Building an AI Agent Orchestrator for Decentralized GPU Compute

I'm Haris Ahmad, Founder & CEO of ZenithOps AI, an AI automation agency based in Burewala, Punjab, Pakistan. I design and ship AI automation systems and agent workflows for clients. For the Decentralize AI Hackathon, I'm proposing something I want to build: an autonomous AI agent orchestration service that runs on decentralized GPU infrastructure, starting with Nosana's peer-to-peer GPU network. This post is my proposal — the idea, the architecture, and the honest state of progress: design stage, with the build happening during the hackathon.The Problem: Centralized AI Prices Out Builders Like MeCentralized AI concentrates power in a handful of hyperscalers. Renting a single H100-class GPU can cost thousands of dollars per month — a wall for indie developers and small agencies like mine in Pakistan. Add vendor lock-in, opaque model APIs with silent policy changes, and data pipelines that feed Big Tech's models while creators get nothing in return, and the pattern is clear: the global majority stays consumers of AI infrastructure rather than builders of it.My Angle: The Economic Leverage of Decentralized GPUsMost commentary frames decentralization as a moral crusade. My take is economic: a market of independent GPU operators can undercut hyperscaler pricing while giving builders in emerging markets real ownership of the infrastructure they depend on. The leverage point is orchestration — autonomous AI agents that schedule inference and training jobs across decentralized GPU networks, turning idle compute into a programmable resource anyone can build on.The Architecture: A Four-Part OrchestratorHere is the design I am working toward, in four moving parts. First, an API gateway (planned: FastAPI) that exposes a REST endpoint for job submissions and status queries. Second, a planner-executor agent (planned: an open-source LLM tuned for scheduling heuristics) that receives a high-level request, resolves resource constraints, and generates a container manifest. Third, a Nosana job submitter that wraps the manifest and posts it to Nosana's peer-to-peer GPU marketplace, then monitors the job. Fourth, an immutable log (planned: Arweave) that stores the prompt, model references, and execution metadata so every run is auditable. In the planned flow, a client submits a task to the gateway; the planner picks the cheapest node satisfying the constraints (for example, minimum GPU memory and maximum latency); the job runs in a container on the decentralized network; and the orchestrator pins a permanent, tamper-evident receipt of the run.Why This Matters to YouIf you are a developer or small agency outside the big tech hubs, this changes the economics: instead of paying hyperscaler rates for GPU time, you route agent workloads to a peer-to-peer network and pay market prices. Your data and agent history stay yours, verifiably recorded on permanent storage instead of inside a vendor's black box. For the ecosystem, it means open, community-run infrastructure — anyone with a GPU can earn, and anyone with an idea can build without asking a platform's permission.What I'm Still Figuring OutHonestly: the build is ahead of me, not behind me. The biggest open questions are how the spot market behaves under sustained multi-tenant demand, how to reconcile token-based GPU payments with the subscription billing my clients expect, and how to monitor and attest heterogeneous nodes I don't control. I'll be working through these during the hackathon, and I'll use the participant Nosana credits to run my first real workloads.The 2030 Vision: An Open Compute GridBy 2030, I want a world where running an AI agent service doesn't require renting from a hyperscaler at all. Millions of independent GPUs form an open compute grid; orchestrator services route workloads to the cheapest reliable node in real time; and agent identities, job histories, and model provenance live on permanent decentralized storage — so any team, from Burewala to Berlin, can audit, fork, and improve AI systems the way we fork open-source code today.Join the ConversationThis post is part of the Decentralize AI Hackathon by HackerNoon. I'm building in the open, and I want to hear from people who have tried running AI workloads on decentralized GPU networks: what broke, what surprised you, what would you do differently? Drop a comment — the best feedback will shape what I build next.