Our first video back covers something I’ve been working on for the better part of six months. Until now we didn’t have the local GPU side in place. A lot of this has been theory, and I’ll be upfront: I don’t know yet whether it will work or pay off. The modeling and simulation say it will, and now we get to test that on real hardware.
The idea
An MCP orchestration layer connects Grok, Claude and OpenAI, so the frontier models stay aligned and work together toward one goal.
The newer piece is taking legacy mining rigs, x1 PCIe risers and all, and turning every GPU into its own small agent. Each card runs one local model (8B, 14B, 32B, or a 70B split across two cards) and handles a narrow, tightly gated job.
The frontier model does the heavy lifting: it plans, breaks the project into a task graph and checks the work. It hands each GPU a short, specific instruction with only the context that task needs. The local model does the job, returns structured results and forgets it. The loop repeats until the project is done.
Why it matters
Context stops snowballing. In a long frontier session, every step re-reads everything before it. Here the frontier only writes short task packets and reads short results, so its context stays lean.
Volume work comes off the API bill. Extraction, tagging, summarizing, SQL, first drafts and checks run on hardware we already own, for the cost of electricity.
Parallel by default. 36 GPUs means up to 36 agents working at once, instead of one model working through a list.
Legacy hardware gets a second life. An x1 riser only slows the moment a model loads. Once it’s loaded, the bus barely matters.
Sensitive work can stay on-site. Tasks tagged local-only never leave the LAN.
The numbers (modeled, not yet benchmarked)
Fleet: 4 rigs (Radeon VII, RTX 3080, RTX A5000 and a mixed NVIDIA/AMD rig), 36 GPUs, 555 GB of VRAM.
Throughput:
About 2,000 tokens/sec combined, or about 7M tokens an hour.
Daily volume:
At 40% utilization, about 70M tokens a day generated locally.
Power: about 8.7 kW with power caps, or about 210 kWh a day. That’s roughly $17–25 a day at $0.08–0.12/kWh (our range at the office).
The same volume through frontier APIs:
Roughly $200–1,000 a day, at a blended $3–15 per million output tokens.
Frontier token reduction on a 20-step project:
About 60% fewer when steps are handed off one at a time. About 85–90% fewer when tasks fan out in parallel.
The caveats. Small models aren’t frontier models. Some tasks will fail, retry or escalate to a bigger model, and a local token isn’t worth a frontier token one-for-one.
That’s exactly what we’re testing: the x16/x8/x4/x1 lane benchmarks, real tokens per second for each card, and how much work the swarm can take off the frontier.