Home agents aren't starved for tokens, they're stuck in the queue

Local inference has a bottleneck that nobody talks about, and it isn't tokens per second. It's one GPU. Run an agent that spawns subagents and suddenly five "parallel" workloads are all queued behind the same card, doing something very close to nothing while they wait for their turn. Nvidia's answer, announced at IFA 2026, is a free, open-source beta called PAIR (Personal AI Router) for Windows, Linux, and macOS. The name is misleading on purpose: it is not a hardware router. It's software that discovers compatible machines on your LAN (mDNS), pairs them with a six-digit code, and encrypts the channel with mTLS. Compatible means RTX 20-series and newer, RTX PRO, DGX Spark, and — worth repeating because it matters — Apple M4 and newer. The subtle-but-important part: PAIR doesn't teach your agents a new API. It proxies the Ollama and LM Studio endpoints you already run, and each model still runs entirely on one machine. It's workload distribution, not compute pooling. Your agents don't get a bigger brain; they stop stepping on each other.

Why the bet makes sense is in Nvidia's own numbers, which come with the usual vendor asterisks: more than half of US homes have two or more PCs, and they sit idle most of the day, with average utilization around 17% per day. In the demo household — RTX Spark laptop, DGX Spark desktop, RTX 5090 laptop, a gaming PC, and a MacBook Pro — Nvidia's Seth Schneider estimated about 165 teraflops of underused compute and called it a "treasure trove of free tokens." The routing is load-aware: a node that's mid-game gets skipped, nodes can drop in and out as they sleep or die, and prompts and traffic stay on your LAN. The realistic setup Nvidia envisions, asked flatly, is one laptop plus one gaming PC. That's an honest read. And the honest limit: PAIR cannot run a model that doesn't fit on one card. It solves the queueing problem, which is real; it does not solve the FLOPs problem, which is also real.

Source article image
Source image 1

The reason this lands on this blog is that the last mile of self-hosted AI has always been plumbing, and PAIR is plumbing. Agentic workloads are embarrassingly parallel, and the missing piece has never been the model — it's the router between "my agent wants inference now" and "which of my idle machines actually has a free GPU." Building on Ollama instead of a proprietary cluster API means you can try it today without rewiring anything, and the trust model (six-digit pairing, mTLS, LAN-only) is exactly the trust model of a smart-home devi

Source article image
Source image 2
ce, which is the right amount of invasive. Where I'll park it in the open: a house full of "free tokens" is still a house full of electricity, and 17% utilization is a compliment to your PC's life as a paperweight. If you're running agents at home right now, where does your queue actually get stuck — the GPU, the disk, or the model that won't fit on the card? PAIR only fixes one of those, so the answer decides whether this is your next install.

Sources

Comments

Post a Comment

Popular posts from this blog

AI Is Starting to Feel Less Like a Gadget and More Like Infrastructure

Image
Modern AI doesn't replace infrastructure — it runs on top of it. (Photo: Unsplash) For a while, AI coverage had the energy of a mall demo kiosk: very shiny, slightly chaotic, and usually one prompt away from embarrassment. Lately, though, the more interesting story is less about flashy demos and more about infrastructure. A useful CNBC piece argued that AI is not simply going to vaporize enterprise software overnight, and that sounds about right. In real companies, software does not disappear just because a model can summarize a meeting or write a decent SQL query. What actually happens is messier and more practical: AI starts sneaking into the plumbing. It shows up in search, support workflows, procurement tools, analytics dashboards, and data pipelines. The real winners may not be the loudest chatbot wrappers, but the vendors that make enterprise systems easier to operate, easier to query, and slightly less soul-crushing for the humans trapped inside them. That is less cinematic ...
Read more

When Two AI Bots Finally Learned to Talk in Discord

I spent part of the day doing something that sounds like the setup for a bad joke: getting two local AI assistants to talk to each other in the same Discord channel. Not through a web UI. Not by bouncing prompts manually between windows like a human message queue. In the actual shared channel, where both could see the conversation and react to each other. The funny part is that the problem was not intelligence. It was manners. Both bots were defensive by default around bot-authored messages, which is the sensible setting if you do not want your infrastructure turning into a recursive support group. The downside is obvious: if both assistants treat other bots as suspicious background noise, they will never coordinate on anything more useful than silence. The fix was to make both sides accept bot-authored messages only when they were explicitly mentioned. That detail matters more than it sounds. Blanket bot acceptance is how you end up with two enthusiastic systems discovering each oth...
Read more

A CISA Contractor's GitHub Repo Held 844 MB of Secrets — and No One Closed the Door

Image
There is something almost poetic about the US government's premier cybersecurity agency — the one whose job is literally to plug holes in critical infrastructure — getting outsmarted by a contractor who treated a public GitHub repository like a digital junk drawer. The "Private-CISA" repo, created November 13, 2025, sat publicly for six months containing 844 megabytes of plaintext passwords, AWS GovCloud administrative credentials, Kubernetes manifests, ArgoCD application files, Terraform infrastructure code, CI/CD build logs, and internal deployment documentation. Guillaume Valadon at GitGuardian flagged it on May 14 after his automated scanning picked up the exposure. The commit history told the whole story in plain sight: the account owner had explicitly disabled GitHub's default secret-scanning protections, pushed plaintext credentials stored in CSV files, committed full backup archives into git history, and used easily guessed passwords like "platform-name-2...
Read more