Local AI Agent Deployment: An AgentSpace Token Cost Test
Why local AI agent deployment landed on my desk
Reading about small, locally-runnable models like LFM2.5-2.6B being used for agentic tasks, one question stuck with me: should I pull some tasks out of AgentSpace’s Claude worker pool and hand them to a local model instead? AgentSpace currently runs on a lead-agent-plus-workers setup: a lead agent plans a sprint, workers pick up tasks from a task board and execute real software work, all running on Claude. Every worker call is a token bill. Could a small local model actually cut a chunk of that bill, or would I just be buying quality loss with extra ops work? I wanted to think through this in writing before touching any code.
This is not a results post. It’s an honest walkthrough of how I plan to test the local-deployment idea, which tasks I think are candidates, and what I genuinely don’t know yet.
Where AgentSpace’s token cost actually comes from
Most of the cost in AgentSpace comes from the context and output tokens worker agents generate per task. The lead agent produces a sprint plan, workers pull work off the task board and run it against a real codebase, and they read persistent memory for past context. All of this runs on Claude today, because Claude has been my only model layer so far.
The catch: not every worker task carries the same weight. Some tasks genuinely need reasoning — multi-step planning, understanding relationships across a codebase — and that’s exactly where I need Claude’s strength. But other tasks are mechanical: reformatting a file into a fixed structure, summarizing a log output, generating text against a rigid template. That second bucket looks like it could be handled without the full capacity of a large model.
What the LFM2.5-2.6B news actually promises
What caught my attention in that piece was a small-parameter model being shown viable for agentic tasks — tool calling, following step-by-step instructions. I want to be precise here: I have not wired this model into AgentSpace yet, and I haven’t benchmarked it against my own workers. I have zero numbers I could point to and say “this saved X tokens on this task type.” What I have is a hypothesis triggered by that news: put a small, self-hosted model in front of certain worker tasks — either as a pre-filter or as the direct executor — and reduce the number of calls that hit Claude.
Which tasks could move local, and which can’t
There’s no point designing a test without drawing this line first, so here’s my classification attempt.
Candidates for a local model
- Fixed-format text generation (task summary cards, log transformation)
- Simple classification (is this task urgent, which category does it belong to)
- A pre-filter step: deciding, before a task ever reaches a worker, whether it actually needs Claude at all
What has to stay on Claude
- Multi-step code changes and actual software task execution
- Interpreting past context from persistent memory and tying it back into the plan
- The lead agent’s sprint-planning and task-splitting decisions
That split is itself an assumption. It wouldn’t surprise me at all if a task I’m calling “simple” turns out to break in unexpected ways on a small model. I’m not claiming this works until I’ve measured it.
My deployment plan
I’m planning to bring local AI agent deployment into AgentSpace in three steps. None of this is in production yet — here’s the order.
1. Model selection and hosting
The first step is standing the model up in isolation, independent of the environment AgentSpace runs in. The goal is to see how consistent the model is on its own before I touch the worker flow at all. I haven’t decided on hardware yet — that’s one of the open questions this post doesn’t answer.
2. Adding a router in front of the worker pool
In the current architecture, every task goes straight to a worker, and every worker goes straight to Claude. My plan is to put a thin router in front of that flow: work coming off the task board would first pass through a simple classifier. Tasks tagged “local-suitable” would go to the local model; everything else would follow the normal Claude worker path. The router itself is a decision point, which means it carries its own failure mode — a misclassified task could let low-quality output slip into a sprint result unnoticed.
3. Task classification logic
For the first pass, I’m keeping classification rule-based (by task-type tag) rather than jumping straight to a model-based classifier. The reason is simple: a rule-based system is easier to debug, and in early testing I want visibility into what’s working and what isn’t, not another opaque model in the loop.
4. What I’ll actually measure
Once the test is set up, I’ll be looking at three things:
- The token/cost difference between the local model and a Claude worker on the same task type
- Whether there’s a visible drop in output quality (a rough measure like acceptance/rejection rate)
- The router’s misclassification rate — of the tasks I labeled “local-suitable,” how many actually needed Claude
I have no numbers for any of this right now. This post is the plan, written before measurement starts.
Risks and what I still don’t know
The biggest risk is that token savings get quietly paid for with quality loss. If a worker task finishes “cheap” on a local model but its output needs correction later in the sprint, the real cost hasn’t gone down — it’s just moved somewhere harder to notice. I won’t claim “local AI agent deployment cuts token cost” until I can measure that trade-off directly.
The second unknown is that the router itself is extra maintenance. Classification rules will go stale as task types change, and I haven’t figured out yet who updates them or how often.
Third: scale. Running this as a solo founder, I don’t yet know whether the operational overhead of hosting a local model is actually worth the token savings it produces. That’s also something I can’t answer without testing it.
Where this leaves me
The idea makes sense on paper: not every task in AgentSpace’s worker pool carries equal weight, and offloading some of them to a smaller model could lower the token bill. But right now I have a plan, not a result. Once the router is built and the first classification pass is live, I’ll write a follow-up on which task types actually showed a difference — and what didn’t behave the way I expected.