You split a complex task across several agents. Each one has a focused job, and the design looks clean on a whiteboard. Then the run spends more time moving context and reconciling outputs than doing the work that justified the split.
The reflex to decompose is not wrong. Parallelism and fault isolation can be valuable, but every split adds context, prompt, review, and handoff costs. When those exceed the benefit, the more elaborate design performs worse.
A warning sign is token spend rising with agent count while completed work stays flat. At that point the system is buying overhead, not capability.
Why This Happens
The overhead is not random. It comes from the same three causes that set the granularity floor for how small a task should be: context coupling (every agent needs its own copy of the surrounding context), prompt overhead (every call needs framing that can cost more than the work itself), and state management at each handoff (validating and reformatting whatever the previous agent produced). How Small Should an AI Task Be? covers why those causes exist.
These three costs do not add linearly. They interact. More handoffs means more context duplication means more prompt overhead, and the failure rate at each seam multiplies the revision loops that generate even more tokens. The model choice makes this worse in a specific way: if you are using a less capable model to save money per token, you often have to decompose further to keep each task within the model’s effective reasoning window. That decomposition drives up total cost even as the per-token rate goes down. A cheaper model can end up being the more expensive option overall.
The Economic Reality
Most teams track API cost as a line item, but they do not instrument it at the task level. They see the bill go up as the system grows and assume that is the cost of more capability. Often it is the cost of more overhead.
A decomposition can cost more than a single well-scoped agent when each worker needs much of the same context or when handoffs trigger revision loops. The amount depends on prompt scaffolding, tool definitions, output schemas, and how ambiguous the intermediate artifacts are.
The wall-clock picture is often worse. Parallel execution helps when the subtasks are genuinely independent, but most real workflows have dependencies. If agent three cannot start until agent two finishes, and agent two is waiting on a validation loop from agent one, you are not running in parallel. You are running sequentially with extra latency at each boundary.
The context window is the primary technical constraint here, as Your AI Doesn’t Need to Read Your Whole Knowledge Base to Use It discusses. Compute is cheap and scales well; tokens are not and do not. A system that duplicates context across many agents is not respecting that asymmetry.
The teams that discover this problem late are usually the ones who built the agent count up incrementally, each step looking locally justified, until the aggregate behavior became unexplainable. Auditing it retrospectively is harder than designing for it up front.
What Good Looks Like
The right architecture for agentic systems amortizes context across the work rather than duplicating it into each agent invocation. The way this works in practice: the shared scope that every agent needs gets prepared once, as a pre-computed work product, before any agent runs. Agents consume from that shared scope rather than each reconstructing it independently.
This sounds straightforward. In implementation it requires discipline about what belongs in shared scope versus what is specific to a particular agent’s invocation. Get that boundary wrong in one direction and you are loading agents with irrelevant context. Get it wrong in the other direction and you are back to duplication.
The agent count itself is not the variable to optimize. The variable is context per agent-turn, which is a function of what each agent is being asked to carry that it could have gotten from shared scope instead. Reducing that redundancy is where the real cost savings come from.
Tiered model dispatch also matters. Not every agent in a system needs the most capable model available. Agents doing deterministic, well-specified subtasks can often run on lighter, faster, cheaper models without quality loss. The coordination and reasoning work, where model capability actually matters, can run on stronger models. Getting that split right requires knowing which subtasks are actually hard, which is a different question than which subtasks look complex on a whiteboard.
When the architecture is right, adding an agent does not require reconstructing the whole problem. Shared scope reduces duplication, but the benefit still has to be measured against coordination and review.
The Benchmark You Should Run Yourself
Before you conclude your current harness is efficient, measure four numbers:
Tokens per task. Not per agent, per completed unit of work. Sum across all agents involved.
Wall-clock per task. Total elapsed time from task submission to final output, including revision loops.
Revision-loop count. How many times does an agent’s output get rejected or need correction before the downstream agent can proceed?
Per-handoff context duplication. Pick two agents with a handoff between them and measure how much background both are carrying. A high ratio of duplicated to unique context is a reason to simplify the split or prepare shared scope once.
If your tokens-per-task scales roughly linearly with the number of agents you involve in a task, the shared-scope problem is real in your system. The benchmark is not complicated to run. Most teams just have not run it because the cost looks like a legitimate scaling curve rather than an overhead curve. They look the same from the billing console.
Instrumenting context duplication can change the model-routing decision because it exposes cost that per-call pricing hides.
Putting It Together
The decomposition reflex is not wrong. Parallelism and fault isolation are real benefits, and they are worth paying for. But paying for them without measuring what you are actually buying is how teams end up with agent swarms that cost more and deliver less than a simpler design would have.
The problem is solvable. The architecture that solves it is not complicated in concept, though the implementation discipline is real. Shared scope, pre-computed work products, tiered dispatch. The hard part is instrumentation: knowing what you are actually spending per unit of work and understanding where the overhead is coming from.
If your team is hitting this wall, the place to start is measuring where the token spend actually goes.