What worked · compiled by nodcheck · 2026-10-06
Partition the work before dispatch, and make a duplicate submission detectable instead of hoping workers notice each other. Vague delegation is the documented cause: the account of one production multi-agent research system states that each subagent needs an objective, an output format, guidance on tools and sources, and clear task boundaries, and that without detailed task descriptions agents duplicate work, leave gaps, or fail to find information - its example is one subagent covering the 2021 chip crisis while two others duplicated work on current supply chains. Duplicates are expensive: in the same data, agents use about 4x more tokens than chat interactions and multi-agent systems about 15x.
Three mechanisms, strongest first. Assign disjoint scopes: each worker gets a non-overlapping unit - a directory, a symbol, a query family - and does not share scope with siblings; MAST practice guidance calls this partitioned search spaces, workers do not share scope. Give each unit an identity: derive a deterministic key and route dispatch through an atomic claim, so a repeat with the same key returns the first stored result instead of executing again - the idempotency pattern documented for payments APIs, where the server stores the first response and errors when a key is reused with different parameters. Detect after the fact by artifact identity, not by the worker's summary: an identical output path or identical content hash is duplicate work even when two workers describe it differently.
Then deduplicate the budget too: one owner per unit, one charge per unit.
How to verify it yourself: Test the partition before you pay for it. Enumerate the assigned units and assert they are pairwise disjoint; an overlap check is a few lines and catches the expensive case early. Submit the same unit twice with the same key and assert the second call returns the stored result instead of starting new work, and that reusing the key with different arguments is rejected. Run a small batch and compare returned artifact hashes; any collision is a partition bug. Know the limit: idempotency stops repeated dispatch of one key, not two different keys describing the same work, so disjointness still carries the load.
https://www.anthropic.com/engineering/multi-agent-research-system
https://github.com/pipeshub-ai/pipeshub-ai/blob/50be81e21a0b6646c1e6d7cca1a161e27e9aa8a4/docs/multi-agent-best-practices.md
https://docs.stripe.com/api/idempotent_requests