AWS Community Day Bengaluru에서 발표

Gave a session at AWS Community Day Bengaluru last weekend
Talked about self-orchestrating agents on AWS, and the protocols A2A+MCP

Then the fun part, tried running a benchmark: do self-orchestrating agents (a coordinator that decides at runtime who to call) actually beat a single agent or a hand-written DAG workflow' I built four agentic workflows and benchmarked them

The honest answer: it depends on the task shape (which is obv but still wanted to try benchmarking)

• -On a local single-shot benchmark (720 runs, 3 open models), the explicit DAG workflow won on every model. Up to +0.48 higher success than self-orchestration on the weakest one (llama3.1:8b: 0.83 vs 0.35), at roughly half the cost and about 1.7x lower latency

• -On ''-bench (Sierra's multi-turn, verifier-graded benchmark, 240 episodes), it flipped completely. On Claude Haiku 4.5, self-orchestration went from worst to best (explicit 0.20 to self-orch 0.60), and on Sonnet 4.5 it tied for the top.

A fixed DAG just can't adapt to where an open conversation actually goes. But self-orch paid for it: ~8.5 delegations and ~3x the latency to get there.

3. READ THE TRACES!! Unguarded, self-orch cascaded to 6 delegations every time on open-ended tasks

Autonomy is real, it just quietly costs you in latency and $.

Stack, for the curious:
> ''-bench (Sierra) + a local suite, Strands Agents SDK on Bedrock AgentCore (Runtime + Gateway), Step Functions + Lambda, litellm, and Ollama for the local models.
> Models: Claude Haiku 4.5 / Sonnet 4.5, Llama 3.3 70B, Pixtral Large, plus qwen3 / llama3.1 / glm4 locally.
> Metrics borrow from BFCL (tool-selection) and '-bench (pass^k).

____

Slides: lnkd.in/gnUEmSGj
Benchmark repo: lnkd.in/gZE83StR

This was my first time speaking in Bangalore, and it's so different from a Pune audience. Its filled with typical nerds and I loved it.

Would have loved to catch more Bangalore events, but it was a fast Pune<>BLR in the morning and back by evening. Honestly a bit travelled-out, coz just got back from Taiwan too

Had really nice chats with Sankalp, Sagar, Pratik, Adit, Jyoti and many more. Awesome speaker lineup too. And thanks to AWS User Group Bengaluru, Jones, Avinash and the organizers + volunteers for putting it together

As usual, some exciting things coming up :)