From JSON to TOON: A Drop-in Swap That Cuts Agent Token Costs by 30-50%
Your agent makes a tool call. The tool returns 40 rows of JSON. That JSON gets stuffed back into the model’s context, the model thinks for a bit, calls another tool, gets another 40 rows back, and on it goes. By iteration 10, your prompt is mostly curly braces, quoted keys, and the same field names repeated hundreds of times. You’re paying for every single one of those tokens. And you’re paying for them on every subsequent model call in the run, because tool results stick around in the conversation history. ...