Problem
To avoid blowing out the LM's context window, when the output size of a tool exceeds 20 KB the agent (Copilot CLI, Claude Code, etc.) will generally not provide the output to the LM as-is. It will instead find some way of cutting down the information, possibly including:
- truncating the output
- deleting data from the middle of the output
- using AI to summarize the data down to a smaller size
- spilling the data to a file, and providing the LM the path to the file
In all cases information is likely to be lost, in the sense that it doesn't make it from the tool back to the LM.
Currently we do not know how often this happens, so we cannot design an appropriate response.
Solution
Add telemetry to every tool call to capture the size of the tool output.
Problem
To avoid blowing out the LM's context window, when the output size of a tool exceeds 20 KB the agent (Copilot CLI, Claude Code, etc.) will generally not provide the output to the LM as-is. It will instead find some way of cutting down the information, possibly including:
In all cases information is likely to be lost, in the sense that it doesn't make it from the tool back to the LM.
Currently we do not know how often this happens, so we cannot design an appropriate response.
Solution
Add telemetry to every tool call to capture the size of the tool output.