Improving instruction following with next-step hints in tool call results

An agent can call every tool correctly and still get lost immediately afterwards. We see this more often with smaller LLMs: they can follow a clear local instruction, but are less consistent at reconstructing the whole research workflow from a long system prompt.

A semantic search hit may look convincing enough to quote as proof. An empty grep result may be treated as proof that the code does not exist. A truncated relationship list may quietly become “all usages.” These are not tool-call errors. The model received valid data, then drew the wrong operational conclusion from it.

In CodeAlive's context research agent, we handle this with a small convention: every tool result ends with a hint.

<artifacts>  <!-- search results --></artifacts><hint>  Grep line previews are context only. Before reasoning about behavior,  call fetch_artifacts and treat the full source as ground truth.</hint>

The position matters. The model first reads what the tool found. The last thing it sees is how to interpret that result and which action makes sense next. The hint closes the tool response by bringing the model's attention back to the workflow.

This does not give a small model new reasoning capabilities. It reduces how much of the workflow the model has to keep active at once. Instead of recalling a rule from the beginning of the conversation, it receives the relevant rule next to the data and immediately before its next decision.

Descriptions before the call, hints after it

Tool descriptions teach the model how to make a valid call. The semantic_search description, for example, asks for a full natural-language question rather than bare keywords. The read_file description requires an exact repository-relative path.

The trailing hint has a different job. It knows what actually happened and can give narrower advice:

• search results explain that descriptions and snippets are triage, not proof;
• empty results suggest rephrasing, broadening filters, or switching search modes;
• fetched source is marked as ground truth;
• truncated graph results tell the agent when another call is needed;
• recoverable errors explain which argument to repair without failing the whole chat.

The hint does not have to be static. get_artifact_relationships builds one from the requested profile, the returned groups, and the available relationship counts. A successful response can look like this:

{  "sourceIdentifier": "CodeAlive.Agents::SearchToolsFactory::Build",  "profile": "callsOnly",  "found": true,  "relationships": [    {      "type": "outgoing_calls",      "totalCount": 2,      "returnedCount": 2,      "truncated": false,      "items": [        { "identifier": "CodeAlive.Search::SemanticSearchAsync" },        { "identifier": "CodeAlive.Search::GetArtifactRelationshipsAsync" }      ]    }  ],  "hint": "Fetch promising related artifacts before making claims about behavior, concrete applications, or how broadly this mechanism is used."}

The tool has already answered successfully: the agent has two outgoing calls. The final hint tells it that relationship metadata is still not enough to explain behavior and points to the next useful action. Left on its own, the agent is often reluctant to fetch the related artifacts; it may start writing from identifiers and summaries alone. After this hint, it frequently corrects itself and loads the source first. The same method produces different guidance when groups are empty, truncated, or available under another relationship profile.

Our agent prompt only needs to establish the convention once: follow <hint>, <next_tool>, and candidate blocks when they match the question. The detailed instruction arrives after the relevant data, at the point where the model is choosing its next action, instead of occupying the system prompt on every turn.

This makes the tool response part of the agent's control surface. The data answers “what did the tool find?” The final hint answers “what now?” For a smaller LLM, that short local instruction can be the difference between continuing the investigation and treating the first plausible result as the final answer.