Skip to content

Commit ebfdad1

Browse files
feat(ai): experimental callbacks in ToolLoopAgent (#12717)
## Background as a follow up to #12654, we will introduce the same callbacks for the `ToolLoopAgent` ## Summary - added callback `experimental_onStart` that exposes the data/events that happen at the very beginning - added callback `experimental_onStepStart` - added callback `experimental_onToolCallStart` - added callback `experimental_onToolCallFinish` ## Manual Verification ## Future Work - callbacks aren't currently passed through for streaming - will need to merge #12708 first ## Checklist - [x] Tests have been added / updated (for bug fixes / features) - [x] Documentation has been added / updated (for bug fixes / features) - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] I have reviewed this pull request (self-review)
1 parent 1a07b86 commit ebfdad1

16 files changed

+2698
-139
lines changed

.changeset/blue-pants-think.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'ai': patch
3+
---
4+
5+
feat(ai): experimental callbacks in ToolLoopAgent

content/docs/03-agents/02-building-agents.mdx

Lines changed: 47 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -329,26 +329,68 @@ export async function POST(request: Request) {
329329
}
330330
```
331331

332-
### Track Step Progress
332+
### Lifecycle Callbacks
333333

334-
Use `onStepFinish` to track each step's progress, including token usage.
335-
The callback receives a `stepNumber` (zero-based) to identify which step just completed:
334+
<Note type="warning">
335+
Experimental callbacks are subject to breaking changes in incremental package
336+
releases.
337+
</Note>
338+
339+
Agents provide lifecycle callbacks that let you hook into different phases of the agent execution.
340+
These are useful for logging, observability, debugging, and custom telemetry.
336341

337342
```ts
338343
const result = await myAgent.generate({
339344
prompt: 'Research and summarize the latest AI trends',
340-
onStepFinish: async ({ stepNumber, usage, finishReason, toolCalls }) => {
345+
346+
experimental_onStart({ model, functionId }) {
347+
console.log('Agent started', { model: model.modelId, functionId });
348+
},
349+
350+
experimental_onStepStart({ stepNumber, model }) {
351+
console.log(`Step ${stepNumber} starting`, { model: model.modelId });
352+
},
353+
354+
experimental_onToolCallStart({ toolCall }) {
355+
console.log(`Tool call starting: ${toolCall.toolName}`);
356+
},
357+
358+
experimental_onToolCallFinish({ toolCall, durationMs, success }) {
359+
console.log(`Tool call finished: ${toolCall.toolName} (${durationMs}ms)`, {
360+
success,
361+
});
362+
},
363+
364+
onStepFinish({ stepNumber, usage, finishReason, toolCalls }) {
341365
console.log(`Step ${stepNumber} completed:`, {
342366
inputTokens: usage.inputTokens,
343367
outputTokens: usage.outputTokens,
344368
finishReason,
345369
toolsUsed: toolCalls?.map(tc => tc.toolName),
346370
});
347371
},
372+
373+
onFinish({ totalUsage, steps }) {
374+
console.log('Agent finished:', {
375+
totalSteps: steps.length,
376+
totalTokens: totalUsage.totalTokens,
377+
});
378+
},
348379
});
349380
```
350381

351-
You can also define `onStepFinish` in the constructor for agent-wide tracking. When both constructor and method callbacks are provided, both are called (constructor first, then the method callback):
382+
The available lifecycle callbacks are:
383+
384+
- **`experimental_onStart`**: Called once when the agent operation begins, before any LLM calls. Receives model info, prompt, settings, and telemetry metadata.
385+
- **`experimental_onStepStart`**: Called before each step (LLM call). Receives the step number, model, messages being sent, tools, and prior steps.
386+
- **`experimental_onToolCallStart`**: Called right before a tool's `execute` function runs. Receives the tool call object with tool name, call ID, and input.
387+
- **`experimental_onToolCallFinish`**: Called right after a tool's `execute` function completes or errors. Receives the tool call, `durationMs`, and a `success` discriminator (`output` when successful, `error` when failed).
388+
- **`onStepFinish`**: Called after each step finishes. Receives step results including usage, finish reason, and tool calls.
389+
- **`onFinish`**: Called when all steps are finished and the response is complete. Receives all step results, total usage, and telemetry metadata.
390+
391+
#### Constructor vs. Method Callbacks
392+
393+
All lifecycle callbacks can be defined in the constructor for agent-wide tracking, in the `generate()`/`stream()` call for per-call tracking, or both. When both are provided, both are called (constructor first, then the method callback):
352394

353395
```ts
354396
const agent = new ToolLoopAgent({

content/docs/07-reference/01-ai-sdk-core/15-agent.mdx

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,10 +65,30 @@ export type AgentCallParameters<CALL_OPTIONS, TOOLS extends ToolSet = {}> = ([
6565
* Can be used alongside abortSignal.
6666
*/
6767
timeout?: number | { totalMs?: number };
68+
/**
69+
* Callback that is called when the agent operation begins, before any LLM calls.
70+
*/
71+
experimental_onStart?: ToolLoopAgentOnStartCallback<TOOLS>;
72+
/**
73+
* Callback that is called when a step (LLM call) begins, before the provider is called.
74+
*/
75+
experimental_onStepStart?: ToolLoopAgentOnStepStartCallback<TOOLS>;
76+
/**
77+
* Callback that is called before each tool execution begins.
78+
*/
79+
experimental_onToolCallStart?: ToolLoopAgentOnToolCallStartCallback<TOOLS>;
80+
/**
81+
* Callback that is called after each tool execution completes.
82+
*/
83+
experimental_onToolCallFinish?: ToolLoopAgentOnToolCallFinishCallback<TOOLS>;
6884
/**
6985
* Callback that is called when each step (LLM call) is finished, including intermediate steps.
7086
*/
7187
onStepFinish?: ToolLoopAgentOnStepFinishCallback<TOOLS>;
88+
/**
89+
* Callback that is called when all steps are finished and the response is complete.
90+
*/
91+
onFinish?: ToolLoopAgentOnFinishCallback<TOOLS>;
7292
};
7393

7494
/**
@@ -142,7 +162,12 @@ Both `generate()` and `stream()` accept an `AgentCallParameters<CALL_OPTIONS, TO
142162
- `options` (optional): Additional call options when `CALL_OPTIONS` is not `never`
143163
- `abortSignal` (optional): An `AbortSignal` to cancel the operation
144164
- `timeout` (optional): A timeout in milliseconds. Can be specified as a number or as an object with a `totalMs` property. The call will be aborted if it takes longer than the specified timeout. Can be used alongside `abortSignal`.
165+
- `experimental_onStart` (optional): Callback invoked when the agent operation begins, before any LLM calls. Experimental.
166+
- `experimental_onStepStart` (optional): Callback invoked when a step (LLM call) begins, before the provider is called. Experimental.
167+
- `experimental_onToolCallStart` (optional): Callback invoked right before a tool's execute function runs. Experimental.
168+
- `experimental_onToolCallFinish` (optional): Callback invoked right after a tool's execute function completes or errors. Experimental.
145169
- `onStepFinish` (optional): A callback invoked after each agent step (LLM/tool call) completes. Useful for tracking token usage or logging.
170+
- `onFinish` (optional): A callback invoked when all steps are finished and the response is complete.
146171

147172
## Example: Custom Agent Implementation
148173

0 commit comments

Comments
 (0)