Skip to content

feat(observability): add ToolObserver hook for structured tool tracing#490

Open
SebastianBoehler wants to merge 1 commit intosipeed:mainfrom
SebastianBoehler:feat/tool-observer-weave-observability
Open

feat(observability): add ToolObserver hook for structured tool tracing#490
SebastianBoehler wants to merge 1 commit intosipeed:mainfrom
SebastianBoehler:feat/tool-observer-weave-observability

Conversation

@SebastianBoehler
Copy link

Summary

Adds a lightweight observer pattern to ToolRegistry that fires after every tool execution. This enables external observability integrations (W&B Weave, OpenTelemetry, custom metrics) without modifying individual tools.

Changes

pkg/tools/registry.go

  • Add ToolObserver func type: func(name string, args map[string]interface{}, result *ToolResult, durationMs int64)
  • Add observer field to ToolRegistry
  • Add SetObserver() method
  • Fire observer in ExecuteWithContext after every tool call

pkg/agent/loop.go

  • Add SetToolObserver() to AgentLoop — propagates observer to all agent tool registries

cmd/picoclaw/main.go

  • Wire observer in agentCmd when PICOCLAW_WEAVE_OBSERVE=1
  • Emits WEAVE_TOOL_EVENT: JSON lines to stderr for downstream consumption

Design

The observer is a zero-cost abstraction when not set — a single nil check before the call. No allocations, no goroutines.

The WEAVE_TOOL_EVENT: line format emitted to stderr:

WEAVE_TOOL_EVENT:{"tool":"exec","duration_ms":42,"is_error":false,"args":{...}}

This is intentionally transport-agnostic — any process reading the agent's stderr can parse these lines. The W&B Weave integration reads them from the agent subprocess and logs structured traces per run, but the hook itself has no Weave dependency.

Usage

// Custom observer — log to any backend
agentLoop.SetToolObserver(func(name string, args map[string]interface{}, result *tools.ToolResult, durationMs int64) {
    myTracer.RecordToolCall(name, durationMs, result.IsError)
})

Or via env var for the built-in stderr emitter:

PICOCLAW_WEAVE_OBSERVE=1 picoclaw agent -m "your prompt"

Copy link

@nikolasdehor nikolasdehor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The observer pattern is well-designed — zero-cost nil check, thread-safe via RLock, clean API. Two notes:

  1. Unrelated changes: Removes OneBot transcriber wiring and a duplicate Name field. These should be separate commits or documented in the PR description.

  2. Fragile JSON construction: The event JSON is built via fmt.Sprintf with %q and raw %s. If argsJSON is malformed, the outer JSON breaks. Safer to use json.Marshal for the entire event struct.

LGTM for the core observer pattern.

SebastianBoehler pushed a commit to SebastianBoehler/picoclaw that referenced this pull request Feb 19, 2026
- Use json.Marshal for entire WEAVE_TOOL_EVENT struct instead of
  fragile fmt.Sprintf with %q/%s interpolation; prevents broken JSON
  if args contain special characters
- Restore accidentally removed OneBot transcriber wiring in gatewayCmd
- Document removal of duplicate ToolCall.Name field (canonical name
  lives in Function.Name only)
SebastianBoehler pushed a commit to SebastianBoehler/picoclaw that referenced this pull request Feb 19, 2026
…ests

## Problem

When agents call external APIs (e.g. Handelsregister, lead discovery,
internal services), they were forced to use exec+curl. This has two
critical drawbacks:

1. The ToolObserver fires with tool="exec" and args={"command": "curl ..."},
   making the actual API call invisible in Weave traces — you see a shell
   command string, not a structured HTTP event.

2. Models with weaker instruction-following (e.g. glm-5) misread curl
   examples as pseudo-syntax and emit literal strings like
   "POST http://..." as shell commands, causing immediate failures.

## Solution

Add http_fetch as a first-class native Go tool alongside web_fetch.

Every http_fetch call surfaces in Weave traces as:
  tool="http_fetch", args={url, method, body, headers}

This makes API calls to internal services fully observable — reviewers
can see exactly which endpoint was called, with what payload, and
whether it succeeded, without parsing shell command strings.

## Changes

pkg/tools/http_fetch.go (new)
- HttpFetchTool struct with Name/Description/Parameters/Execute
- Supports GET, POST, PUT, PATCH, DELETE
- Defaults Content-Type to application/json when body is present
- Pretty-prints JSON responses for LLM readability
- Returns IsError=true for HTTP 4xx/5xx so ToolObserver marks them
- 512KB response cap (configurable via NewHttpFetchTool)
- 30s timeout, max 5 redirects

pkg/agent/loop.go
- Register tools.NewHttpFetchTool(512*1024) alongside NewWebFetchTool

## Relationship to PR sipeed#490

PR sipeed#490 added the ToolObserver hook that fires after every tool call.
http_fetch is the first tool specifically designed to exploit that hook:
because it is a named native tool rather than a shell wrapper, every
HTTP API call now appears as a distinct, structured trace event instead
of being buried inside an opaque exec invocation.
Copy link
Collaborator

@lxowalle lxowalle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Please submit only one feature per PR.
  2. The http_fetch and web_fetch tools conflict.

@lxowalle lxowalle mentioned this pull request Feb 22, 2026
5 tasks
@SebastianBoehler SebastianBoehler force-pushed the feat/tool-observer-weave-observability branch from 4d6b3c9 to 46194b1 Compare February 22, 2026 16:20
@SebastianBoehler
Copy link
Author

Removed conflicting features

@CLAassistant
Copy link

CLAassistant commented Mar 5, 2026

CLA assistant check
All committers have signed the CLA.

Add a lightweight observer pattern to ToolRegistry that fires after
every tool execution. This enables external observability integrations
(Weave, OpenTelemetry, custom metrics) without modifying individual tools.

Changes:
- pkg/tools/registry.go: Add ToolObserver func type, observer field to
  ToolRegistry, SetObserver() method, and fire observer in ExecuteWithContext
  after every tool call with (name, args, result, durationMs)
- pkg/agent/loop.go: Add SetToolObserver() to AgentLoop, propagating
  the observer to all agent tool registries
- cmd/picoclaw/cmd_agent.go: Wire observer in agentCmd when env var
  PICOCLAW_WEAVE_OBSERVE=1 — emits WEAVE_TOOL_EVENT: JSON lines to
  stderr for downstream consumption by observability backends

The observer is a zero-cost abstraction when not set (nil check before
call). The WEAVE_TOOL_EVENT: line format is:
  WEAVE_TOOL_EVENT:{"tool":"exec","duration_ms":42,"is_error":false,"args":{...}}
@SebastianBoehler SebastianBoehler force-pushed the feat/tool-observer-weave-observability branch from 46194b1 to cdc3127 Compare March 15, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants