feat(agent): add vision/image support with streaming base64 and filetype detection

[shikihane]

Description

Add vision/image support to the agent pipeline so that channels can pass media (images) from users to vision-capable LLMs.

This is a rewrite of #990 (which was reverted in #1010 due to dispute over file processing method). All three review comments from @mengzhuo on #990 have been addressed in this version.

Changes from reverted #990

Issue raised by @mengzhuo	#990 (reverted)	This PR
Use h2non/filetype for MIME detection	mimeFromExtension() — relied on file extension	filetype.MatchFile() — magic-bytes detection, no extension dependency
Use file handler + encoder to avoid 2x memory	os.ReadFile + base64.EncodeToString — peak 2.6x file size in memory	os.Open → base64.NewEncoder → io.Copy — streaming, peak ~1.33x (base64 buffer only)
Bake size limitation into config	maxMediaFileSize hardcoded const (20MB)	config.MaxMediaSize JSON/env configurable, 20MB default via GetMaxMediaSize()

Implementation

New file: pkg/agent/loop_media.go — resolveMediaRefs() function (extracted from loop.go for clarity)

• Streaming base64: os.Open → base64.NewEncoder → bytes.Buffer → data URL
• MIME detection: prefer MediaMeta.ContentType from store, fallback to filetype.MatchFile() (magic bytes)
• Size guard: skip files exceeding config.MaxMediaSize with warning log
• Immutable: returns new slice, never mutates original messages
• Graceful degradation: skips unresolvable refs, unknown MIME types, oversized files

Modified: pkg/providers/openai_compat/provider.go — serializeMessages() replaces stripSystemParts()

• Plain text messages: unchanged behavior (string content)
• Messages with Media: OpenAI vision multi-part format (image_url content parts with data URLs)
• Preserves reasoning_content, tool_calls, tool_call_id

Pipeline wiring:

• Message.Media []string field on protocol types
• processOptions.Media threads media refs from InboundMessage through to BuildMessages()
• resolveMediaRefs() called before LLM to convert media:// refs → data: URLs

Data flow

Channel receives image
  → stores in MediaStore → media://uuid ref
  → InboundMessage.Media carries ref through bus
  → processOptions.Media → BuildMessages() attaches to Message.Media
  → resolveMediaRefs(): file → filetype detect → streaming base64 → data URL
  → serializeMessages(): Message.Media → OpenAI multi-part content
  → LLM sees the image

Test coverage

• resolveMediaRefs: 6 tests (resolve to base64, skip oversized, skip unknown type, passthrough non-media URLs, immutability, metadata content-type preference)
• serializeMessages: 4 tests (plain text, with media, media + tool_call_id, system message stripping)

E2E verification

Tested on Radxa Cubie A7A (arm64) with Feishu channel → Dashscope LLM (kimi-k2.5). Sent image via Feishu, LLM correctly identified image content.

Type of Change

• ✨ New feature (non-breaking change which adds functionality)

AI Code Generation

• 🛠️ Mostly AI-generated (AI draft, Human verified/modified)

Test Environment

• Hardware: Radxa Cubie A7A (arm64, 4GB RAM)
• OS: Debian 11 (bullseye)
• Model/Provider: Dashscope (kimi-k2.5, vision-capable)
• Channel: Feishu (WebSocket mode)

Checklist

• My code follows the style of this project
• I have performed a self-review of my own changes
• Backward compatible — messages without Media work exactly as before
• All existing tests pass (make test)
• Addresses all 3 review comments from @mengzhuo on feat(agent): add vision/image support to agent pipeline #990

[yinwm]

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[yinwm]

LGTM

[yinwm]

thanks for the pr

[Orgmar]

@shikihane Solid rewrite of the vision support. Switching to filetype.MatchFile() for magic-bytes detection and streaming base64 via io.Copy to keep memory at ~1.33x instead of 2.6x are both really clean improvements. Nice that you also made the size limit configurable through config.MaxMediaSize instead of keeping it hardcoded. The fact that you verified it end-to-end on actual hardware (Radxa Cubie A7A with Feishu + Dashscope) is a great touch.

We are building the PicoClaw Dev Group on Discord for contributors to connect and collaborate. If you are interested, send an email to support@sipeed.com with the subject [Join PicoClaw Dev Group] + Your GitHub account and we will send you the invite link!