Skip to content

tools.md

Latest commit

 

History

History
513 lines (466 loc) · 53 KB

tools.md

File metadata and controls

513 lines (466 loc) · 53 KB
summary Systematic Peekaboo tool verification plan using Playground and file logs
read_when
planning or executing the comprehensive tool regression pass
picking up the Playground-based test assignment

Peekaboo Tool Playground Test Plan

Assignment & Expectations

  • Validate every native Peekaboo tool/CLI command (see the CLI command reference) against the Playground app so future automation runs have deterministic coverage.
  • For each tool run, capture an OSLog transcript with Apps/Playground/scripts/playground-log.sh --output <file> so we have durable evidence that the action completed (e.g., [Click], [Scroll] entries).
  • Update this document every time you start/finish a tool, and log deeper repro notes or bugs under Apps/Playground/PLAYGROUND_TEST.md so the next person can keep going.
  • Fix any issues you discover while executing the plan. If a fix is large, land it first, then rerun the affected tool plan and refresh the log artifacts.
  • Run the CLI via Poltergeist so you never test stale bits:
    • Preferred (always works): pnpm run peekaboo -- <command>
    • Optional (if your shell is wired for it): polter peekaboo -- <command>
    • For long runs, use tmux.

Environment & Logging Setup

  1. Ensure Poltergeist is healthy: pnpm run poltergeist:status; start it with pnpm run poltergeist:haunt if needed.
  2. Launch Playground (Apps/Playground/Playground.app via Xcode or open Apps/Playground/Playground.xcodeproj). Keep it foregrounded on Space 1 to avoid focus surprises.
    • Prefer the dedicated fixture windows (menu Fixtures, shortcuts ⌘⌃1…⌘⌃8) so each tool targets a stable window title (“Click Fixture”, “Dialog Fixture”, “Scroll Fixture”, etc.) instead of relying on TabView state.
  3. Prepare a log root once per session: 
    LOG_ROOT=${LOG_ROOT:-$PWD/.artifacts/playground-tools}
mkdir -p "$LOG_ROOT"
  4. Before you run any Peekaboo tool, arm a category-specific log capture so we can diff pre/post state: 
    TOOL=Click   # e.g. Click/Text/Menu/Window/Scroll/Drag/Keyboard/Focus/Gesture/Control/App
LOG_FILE="$LOG_ROOT/$(date +%Y%m%d-%H%M%S)-${TOOL,,}.log"
./Apps/Playground/scripts/playground-log.sh -c "$TOOL" --last 10m --all -o "$LOG_FILE"
    • Note: On some macOS 26 setups, unified logging may not retain info lines for long. When collecting evidence, prefer smaller windows (e.g. --last 2m) immediately after each action.
  5. Keep the Playground UI on the matching view (ClickTestingView, TextInputView, etc.) and run pnpm run peekaboo -- see --app Playground anytime you need a fresh snapshot ID for element targeting. Record the snapshot ID in your notes.
  6. After executing the tool, append verification notes (log file path, snapshot ID, observed behavior) to the table below and add detailed findings to Apps/Playground/PLAYGROUND_TEST.md.

Execution Loop

  1. Pick a tool from the matrix (start with Interaction tools, then cover window/app utilities, then the remaining system/automation commands).
  2. Review the tool doc under docs/commands/<tool>.md and skim the command implementation in Apps/CLI/Sources/PeekabooCLI/Commands/** so you understand its parameters and edge cases before running it.
  3. Stage the Playground view + log capture as described above.
  4. Run the suggested CLI smoke tests plus the extra edge cases listed per tool (invalid targets, timing edge cases, multi-step flows).
  5. Confirm Playground reflects the action (UI changes + OSLog evidence). Capture screenshots if a regression needs a visual repro.
  6. File and fix bugs immediately; rerun the plan for the affected tool to prove the fix.
  7. Update the status column and include the log artifact path so the next person knows what already passed.

Performance Checks

  • Capture performance summaries whenever a tool feels “slow” (or after fixing perf regressions) so we have a hard baseline.
  • Use Apps/Playground/scripts/peekaboo-perf.sh to run a command repeatedly and write a *-summary.json alongside the per-run JSON payloads (it reads data.execution_time or data.executionTime when available): 
    ./Apps/Playground/scripts/peekaboo-perf.sh --name see-click-fixture --runs 10 -- \
  see --app boo.peekaboo.playground.debug --mode window --window-title "Click Fixture" --json-output
  • Current reference baseline (2025-12-17, Click Fixture): see p95 ≈ 0.97s, click p95 ≈ 0.18s (.artifacts/playground-tools/20251217-174822-perf-see-click-clickfixture-summary.json).
  • Additional baselines (2025-12-17):
    • Scroll Fixture (scroll --on vertical-scroll, 15 runs): wall p95 ≈ 0.30s, exec p95 ≈ 0.12s (.artifacts/playground-tools/20251217-224849-scroll-vertical-scroll-fixture-summary.json).
    • System menu list-all (3 runs): wall p95 ≈ 0.61s (.artifacts/playground-tools/20251217-224944-menu-list-all-system-summary.json).

Tool Matrix

Vision & Capture

Tool Playground coverage Log focus Sample CLI entry point Status Latest log
see Prefer fixture windows (“Click Fixture”, “Scroll Fixture”, etc.) Capture snapshot metadata via CLI output + optional Playground logs for follow-on actions polter peekaboo -- see --app Playground --mode window --window-title "Click Fixture" Verified – --window-title now resolves against ScreenCaptureKit windows and element detection is pinned to the captured CGWindowID .artifacts/playground-tools/20251217-153107-see-click-for-move.json
image Playground window (full or element-specific) Use Image artifacts; note timestamp in LOG_FILE polter peekaboo -- image window --app Playground --output /tmp/playground-window.png Verified – window + screen captures succeed after capture fallback fix .artifacts/playground-tools/20251116-082109-image-window-playground.json, .artifacts/playground-tools/20251116-082125-image-screen0.json
capture capture live against Playground (5–10s) + capture video ingest smoke Verify artifacts (metadata.json, contact.png, frames) + optional MP4 (--video-out) polter peekaboo -- capture live --mode window --app Playground --duration 5 --threshold 0 --json-output Verified – live writes contact sheet + metadata; video ingest + --video-out covered .artifacts/playground-tools/20251217-133751-capture-live.json, .artifacts/playground-tools/20251217-180155-capture-video.json, .artifacts/playground-tools/20251217-184010-capture-live-videoout.json, .artifacts/playground-tools/20251217-184010-capture-video-videoout.json
list Validate apps, windows, screens, menubar, permissions while Playground is running playground-log optional (Window for focus changes) polter peekaboo -- list windows --app Playground etc. Verified – apps/windows/screens/menubar/permissions captured 2025-11-16 .artifacts/playground-tools/20251116-142111-list-apps.json, .artifacts/playground-tools/20251116-142111-list-windows-playground.json, .artifacts/playground-tools/20251116-142122-list-screens.json, .artifacts/playground-tools/20251116-142122-list-menubar.json, .artifacts/playground-tools/20251116-142122-list-permissions.json
tools Compare CLI output against ToolRegistry No Playground log required; attach output to notes polter peekaboo -- tools > $LOG_ROOT/tools.txt Verified – native tool listing captured 2025-12-19 .artifacts/playground-tools/20251219-001215-tools.txt
run Execute scripted multi-step flows against Playground fixtures Logs depend on embedded commands polter peekaboo -- run docs/testing/fixtures/playground-smoke.peekaboo.json Verified – smoke script drives Text Fixture and type resolves basic-text-field deterministically .artifacts/playground-tools/20251217-221643-run-playground-smoke.json, .artifacts/playground-tools/20251217-221643-run-playground-smoke-text.log
sleep Inserted between Playground actions Observe timestamps in log file polter peekaboo -- sleep 1500 Verified – manual timing around CLI pause python wrapper measuring pnpm run peekaboo -- sleep 2000
clean Snapshot cache after see runs Inspect ~/.peekaboo/snapshots & ensure Playground unaffected polter peekaboo -- clean --snapshot <id> Verified – removed snapshot 5408D893… and confirmed re-run reports none .peekaboo/snapshots/5408D893-E9CF-4A79-9B9B-D025BF9C80BE (deleted)
clipboard Clipboard smoke (text/file/image + save/restore) Verify readback + binary export + restore user clipboard polter peekaboo -- clipboard --action set --image-path assets/peekaboo.png --json-output Verified – CLI set/get (file+image) and cross-invocation save/restore (2025-12-17) .artifacts/playground-tools/20251217-192349-clipboard-get-image.json
config Validate config commands while Playground idle N/A polter peekaboo -- config show Verified – show/validate outputs captured 2025-11-16 .artifacts/playground-tools/20251116-051200-config-show-effective.json
permissions Ensure status/grant flow works with Playground playground-log App category (should log when permissions toggled) polter peekaboo -- permissions status Verified – Screen Recording & Accessibility granted .artifacts/playground-tools/20251116-051000-permissions-status.json
learn Dump agent guide N/A polter peekaboo -- learn > $LOG_ROOT/learn.txt Verified – latest dump saved 2025-11-16 .artifacts/playground-tools/20251116-051300-learn.txt
bridge Bridge host connectivity (local vs Peekaboo.app/Clawdbot) N/A polter peekaboo -- bridge status --json-output Verified – local selection + unauthorized host responses are now structured (no EOF) .artifacts/playground-tools/20251217-133751-bridge-status.json

Interaction Tools

Tool Playground surface Log category Sample CLI Status Latest log
click Click Fixture window Click polter peekaboo -- click "Single Click" --app boo.peekaboo.playground.debug --snapshot <id> Verified – Click Fixture E2E incl. double/right/context menu (2025-12-18) .artifacts/playground-tools/20251218-004335-click.log, .artifacts/playground-tools/20251218-004335-menu.log
type Text Fixture window Text + Focus polter peekaboo -- type "Hello Playground" --clear --snapshot <id> Verified – Text Fixture E2E + text-field focusing (2025-12-18) .artifacts/playground-tools/20251218-001923-text.log
press Keyboard Fixture window Keyboard polter peekaboo -- press return --snapshot <id> Verified – keypresses + repeats logged (2025-12-17) .artifacts/playground-tools/20251217-152138-keyboard.log
hotkey Playground menu shortcuts Keyboard & Menu polter peekaboo -- hotkey --keys "cmd,1" Verified – digit hotkeys (2025-12-17) .artifacts/playground-tools/20251217-152100-menu.log
scroll Scroll Fixture window Scroll polter peekaboo -- scroll --direction down --amount 8 --on vertical-scroll --snapshot <id> Verified – scroll offsets logged (2025-12-18) .artifacts/playground-tools/20251218-012323-scroll.log
swipe Scroll Fixture gesture area Gesture polter peekaboo -- swipe --from-coords <x,y> --to-coords <x,y> Verified – swipe direction + distance logged (2025-12-18), plus long-press hold .artifacts/playground-tools/20251218-012323-gesture.log
drag Drag Fixture window Drag polter peekaboo -- drag --from <elem> --to <elem> --snapshot <id> Verified – item dropped into zone (2025-12-18) .artifacts/playground-tools/20251218-002005-drag.log
move Click Fixture mouse probe Control polter peekaboo -- move --id <elem> --snapshot <id> --smooth Verified – cursor movement emits deterministic probe logs (2025-12-17) .artifacts/playground-tools/20251217-153107-control.log

Windows, Menus, Apps

Tool Playground validation target Log category Sample CLI Status Latest log
window Window Fixture window + list windows bounds Window polter peekaboo -- window move --app boo.peekaboo.playground.debug --window-title "Window Fixture" Verified – focus/move/resize + minimize/maximize covered (2025-12-17) .artifacts/playground-tools/20251217-183242-window.log
space macOS Spaces while Playground anchored on Space 1 Space polter peekaboo -- space list --detailed Verified – list/switch/move now emit [Space] logs (instr. added 2025-11-16) .artifacts/playground-tools/20251116-205548-space.log
menu Playground “Test Menu” Menu polter peekaboo -- menu click --app boo.peekaboo.playground.debug --path "Test Menu>Submenu>Nested Action A" Verified – nested menu click logged (2025-12-18) .artifacts/playground-tools/20251218-002308-menu.log
menubar macOS menu extras (Wi-Fi, Clock) plus Playground status icons Menu (system) polter peekaboo -- menubar list --json-output Verified – list + click captured; logs via Control Center predicate .artifacts/playground-tools/20251116-053932-menubar.log
app Launch/quit/focus Playground + helper apps (TextEdit) App + Focus polter peekaboo -- app list --include-hidden --json-output Verified – Playground app list/switch/hide/launch captured 2025-11-16 .artifacts/playground-tools/20251116-195420-app.log
open Open Playground fixtures/documents App/Focus polter peekaboo -- open Apps/Playground/README.md --app TextEdit --json-output Verified – TextEdit + browser + no-focus covered 2025-11-16 .artifacts/playground-tools/20251116-200220-open.log
dock Dock item interactions w/ Playground icon App + Window polter peekaboo -- dock list --json-output Verified – right-click + menu selection now captured with [Dock] logs .artifacts/playground-tools/20251116-205850-dock.log
dialog Dialogs tab (Save/Open panels + alerts w/ text field) Dialog polter peekaboo -- dialog list --app Playground Verified – use Playground’s built-in dialog fixtures (no TextEdit required) .artifacts/playground-tools/20251116-054316-dialog.log
visualizer Visual feedback overlays while Playground is visible Visual confirmation (overlays render) + JSON dispatch report polter peekaboo -- visualizer --json-output Verified – dispatch report + manual overlay check .artifacts/playground-tools/20251217-204548-visualizer.json

Automation & Integrations

Tool Playground coverage Log category Sample CLI Status Latest log
agent Run natural-language tasks scoped to Playground (“click the single button”) Captures whichever sub-tools fire (Click, Text, etc.) polter peekaboo -- agent "Say hi" --max-steps 1 Verified – GPT-5.1 runs logged 2025-11-17 (see notes re: tool count bug) .artifacts/playground-tools/20251117-011345-agent.log
mcp Verify MCP server can enumerate tools via stdio MCP MCPORTER list peekaboo-local --stdio "$PEEKABOO_BIN mcp" --timeout 20 Verified – MCP tools list captured (2025-12-19) .artifacts/playground-tools/20251219-001200-mcp-list.log

Status Legend: Not started = no logs yet, In progress = partial run logged, Blocked = awaiting fix, Verified = passing with log path recorded.

Per-Tool Test Recipes

The following subsections spell out the concrete steps, required Playground surface, and expected log artifacts for each tool. Check these off (and bump the status above) as you progress.

Vision & Capture

see

  • View: Any (start with ClickTestingView to guarantee clear elements).
  • Steps:
    1. Bring Playground to front (polter peekaboo -- app switch --to Playground).
    2. polter peekaboo -- see --app Playground --output "$LOG_ROOT/see-playground.png".
    3. Record snapshot ID printed to stdout, verify ~/.peekaboo/snapshots/<id>/map.json references Playground elements (single-click-button, etc.).
  • Log capture: Optional Click capture if you immediately chain interactions with the new snapshot; otherwise store the PNG + snapshot metadata path.
  • Pass criteria: Snapshot folder exists, UI map contains Playground identifiers, CLI exits 0.
  • 2025-11-16 verification: Re-enabled the ScreenCaptureKit path inside Core/PeekabooCore/Sources/PeekabooAutomation/Services/Capture/ScreenCaptureService.swift so the modern API runs before falling back to CGWindowList. polter peekaboo -- see --app Playground --json-output --path .artifacts/playground-tools/20251116-082056-see-playground.png now succeeds (snapshot 5B5A2C09-4F4C-4893-B096-C7B4EB38E614) and drops .artifacts/playground-tools/20251116-082056-see-playground.{json,png}.
  • 2025-12-17 rerun: pnpm run peekaboo -- see --app Playground --path .artifacts/playground-tools/20251217-132837-see-playground.png --json-output > .artifacts/playground-tools/20251217-132837-see-playground.json succeeded (Peekaboo main/842434be-dirty).

image

  • View: Keep Playground on ScrollTestingView to capture dynamic content.
  • Steps:
    1. polter peekaboo -- image window --app Playground --output "$LOG_ROOT/image-playground.png".
    2. Repeat with --screen main --bounds 100,100,800,600 to cover coordinate cropping.
  • 2025-11-16 verification: After restoring the ScreenCaptureKit → CGWindowList fallback order, both window and screen captures succeed. Saved .artifacts/playground-tools/20251116-082109-image-window-playground.{json,png} and .artifacts/playground-tools/20251116-082125-image-screen0.{json,png}; CLI debug logs still note tiny background windows but the primary Playground window captures at 1200×852.

capture

  • View: Any; keep Playground frontmost so the window is captureable.
  • Steps:
    1. polter peekaboo -- capture live --mode window --app Playground --duration 5 --threshold 0 --json-output > "$LOG_ROOT/capture-live.json".
    2. Confirm the JSON points at the expected output directory (kept frames + contact.png + metadata.json).
    3. Optional: repeat with --highlight-changes to ensure highlight rendering doesn’t crash.
  • Video ingest add-on:
    1. Generate a deterministic motion video: ffmpeg -hide_banner -loglevel error -y -f lavfi -i testsrc2=size=960x540:rate=30 -t 2 /tmp/peekaboo-capture-src.mp4.
    2. Run: polter peekaboo -- capture video /tmp/peekaboo-capture-src.mp4 --sample-fps 4 --no-diff --json-output > "$LOG_ROOT/capture-video.json".
    3. Confirm framesKept ≥ 2 and the output directory contains keep-*.png, contact.png, and metadata.json.
  • MP4 add-on:
    1. Re-run either live or video ingest with --video-out /tmp/peekaboo-capture.mp4.
    2. Confirm the JSON includes videoOut and the MP4 exists and is non-empty.
  • Pass criteria: ≥1 kept frame, metadata.json exists, and the run exits 0 (a noMotion warning is acceptable for static inputs).
  • Schema check: Cross-check MCP capture meta fields in docs/commands/mcp-capture-meta.md against the JSON payload.
  • 2025-12-18 run:
    • Live window capture (Playground) completed successfully and respects short durations again (no longer stalls ~10s on the ScreenCaptureKit→CG fallback path): .artifacts/playground-tools/20251218-024517-capture-live-window-fast.json and .artifacts/playground-tools/20251218-024517-capture-live-window-fast/.
    • Video ingest (synthetic ffmpeg testsrc2, --sample-fps 4 --no-diff) produced 9 kept frames + contact sheet: .artifacts/playground-tools/20251218-022826-capture-video.json and .artifacts/playground-tools/20251218-022826-capture-video/.

list

  • Scenarios: list apps, list windows --app Playground, list screens, list menubar, list permissions.
  • Steps:
    1. With Playground running, execute each subcommand and ensure Playground appears with expected bundle ID/window title.
    2. For list windows, compare returned bounds vs. WindowTestingView readout.
    3. For list menubar, capture the result and cross-check with actual status items.
  • Logs: Use playground-log Window category when forcing focus changes to validate app switch interplay.

tools

  • Steps:
    1. polter peekaboo -- tools > "$LOG_ROOT/tools.txt".
    2. Compare entries to the Interaction/Window commands listed here; flag gaps.
  • Verification: Output includes click/type/etc. with descriptions.

run

  • Setup: Create a sample .peekaboo.json (store under docs/testing/fixtures/ once defined) that performs see, click, type, and scroll.
  • Steps:
    1. Start Keyboard, Click, and Text log captures.
    2. polter peekaboo -- run docs/testing/fixtures/playground-smoke.peekaboo.json --output "$LOG_ROOT/run-playground.json" --json-output.
    3. Confirm each embedded step produced matching log entries (the script opens the Text Fixture window via ⌘⌃2 before running see/click/type).
  • No-fail-fast add-on:
    1. Run polter peekaboo -- run docs/testing/fixtures/playground-no-fail-fast.peekaboo.json --no-fail-fast --json-output > "$LOG_ROOT/run-no-fail-fast.json".
    2. Verify the JSON is a single payload (no double-printed JSON) and reports success=false with failedSteps=1.
    3. Confirm the Playground Click log includes a Single click entry even though the script intentionally includes a failing step first.
  • Notes: Update fixture when tools change to keep coverage aligned.
  • 2025-12-17 run: Updated docs/testing/fixtures/playground-smoke.peekaboo.json to open the Text Fixture window (hotkey ⌘⌃2) and reran successfully: .artifacts/playground-tools/20251217-173849-run-playground-smoke.json plus matching OSLog evidence in .artifacts/playground-tools/20251217-173849-run-playground-smoke-{keyboard,click,text}.log.

sleep

  • Steps:
    1. Run date +%s then polter peekaboo -- sleep 2000 within tmux.
    2. Immediately issue a click command and ensure the log timestamps show ≥2s gap.
  • Verification: Playground log lines prove no action fired during sleep window.
  • 2025-11-16 run: Measured via python - <<'PY' ... subprocess.run(["pnpm","run","peekaboo","--","sleep","2000"]) ... → actual pause ≈2.24 s (CLI printed ✅ Paused for 2.0s). No Playground interaction necessary.

clean

  • Steps:
    1. Generate two snapshots via see.
    2. polter peekaboo -- clean --older-than 1m and confirm only newest snapshot remains.
    3. Attempt to interact using purged snapshot ID and assert command fails with helpful error.
  • Artifacts: Directory listing before/after.
  • 2025-11-16 run: Created snapshots 5408D893-… and 129101F5-… via back-to-back see captures (artifacts saved under .artifacts/playground-tools/*clean-see*.png). Ran polter peekaboo -- clean --snapshot 5408D893-… (freed 453 KB), verified folder removal (ls ~/.peekaboo/snapshots). Re-running the same clean command returned “No snapshots to clean”, confirming deletion.
  • 2025-12-17 rerun: Using a cleaned snapshot now yields SNAPSHOT_NOT_FOUND for snapshot-scoped commands (instead of ELEMENT_NOT_FOUND), which is much clearer for end-to-end scripts.
    • Snapshot + clean: .artifacts/playground-tools/20251217-201134-see-for-snapshot-missing.json, .artifacts/playground-tools/20251217-201134-clean-snapshot.json
    • Command failures:
      • .artifacts/playground-tools/20251217-201134-click-snapshot-missing.json
      • .artifacts/playground-tools/20251217-201134-move-snapshot-missing.json
      • .artifacts/playground-tools/20251217-201134-scroll-snapshot-missing.json
      • .artifacts/playground-tools/20251217-202239-snapshot-missing-drag.json
      • .artifacts/playground-tools/20251217-202239-snapshot-missing-swipe.json
      • .artifacts/playground-tools/20251217-202239-snapshot-missing-type.json
      • .artifacts/playground-tools/20251217-202239-snapshot-missing-hotkey.json
      • .artifacts/playground-tools/20251217-202239-snapshot-missing-press.json

clipboard

  • Steps:
    1. Scripted smoke: polter peekaboo -- run docs/testing/fixtures/clipboard-smoke.peekaboo.json --json-output > "$LOG_ROOT/clipboard-smoke.json".
    2. Cross-invocation save/restore: polter peekaboo -- clipboard --action save --slot original, then --action clear, then --action restore --slot original.
    3. File payload: polter peekaboo -- clipboard --action set --file-path /tmp/peekaboo-clipboard-smoke.txt --json-output.
    4. Image payload + export: polter peekaboo -- clipboard --action set --image-path assets/peekaboo.png --also-text "Peekaboo clipboard image smoke" --json-output, then polter peekaboo -- clipboard --action get --prefer public.png --output /tmp/peekaboo-clipboard-out.png --json-output.
  • Pass criteria: Script succeeds and clipboard is restored.
  • 2025-12-17 CLI evidence: .artifacts/playground-tools/20251217-192349-clipboard-{save-original,set-file,get-file-text,set-image,get-image,restore-original}.json plus exported /tmp/peekaboo-clipboard-out.png.

config

  • Focus: config show, config validate, config models.
  • Steps:
    1. Snapshot ~/.peekaboo/config.json (read-only).
    2. Run polter peekaboo -- config validate --verbose.
    3. Document provider list for later cross-check.
  • Notes: No Playground tie-in; just ensure CLI stability.
  • 2025-11-16 run: polter peekaboo -- config show --effective --json-output > .artifacts/playground-tools/20251116-051200-config-show-effective.json plus polter peekaboo -- config validate both succeeded; output confirms OpenAI key set + default save path. No edits performed.

permissions

  • Steps:
    1. polter peekaboo -- permissions status to confirm Accessibility/Screen Recording show Granted.
    2. If a permission is missing, follow docs/permissions.md to re-grant and note the steps.
    3. Capture console output.
  • 2025-11-16 run: polter peekaboo -- permissions status --json-output > .artifacts/playground-tools/20251116-051000-permissions-status.json returned both Screen Recording and Accessibility as granted (matching expectations); no Playground interaction required.

learn

  • Steps: polter peekaboo -- learn > "$LOG_ROOT/learn-latest.txt"; record commit hash displayed at top.
  • 2025-11-16 run: Saved .artifacts/playground-tools/20251116-051300-learn.txt for reference; includes commit metadata from peekaboo binary.

bridge

  • Steps:
    1. polter peekaboo -- bridge status and confirm it reports local execution vs. a remote host (Peekaboo.app / Clawdbot).
    2. polter peekaboo -- bridge status --verbose --json-output > "$LOG_ROOT/bridge-status.json" and sanity-check the selected host + probed sockets.
    3. Repeat with --no-remote to confirm local-only mode is explicit and stable.
  • Unauthorized host behavior:
    • If a remote host rejects the CLI due to TeamID allowlisting, the host should reply with unauthorizedClient (not close the socket/EOF).
    • This is regression-covered by Apps/CLI/Tests/CoreCLITests/PeekabooBridgeHostUnauthorizedResponseTests.swift (landed 2025-12-18).
  • Pass criteria: Clear host selection output and no crashes.
  • 2025-12-18 run:
    • Remote sockets were probed but both candidates returned internalError (“Bridge host returned no response”), so the CLI selected source=local as expected.
    • Note: this typically indicates an older Peekaboo/Clawdbot host build. Hosts built from main after 2025-12-18 should respond with a structured unauthorizedClient error instead.
    • Evidence: .artifacts/playground-tools/20251218-022612-bridge-status.json, .artifacts/playground-tools/20251218-022612-bridge-status-verbose.json, .artifacts/playground-tools/20251218-022612-bridge-status-no-remote.json.

Interaction Tools

click

  • View: ClickTestingView.
  • Log capture: ./Apps/Playground/scripts/playground-log.sh -c Click --last 10m --all -o "$LOG_ROOT/click-$(date +%s).log".
  • Test cases:
    1. Query-based click: polter peekaboo -- click "Single Click" (expect Click log + counter increment).
    2. ID-based click: polter peekaboo -- click --on B1 --snapshot <id> targeting single-click-button.
    3. Coordinate click: polter peekaboo -- click --coords 400,400 hitting the nested area.
    4. Coordinate validation: polter peekaboo -- click --coords , --json-output should fail with VALIDATION_ERROR (no crash).
    5. Error path: attempt to click disabled button and confirm descriptive elementNotFound guidance.
  • Verification: Playground counter increments, log file shows [Click] Single click... entries.
  • 2025-11-16 run:
    • Captured Click logs to .artifacts/playground-tools/20251116-051025-click.log.
    • Generated fresh snapshot 263F8CD6-E809-4AC6-A7B3-604704095011 via see (.artifacts/playground-tools/20251116-051120-click-see.{json,png}).
    • polter peekaboo -- click "Single Click" --snapshot <legacy snapshot> succeeded but targeted Ghostty (click hit terminal input); highlighting importance of focusing Playground first.
    • polter peekaboo -- app switch --to Playground followed by polter peekaboo -- click --on elem_6 --snapshot 263F8CD6-... successfully hit the “View Logs” button (Playground log recorded the click).
    • Coordinate click --coords 600,500 succeeded (see log); attempting --on elem_disabled produced expected elementNotFound error.
    • IDs like B1 are not stable in this build; rely on elem_* IDs from the see output.
  • 2025-12-17 Controls Fixture add-on:
    • Open “Controls Fixture” via ⌘⌃3, then drive checkboxes + segmented control by clicking snapshot IDs (--on elem_…) captured from see.
    • Important: ControlsView is scrollable; after any scroll, re-run see before clicking elements further down (otherwise snapshot coordinates can be stale).
    • Evidence: .artifacts/playground-tools/20251217-230454-control.log plus .artifacts/playground-tools/20251217-230454-see-controls-top.json and .artifacts/playground-tools/20251217-230454-see-controls-progress.json.

type

  • View: TextInputView.
  • Log capture: Text + Focus categories.
  • Test cases:
    1. polter peekaboo -- type "Hello Playground" --query "Basic" to fill the basic field.
    2. Use --clear then --append flows to verify editing.
    3. Tab-step typing with --tabs 2 into the secure field.
    4. Unicode input (emoji) to ensure no crash.
  • Verification: Field contents update, log shows [Text] Basic field changed entries.
  • 2025-11-16 run:
    • Logged .artifacts/playground-tools/20251116-051202-text.log.
    • Focused field via polter peekaboo -- click "Focus Basic Field" --snapshot 263F8CD6-… (snapshot from .artifacts/playground-tools/20251116-051120-click-see.json).
    • polter peekaboo -- type "Hello Playground" --clear --snapshot 263F8CD6-… updated the Basic Text Field (log shows “Basic text changed …”).
    • polter peekaboo -- type --tab 1 --snapshot 263F8CD6-… advanced focus to the Number field, followed by polter peekaboo -- type "42" --snapshot 263F8CD6-….
    • Validation error confirmed via polter peekaboo -- type "bad" --profile warp (proper error message).
    • Note: targets are determined by current focus; use helper buttons and click to focus before typing. Legacy --on / --query flags no longer exist.

press

  • View: KeyboardView “Key Press Detection” field (Keyboard tab).
  • Test cases:
    1. polter peekaboo -- press return --snapshot <id> after focusing the detection text field.
    2. polter peekaboo -- press up --count 3 --snapshot <id> to ensure repeated presses log individually.
    3. Invalid key handling (polter peekaboo -- press foo) should error.
  • 2025-11-16 verification:
    • Switched to the Keyboard tab via polter peekaboo -- hotkey --keys "cmd,option,7", captured .artifacts/playground-tools/20251116-090141-see-keyboardtab.{json,png} (snapshot C106D508-930C-4996-A4F4-A50E2E0BA91A), and focused the “Press keys here…” field with a coordinate click (--coords 760,300).
    • polter peekaboo -- press return --snapshot C106D508-… and polter peekaboo -- press up --count 3 --snapshot C106D508-… produced [boo.peekaboo.playground:Keyboard] Key pressed: … entries in .artifacts/playground-tools/20251116-090455-keyboard.log.
    • polter peekaboo -- press foo reports Unknown key: 'foo'. Run 'peekaboo press --help' for available keys. confirming validation and documenting the negative path.

hotkey

  • View: KeyboardView hotkey demo or main window (use cmd+shift+l to open log viewer).
  • Test cases:
    1. polter peekaboo -- hotkey cmd,shift,l should toggle the “Clear All Logs” command (log viewer clears entries).
    2. polter peekaboo -- hotkey cmd,1 to trigger Test Menu action; watch Menu logs.
    3. Negative test: provide invalid chord order to ensure validation message.
  • Verification: Playground Keyboard log file shows the keystrokes fired.
  • 2025-11-16 run:
    • Logs stored at .artifacts/playground-tools/20251116-051654-keyboard-hotkey.log (contains entries for L and 1 corresponding to the combos).
    • polter peekaboo -- hotkey --keys "cmd,shift,l" --snapshot 11227301-05DE-4540-8BE7-617F99A74156 (clears logs via shortcut).
    • polter peekaboo -- hotkey --keys "cmd,1" --snapshot … switches Playground tabs.
    • polter peekaboo -- hotkey --keys "foo,bar" correctly fails with Unknown key: 'foo'.

scroll

  • View: ScrollTestingView vertical/horizontal sections (switch using polter peekaboo -- hotkey --keys "cmd,option,4" to trigger the new Test Menu shortcut).
  • Test cases:
    1. polter peekaboo -- scroll --direction down --amount 6 --snapshot <id> for vertical movement.
    2. polter peekaboo -- scroll --direction right --amount 4 --smooth --snapshot <id> for horizontal smooth scrolling.
    3. polter peekaboo -- scroll --direction down --amount 6 --on vertical-scroll --snapshot <id> and ... --direction right --amount 4 --on horizontal-scroll --snapshot <id> to prove the new identifiers work end-to-end.
    4. Nested scroll targeting: --on nested-inner-scroll and --on nested-outer-scroll (Scroll Fixture “Nested Scroll Views” section).
  • 2025-11-16 verification:
    • Captured snapshot .artifacts/playground-tools/20251116-194615-see-scrolltab.json (snapshot 649EB632-ED4B-4935-9F1F-1866BB763804) and re-ran both scroll commands with --on vertical-scroll and --on horizontal-scroll. The CLI outputs live at .artifacts/playground-tools/20251116-194652-scroll-vertical.json and .artifacts/playground-tools/20251116-194708-scroll-horizontal.json (both ✅ now that the Playground view exposes identifiers and the ScrollService snapshot cache preserves them).
    • Added .artifacts/playground-tools/20251116-194730-scroll.log via ./Apps/Playground/scripts/playground-log.sh -c Scroll --last 10m --all -o …; it shows the [Scroll] direction=down and [Scroll] direction=right events emitted by AutomationEventLogger.
  • 2025-12-17 rerun:
    • Re-validated Scroll Fixture window-scoped scrolling (vertical/horizontal + nested target commands) with .artifacts/playground-tools/20251217-222958-scroll.log.
  • 2025-12-18 rerun:
    • Verified Scroll Fixture again, but this time with another app frontmost (Ghostty) to prove auto-focus uses snapshot metadata reliably even when see snapshots do not include windowID.
    • Evidence:
      • .artifacts/playground-tools/20251218-012323-scroll.log (Scroll offsets + nested inner/outer offsets logged by Playground).
      • .artifacts/playground-tools/20251218-012323-click-scroll-{top,middle,bottom}.json (Clicking fixture buttons via snapshot IDs).
      • .artifacts/playground-tools/20251218-012323-scroll-{vertical-down,vertical-up,horizontal-right,horizontal-left,nested-outer-down,nested-inner-down}.json (CLI evidence per scroll variant).

swipe

  • View: Gesture Testing area.
  • Test cases:
    1. polter peekaboo -- swipe --from-coords 1100,520 --to-coords 700,520 --duration 600.
    2. polter peekaboo -- swipe --from-coords 850,600 --to-coords 850,350 --duration 800 --profile human.
    3. Negative test: polter peekaboo -- swipe … --right-button should error.
  • 2025-11-16 verification:
    • Used snapshot DBFDD053-4513-4603-B7C3-9170E7386BA7 (see .artifacts/playground-tools/20251116-085714-see-scrolltab.{json,png}) to keep the tab selection stable.
    • Horizontal and vertical commands above completed successfully; Playground log .artifacts/playground-tools/20251116-090041-gesture.log shows [boo.peekaboo.playground:Gesture] entries with exact coordinates, profiles, and step counts.
    • polter peekaboo -- swipe --from-coords 900,520 --to-coords 700,520 --right-button returns Right-button swipe is not currently supported…, matching expectations.
  • 2025-12-18 rerun:
    • Verified swipe-direction logging + long-press detection on the Scroll Fixture gesture tiles.
    • Evidence: .artifacts/playground-tools/20251218-012323-gesture.log plus .artifacts/playground-tools/20251218-012323-swipe-right.json and .artifacts/playground-tools/20251218-012323-long-press.json.

drag

  • View: DragDropView (tab is hidden on launch—run polter peekaboo -- click --snapshot <id> --on elem_79 right after see to activate the “Drag & Drop” tab radio button).
  • Test cases:
    1. Drag Item A (elem_15) into drop zone 1 (elem_24) via --from/--to.
    2. Drag Item B (elem_17) into drop zone 2 (elem_26) and capture JSON output for artifacting.
    3. (Optional) Drag the reorderable list rows (elem_37…elem_57) once additional coverage is needed.
  • 2025-11-16 verification:
    • A reusable PlaygroundTabRouter + header “Go to Drag & Drop” control keep the TabView state predictable, and more importantly elem_79 now works deterministically—clicking it flips the TabView so subsequent see runs expose DragDropView element IDs (see .artifacts/playground-tools/20251116-085142-see-afterclick-elem79.{json,png} with snapshot BBF9D6B9-26CB-4370-8460-6C8188E7466C).
    • polter peekaboo -- drag --snapshot BBF9D6B9-26CB-4370-8460-6C8188E7466C --from elem_15 --to elem_24 --duration 800 --steps 40 succeeded; Playground log .artifacts/playground-tools/20251116-085233-drag.log shows “Started dragging: Item A”, “Hovering over zone1”, and “Item dropped… zone1”, plus the CLI-side [boo.peekaboo.playground:Drag] drag from=… entry.
    • Captured a second run with JSON output (.artifacts/playground-tools/20251116-085346-drag-elem17.json) dragging Item B to zone2 so we have structured metadata (coords, duration, profile) for regression diffs.
    • We still keep the older coordinate-only recipe around as a fallback, but the default regression loop is now: focus Playground → see → click --on elem_79 → drag --snapshot … --from elem_XX --to elem_YY → archive the Drag log + CLI JSON.
  • 2025-12-17 Controls Fixture add-on:
    • Slider adjustment works via drag when you compute a --to-coords inside the slider’s frame using the snapshot JSON.
    • Evidence: .artifacts/playground-tools/20251217-230454-drag-slider.json and the corresponding [Control] Slider moved … lines in .artifacts/playground-tools/20251217-230454-control.log.

move

  • View: ClickTestingView (target nested button) or ScrollTestingView.
  • Test cases:
    1. polter peekaboo -- move 600,600 for instant pointer relocation.
    2. Smooth query-based move: polter peekaboo -- move --to "Focus Basic Field" --snapshot <id> --smooth.
    3. polter peekaboo -- move --center --duration 300 --steps 15.
    4. polter peekaboo -- move --coords 600,600 (alias coverage).
    5. Negative test: polter peekaboo -- move 1,2 --center should error (conflicting targets).
  • 2025-11-16 verification:
    • Commands above rerun with snapshot DBFDD053-4513-4603-B7C3-9170E7386BA7; CLI outputs saved implicitly (no JSON mode). Pointer jumps succeeded (move 600,600, move --center).
    • move --to "Focus Basic Field" --snapshot ... --smooth works with snapshot-based targeting; repeated runs confirm the lookup is stable.
    • Focus logger still doesn’t capture these events (playground-log -c Focus remains empty), so we rely on CLI output for evidence until instrumentation is added.
  • 2025-12-17 re-verification:
    • --coords is now accepted (Commander metadata updated) and treated as an alias for the positional coordinates.
    • Conflicting targets now fail at runtime (MoveCommand explicitly runs validate() before executing).
    • Playground evidence loop using Click Fixture probe:
      • Snapshot: .artifacts/playground-tools/20251217-194922-see-click-fixture.json
      • CLI: .artifacts/playground-tools/20251217-194947-move-coords-probe.json
      • Playground logs: .artifacts/playground-tools/20251217-195012-move-out-control.log (contains Mouse entered probe area / Mouse exited probe area).

Windows, Menus, Apps

window

  • View: WindowTestingView (or any app with a movable window; Playground itself works for focus/move/resize).
  • Test cases:
    1. polter peekaboo -- window focus --app Playground.
    2. polter peekaboo -- window move --app Playground -x 100 -y 100.
    3. polter peekaboo -- window resize --app Playground --width 900 --height 600.
    4. polter peekaboo -- window set-bounds --app Playground --x 200 --y 200 --width 1100 --height 700.
    5. polter peekaboo -- window list --app Playground --json-output.
  • 2025-11-16 verification:
    • Commands rerun with Playground as the target: .artifacts/playground-tools/20251116-194858-window-list-playground.json, ...-window-move-playground.json, ...-window-resize-playground.json, ...-window-setbounds-playground.json, and ...-window-focus-playground.json capture each CLI invocation.
    • Window log .artifacts/playground-tools/20251116-194900-window.log shows [Window] focus, move, resize, and set_bounds entries with updated bounds, confirming instrumentation now covers the Playground window itself.
  • 2025-12-18 regression fix:
    • window list no longer returns duplicate entries for the same window_id (which previously happened for Playground’s fixture windows, confusing scripts that key off window_id).
    • Evidence: .artifacts/playground-tools/20251218-022217-window-list-playground-dedup.json (no duplicate window_id values).

space

  • Scenario: Single Space (current setup). Need additional Space to test multi-space behavior.
  • Test cases:
    1. polter peekaboo -- space list --detailed --json-output.
    2. polter peekaboo -- space switch --to 1 (happy path) and expect error for --to 2 when only one Space exists.
    3. polter peekaboo -- space move-window --app Playground --window-index 0 --to 1 --follow.
  • 2025-11-16 run:
    • Latest artifacts: .artifacts/playground-tools/20251116-205527-space-list.json, ...205532-space-list-detailed.json, ...205536-space-switch-1.json, ...205541-space-move-window.json, plus ...195602-space-switch-2.json for the expected validation error.
    • AutomationEventLogger now emits [Space] entries (list count + actions) captured via .artifacts/playground-tools/20251116-205548-space.log.
    • Still only one desktop (Space IDs 1-1), so the --to 2 path continues to produce VALIDATION_ERROR (Available: 1-1) as designed.

menu

  • View: Playground’s “Test Menu” items (standard menu bar). Context menus on the right-click-area still require click rather than menu because menu click doesn’t accept coordinate targets yet.
  • Test cases:
    1. polter peekaboo -- menu click --app Playground --path "Test Menu>Test Action 1".
    2. polter peekaboo -- menu click --app Playground --path "Test Menu>Submenu>Nested Action A".
    3. Disabled menu handling: polter peekaboo -- menu click --app Playground --path "Test Menu>Disabled Action" should fail with a descriptive error.
  • 2025-11-16 verification:
    • Re-ran the command set; artifacts include .artifacts/playground-tools/20251116-195020-menu-click-action.json, ...195024-menu-click-submenu.json, and ...195022-menu-click-disabled.json (the last exits with INTERACTION_FAILED and message Menu item is disabled: ...).
    • Playground Menu log .artifacts/playground-tools/20251116-195020-menu.log now shows each click (Test Action 1, Submenu > Nested Action A, and the disabled error), proving AutomationEventLogger coverage.
    • Context menu coverage is verified via click --right on the Click Fixture: .artifacts/playground-tools/20251217-165443-context-menu.log contains Context menu: Action 1/2/Delete entries emitted by Playground.
  • 2025-12-18 re-verification:
    • Confirmed a “real world” nested menu path with spaces (Fixtures > Open Window Fixture) opens the expected window.
    • Evidence: .artifacts/playground-tools/20251218-021541-menu-open-windowfixture.json + .artifacts/playground-tools/20251218-021541-window.log (Window became key for “Window Fixture”).

menubar

  • Target: macOS status items (Wi-Fi, Battery) or custom extras.
  • Test cases:
    1. polter peekaboo -- menubar list --json-output > .artifacts/playground-tools/20251116-141824-menubar-list.json.
    2. polter peekaboo -- menubar click "Wi-Fi" (or --index 9) and close Control Center manually afterward.
    3. polter peekaboo -- menubar click --index 2 to exercise Control Center by index.
  • 2025-11-16 run: Commands above succeeded; no dedicated Playground log yet (menu bar actions don’t flow through the app logger). The new list artifact reflects the current order, and the CLI output confirms the clicked items (Wi-Fi and Control Center).

app

  • Scenarios:
    1. polter peekaboo -- app list --include-hidden --json-output > $LOG_ROOT/app-list.json
    2. polter peekaboo -- app switch --to Playground
    3. polter peekaboo -- app hide --app Playground / polter peekaboo -- app unhide --app Playground
    4. polter peekaboo -- app launch "TextEdit" --json-output followed by polter peekaboo -- app quit --app TextEdit --json-output
  • 2025-11-16 verification:
    • Re-ran the flow: .artifacts/playground-tools/20251116-195420-app-list.json, ...195421-app-switch.json, ...195422-app-hide.json, ...195423-app-unhide.json, ...195424-app-launch-textedit.json, and ...195425-app-quit-textedit.json capture the CLI outputs.
    • App log .artifacts/playground-tools/20251116-195420-app.log shows the matching [App] list, switch, hide, unhide, launch, and quit entries with bundle IDs + PIDs.

open

  • Tests:
    1. polter peekaboo -- open Apps/Playground/README.md --app TextEdit --json-output > .artifacts/playground-tools/20251116-091415-open-readme-textedit.json.
    2. polter peekaboo -- open https://example.com --json-output > .artifacts/playground-tools/20251116-091422-open-example.json.
    3. polter peekaboo -- open Apps/Playground/README.md --app TextEdit --no-focus --json-output > .artifacts/playground-tools/20251116-091435-open-readme-textedit-nofocus.json.
  • 2025-11-16 verification: Latest run captured .artifacts/playground-tools/20251116-200220-open.log with the three [Open] entries (TextEdit focus, browser focus, TextEdit --no-focus), alongside the corresponding CLI JSON artifacts.

dock

  • Tests:
    1. polter peekaboo -- dock list --json-output (artifact .artifacts/playground-tools/20251116-200750-dock-list.json).
    2. polter peekaboo -- dock launch Playground.
    3. polter peekaboo -- dock hide / polter peekaboo -- dock show.
    4. polter peekaboo -- dock right-click --app Finder --select "New Finder Window" (JSON artifact .artifacts/playground-tools/20251116-205828-dock-right-click.json).
  • 2025-11-16 verification:
    • [Dock] logger entries captured via .artifacts/playground-tools/20251116-205850-dock.log show list, launch Playground, hide, show, and the Finder right-click with selection=New Finder Window.
    • Context menu selection works once Finder is present in the Dock; if the menu doesn’t surface, re-run after focusing the Dock. No additional code changes required.

dialog

  • Scenario: Use Playground’s Dialogs tab to spawn deterministic Save/Open panels and alerts.
  • Steps to spawn dialogs:
    1. Launch Playground and switch to the Dialogs tab (Header button “Go to Dialogs”).
    2. Click “Show Save Panel” (or “Show Save Panel (Overwrite /tmp)” to exercise Replace flows). Use “Show Save Panel (TextEdit-like)” to add a file-format accessory view + tags field closer to real-world apps.
    3. Optional: Click “Show Alert (Text Field)” to exercise dialog input against a sheet-local text field.
  • Tests:
    1. polter peekaboo -- dialog list --app Playground --json-output > .artifacts/playground-tools/<timestamp>-dialog-list.json.
    2. polter peekaboo -- dialog click --button "Cancel" --app Playground --json-output > .artifacts/playground-tools/<timestamp>-dialog-click-cancel.json.
    3. (Alert w/ text field) polter peekaboo -- dialog input --app Playground --index 0 --text "NAME0" --clear --json-output > .artifacts/playground-tools/<timestamp>-dialog-input.json.
    4. (Save panel) polter peekaboo -- dialog file --app Playground --path /tmp --name playground-dialog-out.txt --ensure-expanded --select default --json-output > .artifacts/playground-tools/<timestamp>-dialog-file-save.json.
  • Verification notes:
    • Prefer Playground’s Dialogs tab over TextEdit for repeatable coverage (no “dirty document” preconditions).
    • Capture a Playground log excerpt for each run (category Dialog) so the result is verifiable without screenshots.

visualizer

  • Setup: Ensure Peekaboo.app is running (visual feedback host) and keep Playground visible so you can quickly spot overlays.
  • Steps:
    1. polter peekaboo -- visualizer --json-output > .artifacts/playground-tools/<timestamp>-visualizer.json
    2. Visually confirm you see (in order): screenshot flash, capture HUD, click ripple, typing overlay, scroll indicator, mouse trail, swipe path, hotkey HUD, window move overlay, app launch/quit animation, menu breadcrumb, dialog highlight, space switch indicator, and element detection overlay.
  • Pass criteria: No CLI errors, the JSON report shows every step dispatched=true, and the full overlay sequence renders end-to-end.
  • 2025-12-18 run:
    • JSON reports all 15 steps dispatched=true (manual “eyes on overlay” still required for full pass criteria).
    • Evidence: .artifacts/playground-tools/20251218-022612-visualizer.json.

Automation & Integrations

agent

  • Scope: Playground-specific instructions to exercise multiple tools automatically.
  • Tests:
    1. polter peekaboo -- agent --model gpt-5.1 --list-sessions --json-output > .artifacts/playground-tools/20251117-010912-agent-list.json.
    2. polter peekaboo -- agent "Say hi to the Playground app." --model gpt-5.1 --max-steps 2 --json-output > .artifacts/playground-tools/20251117-010919-agent-hi.json.
    3. polter peekaboo -- agent "Switch to Playground and press the Single Click button once." --model gpt-5.1 --max-steps 4 --json-output > .artifacts/playground-tools/20251117-010935-agent-single-click.json.
    4. For long interactive runs, use tmux: tmux new-session -- bash -lc 'pnpm run peekaboo -- agent "Click the Single Click button in Playground." --model gpt-5.1 --max-steps 6 --no-cache | tee .artifacts/playground-tools/20251117-011500-agent-single-click.log'.
    5. Spot-check metadata: polter --force peekaboo -- agent "Say hi to Playground again." --model gpt-5.1 --max-steps 2 --json-output > .artifacts/playground-tools/20251117-012655-agent-hi.json.
  • 2025-11-17 run:
    • GPT-5.1 executes happily; Playground [Agent] log is captured in .artifacts/playground-tools/20251117-011345-agent.log.
    • Non-tmux invocations can time out; move anything beyond quick dry-runs into tmux ... so long runs complete.
    • Manual verification: observed the agent perform see + click against the Playground “Single Click” button (tmux transcript stored in .artifacts/playground-tools/20251117-011500-agent-single-click.log).
    • JSON mode now reports the correct toolCallCount (see .artifacts/playground-tools/20251117-012655-agent-hi.json which shows toolCallCount: 1 for the done tool).

mcp

  • Steps:
    1. MCPORTER list peekaboo-local --stdio "$PEEKABOO_BIN mcp" --timeout 20 --schema > .artifacts/playground-tools/20251219-001230-mcp-list.json.
    2. MCPORTER call peekaboo-local.permissions --stdio "$PEEKABOO_BIN mcp" --timeout 15 > .artifacts/playground-tools/20251219-001245-mcp-call-permissions.json.
    3. Capture the OSLog stream with ./Apps/Playground/scripts/playground-log.sh -c MCP --last 15m --all -o .artifacts/playground-tools/20251219-001255-mcp.log.
  • 2025-12-19 verification:
    • MCPORTER list returns the native Peekaboo tool catalog via stdio.
    • permissions call returns the expected Screen Recording + Accessibility statuses.
    • Playground [MCP] log records the server requests for later regression diffs.

Reporting & Follow-Up

  • Record every executed test case (command, arguments, snapshot ID, log file path, outcome) in Apps/Playground/PLAYGROUND_TEST.md.
  • When a bug is fixed, update this doc’s table row to Verified and link to the log artifact plus commit hash.
  • If a tool is blocked (e.g., Swift compiler crash), set status to Blocked, explain the reason inline, and add a TODO referencing the GitHub issue/Swift crash log.
  • Keep this plan synchronized with any changes under docs/commands/—when new tools land, add rows + recipes immediately so coverage never regresses.