feat: audit log improvements — CSV/JSON export and real-time WebSocket stream#215
feat: audit log improvements — CSV/JSON export and real-time WebSocket stream#215BillChirico merged 3 commits intomainfrom
Conversation
…tream - Add GET /:id/audit-log/export endpoint (CSV and JSON, up to 10k rows) - Add /ws/audit-log WebSocket server for real-time audit entry broadcast - Refactor buildFilters() shared helper to eliminate duplication - Hook broadcastAuditEntry() into insertAuditEntry (RETURNING id+created_at) - Wire setupAuditStream/stopAuditStream into startServer/stopServer lifecycle - Add escapeCsvValue/rowsToCsv helpers with full test coverage - 30 route tests + 17 WebSocket stream tests, all green Closes #136
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📒 Files selected for processing (6)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Pull request overview
Adds audit log export (CSV/JSON) and a real-time WebSocket audit stream to improve dashboard audit log consumption.
Changes:
- Added
GET /guilds/:id/audit-log/exportwith CSV/JSON output, larger limits, and export-specific rate limiting. - Introduced
/ws/audit-logWebSocket stream with HMAC ticket auth, optional filtering, heartbeat, and client cap. - Updated audit log insertion to
RETURNINGthe inserted row and broadcast it to connected WebSocket clients.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/api/middleware/auditLog.js | Broadcasts newly inserted audit entries to the audit WebSocket stream using RETURNING row data. |
| src/api/routes/auditLog.js | Adds export endpoint, CSV helpers, and refactors shared filter-building logic. |
| src/api/server.js | Wires audit stream lifecycle into server start/stop. |
| src/api/ws/auditStream.js | Implements WebSocket audit log streaming server (auth, filter, heartbeat, broadcast). |
| tests/api/routes/auditLog.test.js | Adds unit tests for CSV helpers and integration tests for the export endpoint. |
| tests/api/ws/auditStream.test.js | Adds coverage for audit stream connection/auth/filter/broadcast behaviors. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
| Filename | Overview |
|---|---|
| src/api/middleware/auditLog.js | Modified to broadcast audit entries to WebSocket clients via RETURNING clause integration |
| src/api/routes/auditLog.js | Added export endpoint with CSV/JSON formats and refactored filter logic into reusable buildFilters helper |
| src/api/server.js | Integrated audit stream lifecycle (setup/teardown) into server startup and shutdown |
| src/api/ws/auditStream.js | New WebSocket server for real-time audit log streaming with HMAC auth, filtering, and heartbeat |
Sequence Diagram
sequenceDiagram
participant Client as Dashboard Client
participant HTTP as Express Server
participant WS as WebSocket Server
participant Middleware as Audit Middleware
participant DB as PostgreSQL
Note over Client,DB: Real-time Audit Stream Flow
Client->>WS: Connect to /ws/audit-log
WS->>Client: Connection established
Client->>WS: {type: 'auth', ticket: 'hmac'}
WS->>WS: Validate HMAC ticket
WS->>Client: {type: 'auth_ok'}
Client->>WS: {type: 'filter', guildId, action}
WS->>Client: {type: 'filter_ok'}
Note over Client,DB: Mutation triggers broadcast
Client->>HTTP: PUT /guilds/:id/config
HTTP->>Middleware: Audit middleware intercepts
Middleware->>DB: INSERT ... RETURNING *
DB->>Middleware: Return new entry with id
Middleware->>WS: broadcastAuditEntry(entry)
WS->>WS: Match against client filters
WS->>Client: {type: 'entry', entry: {...}}
Note over Client,DB: Export Flow
Client->>HTTP: GET /guilds/:id/audit-log/export?format=csv
HTTP->>HTTP: Validate auth & guild admin
HTTP->>DB: SELECT * WHERE ... LIMIT 10000
DB->>HTTP: Return matching entries
HTTP->>Client: CSV file download
Last reviewed commit: 7bbcade
- ws.ping() crash: guard with readyState check + try/catch to avoid crashing heartbeat interval when socket not OPEN - stopAuditStream race: make setupAuditStream async and await stopAuditStream() to prevent concurrent WebSocketServer creation - Query param array coercion: add typeof === 'string' checks for startDate/endDate to handle Express string|string[]|undefined - CSV CRLF quoting: add \r to RFC 4180 special-char check for proper Windows line ending handling - Test timeouts: make AUTH_TIMEOUT_MS configurable via AUDIT_STREAM_AUTH_TIMEOUT_MS env var, use 100ms in tests
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…t stream (#215) * feat: audit log improvements — CSV/JSON export, real-time WebSocket stream - Add GET /:id/audit-log/export endpoint (CSV and JSON, up to 10k rows) - Add /ws/audit-log WebSocket server for real-time audit entry broadcast - Refactor buildFilters() shared helper to eliminate duplication - Hook broadcastAuditEntry() into insertAuditEntry (RETURNING id+created_at) - Wire setupAuditStream/stopAuditStream into startServer/stopServer lifecycle - Add escapeCsvValue/rowsToCsv helpers with full test coverage - 30 route tests + 17 WebSocket stream tests, all green Closes #136 * fix: PR #215 review feedback - audit stream fixes - ws.ping() crash: guard with readyState check + try/catch to avoid crashing heartbeat interval when socket not OPEN - stopAuditStream race: make setupAuditStream async and await stopAuditStream() to prevent concurrent WebSocketServer creation - Query param array coercion: add typeof === 'string' checks for startDate/endDate to handle Express string|string[]|undefined - CSV CRLF quoting: add \r to RFC 4180 special-char check for proper Windows line ending handling - Test timeouts: make AUTH_TIMEOUT_MS configurable via AUDIT_STREAM_AUTH_TIMEOUT_MS env var, use 100ms in tests
* security: escape user content in triage prompt delimiters (#164) Add escapePromptDelimiters() to HTML-encode < and > in user-supplied message content before it is inserted between XML-style section tags in the LLM prompt. Without escaping, a crafted message containing the literal text `</messages-to-evaluate>` could break out of the user-content section and inject attacker-controlled instructions into the prompt structure. Changes: - Add escapePromptDelimiters(text) utility exported from triage-prompt.js - Apply escape to m.content and m.replyTo.content in buildConversationText() - Add 13 new tests covering the escape function and injection scenarios Closes #164 * security: escape & chars and author fields in prompt delimiters * fix(security): escape & in prompt delimiters and escape author fields - Add & → & escape first in escapePromptDelimiters() to prevent HTML entity bypass attacks (e.g. </messages-to-evaluate>) - Also escape m.author and m.replyTo.author since Discord display names are user-controlled and can contain < / > characters Addresses review feedback on PR #204. * fix: guard replyTo.content before .slice() to handle null/undefined * perf: SQL-based conversation pagination + missing DB indexes (#221) Fixes three performance bottlenecks identified in code review of recently merged features (PR #121 conversations viewer, PR #190 AI feedback). ## Changes ### migrations/004_performance_indexes.cjs (new) Four new indexes targeting hot query paths: - idx_ai_feedback_guild_created (guild_id, created_at DESC) getFeedbackTrend() and getRecentFeedback() filtered by guild_id AND created_at but only had a single-column guild_id index, forcing a full guild scan + sort on every trend/recent call. - idx_conversations_content_trgm (GIN, pg_trgm) content ILIKE '%...%' search was a sequential scan. GIN/trgm index reduces this from O(n) to O(log n * trigram matches). Requires pg_trgm extension (added idempotently). - idx_conversations_guild_created (guild_id, created_at DESC) Default 30-day listing query filters guild_id + created_at. The existing 3-column (guild_id, channel_id, created_at) composite is suboptimal when channel_id is not in the predicate. - idx_flagged_messages_guild_message (guild_id, message_id) Conversation detail + flag endpoints query flagged_messages by guild_id AND message_id = ANY(...). Existing index only covers (guild_id, status). ### src/api/routes/conversations.js **GET / — Replace in-memory pagination with SQL CTE grouping** Before: fetched up to 10,000 message rows into Node memory, grouped them in JavaScript (O(n) time + memory), then sliced for pagination. Every page request loaded the full 10k row dataset. After: single SQL query using window functions (LAG + SUM OVER) to identify conversation boundaries and aggregate summaries directly. COUNT(*) OVER() provides total count without a second query. Pagination happens at the DB with LIMIT/OFFSET on summary rows. Memory overhead is now proportional to page size (default 25), not total conversation volume. Removed now-unused buildConversationSummary() helper (logic inlined into the SQL-side aggregation). **POST /:conversationId/flag — Parallel verification queries** Before: msgCheck and anchorCheck ran sequentially (~2× RTT). After: both run in parallel via Promise.all (1× RTT for verification). ### tests/api/routes/conversations.test.js Updated 'should return paginated conversations' test to mock the new SQL CTE response shape (pre-aggregated summary rows) instead of raw message rows. All 41 conversation tests pass. * feat: channel-level quiet mode via bot mention (#173) (#213) * feat: quiet mode per-channel via bot mention (#173) - Add quietMode.js module with Redis+memory storage - Parse duration from natural language (30m, 1 hour, etc.) - Permission gated via config.quietMode.allowedRoles - Commands: quiet, unquiet, status - Suppress AI responses during quiet mode in events.js - Add quietMode section to config.json (disabled by default) - Add quietMode to configAllowlist.js for dashboard editing * test: add quiet mode tests (41 tests, all passing) * style: fix biome formatting in quietMode.js, events.js, and test * fix(web): fix ai-feedback-stats TypeScript and formatting errors * fix: gate quiet mode checks on enabled flag, validate TTL, honor maxDurationMinutes config - events.js: Wrap isQuietMode() calls in guildConfig.quietMode?.enabled check to avoid unnecessary Redis lookups and prevent stale records from suppressing AI responses when the feature is disabled (PRRT_kwDORICdSM5xdbmp, PRRT_kwDORICdSM5xdbmx) - quietMode.js: Add TTL validation in setQuiet() to guard against 0, negative, or NaN values that would error in Redis (PRRT_kwDORICdSM5xdbm3) - quietMode.js: Update parseDurationFromContent() to accept config parameter and honor guildConfig.quietMode.maxDurationMinutes. Also clamp defaultSeconds to the effective max (PRRT_kwDORICdSM5xdbm_) - configValidation.js: Add quietMode schema entry with enabled, maxDurationMinutes, and allowedRoles properties (PRRT_kwDORICdSM5xdbnH) * style: fix biome formatting in quietMode.js and ai-feedback-stats.tsx * feat: audit log improvements — CSV/JSON export and real-time WebSocket stream (#215) * feat: audit log improvements — CSV/JSON export, real-time WebSocket stream - Add GET /:id/audit-log/export endpoint (CSV and JSON, up to 10k rows) - Add /ws/audit-log WebSocket server for real-time audit entry broadcast - Refactor buildFilters() shared helper to eliminate duplication - Hook broadcastAuditEntry() into insertAuditEntry (RETURNING id+created_at) - Wire setupAuditStream/stopAuditStream into startServer/stopServer lifecycle - Add escapeCsvValue/rowsToCsv helpers with full test coverage - 30 route tests + 17 WebSocket stream tests, all green Closes #136 * fix: PR #215 review feedback - audit stream fixes - ws.ping() crash: guard with readyState check + try/catch to avoid crashing heartbeat interval when socket not OPEN - stopAuditStream race: make setupAuditStream async and await stopAuditStream() to prevent concurrent WebSocketServer creation - Query param array coercion: add typeof === 'string' checks for startDate/endDate to handle Express string|string[]|undefined - CSV CRLF quoting: add \r to RFC 4180 special-char check for proper Windows line ending handling - Test timeouts: make AUTH_TIMEOUT_MS configurable via AUDIT_STREAM_AUTH_TIMEOUT_MS env var, use 100ms in tests * feat: voice channel activity tracking — join/leave/move, leaderboard, export (#212) * feat: add voice_sessions migration (#135) * feat: add voice tracking module — join/leave/move/flush/leaderboard (#135) * feat: wire voiceStateUpdate handler into event registration (#135) * feat: add /voice command — leaderboard, stats, export subcommands (#135) * feat: add voice config defaults to config.json (#135) * feat: wire voice flush start/stop into bot lifecycle (#135) * feat: add voice to config API allowlist (#135) * fix: SQL UPDATE subquery for closeSession, fix import order (#135) * fix(voice): resolve race conditions and missing config schema - Fix openSession: update in-memory state only AFTER DB INSERT succeeds - Fix closeSession: delete from in-memory state only AFTER DB UPDATE succeeds - Fix: allow closeSession on leave/move even when feature is disabled - Fix migration: add UNIQUE constraint to partial index to prevent duplicates - Fix: move 'Voice join' log to after openSession succeeds - Add voice config to CONFIG_SCHEMA for validation --------- Co-authored-by: Bill <bill@example.com> * feat(dashboard): auto-save config with 500ms debounce (#199) * feat(dashboard): replace manual save with auto-save (500ms debounce) - Remove 'Save Changes' button; saving now fires automatically 500ms after the last config change (no changes → no network call) - Add saveStatus state ('idle' | 'saving' | 'saved' | 'error') with AutoSaveStatus component showing spinner, check, or error+retry - Add isLoadingConfigRef guard so the initial fetchConfig load never triggers a spurious PATCH - Ctrl+S still works: clears debounce timer and saves immediately - Keep 'beforeunload' warning for validation errors that block save - Replace yellow unsaved-changes banner with a destructive validation error banner (only shown when save is actually blocked) - Error state shows 'Save failed' + 'Retry' button for user recovery Closes #189 * test(dashboard): add auto-save tests for ConfigEditor - No PATCH on initial config load - Validation error banner suppresses auto-save - 'Saving...' spinner visible while PATCH in-flight - 'Save failed' + Retry button on PATCH error * fix(dashboard): prevent fetchConfig from overwriting saveStatus after successful save Add skipSaveStatusReset parameter to fetchConfig so that post-save reloads preserve the 'saved' status indicator instead of immediately resetting to 'idle'. * test(dashboard): use fake timers, restore vi.stubGlobal, fix assertions, add idle/saved coverage - Replace real setTimeout delays with vi.useFakeTimers() + vi.advanceTimersByTimeAsync() for deterministic, fast debounce tests - Add afterEach cleanup: vi.unstubAllGlobals() + vi.useRealTimers() - Replace toBeTruthy() with toBeInTheDocument() for Testing Library queries - Add idle state test (no status indicator shown after load) - Add saved state test (shows 'Saved' after successful save) - Update file-level comment to list all four states --------- Co-authored-by: Bill Chirico <bill@volvox.gg> * feat: Reaction role menus (#162) (#205) * feat: reaction role menus - core module, command, event hooks, migration Implements issue #162: reaction role menus. - Add migration 004 creating reaction_role_menus and reaction_role_entries tables - Add src/modules/reactionRoles.js with DB helpers, embed builder, event handlers - Add src/commands/reactionrole.js with /reactionrole create|add|remove|delete|list - Wire handleReactionRoleAdd/Remove into registerReactionHandlers in events.js Roles are granted on reaction add and revoked on reaction remove. All mappings persist in PostgreSQL across bot restarts. * test: reaction role menus - 40 tests covering module and command - tests/modules/reactionRoles.test.js: resolveEmojiString, buildReactionRoleEmbed, all DB helpers, handleReactionRoleAdd, handleReactionRoleRemove - tests/commands/reactionrole.test.js: all 5 subcommands (create, add, remove, delete, list) including error paths and guild ownership checks - Fix biome lint: import sort order + unused import removal * fix: remove unused import in reactionrole command --------- Co-authored-by: Bill Chirico <bill@volvox.gg> * fix(security): validate GitHub owner/repo format before gh CLI call (#198) * fix(security): validate GitHub owner/repo format before gh CLI call Prevents API path traversal by validating owner/repo segments against a strict allowlist regex before interpolating them into the gh CLI invocation. Adds: - VALID_GH_NAME regex (/^[a-zA-Z0-9._-]+$/) - isValidGhRepo() helper (exported for testing) - Guard in fetchRepoEvents() — returns [] and warns on invalid input - Strengthened guard in pollGuildFeed() split logic Fixes #160 * test(security): add validation tests for GitHub owner/repo format Covers isValidGhRepo(), VALID_GH_NAME regex, and fetchRepoEvents() validation guard introduced in fix for #160. 19 new tests verify: - Valid alphanumeric/dot/hyphen/underscore names pass - Path traversal (../../etc/passwd) is rejected at both entry points - Slashes, empty strings, non-strings, spaces all rejected - Shell metacharacters (; && $()) blocked - gh CLI is NOT invoked when validation fails - warn() fires with the invalid values (observable audit trail) - Valid owner/repo still reach gh CLI unchanged * fix(security): reject pure-dot owner/repo names to prevent path traversal * test(githubFeed): add tests for pure-dot path traversal bypass --------- Co-authored-by: Bill Chirico <bill@volvox.gg> --------- Co-authored-by: Bill <bill@example.com> Co-authored-by: Bill Chirico <bill@volvox.gg>
Summary
Enhances the audit log system with two missing features from the issue:
✅ Export to CSV/JSON
GET /guilds/:id/audit-log/exportformat=csvorformat=json(default)escapeCsvValue)Content-Disposition: attachment✅ Real-time WebSocket updates
/ws/audit-log(same HMAC ticket auth as/ws/logs)entryevents as mutations are committed{ type: 'filter', guildId, action, userId }insertAuditEntryviaRETURNINGclause (no extra query)startServer/stopServerAlready present (no changes needed)
Tests
Diff stat
Closes #136