feat(core): add robustness tests, logging, and metrics for CodeAssistServer SSE parsing

[yunaseoul]

Summary

This PR introduces manual fuzzing and robustness tests to the CodeAssistServer to ensure resilient handling of SSE parsing and error responses. It also adds logging and metrics for invalid SSE chunks.

Details

• Added manual fuzzing and robustness tests in server.test.ts to cover:
  • Randomly fragmented SSE streams.
  • Multi-line JSON parsing within SSE data lines.
  • Filtering of arbitrary SSE comments and ID lines.
  • VPC SC error object structure validation.
  • Credit consumption and remaining credit tracking in streaming responses.
• Implemented InvalidChunkEvent logging on SSE parse failure in CodeAssistServer.
• Integrated metrics into logging: every time an invalid chunk is logged, the corresponding metric is also incremented.

Related Issues

Fixes #20191
Partially fixes #20189

How to Validate

Run the newly added tests:

npm test -w @google/gemini-cli-core -- src/code_assist/server.test.ts
npm test -w @google/gemini-cli-core -- src/telemetry/loggers.test.ts
npm test -w @google/gemini-cli-core -- src/telemetry/metrics.test.ts

Pre-Merge Checklist

• Updated relevant documentation and README (if needed)
• Added/updated tests (if needed)
• Noted breaking changes (if any)
• Validated on required platforms/methods:
  • MacOS
    • npm run

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the reliability and stability of the CodeAssistServer by integrating property-based fuzz testing. The new tests target critical areas such as Server-Sent Events (SSE) parsing, error handling for VPC SC responses, and credit tracking in streaming data, ensuring the server can withstand unexpected or malformed inputs without crashing or misbehaving. This proactive approach to testing helps prevent future regressions and improves the overall resilience of the core service.

Highlights

• Fuzz Testing Introduction: Introduced property-based fuzz testing to the CodeAssistServer using fast-check and @fast-check/vitest to enhance robustness.
• SSE Parsing Robustness: Added tests to ensure the server gracefully handles randomly fragmented Server-Sent Events (SSE) streams and correctly parses multi-line JSON data within SSE events.
• Error Handling Validation: Implemented fuzz tests to validate the server's resilience against malformed or partial VPC SC error objects, preventing crashes during error processing.
• SSE Noise Filtering: Included tests to confirm the server correctly ignores arbitrary SSE comments, ID lines, and empty lines, focusing only on valid data.
• Credit Tracking Stability: Added fuzz tests to ensure safe processing of streaming responses, specifically validating the handling of consumed and remaining credits data.

Changelog

• package-lock.json
  • Added fast-check as a development dependency.
  • Added @fast-check/vitest as a development dependency.
• packages/core/package.json
  • Added fast-check to devDependencies.
  • Added @fast-check/vitest to devDependencies.
• packages/core/src/code_assist/server.test.ts
  • Imported test from @fast-check/vitest and fc from fast-check.
  • Added a new describe block for 'fuzz testing'.
  • Implemented a fuzz test for loadCodeAssist to handle random error objects without crashing.
  • Added a fuzz test to verify graceful handling of randomly fragmented SSE streams.
  • Included a fuzz test to ensure correct parsing of valid JSON split across multiple SSE data lines.
  • Created a fuzz test to prevent crashes from partially matching VPC SC error structures.
  • Added a fuzz test to confirm arbitrary SSE comments, ID lines, and empty lines are ignored.
  • Implemented a fuzz test for generateContentStream to safely process random response streams, including credit information.

Activity

• No human activity has occurred on this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[github-actions]

Size Change: +1.74 kB (+0.01%)

Total Size: 26 MB

Filename	Size	Change
./bundle/gemini.js	25.5 MB	+1.74 kB (+0.01%)

ℹ️ View Unchanged

Filename	Size
./bundle/node_modules/@google/gemini-cli-devtools/dist/client/main.js	221 kB
./bundle/node_modules/@google/gemini-cli-devtools/dist/src/_client-assets.js	227 kB
./bundle/node_modules/@google/gemini-cli-devtools/dist/src/index.js	11.5 kB
./bundle/node_modules/@google/gemini-cli-devtools/dist/src/types.js	132 B
./bundle/sandbox-macos-permissive-open.sb	890 B
./bundle/sandbox-macos-permissive-proxied.sb	1.31 kB
./bundle/sandbox-macos-restrictive-open.sb	3.36 kB
./bundle/sandbox-macos-restrictive-proxied.sb	3.56 kB
./bundle/sandbox-macos-strict-open.sb	4.82 kB
./bundle/sandbox-macos-strict-proxied.sb	5.02 kB

_{compressed-size-action}

[gemini-code-assist]

Code Review

This pull request introduces valuable fuzz tests for the CodeAssistServer using fast-check, significantly improving the robustness of SSE parsing and error handling. The tests are well-structured and cover a good range of edge cases. I have one suggestion to make one of the new tests more resilient to different types of parsing errors.

[gemini-code-assist]

Code Review

This pull request enhances the robustness of the CodeAssistServer's Server-Sent Events (SSE) parsing by adding error handling for invalid JSON chunks, introducing new logging and metrics, and adding robustness tests. A critical security vulnerability was identified where logging malformed SSE chunks can lead to PII or secret leakage, as JSON.parse error messages might contain sensitive input data and are sent to telemetry. Additionally, a high-severity issue was found where a failure in the telemetry pipeline could still crash the stream, undermining the goal of increased robustness.