feat: Script Output Summarization

[mgajda]

This PR implements the Script Output Summarization strategy. It allows the CLI to summarize lengthy script outputs using the flash-lite model while preserving key excerpts. Short outputs are skipped. The strategy is integrated into the model router. Related to issue #2.

[gemini-code-assist]

Summary of Changes

Hello @mgajda, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new feature to summarize lengthy script outputs before they are processed by the main thinking model. The primary goal is to enhance efficiency and clarity by utilizing the flash-lite model for cost-effective and fast summarization, while ensuring that critical information and excerpts from the original output are retained. The changes involve integrating this new summarization logic into the existing model routing system and explicitly tagging shell tool outputs for proper identification.

Highlights

• Script Output Summarization: Implemented a new strategy to summarize lengthy script outputs using the flash-lite model, ensuring key excerpts are preserved.
• Conditional Summarization: Short script outputs (less than 500 characters) are now automatically skipped from summarization to optimize performance and avoid unnecessary processing.
• Model Router Integration: A new ScriptOutputSummarizationStrategy has been integrated into the ModelRouterService's CompositeStrategy, allowing the system to intelligently route shell command outputs for summarization.
• Shell Tool Output Tagging: Shell command results are now explicitly tagged with a [SHELL_OUTPUT] prefix and a toolSpecificInfo.isShellOutput: true flag, enabling the new summarization strategy to accurately identify and process them.
• Refactored Summarization Logic: The previous, more generic summarization logic within packages/core/src/tools/shell.ts has been removed, centralizing summarization responsibility within the new dedicated routing strategy.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new ScriptOutputSummarizationStrategy to handle long script outputs by summarizing them. The core idea is solid, but the current implementation has a critical flaw where the generated summary is not used, and the strategy ordering prevents it from running effectively. I've provided suggestions to fix the implementation, correct the strategy order, and improve the summarization prompt. Addressing these points will make the feature functional as intended.

[gemini-code-assist]

The summarization logic is incomplete and contains a potential type error.

1. Summary Not Used: A summary is generated but the summarizedOutput is never used. The strategy returns a RoutingDecision to use flash-lite, but the original, unsummarized content would be passed to the model. This defeats the purpose of the summarization. The strategy should modify shellOutputPart.text with the summary and return null to allow subsequent strategies to process the summarized content.
2. Incorrect contents Type: The contents property of generateContent expects a Content[] (i.e., Array<{role: string, parts: Part[]}>), but it's being passed a string[]. This could cause a runtime error.

The suggestion below fixes both issues.

      const summaryResult = await baseLlmClient.generateContent({
        modelConfigKey: { model: 'flash-lite' }, // Using flash-lite for summarization
        contents: [{ role: 'user', parts: [{ text: summaryPrompt }] }],
        abortSignal: context.signal,
      });

      const summarizedOutput = summaryResult.text;

      // Replace the long output with the summary in the context for subsequent strategies.
      shellOutputPart.text = summarizedOutput;

      // Pass to the next strategy to make a final model decision on the summarized content.
      return null;

[gemini-code-assist]

The ScriptOutputSummarizationStrategy is placed after the ClassifierStrategy. For the summarization to be effective, it needs to run before the classifier. The current order means that if the ClassifierStrategy makes a decision (e.g., routing to 'pro' model due to the long script output), the ScriptOutputSummarizationStrategy will never be executed. The summarization strategy should run first to shorten the input, and then the classifier can run on the summarized, shorter content.

Suggested change

	new ClassifierStrategy(),
	new ScriptOutputSummarizationStrategy(), // Added new strategy here
	new ScriptOutputSummarizationStrategy(),
	new ClassifierStrategy(),

[gemini-code-assist]

The summaryPrompt contains an unclosed markdown code block. The prompt ends with \``\n` which starts a code block but doesn't close it. This can confuse the model and lead to poor quality summaries. The content to be summarized should be wrapped in a closed code block for clarity.

Suggested change

	const summaryPrompt = `Summarize the following script output concisely. Preserve key phrases and sentences that are critical to understanding the output verbatim. Present these excerpts clearly, perhaps by quoting them or using a specific marker like [EXCERPT]...[/EXCERPT]. Avoid paraphrasing or distorting the meaning of these excerpts.
	Text to summarize:
	\n\n${actualOutput}\n\n\n\
	```\n`;
	const summaryPrompt = `Summarize the following script output concisely. Preserve key phrases and sentences that are critical to understanding the output verbatim. Present these excerpts clearly, perhaps by quoting them or using a specific marker like [EXCERPT]...[/EXCERPT]. Avoid paraphrasing or distorting the meaning of these excerpts.
	Text to summarize:
	\`\`\`
	${actualOutput}
	\`\`\``;

[jackwotherspoon]

Hi there @mgajda

While we appreciate these pull requests we need a proper feature request to be discussed prior to implementation.

Please open a feature request to be discussed with the team here: https://github.com/google-gemini/gemini-cli/issues and not as a file in the PR.

Once a design decision has been agreed upon then is the proper time to open a PR.

Hope this makes sense 👍

Maybe I did not make myself clear earlier, my apologies for that.