Skip to content

feat(pattern): support multi-word alternation syntax in command position#94

Merged
fohte merged 4 commits intomainfrom
fohte/multi-word-alternation
Feb 24, 2026
Merged

feat(pattern): support multi-word alternation syntax in command position#94
fohte merged 4 commits intomainfrom
fohte/multi-word-alternation

Conversation

@fohte
Copy link
Owner

@fohte fohte commented Feb 24, 2026

Why

  • Patterns like "npx prettier"|"bunx prettier"|prettier * should allow multi-word alternatives containing spaces
    • Covering npx prettier, bunx prettier, and prettier under a single rule currently requires three separate rules

What

  • Expand quoted multi-word alternation into multiple independent Pattern instances at parse time, enabling matching without changes to the existing matcher

Open with Devin

Single-word alternation (`ast-grep|sg scan *`) was supported but
users needed separate rules for commands with different prefixes
(e.g., `npx prettier *` and `prettier *`). Multi-word alternation
allows combining these into a single rule using quoted alternatives:

  `"npx prettier"|"bunx prettier"|prettier *`

Implementation approach: multi-word alternation patterns are expanded
into multiple independent Pattern instances at parse time, so the
matcher requires no changes. The lexer detects quoted-string `|`
continuations and emits a new `MultiWordAlternation` token, which
the parser's `parse_multi()` expands into separate patterns. The
rule engine iterates over expanded patterns for each rule.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@gemini-code-assist
Copy link

Summary of Changes

Hello @fohte, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the flexibility of pattern matching by introducing support for multi-word alternations in the command position. This allows users to define a single, more powerful rule that can match several variations of a command, including those with multiple words or different prefixes, without altering the core matching logic. The change simplifies rule definition and makes the pattern language more robust for complex command structures.

Highlights

  • Multi-word Alternation Syntax: Introduced support for multi-word alternation in the command position, allowing patterns like "npx prettier"|prettier * to be expressed as a single rule. This enhances pattern expressiveness and reduces the need for multiple rules to cover variations of a command.
  • Lexer Enhancements: The pattern lexer was updated to recognize and tokenize multi-word alternations, distinguishing them from single-word alternations and literals. This includes handling quoted multi-word segments and combining them with bare-word alternatives.
  • Parser Expansion: A new parse_multi function was added to the pattern parser. This function expands multi-word alternation tokens into multiple distinct Pattern instances at parse time, allowing the existing matcher to process them without modification.
  • Rule Engine Integration: The rule engine was modified to utilize the new parse_multi function, enabling it to correctly evaluate rules containing multi-word alternations by iterating through the expanded patterns until a match is found.
  • Comprehensive Testing: Extensive unit and integration tests were added to cover various scenarios of multi-word alternation, including mixed quoted and bare-word alternatives, error cases, and backward compatibility with existing single-word alternations.
Changelog
  • src/rules/pattern_lexer.rs
    • Added MultiWordAlternation variant to LexToken enum.
    • Modified tokenize function to detect and parse multi-word alternations, especially those involving quoted strings.
    • Introduced consume_alternation_continuation to handle subsequent alternatives in a multi-word sequence.
    • Added classify_multi_word_alternation to determine if a set of alternatives should be MultiWordAlternation or Alternation.
    • Updated is_word_boundary to include quotes as word boundaries.
    • Added new unit tests for multi-word alternation parsing and error handling.
  • src/rules/pattern_matcher.rs
    • Added new test cases to verify correct matching behavior for multi-word alternations.
    • Included tests for backward compatibility with existing single-word alternations.
  • src/rules/pattern_parser.rs
    • Introduced parse_multi function to handle patterns that may contain multi-word alternations, returning a vector of Pattern instances.
    • Modified build_pattern_tokens and lex_to_pattern_value to explicitly disallow MultiWordAlternation tokens outside of the command position.
    • Added unit tests for the parse_multi function, demonstrating its expansion capabilities.
  • src/rules/rule_engine.rs
    • Updated evaluate_command_inner to use parse_multi instead of parse for pattern parsing, allowing it to process expanded patterns.
    • Modified try_unwrap_wrapper to also use parse_multi when evaluating wrapper patterns, ensuring consistent behavior.
  • tests/integration/config_to_rule_evaluation.rs
    • Added integration tests to confirm that multi-word alternation rules defined in YAML configurations are correctly evaluated.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@codecov
Copy link

codecov bot commented Feb 24, 2026

Codecov Report

❌ Patch coverage is 93.25843% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.07%. Comparing base (6fabb81) to head (ae5a01b).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
src/rules/pattern_lexer.rs 94.00% 6 Missing ⚠️
src/rules/pattern_parser.rs 81.81% 6 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #94      +/-   ##
==========================================
+ Coverage   89.00%   89.07%   +0.06%     
==========================================
  Files          29       29              
  Lines        5821     5966     +145     
==========================================
+ Hits         5181     5314     +133     
- Misses        640      652      +12     
Flag Coverage Δ
Linux 88.90% <93.25%> (+0.07%) ⬆️
macOS 91.08% <93.25%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

gemini-code-assist[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

…re alternatives

`consume_word` did not treat `|` as a word boundary, so inside
`consume_alternation_continuation` a bare-word branch like
`"npx prettier"|foo|bar` would merge `foo|bar` into a single word
instead of yielding two separate alternatives.

Add an `extra_stop` parameter to `consume_word` and pass `Some('|')`
from the alternation continuation context.

Also extract shared tokenize/build logic between `parse` and
`parse_multi` into `tokenize_pattern` / `build_pattern_from_tokens`
helpers, and use iterator methods for alternation construction.
devin-ai-integration[bot]

This comment was marked as resolved.

fohte and others added 2 commits February 25, 2026 01:19
…e-word pipe

The error message for an unclosed quote after a bare-word pipe
reported the position of the bare word start instead of the opening
quote character. Capture the quote position from `chars.peek()` and
use it in the error message.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Verify that when a bare-word pipe is followed by an unclosed quote
(e.g., `prettier|"npx prettier`), the error message reports the
position of the opening quote character, not the bare word start.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@fohte fohte merged commit 2fdea25 into main Feb 24, 2026
6 checks passed
@fohte fohte deleted the fohte/multi-word-alternation branch February 24, 2026 16:40
@fohte-bot fohte-bot bot mentioned this pull request Feb 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant