Skip to content

Conversation

@gnodet
Copy link
Contributor

@gnodet gnodet commented Oct 30, 2025

Description

Fixes #11363

This PR fixes the issue where Maven 4 fails to parse pipe symbols (|) and other special characters in .mvn/jvm.config files, causing shell/batch script parsing errors.

Problem

When users tried to use JVM arguments with pipe symbols in .mvn/jvm.config, such as:

-Dhttp.nonProxyHosts=de|*.de|my.company.mirror.de
-Dprop.with.pipes="value|with|pipes"

Maven 4 would fail with errors:

On Unix/Linux/macOS:

*.de: command not found
my.company.mirror.de: command not found

On Windows:

'*.de' is not recognized as an internal or external command

Root Cause

The issue had different root causes on different platforms:

Unix/Linux/macOS

The concat_lines function used xargs -n 1 which split arguments on whitespace, breaking quoted arguments. Additionally, pipe symbols were being interpreted as shell command separators during eval exec execution.

Windows

Windows batch script variable expansion with delayed expansion (!var!) causes special characters like pipes to be interpreted as command separators. The complex batch script logic couldn't reliably handle pipes, quotes, and spaces simultaneously.

Solution

We implemented different solutions for Unix and Windows due to their different capabilities and performance characteristics:

Windows: Java-based Parser (Required)

Created JvmConfigParser.java that:

  • Parses .mvn/jvm.config line by line
  • Handles comments (both # at start and end-of-line)
  • Performs ${MAVEN_PROJECTBASEDIR} and $MAVEN_PROJECTBASEDIR substitution
  • Splits each line into individual arguments, respecting quoted strings
  • Strips quotes from arguments (they're for grouping only)
  • Outputs space-separated quoted arguments safe for batch scripts

The Windows batch script (mvn.cmd):

  • Compiles the Java parser to a random temp directory (unique per invocation)
  • Runs the compiled parser to process jvm.config
  • Reads the output into a variable using set /p (no loops, no delayed expansion)
  • Cleans up temp files and directory after use
  • Avoids all batch script parsing complexity

Why Java for Windows? Windows batch scripts fundamentally cannot handle pipes in variables reliably. Any attempt to use delayed expansion (!var!) causes special characters to be interpreted. The Java approach is the only robust solution.

Unix: Improved awk-based Parser

Updated the concat_lines function to:

  • Use awk instead of xargs to preserve argument boundaries
  • Escape unquoted pipe symbols (|\|) for eval safety
  • Preserve quoted strings correctly
  • Handle comments and variable substitution

Why different approaches? Unix shells have powerful text processing tools (awk, sed) that can handle this efficiently and with minimal overhead. Windows batch scripts lack these capabilities, making the Java approach necessary there. Performance testing showed the awk approach is ~200ms faster than Java compilation on Unix (320ms vs 520ms for mvn -v).

Changes

Core Changes

  • apache-maven/src/assembly/maven/bin/JvmConfigParser.java (NEW): Java parser for jvm.config files
  • apache-maven/src/assembly/maven/bin/mvn.cmd: Replaced complex batch script parsing with Java parser invocation
  • apache-maven/src/assembly/maven/bin/mvn: Improved concat_lines function with awk-based pipe escaping
  • apache-maven/src/assembly/component.xml: Include *.java file in Windows distribution

Testing

  • MavenITgh11363PipeSymbolsInJvmConfigTest: Integration test verifying pipe symbols work correctly
  • Test resources with .mvn/jvm.config containing pipe symbols
  • Existing tests (MavenITmng4559MultipleJvmArgsTest, MavenITmng4559SpacesInJvmOptsTest) continue to pass

Testing

The integration tests verify:

  • ✅ Pipe symbols are preserved in JVM arguments
  • ✅ Quoted values with pipes work correctly
  • ✅ Multiple arguments per line work
  • ✅ Variable substitution works
  • ✅ Comments are properly removed
  • ✅ No shell/batch parsing errors occur
  • ✅ Spaces in quoted values are preserved

Compatibility

  • Backward compatible: All existing jvm.config files continue to work
  • Cross-platform: Works on Windows, Linux, macOS
  • Java version: Requires Java 17+ (already required by Maven 4)
  • No breaking changes: Existing functionality preserved

Example

Input (.mvn/jvm.config):

# JVM configuration
-Xmx2048m -Xms1024m -Dtest.prop1=value1
-Dhttp.nonProxyHosts=de|*.de|my.company.mirror.de
-Dprop.with.pipes="value|with|pipes"

Output (arguments passed to JVM):

-Xmx2048m
-Xms1024m
-Dtest.prop1=value1
-Dhttp.nonProxyHosts=de|*.de|my.company.mirror.de
-Dprop.with.pipes=value|with|pipes

All special characters are preserved correctly! 🎉

@gnodet gnodet changed the title [MNG-11363] Fix pipe symbol parsing in .mvn/jvm.config Fix pipe symbol parsing in .mvn/jvm.config (fix #11363) Oct 30, 2025
@gnodet gnodet added bug Something isn't working backport-to-4.0.x labels Oct 30, 2025
@gnodet gnodet force-pushed the fix/mng-11363-pipe-symbols-jvm-config branch 2 times, most recently from 465eef5 to bdc5130 Compare November 1, 2025 07:39
@gnodet
Copy link
Contributor Author

gnodet commented Nov 1, 2025

Updated all commit messages to use [gh-11363] instead of [MNG-11363] to follow the correct issue reference format.

@gnodet
Copy link
Contributor Author

gnodet commented Nov 1, 2025

Fixed the integration test failures. The issue was that the previous implementation was escaping pipes in concat_lines but then using eval which still interpreted them as shell operators.

The updated fix:

  1. Uses awk to escape unquoted pipes with backslashes in the concat_lines function
  2. Uses eval with the escaped MAVEN_OPTS to properly handle the pipes

This ensures that pipe symbols in JVM arguments (like -Dhttp.nonProxyHosts=de|*.de) are treated as literal characters rather than shell pipe operators.

All previously failing tests now pass:

  • ✅ MavenITgh11363PipeSymbolsInJvmConfigTest
  • ✅ MavenITmng4559MultipleJvmArgsTest
  • ✅ MavenITmng5889FindBasedir
  • ✅ MavenITmng6223FindBasedir
  • ✅ MavenITmng8598JvmConfigSubstitutionTest

@gnodet gnodet force-pushed the fix/mng-11363-pipe-symbols-jvm-config branch 3 times, most recently from c2acb11 to c246af5 Compare November 3, 2025 19:20
The concat_lines function in the Unix mvn script was using xargs -n 1
which split arguments by whitespace, causing pipe symbols (|) in JVM
arguments to be interpreted as shell command separators.

This fix replaces the problematic implementation with a line-by-line
reader that preserves quoted arguments and special characters while
maintaining all existing functionality:
- Comments are still removed
- Empty lines are still filtered out
- Variable substitution (MAVEN_PROJECTBASEDIR) still works
- Pipe symbols and other special characters are preserved

The Windows mvn.cmd seems too limited, so we use a custom java
class to parse the jvm.config file and process it.

Added integration test to verify pipe symbols in jvm.config work
correctly and don't cause shell parsing errors.
@gnodet gnodet force-pushed the fix/mng-11363-pipe-symbols-jvm-config branch from 4d00db1 to d1dc6f1 Compare November 5, 2025 11:13
@gnodet gnodet requested a review from cstamas November 5, 2025 11:13
Copy link
Member

@cstamas cstamas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One nit: why not use same Java helper on all platforms?

@gnodet
Copy link
Contributor Author

gnodet commented Nov 5, 2025

One nit: why not use same Java helper on all platforms?

Answered in the description.
When doing perfs measurements, running mvn -v went from 320 ms (using awk) to 520 ms (using the java parser).
So I don't think it's better to keep two separate solutions at this point.

@gnodet gnodet force-pushed the fix/mng-11363-pipe-symbols-jvm-config branch from 4442e67 to d1dc6f1 Compare November 6, 2025 12:12
- Always initialize JVM_CONFIG_MAVEN_OPTS to empty to avoid inheriting stale values from environment
- Add debug output to see actual values during IT runs
- This prevents previous test runs from interfering with current test
Write JVM_CONFIG_MAVEN_OPTS and MAVEN_OPTS values to mvn-debug.txt
in the project directory so we can check what values are being used
during IT runs.
- Use random temp directory for each invocation to avoid conflicts
- Capture stderr from Java parser to debug output file
- Clean up temp directory after use
- This ensures complete isolation between different Maven invocations
- Removed debug output to mvn-debug.txt
- Cleaned up error capture logic
- Ready for production use
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-to-4.0.x bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[REGRESSION] Maven 4 fails to parse pipe symbols in .mvn/jvm.config

2 participants