Use GraalVM Native Image for Mill Client (1500USD Bounty)

[lihaoyi]

---

From the maintainer Li Haoyi: I'm putting a 1500USD bounty on this issue, payable by bank transfer on a merged PR implementing this.

---

Mill's client is currently a Java/JVM application. This means it suffers from the normal JVM slow startup (>100ms) and warmup times, and it prevents us from moving more logic into the client otherwise it will become even slower.

We should explore using GraalVM to generate cross-platform native image launchers for the Mill client, and integrate that into our ./mill script. This will allow us to make the client smarter:

• Using Scala rather than Java
• Parsing command line arguments in the client, which would remove the -i must be passed as first argument restriction
• Allow the client to use Coursier to bootstrap the JVM for the server to use, i.e. Allow Mill to bootstrap its own Java version #3951

For this bounty, the success criteria of this issue is twofold:

1. An example in the Mill documentation of the recommended way to use Graal native image to compile a native binary from a JavaModule, ScalaModule, or KotlinModule

2. Have a Mill generate a native image version of the runner.client jar on different operating systems and CPU architectures (linux,mac,windows X arm,intel) that is able to replace the current JVM client launcher to launch and manage the Mill backend server process, using the same Java code that runner.client is implemented with today.

Reference https://blogs.oracle.com/developers/post/building-cross-platform-native-images-with-graalvm

[ajaychandran]

@lihaoyi Any tips on how to resolve the following?

Exception in thread "main" java.lang.RuntimeException: MILL_CLASSPATH is empty!
	at mill.runner.client.MillProcessLauncher.millClasspath(MillProcessLauncher.java:152)
	at mill.runner.client.MillProcessLauncher.millLaunchJvmCommand(MillProcessLauncher.java:190)
	at mill.runner.client.MillProcessLauncher.launchMillServer(MillProcessLauncher.java:57)
	at mill.runner.client.MillClientMain$1.initServer(MillClientMain.java:56)
	at mill.main.client.ServerLauncher.run(ServerLauncher.java:131)
	at mill.main.client.ServerLauncher.acquireLocksAndRun(ServerLauncher.java:111)
	at mill.runner.client.MillClientMain.main(MillClientMain.java:63)
	at java.base@24/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)

IIUC, the value is read from java.class.path system property. But this is empty for the native image.

[alexarchambault]

1. An example in the Mill documentation of the recommended way to use Graal native image to compile a native binary from a JavaModule, ScalaModule, or KotlinModule

@lihaoyi What do you have in mind for this point? I have a mill-native-image plugin, used by Scala CLI in particular, that can be used for that, and could be added to the contrib folder of Mill if needed

[lihaoyi]

@ajaychandran I am assuming you're using #4009 because without that PR I can't build a native-image at all. w.r.t. the MILL_CLASSPATH.

Currently, the mill client and mill server use the same assembly jar with a bash prelude launcher, which is why we can do that.

If we use a native client, we'll need to find another way to package them together. Maybe we can use the native image as the prelude launcher for the assembly and have it unzip or unpack itself somehow before execution so it can reference the jar file? I don't actually know what the answer here is, but we can probably come up with something

[lihaoyi]

@alexarchambault I was thinking to inline native-image support into mill.javalib.* and having examples of using it in all 3 JVM languages (Java/Scala/Kotlin). This will also open up the possibility of using it for other examples, such as Micronaut which is meant to be native-image compatible

In general we cannot use third-party plugins in our example test suite without causing circular dependency and bootstrapping issues, so if we want to begin using native image (which I do) we need it to be in-sourced into the Mill repo

[lihaoyi]

@ajaychandran seems like if we bundle the graal launcher and the rest of the jars in a zip, like we do now for the bash launcher, we can access the executable path via https://stackoverflow.com/a/77736422/871202

[lolgab]

If we use a native client, we'll need to find another way to package them together. Maybe we can use the native image as the prelude launcher for the assembly and have it unzip or unpack itself somehow before execution so it can reference the jar file? I don't actually know what the answer here is, but we can probably come up with something

Since binary headers are at the beginning of files and zip headers (jar is a zip) are at the end, one hack you can use ( cosmopolitan libc does that) is to concatenate the binary and the jar which makes the file both a native binary and a jar. If you run the binary directly you are running the client, if you run it with java -jar you run the server.

//> using platform scalaNative
@main
def run =
  println("hello world from scala native!")

scala-cli package -f .

generates run

@main
def run =
  println("hello world from java!")

scala-cli package -f --assembly .

generates run.jar

Then you concatenate them:

cat run run.jar > concatenated
chmod +x concatenated

And you get two applications in the same file.

Running the Scala Native application

time ./concatenated
hello world from scala native!

________________________________________________________
Executed in    4.63 millis    fish           external
   usr time    1.26 millis  131.00 micros    1.13 millis
   sys time    2.49 millis  714.00 micros    1.77 millis

Running the Scala JVM application

time java -jar ./concatenated
hello world from java!

________________________________________________________
Executed in  137.97 millis    fish           external
   usr time  139.35 millis  131.00 micros  139.22 millis
   sys time   22.56 millis  368.00 micros   22.19 millis

[ajaychandran]

@lolgab Thank you for the wonderful tip!

Initial results look promising:

$ time ./mill --version
Mill Build Tool version 0.12.2-50-b1f6be
Java version: 24, vendor: Oracle Corporation, runtime: ~/.sdkman/candidates/java/24.ea.22-graal
Default locale: en_US, platform encoding: UTF-8
OS name: "Linux", version: 6.8.0-49-generic, arch: amd64

real	0m0.116s
user	0m0.204s
sys	0m0.084s

$ time ./native --version
Mill Build Tool version 0.12.2-55-2482ce-DIRTY700a3522
Java version: 24, vendor: Oracle Corporation, runtime: ~/.sdkman/candidates/java/24.ea.22-graal
Default locale: en_US, platform encoding: UTF-8
OS name: "Linux", version: 6.8.0-49-generic, arch: amd64

real	0m0.023s
user	0m0.002s
sys	0m0.016s

$ time java -jar native --version
Mill Build Tool version 0.12.2-55-2482ce-DIRTY700a3522
Java version: 24, vendor: Oracle Corporation, runtime: ~/.sdkman/candidates/java/24.ea.22-graal
Default locale: en_US, platform encoding: UTF-8
OS name: "Linux", version: 6.8.0-49-generic, arch: amd64

real	0m0.100s
user	0m0.201s
sys	0m0.069s

[ajaychandran]

@lihaoyi

An example in the Mill documentation of the recommended way to use Graal native image to compile a native binary from a JavaModule, ScalaModule, or KotlinModule

Should these be added under example directory?
This would require setup-graalvm Github Action (for publishing docs).

[sideeffffect]

This would require setup-graalvm Github Action

Why? GraalVM seems to be well supported by ordinary setup-java: https://github.com/actions/setup-java/?tab=readme-ov-file#supported-distributions

[lihaoyi]

The new Configuring JVM versions functionality should be able to pull in the desired Graal versions we want without needing a global install I think. In my cursory experiments I was able to do so

https://mill-build.org/mill/fundamentals/configuring-jvm-versions.html