Progress reporting for module loading and compilation #5883

theduke · 2025-11-18T13:50:58Z

feat: Progress reporting and abort for module compilation

Add a new Module::new_with_progress function, which will call a progress
callback with progress information.

The callback may also return an error , which will abort compilation.
feat(wasix): Module load/compile progress reporting

Implement progress reporting for module loading.
- Extends ModuleCache with a load_with_progress() method, which receives
  a progress callback
- Clean up and unify the module loading methods on the Runtime trait.
  Replace old methods with a new, general fn resolve_module(), which
  replaces all the old methods.
  This new method also takes an optional progress callback.
feat(cli): run command: Show compilation progress

Show compilation progress using the newly added progress tracking.

NOTE: needs a test or two.
Also can't currently test the CLI progress reporting because of the Cloudflare
outage, will do so later.

Add a new Module::new_with_progress function, which will call a progress callback with progress information. The callback may also return an error , which will abort compilation.

Implement progress reporting for module loading. * Extends ModuleCache with a load_with_progress() method, which receives a progress callback * Clean up and unify the module loading methods on the Runtime trait. Replace old methods with a new, general fn resolve_module(), which replaces all the old methods. This new method also takes an optional progress callback.

Show compilation progress using the newly added progress tracking.

Copilot

Pull Request Overview

This PR adds comprehensive progress reporting capabilities for WebAssembly module loading and compilation. It introduces a new callback-based mechanism that allows consumers to track compilation progress, receive detailed phase information, and abort compilation if needed.

Key changes:

New Module::new_with_progress() API for compilation progress tracking with abort support
Unified module loading through Runtime::resolve_module() replacing multiple deprecated methods
Progress reporting infrastructure with CompilationProgress and ModuleLoadProgress types

Reviewed Changes

Copilot reviewed 26 out of 26 changed files in this pull request and generated 17 comments.

Show a summary per file

File	Description
`lib/types/src/progress.rs`	Defines core progress types (`CompilationProgress`, `CompilationProgressCallback`, `UserAbort`)
`lib/types/src/error.rs`	Adds `UserAbort` variant to `CompileError` for user-initiated aborts
`lib/wasix/src/runtime/module_cache/progress.rs`	Defines module load progress types for cache operations and downloads
`lib/wasix/src/runtime/module_cache/types.rs`	Extends `ModuleCache` trait with `load_with_progress()` default implementation
`lib/wasix/src/runtime/mod.rs`	Introduces `ModuleInput` enum and `resolve_module()` method, deprecates old loading methods
`lib/api/src/entities/module/mod.rs`	Adds public `new_with_progress()` method to Module API
`lib/api/src/entities/module/inner.rs`	Dispatches progress callbacks to backend-specific implementations
`lib/api/src/backend/sys/entities/module.rs`	Implements progress reporting for sys backend with compiler feature support
`lib/api/src/backend/{wasmi,wamr,v8,js,jsc}/entities/module.rs`	Stub implementations for other backends (progress ignored)
`lib/compiler/src/engine/inner.rs`	Adds `compile_with_progress()` method to Engine
`lib/compiler/src/engine/artifact.rs`	Threads progress callback through artifact creation
`lib/compiler/src/compiler.rs`	Extends Compiler trait with progress callback parameter
`lib/compiler-llvm/src/compiler.rs`	Implements progress tracking with atomic counters for parallel compilation
`lib/compiler-cranelift/src/compiler.rs`	Implements progress tracking for both parallel and sequential compilation
`lib/wasix/src/runners/wcgi/runner.rs`	Migrates to new `resolve_module_sync()` API
`lib/wasix/src/bin_factory/exec.rs`	Migrates to new `resolve_module()` API
`lib/cli/src/commands/run/runtime.rs`	Implements progress bar UI for compilation using new progress reporting
`lib/wasix/src/bin_factory/binary_package.rs`	Adds `atom_ref()` helper method and fixes trailing whitespace

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

lib/types/src/progress.rs

lib/wasix/src/runtime/module_cache/progress.rs

lib/compiler/src/engine/inner.rs

lib/compiler-cranelift/src/compiler.rs

lib/wasix/src/runtime/mod.rs

lib/wasix/src/runtime/module_cache/progress.rs

Copilot · 2025-11-18T14:02:36Z

lib/wasix/src/runtime/module_cache/progress.rs

+#[derive(Clone, Debug)]
+pub struct ProgressError {
+    message: String,
+}
+
+impl ProgressError {
+    pub fn new(message: impl Into<String>) -> Self {
+        Self {
+            message: message.into(),
+        }
+    }
+}


Missing documentation: The ProgressError struct and its methods lack documentation. Consider adding doc comments explaining its purpose and how it differs from other error types.

syrusakbary

Im in favor of having something like this functionality wise, but really against how it exposes things.

The method should be not inside of the module, but the engine. More specifically the Sys engine.

theduke · 2025-11-18T16:31:36Z

Why though?

We should have a generic progress reporting system that isn't tied to any particular backend.

marxin

Generally speaking, I really like the idea and I am also fine with the suggested callback mechanism.

Have 2 questions/comments about the CLI:

after the change, wasmer run --llvm python stopped producing the progress messages: ⠲ Compiling WebAssembly module for command... and so on.
can we provide proper progress bar for the compilation progress based on the new API (indicatif)?

lib/compiler-cranelift/src/compiler.rs

lib/compiler-singlepass/src/compiler.rs

Makes more sense.

…ith_progress To keep the main API clean.

To enable reuse.

theduke · 2025-11-18T22:07:07Z

@marxin

Changes since the review:

Added singlepass
Added trampolines to counters
Added abort and progress tests for llvm/cranelift/singlepass in lib/api/tests/module_compilation_progress.rs
Moved method to new extension trait ProgressEngineExt (as per discussion with Syrus)
Improved the progress bar:

Other small tweaks, docs, etc

theduke · 2025-11-18T22:09:41Z

@syrusakbary

I instead moved the method to a new ProgressEngineExt::new_module_with_progress(engine, ..).

Why not on NativeEngineExt?
To have a uniform interface for all backends, and not require cumbersome #[cfg(...)] everywhere.

syrusakbary · 2025-11-18T22:13:50Z

lib/api/src/entities/engine/progress_ext.rs

+    ) -> Result<crate::Module, wasmer_types::CompileError>;
+}
+
+impl ProgressEngineExt for crate::Engine {


This impl and work should not live here. Should live only on the sys implementation that already exists (NativeEngineExt):

https://github.com/wasmerio/wasmer/blob/main/lib/api/src/backend/sys/entities/engine.rs#L53

syrusakbary · 2025-11-18T22:15:33Z

lib/api/src/backend/js/entities/module.rs

        unsafe { Self::from_binary_unchecked(_engine, binary) }
    }

+    pub(crate) fn from_binary_with_progress(
