Skip to content

Conversation

@pepijndevos
Copy link
Contributor

@pepijndevos pepijndevos commented Aug 11, 2025

📝 Summary

Update pyodide to 0.28 which allows decoupling the package version from the pyodide version making it easier in the future for people to use new packages without requiring bumping pyodide

🔍 Description of Changes

https://pyodide.org/en/stable/project/changelog.html#packages

This decouples the set of packages from the Pyodide version and allows people to use the different sets of packages easily with different Pyodide versions. pyodide/pyodide#5699

My particular interest is in pyodide/pyodide-recipes#102

📋 Checklist

  • I have read the contributor guidelines.
  • For large changes, or changes that affect the public API: this change was discussed or approved through an issue, on Discord, or the community discussions (Please provide a link if applicable).
  • I have added tests for the changes made.
  • I have run the code and verified that it works as expected.

@vercel
Copy link

vercel bot commented Aug 11, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
marimo-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 11, 2025 11:48am

Copy link
Contributor

@mscolnick mscolnick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this removes a lot of used packages at the moment, so we cannot do this upgrade.

https://pyodide.org/en/stable/project/changelog.html#version-0-28-0

BREAKING CHANGE The following packages are removed from the Pyodide distribution because of the build issues. We will try to fix them in the future:
arro3-compute
arro3-core
arro3-io
Cartopy
duckdb
gensim
geopandas
mne
osqp
polars
pyarrow
pygame-ce
pyproj
zarr

@pepijndevos
Copy link
Contributor Author

You mean Marimo actively relies on some of those packages or it might break user code?

The broken packages are being tracked here: pyodide/pyodide-recipes#99

Maybe it would be worth weighing in on that issue with packages that are critical to Marimo.

It seems that some of them are an easy update while others are unlikely to resolve themselves any time soon, and at some point the increasing number of missing and outdated packages in 0.27 would seem to outweigh some obscure broken packages. I for one already depend on a package that is not in 0.27.

@mscolnick
Copy link
Contributor

We rely on them and so do our users. Especially polars, duckdb, and pyarrow

@agriyakhetarpal
Copy link

Hi! Coming from pyodide/pyodide-recipes#99: zarr is a non-trivial update, and it would work only in Chrome 137+ anyway due to JSPI (zarr-developers/zarr-python#1903). However, it might be worth adding only these three packages (polars, duckdb, pyarrow) and updating to Pyodide 0.28, and then the rest of them can be added when enabled in another lockfile update – so marimo can bump to a newer version with just a limited number of packages and include only the popular ones. Unfortunately, there hasn't been much progress on them yet. I'll try to see if I can update PyArrow, and DuckDB and arro3 were already in progress, IIRC. We already have stable builds for the pyodide_2025 ABI in place for downstream packagers.

@mscolnick
Copy link
Contributor

thanks @agriyakhetarpal! let us know how we can be helpful

@agriyakhetarpal
Copy link

I've opened an issue at apache/arrow#47329 and a PR at apache/arrow#47330 to test PyArrow with Pyodide 0.28, and then we can issue a build of PyArrow if most tests pass (as we don't want it to simply fail on import with the new ABI).

As per duckdb/duckdb-pyodide#7, DuckDB is building well against the new ABI, and all that is needed is a dedicated build somewhere available as a GitHub release or similar (ideally with attestations to prove provenance).

@agriyakhetarpal
Copy link

Hi @ryanking13, I don't see an issue for polars in pyodide/pyodide-recipes#99. There is comment from you about how they need to pass -fwasm-exceptions when building, and I see that they set the Emscripten version as unpinned in https://github.com/pola-rs/polars/blob/0031e4e21a95b2e2d23ef3f270cee64a7f4bb02b/.github/workflows/release-python.yml#L338-L343, and that means that inadvertently a build with Emscripten 4.0.12 was created at https://github.com/pola-rs/polars/releases/tag/py-1.32.2 rather than with Emscripten 4.0.9. It also looks like they are still testing against Pyodide 0.26, unfortunately: https://github.com/pola-rs/polars/blob/0031e4e21a95b2e2d23ef3f270cee64a7f4bb02b/.github/workflows/test-pyodide.yml. I'll put up a PR to add -fwasm-exceptions and also update their Pyodide version.

@pepijndevos
Copy link
Contributor Author

Hm, I tried to use my patched version locally and it seems to want to install packages from eg https://wasm.marimo.app/micropip-0.10.1-py3-none-any.whl

This seems to be coming from

lockFileURL: `https://wasm.marimo.app/pyodide-lock.json?v=${opts.version}&pyodide=${opts.pyodideVersion}`,

So I guess further patches on my end would be required, or I need to wait for the Marimo team to publish a 0.28.1 lockfile and packages?

@agriyakhetarpal
Copy link

Yes, I think it would be useful if marimo could allow overriding the value of the lockFileURL. I am not familiar with how marimo's configuration system works. If there isn't something to put in a YAML/JSON/TOML file for this already, maybe a [tool.marimo.wasm] table in pyproject.tomland a pyodide-lockfile key (or similar) would be apt.

@MattOates
Copy link

For duckdb support at least, this is waiting on this way upstream change duckdb/duckdb-pyodide#7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants