Skip to content

Conversation

@mscolnick
Copy link
Contributor

@mscolnick mscolnick commented Oct 14, 2025

This migrates mo.ui.dataframe logic to be backed by narwhals to both 1) reduce maintenance burden, 2) support more dataframe types.

We now support: duckdb, ibis, pandas, polars

This will also fix duckdb filtering when using mo.ui.table()

@vercel
Copy link

vercel bot commented Oct 14, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
marimo-docs Ready Ready Preview Comment Oct 14, 2025 7:50pm

@mscolnick mscolnick requested a review from Light2Dark October 14, 2025 13:49
@mscolnick mscolnick changed the title improvement: migrate dataframes to narwhals improvement: migrate mo.ui.dataframe to narwhals; support duckdb, ibis, pandas, polars Oct 14, 2025
@mscolnick mscolnick requested a review from Copilot October 14, 2025 13:49
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR migrates the mo.ui.dataframe component to use narwhals as the backend abstraction layer, expanding support from just pandas and polars to include duckdb, ibis, pandas, and polars. This reduces maintenance burden by consolidating dataframe handling logic under a single API while fixing duckdb filtering when using mo.ui.table().

  • Replaces individual transform handlers (PandasTransformHandler, PolarsTransformHandler, IbisTransformHandler) with a unified NarwhalsTransformHandler
  • Updates test infrastructure to use parameterized tests with create_dataframes helper
  • Changes column ID types from Union[str, int] to str for consistency across backends

Reviewed Changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/_plugins/ui/_impl/dataframes/test_print_code.py Updates tests to use NarwhalsTransformHandler and narwhals lazy dataframes
tests/_plugins/ui/_impl/dataframes/test_handlers.py Refactors tests to use unified narwhals approach with helper functions
tests/_plugins/ui/_impl/dataframes/test_dataframe.py Updates dataframe tests to work with narwhals-backed implementation
marimo/_smoke_tests/dataframes/transforms.py Adds smoke test script for testing transforms across multiple dataframe backends
marimo/_plugins/ui/_impl/dataframes/transforms/types.py Updates type definitions to use narwhals types and removes int from ColumnId
marimo/_plugins/ui/_impl/dataframes/transforms/handlers.py Replaces backend-specific handlers with unified NarwhalsTransformHandler
marimo/_plugins/ui/_impl/dataframes/transforms/apply.py Updates transform application to work with narwhals lazy frames
marimo/_plugins/ui/_impl/dataframes/dataframe.py Modifies dataframe component to use narwhals internally while preserving external API

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Light2Dark
Light2Dark previously approved these changes Oct 14, 2025
@mscolnick mscolnick merged commit ebc0ac2 into main Oct 14, 2025
32 of 41 checks passed
@mscolnick mscolnick deleted the ms/migrate-dataframe-to-narwhals branch October 14, 2025 20:04
@dmadisetti dmadisetti added the enhancement New feature or request label Oct 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants