Skip to content

tests: pytest-changed #6761

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 9 commits into from
Oct 14, 2025
Merged

tests: pytest-changed #6761

merged 9 commits into from
Oct 14, 2025

Conversation

@mscolnick
Copy link
Contributor

@mscolnick mscolnick commented Oct 13, 2025 •
edited
Loading

A new local pytest plugin, pytest-changed to run only affected tests (including transitively affected)

  1. Find Changes: Uses git diff --name-only <ref> to find Python files that have changed
  2. Build Graph: Runs ruff analyze graph --direction dependents to build a dependency graph
  3. Graph Traversal: Performs BFS from changed files to find all affected files
  4. Filter Tests: Identifies which affected files are test files
  5. Run Tests: Only runs pytest on the affected test files

@mscolnick mscolnick requested a review from manzt as a code owner October 13, 2025 19:02
@vercel
Copy link

vercel bot commented Oct 13, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
marimo-docs Ready Ready Preview Comment Oct 14, 2025 5:10pm

@mscolnick mscolnick requested a review from Light2Dark as a code owner October 13, 2025 19:06
@mscolnick mscolnick requested a review from dmadisetti October 14, 2025 00:55
@dmadisetti
Copy link
Collaborator

Can you pull main since we put in 3.14 CI?

dmadisetti
dmadisetti previously approved these changes Oct 14, 2025
View reviewed changes
Copy link
Collaborator

@dmadisetti dmadisetti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I almost would want unit tests for this just a quick sanity check- but nice to see it applied in runners

packages/pytest_changed/__init__.py
# Check if it's a test file
if file_path.exists() and (
file_path.name.startswith("test_")
or file_path.name.endswith("_test.py")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a note, in source:

marimo/_ast/pytest.py
marimo/_runtime/pytest.py

(I guess not touched)

but then in smoke tests:

marimo/_smoke_tests/lens_test.py
marimo/_smoke_tests/pdb_test.py
marimo/_smoke_tests/pygwalker_test.py
marimo/_smoke_tests/reactive_pytest.py
marimo/_smoke_tests/tqdm_update_test.py

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is ok since doesn't add more tests. i could remove file_path.name.endswith("_test.py")

@dmadisetti
Copy link
Collaborator

dmadisetti commented Oct 14, 2025 •
edited
Loading

I guess looking at the action are these only all selected because we changed the test infra itself?

  - tests/_ai/llm/test_impl.py
  - tests/_ai/test_chat_model.py
  - tests/_ai/tools/test_base.py
  - tests/_ai/tools/tools/test_cells.py
  - tests/_ai/tools/tools/test_datasource_tool.py
  - tests/_ai/tools/tools/test_errors_tool.py
  - tests/_ai/tools/tools/test_notebooks.py
  - tests/_ai/tools/tools/test_rules.py
  - tests/_ai/tools/tools/test_tables_variables.py
  - tests/_ast/test_app_cell.py
  - tests/_ast/test_cell.py
  - tests/_ast/test_cell_manager.py
  - tests/_ast/test_codegen.py
  - tests/_ast/test_compiler.py
  - tests/_ast/test_load.py
  - tests/_ast/test_pytest.py
  - tests/_ast/test_pytest_scoped.py
  - tests/_ast/test_pytest_toplevel.py
  - tests/_ast/test_sql_utils.py
  - tests/_ast/test_toplevel.py
  - tests/_ast/test_visitor.py
  - tests/_cli/test_cli.py
  - tests/_cli/test_cli_check.py
  - tests/_cli/test_cli_convert.py
  - tests/_cli/test_cli_export.py
  - tests/_cli/test_file_path.py
  - tests/_cli/test_parse_args.py
  - tests/_cli/test_sandbox.py
  - tests/_convert/test_basic_marimo_example.py
  - tests/_convert/test_convert_ipynb.py
  - tests/_convert/test_convert_non_marimo_python_script.py
  - tests/_convert/test_convert_utils.py
  - tests/_convert/test_ipynb.py
  - tests/_convert/test_ipynb_to_marimo.py
  - tests/_convert/test_markdown_conversion.py
  - tests/_data/test_charts.py
  - tests/_data/test_get_datasets.py
  - tests/_data/test_models.py
  - tests/_data/test_preview_column.py
  - tests/_data/test_sql_summaries.py
  - tests/_ipc/test_kernel_communication.py
  - tests/_islands/test_island_generator.py
  - tests/_lint/test_async_context_system.py
  - tests/_lint/test_empty_cells.py
  - tests/_lint/test_generated_with_comparison.py
  - tests/_lint/test_ignore_scripts.py
  - tests/_lint/test_json_formatter.py
  - tests/_lint/test_lint_system.py
  - tests/_lint/test_log_rules.py
  - tests/_lint/test_run_check.py
  - tests/_lint/test_runtime_errors_snapshot.py
  - tests/_lint/test_sql_log_rules_snapshot.py
  - tests/_lint/test_streaming_early_stopping.py
  - tests/_lint/test_validate_graph.py
  - tests/_lint/utils.py
  - tests/_mcp/server/prompts/test_notebooks_prompts.py
  - tests/_mcp/server/test_mcp_server.py
  - tests/_messaging/mocks.py
  - tests/_messaging/test_cell_output.py
  - tests/_messaging/test_console_output_worker.py
  - tests/_messaging/test_messaging_context.py
  - tests/_messaging/test_messaging_context_vars.py
  - tests/_messaging/test_messaging_errors.py
  - tests/_messaging/test_messaging_run_id_context.py
  - tests/_messaging/test_ops.py
  - tests/_messaging/test_print_override.py
  - tests/_messaging/test_serde.py
  - tests/_messaging/test_streams.py
  - tests/_messaging/test_variables.py
  - tests/_output/formatters/test_ai_formatters.py
  - tests/_output/formatters/test_altair.py
  - tests/_output/formatters/test_altair_formatters.py
  - tests/_output/formatters/test_formatter_utils.py
  - tests/_output/formatters/test_formatters.py
  - tests/_output/formatters/test_ibis_formatters.py
  - tests/_output/formatters/test_iframe.py
  - tests/_output/formatters/test_ipython_formatters.py
  - tests/_output/formatters/test_ipython_update.py
  - tests/_output/formatters/test_matplotlib.py
  - tests/_output/formatters/test_pandas.py
  - tests/_output/formatters/test_plotly_formatters.py
  - tests/_output/formatters/test_structures.py
  - tests/_output/formatters/test_sympy.py
  - tests/_output/test_data.py
  - tests/_output/test_formatter_registry.py
  - tests/_output/test_hypertext.py
  - tests/_output/test_md.py
  - tests/_output/test_show_code.py
  - tests/_output/test_try_format.py
  - tests/_plugins/core/test_json_encoder.py
  - tests/_plugins/core/test_web_component.py
  - tests/_plugins/stateless/status/test_progress.py
  - tests/_plugins/stateless/test_audio.py
  - tests/_plugins/stateless/test_flex.py
  - tests/_plugins/stateless/test_icon.py
  - tests/_plugins/stateless/test_image.py
  - tests/_plugins/stateless/test_image_compare.py
  - tests/_plugins/stateless/test_inspect.py
  - tests/_plugins/stateless/test_lazy.py
  - tests/_plugins/stateless/test_mpl.py
  - tests/_plugins/stateless/test_plain_text.py
  - tests/_plugins/stateless/test_routes.py
  - tests/_plugins/stateless/test_sidebar.py
  - tests/_plugins/stateless/test_style.py
  - tests/_plugins/test_download.py
  - tests/_plugins/ui/_core/test_registry.py
  - tests/_plugins/ui/_core/test_ui_element.py
  - tests/_plugins/ui/_impl/charts/test_altair_transformers.py
  - tests/_plugins/ui/_impl/chat/test_chat.py
  - tests/_plugins/ui/_impl/dataframes/test_dataframe.py
  - tests/_plugins/ui/_impl/tables/test_default_table.py
  - tests/_plugins/ui/_impl/tables/test_format.py
  - tests/_plugins/ui/_impl/tables/test_ibis_table.py
  - tests/_plugins/ui/_impl/tables/test_narwhals.py
  - tests/_plugins/ui/_impl/tables/test_pandas_table.py
  - tests/_plugins/ui/_impl/tables/test_polars_table.py
  - tests/_plugins/ui/_impl/tables/test_selection.py
  - tests/_plugins/ui/_impl/tables/test_table_utils.py
  - tests/_plugins/ui/_impl/test_altair_chart.py
  - tests/_plugins/ui/_impl/test_anywidget.py
  - tests/_plugins/ui/_impl/test_array.py
  - tests/_plugins/ui/_impl/test_batch.py
  - tests/_plugins/ui/_impl/test_comm.py
  - tests/_plugins/ui/_impl/test_data_editor.py
  - tests/_plugins/ui/_impl/test_data_explorer.py
  - tests/_plugins/ui/_impl/test_dates.py
  - tests/_plugins/ui/_impl/test_dictionary.py
  - tests/_plugins/ui/_impl/test_file_browser.py
  - tests/_plugins/ui/_impl/test_input.py
  - tests/_plugins/ui/_impl/test_nav_menu.py
  - tests/_plugins/ui/_impl/test_panel.py
  - tests/_plugins/ui/_impl/test_run_button.py
  - tests/_plugins/ui/_impl/test_table.py
  - tests/_plugins/ui/_impl/utils/test_dataframe_utils.py
  - tests/_pyodide/test_bootstrap.py
  - tests/_pyodide/test_pyodide_session.py
  - tests/_pyodide/test_pyodide_streams.py
  - tests/_runtime/output/test_output.py
  - tests/_runtime/packages/test_module_registry.py
  - tests/_runtime/packages/test_package_managers.py
  - tests/_runtime/packages/test_pypi_package_manager.py
  - tests/_runtime/reload/test_autoreload.py
  - tests/_runtime/reload/test_module_watcher.py
  - tests/_runtime/runner/test_cell_runner.py
  - tests/_runtime/script_data/contains_tests.py
  - tests/_runtime/script_data/fn_exception.py
  - tests/_runtime/script_data/script_exception.py
  - tests/_runtime/script_data/script_exception_function.py
  - tests/_runtime/script_data/script_exception_setup_cell.py
  - tests/_runtime/script_data/script_exception_with_imported_function.py
  - tests/_runtime/script_data/script_exception_with_output.py
  - tests/_runtime/script_data/script_global_setup_difference.py
  - tests/_runtime/test_app_mode.py
  - tests/_runtime/test_capture.py
  - tests/_runtime/test_complete.py
  - tests/_runtime/test_context.py
  - tests/_runtime/test_control_flow.py
  - tests/_runtime/test_dataflow.py
  - tests/_runtime/test_dataflow_cases.py
  - tests/_runtime/test_dotenv.py
  - tests/_runtime/test_functions.py
  - tests/_runtime/test_manage_script_metadata.py
  - tests/_runtime/test_marimo_pdb.py
  - tests/_runtime/test_patches.py
  - tests/_runtime/test_primitives.py
  - tests/_runtime/test_pytest_runtime.py
  - tests/_runtime/test_query_params.py
  - tests/_runtime/test_requests.py
  - tests/_runtime/test_runtime.py
  - tests/_runtime/test_runtime_datasets.py
  - tests/_runtime/test_runtime_secrets.py
  - tests/_runtime/test_single_execution.py
  - tests/_runtime/test_state.py
  - tests/_runtime/test_threads.py
  - tests/_runtime/test_trace.py
  - tests/_runtime/test_virtual_file.py
  - tests/_runtime/watch/test_watch.py
  - tests/_save/external_decorators/app.py
  - tests/_save/external_decorators/transitive_imports.py
  - tests/_save/external_decorators/transitive_wrappers_1.py
  - tests/_save/external_decorators/transitive_wrappers_2.py
  - tests/_save/loaders/mocks.py
  - tests/_save/loaders/test_loader.py
  - tests/_save/store/mocks.py
  - tests/_save/store/test_store.py
  - tests/_save/stores/test_file.py
  - tests/_save/stores/test_store_config.py
  - tests/_save/stores/test_tiered.py
  - tests/_save/stubs/test_stubs.py
  - tests/_save/test_cache.py
  - tests/_save/test_cache_versions.py
  - tests/_save/test_external_decorators.py
  - tests/_save/test_hash.py
  - tests/_server/ai/test_ai_config.py
  - tests/_server/ai/test_prompts.py
  - tests/_server/ai/test_providers.py
  - tests/_server/ai/tools/test_tool_manager.py
  - tests/_server/api/endpoints/test_ai.py
  - tests/_server/api/endpoints/test_assets.py
  - tests/_server/api/endpoints/test_config_endpoints.py
  - tests/_server/api/endpoints/test_datasources.py
  - tests/_server/api/endpoints/test_documentation.py
  - tests/_server/api/endpoints/test_editing.py
  - tests/_server/api/endpoints/test_execution.py
  - tests/_server/api/endpoints/test_export.py
  - tests/_server/api/endpoints/test_file_explorer.py
  - tests/_server/api/endpoints/test_files.py
  - tests/_server/api/endpoints/test_health.py
  - tests/_server/api/endpoints/test_home.py
  - tests/_server/api/endpoints/test_kiosk.py
  - tests/_server/api/endpoints/test_login.py
  - tests/_server/api/endpoints/test_packages.py
  - tests/_server/api/endpoints/test_resume_session.py
  - tests/_server/api/endpoints/test_sql_endpoints.py
  - tests/_server/api/endpoints/test_terminal.py
  - tests/_server/api/endpoints/test_ws.py
  - tests/_server/api/endpoints/test_ws_rtc.py
  - tests/_server/api/test_auth.py
  - tests/_server/api/test_middleware.py
  - tests/_server/conftest.py
  - tests/_server/export/test_export_utils.py
  - tests/_server/export/test_exporter.py
  - tests/_server/files/test_os_file_system.py
  - tests/_server/mocks.py
  - tests/_server/rtc/test_rtc_doc.py
  - tests/_server/session/test_serialize_session.py
  - tests/_server/session/test_serialize_session_missing_type.py
  - tests/_server/session/test_session_view.py
  - tests/_server/templates/test_templates.py
  - tests/_server/test_asgi.py
  - tests/_server/test_errors.py
  - tests/_server/test_file_manager.py
  - tests/_server/test_file_manager_filename.py
  - tests/_server/test_file_router.py
  - tests/_server/test_lsp.py
  - tests/_server/test_recents.py
  - tests/_server/test_session_manager.py
  - tests/_server/test_sessions.py
  - tests/_server/test_templates_filename.py
  - tests/_snippets/test_snippets.py
  - tests/_sql/test_clickhouse.py
  - tests/_sql/test_dbapi.py
  - tests/_sql/test_duckdb.py
  - tests/_sql/test_engine_utils.py
  - tests/_sql/test_engines.py
  - tests/_sql/test_get_engines.py
  - tests/_sql/test_ibis.py
  - tests/_sql/test_pyiceberg.py
  - tests/_sql/test_redshift.py
  - tests/_sql/test_sql.py
  - tests/_sql/test_sql_error_handling.py
  - tests/_sql/test_sqlalchemy.py
  - tests/_utils/test_docs.py
  - tests/_utils/test_inline_script_metadata.py
  - tests/_utils/test_marimo_path.py
  - tests/_utils/test_msgspec_basestruct.py
  - tests/_utils/test_parse_dataclass.py
  - tests/_utils/test_uv_tree.py
  - tests/conftest.py
  - tests/mocks.py
  - tests/test_api.py
  - tests/test_entrypoints.py

@mscolnick mscolnick merged commit 07dff1a into main Oct 14, 2025
37 of 40 checks passed
@mscolnick mscolnick deleted the ms/pytest-unchanged branch October 14, 2025 17:30
@dmadisetti dmadisetti added bug Something isn't working internal A refactor or improvement that is not user facing labels Oct 15, 2025
@dmadisetti dmadisetti removed the bug Something isn't working label Oct 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dmadisetti dmadisetti dmadisetti left review comments

@manzt manzt Awaiting requested review from manzt manzt is a code owner

@Light2Dark Light2Dark Awaiting requested review from Light2Dark Light2Dark is a code owner

Assignees

No one assigned

Labels

internal A refactor or improvement that is not user facing

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@mscolnick @dmadisetti