Skip to content

Remove dataframe protocol#17909

Merged
rapids-bot[bot] merged 10 commits intorapidsai:branch-25.04from
vyasr:chore/remove_dataframe_protocol
Feb 6, 2025
Merged

Remove dataframe protocol#17909
rapids-bot[bot] merged 10 commits intorapidsai:branch-25.04from
vyasr:chore/remove_dataframe_protocol

Conversation

@vyasr
Copy link
Copy Markdown
Contributor

@vyasr vyasr commented Feb 4, 2025

Description

Follow-up to #17736

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@vyasr vyasr added improvement Improvement / enhancement to an existing function breaking Breaking change labels Feb 4, 2025
@vyasr vyasr self-assigned this Feb 4, 2025
@vyasr vyasr requested a review from a team as a code owner February 4, 2025 01:04
@github-actions github-actions Bot added the Python Affects Python cuDF API. label Feb 4, 2025
@mroeschke
Copy link
Copy Markdown
Contributor

Hmm darn. Ideally we should have had warnings as errors enabled for cudf pandas tests based on the configs in pyproject.toml, but because we're using a pytest.ini for these tests it looks like we didn't anticipate the test failure here

@github-actions github-actions Bot added the cudf.pandas Issues specific to cudf.pandas label Feb 4, 2025
@vyasr
Copy link
Copy Markdown
Contributor Author

vyasr commented Feb 4, 2025

Hmm yeah that is unfortunate. We should fix that in a follow-up PR. I don't know that we can reuse existing pyproject.toml files though given the special nature of these tests. Maybe our best bet is to just copy over filters from our other pyprojects?

@galipremsagar
Copy link
Copy Markdown
Contributor

/merge

@galipremsagar
Copy link
Copy Markdown
Contributor

galipremsagar commented Feb 4, 2025

I think we will need to investigate the massive increase in duration of pandas-tests job.

On a different pr it is(50 mins): https://github.com/rapidsai/cudf/actions/runs/13120888958/job/36623191308

On this pr it is consistently going beyond (1hr 30mins): https://github.com/rapidsai/cudf/actions/runs/13128280990/job/36670004745?pr=17909

@galipremsagar
Copy link
Copy Markdown
Contributor

I think we will need to investigate the massive increase in duration of pandas-tests job.

On a different pr it is(50 mins): https://github.com/rapidsai/cudf/actions/runs/13120888958/job/36623191308

On this pr it is consistently going beyond (1hr 30mins): https://github.com/rapidsai/cudf/actions/runs/13128280990/job/36670004745?pr=17909

I think we will need to investigate the massive increase in duration of pandas-tests job.

On a different pr it is(50 mins): https://github.com/rapidsai/cudf/actions/runs/13120888958/job/36623191308

On this pr it is consistently going beyond (1hr 30mins): https://github.com/rapidsai/cudf/actions/runs/13128280990/job/36670004745?pr=17909

I'm investigating what is causing this slowdown.

@mroeschke
Copy link
Copy Markdown
Contributor

Maybe not the root of the slowdown, but it appears tests/io/test_pickle.py::test_pickle_generalurl_read is taking a long amount of time on the 3 times pandas-tests was retried (the http calls are supposed to be mocked, but maybe not well enough)

2025-02-04T17:07:30.8282675Z [gw2] [ 99%] FAILED tests/io/test_pickle.py::test_pickle_generalurl_read[http://url.com] 
2025-02-04T17:36:11.9497790Z tests/io/test_pickle.py::test_pickle_generalurl_read[ftp://test.com] 

@vyasr vyasr added the DO NOT MERGE Hold off on merging; see PR for details label Feb 4, 2025
@vyasr
Copy link
Copy Markdown
Contributor Author

vyasr commented Feb 4, 2025

Marked as do not merge until Prem has a chance to get to the bottom of the perf hit.

@galipremsagar galipremsagar requested a review from a team as a code owner February 4, 2025 23:51
@galipremsagar galipremsagar requested a review from bdice February 4, 2025 23:51
Comment thread ci/cudf_pandas_scripts/pandas-tests/run.sh Outdated
@galipremsagar
Copy link
Copy Markdown
Contributor

/merge

@galipremsagar galipremsagar added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed DO NOT MERGE Hold off on merging; see PR for details labels Feb 6, 2025
@rapids-bot rapids-bot Bot merged commit 92cf560 into rapidsai:branch-25.04 Feb 6, 2025
PYTEST_IGNORES="--ignore=tests/io/parser/common/test_read_errors.py \
--ignore=tests/io/test_clipboard.py"
--ignore=tests/io/test_clipboard.py \
--ignore=tests/io/test_pickle.py"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-opened the issue related to this: #15459

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

5 - Ready to Merge Testing and reviews complete, ready to merge breaking Breaking change cudf.pandas Issues specific to cudf.pandas improvement Improvement / enhancement to an existing function Python Affects Python cuDF API.

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants