Skip to content

Conversation

@TyurinIvan
Copy link

remote: fix problem with dvc pull from HDFS

Prepared a patch for version 4.0.1 to fix issue. I want to publish this fix in PyPi as version 4.0.1->4.0.2 to be able to use dvc for HDFS and python 3.8

Issue: dvc/issues/8642

@TyurinIvan
Copy link
Author

TyurinIvan commented Feb 21, 2024

Hello! Please tell me if it is possible to publish my changes for version 4.0.1->4.0.2 to support HDFS with python 3.8. If I sent an incorrect pull request, please clarify what needs to be corrected to publish these changes to PyPi.

The proposed changes allow us to get rid of the dvc/issues/8642 error without limiting the fsspec version

@TyurinIvan TyurinIvan changed the title 4.0.2 issue-8642 fix. DVC pull from HDFS remote: issue-8642 fix. DVC pull from HDFS error Feb 21, 2024
@TyurinIvan TyurinIvan changed the title remote: issue-8642 fix. DVC pull from HDFS error remote: issue-8642 fix. DVC pull from HDFS error (4.0.1->4.0.2) Feb 21, 2024
@skshetry
Copy link
Collaborator

skshetry commented Feb 21, 2024

Hi, @TyurinIvan. dvc-objects dropped support for Python 3.8 recently in #280.

Regarding the callback issue, it will have to be fixed upstream in fsspec.

@TyurinIvan
Copy link
Author

Hi, @skshetry. The attached issue suggests downgrading the fsspec version (current actual version is 2024.2.0). For versions fsspec>2022.10.0, the described problem persists.
I have prepared a patch (for dvc-objects==4.0.1) that works for versions fsspec>2022.10.0 as well. I suggest releasing dvc-objects version 4.0.1->4.0.2 to allow the old version of dvc[hdfs] to be used with python 3.8 for those who cannot yet use the newer version of python by any reasons.

Thus, to use any available version of dvc for python3.8, I need to use the outdated version of fsspec<=2022.10.0, which can lead to unresolved conflicts when using other libraries (for example, huggingface-hub).

Please tell me, is it possible to release dvc-objects==4.0.2 while the current version dvc-objects==5.0.0 exists?

@skshetry
Copy link
Collaborator

We are trying to get huggingface support newer fsspec versions: huggingface/datasets#6687.

So you'd be able to use newer versions of fsspec when that gets merged. Regarding the callback bug, it'll have to be fixed in the fsspec.

Regarding this PR, we don't make patch releases for older versions. So I am sorry, we can't accept this PR. :(

@dependabot dependabot bot deleted the branch treeverse:dependabot/github_actions/codecov/codecov-action-4.0.1 February 26, 2024 22:23
@dependabot dependabot bot closed this Feb 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants