Skip to content

using DownloadManager to download from local filesystem and disable_progress_bar, there will be an exception  #6409

@neiblegy

Description

@neiblegy

Describe the bug

i'm using datasets.download.download_manager.DownloadManager to download files like "file:///a/b/c.txt", and i disable_progress_bar() to disable bar. there will be an exception as follows:

AttributeError: 'function' object has no attribute 'close' Exception ignored in: <function TqdmCallback.__del__ at 0x7fa8683d84c0> Traceback (most recent call last): File "/home/protoss.gao/.local/lib/python3.9/site-packages/fsspec/callbacks.py", line 233, in __del__ self.tqdm.close()

i check your source code in datasets/utils/file_utils.py:348 you define TqdmCallback derive from fsspec.callbacks.TqdmCallback

but in the newest fsspec code https://github.com/fsspec/filesystem_spec/blob/master/fsspec/callbacks.py , line 146, in this case, _DEFAULT_CALLBACK will take effect, but in line 234, it calls "close()" function which _DEFAULT_CALLBACK don't have such thing.

so i think the class "TqdmCallback" in datasets/utils/file_utils.py may override "del" function or report this bug to fsspec.

Steps to reproduce the bug

as i said

Expected behavior

no exception

Environment info

datasets: 2.14.4
python: 3.9
platform: x86_64

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions