Skip to content

Release 2.3.0 broke custom iterable datasets #4612

@aapot

Description

@aapot

Describe the bug

Trying to iterate examples from custom iterable dataset fails to bug introduced in torch_iterable_dataset.py since the release of 2.3.0.

Steps to reproduce the bug

next(iter(custom_iterable_dataset))

Expected results

next(iter(custom_iterable_dataset)) should return examples from the dataset

Actual results

/usr/local/lib/python3.7/dist-packages/datasets/formatting/dataset_wrappers/torch_iterable_dataset.py in _set_fsspec_for_multiprocess()
     16     See https://github.com/fsspec/gcsfs/issues/379
     17     """
---> 18     fsspec.asyn.iothread[0] = None
     19     fsspec.asyn.loop[0] = None
     20 

AttributeError: module 'fsspec' has no attribute 'asyn'

Environment info

  • datasets version: 2.3.0
  • Platform: Linux-5.4.188+-x86_64-with-Ubuntu-18.04-bionic
  • Python version: 3.7.13
  • PyArrow version: 8.0.0
  • Pandas version: 1.3.5

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions