fix webdataset filename split #6849

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Closed

Bowser1704 wants to merge 1 commit into huggingface:main from Bowser1704:fix/webdataset-filename

Bowser1704 commented Apr 29, 2024

use os.path.splitext to parse field_name.

fix filename which has dot. like:

a.b.jpeg
a.b.txt


          fix webdataset filename split

3782bd5

Bowser1704 force-pushed the fix/webdataset-filename branch from d404bc9 to 3782bd5 Compare

April 29, 2024 11:14

Member

lhoestq commented Jun 4, 2024

Hi ! This was fixed recently in #6904 and #6931

lhoestq closed this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet