Skip to content

Conversation

@lhoestq
Copy link
Member

@lhoestq lhoestq commented Oct 5, 2022

Added support for image and audio formatting for numpy, torch, tf and jax.

For images, the dtype used is the one of the image (the one returned by PIL.Image), e.g. uint8

I also added support for string, binary and None types. In particular for torch and jax, strings are kept unchanged (previously it was returning an error because you can't create a tensor of strings)

@lhoestq lhoestq requested a review from mariosasko October 5, 2022 13:07
@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Oct 5, 2022

The documentation is not available anymore as the PR was closed or merged.

@lhoestq
Copy link
Member Author

lhoestq commented Oct 6, 2022

I just added a consolidation step so that numpy arrays or tensors of images are stacked together if the shapes match, instead of having lists of tensors

Feel free to review @mariosasko :)

Copy link
Collaborator

@mariosasko mariosasko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! A few nits, but other than that looks great.

@lhoestq
Copy link
Member Author

lhoestq commented Oct 7, 2022

I added a few lines in the docs and reverted the ragged numpy array change :)

ready for another review @mariosasko !

Copy link
Collaborator

@mariosasko mariosasko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last nit.

Copy link
Collaborator

@mariosasko mariosasko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks all good now!

@lhoestq lhoestq merged commit 1b935da into main Oct 10, 2022
@lhoestq lhoestq deleted the image-formatting branch October 10, 2022 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants