Add documentaton for dataset README.md files

[PhilipMay]

Hi,
the dataset README files have special headers.
Somehow a documenation of the allowed values and tags is missing.
Could you add that?

Just to give some concrete questions that should be answered imo:

• which values can be passted to multilinguality?
• what should be passed to language_creators?
• which values should licenses have? What do I say when it is a custom license? Should I add a link?
• how should I choose size_categories ? What are valid ranges?
• what are valid task_categories?

Thanks
Philip

[lhoestq]

Hi ! We are using the datasets-tagging app to select the tags to add.

We are also adding the full list of tags in #2107
This covers multilinguality, language_creators, licenses, size_categories and task_categories.

In general if you want to add a tag that doesn't exist (for example for a custom license) you must make it start with other- and then a custom tag name.

edit (@theo-m) if you ever find yourself resorting to adding an other-* tag, please do ping us somewhere so we can think about adding it to the "official" list :)

[PhilipMay]

@lhoestq hmm - ok thanks for the answer.
To be honest I am not sure if this issue can be closed now.
I just wanted to point out that this should either be documented or linked in the documentation.
If you feel like it is (will be) please just close this.

[lhoestq]

We're still working on the validation+documentation in this.
Feel free to keep this issue open till we've added them

[PhilipMay]

@lhoestq what is the status on this? Did you add documentation?

[lhoestq]

Hi ! There's the tagging app at https://huggingface.co/datasets/tagging/ that you can use.
It shows the list of all the tags you can use.

It is based on all the tag sets defined in this folder:
https://github.com/huggingface/datasets/tree/master/src/datasets/utils/resources

[PhilipMay]

@lhoestq is there something like this form Models?

[lhoestq]

I don't think so. Feel free to take a look at the tags of other models (example here). But we should definitely have some docs or an app to write the tags. Feel free to open an issue in the transformers repo or in the huggingface_hub repo so we can discuss this

[mariosasko]

When modifying a README file, the Hub now displays a special UI with allowed values (see https://huggingface.co/docs/datasets/main/en/upload_dataset#create-a-dataset-card).