This repository was archived by the owner on Jun 30, 2022. It is now read-only.
Update tagging_app.py #11
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add more size categories in the tagging app.
lhoestq
reviewed
Mar 22, 2021
Contributor
Author
|
@lhoestq 🙈 I missed it. |
Member
|
cc @yjernite can you take a look at these changes ? |
Member
|
Looks good to me, although I'm not sure how many datasets we have that fall in any of these categories (the size here is the total number of rows in the table in a single config) @gchhablani did you have specific datasets in mind? |
Contributor
Author
|
Hi @yjernite, In the existing datasets, there are some above a billion, some above 10 million, I just thought it would be better for such datasets. You can check the PR I made on datasets, a lot of the datasets(configs) are between 1M and 10M and some above 10M, where it was previous labelled as "n>1M". I don't have anything specific in mind for 1T, but I think we can have it there, just in case something comes along. Up to you and @lhoestq to decide :) I'll change everything back to "n>1M" if you'd like on the PR. |
theo-m
pushed a commit
to huggingface/datasets
that referenced
this pull request
Mar 24, 2021
SBrandeis
added a commit
to huggingface/datasets
that referenced
this pull request
Apr 26, 2021
* basic validation * ci script and test change * color is better * check all option * validate size cats & multiling, point to reference file urls on error * add validation to ci and rename files * spurrious change to trigger CI * add qa reqs * disallow empty lists * better error msg: show all invalid values rather than first one * some code shuffling & better error msg for langcodes * add pyyaml to qa reqs * fix package file loading * include json resources * reflect changes to size cats from huggingface/datasets-tagging#11 * trying another format for package_data * ci works! fixing the readme like a good citizen 🤗 * escape validation everywhere it's allowed in the tagging app * code review: more json files, conditional import * pointers to integrate readme metadata in class (wip) * no pydantic * fix docs? * Revert "fix docs?" This reverts commit ab82a6c. * remove pointers to add readme to loader * Get rid of langcodes, some refactor * Update languages.json * Refactor, add tests * I said, tests!! Co-authored-by: theo <[email protected]> Co-authored-by: SBrandeis <[email protected]>
lvwerra
pushed a commit
to huggingface/evaluate
that referenced
this pull request
Mar 31, 2022
* basic validation * ci script and test change * color is better * check all option * validate size cats & multiling, point to reference file urls on error * add validation to ci and rename files * spurrious change to trigger CI * add qa reqs * disallow empty lists * better error msg: show all invalid values rather than first one * some code shuffling & better error msg for langcodes * add pyyaml to qa reqs * fix package file loading * include json resources * reflect changes to size cats from huggingface/datasets-tagging#11 * trying another format for package_data * ci works! fixing the readme like a good citizen 🤗 * escape validation everywhere it's allowed in the tagging app * code review: more json files, conditional import * pointers to integrate readme metadata in class (wip) * no pydantic * fix docs? * Revert "fix docs?" This reverts commit ab82a6cbb1dd5fbc7f0ea70e98156d7419c54bf1. * remove pointers to add readme to loader * Get rid of langcodes, some refactor * Update languages.json * Refactor, add tests * I said, tests!! Co-authored-by: theo <[email protected]> Co-authored-by: SBrandeis <[email protected]>
lvwerra
pushed a commit
to huggingface/evaluate
that referenced
this pull request
Mar 31, 2022
* basic validation * ci script and test change * color is better * check all option * validate size cats & multiling, point to reference file urls on error * add validation to ci and rename files * spurrious change to trigger CI * add qa reqs * disallow empty lists * better error msg: show all invalid values rather than first one * some code shuffling & better error msg for langcodes * add pyyaml to qa reqs * fix package file loading * include json resources * reflect changes to size cats from huggingface/datasets-tagging#11 * trying another format for package_data * ci works! fixing the readme like a good citizen 🤗 * escape validation everywhere it's allowed in the tagging app * code review: more json files, conditional import * pointers to integrate readme metadata in class (wip) * no pydantic * fix docs? * Revert "fix docs?" This reverts commit ab82a6cbb1dd5fbc7f0ea70e98156d7419c54bf1. * remove pointers to add readme to loader * Get rid of langcodes, some refactor * Update languages.json * Refactor, add tests * I said, tests!! Co-authored-by: theo <[email protected]> Co-authored-by: SBrandeis <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add more size categories in the tagging app.