Skip to content

Fix download_mode docstrings #2701

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
albertvillanova merged 3 commits into from
Jul 22, 2021
Merged

Fix download_mode docstrings #2701

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6da78ee
Add missing download_mode to prepare_module docstring
albertvillanova Jul 22, 2021
c6d76fa
Fix cross-reference of GenerateMode in load_metric docstring
albertvillanova Jul 22, 2021
7ba2162
Update download_mode docstring in load_dataset and load_dataset_builder
albertvillanova Jul 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions src/datasets/load.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -249,7 +249,8 @@ def prepare_module(
- it is set to the local version fo the lib.
- it will also try to load it from the master branch if it's not available at the local version fo the lib.
Specifying a version that is different from your local version of the lib might cause compatibility issues.
download_config (Optional ``datasets.DownloadConfig``: specific download configuration parameters.
download_config (:class:`DownloadConfig`, optional): Specific download configuration parameters.
download_mode (:class:`GenerateMode`, default ``REUSE_DATASET_IF_EXISTS``): Download/generate mode.
dataset (bool): True if the script to load is a dataset, False if the script is a metric.
force_local_path (Optional str): Optional path to a local path to download and prepare the script to.
Used to inspect or modify the script folder.
Expand Down Expand Up @@ -594,7 +595,7 @@ def load_metric(
This is useful to compute metrics in distributed setups (in particular non-additive metrics like F1).
keep_in_memory (bool): Whether to store the temporary results in memory (defaults to False)
download_config (Optional ``datasets.DownloadConfig``: specific download configuration parameters.
download_mode (Optional `datasets.GenerateMode`): select the download/generate mode - Default to REUSE_DATASET_IF_EXISTS
download_mode (:class:`GenerateMode`, default ``REUSE_DATASET_IF_EXISTS``): Download/generate mode.
script_version (Optional ``Union[str, datasets.Version]``): if specified, the module will be loaded from the datasets repository
at this version. By default it is set to the local version fo the lib. Specifying a version that is different from
your local version of the lib might cause compatibility issues.
Expand Down Expand Up @@ -658,7 +659,7 @@ def load_dataset_builder(
cache_dir (:obj:`str`, optional): Directory to read/write data. Defaults to "~/datasets".
features (:class:`Features`, optional): Set the features type to use for this dataset.
download_config (:class:`~utils.DownloadConfig`, optional): Specific download configuration parameters.
download_mode (:class:`GenerateMode`, optional): Select the download/generate mode - Default to REUSE_DATASET_IF_EXISTS
download_mode (:class:`GenerateMode`, default ``REUSE_DATASET_IF_EXISTS``): Download/generate mode.
script_version (:class:`~utils.Version` or :obj:`str`, optional): Version of the dataset script to load:

- For canonical datasets in the `huggingface/datasets` library like "squad", the default version of the module is the local version fo the lib.
Expand Down Expand Up @@ -768,7 +769,7 @@ def load_dataset(
cache_dir (:obj:`str`, optional): Directory to read/write data. Defaults to "~/datasets".
features (:class:`Features`, optional): Set the features type to use for this dataset.
download_config (:class:`~utils.DownloadConfig`, optional): Specific download configuration parameters.
download_mode (:class:`GenerateMode`, optional): Select the download/generate mode - Default to REUSE_DATASET_IF_EXISTS
download_mode (:class:`GenerateMode`, default ``REUSE_DATASET_IF_EXISTS``): Download/generate mode.
ignore_verifications (:obj:`bool`, default ``False``): Ignore the verifications of the downloaded/processed dataset information (checksums/size/splits/...).
keep_in_memory (:obj:`bool`, default ``None``): Whether to copy the dataset in-memory. If `None`, the dataset
will not be copied in-memory unless explicitly enabled by setting `datasets.config.IN_MEMORY_MAX_SIZE` to
Expand Down