[LibriSpeech] Fix dev split local_extracted_archive for 'all' config

[sanchit-gandhi]

We define the keys for the _DL_URLS of the dev split as dev.clean and dev.other:

datasets/datasets/librispeech_asr/librispeech_asr.py

Lines 60 to 61 in 2e7142a

	"dev.clean": _DL_URL + "dev-clean.tar.gz",
	"dev.other": _DL_URL + "dev-other.tar.gz",

These keys get forwarded to the dl_manager and thus the local_extracted_archive.

However, when calling SplitGenerator for the dev sets, we query the local_extracted_archive keys validation.clean and validation.other:

datasets/datasets/librispeech_asr/librispeech_asr.py

Line 212 in 2e7142a

"local_extracted_archive": local_extracted_archive.get("validation.clean"),

datasets/datasets/librispeech_asr/librispeech_asr.py

Line 219 in 2e7142a

"local_extracted_archive": local_extracted_archive.get("validation.other"),

The consequence of this is that the local_extracted_archive arg passed to _generate_examples is always None, as the keys validation.clean and validation.other do not exists in the local_extracted_archive.

When defining the audio_file in _generate_examples, since local_extracted_archive is always None, we always omit the local_extracted_archive path from the audio_file path, even if in non-streaming mode:

datasets/datasets/librispeech_asr/librispeech_asr.py

Lines 259 to 263 in 2e7142a

	audio_file = (
	os.path.join(local_extracted_archive, audio_file)
	if local_extracted_archive
	else audio_file
	)

Thus, audio_file will only ever be the streaming path (audio_file, not os.path.join(local_extracted_archive, audio_file)).

This PR fixes the .get() keys for the local_extracted_archive for the dev splits.

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[albertvillanova]

Good catch, @sanchit-gandhi!!!

I'm just mentioning the original PR below, so that both are linked in GitHub.

[albertvillanova]

This PR fixes a bug introduced in:

• [Librispeech] Add 'all' config #4184

[lhoestq]

Good catch !