Cannot load timit_asr data set

[bhaddow]

Describe the bug

I am trying to load the timit_asr data set. I have tried with a copy from the LDC, and a copy from deepai. In both cases they fail with a "duplicate key" error. With the LDC version I have to convert the file extensions all to upper-case before I can load it at all.

Steps to reproduce the bug

timit = datasets.load_dataset("timit_asr", data_dir = "/path/to/dataset")
# Sample code to reproduce the bug

Expected results

The data set should load without error. It worked for me before the LDC url change.

Actual results

datasets.keyhash.DuplicatedKeysError: FAILURE TO GENERATE DATASET !
Found duplicate Key: SA1
Keys should be unique and deterministic in nature

Environment info

• datasets version:
• datasets version: 2.2.2
• Platform: Linux-5.4.0-90-generic-x86_64-with-glibc2.17
• Python version: 3.8.12
• PyArrow version: 8.0.0
• Pandas version: 1.4.2

[albertvillanova]

Thanks for reporting, @bhaddow.

I'm fixing it.

[bhaddow]

Thanks for the quick fix!

[albertvillanova]

@bhaddow we have also made a fix so that you don't have to convert to uppercase the file extensions of the LDC data.

Would you mind checking if it works OK now for you and reporting if there are any issues? Thanks.

[bhaddow]

Hi @albertvillanova -It loads fine on a copy of the data from deepai - although I have to remove the copies of the .WAV files (with extension .WAV,wav). On a copy of the data that was obtained from the LDC, the glob still fails to find the files. The LDC copy looks like it was copied from CD, in 2004, so the structure may be different to a current download.

[bhaddow]

Ah, if I change the train/ and test/ directories to TRAIN/ and TEST/ then it works!

[albertvillanova]

Thanks for your investigation and report, @bhaddow. I'm adding another fix for the TRAIN/train and TEST/test directory names.