"File name too long" error for file locks

[gar1t]

Describe the bug

Getting the following error when calling load_dataset("gar1t/test"):

OSError: [Errno 36] File name too long: '<user>/.cache/huggingface/datasets/_home_garrett_.cache_huggingface_datasets_csv_test-7c856aea083a7043_0.0.0_9144e0a4e8435090117cea53e6c7537173ef2304525df4a077c435d8ee7828ff.incomplete.lock'

Steps to reproduce the bug

Where the user cache dir (e.g. ~/.cache) is on a file system that limits filenames to 255 chars (e.g. ext4):

from datasets import load_dataset
load_dataset("gar1t/test")

Expected results

Expect the function to return without an error.

Actual results

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<python_venv>/lib/python3.9/site-packages/datasets/load.py", line 1112, in load_dataset
    builder_instance.download_and_prepare(
  File "<python_venv>/lib/python3.9/site-packages/datasets/builder.py", line 644, in download_and_prepare
    self._save_info()
  File "<python_venv>/lib/python3.9/site-packages/datasets/builder.py", line 765, in _save_info
    with FileLock(lock_path):
  File "<python_venv>/lib/python3.9/site-packages/datasets/utils/filelock.py", line 323, in __enter__
    self.acquire()
  File "<python_venv>/lib/python3.9/site-packages/datasets/utils/filelock.py", line 272, in acquire
    self._acquire()
  File "<python_venv>/lib/python3.9/site-packages/datasets/utils/filelock.py", line 403, in _acquire
    fd = os.open(self._lock_file, open_mode)
OSError: [Errno 36] File name too long: '<user>/.cache/huggingface/datasets/_home_garrett_.cache_huggingface_datasets_csv_test-7c856aea083a7043_0.0.0_9144e0a4e8435090117cea53e6c7537173ef2304525df4a077c435d8ee7828ff.incomplete.lock'

Environment info

• datasets version: 1.12.1
• Platform: Linux-5.11.0-27-generic-x86_64-with-glibc2.31
• Python version: 3.9.7
• PyArrow version: 5.0.0

[lhoestq]

Hi, the filename here is less than 255

>>> len("_home_garrett_.cache_huggingface_datasets_csv_test-7c856aea083a7043_0.0.0_9144e0a4e8435090117cea53e6c7537173ef2304525df4a077c435d8ee7828ff.incomplete.lock")
154

so not sure why it's considered too long for your filesystem.
(also note that the lock files we use always have smaller filenames than 255)

datasets/src/datasets/utils/filelock.py

Line 135 in 5d1a9f1

def __init__(self, lock_file, timeout=-1, max_filename_length=255):

[gar1t]

Yes, you're right! I need to get you more info here. Either there's something going with the name itself that the file system doesn't like (an encoding that blows up the name length??) or perhaps there's something with the path that's causing the entire string to be used as a name. I haven't seen this on any system before and the Internet's not forthcoming with any info.

[lmmx]

Snap, encountered when trying to run this example from PyTorch Lightning Flash:

import torch

import flash
from flash.audio import SpeechRecognition, SpeechRecognitionData
from flash.core.data.utils import download_data

# 1. Create the DataModule
download_data("https://pl-flash-data.s3.amazonaws.com/timit_data.zip", "./data")

datamodule = SpeechRecognitionData.from_json(
    input_fields="file",
    target_fields="text",
    train_file="data/timit/train.json",
    test_file="data/timit/test.json",
)

Gave this traceback:

Traceback (most recent call last):
  File "lf_ft.py", line 10, in <module>
    datamodule = SpeechRecognitionData.from_json(
  File "/home/louis/miniconda3/envs/w2vlf/lib/python3.8/site-packages/flash/core/data/data_module.py", line 1005, in from_json
    return cls.from_data_source(
  File "/home/louis/miniconda3/envs/w2vlf/lib/python3.8/site-packages/flash/core/data/data_module.py", line 571, in from_data_source
    train_dataset, val_dataset, test_dataset, predict_dataset = data_source.to_datasets(
  File "/home/louis/miniconda3/envs/w2vlf/lib/python3.8/site-packages/flash/core/data/data_source.py", line 307, in to_datasets
    train_dataset = self.generate_dataset(train_data, RunningStage.TRAINING)
  File "/home/louis/miniconda3/envs/w2vlf/lib/python3.8/site-packages/flash/core/data/data_source.py", line 344, in generate_dataset
    data = load_data(data, mock_dataset)
  File "/home/louis/miniconda3/envs/w2vlf/lib/python3.8/site-packages/flash/audio/speech_recognition/data.py", line 103, in load_data
    dataset_dict = load_dataset(self.filetype, data_files={stage: str(file)})
  File "/home/louis/miniconda3/envs/w2vlf/lib/python3.8/site-packages/datasets/load.py", line 1599, in load_dataset
    builder_instance = load_dataset_builder(
  File "/home/louis/miniconda3/envs/w2vlf/lib/python3.8/site-packages/datasets/load.py", line 1457, in load_dataset_builder
    builder_instance: DatasetBuilder = builder_cls(
  File "/home/louis/miniconda3/envs/w2vlf/lib/python3.8/site-packages/datasets/builder.py", line 285, in __init__
    with FileLock(lock_path):
  File "/home/louis/miniconda3/envs/w2vlf/lib/python3.8/site-packages/datasets/utils/filelock.py", line 323, in __enter__
    self.acquire()
  File "/home/louis/miniconda3/envs/w2vlf/lib/python3.8/site-packages/datasets/utils/filelock.py", line 272, in acquire
    self._acquire()
  File "/home/louis/miniconda3/envs/w2vlf/lib/python3.8/site-packages/datasets/utils/filelock.py", line 403, in _acquire
    fd = os.open(self._lock_file, open_mode)
OSError: [Errno 36] File name too long: '/home/louis/.cache/huggingface/datasets/_home_louis_.cache_huggingface_datasets_json_default-98e6813a547f72fa_0.0.0_c2d554c3377ea79c7664b93dc65d0803b45e3279000f993c7bfd18937fd7f426.lock'

My home directory is encrypted, therefore the maximum length is 143 (source 1, source 2)

From what I've read I think the error is in reference to the file name (just the final part of the path) which is 145 chars long:

>>> len("_home_louis_.cache_huggingface_datasets_json_default-98e6813a547f72fa_0.0.0_c2d554c3377ea79c7664b93dc65d0803b45e3279000f993c7bfd18937fd7f426.lock")
145

I also have a file in this directory (i.e. whose length is not a problem):

>>> len("_home_louis_.cache_huggingface_datasets_librispeech_asr_clean_2.1.0_468ec03677f46a8714ac6b5b64dba02d246a228d92cbbad7f3dc190fa039eab1.lock")
137

[lmmx]

Perhaps this could be exposed as a config setting so you could change it manually?

datasets/src/datasets/utils/filelock.py

Line 135 in 5d1a9f1

def __init__(self, lock_file, timeout=-1, max_filename_length=255):

Rather than hard-code 255, default it to 255, and allow it to be changed, the same way is done for datasets.config.IN_MEMORY_MAX_SIZE:

datasets/src/datasets/config.py

Lines 171 to 173 in 12b7e13

	# In-memory
	DEFAULT_IN_MEMORY_MAX_SIZE = 0 # Disabled
	IN_MEMORY_MAX_SIZE = float(os.environ.get("HF_DATASETS_IN_MEMORY_MAX_SIZE", DEFAULT_IN_MEMORY_MAX_SIZE))

In fact there already appears to be an existing variable to do so:

datasets/src/datasets/config.py

Line 187 in 12b7e13

MAX_DATASET_CONFIG_ID_READABLE_LENGTH = 255

It's used here:

datasets/src/datasets/builder.py

Lines 163 to 165 in efe89ed

	config_id = self.name + "-" + suffix
	if len(config_id) > config.MAX_DATASET_CONFIG_ID_READABLE_LENGTH:
	config_id = self.name + "-" + Hasher.hash(suffix)

Perhaps it could be set based on a test (trying to create a 255 char length named lock file and seeing if it fails)

[lmmx]

Just fixed it, sending a PR 😄

[mariosasko]

Hi @lmmx @gar1t ,

it would be helpful if you could run the following code and copy-paste the output here:

import datasets
import os
os.statvfs(datasets.config.HF_DATASETS_CACHE)

[lmmx]

os.statvfs_result(f_bsize=4096, f_frsize=4096, f_blocks=240046344, f_bfree=96427610, f_bavail=84216487, f_files=61038592, f_ffree=58216027, f_favail=58216027, f_flag=4102, f_namemax=143)

[mariosasko]

Hi @lmmx,

Thanks for providing the result of the command. I've opened a PR, and it would be great if you could verify that the fix works on your system. To install the version of the datasets with the fix, please run the following command:

pip install git+https://github.com/huggingface/datasets.git@fix-2924

Btw, I saw your PR, and I appreciate your effort. However, my approach is a bit simpler for the end-user, so that's why I decided to fix the issue myself.

[lmmx]

No problem Mario I didn't know that was where that value was recorded so I learnt something 😃 I just wanted to get a local version working, of course you should implement whatever fix is best for HF. Yes can confirm this fixes it too. Thanks!

[shashwat1002]

Hello @mariosasko

Has this fix shown up in the 2.10.1 version of huggingface datasets?