Support remote data files

[albertvillanova]

Add support for (streaming) remote data files:

data_files = f"https://huggingface.co/datasets/{repo_id}/resolve/main/{relative_file_path}"
ds = load_dataset("json", split="train", data_files=data_files, streaming=True)

cc: @thomwolf

[albertvillanova]

@lhoestq maybe we could also use (if available) the ETag of the remote file in create_config_id?

[lhoestq]

@lhoestq maybe we could also use (if available) the ETag of the remote file in create_config_id?

Sure ! We can get the ETag with

headers = get_authentication_headers_for_url(url, use_auth_token=use_auth_token)  # auth for private repos
etag = http_head(url, headers=headers).headers.get("ETag")

Since the computation of the config_id is done in the DatasetBuilder.__init__, then this means that we need to add a new parameter use_auth_token in DatasetBuilder.__init__

Does that sound good ? We can add this in a following PR