Skip to content

Reset file pointer to 0 when reading file stream #7304

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dcherian merged 8 commits into from
Dec 1, 2022
Merged

Reset file pointer to 0 when reading file stream #7304

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
4283b95
Reset file pointer to 0 when reading file stream
weiji14 Nov 21, 2022
68989c3
Expect warning rather than ValueError for test_open_twice
weiji14 Nov 21, 2022
befda31
Remove checks for errors raised in test_open_fileobj
weiji14 Nov 21, 2022
6f6fbe6
Fix typo form -> from
weiji14 Nov 21, 2022
93711b7
Add changelog entry for bugfix
weiji14 Nov 21, 2022
d2e84d2
Merge branch 'main' into fix_s3_start_byte
weiji14 Nov 29, 2022
929cb62
Remove warning about resetting file pointer to zero
weiji14 Nov 29, 2022
4e6e59c
Merge branch 'main' into fix_s3_start_byte
weiji14 Nov 29, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion xarray/core/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -654,9 +654,11 @@ def read_magic_number_from_file(filename_or_obj, count=8) -> bytes:
magic_number = filename_or_obj[:count]
elif isinstance(filename_or_obj, io.IOBase):
if filename_or_obj.tell() != 0:
raise ValueError(
filename_or_obj.seek(0)
warnings.warn(
"cannot guess the engine, "
"file-like object read/write pointer not at the start of the file, "
"so resetting file pointer to zero. If this does not work, "
"please close and reopen, or use a context manager"
)
Copy link
Copy Markdown
Contributor Author

@weiji14 weiji14 Nov 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is a warning actually necessary here, or can we remove the warning and let the file pointer reset to zero silently?

Edit: warning has been removed in 929cb62

magic_number = filename_or_obj.read(count)
Expand Down
12 changes: 2 additions & 10 deletions xarray/tests/test_backends.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3031,7 +3031,7 @@ def test_open_badbytes(self) -> None:
def test_open_twice(self) -> None:
expected = create_test_data()
expected.attrs["foo"] = "bar"
with pytest.raises(ValueError, match=r"read/write pointer not at the start"):
with pytest.warns(match=r"read/write pointer not at the start"):
with create_tmp_file() as tmp_file:
expected.to_netcdf(tmp_file, engine="h5netcdf")
with open(tmp_file, "rb") as f:
Expand Down Expand Up @@ -3069,15 +3069,7 @@ def test_open_fileobj(self) -> None:
# `raises_regex`?). Ref https://github.com/pydata/xarray/pull/5191
with open(tmp_file, "rb") as f:
f.seek(8)
with pytest.raises(
ValueError,
match="match in any of xarray's currently installed IO",
):
with pytest.warns(
RuntimeWarning,
match=re.escape("'h5netcdf' fails while guessing"),
):
open_dataset(f)
open_dataset(f)


@requires_h5netcdf
Expand Down