Fix download_demo for data.zip files
#2699
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## feature-branch-download-demo #2699 +/- ##
================================================================
- Coverage 98.16% 96.88% -1.29%
================================================================
Files 74 74
Lines 7896 7923 +27
================================================================
- Hits 7751 7676 -75
- Misses 145 247 +102
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
This Pull Request is not linked to an issue. To ensure our community is able to accurately track resolved issues, please link any issue that will be closed by this PR! |
fealho
force-pushed
the
issue-2687-download-demo-output-folder
branch
from
c3bdfa2 to
62ee13b
Compare
fealho
force-pushed
the
issue-2688-download-demo-zip
branch
2 times, most recently
from
32da919 to
74d3cc5
Compare
fealho
force-pushed
the
issue-2688-download-demo-zip
branch
from
74d3cc5 to
af00578
Compare
|
This Pull Request is not linked to an issue. To ensure our community is able to accurately track resolved issues, please link any issue that will be closed by this PR! |
amontanez24
left a comment
amontanez24
left a comment
There was a problem hiding this comment.
Can we make latin-1 a constant and explain why that encoding is special
| try: | ||
| data[table_name] = pd.read_csv(io.BytesIO(file_), low_memory=False) | ||
| except UnicodeDecodeError: | ||
| data[table_name] = pd.read_csv(io.BytesIO(file_), low_memory=False, encoding='latin-1') | ||
| except Exception as e: | ||
| skipped_files.append(f'{filename}: {e}') |
There was a problem hiding this comment.
This and the previous bit of reading seems very similar, could we move it to its own function like read data ?
There was a problem hiding this comment.
That's actually tricky. One approach for example would be to substitute lines 241-244 with:
def _read_csv_with_fallback(filepath_or_buffer, **kwargs):
"""Read a CSV with a fallback encoding on UnicodeDecodeError."""
try:
return pd.read_csv(filepath_or_buffer, **kwargs)
except UnicodeDecodeError:
kwargs = {**kwargs, 'encoding': FALLBACK_ENCODING}
return pd.read_csv(filepath_or_buffer, **kwargs)But this doesn't work because you need to rewind the BytesIO for the second call, and implementing that into this function makes it confusing.
Did you have some other approach in mind?
There was a problem hiding this comment.
Yeah I had that in mind but I see why this won't work. It's okay to go ahead with your implementation. I think that if we did anything it would just be 'overcomplicating' a simple process.
|
This Pull Request is not linked to an issue. To ensure our community is able to accurately track resolved issues, please link any issue that will be closed by this PR! |
|
This Pull Request is not linked to an issue. To ensure our community is able to accurately track resolved issues, please link any issue that will be closed by this PR! |
|
This Pull Request is not linked to an issue. To ensure our community is able to accurately track resolved issues, please link any issue that will be closed by this PR! |
CU-86b6xp7a0, Resolve #2688
CU-86b6xrcah, Resolve #2690