Skip to content

remote: Add zstd to encoding types#330

Merged
joverlee521 merged 1 commit into
masterfrom
zstd-encoding
Nov 16, 2023
Merged

remote: Add zstd to encoding types#330
joverlee521 merged 1 commit into
masterfrom
zstd-encoding

Conversation

@joverlee521
Copy link
Copy Markdown
Contributor

@joverlee521 joverlee521 commented Nov 16, 2023

Allows us to skip gzip compression when uploading .zst files. Includes adding ".zst" to the mimetypes encoding types dictionary because it is not yet included in the standard library.¹

¹ https://github.com/python/cpython/blob/2ef3676a5bb8fba531fb8237ce50c27ebe37fb96/Lib/mimetypes.py#L411-L417

Checklist

  • Checks pass

Allows us to skip gzip compression when uploading .zst files.
Includes adding ".zst" to the mimetypes encoding types dictionary
because it is not yet included in the standard library.¹

¹ https://github.com/python/cpython/blob/2ef3676a5bb8fba531fb8237ce50c27ebe37fb96/Lib/mimetypes.py#L411-L417
@joverlee521
Copy link
Copy Markdown
Contributor Author

Tested locally with

nextstrain remote upload s3://nextstrain-data/files/dengue/trial/ /tmp/sequences_all.fasta.zst

Seems to have worked as expected

$ curl -I https://data.nextstrain.org/files/dengue/trial/sequences_all.fasta.zst
HTTP/2 200 
content-type: application/zstd

Copy link
Copy Markdown
Contributor

@tsibley tsibley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for adding this.

Comment thread nextstrain/cli/remote/s3.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants