Skip to content

fix(gatsby-source-filesystem): fix broken stream with gzipped files - #28913

Merged
sidharthachatterjee merged 5 commits into
gatsbyjs:masterfrom
wardpeet:fix/source-encoding
Jan 8, 2021
Merged

fix(gatsby-source-filesystem): fix broken stream with gzipped files#28913
sidharthachatterjee merged 5 commits into
gatsbyjs:masterfrom
wardpeet:fix/source-encoding

Conversation

@wardpeet

@wardpeet wardpeet commented Jan 7, 2021

Copy link
Copy Markdown
Contributor

Description

When creating #28547 we didn't think of gzip, brotli compression when doing remote file downloads. SVGs for example have this issue. Costco reported this issue. I did a fix and created some integration tests to see that it actually works.

@gatsbot gatsbot Bot added the status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer label Jan 7, 2021
@wardpeet wardpeet added status: needs core review Currently awaiting review from Core team member topic: source-plugins Relates to the Gatsby source plugins (e.g. -filesystem) and removed status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer labels Jan 7, 2021

@sidharthachatterjee sidharthachatterjee left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love those integration tests! And the check at

responseStream.on(`downloadProgress`, progress => {
      if (progress.transferred === progress.total || progress.total === null)

is a lot less brittle. Thank you so much for this fix!

@sidharthachatterjee
sidharthachatterjee merged commit a8b516f into gatsbyjs:master Jan 8, 2021
@sidharthachatterjee sidharthachatterjee removed the status: needs core review Currently awaiting review from Core team member label Jan 8, 2021
@wardpeet
wardpeet deleted the fix/source-encoding branch January 8, 2021 14:02
Comment thread yarn.lock
@vladar

vladar commented Jan 16, 2021

Copy link
Copy Markdown
Contributor

Published in gatsby-source-filesystem@2.9.1

@ertrzyiks

Copy link
Copy Markdown

Is this fix available in @3.0.0 ? I had some issues with remote images and downgrading to 2.x helped.

@wardpeet

Copy link
Copy Markdown
Contributor Author

@ertrzyiks it should be, can you open an issue and a reproduction? 🙏

@ertrzyiks

Copy link
Copy Markdown

@wardpeet thanks for the quick answer, I will try to reproduce it in a simple setup

@rroslaniec

Copy link
Copy Markdown

@ertrzyiks it is related I think with my comment: #28983 (comment)

@ahmadkhalaf1

ahmadkhalaf1 commented Dec 9, 2021

Copy link
Copy Markdown

Published in gatsby-source-filesystem@2.9.1

Hello @vladar

seems like this fix was not merged into the next releases
trying gatsby-source-filesystem@2.9.1 works tottaly fine

however on the same code and image trying it with version 3.0.0 or 3.14.0 , or even version 4 it shows the error .
is it possible to check this and merge it to the latest release for gatsby 3 please?

Thanks

@ahmadkhalaf1

Copy link
Copy Markdown

@wardpeet
can you please check if the fix that was published into gatsby-source-filesystem@2.9.1 is released to the latest version ?
it does not work with Gatsby 3 or Gatsby 4
only when i set the version to 2.9.1 it work fine.
please answer me if u can
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic: source-plugins Relates to the Gatsby source plugins (e.g. -filesystem)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants