feat(gatsby-source-filesystem): Only generate hashes when a file has changed, and add an option for skipping hashing #37464
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…her packages that use this, add tests
Update feature branch with latest from master
gatsbot
bot
added
the
status: triage needed
LekoArts
added
topic: source-plugins
LekoArts
suggested changes
Jan 16, 2023
Contributor
LekoArts
left a comment
LekoArts
left a comment
There was a problem hiding this comment.
Thanks for the PR! I'll change the README a little bit more later, so code wise I only have some small requests :)
packages/gatsby-source-filesystem/src/__tests__/create-file-node.js
Outdated
Show resolved
Hide resolved
packages/gatsby-source-filesystem/src/__tests__/create-file-node.js
Outdated
Show resolved
Hide resolved
…to fastHash setting
latest from master
LekoArts
approved these changes
Jan 17, 2023
Contributor
LekoArts
left a comment
LekoArts
left a comment
There was a problem hiding this comment.
Thank you! A lot of people will benefit from that ❤️
This was referenced Jan 24, 2023
This was referenced Jan 25, 2023
This was referenced Jan 25, 2023
mwfrost
pushed a commit
to mwfrost/gatsby
that referenced
this pull request
Apr 20, 2023
…changed, and add an option for skipping hashing (gatsbyjs#37464) Co-authored-by: LekoArts <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Generating an MD5 hash in
gatsby-source-filesystemis quite an expensive operation for large files, and can increase build times significantly, as well as introduce other issues with open file handles.To mitigate this:
Documentation
I've added a section to the README explaining the new option. Not sure if what I wrote goes into sufficient detail (or too much detail) though.
Related Issues
Addresses #12011
Fixes #37425