Skip to content

feat: add config iavl-lazy-loading to enable lazy loading of iavl store#14189

Merged
tac0turtle merged 12 commits into
cosmos:mainfrom
yihuang:iavl-lazy-loading
Jan 31, 2023
Merged

feat: add config iavl-lazy-loading to enable lazy loading of iavl store#14189
tac0turtle merged 12 commits into
cosmos:mainfrom
yihuang:iavl-lazy-loading

Conversation

@yihuang
Copy link
Copy Markdown
Collaborator

@yihuang yihuang commented Dec 7, 2022 •
edited
Loading

Description

It improve start time of archive nodes (from dozens of minutes to seconds).

It seems to be running alright on production node, but not sure of unknown side effects.

related: cosmos/iavl#638

Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

@yihuang yihuang requested a review from a team December 7, 2022 09:32
@yihuang yihuang marked this pull request as draft December 7, 2022 09:33
yihuang
yihuang commented Dec 7, 2022
View reviewed changes
Comment thread CHANGELOG.md Outdated
@yihuang yihuang force-pushed the iavl-lazy-loading branch 2 times, most recently from d14d643 to cdcfc1d Compare December 7, 2022 09:46
@yihuang yihuang mentioned this pull request Dec 7, 2022
Merged
@yihuang yihuang changed the title feat: add iavl-lazy-loading to enable lazy loading of iavl store feat: add config iavl-lazy-loading to enable lazy loading of iavl store Dec 8, 2022
@yihuang yihuang mentioned this pull request Dec 8, 2022
Merged
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jan 8, 2023

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Jan 8, 2023
@yihuang
Copy link
Copy Markdown
Collaborator Author

yihuang commented Jan 8, 2023

unstall

@alexanderbez alexanderbez removed the stale label Jan 8, 2023
feat: add iavl-lazy-loading to enable lazy loading of iavl store
cc84e97 
improve start time of archive nodes.

Update CHANGELOG.md

fix test
@tac0turtle
Copy link
Copy Markdown
Contributor

tac0turtle commented Jan 30, 2023

@yihuang want to update this to alpha.2?

@yihuang
Copy link
Copy Markdown
Collaborator Author

yihuang commented Jan 31, 2023 •
edited
Loading

@yihuang want to update this to alpha.2?

done, added a bunch of replacements to make it build though, not sure what's the best way to do it, do I need to make a separate PR for store package?

@yihuang yihuang marked this pull request as ready for review January 31, 2023 00:56
@ghost ghost requested review from a team, amaury1093 and kocubinski and removed request for a team January 31, 2023 00:56
@tac0turtle tac0turtle enabled auto-merge (squash) January 31, 2023 21:08
@tac0turtle tac0turtle merged commit b266522 into cosmos:main Jan 31, 2023
@yihuang
Copy link
Copy Markdown
Collaborator Author

yihuang commented Feb 1, 2023 •
edited
Loading

can we backport to 0.47 and 0.46, it's important UX improvement for existing chains. @tac0turtle

@yihuang yihuang mentioned this pull request Feb 1, 2023
Closed
@tac0turtle
Copy link
Copy Markdown
Contributor

tac0turtle commented Feb 1, 2023

@Mergifyio backport release/v0.46.x

@tac0turtle
Copy link
Copy Markdown
Contributor

tac0turtle commented Feb 1, 2023

@Mergifyio backport release/v0.47.x

@tac0turtle
Copy link
Copy Markdown
Contributor

tac0turtle commented Feb 1, 2023

heads up these prs will need to be reduced to 0.19.5

@mergify mergify Bot mentioned this pull request Feb 1, 2023
Merged
@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Feb 1, 2023

backport release/v0.46.x

✅ Backports have been created

Details

@mergify mergify Bot mentioned this pull request Feb 1, 2023
Merged
mergify Bot pushed a commit that referenced this pull request Feb 1, 2023
@mergify
feat: add config iavl-lazy-loading to enable lazy loading of iavl s…
c71dc1c 
…tore (#14189)

Co-authored-by: Marko <marbar3778@yahoo.com>
(cherry picked from commit b266522)

# Conflicts:
#	CHANGELOG.md
#	go.mod
#	go.sum
#	simapp/go.mod
#	simapp/go.sum
#	store/go.mod
#	store/go.sum
#	tests/go.mod
#	tests/go.sum
#	x/nft/go.mod
#	x/nft/go.sum
#	x/upgrade/go.mod
#	x/upgrade/go.sum
@mergify
Copy link
Copy Markdown
Contributor

mergify Bot commented Feb 1, 2023

backport release/v0.47.x

✅ Backports have been created

Details

yihuang pushed a commit to yihuang/cosmos-sdk that referenced this pull request Feb 1, 2023
feat: add config iavl-lazy-loading to enable lazy loading of iavl s…
74bd79b 
…tore (cosmos#14189)

Co-authored-by: Marko <marbar3778@yahoo.com>
(cherry picked from commit b266522)

# Conflicts:
#	CHANGELOG.md
#	cosmovisor/go.sum
#	go.mod
#	go.sum
#	server/util.go
#	simapp/go.mod
#	simapp/go.sum
#	store/go.mod
#	store/go.sum
#	tests/go.mod
#	tests/go.sum
#	x/nft/go.mod
#	x/upgrade/go.mod
#	x/upgrade/go.sum
julienrbrt pushed a commit that referenced this pull request Feb 1, 2023
@mergify @tac0turtle
feat: add config iavl-lazy-loading to enable lazy loading of iavl s…
3a1c797 
…tore (backport #14189) (#14863)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
tac0turtle added a commit that referenced this pull request Feb 2, 2023
@mergify @tac0turtle
feat: add config iavl-lazy-loading to enable lazy loading of iavl s…
70589eb 
…tore (backport #14189) (#14862)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
@fedekunze fedekunze mentioned this pull request Feb 9, 2023
Merged
19 tasks
SpicyLemon added a commit to provenance-io/cosmos-sdk that referenced this pull request Feb 16, 2023
@SpicyLemon @mergify
@likhita-809 @julienrbrt @tac0turtle @mmsqe @robert-zaremba @elias-orijtech @facundomedica
Backport: #505 (v0.46.9) #517 (v0.46.10) and #510 (sanction Tx CLI) (#…
cf6f99f 
…518)

* fix: Fix Evidence CLI query flag parsing (backport cosmos#13458) (cosmos#14799)

Co-authored-by: Likhita Polavarapu <78951027+likhita-809@users.noreply.github.com>
Co-authored-by: HuangYi <huang@crypto.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: nested iterator on cache store (cosmos#14798)

* chore: update IAVL  (backport cosmos#14830) (cosmos#14835)

Co-authored-by: Marko <marbar3778@yahoo.com>

* Mark v0.46.8-pio-2 in the changelog and release notes.

* feat: add config `iavl-lazy-loading` to enable lazy loading of iavl store (backport cosmos#14189) (cosmos#14862)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>

* chore: bump IAVL to v0.19.5 (backport cosmos#14889) (cosmos#14892)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* build(deps): bump tendermint to v0.34.25 (cosmos#14903)

* build(deps): bump tendermint to v0.34.26 (cosmos#14911)

* fix: return named error for write validators (backport cosmos#14919) (cosmos#14921)

Co-authored-by: mmsqe <mavis@crypto.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* Bump Tendermint to v0.34.25 (from v0.34.24). (#498)

* Fix a few listener proto comments. (#499)

* build(deps): bump btcd in v0.46 (cosmos#14846)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: Elias Naur <103319121+elias-orijtech@users.noreply.github.com>
Co-authored-by: Marko <marbar3778@yahoo.com>

* ci: update release slack channel and algolia api key (backport cosmos#14942) (cosmos#14945)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: support profiling block replay during abci handshake (backport cosmos#14953) (cosmos#14964)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* Revert "Revert cosmos#14349 - cache iterator optimizations (#444)"

This reverts commit cd9d192.

* Mark v0.46.8-pio-3 in the changelog and release notes.

* fix: avoid broadcasting tx through query (backport cosmos#15044) (cosmos#15047)

Co-authored-by: Facundo Medica <14063057+facundomedica@users.noreply.github.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* Add CLI commands for sanction Tx gov props (#510)

* Fix the changelog.

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Likhita Polavarapu <78951027+likhita-809@users.noreply.github.com>
Co-authored-by: HuangYi <huang@crypto.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: Marko <marbar3778@yahoo.com>
Co-authored-by: mmsqe <mavis@crypto.com>
Co-authored-by: Robert Zaremba <robert@zaremba.ch>
Co-authored-by: Elias Naur <103319121+elias-orijtech@users.noreply.github.com>
Co-authored-by: Facundo Medica <14063057+facundomedica@users.noreply.github.com>
ankurdotb pushed a commit to cheqd/cosmos-sdk that referenced this pull request Feb 28, 2023
@mergify @tac0turtle @ankurdotb
feat: add config iavl-lazy-loading to enable lazy loading of iavl s…
6c4dd40 
…tore (backport cosmos#14189) (cosmos#14862)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
@DracoLi DracoLi mentioned this pull request Mar 13, 2023
Merged
tsenart pushed a commit to meka-dev/cosmos-sdk that referenced this pull request Apr 12, 2023
@tac0turtle @tsenart
feat: add config iavl-lazy-loading to enable lazy loading of iavl s…
cb9dd65 
…tore (cosmos#14189)

Co-authored-by: Marko <marbar3778@yahoo.com>
larry0x pushed a commit to larry0x/cosmos-sdk that referenced this pull request May 22, 2023
@tac0turtle
feat: add config iavl-lazy-loading to enable lazy loading of iavl s…
3d37ad2 
…tore (cosmos#14189)

Co-authored-by: Marko <marbar3778@yahoo.com>
@LampardNguyen234 LampardNguyen234 mentioned this pull request Aug 15, 2023
Merged
@0xstepit 0xstepit mentioned this pull request Jul 1, 2024
Merged
JeancarloBarrios pushed a commit to agoric-labs/cosmos-sdk that referenced this pull request Sep 28, 2024
@mergify @tac0turtle @JeancarloBarrios
feat: add config iavl-lazy-loading to enable lazy loading of iavl s…
3bf9383 
…tore (backport cosmos#14189) (cosmos#14862)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kocubinski kocubinski Awaiting requested review from kocubinski

@amaury1093 amaury1093 Awaiting requested review from amaury1093

2 more reviewers

@alexanderbez alexanderbez alexanderbez approved these changes

@tac0turtle tac0turtle tac0turtle approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

C:Store C:x/upgrade

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@yihuang @tac0turtle @alexanderbez