Skip to content

[Merged by Bors] - Add commmand for pruning states#4835

Closed
jimmygchen wants to merge 19 commits intosigp:unstablefrom
jimmygchen:prune-states
Closed

[Merged by Bors] - Add commmand for pruning states#4835
jimmygchen wants to merge 19 commits intosigp:unstablefrom
jimmygchen:prune-states

Conversation

@jimmygchen
Copy link
Copy Markdown
Member

@jimmygchen jimmygchen commented Oct 12, 2023

Issue Addressed

Closes #4481.

(Continuation of #4648)

Proposed Changes

  • Add lighthouse db prune-states
  • Make it work
  • Ensure block roots are handled correctly (to be addressed in 4735)
  • Check perf on mainnet/Goerli/Gnosis (takes a few seconds max)
  • Run block root healing logic ([Merged by Bors] - Add block roots heal logic in v18 schema migration. #4875 ) at the beginning
  • Add some tests
  • Update docs
  • Add --freezer flag and other improvements to lighthouse db inspect

@jimmygchen jimmygchen changed the base branch from stable to unstable October 12, 2023 04:42
@jimmygchen jimmygchen mentioned this pull request Oct 12, 2023
7 tasks
bors bot pushed a commit that referenced this pull request Oct 25, 2023
## Issue Addressed

Fixes #4697. 

This also unblocks the state pruning PR (#4835). Because self healing breaks if state pruning is applied to a database with missing block roots.

## Proposed Changes

- Fill in the missing block roots between last restore point slot and split slot when upgrading to latest database version.
# Conflicts:
#	beacon_node/beacon_chain/tests/store_tests.rs
#	beacon_node/store/src/hot_cold_store.rs
#	beacon_node/store/src/lib.rs
#	database_manager/src/lib.rs
# Conflicts:
#	beacon_node/store/src/lib.rs
#	beacon_node/store/src/memory_store.rs
#	database_manager/src/lib.rs
@jimmygchen jimmygchen added ready-for-review The code is ready for review and removed work-in-progress PR is a work-in-progress labels Nov 1, 2023
@michaelsproul michaelsproul added the backwards-incompat Backwards-incompatible API change label Nov 2, 2023
Copy link
Copy Markdown
Member

@michaelsproul michaelsproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge!

@michaelsproul michaelsproul added ready-for-merge This PR is ready to merge. and removed backwards-incompat Backwards-incompatible API change labels Nov 3, 2023
@michaelsproul
Copy link
Copy Markdown
Member

I added and removed a backwards-incompat label because I thought we'd broken the prune_payloads command. But the alias exists, so it's fine.

bors r+

bors bot pushed a commit that referenced this pull request Nov 3, 2023
## Issue Addressed

Closes #4481. 

(Continuation of #4648)

## Proposed Changes

- [x] Add `lighthouse db prune-states`
- [x] Make it work
- [x] Ensure block roots are handled correctly (to be addressed in 4735)
- [x] Check perf on mainnet/Goerli/Gnosis (takes a few seconds max)
- [x] Run block root healing logic (#4875 ) at the beginning
- [x] Add some tests
- [x] Update docs
- [x] Add `--freezer` flag and other improvements to `lighthouse db inspect`

Co-authored-by: Michael Sproul <[email protected]>
Co-authored-by: Jimmy Chen <[email protected]>
Co-authored-by: Michael Sproul <[email protected]>
@jimmygchen jimmygchen removed the ready-for-review The code is ready for review label Nov 3, 2023
@bors
Copy link
Copy Markdown

bors bot commented Nov 3, 2023

Pull request successfully merged into unstable.

Build succeeded!

The publicly hosted instance of bors-ng is deprecated and will go away soon.

If you want to self-host your own instance, instructions are here.
For more help, visit the forum.

If you want to switch to GitHub's built-in merge queue, visit their help page.

@bors bors bot changed the title Add commmand for pruning states [Merged by Bors] - Add commmand for pruning states Nov 3, 2023
@bors bors bot closed this Nov 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

database ready-for-merge This PR is ready to merge. v4.6.0 ETA Q1 2024

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add logic to prune all historic states

2 participants