Skip to content

Lodestar devnet 3 #52

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dgcoffman merged 29 commits into from
Nov 11, 2022
Merged

Lodestar devnet 3 #52

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
6c6c56d
Add historical sync test (#44)
Inphi Oct 28, 2022
9b27230
Add Lodestar
dgcoffman Oct 18, 2022
5fb61e5
Supply chain config
dgcoffman Oct 18, 2022
e5e0351
Cache all docker work before yarn build
dgcoffman Oct 18, 2022
48fe02f
Build packages individually to allow them to cache
dgcoffman Oct 18, 2022
042c4eb
We must also build the CLI
dgcoffman Oct 18, 2022
5fe616d
Fix clean
dgcoffman Oct 18, 2022
ebccdf8
Update to head of Lodestar
dgcoffman Oct 20, 2022
f8f5d42
Fun through Capella
dgcoffman Oct 28, 2022
2b5e083
Latest Lodestar
dgcoffman Oct 28, 2022
75c78da
Remove nonfunctional setting
dgcoffman Oct 29, 2022
ea6482c
Bump Lodestar
dgcoffman Oct 29, 2022
b4f19b8
Update lodestar submodule. Copy trusted setup
dgcoffman Nov 8, 2022
b3c004d
Update test harness to start either prysm or lodestar
dgcoffman Nov 9, 2022
05d7355
Fix the Prysm communication
dgcoffman Nov 9, 2022
7da4527
Fix the Lodestar wiring
dgcoffman Nov 9, 2022
109b469
Fix up validator <-> beacon node communication
dgcoffman Nov 9, 2022
d39f2e9
Start switching tests to real beacon API
dgcoffman Nov 10, 2022
ff52ace
fee-market tests pass against Prysm without gRPC
dgcoffman Nov 10, 2022
9bf202f
Fix initial-sync test
dgcoffman Nov 10, 2022
0b427c7
Switch prysm to my branch which has GetBlockJSON
dgcoffman Nov 10, 2022
4eaf186
Configure Prysm to upgrade later (as though it had Capella). Also run…
dgcoffman Nov 10, 2022
ad8a2ee
Fix make lodestar-up
dgcoffman Nov 10, 2022
c9a1883
Merge devnet-v3
dgcoffman Nov 10, 2022
8fe2d40
Remove Prysm-specific chain config. We used a shared config now.
dgcoffman Nov 10, 2022
002b39a
Point Prysm back at the devnet-v3 branch
dgcoffman Nov 10, 2022
9573bb2
Update geth to latest from devnet-v3
dgcoffman Nov 10, 2022
44b2b3f
Put erigon back
dgcoffman Nov 10, 2022
4a6ee67
Remove preset base, let it take the default
dgcoffman Nov 11, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,12 @@ jobs:
# go 1.19 is incompatible with the github.com/lucas-clemente/quic-go v0.27.2 dependency
go-version: '1.18'

- name: Run pre-EIP4844 tests
- name: Prysm - Run pre-EIP4844 tests
run: go run ./tests/pre-4844

- name: Lodestar - Run pre-EIP4844 tests
run: go run ./tests/pre-4844 lodestar

# TODO: A bit redundant. combine this test with fee-market
- name: Run Blob transaction tests
run: go run ./tests/blobtx
Expand Down
4 changes: 4 additions & 0 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@
path = prysm/prysm
url = https://github.com/Inphi/prysm
branch = devnet-v3
[submodule "lodestar/lodestar"]
path = lodestar/lodestar
url = https://github.com/dgcoffman/lodestar
branch = dgc/eip-4844
[submodule "erigon/erigon"]
path = erigon/erigon
url = https://github.com/roberto-bayardo/erigon
Expand Down
17 changes: 12 additions & 5 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,26 @@ devnet-up:
docker compose up -d\
execution-node\
execution-node-2\
beacon-node\
beacon-node-follower\
validator-node\
prysm-beacon-node\
prysm-beacon-node-follower\
prysm-validator-node\
jaeger-tracing

lodestar-up:
docker compose up -d\
execution-node\
execution-node-2\
lodestar-beacon-node\
lodestar-beacon-node-follower\

devnet-down:
docker compose down -v

devnet-restart: devnet-down devnet-up

devnet-clean:
docker compose down
docker image ls 'eip4844-interop*' --format='{{.Repository}}' | xargs -r docker rmi
docker volume ls --filter name=eip4844-interop --format='{{.Name}}' | xargs -r docker volume rm
docker image ls 'interop*' --format='{{.Repository}}' | xargs -r docker rmi
docker volume ls --filter name=interop --format='{{.Name}}' | xargs -r docker volume rm

.PHONY: devnet-clean
107 changes: 79 additions & 28 deletions docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ services:
context: ./geth
dockerfile: Dockerfile.geth
ports:
- "8545:8545"
- '8545:8545'
environment:
GETH_VERBOSITY: 4
volumes:
- "geth_data:/db"
- 'geth_data:/db'
- type: bind
source: ./geth/geth-genesis.json
target: /genesis.json
Expand All @@ -31,14 +31,14 @@ services:
PEER: execution-node:30303
GETH_VERBOSITY: 4
ports:
- "8546:8545"
- '8546:8545'
volumes:
- "geth_data2:/db"
- 'geth_data2:/db'
- type: bind
source: ./geth/geth-genesis.json
target: /genesis.json

beacon-node:
prysm-beacon-node:
depends_on:
- execution-node
- jaeger-tracing
Expand All @@ -51,13 +51,14 @@ services:
PROCESS_NAME: beacon-node
VERBOSITY: debug
P2P_PRIV_KEY: /etc/prysm-priv-key
entrypoint: ["run_beacon_node.sh", "--min-sync-peers=0", "--bootstrap-node="]
entrypoint:
['run_beacon_node.sh', '--min-sync-peers=0', '--bootstrap-node=']
ports:
- "3500:3500"
- "4000:4000"
- "13000:13000"
- '3500:3500'
- '4000:4000'
- '13000:13000'
volumes:
- "beacon_node_data:/chaindata"
- 'beacon_node_data:/chaindata'
- type: bind
source: ./shared/chain-config.yml
target: /config/chain-config.yml
Expand All @@ -68,10 +69,10 @@ services:
source: ./prysm/prysm-priv-key
target: /etc/prysm-priv-key

beacon-node-follower:
prysm-beacon-node-follower:
depends_on:
- execution-node-2
- beacon-node
- prysm-beacon-node
- jaeger-tracing
build:
context: ./prysm
Expand All @@ -80,16 +81,16 @@ services:
EXECUTION_NODE_URL: http://execution-node-2:8545
TRACING_ENDPOINT: http://jaeger-tracing:14268/api/traces
PROCESS_NAME: beacon-node-follower
BEACON_NODE_RPC: http://beacon-node:3500
BEACON_NODE_RPC: http://prysm-beacon-node:3500
VERBOSITY: debug
P2P_PRIV_KEY: /etc/prysm-priv-key
entrypoint: run_beacon_node_peer.sh
ports:
- "3501:3500"
- "4001:4000"
- "13001:13000"
- '3501:3500'
- '4001:4000'
- '13001:13000'
volumes:
- "beacon_node_follower_data:/chaindata"
- 'beacon_node_follower_data:/chaindata'
- type: bind
source: ./shared/chain-config.yml
target: /config/chain-config.yml
Expand All @@ -103,17 +104,17 @@ services:
source: ./prysm/prysm-follower-priv-key
target: /etc/prysm-priv-key

validator-node:
prysm-validator-node:
depends_on:
- beacon-node
- prysm-beacon-node
- jaeger-tracing
build:
context: ./prysm
dockerfile: Dockerfile.prysm
command: >
validator
--accept-terms-of-use
--beacon-rpc-provider beacon-node:4000
--beacon-rpc-provider prysm-beacon-node:4000
--rpc
--grpc-gateway-host 0.0.0.0
--grpc-gateway-port 7500
Expand All @@ -126,21 +127,71 @@ services:
--tracing-endpoint http://jaeger-tracing:14268/api/traces
--tracing-process-name validator-node
ports:
- "7500:7500"
- '7500:7500'
volumes:
- type: bind
source: ./shared/chain-config.yml
target: /config/chain-config.yml

lodestar-beacon-node:
depends_on:
- execution-node
build:
context: ./lodestar
dockerfile: Dockerfile.lodestar
environment:
EXECUTION_NODE_URL: http://execution-node:8545
PROCESS_NAME: lodestar-beacon-node
entrypoint: ['run_beacon_node.sh']
ports:
- '3500:3500'
- '13000:13000'
volumes:
- 'beacon_node_data:/chaindata'
- type: bind
source: ./shared/chain-config.yml
target: /config/chain-config.yml
- type: bind
source: ./lodestar/run_beacon_node.sh
target: /usr/local/bin/run_beacon_node.sh

lodestar-beacon-node-follower:
depends_on:
- execution-node-2
- lodestar-beacon-node
build:
context: ./lodestar
dockerfile: Dockerfile.lodestar
environment:
EXECUTION_NODE_URL: http://execution-node-2:8545
BEACON_NODE_RPC: http://lodestar-beacon-node:3500
PROCESS_NAME: lodestar-beacon-node-follower
VERBOSITY: debug
entrypoint: run_beacon_node_peer.sh
ports:
- '3501:3500'
- '13001:13000'
volumes:
- 'beacon_node_follower_data:/chaindata'
- type: bind
source: ./shared/chain-config.yml
target: /config/chain-config.yml
- type: bind
source: ./lodestar/run_beacon_node.sh
target: /usr/local/bin/run_beacon_node.sh
- type: bind
source: ./lodestar/run_beacon_node_peer.sh
target: /usr/local/bin/run_beacon_node_peer.sh

jaeger-tracing:
image: jaegertracing/all-in-one:1.35
environment:
COLLECTOR_ZIPKIN_HTTP_PORT: 9411
ports:
- "5775:5775/udp"
- "6831:6831/udp"
- "6832:6832/udp"
- "5778:5778"
- "16686:16686"
- "14268:14268"
- "9411:9411"
- '5775:5775/udp'
- '6831:6831/udp'
- '6832:6832/udp'
- '5778:5778'
- '16686:16686'
- '14268:14268'
- '9411:9411'
Copy link
Copy Markdown
Collaborator Author

@dgcoffman dgcoffman Nov 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm sorry, my editor wants single quotes real bad 🤷‍♂️

Copy link
Copy Markdown
Owner

@Inphi Inphi Nov 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's fine :(

2 changes: 1 addition & 1 deletion geth/geth-genesis.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"muirGlacierBlock": 0,
"berlinBlock": 0,
"londonBlock": 0,
"shardingForkBlock": 5,
"shardingForkBlock": 8,
"clique": {
"period": 5,
"epoch": 30000
Expand Down
2 changes: 1 addition & 1 deletion geth/go-ethereum
Open in desktop
Submodule go-ethereum updated 7 files
+26 −104 core/types/data_blob.go
+1 −63 core/vm/contracts.go
+0 −79 crypto/kzg/kzg.go
+142 −0 crypto/kzg/kzg_new.go
+4 −2 signer/core/apitypes/types.go
+14 −51 tests/kzg_bench_test.go
+7 −7 tests/kzg_test.go
5 changes: 4 additions & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ require (
github.com/prysmaticlabs/prysm/v3 v3.1.1
github.com/wealdtech/go-bytesutil v1.1.1
golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f
google.golang.org/grpc v1.40.0
gopkg.in/yaml.v2 v2.4.0
)

Expand Down Expand Up @@ -60,6 +59,7 @@ require (
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/gopacket v1.1.19 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/gorilla/mux v1.8.0 // indirect
github.com/gorilla/websocket v1.5.0 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.0.1 // indirect
github.com/hashicorp/go-bexpr v0.1.11 // indirect
Expand Down Expand Up @@ -147,6 +147,7 @@ require (
github.com/prysmaticlabs/go-bitfield v0.0.0-20210809151128-385d8c5e3fb7 // indirect
github.com/prysmaticlabs/gohashtree v0.0.2-alpha // indirect
github.com/prysmaticlabs/prombbolt v0.0.0-20210126082820-9b7adba6db7c // indirect
github.com/r3labs/sse v0.0.0-20210224172625-26fe804710bc // indirect
github.com/raulk/clock v1.1.0 // indirect
github.com/raulk/go-watchdog v1.2.0 // indirect
github.com/rivo/uniseg v0.2.0 // indirect
Expand Down Expand Up @@ -189,7 +190,9 @@ require (
google.golang.org/api v0.34.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20210426193834-eac7f76ac494 // indirect
google.golang.org/grpc v1.40.0 // indirect
google.golang.org/protobuf v1.28.0 // indirect
gopkg.in/cenkalti/backoff.v1 v1.1.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/natefinch/npipe.v2 v2.0.0-20160621034901-c1b8fa8bdcce // indirect
gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
Expand Down
7 changes: 7 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -384,6 +384,8 @@ github.com/gordonklaus/ineffassign v0.0.0-20200309095847-7953dde2c7bf/go.mod h1:
github.com/gorilla/context v1.1.1/go.mod h1:kBGZzfjB9CEq2AlWe17Uuf7NDRt0dE0s8S51q0aT7Yg=
github.com/gorilla/mux v1.6.2/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/mux v1.7.3/go.mod h1:1lud6UwP+6orDFRuTfBEV8e9/aOM/c4fVVCaMa2zaAs=
github.com/gorilla/mux v1.8.0 h1:i40aqfkR1h2SlN9hojwV5ZA91wcXFOvkdNIeFDP5koI=
github.com/gorilla/mux v1.8.0/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
github.com/gorilla/websocket v0.0.0-20170926233335-4201258b820c/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/gorilla/websocket v1.5.0 h1:PPwGk2jz7EePpoHN/+ClbZu8SPxiqlu12wZP/3sWmnc=
Expand Down Expand Up @@ -927,6 +929,8 @@ github.com/prysmaticlabs/grpc-gateway/v2 v2.3.1-0.20220721162526-0d1c40b5f064/go
github.com/prysmaticlabs/prombbolt v0.0.0-20210126082820-9b7adba6db7c h1:9PHRCuO/VN0s9k+RmLykho7AjDxblNYI5bYKed16NPU=
github.com/prysmaticlabs/prombbolt v0.0.0-20210126082820-9b7adba6db7c/go.mod h1:ZRws458tYHS/Zs936OQ6oCrL+Ict5O4Xpwve1UQ6C9M=
github.com/prysmaticlabs/protoc-gen-go-cast v0.0.0-20211014160335-757fae4f38c6 h1:+jhXLjEYVW4qU2z5SOxlxN+Hv/A9FDf0HpfDurfMEz0=
github.com/r3labs/sse v0.0.0-20210224172625-26fe804710bc h1:zAsgcP8MhzAbhMnB1QQ2O7ZhWYVGYSR2iVcjzQuPV+o=
github.com/r3labs/sse v0.0.0-20210224172625-26fe804710bc/go.mod h1:S8xSOnV3CgpNrWd0GQ/OoQfMtlg2uPRSuTzcSGrzwK8=
github.com/raulk/clock v1.1.0 h1:dpb29+UKMbLqiU/jqIJptgLR1nn23HLgMY0sTCDza5Y=
github.com/raulk/clock v1.1.0/go.mod h1:3MpVxdZ/ODBQDxbN+kzshf5OSZwPjtMDx6BBXBmOeY0=
github.com/raulk/go-watchdog v1.2.0 h1:konN75pw2BMmZ+AfuAm5rtFsWcJpKF3m02rKituuXNo=
Expand Down Expand Up @@ -1220,6 +1224,7 @@ golang.org/x/net v0.0.0-20190628185345-da137c7871d7/go.mod h1:z5CRVTTTmAJ677TzLL
golang.org/x/net v0.0.0-20190724013045-ca1201d0de80/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20190813141303-74dc4d7220e7/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191004110552-13f9640d40b9/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191116160921-f9c825593386/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20191209160850-c0dbc17a3553/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200114155413-6afb5195e5aa/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
Expand Down Expand Up @@ -1580,6 +1585,8 @@ google.golang.org/protobuf v1.28.0 h1:w43yiav+6bVFTBQFZX0r7ipe9JQ1QsbMgHwbBziscL
google.golang.org/protobuf v1.28.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/bsm/ratelimit.v1 v1.0.0-20160220154919-db14e161995a/go.mod h1:KF9sEfUPAXdG8Oev9e99iLGnl2uJMjc5B+4y3O7x610=
gopkg.in/cenkalti/backoff.v1 v1.1.0 h1:Arh75ttbsvlpVA7WtVpH4u9h6Zl46xuptxqLxPiSo4Y=
gopkg.in/cenkalti/backoff.v1 v1.1.0/go.mod h1:J6Vskwqd+OMVJl8C33mmtxTBs2gyzfv7UDAkHu8BrjI=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
Expand Down
3 changes: 3 additions & 0 deletions lodestar/.dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
.git
.github
node_modules
Loading