Skip to content

Contracts: Only exec parsed code in benchmarks#3915

Merged
pgherveou merged 90 commits intomasterfrom
pg/bench_tweaks
Apr 10, 2024
Merged

Contracts: Only exec parsed code in benchmarks#3915
pgherveou merged 90 commits intomasterfrom
pg/bench_tweaks

Conversation

@pgherveou
Copy link
Contributor

@pgherveou pgherveou commented Mar 31, 2024

Weights compare

Note: Raw weights change does not mean much here, as this PR reduce the scope of what is benchmarked, they are therefore decreased by a good margin. One should instead print the Schedule using

cargo test --features runtime-benchmarks bench_print_schedule -- --nocapture
or following the instructions from the README for looking at the Schedule of a specific runtime

@pgherveou pgherveou force-pushed the pg/bench_tweaks branch 2 times, most recently from f08e071 to fb8d6e2 Compare March 31, 2024 16:32
@pgherveou
Copy link
Contributor Author

pgherveou commented Mar 31, 2024

@athei @Robbepop
I think this approach address the comment of this PR

#2941 (review)

@pgherveou pgherveou marked this pull request as ready for review March 31, 2024 16:44
@pgherveou pgherveou requested a review from athei as a code owner March 31, 2024 16:44
@pgherveou
Copy link
Contributor Author

bot bench substrate-pallet --pallet=pallet_contracts

@command-bot
Copy link

command-bot bot commented Mar 31, 2024

@pgherveou https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5705896 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 1-de455737-5853-415e-a6dd-b0b70806389c to cancel this command or bot cancel to cancel all commands in this pull request.

@command-bot
Copy link

command-bot bot commented Mar 31, 2024

@pgherveou Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5705896 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5705896/artifacts/download.

@pgherveou
Copy link
Contributor Author

pgherveou commented Mar 31, 2024

Looking the Schedule result might be more relevant here than the raw benchmark

-  now: 329 ns, 6 bytes,
+  now: 280 ns, 0 bytes,

I think these results make a bit more sense. I guess the reason we had some PoV previously is because the length of the wasm code benchmarked was increasing linearly and thus was showing up in the benchmark equation

@pgherveou
Copy link
Contributor Author

bot bench substrate-pallet --pallet=pallet_contracts

@command-bot
Copy link

command-bot bot commented Apr 1, 2024

@pgherveou "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts (https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5715482) was cancelled in #3915 (comment)

@pgherveou
Copy link
Contributor Author

bot cancel 2-11f65051-854c-4ba5-93f1-47f2d7449529

@command-bot
Copy link

command-bot bot commented Apr 1, 2024

@pgherveou Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5715482 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5715482/artifacts/download.

@pgherveou
Copy link
Contributor Author

bot bench substrate-pallet --pallet=pallet_contracts

@command-bot
Copy link

command-bot bot commented Apr 1, 2024

@pgherveou https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5715784 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 3-dd926209-4239-443d-8c29-322948c88ecb to cancel this command or bot cancel to cancel all commands in this pull request.

@command-bot
Copy link

command-bot bot commented Apr 1, 2024

@pgherveou Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5715784 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5715784/artifacts/download.

@pgherveou pgherveou added the T7-smart_contracts This PR/Issue is related to smart contracts. label Apr 8, 2024
@paritytech-review-bot paritytech-review-bot bot requested a review from a team April 9, 2024 08:52
Base automatically changed from pg/bench_update to master April 10, 2024 07:20
@pgherveou pgherveou enabled auto-merge April 10, 2024 07:31
@pgherveou
Copy link
Contributor Author

bot bench substrate-pallet --pallet=pallet_contracts

@command-bot
Copy link

command-bot bot commented Apr 10, 2024

@pgherveou https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5856612 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts. Check out https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 1-36936205-076c-4cf6-a78d-8965e72caf6f to cancel this command or bot cancel to cancel all commands in this pull request.

@pgherveou pgherveou disabled auto-merge April 10, 2024 07:33
jonathanudd pushed a commit to jonathanudd/polkadot-sdk that referenced this pull request Apr 10, 2024
commit 265365920836bb1d286c9b48b1902a2de278fdd9
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Wed Jan 29 19:51:15 2020 -0500

    Move hc-jp-bridge repo to different folder

commit 8271991e95320baba70bd1cb9c4234d0ffd5b638
Merge: 57d0811 304cbc5
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Wed Jan 29 19:36:41 2020 -0500

    Merge branch 'hc-jp-bridge-module' of hc-jp-bridge-module

commit 304cbc5f02d003ffa5404c1c01e461e5b8539888
Author: Hernando Castano <HCastano@users.noreply.github.com>
Date:   Wed Jan 29 00:38:27 2020 -0500

    Update bridge pallet to work with the (almost) lastest master (paritytech#4672)

    * Update decl_error usage

    * WIP: Update error handling to use DispatchResult

    * Get module compiling with new error handling

    * Make tests compile again

    Main change was updating the usage of InMemoryBackend

    * Move `sp-state-machine` into dev-dependencies

    * Bump dependencies to v2.0.0

    * Remove some stray comments

    * Appy code review suggestion

commit 510cd6d96372688517496efa61773ea2839f8474
Author: Hernando Castano <HCastano@users.noreply.github.com>
Date:   Tue Dec 17 12:52:51 2019 -0500

    Move Bridge Pallet into FRAME (paritytech#4373)

    * Move `bridge` crate into `frame` folder

    * Make `bridge` pallet compile after `the-big-reorg`

commit ab54e838ef75e6a3f68fd0944bf22598c10c552f
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Mon Nov 11 21:56:40 2019 +0100

    Use new StorageProof type from paritytech#3834

commit 8fc8911fd1b4acc2274c6863fb3dba91b30c90af
Author: Hernando Castano <HCastano@users.noreply.github.com>
Date:   Tue Nov 5 00:50:34 2019 +0100

    Verify Ancestry between Headers (paritytech#3963)

    * Create module for checking ancestry proofs

    * Use Vec of Headers instead of a HashMap

    * Move the ancestry verification into the lib.rs file

    * Change the proof format to exclude `child` and `ancestor` headers

    * Add a testing function for building header chains

    * Rename AncestorNotFound error to InvalidAncestryProof

    * Use ancestor hash instead of header when verifying ancestry

    * Clean up some stuff missed in the merge

commit dbe85738b68358b790cf927b34a804b965a88f96
or: Hernando Castano <HCastano@users.noreply.github.com>
Date:   Fri Nov 1 15:41:58 2019 +0100

    Check given Grandpa validator set against set found in storage (paritytech#3915)

    * Make StorageProofChecker happy

    * Update some tests

    * Check given validator set against set found in storage

    * Use Finality Grandpa's Authority Id and Weight

    * Add better error handling

    * Use error type from decl_error! macro

commit 31b09216603d3e9c21144ce8c0b6bf59307a4f97
or: Hernando Castano <HCastano@users.noreply.github.com>
Date:   Wed Oct 23 14:55:37 2019 +0200

    Make tests work after the changes introduced in paritytech#3793 (paritytech#3874)

    * Make tests work after the changes introduced in paritytech#3793

    * Remove unneccessary import

commit bce6d804aa86504599ff912387295c58f846cbf3
Author: Jim Posen <jim.posen@gmail.com>
Date:   Thu Oct 10 12:18:58 2019 +0200

    Logic for checking Substrate proofs from within runtime module. (paritytech#3783)

commit a7013e94b6c772c1d45a7cacbb445f73f6554fca
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Fri Oct 4 15:21:00 2019 +0300

    Allow tracking of multiple bridges

commit 3cf648242d631e32bd553a67df54bf5a48912839
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Tue Oct 1 14:55:04 2019 +0200

    Add BridgeId => Bridge mapping

commit 001c74c45072213e01857d0a2454379b447c5a76
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Tue Oct 1 11:10:19 2019 +0200

    Get the mock runtime for tests set up

commit 38443a1e8b424ed2f148eb95121d009f730e3b5a
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Fri Sep 27 14:52:53 2019 +0200

    Clean up some warnings

commit bdc3b01401e89c7111f8bf71f84c50750d25089f
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Thu Sep 26 16:41:01 2019 +0200

    Add more skeleton code

commit 26995efbf4bac2842eb2822322f7ad3c3e88feb8
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Wed Sep 25 15:16:57 2019 +0200

    Create `bridge` module skeleton
jonathanudd pushed a commit to jonathanudd/polkadot-sdk that referenced this pull request Apr 10, 2024
commit 657deb4cf4b90f24b9c5bfd62764b197776c262c
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Wed Jan 29 20:14:20 2020 -0500

    Move Slava's bridge code into relays folder

commit 4868c42c7da959dde7252766996b3ed4e408e439
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Wed Jan 29 20:01:06 2020 -0500

    Move files into `modules/ethereum`

commit d1093f3e4238acb1a1a020011452cb928d3f8d7a
Merge: 29dc6f9 bfd30ef
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Wed Jan 29 19:59:27 2020 -0500

    Merge branch 'master' of slava-async-bridge

commit 29dc6f97b1b7d1db99086d35a5336f43d2f0f8af
Author: Hernando Castano <castano.ha@gmail.com>
Date:   Wed Jan 29 19:51:31 2020 -0500

    Squashed commit of the following:

    commit 265365920836bb1d286c9b48b1902a2de278fdd9
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Wed Jan 29 19:51:15 2020 -0500

        Move hc-jp-bridge repo to different folder

    commit 8271991e95320baba70bd1cb9c4234d0ffd5b638
    Merge: 57d0811 304cbc5
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Wed Jan 29 19:36:41 2020 -0500

        Merge branch 'hc-jp-bridge-module' of hc-jp-bridge-module

    commit 304cbc5f02d003ffa5404c1c01e461e5b8539888
    Author: Hernando Castano <HCastano@users.noreply.github.com>
    Date:   Wed Jan 29 00:38:27 2020 -0500

        Update bridge pallet to work with the (almost) lastest master (paritytech#4672)

        * Update decl_error usage

        * WIP: Update error handling to use DispatchResult

        * Get module compiling with new error handling

        * Make tests compile again

        Main change was updating the usage of InMemoryBackend

        * Move `sp-state-machine` into dev-dependencies

        * Bump dependencies to v2.0.0

        * Remove some stray comments

        * Appy code review suggestion

    commit 510cd6d96372688517496efa61773ea2839f8474
    Author: Hernando Castano <HCastano@users.noreply.github.com>
    Date:   Tue Dec 17 12:52:51 2019 -0500

        Move Bridge Pallet into FRAME (paritytech#4373)

        * Move `bridge` crate into `frame` folder

        * Make `bridge` pallet compile after `the-big-reorg`

    commit ab54e838ef75e6a3f68fd0944bf22598c10c552f
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Mon Nov 11 21:56:40 2019 +0100

        Use new StorageProof type from paritytech#3834

    commit 8fc8911fd1b4acc2274c6863fb3dba91b30c90af
    Author: Hernando Castano <HCastano@users.noreply.github.com>
    Date:   Tue Nov 5 00:50:34 2019 +0100

        Verify Ancestry between Headers (paritytech#3963)

        * Create module for checking ancestry proofs

        * Use Vec of Headers instead of a HashMap

        * Move the ancestry verification into the lib.rs file

        * Change the proof format to exclude `child` and `ancestor` headers

        * Add a testing function for building header chains

        * Rename AncestorNotFound error to InvalidAncestryProof

        * Use ancestor hash instead of header when verifying ancestry

        * Clean up some stuff missed in the merge

    commit dbe85738b68358b790cf927b34a804b965a88f96
    or: Hernando Castano <HCastano@users.noreply.github.com>
    Date:   Fri Nov 1 15:41:58 2019 +0100

        Check given Grandpa validator set against set found in storage (paritytech#3915)

        * Make StorageProofChecker happy

        * Update some tests

        * Check given validator set against set found in storage

        * Use Finality Grandpa's Authority Id and Weight

        * Add better error handling

        * Use error type from decl_error! macro

    commit 31b09216603d3e9c21144ce8c0b6bf59307a4f97
    or: Hernando Castano <HCastano@users.noreply.github.com>
    Date:   Wed Oct 23 14:55:37 2019 +0200

        Make tests work after the changes introduced in paritytech#3793 (paritytech#3874)

        * Make tests work after the changes introduced in paritytech#3793

        * Remove unneccessary import

    commit bce6d804aa86504599ff912387295c58f846cbf3
    Author: Jim Posen <jim.posen@gmail.com>
    Date:   Thu Oct 10 12:18:58 2019 +0200

        Logic for checking Substrate proofs from within runtime module. (paritytech#3783)

    commit a7013e94b6c772c1d45a7cacbb445f73f6554fca
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Fri Oct 4 15:21:00 2019 +0300

        Allow tracking of multiple bridges

    commit 3cf648242d631e32bd553a67df54bf5a48912839
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Tue Oct 1 14:55:04 2019 +0200

        Add BridgeId => Bridge mapping

    commit 001c74c45072213e01857d0a2454379b447c5a76
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Tue Oct 1 11:10:19 2019 +0200

        Get the mock runtime for tests set up

    commit 38443a1e8b424ed2f148eb95121d009f730e3b5a
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Fri Sep 27 14:52:53 2019 +0200

        Clean up some warnings

    commit bdc3b01401e89c7111f8bf71f84c50750d25089f
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Thu Sep 26 16:41:01 2019 +0200

        Add more skeleton code

    commit 26995efbf4bac2842eb2822322f7ad3c3e88feb8
    Author: Hernando Castano <castano.ha@gmail.com>
    Date:   Wed Sep 25 15:16:57 2019 +0200

        Create `bridge` module skeleton

commit bfd30ef8363b1483ef1107ae1eb958a4e944c93b
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Tue Dec 10 12:10:53 2019 +0300

    actually use signer from CLI to sign Substrate transactions

commit 504028eac60d9d14ba95b506cd355b0d2f405ce0
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Tue Dec 10 12:02:22 2019 +0300

    go offline for a bit on connection error

commit 446d0c8d20187dfd1beb173958ea28f2ad97887d
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Tue Dec 10 11:25:50 2019 +0300

    enable info logs by default

commit d039c60ec72bc91adfdad85442bc99a93b7f8e8d
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Tue Dec 10 11:12:51 2019 +0300

    support basic CLI arguments

commit 65c6d48e23576f36e8541878b920a03730226392
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 15:37:48 2019 +0300

    fix restart

commit 96e94c1c4b22d732078f8c401b872c5f8246c3fe
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 14:57:53 2019 +0300

    license

commit 68f4191e6cdd211ac8975e0b79f8a6f46a3ca953
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 14:56:05 2019 +0300

    restart sync when Substrate reorgs && we are unlucky

commit 29887c446167d580d73cc03a0b71c31890cafb51
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 13:49:31 2019 +0300

    only read genesis hash once

commit 832492b8393fe2063adf9c58c2b9e060dc3e4efb
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 13:23:26 2019 +0300

    changed TODO

commit 9dbc130e5fa036ae63d973819daf30f4ed6ffb5b
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 13:16:56 2019 +0300

    removed obsolete exit future

commit d03408cd8284eb0c61e7e96429b4f6199353e030
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 13:16:17 2019 +0300

    removed obsolete TODOs + moved a couple of TODOs to runtime module

commit ed8bec44b79f9a2ce829e59f10181368b2f42139
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 12:37:05 2019 +0300

    explained TODO fix

commit aa9c4c66ec2904eeb6072d654718b0ac0b7d8803
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 12:28:09 2019 +0300

    fix tx outcome serialization

commit 126f8f5484dac8c4af588ae86dc8855919d6c822
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Mon Dec 9 12:05:05 2019 +0300

    prune old ethereum headers when Substrate best header is too far in the future

commit c7bd301e631a44fe3263e188d0956081aa84f31e
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Fri Dec 6 12:51:50 2019 +0300

    fix trace

commit 549bb7acdb30cfdafe6c8600f0410212539ea63d
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Fri Dec 6 12:51:26 2019 +0300

    tx hashes are already a part of Block response

commit 7864017909f87ea36955d605a924c3c88bc88df3
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Thu Dec 5 12:29:37 2019 +0300

    submit bunch of headers at once + some fixes

commit 96485f85d38c144f0771f02ba692216a60356665
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Wed Dec 4 17:22:13 2019 +0300

    print status messages

commit ae0ec4c087136db653339537daab7f96a8c21b65
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Wed Dec 4 17:06:00 2019 +0300

    continue actual Substrate client implementation

commit 8146293740d70b88904568ff8e5acdfbadf06fd3
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Wed Dec 4 13:49:30 2019 +0300

    fix IncompleteHeader condition

commit 767c6201157dabcccf7f62e643681ca298224fb1
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Wed Dec 4 10:55:06 2019 +0300

    actual Substrate client implementation

commit 221fd4ccd2b1eea12c9dacf800d80e15ec115c1b
Author: Svyatoslav Nikolsky <svyatonik@gmail.com>
Date:   Wed Nov 20 17:28:13 2019 +0300

    initial commit
…=dev --target_dir=substrate --pallet=pallet_contracts
@command-bot
Copy link

command-bot bot commented Apr 10, 2024

@pgherveou Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=dev --target_dir=substrate --pallet=pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5856612 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/5856612/artifacts/download.

@athei athei enabled auto-merge April 10, 2024 10:45
@athei athei added this pull request to the merge queue Apr 10, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to no response for status checks Apr 10, 2024
@pgherveou pgherveou added this pull request to the merge queue Apr 10, 2024
Merged via the queue into master with commit 0d71753 Apr 10, 2024
@pgherveou pgherveou deleted the pg/bench_tweaks branch April 10, 2024 14:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

T7-smart_contracts This PR/Issue is related to smart contracts.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants