Skip to content

git pull#1

Merged
Daanvdplas merged 34 commits intoDaanvdplas:masterfrom
paritytech:master
Jun 13, 2024
Merged

git pull#1
Daanvdplas merged 34 commits intoDaanvdplas:masterfrom
paritytech:master

Conversation

@Daanvdplas
Copy link
Copy Markdown
Owner

No description provided.

rzadp and others added 30 commits June 4, 2024 06:43
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
This PR implements the `READ_ONLY` flag to be used as a `Callflag` in
the `call` function.
The flag indicates that the callee is restricted from modifying the
state during call execution.
It is equivalent to Ethereum's
[STATICCALL](https://eips.ethereum.org/EIPS/eip-214).

---------

Co-authored-by: command-bot <>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Since the fix for #4559
was to add a codeSubstitute to the rococo-parachains, we should update
the chain-specs in the repo.

The json diffs are not super easy to review, so if you want to manually
check:
- Run for example `jd -o people-rococo.diff
people-rococo-from-this-branch.json people-rococo-master.json`
- Check in the resulting diff file that only `codeSubstitutes` field is
changed
This allows to use custom para_ids with cumulus-test-runtime. 

Zombienet is patching the genesis entries for `ParachainInfo`. This did
not work with `test-parachain` because it was using the `test_pallet`
for historic reasons I guess.
Added reference doc about:
- the pallet genesis config and genesis build, 
- runtime `genesis-builder` API,
- presets,
- interacting with the `chain-spec-builder` tool

I've added [minimal
runtime](https://github.com/paritytech/polkadot-sdk/tree/mku-chain-spec-guide/docs/sdk/src/reference_docs/chain_spec_runtime)
to demonstrate above topics.

I also sneaked in some little improvement to `chain-spec-builder` which
allows to parse output of the `list-presets` command.

---------

Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
In order to receive a username in `pallet-identity`, users have to,
among other things, provide a signature of the desired username. Right
now, there is an [extra encoding
step](https://github.com/paritytech/polkadot-sdk/blob/4ab078d6754147ce731523292dd1882f8a7b5775/substrate/frame/identity/src/lib.rs#L1119)
when generating the payload to sign.

Encoding a `Vec` adds extra bytes related to the length, which changes
the payload. This is unnecessary and confusing as users expect the
payload to sign to be just the username bytes. This PR fixes this issue
by validating the signature directly against the username bytes.

---------

Signed-off-by: georgepisaltu <george.pisaltu@parity.io>
This Dockerfile seems outdated - it currently fails to build (on my
machine).
I don't see it being built anywhere on CI.

I did a couple of tweaks to make it build.

---------

Co-authored-by: Alexander Samusev <41779041+alvicsam@users.noreply.github.com>
Follow up on #4414 to clean up the old storage.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
In preparation for launching re-enabling
(#2418), we need to
adjust the disabling strategy of statement-distribution to use the relay
parent's state instead of the latest state (union of active leaves).
This will also ensure no raciness of getting the latest state vs
accepting statements from disabling validators at the cost of being more
lenient/potentially accepting more statements from disabled validators.

- [x] PRDoc
Inherited workspace dependencies cannot be renamed by the crate using
them (see [1](rust-lang/cargo#12546),
[2](https://stackoverflow.com/questions/76792343/can-inherited-dependencies-in-rust-be-aliased-in-the-cargo-toml-file)).
Since we want to use inherited workspace dependencies everywhere, we
first need to unify all aliases that we use for a dependency throughout
the workspace.
The umbrella crate is currently excluded from this procedure, since it
should be able to export the crates by their original name without much
hassle.

For example: one crate may alias `parity-scale-codec` to `codec`, while
another crate does not alias it at all. After this change, all crates
have to use `codec` as name. The problematic combinations were:
- conflicting aliases: most crates aliases as `A` but some use `B`.
- missing alias: most of the crates alias a dep but some dont.
- superfluous alias: most crates dont alias a dep but some do.

The script that i used first determines whether most crates opted to
alias a dependency or not. From that info it decides whether to use an
alias or not. If it decided to use an alias, the most common one is used
everywhere.

To reproduce, i used
[this](https://github.com/ggwpez/substrate-scripts/blob/master/uniform-crate-alias.py)
python script in combination with
[this](https://github.com/ggwpez/zepter/blob/38ad10585fe98a5a86c1d2369738bc763a77057b/renames.json)
error output from Zepter.

---------

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
…ons (#4709)

`secp256k1 v0.28.0` and `secp256k1-sys v0.9.0` were yanked because
building them fails for `aarch64-apple-darwin` targets.

Use the `secp256k1 v0.28.2` and `secp256k1-sys v0.9.2` patched versions
that build fine on ARM chipset MacOS.
Nobody seems to maintain this and the job is disabled since months. I
think unless the Security team wants to pick this up we delete it for
now.

Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
The link to
[releases](https://github.com/paritytech/polkadot-sdk/releases) usually
takes you to a list with a bunch of drafts at the top so you have to
scroll.

[This link](https://github.com/paritytech/polkadot-sdk/releases/latest)
takes you directly to the latest release.
Part of #4324
We don't change but extend the existing cleanup strategy. 
- We still don't touch artifacts being stale less than 24h
- First time we attempt pruning only when we hit cache limit (10 GB)
- If somehow happened that after we hit 10 GB and least used artifact is
stale less than 24h we don't remove it.

---------

Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
…er (#4718)

Adds a link to the [Polkadot SDK Version
Manager](https://github.com/paritytech/psvm) since this tool is not well
known, but very useful for developers using the Polkadot SDK.

---------

Co-authored-by: Bastian Köcher <git@kchr.de>
... when backing group is of size 1.

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
Co-authored-by: eskimor <eskimor@no-such-url.com>
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com>
<img width="484" alt="Screenshot 2024-05-30 at 10 30 41"
src="https://github.com/paritytech/polkadot-sdk/assets/5588131/92ece1d0-cda3-46cf-9f46-4026c565b52d">

---------

Co-authored-by: Dónal Murray <donal.murray@parity.io>
Thanks for @xlc for the original seed info, I've just fixed it up a bit
and added example links.

I've moved the comparison between eth-rpc-api and frontier outside, as
it is opinionation. I think the content there was good but should live
in the README of the corresponding repos. No strong opinion, happy
either way.

---------

Co-authored-by: Bryan Chen <xlchen1291@gmail.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Gonçalo Pestana <g6pestana@gmail.com>
Co-authored-by: command-bot <>
Small refactoring PR to improve the readability of the proc macros.
- small improvement in docs
- use new `let Some(..) else` expression
- removed extra indentations by early returns

Discovered during metadata v16 poc, extracted from:
#4358

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: command-bot <>
Co-authored-by: gupnik <mail.guptanikhil@gmail.com>
…r the "Releases" section (#4734)

This update implements the suggestion from @kianenigma mentioned in
#4718 (comment)

Replaces the "Other useful resources and tooling" section at the bottom
with a new (nicer) "Tooling" section just under the "Releases" section.
- Addresses
[this](#3155 (comment)).
- Revamps the Readme, very similar to [the minimal
template](#4649).
- Changed `polkadot-launch` to `zombienet`, with instructions how to run
it.
- See the [rendered
version](https://github.com/paritytech/polkadot-sdk/blob/rzadp/parachain-template-readme/templates/parachain/README.md).
Add some debug logs to be able to identify the validators and parachains
that have most no-shows, this metric is valuable because it will help us
identify validators and parachains that regularly have this problem.

From the validator_index we can then query the on-chain information and
identify the exact validator that is causing the no-shows.

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
- unit tests for pov-recovery
- elastic scaling support (recovering multiple candidates in a single
relay chain block)
- also some small cleanups
- also switches to candidates_pending_availability in
`handle_empty_block_announce_data`

Fixes #3577

After #4097 is merged, we
should also add a zombienet test, similar to the existing
`0002-pov_recovery.toml` but which has a single collator using elastic
scaling on multiple cores.
... this is superfluous because set_reserved_peers implementation
already calls this method here:

https://github.com/paritytech/polkadot-sdk/blob/cdb297b15ad9c1d952c0501afaf6b764e5fd147c/substrate/client/network/src/protocol_controller.rs#L571,
so the call just ends producing this warnings whenever we manipulate the
peers set.

```
Trying to remove unknown reserved node 12D3KooWRCePWvHoBbz9PSkw4aogtdVqkVDhiwpcHZCqh4hdPTXC from SetId(3)
peerset warnings (from different peers)
```

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
skunert and others added 4 commits June 11, 2024 13:02
## Issue

Currently, syncing parachains from scratch can lead to a very long
finalization time once they reach the tip of the chain. The problem is
that we try to finalize everything from 0 to the tip, which can be
thousands or even millions of blocks.

We finalize sequentially and try to compute displaced branches during
finalization. So for every block on the way, we compute an expensive
tree route.

## Proposed Improvements

In this PR, I propose improvements that solve this situation:

- **Skip tree route calculation if `leaves().len() == 1`:** This should
be enough for 90% of cases where there is only one leaf after sync.
- **Optimize finalization for long distances:** It can happen that the
parachain has imported some leaf and then receives a relay chain
notification with the finalized block. In that case, the previous
optimization will not trigger. A second mechanism should ensure that we
do not need to compute the full tree route. If the finalization distance
is long, we check the lowest common ancestor of all the leaves. If it is
above the to-be-finalized block, we know that there are no displaced
leaves. This is fast because forks are short and close to the tip, so we
can leverage the header cache.

## Alternative Approach

- The problem was introduced in #3962. Reverting that PR is another
possible strategy.
- We could store for every fork where it begins, however sounds a bit
more involved to me.


fixes #4614
This branch propose to avoid clones in append by storing offset and size
in previous overlay depth.
That way on rollback we can just truncate and change size of existing
value.
To avoid copy it also means that :

- append on new overlay layer if there is an existing value: create a
new Append entry with previous offsets, and take memory of previous
overlay value.
- rollback on append: restore value by applying offsets and put it back
in previous overlay value
- commit on append: appended value overwrite previous value (is an empty
vec as the memory was taken). offsets of commited layer are dropped, if
there is offset in previous overlay layer they are maintained.
- set value (or remove) when append offsets are present: current
appended value is moved back to previous overlay value with offset
applied and current empty entry is overwrite (no offsets kept).

The modify mechanism is not needed anymore.
This branch lacks testing and break some existing genericity (bit of
duplicated code), but good to have to check direction.

Generally I am not sure if it is worth or we just should favor
differents directions (transients blob storage for instance), as the
current append mechanism is a bit tricky (having a variable length in
first position means we sometime need to insert in front of a vector).

Fix #30.

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: EgorPopelyaev <egor@parity.io>
Co-authored-by: Alexandru Vasile <60601340+lexnv@users.noreply.github.com>
Co-authored-by: Bastian Köcher <git@kchr.de>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Liam Aharon <liam.aharon@hotmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Branislav Kontur <bkontur@gmail.com>
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
Addressing comment:
https://github.com/paritytech/polkadot-sdk/pull/4102/files#r1635502496

---------

Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
After using this tutorial in PBA, there was a few areas to improve it.
Moreover, I have:

- Improve `your_first_pallet`, link it in README, improve the parent
`guide` section.
- Updated the templates page, in light of recent efforts related to in
#3155
- Added small ref docs about metadata, completed the one about native
runtime, added one about host functions.
- Remove a lot of unfinished stuff from sdk-docs
- update diagram for `Hooks`
@Daanvdplas Daanvdplas merged commit a91e7b1 into Daanvdplas:master Jun 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.