Skip to content

CLI anchor idl command to use dump base64 for set-buffer and close#2486

Merged
acheroncrypto merged 4 commits intosolana-foundation:masterfrom
marinade-finance:play-with-anchor-cli
May 18, 2023
Merged

CLI anchor idl command to use dump base64 for set-buffer and close#2486
acheroncrypto merged 4 commits intosolana-foundation:masterfrom
marinade-finance:play-with-anchor-cli

Conversation

@ochaloup
Copy link
Copy Markdown
Contributor

@ochaloup ochaloup commented May 11, 2023

A proposal for discussion.

We need to use IDL upgrade from anchor being called from multisig wallet of SPL Governance.
There is a way to pass a custom base64 instruction that is placed into proposal.
A way is to ask anchor idl cli to generate such instruction format instead of execution. As there could be called write-buffer -> set-authority it's all fine to have such functionality only for the set-buffer command.

I asked about it at discord (https://discord.com/channels/889577356681945098/889577399308656662/1105766024650956851) but haven't got any response if there is some way to do so.
This is a solution to get base64 printed. I will be glad for review and happy to hear about better solutions.

@vercel
Copy link
Copy Markdown

vercel bot commented May 11, 2023

@ochaloup is attempting to deploy a commit to the coral-xyz Team on Vercel.

A member of the Team first needs to authorize it.

acheroncrypto
acheroncrypto reviewed May 14, 2023
View reviewed changes
Copy link
Copy Markdown
Collaborator

@acheroncrypto acheroncrypto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, this seems like a plausible solution. My question is would IDL commands other than set-buffer and close be useful for SPL Governance?

@acheroncrypto acheroncrypto added cli idl related to the IDL, either program or client side labels May 14, 2023
@ochaloup ochaloup force-pushed the play-with-anchor-cli branch from 1860a96 to 375f21b Compare May 14, 2023 21:28
@ochaloup
Copy link
Copy Markdown
Contributor Author

ochaloup commented May 15, 2023

Thank your @acheroncrypto for the review and your points.
I updated the code to address the review part.

About the other commands. This change was done from perspective where the idl authority is in use. When the authority is a multisig - it could not be just spl gov but it could be some other multisig system - then the multisig requires to sign.
From that I think the set-authority (https://github.com/coral-xyz/anchor/blob/master/lang/syn/src/codegen/program/idl.rs#L289) is one more command to be added into the scope.
(erase authority and write buffer can be done in two steps)

@ochaloup
Copy link
Copy Markdown
Contributor Author

ochaloup commented May 15, 2023

@acheroncrypto I discussed this PR with my colleague and he has got a really nice point that the flag should be only like --print-only without forcing to provide the current authority manually.
If you be so kind and check the PR with this new adjustment it would be great. Thank you.

@acheroncrypto
Copy link
Copy Markdown
Collaborator

acheroncrypto commented May 15, 2023

@acheroncrypto I discussed this PR with my colleague and he has got a really nice point that the flag should be only like --print-only without forcing to provide the current authority manually. If you be so kind and check the PR with this new adjustment it would be great. Thank you.

I think this looks much better.

One issue is that we are completely recreating Instruction and AccountMeta structs. What would you say to using bincode encoding instead of borsh to get rid of this duplication?

@ochaloup
Copy link
Copy Markdown
Contributor Author

ochaloup commented May 16, 2023 •
edited
Loading

One issue is that we are completely recreating Instruction and AccountMeta structs. What would you say to using bincode encoding instead of borsh to get rid of this duplication?

Thanks @acheroncrypto for the feedback. Sorry I missed the point about the bincode in the first post.
I'm fine to use the bincode serialization but the deserialization at spl gov or serum multisig work with borsh and I'm not sure how those two (bincode vs borsh) may work together.

I did some experiments
ochaloup@3db0202#diff-c1f8f7498da827a634bddc8a7559198bc99b296e9d9e8b91a70b503662995b8cR1975
and the results are different. It seems the borsh align the output to 4 bytes size while bicode somehow strip additional 0. The bincode output fails to be decoded by e.g., spl gov UI processing.

It's like

borsh

 DPRl9bEiiOHiF8/bqgrjrSFA8fetonnejseq+Zra59sDAAAAZNYA/eqUTTit8mSlVRWZCfA/KzcYPpqXMVZTCg5j64cAAQEUEwbrCwK1nWBRfcsHmsSPv3Hq9xKaNnH3AoQH4ngFAAFY1yGdsjj432ypEHVaegNWnDLmTi9zdrRFcIm8Q9RyBwEBCQAAAED0vHin6WkKAw==

[12,244,101,245,177,34,136,225,226,23,207,219,170,10,227,173,33,64,241,247,173,162,121,222,142,199,170,249,154,218,231,219,3,0,0,0,100,...

bincode

DPRl9bEiiOHiF8/bqgrjrSFA8fetonnejseq+Zra59sDZNYA/eqUTTit8mSlVRWZCfA/KzcYPpqXMVZTCg5j64cAAQEUEwbrCwK1nWBRfcsHmsSPv3Hq9xKaNnH3AoQH4ngFAAFY1yGdsjj432ypEHVaegNWnDLmTi9zdrRFcIm8Q9RyBwEBCUD0vHin6WkKAw==

[12,244,101,245,177,34,136,225,226,23,207,219,170,10,227,173,33,64,241,247,173,162,121,222,142,199,170,249,154,218,231,219,3,100,...

Would you know how to make the bincode to generate the serialized output aligned to borsh expectations?

@acheroncrypto
Copy link
Copy Markdown
Collaborator

acheroncrypto commented May 16, 2023

Would you know how to make the bincode to generate the serialized output aligned to borsh expectations?

Yes, we can transform the bincode serialized data if SPL Governance only accepts borsh encoding. I can add this functionality if you'd like.

@ochaloup
Copy link
Copy Markdown
Contributor Author

ochaloup commented May 16, 2023

Yes, we can transform the bincode serialized data if SPL Governance only accepts borsh encoding. I can add this functionality if you'd like.

If you link me to an example in a repo or some doc I would be glad to learn.
If it's easier for you to add it then I will be happy if you can.

@acheroncrypto
Copy link
Copy Markdown
Collaborator

acheroncrypto commented May 16, 2023 •
edited
Loading

If you link me to an example in a repo or some doc I would be glad to learn.

As you've figured, bincode uses 8 bytes(LE) to store the length of the elements meanwhile borsh is using 4 bytes(LE). We can remove extra 4 bytes to make the instructions compatible with borsh.

If it's easier for you to add it then I will be happy if you can.

I've added it but seems like I don't have the rights to push to this branch, could you allow edits by maintainers?

@ochaloup ochaloup force-pushed the play-with-anchor-cli branch from 43aca39 to b99bece Compare May 16, 2023 15:44
@ochaloup
Copy link
Copy Markdown
Contributor Author

ochaloup commented May 16, 2023

@acheroncrypto awesome, it works nicely. I fixed merge conflicts, squashed my commits and placed your one on top.
It's ready for a final review, I assume.

acheroncrypto
acheroncrypto reviewed May 17, 2023
View reviewed changes
@acheroncrypto
Copy link
Copy Markdown
Collaborator

acheroncrypto commented May 17, 2023

@acheroncrypto awesome, it works nicely. I fixed merge conflicts, squashed my commits and placed your one on top. It's ready for a final review, I assume.

Thanks, it looks great. Could you add a feature entry to the CHANGELOG for this PR?

@ochaloup
Copy link
Copy Markdown
Contributor Author

ochaloup commented May 18, 2023

Thanks, it looks great. Could you add a feature entry to the CHANGELOG for this PR?

I updated the changelog, hopefully in format similar to others.

acheroncrypto
acheroncrypto reviewed May 18, 2023
View reviewed changes
@acheroncrypto acheroncrypto merged commit 41a4d82 into solana-foundation:master May 18, 2023
Aursen added a commit to Aursen/anchor that referenced this pull request Jun 10, 2023
@Aursen
Squashed commit of the following:
d8ace33 
commit e1afcbf
Author: acheron <[email protected]>
Date:   Fri Jun 9 18:00:35 2023 +0200

    v0.28.0 (solana-foundation#2527)

commit c7c7319
Author: acheron <[email protected]>
Date:   Thu Jun 8 18:59:44 2023 +0200

    Allow wider range of dependency versions to reduce dependency issues (solana-foundation#2524)

commit 6df34e7
Author: acheron <[email protected]>
Date:   Wed Jun 7 19:12:56 2023 +0200

    Update crate authors and remove outdated registry (solana-foundation#2522)

commit 1705d16
Author: Jean Marchand (Exotic Markets) <[email protected]>
Date:   Wed Jun 7 16:29:23 2023 +0200

    docs: Add doc for InitSpace macro (solana-foundation#2521)

commit 3d7c97b
Author: acheron <[email protected]>
Date:   Tue Jun 6 19:28:24 2023 +0200

    cli: Accept program lib name for `anchor deploy --program-name` (solana-foundation#2519)

commit a88be42
Author: Sergo <[email protected]>
Date:   Tue Jun 6 14:07:33 2023 +0300

    ts: Validate `error.data` exists on simulation response (solana-foundation#2508)

commit 65c9d6e
Author: Jean Marchand (Exotic Markets) <[email protected]>
Date:   Tue Jun 6 09:43:46 2023 +0200

    client: Add async to anchor-client (solana-foundation#2488)

    Co-authored-by: acheron <[email protected]>

commit b8eda69
Author: Deep Mehta <[email protected]>
Date:   Mon Jun 5 22:35:24 2023 +0530

    cli: Print not found message if the given program cannot be found during deployment (solana-foundation#2517)

commit 1902b8e
Author: CanardMandarin <[email protected]>
Date:   Mon Jun 5 14:16:10 2023 +0200

    cli: Update programs in `Anchor.toml` when using `anchor new` (solana-foundation#2516)

commit 383e440
Author: acheron <[email protected]>
Date:   Sun Jun 4 21:02:16 2023 +0200

    cli: Initialize with the correct program id (solana-foundation#2509)

commit 835dc5b
Author: Sarfaraz Nawaz <[email protected]>
Date:   Sun Jun 4 23:20:03 2023 +0530

    lang: Rename derive_anchor_deserialize -> derive_init_space (solana-foundation#2510)

commit 1c6f86e
Author: acheron <[email protected]>
Date:   Sun Jun 4 13:09:39 2023 +0200

    Upgrade Solana to 1.16.0 (solana-foundation#2512)

commit 2bf8afe
Author: acheron <[email protected]>
Date:   Tue May 30 19:50:45 2023 +0200

    cli: Use `confirmed` commitment level in commands (solana-foundation#2506)

commit 70d9223
Author: acheron <[email protected]>
Date:   Sun May 28 22:34:53 2023 +0200

    cli: Add `anchor keys sync` command (solana-foundation#2505)

commit 0c8498d
Author: cavemanloverboy <[email protected]>
Date:   Sat May 27 06:53:02 2023 -0700

    cli: Exit `anchor clean` without error when dirs don't exist (solana-foundation#2504)

commit 23b90bf
Author: Noah Gundotra <[email protected]>
Date:   Fri May 26 12:36:46 2023 -0400

    Feature: CPI Events API (solana-foundation#2438)

    Co-authored-by: acheron <[email protected]>

commit c3625c8
Author: Last Emperor <[email protected]>
Date:   Wed May 24 15:05:47 2023 +0300

    examples: Add an example with `instruction` method (solana-foundation#2501)

    Co-authored-by: acheron <[email protected]>

commit 67eb752
Author: acheron <[email protected]>
Date:   Sat May 20 20:34:38 2023 +0200

    tests: Fix zero-copy tests (solana-foundation#2498)

commit f9d0eca
Author: acheron <[email protected]>
Date:   Fri May 19 13:18:14 2023 +0200

    spl: Update `spl-token-2022` to 0.6.1 (solana-foundation#2496)

commit 4793b90
Author: acheron <[email protected]>
Date:   Fri May 19 10:58:16 2023 +0200

    Fix `toml_datetime` 1.64.0 MSRV error (solana-foundation#2495)

commit 41a4d82
Author: chalda <[email protected]>
Date:   Thu May 18 19:12:25 2023 +0200

    cli: Add print base64 instruction option for some of the IDL commands (solana-foundation#2486)

    Co-authored-by: acheron <[email protected]>

commit b7bada1
Author: Pierre <[email protected]>
Date:   Tue May 16 23:46:40 2023 +1000

    fix: remove skip preflight from cli (solana-foundation#2492)

    ---------

    Co-authored-by: acheron <[email protected]>

commit 89e94d1
Author: Ryan De La O <[email protected]>
Date:   Sat May 13 02:17:47 2023 -0700

    cli: Fix incorrect metadata.address generation (solana-foundation#2485)

    Currently when running 'anchor deploy --program-name <name> --program-keypair <specified keypair>' the cli still uses the auto-generated keypair when fetching the program id to add to the IDL metadata at the end. It should instead use the address from the specified keypair.

    ---------

    Co-authored-by: acheron <[email protected]>

commit 714d524
Author: CanardMandarin <[email protected]>
Date:   Tue May 9 16:17:11 2023 +0200

    lang: Add error message when Mint and TokenAccount with `init` are not ordered correctly (solana-foundation#2484)

commit 9a93a2e
Author: James <[email protected]>
Date:   Mon May 8 10:17:51 2023 +0100

    ts: Improve IDL typing (solana-foundation#2482)

    * Use XOR pattern for enum variants to prevent two variants being used at the same time.

    * Fix unknown for types like Option<[u8; 32]>

commit d1ddf00
Author: CanardMandarin <[email protected]>
Date:   Sun May 7 11:03:37 2023 +0200

    lang: Fix incorrectly checking the first init constraint (solana-foundation#2483)
vadorovsky pushed a commit to Lightprotocol/anchor that referenced this pull request Jun 20, 2023
@ochaloup @acheroncrypto @vadorovsky
cli: Add print base64 instruction option for some of the IDL commands (…
e401c25 
…solana-foundation#2486)

Co-authored-by: acheron <[email protected]>
ananas-block pushed a commit to Lightprotocol/anchor that referenced this pull request Jun 20, 2023
@ochaloup @acheroncrypto @ananas-block
cli: Add print base64 instruction option for some of the IDL commands (…
ba018c3 
…solana-foundation#2486)

Co-authored-by: acheron <[email protected]>
Otter-0x4ka5h pushed a commit to Otter-0x4ka5h/anchor that referenced this pull request Mar 25, 2026
@ochaloup @acheroncrypto
cli: Add print base64 instruction option for some of the IDL commands (…
dd82771 
…solana-foundation#2486)

Co-authored-by: acheron <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@acheroncrypto acheroncrypto acheroncrypto approved these changes

Assignees

No one assigned

Labels

cli idl related to the IDL, either program or client side

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ochaloup @acheroncrypto