-
Notifications
You must be signed in to change notification settings - Fork 12.4k
Add Base58 library #5762
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Add Base58 library #5762
Changes from 45 commits
Commits
Show all changes
52 commits
Select commit
Hold shift + click to select a range
5a400eb
Add Bytes.splice, an inplace variant of Buffer.slice
Amxx 65292d5
Add Base58 library
Amxx 99a1835
docs
Amxx bddf4f6
Merge branch 'feature/Bytes-splice' into feature/base58
Amxx 88c03e7
Add Bytes.countConsecutive and Bytes.countLeading
Amxx a3c4667
fix
Amxx 41b586b
efficient decoding
Amxx c6d6bdd
coverage
Amxx 48bf13b
Update thirty-pugs-pick.md
Amxx eebd51e
docs
Amxx 296a87e
pragma
Amxx 8c94acc
pragma
Amxx d09ebfa
coverage
Amxx a25bd11
rewrite _encode in assembly
Amxx a4ce8c8
more inline documentation
Amxx 7474f2a
test vectors
Amxx bef2e4f
document
Amxx ce1c5ad
remove auxiliary utils
Amxx c33e933
mload is actually cheaper than jump
Amxx 855a1c6
up
Amxx ec641c7
Update contracts/utils/Base58.sol
Amxx 7429bcc
up
Amxx 45edb76
do base58 arithmetics in chunks of 248 bits
Amxx 20f3611
update
Amxx 8e60a99
codespell
Amxx dd8e895
decode assembly
Amxx 45f04b4
char valdity filter
Amxx da84743
slither
Amxx c80f693
slither
Amxx f7ac27d
fix custom error name + testing
Amxx 2696cd8
Apply suggestions from code review
Amxx 1736f38
optimize zero limbs accounting
Amxx 8652d20
Update contracts/utils/Base58.sol
Amxx 8098fb2
Update test/utils/Base58.t.sol
Amxx d0ece81
Apply suggestions from code review
Amxx 3974f6d
Merge branch 'master' into feature/base58
Amxx 59b2866
fix compilation and update custom error name
Amxx 66ef584
minify change by removing unecessary feature in Bytes.sol
Amxx 8b1ae97
Apply suggestions from code review
Amxx bb3aaf3
Update contracts/utils/Base58.sol
Amxx ac75789
clarify
Amxx 930a03f
update estimation to always overestimate
Amxx c3a4e76
closer approximation
Amxx f372c20
Update Base58.sol
Amxx 377ec53
Enhance comments
ernestognw aeaa73c
fraction values
Amxx 107328f
Update contracts/utils/Base58.sol
ernestognw e3268cf
Update contracts/utils/Base58.sol
ernestognw fe288ea
Enhance _decode comments
ernestognw 8e32c79
Add custom error changes section in changelog
ernestognw 0c488da
Apply suggestions from code review
Amxx 23221e1
Update CHANGELOG.md
ernestognw File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| 'openzeppelin-solidity': minor | ||
| --- | ||
|
|
||
| `Base58`: Add a library for encoding and decoding bytes buffers into base58 strings. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,230 @@ | ||
| // SPDX-License-Identifier: MIT | ||
|
|
||
| pragma solidity ^0.8.20; | ||
|
|
||
| /** | ||
| * @dev Provides a set of functions to operate with Base58 strings. | ||
| * | ||
| * Base58 is an encoding scheme that converts binary data into a human-readable text format. | ||
| * Similar to {Base64} but specifically designed for better human usability. | ||
| * | ||
| * 1. Human-friendly alphabet: Excludes visually similar characters to reduce human error: | ||
| * * No 0 (zero) vs O (capital O) confusion | ||
Amxx marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| * * No I (capital i) vs l (lowercase L) confusion | ||
| * * No non-alphanumeric characters like + or = | ||
| * 2. URL-safe: Contains only alphanumeric characters, making it safe for URLs without encoding. | ||
| * | ||
ernestognw marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| * Initially based on https://github.com/storyicon/base58-solidity/commit/807428e5174e61867e4c606bdb26cba58a8c5cb1[storyicon's implementation] (MIT). | ||
| * Based on the updated and improved https://github.com/Vectorized/solady/blob/208e4f31cfae26e4983eb95c3488a14fdc497ad7/src/utils/Base58.sol[Vectorized version] (MIT). | ||
| */ | ||
| library Base58 { | ||
| /// @dev Unrecognized Base58 character on decoding. | ||
| error InvalidBase58Char(bytes1); | ||
|
|
||
| /** | ||
| * @dev Encode a `bytes` buffer as a Base58 `string`. | ||
| */ | ||
| function encode(bytes memory input) internal pure returns (string memory) { | ||
| return string(_encode(input)); | ||
| } | ||
|
|
||
| /** | ||
| * @dev Decode a Base58 `string` into a `bytes` buffer. | ||
| */ | ||
| function decode(string memory input) internal pure returns (bytes memory) { | ||
| return _decode(bytes(input)); | ||
| } | ||
|
|
||
| function _encode(bytes memory input) private pure returns (bytes memory output) { | ||
| uint256 inputLength = input.length; | ||
| if (inputLength == 0) return ""; | ||
|
|
||
| assembly ("memory-safe") { | ||
| // Count number of zero bytes at the beginning of `input`. These are encoded using the same number of '1's | ||
| // at the beginning of the encoded string. | ||
| let inputLeadingZeros := 0 | ||
| for {} lt(byte(0, mload(add(add(input, 0x20), inputLeadingZeros))), lt(inputLeadingZeros, inputLength)) {} { | ||
coderabbitai[bot] marked this conversation as resolved.
Show resolved
Hide resolved
ernestognw marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| inputLeadingZeros := add(inputLeadingZeros, 1) | ||
| } | ||
|
|
||
| // Start the output offset by an over-estimate of the length. | ||
| // When converting from base-256 (bytes) to base-58, the theoretical length ratio is ln(256)/ln(58). | ||
| // We use 9886/7239 ≈ 1.3657 as a rational approximation that slightly over-estimates to ensure | ||
| // sufficient memory allocation. (ln = natural logarithm) | ||
Amxx marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| let outputLengthEstim := add(inputLeadingZeros, div(mul(sub(inputLength, inputLeadingZeros), 9886), 7239)) | ||
ernestognw marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| // This is going to be our "scratch" workspace. We leave enough room before FMP to later store length + encoded output. | ||
ernestognw marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| // 0x21 = 0x20 (32 bytes for result length prefix) + 0x1 (safety buffer for division truncation) | ||
| let scratch := add(mload(0x40), add(outputLengthEstim, 0x21)) | ||
Amxx marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| // Chunk input into 31-byte limbs (248 bits) for efficient batch processing. | ||
| // Each limb fits safely in a 256-bit word with 8-bit overflow protection. | ||
| // Memory layout: [output chars] [limb₁(248 bits)][limb₂(248 bits)][limb₃(248 bits)]... | ||
| // ↑ scratch | ||
| // ↑ ptr (moves right) | ||
| let ptr := scratch | ||
| for { | ||
| // Handle partial first limb if input length isn't divisible by 31 | ||
| let i := mod(inputLength, 31) | ||
| if i { | ||
| // Right-shift to align partial limb in high bits of 256-bit word | ||
| mstore(ptr, shr(mul(sub(32, i), 8), mload(add(input, 0x20)))) | ||
| ptr := add(ptr, 0x20) // next limb | ||
| } | ||
| } lt(i, inputLength) { | ||
| ptr := add(ptr, 0x20) // next limb | ||
| i := add(i, 31) // move in buffer | ||
| } { | ||
| // Load 31 bytes from input, right-shift by 8 bits to leave 1 zero byte in low bits | ||
ernestognw marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| mstore(ptr, shr(8, mload(add(add(input, 0x20), i)))) | ||
| } | ||
|
|
||
| // Store the encoding table. This overlaps with the FMP that we are going to reset later anyway. | ||
| // See https://datatracker.ietf.org/doc/html/draft-msporny-base58-03#section-2 | ||
| mstore(0x1f, "123456789ABCDEFGHJKLMNPQRSTUVWXY") | ||
| mstore(0x3f, "Zabcdefghijkmnopqrstuvwxyz") | ||
|
|
||
| // Core Base58 encoding: repeated division by 58 on input limbs | ||
| // Memory layout: [output chars] [limb₁(248 bits)][limb₂(248 bits)][limb₃(248 bits)]... | ||
| // ↑ scratch ↑ ptr | ||
| // ↑ output (moves left) | ||
| // ↑ data (moves right) | ||
| for { | ||
| let data := scratch // Points to first non-zero limb | ||
| output := scratch // Builds result right-to-left from scratch | ||
| } 1 {} { | ||
| // Skip zero limbs at the beginning (limbs become 0 after repeated divisions) | ||
| for {} and(iszero(mload(data)), lt(data, ptr)) { | ||
| data := add(data, 0x20) | ||
| } {} | ||
| // Exit when all limbs are zero (conversion complete) | ||
| if eq(data, ptr) { | ||
| break | ||
| } | ||
|
|
||
| // Division by 58 across all remaining limbs | ||
| let carry := 0 | ||
| for { | ||
| let i := data | ||
| } lt(i, ptr) { | ||
| i := add(i, 0x20) | ||
| } { | ||
| let acc := add(shl(248, carry), mload(i)) // Combine carry from previous limb with current limb | ||
| mstore(i, div(acc, 58)) // Store quotient back in limb | ||
| carry := mod(acc, 58) // Remainder becomes next carry | ||
| } | ||
|
|
||
| // Convert remainder (0-57) to Base58 character and store right-to-left | ||
Amxx marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| output := sub(output, 1) | ||
| mstore8(output, mload(carry)) | ||
| } | ||
|
|
||
| // Write the input leading zeros at the left of the encoded. | ||
| // This may spill to the left into the "length" of the buffer. | ||
| for { | ||
| let i := 0 | ||
| } lt(i, inputLeadingZeros) {} { | ||
| i := add(i, 0x20) | ||
| mstore(sub(output, i), "11111111111111111111111111111111") | ||
| } | ||
|
|
||
| // Move output pointer to account for inputLeadingZeros | ||
| output := sub(output, add(inputLeadingZeros, 0x20)) | ||
|
|
||
| // Store length and allocate (reserve) memory up to scratch. | ||
| mstore(output, sub(scratch, add(output, 0x20))) // Overwrite spilled bytes | ||
| mstore(0x40, scratch) | ||
| } | ||
| } | ||
|
|
||
| function _decode(bytes memory input) private pure returns (bytes memory output) { | ||
| bytes4 errorSelector = InvalidBase58Char.selector; | ||
|
|
||
| uint256 inputLength = input.length; | ||
| if (inputLength == 0) return ""; | ||
|
|
||
| assembly ("memory-safe") { | ||
| let inputLeadingZeros := 0 // Number of leading '1' in `input`. | ||
| // Count leading zeros. In base58, zeros are represented using '1' (chr(49)). | ||
| for {} and( | ||
| eq(byte(0, mload(add(add(input, 0x20), inputLeadingZeros))), 49), | ||
| lt(inputLeadingZeros, inputLength) | ||
| ) {} { | ||
| inputLeadingZeros := add(inputLeadingZeros, 1) | ||
| } | ||
|
|
||
| // Start the output offset by an over-estimate of the length. | ||
| // When converting from base-58 to base-256 (bytes), the theoretical length ratio is ln(58)/ln(256). | ||
| // We use 6115/8351 ≈ 0.7322 as a rational approximation that slightly over-estimates to ensure | ||
| // sufficient memory allocation. (ln = natural logarithm) | ||
Amxx marked this conversation as resolved.
Outdated
Show resolved
Hide resolved
|
||
| let outputLengthEstim := add(inputLeadingZeros, div(mul(sub(inputLength, inputLeadingZeros), 7239), 9886)) | ||
|
|
||
| // This is going to be our "scratch" workspace. Be leave enough room on the left to store length + encoded input. | ||
| let scratch := add(mload(0x40), add(outputLengthEstim, 0x21)) | ||
|
|
||
| // Store the decoding table. This overlaps with the FMP that we are going to reset later anyway. | ||
| mstore(0x2a, 0x30313233343536373839) | ||
| mstore(0x20, 0x1718191a1b1c1d1e1f20ffffffffffff2122232425262728292a2bff2c2d2e2f) | ||
| mstore(0x00, 0x000102030405060708ffffffffffffff090a0b0c0d0e0f10ff1112131415ff16) | ||
|
|
||
| // Decode each char of the input string, and store it in sections (limbs) of 31 bytes. Store in scratch. | ||
| let ptr := scratch | ||
| let mask := shr(8, not(0)) | ||
| for { | ||
| let j := 0 | ||
| } lt(j, inputLength) { | ||
| j := add(j, 1) | ||
| } { | ||
| // for each char, decode it ... | ||
| let c := sub(byte(0, mload(add(add(input, 0x20), j))), 49) | ||
| // slither-disable-next-line incorrect-shift | ||
| if iszero(and(shl(c, 1), 0x3fff7ff03ffbeff01ff)) { | ||
| mstore(0, errorSelector) | ||
| mstore(4, shl(248, add(c, 49))) | ||
| revert(0, 0x24) | ||
| } | ||
| let carry := byte(0, mload(c)) | ||
|
|
||
| // ... and add it to the limbs starting a `scratch` | ||
| for { | ||
| let i := scratch | ||
| } lt(i, ptr) { | ||
| i := add(i, 0x20) | ||
| } { | ||
| let acc := add(carry, mul(58, mload(i))) | ||
| mstore(i, and(mask, acc)) | ||
| carry := shr(248, acc) | ||
| } | ||
| // If the char just read result in a leftover carry, extend the limbs with the new value | ||
| if carry { | ||
| mstore(ptr, carry) | ||
| ptr := add(ptr, 0x20) | ||
| } | ||
| } | ||
|
|
||
| // Copy and compact the uint248 limbs + remove any zeros at the beginning. | ||
| output := scratch | ||
| for { | ||
| let i := scratch | ||
| } lt(i, ptr) { | ||
| i := add(i, 0x20) | ||
| } { | ||
| output := sub(output, 31) | ||
| mstore(sub(output, 1), mload(i)) | ||
| } | ||
| for {} lt(byte(0, mload(output)), lt(output, scratch)) {} { | ||
| output := add(output, 1) | ||
| } | ||
|
|
||
| // Add the zeros that were encoded in the input (prefix '1's) | ||
| calldatacopy(sub(output, inputLeadingZeros), calldatasize(), inputLeadingZeros) | ||
|
|
||
| // Move output pointer to account for inputLeadingZeros | ||
| output := sub(output, add(inputLeadingZeros, 0x20)) | ||
|
|
||
| // Store length and allocate (reserve) memory up to scratch. | ||
| mstore(output, sub(scratch, add(output, 0x20))) | ||
| mstore(0x40, scratch) | ||
| } | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| // SPDX-License-Identifier: MIT | ||
|
|
||
| pragma solidity ^0.8.26; | ||
|
|
||
| import {Test} from "forge-std/Test.sol"; | ||
| import {Base58} from "@openzeppelin/contracts/utils/Base58.sol"; | ||
|
|
||
| contract Base58Test is Test { | ||
| function testEncodeDecodeEmpty() external pure { | ||
| assertEq(Base58.decode(Base58.encode(hex"")), hex""); | ||
| } | ||
|
|
||
| function testEncodeDecodeZeros() external pure { | ||
| bytes memory zeros = hex"0000000000000000"; | ||
| assertEq(Base58.decode(Base58.encode(zeros)), zeros); | ||
|
|
||
| bytes memory almostZeros = hex"00000000a400000000"; | ||
| assertEq(Base58.decode(Base58.encode(almostZeros)), almostZeros); | ||
| } | ||
|
|
||
| function testEncodeDecode(bytes memory input) external pure { | ||
| assertEq(Base58.decode(Base58.encode(input)), input); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| const { ethers } = require('hardhat'); | ||
| const { expect } = require('chai'); | ||
| const { loadFixture } = require('@nomicfoundation/hardhat-network-helpers'); | ||
|
|
||
| async function fixture() { | ||
| const mock = await ethers.deployContract('$Base58'); | ||
| return { mock }; | ||
| } | ||
|
|
||
| describe('Base58', function () { | ||
| beforeEach(async function () { | ||
| Object.assign(this, await loadFixture(fixture)); | ||
| }); | ||
|
|
||
| describe('base58', function () { | ||
| describe('encode/decode random buffers', function () { | ||
| // length 512 runs out of gas. | ||
| // this checks are very slow when running coverage, causing CI to timeout. | ||
| for (const length of [0, 1, 2, 3, 4, 32, 42, 128, 384]) | ||
| it( | ||
| [length > 32 && '[skip-on-coverage]', `buffer of length ${length}`].filter(Boolean).join(' '), | ||
| async function () { | ||
| const buffer = ethers.randomBytes(length); | ||
| const hex = ethers.hexlify(buffer); | ||
| const b58 = ethers.encodeBase58(buffer); | ||
|
|
||
| await expect(this.mock.$encode(hex)).to.eventually.equal(b58); | ||
| await expect(this.mock.$decode(b58)).to.eventually.equal(hex); | ||
| }, | ||
| ); | ||
| }); | ||
|
|
||
| // Tests case from section 5 of the (no longer active) Base58 Encoding Scheme RFC | ||
| // https://datatracker.ietf.org/doc/html/draft-msporny-base58-03 | ||
| describe('test vectors', function () { | ||
| for (const { raw, b58 } of [ | ||
| { raw: 'Hello World!', b58: '2NEpo7TZRRrLZSi2U' }, | ||
| { | ||
| raw: 'The quick brown fox jumps over the lazy dog.', | ||
| b58: 'USm3fpXnKG5EUBx2ndxBDMPVciP5hGey2Jh4NDv6gmeo1LkMeiKrLJUUBk6Z', | ||
| }, | ||
| { raw: '0x0000287fb4cd', b58: '11233QC4' }, | ||
| ]) | ||
| it(raw, async function () { | ||
| const buffer = (ethers.isHexString(raw) ? ethers.getBytes : ethers.toUtf8Bytes)(raw); | ||
| const hex = ethers.hexlify(buffer); | ||
|
|
||
| await expect(this.mock.$encode(hex)).to.eventually.equal(b58); | ||
| await expect(this.mock.$decode(b58)).to.eventually.equal(hex); | ||
| }); | ||
| }); | ||
|
|
||
| describe('decode invalid format', function () { | ||
| for (const chr of ['I', '-', '~']) | ||
| it(`Invalid base58 char ${chr}`, async function () { | ||
| const getHexCode = str => ethers.hexlify(ethers.toUtf8Bytes(str)); | ||
| const helper = { interface: ethers.Interface.from(['error InvalidBase58Char(bytes1)']) }; | ||
|
|
||
| await expect(this.mock.$decode(`VYRWKp${chr}pnN7`)) | ||
| .to.be.revertedWithCustomError(helper, 'InvalidBase58Char') | ||
| .withArgs(getHexCode(chr)); | ||
| }); | ||
| }); | ||
| }); | ||
| }); |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.