Skip to content
This repository was archived by the owner on Jan 19, 2021. It is now read-only.

Promisify rest of library #107

Merged
holgerd77 merged 4 commits into from
Apr 13, 2020
Merged

Promisify rest of library #107

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2e4c40e
Upgrade from callbacks to promises
ryanio Apr 2, 2020
056152c
Upgrade from callbacks to promises
ryanio Apr 6, 2020
dfaa274
Improve benchmarks code
ryanio Apr 10, 2020
9390ab1
Fix typedoc (method execute not async)
ryanio Apr 10, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 68 additions & 76 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# SYNOPSIS

[![NPM Package](https://img.shields.io/npm/v/merkle-patricia-tree)](https://www.npmjs.org/package/merkle-patricia-tree)
[![Actions Status](https://github.com/ethereumjs/ethereumjs-util/workflows/Build/badge.svg)](https://github.com/ethereumjs/merkle-patricia-tree/actions)
[![Actions Status](https://github.com/ethereumjs/merkle-patricia-tree/workflows/Build/badge.svg)](https://github.com/ethereumjs/merkle-patricia-tree/actions)
[![Coverage Status](https://img.shields.io/coveralls/ethereumjs/merkle-patricia-tree.svg)](https://coveralls.io/r/ethereumjs/merkle-patricia-tree)
[![Gitter](https://img.shields.io/gitter/room/ethereum/ethereumjs.svg)](https://gitter.im/ethereum/ethereumjs)

Expand All @@ -23,95 +23,84 @@ There are 3 variants of the tree implemented in this library, namely: `BaseTrie`

## Initialization and Basic Usage

```javascript
const Trie = require('merkle-patricia-tree').BaseTrie,
level = require('level'),
db = level('./testdb'),
trie = new Trie(db)
```typescript
import level from 'level'
import { BaseTrie as Trie } from 'merkle-patricia-tree'

trie.put(Buffer.from('test'), Buffer.from('one'), function () {
trie.get(Buffer.from('test'), function (err, value) {
if (value) console.log(value.toString())
})
})
const db = level('./testdb')
const trie = new Trie(db)

await trie.put(Buffer.from('test'), Buffer.from('one'))
const value = await trie.get(Buffer.from('test'))
console.log(value.toString())
```

## Merkle Proofs

```javascript
Trie.prove(trie, Buffer.from('test'), function (err, prove) {
if (err) return cb(err)
Trie.verifyProof(trie.root, Buffer.from('test'), prove, function (err, value) {
if (err) return cb(err)
console.log(value.toString())
cb()
})
})
```typescript
const prove = await Trie.prove(trie, Buffer.from('test'))
const value = await Trie.verifyProof(trie.root, Buffer.from('test'), prove)
console.log(value.toString())
```

## Read stream on Geth DB

```javascript
var level = require('level')
var Trie = require('merkle-patricia-tree').SecureTrie

var stateRoot = '0xd7f8974fb5ac78d9ac099b9ad5018bedc2ce0a72dad1827a1709da30580f0544' // Block #222
```typescript
import level from 'level'
import { SecureTrie as Trie } from 'merkle-patricia-tree'

var db = level('YOUR_PATH_TO_THE_GETH_CHAIN_DB')
var trie = new Trie(db, stateRoot)
const db = level('YOUR_PATH_TO_THE_GETH_CHAIN_DB')
// Set stateRoot to block #222
const stateRoot = '0xd7f8974fb5ac78d9ac099b9ad5018bedc2ce0a72dad1827a1709da30580f0544'
// Initialize trie
const trie = new Trie(db, stateRoot)

trie
.createReadStream()
.on('data', function (data) {
console.log(data)
})
.on('end', function () {
.on('data', console.log)
.on('end', () => {
console.log('End.')
})
```

## Read Account State including Storage from Geth DB

```javascript
var level = require('level')
var rlp = require('rlp')
var ethutil = require('ethereumjs-util')

var Trie = require('merkle-patricia-tree').SecureTrie
var Account = require('ethereumjs-account').default
var BN = ethutil.BN

var stateRoot = 'STATE_ROOT_OF_A_BLOCK'

var db = level('YOUR_PATH_TO_THE_GETH_CHAINDATA_FOLDER')
var trie = new Trie(db, stateRoot)

var address = 'AN_ETHEREUM_ACCOUNT_ADDRESS'

trie.get(address, function (err, data) {
if (err) return cb(err)

var acc = new Account(data)
console.log('-------State-------')
console.log(`nonce: ${new BN(acc.nonce)}`)
console.log(`balance in wei: ${new BN(acc.balance)}`)
console.log(`storageRoot: ${ethutil.bufferToHex(acc.stateRoot)}`)
console.log(`codeHash: ${ethutil.bufferToHex(acc.codeHash)}`)

var storageTrie = trie.copy()
storageTrie.root = acc.stateRoot

console.log('------Storage------')
var stream = storageTrie.createReadStream()
stream
.on('data', function (data) {
console.log(`key: ${ethutil.bufferToHex(data.key)}`)
console.log(`Value: ${ethutil.bufferToHex(rlp.decode(data.value))}`)
})
.on('end', function () {
console.log('Finished reading storage.')
})
})
```typescript
import level from 'level'
import rlp from 'rlp'
import { BN, bufferToHex } from 'ethereumjs-util'
import Account from 'ethereumjs-account'
import { SecureTrie as Trie } from 'merkle-patricia-tree'

const stateRoot = 'STATE_ROOT_OF_A_BLOCK'

const db = level('YOUR_PATH_TO_THE_GETH_CHAINDATA_FOLDER')
const trie = new Trie(db, stateRoot)

const address = 'AN_ETHEREUM_ACCOUNT_ADDRESS'

const data = await trie.get(address)
const acc = new Account(data)

console.log('-------State-------')
console.log(`nonce: ${new BN(acc.nonce)}`)
console.log(`balance in wei: ${new BN(acc.balance)}`)
console.log(`storageRoot: ${bufferToHex(acc.stateRoot)}`)
console.log(`codeHash: ${bufferToHex(acc.codeHash)}`)

let storageTrie = trie.copy()
storageTrie.root = acc.stateRoot

console.log('------Storage------')
const stream = storageTrie.createReadStream()
stream
.on('data', (data) => {
console.log(`key: ${bufferToHex(data.key)}`)
console.log(`Value: ${bufferToHex(rlp.decode(data.value))}`)
})
.on('end', () => {
console.log('Finished reading storage.')
})
```

# API
Expand All @@ -124,11 +113,14 @@ trie.get(address, function (err, data) {

# REFERENCES

- ["Exploring Ethereum's state trie with Node.js"](https://wanderer.github.io/ethereum/nodejs/code/2014/05/21/using-ethereums-tries-with-node/) blog post
- ["Merkling in Ethereum"](https://blog.ethereum.org/2015/11/15/merkling-in-ethereum/) blog post
- [Ethereum Trie Specification](https://github.com/ethereum/wiki/wiki/Patricia-Tree) Wiki
- ["Understanding the ethereum trie"](https://easythereentropy.wordpress.com/2014/06/04/understanding-the-ethereum-trie/) blog post
- ["Trie and Patricia Trie Overview"](https://www.youtube.com/watch?v=jXAHLqQthKw&t=26s) Video Talk on Youtube
- Wiki
- [Ethereum Trie Specification](https://github.com/ethereum/wiki/wiki/Patricia-Tree)
- Blog posts
- [Exploring Ethereum's State Trie with Node.js](https://wanderer.github.io/ethereum/nodejs/code/2014/05/21/using-ethereums-tries-with-node/)
- [Merkling in Ethereum](https://blog.ethereum.org/2015/11/15/merkling-in-ethereum/)
- [Understanding the Ethereum Trie](https://easythereentropy.wordpress.com/2014/06/04/understanding-the-ethereum-trie/)
- Videos
- [Trie and Patricia Trie Overview](https://www.youtube.com/watch?v=jXAHLqQthKw&t=26s)

# EthereumJS

Copy link
Member

@holgerd77 holgerd77 Apr 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did an overall README check, everything covered, examples look good as well.

Expand Down
92 changes: 42 additions & 50 deletions benchmarks/checkpointing.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,60 +1,52 @@
import * as async from 'async'
import * as crypto from 'crypto'
const Trie = require('../dist/index.js').CheckpointTrie
import { pseudoRandomBytes } from 'crypto'
import { CheckpointTrie } from '../dist'

let iterations = 500
let samples = 20
let i = 0
const iterations = 500
const samples = 20

function iterTest(numOfIter: number, cb: Function) {
let vals = [] as any
let keys = [] as any
const iterTest = async (numOfIter: number): Promise<Array<number>> => {
return new Promise(async (resolve) => {
let vals = [] as any
let keys = [] as any

for (i = 0; i <= numOfIter; i++) {
vals.push(crypto.pseudoRandomBytes(32))
keys.push(crypto.pseudoRandomBytes(32))
}
for (let i = 0; i <= numOfIter; i++) {
vals.push(pseudoRandomBytes(32))
keys.push(pseudoRandomBytes(32))
}

let hrstart = process.hrtime()
let numOfOps = 0
let trie = new Trie()
let hrstart = process.hrtime()
let numOfOps = 0
let trie = new CheckpointTrie()

for (i = 0; i < numOfIter; i++) {
trie.put(vals[i], keys[i], function () {
for (let i = 0; i < numOfIter; i++) {
await trie.put(vals[i], keys[i])
trie.checkpoint()
trie.get(Buffer.from('test'), function () {
numOfOps++
if (numOfOps === numOfIter) {
const hrend = process.hrtime(hrstart)
cb(hrend)
}
})
})
}
await trie.get(Buffer.from('test'))
numOfOps++
if (numOfOps === numOfIter) {
const hrend = process.hrtime(hrstart)
resolve(hrend)
}
}
})
}

i = 0
let avg = [0, 0]
const go = async () => {
let i = 0
let avg = [0, 0]

async.whilst(
function () {
while (i <= samples) {
const hrend = await iterTest(iterations)
avg[0] += hrend[0]
avg[1] += hrend[1]
// console.log('benchmarks/checkpointing.ts | execution time: %ds %dms', hrend[0], (hrend[1] / 1000000).toFixed(3))
i++
return i <= samples
},
function (done) {
iterTest(iterations, function (hrend: Array<number>) {
avg[0] += hrend[0]
avg[1] += hrend[1]

console.info('Execution time (hr): %ds %dms', hrend[0], hrend[1] / 1000000)
done()
})
},
function () {
console.info(
'Average Execution time (hr): %ds %dms',
avg[0] / samples,
avg[1] / 1000000 / samples,
)
},
)
}
console.log(
'benchmarks/checkpointing.ts | average execution time: %ds %dms',
avg[0] / samples,
(avg[1] / 1000000 / samples).toFixed(3),
)
}

go()
65 changes: 33 additions & 32 deletions benchmarks/random.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,45 +1,46 @@
// https://github.com/ethereum/wiki/wiki/Benchmarks
'use strict'
import * as async from 'async'
import * as ethUtil from 'ethereumjs-util'
const Trie = require('../dist/index.js').BaseTrie
import { keccak256 } from 'ethereumjs-util'
import { BaseTrie } from '../dist'

const ROUNDS = 1000
const SYMMETRIC = true
const ERA_SIZE = 1000

let trie = new Trie()
const trie = new BaseTrie()
let seed = Buffer.alloc(32).fill(0)

let testName = 'rounds ' + ROUNDS + ' ' + ERA_SIZE + ' ' + SYMMETRIC ? 'sys' : 'rand'
console.time(testName)
run(() => {
console.timeEnd(testName)
})

function run(cb: any) {
const run = async (): Promise<void> => {
let i = 0
async.whilst(
() => {
i++
return i <= ROUNDS
},
function (done) {
seed = ethUtil.keccak256(seed)
if (SYMMETRIC) {
trie.put(seed, seed, genRoot)
} else {
let val = ethUtil.keccak256(seed)
trie.put(seed, val, genRoot)
}
while (i <= ROUNDS) {
seed = keccak256(seed)

function genRoot() {
if (i % ERA_SIZE === 0) {
seed = trie.root
}
done()
const genRoot = () => {
if (i % ERA_SIZE === 0) {
seed = trie.root
}
},
cb,
)
}

if (SYMMETRIC) {
await trie.put(seed, seed)
genRoot()
} else {
const val = keccak256(seed)
await trie.put(seed, val)
genRoot()
}

i++
}
}

const go = async () => {
const testName = `benchmarks/random.ts | rounds: ${ROUNDS}, ERA_SIZE: ${ERA_SIZE}, ${
SYMMETRIC ? 'sys' : 'rand'
}`
console.time(testName)
await run()
console.timeEnd(testName)
}

go()
Copy link
Member

@holgerd77 holgerd77 Apr 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested both manually locally, updated benchmark suites at least run through. This seems to need more work to be usable - the checkpointing benchmark file e.g. is not taking the samples into account at all e.g., think due to a bug with a reused/side-used i counter variable as far as I see it - nothing for this PR though but just as a note.

Copy link
Contributor Author

@ryanio ryanio Apr 8, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah thanks for pointing out, I will give it a closer look.

Loading