This is the second blog post for hacking Ethereum. If you have not gone through the first blog post please click here and have your local environment set up. The main entry point for Ethereum is /go-ethereum/cmd/geth/main.go
Open up main.go and look at ‘main’ method, which is the entry point whenever we execute ./geth binary. This function basically executes the application with whatever command parameter we have supplied to it. As we know in ‘go’ there is specific function ‘init’ which gets executed before calling the module, let’s see what’s inside in initialisation code. The important line to notice is:
app.Action = geth
So default action if we don’t supply any via command line parameter is geth. ‘geth’ is a function inside main.go. This function is actually starting up all background services for a node.
func geth(ctx *cli.Context) error { node := makeFullNode(ctx) startNode(ctx, node) node.Wait() return nil }
Let further analyse the function ‘makeFullNode(ctx)’. Go to its implementation, we see at 2nd line “utils.RegisterEthService(stack, &cfg.Eth)”
It is registering “eth” service using utils. As we are running full node following is the code which actually gets executed:
func RegisterEthService(stack *node.Node, cfg *eth.Config) { ..... err = stack.Register(func(ctx *node.ServiceContext) (node.Service, error) { fullNode, err := eth.New(ctx, cfg) if fullNode != nil && cfg.LightServ > 0 { ls, _ := les.NewLesServer(fullNode, cfg) fullNode.AddLesServer(ls) } return fullNode, err }) .....
This function is registering new service to the stack which is basically a node. It means whenever we start a node this service will be executed. Let’s analyse the line fullNode, err := eth.New(ctx, cfg)
Going to the implementation of the ‘eth.New’ we go to ‘backend.go’ file. Looking at the constructor code, this service is nothing but Ethereum service. This constructor function configures the whole Ethereum. As our goal for this tutorial is to change the consensus algorithm following is the code of interest under ‘backend.go -> func New’
eth := &Ethereum{ config: config, chainDb: chainDb, chainConfig: chainConfig, eventMux: ctx.EventMux, accountManager: ctx.AccountManager, engine: CreateConsensusEngine(ctx, &config.Ethash, chainConfig, chainDb), shutdownChan: make(chan bool), networkID: config.NetworkId, gasPrice: config.GasPrice, etherbase: config.Etherbase, bloomRequests: make(chan chan *bloombits.Retrieval), bloomIndexer: NewBloomIndexer(chainDb, params.BloomBitsBlocks), }
myalgo.go
func (MyAlgo *MyAlgo) VerifyHeader(chain consensus.ChainReader, header *types.Header, seal bool) error { log.Info("will verfiyHeader") return nil }
func (MyAlgo *MyAlgo) VerifyHeaders(chain consensus.ChainReader, headers []*types.Header, seals []bool) (chan<- struct{}, <-chan error){ log.Info("will verfiyHeaders") abort := make(chan struct{}) results := make(chan error, len(headers))
go func() { for _, header := range headers { err := MyAlgo.VerifyHeader(chain, header, false) select { case <-abort: return case results <- err: } } }() return abort, results }
func (MyAlgo *MyAlgo) VerifyUncles(chain consensus.ChainReader, block *types.Block) error { log.Info("will verfiy uncles") return nil }
func (MyAlgo *MyAlgo) VerifySeal(chain consensus.ChainReader, header *types.Header) error{ log.Info("will verfiy VerifySeal") return nil }
func (MyAlgo *MyAlgo) Prepare(chain consensus.ChainReader, header *types.Header) error{ log.Info("will prepare the block") parent := chain.GetHeader(header.ParentHash, header.Number.Uint64()-1) if parent == nil { return consensus.ErrUnknownAncestor }
header.Difficulty = MyAlgo.CalcDifficulty(chain, header.Time.Uint64(), parent) return nil }
func (MyAlgo *MyAlgo) CalcDifficulty(chain consensus.ChainReader, time uint64, parent *types.Header) *big.Int { return calcDifficultyHomestead(time, parent) }
func (MyAlgo *MyAlgo) Finalize(chain consensus.ChainReader, header *types.Header, state *state.StateDB, txs []*types.Transaction, uncles []*types.Header, receipts []*types.Receipt) (*types.Block, error){ log.Info("will Finalize the block") header.Root = state.IntermediateRoot(chain.Config().IsEIP158(header.Number)) b := types.NewBlock(header, txs, uncles, receipts) return b, nil }
func (MyAlgo *MyAlgo) Seal(chain consensus.ChainReader, block *types.Block, stop <-chan struct{}) (*types.Block, error){ log.Info("will Seal the block") //time.Sleep(15 * time.Second) header := block.Header() header.Nonce, header.MixDigest = getRequiredHeader() return block.WithSeal(header), nil }
As you can see most of them are just dummy function, always returning success result. In this algorithm, we are not actually finding any nonce, so after plugging this consensus algorithm it is expected that block is generated in an infinite loop without any delay. Let’s plug this algorithm and see what we are getting in logs. To plug this algorithm in Ethereum insert following code to “CreateConsensusEngine” function in ‘go-ethereum/eth/backend.go’ file.
if chainConfig.MyAlgo != nil{ fmt.Println(“myalgo is configured as consensus engine") return myalgo.New(chainConfig.MyAlgo, db) }
So this method basically checks if we have configured our configuration to use ‘MyAlgo’ or not. To enable ‘MyAlgo’ we need to make changes to our ‘privategensis.json’, Replace the content of the file with the following:
{ "config": { "chainId": 15, "homesteadBlock": 0, "eip155Block": 0, "eip158Block": 0, "myalgo" : {} }, "difficulty": "2000000", "gasLimit": "21000000", "alloc": { } }
Note the additional change “myalgo:{}” this line will enable our algorithm instead of default one. Delete previously create a directory from tutorial-1 and initialise Ethereum with new configuration:
> rm -rf ~/.ethereum/myprivatenet > ./geth --datadir ~/.ethereum/myprivatenet init privategensis.json
Start the node as we did previously in the last tutorial. You will notice some additional logs and module after starting the node:
INFO [06-21|16:49:41.400938] Starting P2P networking
myalgo is configured as consensus engine
INFO [06-21|16:49:43.658541] UDP listener up self=enode://f08e5aa4fbde25b09ee977001af4a642abfd150c47e571ec5b69deed8c8c644c0a785d47a890adb6b2b55155cd32d7c035f4ec3ce7f8af54cd07219c8ad06ee7@[::]:30301 INFO [06-21|16:49:43.658717] RLPx listener up self=enode://f08e5aa4fbde25b09ee977001af4a642abfd150c47e571ec5b69deed8c8c644c0a785d47a890adb6b2b55155cd32d7c035f4ec3ce7f8af54cd07219c8ad06ee7@[::]:30301 INFO [06-21|16:49:43.661563] IPC endpoint opened url=/Users/hemants/.ethereum/myprivatenet/geth.ipc INFO [06-21|16:49:43.662937] HTTP endpoint opened url=http://localhost:8101 cors=http://localhost:8000 vhosts=localhost Welcome to the Geth JavaScript console!
instance: Geth/v1.8.12-unstable/darwin-amd64/go1.9.2 modules: admin:1.0 debug:1.0 eth:1.0 miner:1.0 myalgo:1.0 net:1.0 personal:1.0 rpc:1.0 txpool:1.0 web3:1.0
Log message: myalgo is configured as consensus engine, confirm that our configuration has been picked up and now Ethereum is configured with our own consensus algorithm. Also, a new module gets added to the console, it means we can interact with our consensus algorithm from the console. Great !!. This ‘myalgo’ got added due to the following two changes:
In consensus/myalgo/api.go:
type API struct { chain consensus.ChainReader myAlgo *MyAlgo } func (api *API) EchoNumber(ctx context.Context, number uint64) (uint64, error) { fmt.Println("called echo number") return number, nil }
In internal/web3ext/web3ext.go:
var Modules = map[string]string{ "admin": Admin_JS, ... "txpool": TxPool_JS, "myalgo": MyAlgo_JS, } const MyAlgo_JS = ` web3._extend({ property: 'myalgo', methods: [ new web3._extend.Method({ name: 'echoNumber', call: 'myalgo_echoNumber', params: 1, inputFormatter: [null] }), ] })
This ‘echoNumber’ function is just echoing whatever we pass as an input parameter. This is to just show how we can add our own custom console modules.
> myalgo { echoNumber: function() } > myalgo.echoNumber(5) called echo number 5
Now let’s start miner and see which function is being called from our consensus algorithm. As you start the miner you will notice that instantly it start mining blocks, here is the log:
> miner.start(1) null > INFO [06-21|17:01:52.43016] Transaction pool price threshold updated price=18000000000 INFO [06-21|17:01:52.43104] Etherbase automatically configured address=0x33954C10f5789A5914a09F0B1ef64CD26E0FA66E INFO [06-21|17:01:52.43109] Starting mining operation INFO [06-21|17:01:52.43113] will prepare the block INFO [06-21|17:01:52.431516] will Finalize the block INFO [06-21|17:01:52.431532] Commit new mining work number=1 txs=0 uncles=0 elapsed=406.763µs INFO [06-21|17:01:52.431749] will Seal the block INFO [06-21|17:01:52.431797] Successfully sealed new block number=1 hash=71810a…2882f3 INFO [06-21|17:01:52.436641] ? mined potential block number=1 hash=71810a…2882f3 INFO [06-21|17:01:52.436681] will prepare the block INFO [06-21|17:01:52.43676] will Finalize the block INFO [06-21|17:01:52.436768] Commit new mining work number=2 txs=0 uncles=0 elapsed=90.323µs INFO [06-21|17:01:52.436991] will Seal the block INFO [06-21|17:01:52.437038] Successfully sealed new block number=2 hash=b6bef3…f2ed9b INFO [06-21|17:01:52.437498] ? mined potential block number=2 hash=b6bef3…f2ed9b INFO [06-21|17:01:52.437641] Mining too far in the future wait=2s INFO [06-21|17:01:54.441536] will prepare the block INFO [06-21|17:01:54.441837] will Finalize the block INFO [06-21|17:01:54.441891] Commit new mining work number=3 txs=0 uncles=0 elapsed=2.004s INFO [06-21|17:01:54.442025] will Seal the block INFO [06-21|17:01:54.442176] Successfully sealed new block number=3 hash=1d3448…e8ec87 INFO [06-21|17:01:54.442574] ? mined potential block number=3 hash=1d3448…e8ec87 INFO [06-21|17:01:54.442616] will prepare the block INFO [06-21|17:01:54.442739] will Finalize the block INFO [06-21|17:01:54.442757] Commit new mining work number=4 txs=0 uncles=0 elapsed=144.138µs INFO [06-21|17:01:54.442807] will Seal the block
Let’s check the height of blockchain :
> web3.eth.getBlockNumber(function(e,r){ console.log(r)}) 17
Wow in a matter of seconds 17 blocks got generated. Now we have our own private Ethereum running locally with our own consensus algorithm, That’s amazing. Now we can implement logic into consensus algorithm.
Before we started implementing our own algorithm I would like to highlight one more insight of the code. Ethereum service (backend.go) contains a miner which orchestrate two very important module “worker” and “agent”.
The role of the worker is to generate blocks or work. It collects all pending transaction and makes them part of the block. Please note that here ‘generate block’ does not mean that it mined the block, it simply means we are initialising the block structure and doing some preliminary validations or simply it generate the work. Worker first calls engine’s Prepare function, in case of ethereum’s standard algorithm (ethash), block difficulty is calculated. Then it calls the engine’s Finalize method.
Role of an agent (cpuAgent) is to do work generated by the worker. So the agent actually mines the block, by finding correct nonce in case of POW. The communication between the worker and agent is done via event. When the worker generates a block it generates an event which agent is listening to. Once agent mine the block, it generate the event which worker is listening to and worker starts creating new work for an agent to work on. This loop continues.
For this tutorial, I am planning to implement a very basic consensus algorithm. The details of which are as follows :
At Miner side
– There is a file called problems.json, which contains 10 problems to be solved by the miner. These problems are a simple arithmetic equation like “3 + 14 * 49”.
– Miner, while sealing the block, will first select the problem from problems.json based upon index which will the first character of current block’s parent hash. So if current block’s parent hash is
“0x29c9f33844df1df9d808412104f08dd318c016f7f06faad241fb0f8c79911c75” then the index to pick problem will be ‘2’ ( ignore 0x which represents hexadecimal),
“0xc4fbd0b2c6bc524794fbff7af77eb0d98ad42aa4c6a34776bc66210502e908e2” then the index to pick problem will be ‘2′ ( c in decimal is 12, so 12%10 = 2)
– Once the miner finds the problem, he will try to solve the problem and once the solution is found he put the solution in ‘nounce’ field of the block.
At Validator side
– Whenever the validator node got a block, it first selects the problem from “problems.json” using block’s parent hash as described above.
– Validator computes the solution and compares its solution with the Nonce.
– If both are equal, validator put the block in his chain, else he rejects the block.
Run Ethereum Local cluster
To run multiple nodes we just need to specify different ‘data-dir’ and different ‘rpc-port’. So to initialise node 1
./geth --datadir ~/.ethereum/myprivatenet-1 init privategensis.json
To initialise node 2
./geth --datadir ~/.ethereum/myprivatenet-2 init privategensis.json
Similarly, you can initialise as many nodes as you want.
To start node, you need to mention previously created ‘data-dir’ and port. So to start node 1
./geth -rpc -rpcapi 'web3,eth,debug,personal' -rpcport 8545 --rpccorsdomain '*' --datadir ~/.ethereum/myprivatenet-1 --networkid 15
To start node 2
./geth -rpc -rpcapi 'web3,eth,debug,personal' -rpcport 8546 --rpccorsdomain '*' --datadir ~/.ethereum/myprivatenet-2 --networkid 15
As of now, these two nodes are running independently without knowing each other. So to connect node1 to node1, go to geth console and find node1 enode address by issuing following command:
node1 > admin.nodeInfo.enode
This will return enode address of the node1 like
node1 > "enode://c7a9acd7a6f381bb33a51767e07989939448fae4db2ec95cc736f88bebd39c6fce1e75b333225b435a0398ff7377805b08fb8792e6a0886d855ff315e54d5cf6@[::]:30303”
Go to node2 geth console and add node1 using its enode address as follows
node2 > admin.addPeer("enode://c7a9acd7a6f381bb33a51767e07989939448fae4db2ec95cc736f88bebd39c6fce1e75b333225b435a0398ff7377805b08fb8792e6a0886d855ff315e54d5cf6@[::]:30303”)
You can check if the connection is successful or not by issue ‘net’ command. This command will return the count of connected nodes.
> net { listening: true, peerCount: 1, version: "1", getListening: function(callback), getPeerCount: function(callback), getVersion: function(callback) }
The complete code is available here: Hope I am able to give you a taste of Ethereum internals and you must be equipped enough to hack it further.