Introduction

Knowing the main concepts of Blockchain is a good start. But before going into details on how to use this technology, it is important do understand better how this technology works. In this lesson, we give more details on the core of Blockchain: consensus protocols.

What is a consensus protocol?

In blockchains like Bitcoin or Ethereum, the nodes in the p2p network need to reach an agreement on the ledger’s current state. Consensus protocols 📚 help achieving the agreement or consensus in a trustless setting. To be able to secure the consensus in such setting, the protocol must achieve Byzantine Fault Tolerance 📚.

This security property is explained in this video 🎬 [04:09].

Consensus protocols ensure a total order over the blocks and prevent certain kinds of attacks by relying on both mathematical and economic hypothesis. Theoretically, an attacker could compromise consensus by controlling 51% of the nodes of the network. However, consensus protocols are designed to make this economically unfeasible (checkout this website that evaluates the cost of the attack for each blockchain: crypto51.app .

To learn more about the 51% attack, you can watch this video 🎬 [03:54].

What are the most famous consensus protocols?

If you want to dig into consensus mechanisms, you can have a look at this academic paper 📖 or this academic survey 📖

What is Proof of Work?

Like Bitcoin, Ethereum once used a Proof of Work (PoW) 📚 based consensus protocol. This protocol allows all nodes in the p2P network to agree on the current state of the blockchain, and secures the network against a variety of attacks. The nodes participating in this protocol are called miners and the execution of this protocol is called mining 📚. They are involved in the process of verifying and approving Bitcoin transactions, through solving complex mathematical problems using high-performance mining computers.

❗ Disclaimer: If you want to understand how Ethereum used to work with proof of work (reading which we recommend) stay on track. If you want an updated tour on Ethereum’s consensus, go directly to the next section.

Hashes & Puzzle

Understanding hashes is the first step in understanding mining. A hash will take an input of any length, and generate seemingly randomised output of a specific length (SHA-256 is a one-way function – every input string, no matter the size, transforms as a fixed 256 bit output, but the reverse is not possible). The same input will always generate the same output, but changing just one character will drastically change the output. For example, b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9 is the hash (SHA-256) of “hello world”, and 0fc30e735a0228a31cbbb969988b4f50e02e737f979f091d7d224b765443f5d4 is the hash of “hello worle”. You can try it using: cryptii.com (replace Enigma Machine with Modern cryptography/Hash function and configure the output to bytes and group by None).

This behaviour makes it very difficult to predict what input gives a particular output. To keep track of the map between an input and its hash, some lookup tables called rainbow tables are available online 📖. However it is nearly impossible to find a hash a specific pattern like for instance a hash with a single zero at the front. For example, you can try altering the last letter of “hello world” to get a hash starting with a zero. It will take you multiple attempts to finally get “hello worlz” which equates to 0309d44b424edadb9f3b16ba2a716619275e4eebd6cbbe0621e0aefd51bbf08d using SHA-256.

This pattern of hash starting with a zero creates a puzzle whose difficulty is measurable and which it’s impossible to perform better than blind guessing (brute force approach). This blind guessing is important because its creates a fair competition. Miners have to achieve a hash with a numeric value lower than a specified number. This number is called the target. If your hash attempt gives you a number less than the target, which is the same thing as having a bunch of zeros at the front of the hash, then you win and you get to “mine the block” (finds the appropriate hash). To find such a small hash takes millions of attempts, or more accurately, the whole mining network, with everyone trying at the same time, needs millions of billions of tries to get it right.

Miners have to solve such puzzles with a serie of zero based on a formula that evaluates the difficulty of guessing after a fixed amount of time (around 10 minutes for Bitcoin and 12-14 secondsfor Ethereum). Each validated block includes a reward in Bitcoins for the successfull miner. It’s not really that mining “generates” the Bitcoin in any sense, it’s just that it’s written into Bitcoin code that a transaction block starts with a unique transaction called a “coinbase” transaction, which is the only type of transaction with no inputs. It only has an output, consisting of the reward plus the transaction fees. So in exchange for finding the correct hash, miners earn both transaction fees and the block reward. The number of Bitcoins in that reward decreases over time, with the current block reward being 6.25 Bitcoin.

Note that computing all the hashes requires a huge investments in Mining hardware (using GPU 📖 or an ASIC 📚) and electrical energy 📖, which means you’re likely to end up spending more than you would earn. It is in the miners best interest to produce valid blocks, because if they are caught lying (which is very easy since verification of the certificate is easy) - they just wasted all that energy and computation for nothing and therefore lost their money and time. As such, solving the mathematical problem means the miner really, really wants to propose the new block, and is willing to spend energy and computation to receive the reward. To explore mining profitability, you can use an online calculator 📖.

Fork

Occasionally, two miners will produce valid blocks roughly at the same time. This can cause different nodes in the network to include different blocks in their blockchain. The technical term for this is a fork. However, for blockchains to proceed in a stable fashion, a single contiguous chain needs to be chosen as the “correct chain” to prevent the splitting of state.

Bitcoin and Ethereum use the “longest chain” rule to do this. Whichever chain is accepted by a higher number of nodes and continues to grow longer is chosen as the ‘correct chain’, and the forked chain is gotten rid of. Occasionally, this means that transactions which get mined as part of a temporary fork may be rolled back once the fork is gotten rid of in favour of a different longer chain. This introduces the concept of Finality.

The blocks which form the forked chain and end up getting deleted are called Uncle Blocks. The miner clearly put in work to produce that Uncle block, and likely just lost out on the mining reward due to network latencies. As such, the Ethereum network still rewards Uncle block miners with 1.75 ETH for their hard work.

A transaction has “finality” on Ethereum when it’s part of a block that can’t change. Because miners work in a decentralized way, two valid blocks can get mined at the same time. This creates a temporary fork. Eventually, one of these chains will become the accepted chain after a subsequent block has been mined and added, making it longer. But to complicate things further, transactions rejected on the temporary fork may have been included in the accepted chain. This means it could get reversed. So finality refers to the time you should wait before considering a transaction irreversible. For Ethereum, the recommended time is six blocks or just over 1 minute. After six blocks, you can say with relative confidence that the transaction was successful (more than 99.999% chance it will not be reverted now). You can wait longer for even greater assurances.

Ethash

The Ethereum proof-of-work protocol, Ethash 📖, requires miners to go through an intense race of trial and error. The process goes as follows:

• The miner selects a group of transactions to include in a potential block
• Based on the block they create, the network has rules to choose a slice of data (roughly ~1GB in size) from the current state of the blockchain network. These rules are not particularly relevant, but you can read more about them in the Ethash documenation 📖.
• They put the dataset through a hashing function to calculate a target value. This target is a number, which is inversely proportionate to the mining difficulty. The higher the mining difficulty, the lower the target, and vice versa.
• Then, the miner uses brute force to try to find another random number called the nonce.
• Putting the combination of the dataset, target, nonce, and a couple other values through a hashing function should result in a number that is lower than the target.
• HashFunction(dataset, nonce, ...) = a number
• The higher the mining difficulty, the lower the target, and hence the harder it is to find a nonce which satisfies this condition.
• Miners keep using trial-and-error to find a valid value for the nonce which satisfies the condition. There is no formula to calculate the nonce.

The mining difficulty becomes less or more difficult based on how many miners are on the network, to ensure that a block can be reliably produced roughly every ~15 seconds. If it becomes too easy and there are a lot of miners, blocks will be produced much faster than 15 seconds. Likewise, if it becomes too hard and there are not a lot of miners, blocks will take a long time to be produced. Difficulty is calculated by the network automatically.

There is no way to solve this other than brute force, as currently there does not exist a way to reverse hash functions. Therefore, to know if a certain nonce, when appended to the dataset results in a specific hash, the only way possible is to try random values for the nonce and check.

Take some time to read this section thoroughly, this is all there is to Proof of Work. It might seem quite anticlimactic, but think about what we started off wanting to prove. We wanted to prove that miners did hard work to produce this block, and it wasn’t a piece of cake for them. Since they did hard work, it is in their best interest to not lie. This sort of computation puzzle does exactly that, because finding a valid nonce is a computationally hard problem - but verifying it is easy, so other nodes can easily verify that the miner did in fact find a valid nonce, which means they did in fact put in the work.

As explained earlier, Ethereum once used the proof of work. Now, Ethereum runs on Proof of Stake 📚 with Ethereum 2.0 and “The Merge”. Let’s talk about this in the next section.

What is Proof of Stake?

To deep dive into how it works, we also suggest the following resources:

• Dan Boneh: Blockchain Primitives: Cryptography and Consensus 🎬 [56:36]
• PoS - Ethereum implementation Doc

Conclusion

In this lesson, we explained what is a consensus, proof of work is used in Bitcoin, proof of stake is used in Ethereum. In the next lesson, we will see how to interact with a blockchain using a cryptocurrency wallet.