Blockchain Conensus

How to build global consensus

BUILDING CONSENSUS

The challenge with any system is communication. It becomes tougher with millions of users doing millions of transactions simultaneously. To solve this problem, we need to find consensus.

TYPES OF NETWORKS

What are centralized systems? Centralized systems are systems that use client/server architecture where one or more client nodes are directly connected to a central server. This is the most commonly used type of system in many organizations where a client sends a request to a company server and receives the response.

What are decentralized systems? Decentralized means that there is no single point where the decision is made. A decentralized system in systems theory is a system in which lower level components operate on local information to accomplish global goals.

What are distributed systems? A network of independent nodes, each representing a “process” with the purpose is to accomplish a common goal. There can still be a potential failure of individual components. The correctness of a distributed consensus is defined as achieving one’s goal.

How are Decentralized systems similar but different from Distributed systems? Distributed means that the processing is shared across multiple nodes, but the decisions may still be centralized and use complete system knowledge.

Combined, distributed building and decentralized systems can deliver collaborative problem solving. These methods differ in how they build communities and how they would handle potential attacks. Behind every great blockchain there is a great consensus algorithm

BLOCKCHAIN NODES

Why do blockchains need to agree? For decentralization to work, there has to be a way for the network to maintain its integrity. Everyone has to be assured that all transactions are valid and that no one on the network is cheating by double spending or reversing transactions. The process of everyone on the network agreeing that transactions are valid in the absence of a central authority is known as “achieving consensus.” It is the network nodes that achieve this consensus among users, helping to make the blockchain secure. {{word-count}}

What Is a Blockchain Node? A node is generally a point of intersection or connection in a telecommunications network. A node may also mean any system or physical device that is connected to a network and can execute certain functions like creating, receiving or sending information via a communication channel. The explanation of a node varies depending on the protocol layer being referred to.

For example, a basic resident network may consist of a file server, two laptops and a fax machine. In this case, the network has four nodes, each equipped with a MAC address to uniquely identify them.

The most popular usage of the term “node” is seen in the blockchain space. In this guide, we will explain what nodes are in more detail, including the different types of blockchain nodes being used today.

Blockchain nodes refer to a network’s stakeholders and/or their devices, which are designated to keep a copy of the distributed ledger and serve as communication points that execute various essential network functions. A blockchain node’s main purpose is to verify the validity of each succeeding batch of network transactions, called blocks. Each node has a unique identifier attached to its device that allows it to be distinguished from others in the network.

A proof-of-work (PoW) blockchain like Bitcoin (BTC) or Monero (XMR) incorporates miners, which are tasked with the following.

However, only “full nodes” are required to keep all of the blockchain transactions on their devices. These types of nodes are responsible for validating blocks and transactions. Lightweight or light nodes, on the other hand, have minimal storage requirements as they only need to download block headers in order to verify transactions. Neither of these versions of a full node necessarily come with a block reward.

Another way to classify a blockchain node is its availability. For instance, a node that is designated to continuously send updates throughout the network and always be online is considered an “online node.” In contrast, offline nodes are only required to download an updated copy of the ledger each time they reconnect to the network in order to ensure that they are synchronized with every other node.

People might choose to run full nodes for a variety of reasons, including increased privacy or a desire to support their network of choice. Lightweight nodes and full nodes alike come with wallets that can be used for making cryptocurrency transactions. Full nodes provide greater privacy, as outside observers have a hard time distinguishing between transactions being processed by the node and transactions sent by the person running the node.

7 Types of Blockchain Nodes: While there are several types of full nodes, there are also lightweight nodes. Below, we’ll highlight both lightweight and full nodes.

  • Light Nodes: Lightweight nodes or “light nodes” do not hold full copies of the blockchain. Light nodes only download blockheaders, saving users significant download time and storage space. Nodes of this nature depend on full nodes to function and are used for simplified payment verification (SPV).

  • Archival Full Nodes: Most often, when someone uses the term “full node,” they are referring to an archival full node. This is the primary node type that forms the backbone of a blockchain network. Archival full nodes are servers that host the entire blockchain, with every single transaction recorded in their databases. The main task of these nodes is to validate blocks and maintain consensus. Archival nodes can be broken down further into two subcategories: nodes that can add blocks to the chain and those that cannot.

  • Pruned Full Nodes: A pruned full node is one that saves hard disk space for its users by “pruning” older blocks in the blockchain. This type of node will first have to download the entire blockchain from the beginning. After that, it will begin deleting blocks beginning with the oldest and continue until the node only holds the most recent transactions up to a set size limit. If a node operator were to set the size limit to 250 MB, then a pruned full node would hold the most recent 250 MB worth of transactions.

  • Mining Nodes: In crypto mining, miners are either full or light nodes that try to prove they’ve completed the work required to create a new block. This is where the term “proof-of-work” comes from. To accomplish this task, miners have to either be an archival full node themselves, or get data from other nodes to learn the current status of the blockchain and how to work on finding the next block. (Those who seek to run mining nodes might want to take into account crypto mining electricity costs.)

  • Authority Nodes: Authority nodes are used by consensus algorithms for networks that aren’t fully decentralized, including Delegated Proof of Stake and Proof of Authority. In these networks, either the development team will decide how many authority nodes are needed and who will run them, or the community could vote for the decision. The task of these nodes is the same as full nodes in other networks.

  • Masternodes: Masternodes cannot add blocks to a blockchain. They only serve to validate and record transactions. Running a masternode can earn users a share of the network’s rewards. Doing so requires first locking away a certain amount of money in the form of the network’s native token. DASH is an example of a network that uses masternodes.

  • Lightning Nodes: Lightning nodes don’t quite fit the mold of any of the nodes discussed so far. The main idea of a lightning node is to establish a connection between users outside of the blockchain, enabling what are referred to as “off-chain transactions.” This reduces the load on the network and allows for much faster and cheaper transactions. Bitcoin lightning transactions typically cost 10 or 20 satoshis, or the equivalent of a fraction of a penny.

How Does a Blockchain Work? Before we move on to discussing blockchain nodes, it is important to understand how blockchains work first. Blockchain is a class of distributed ledger technology (DLT) that functions as a decentralized digital ledger of transactions that is designed to copy itself to multiple devices within a network. This means that a chronological and complete record of every network transaction gets to be distributed to a series of devices. These devices are called nodes and they are responsible for keeping a blockchain fair, secure and immutable.

How to Run a Node? Being a node operator on any particular blockchain has several system requirements. Ensuring that you have the proper node setup is crucial for your node to be fully functional and synchronized to the network you aim to serve. As mentioned earlier, you can refer to each blockchain’s respective websites to check hardware and software requirements, as well as setup guides.

What does a Node do? Depending upon the Blockchain, there can be innumerable roles based on certain tasks but the basic functions of a node come down to: 1) Accepting or rejecting transactions; 2) Managing the transactions and their validity; 3) Storing the cryptographically linked blocks; and 4) Acting as a point of communication.

Why we need nodes? Nodes are essential to a blockchain network’s operation since they keep every participant honest and ensure data reliability. Most blockchain networks employ methods to incentivize users to run full nodes, mostly through monetary rewards like mining or staking. Regardless of the incentives, there are also users who voluntarily set up their own full nodes simply because they believe in a project’s future and want to support and protect it as much as they can.

However, keep in mind that running a full node incurs costs and risks. And although there are multiple guides available online, the process of setting them up can be too technical for those who are unexposed to blockchain and programming. Basically, it all comes down to how much time and effort you are willing to invest in this endeavor.

Node recap: A node is one computer in a network of many that follows rules and shares information. The term “node” is sometimes used interchangeably with the term “full node,” but they are not the same. A “full node” is a computer in the Bitcoin network that stores and synchronizes a copy of the Bitcoin network’s entire blockchain history. Full nodes are important for several reasons, not the least of which being that they vote on proposed changes to the network. When more than 51% of full nodes don’t agree on a proposal, it gets skipped. Sometimes this leads to a hard fork, as was the case in 2017 with the Bitcoin Cash fork. {{word-count}}

CORRECTNESS = CONSENSUS ALGORITHM

What is Consensus? Consensus refers to the rules by which a blockchain network operates and confirms the validity of information written in blocks. Confirming this information can be complicated with large networks involving large numbers of people, hence the need for a consensus algorithm.

To ensure correctness, one uses a consensus algorithm achieving the following:

  • Validity: any value decided upon must be proposed by one of the processes

  • Agreement: all non-faulty processes must agree on the same value

  • Agreement and validity are safety properties: honest nodes will never decide on trivial, random, or different value

  • Termination: all non-faulty nodes eventually decide

  • Termination is a liveness property: all nodes eventually decide on a value

How did the the original consensus algorithm work?

The original consensus algorithm is Bitcoin’s proof-of-work (PoW) algorithm. Proof-of-Stake (PoS) is another popular consensus algorithm that works somewhat differently but seeks to achieve the same goal. Many DeFi protocols utilize PoS. Both algorithms rely on full nodes for the validation of transactions and enforcement of network rules.

Anyone can download the entire Bitcoin blockchain and validate blocks. This increases both the security and the decentralization of the network, as more copies of the ledger come into existence and can be referenced by others. Bitcoin nodes can be run by anyone in the world with the proper hardware and an internet connection.

What is a Hashing function? A hash is a function that meets the encrypted demands needed to solve for a blockchain computation. Hashes are of a fixed length since it makes it nearly impossible to guess the length of the hash if someone was trying to crack the blockchain. The same data will always produce the same hashed value.

What is a Cryptographic hash function? A hash function with three special properties: Preimage resistance, Second preimage resistance, Collision resistance. This ensures trust in a trustless environment. Hash functions = mathematical fingerprint.

  • Preimage resistance: Whose fingerprint is this?

  • Second preimage resistance: Can you find someone with the same fingerprint as you?

  • Collision resistance: Can you find two random people with the same fingerprint?

What are digital signatures? Recipients given the (message, signature) pair should be able to verify:

  • Message origin: original sender (owner of private key) has authorized this message/transaction

  • Message integrity: message cannot have been modified since sending

  • Non-repudiation: original sender (owner of private key) cannot backtrack

What makes valid transactions? Proof of ownership (a signature), Available funds, No other transactions using the same funds

What is a Hash puzzle? A Hash puzzles need to be: 1) Computationally difficult 2) Parameterizable 3) Easily verifiable

What is a Validator?

At a high level, a validator is used to approve a transaction that has been submitted by a user or blockchain client. A transaction is simply data that has been submitted to be added to a blockchain and can include anything the blockchain was created to be used for. The most known example is a record of a cryptocurrency payment but can also include ledger data such as supply chain tracking information.

The logic that is used to run the validator in the blockchain are called transaction processors. These processors are coded in the appropriate blockchain class and called when transaction parameters need to be checked for completeness. If all parameters are accepted, the validator will then pass the transaction to a Node which will add it to the blockchain.

Nodes are computer systems that use their computational power to confirm these transactions and act as the physical hardware validation of the process. After the validator code uses logic to determine if a transaction should be approved, the heavy processing power needed to update the Blockchain Merkle Tree is completed by these Nodes. Miners typically are the users that run Nodes as there is no other incentive to run the software on the computer. Miners get rewards in the form of fractions of a block reward (such as BTC) for their efforts of using their computational power. source

What is a Smart Contract? Code that facilitates, verifies, or enforces the negotiation or execution of a digital contract. A trusted entity must run this code. Ethereum empowers developers to design and implement their own game theory systems in the form of smart contracts. Smart contracts render transactions traceable, transparent, and irreversible.

A smart contract is a self-executing contract with the terms of the agreement between buyer and seller being directly written into lines of code. The code and the agreements contained therein exist across a distributed, decentralized blockchain network. The code controls the execution, and transactions are trackable and irreversible.

Smart contracts permit trusted transactions and agreements to be carried out among disparate, anonymous parties without the need for a central authority, legal system, or external enforcement mechanism.

What is a Merkle tree? A tree constructed by hashing paired data (the leaves), then pairing and hashing the results until a single hash remains, the merkle root. In Bitcoin, the leaves are almost always transactions from a single block.

What is a Nonce? A nonce ("number only used once") is a number added to a hashed block that, when rehashed, meets the difficulty level restrictions. The nonce is the number that blockchain miners are solving for. What is the nonce range?

What is a Merkle root? The root node of a merkle tree, a descendant of all the hashed pairs in the tree. Block headers must include a valid merkle root descended from all transactions in that block.

HOW TO FIND CONSENSUS

What is the Byzantine fault tolerance? Describe the generals attack story: The problem is that several Byzantine generals and their respective portions of the Byzantine army and have surrounded a city. They must decide in unison whether or not to attack. If some generals attack without the others, their siege will end in tragedy. The generals are usually separated by distance and have to pass messages to communicate. The practical Byzantine fault tolerance algorithm (PBFT) solves for high throughput, scalable, low cost, and is semi-trusted.

How do we ensure that all nodes make the same decision? Nodes can send incorrect/corrupted values.

What is the Byzantine agreement? In a distributed system, a quorum is a set of nodes sufficient to reach agreement.

What if you don’t necessarily trust certain nodes in the quorum? How can the we still achieve consensus?

Decentralized control: no central authority that authorizes consensus

  • Low latency: consensus achieve in a few seconds

  • Flexible trust: nodes choose who they trust, don’t have to trust the entire network

How do we determine quorums in a decentralized way? Solution: introduce quorum slices. Subset of a quorum that can convince one particular node of agreement. Individual nodes decide on other participants they trust for information

The Proof of Stake (PoS) algorithm now used by Ethereum. Attacks are more expensive, more decentralized, energy efficient

What is Proof of stake?

Alternative Consensus algorithm where voting power is directly proportional to economic stake locked in the network.

In Proof-of-Stake, you can specific properties you want to maintain a “defender’s advantage”. Introduce penalties, Punish malicious behavior much more greatly than in POW but Security comes from putting up economic value-at-loss. Discourage malicious behavior with explicit consequences.

What are the drawbacks of Proof of stake? Liquidity problem as funds are locked up, Low participation can cause centralization, Can rewrite history of blockchain if someone with a huge stake sells private keys and the rich get richer: Richest participants given easiest mining puzzles in pure proof-of-stake

The Proof of Work (PoW) algorithm used by Bitcoin. It has slow throughput and is expensive to mine. Reward subsidies will eventually end and miners will eventually receive only subsidies. System limited to 21m bitcoins.

How to use PoW? What is the PoW process? Everyone accepts valid transactions. How do we ensure no one is cheating? Proof of work requires CapEx (buy ASICs and other equipments) and OpEx (power and real estate cost etc) to participate in securing the network. Those are realized costs, whether or not the miner is successful.

What are the drawbacks of proof of work? In POW, there is no defender’s advantage: The cost of attack and the cost of defense are 1:1. The constraints are inflexible (i.e. computation, electricity) There is nothing in place to prevent or discourage malicious actors. Proof of Work simply allows it.

How does Proof of work validate peers? Instead of siloed decisions, PoW has proposers and voters. The proposer submits a transaction to everyone else. Peers cast votes. Only save after receiving a certain number of votes.

Consensus is critical in the online and offline world. Blockchains solve this by using math as a single source of truth. It's important to understand how different chains operate, ranging from central to decentralized nodes.

Economic incentives determine everything.

To build a trustless system, you need to incentivize for the truth. Remember all third parties are loopholes. And decentralized require everyone to be on the same page.

What is Blockchain mining?

Mining begins and ends with incentives. In this section, we'll break down the different types of miners, how they interact and the economics of mining incentives.

How does cryptocurrency mining work?

Mining updates the blockchain. Transactions are bundled into blocks and verified. New blocks are appended to the blockchain. Mining introduces new coins to the network. The mining process decentralizes the coin minting process with no central party issuing out coins/currency.

Mining Pools and Incentives

What are the incentives for mining and transaction fees?

If the Genesis block is the first block of the blockchain, then Proof of work is the amount of computational work required to build the next block. Parent block is the block that immediately precedes the next block

Miners can cheat with invalid transactions, add blocks randomly without proof, mine on top of invalid blocks

Mining solves the problems with double spending

Double spend: successfully spending the same money more than once

Tom promises ten bitcoins (BTC) to Billy in one transaction and promises ten bitcoins (BTC) to Ted in another, but he only has ten bitcoins (BTC) in total. Tom is performing a double spend attack.

When Tom attempts to double spend, he will be rejected by observing peers. Peers vote “no” on his proposal, as they notice multiple transaction trying to spend the same funds.

Miners are rewarded with a certain number of bitcoins for finding each successful nonce; ethereum adds punishment because it requires stake in mining, hence, strengthening the protocol security features.

Limited number of transactions per second

How do miners verify transactions? Miners verify new transactions.

A full-fledged Bitcoin miner (“Full node”) must:

  • 1. Download the entire blockchain to store the entire transaction history

  • 2. Verify incoming transactions by checking signatures and confirming the existence of valid bitcoins

  • 3. Create a block using collected valid transactions

  • 4. Find a valid nonce to create a valid block header

  • 5. Hope that your block is accepted by other nodes and not defeated by a competitor block

  • 6. Profit!

How to mine the proof of work problem?

Proof of work problem: costly and time consuming to solve, easy to verify. The Goal is to find the hash of a valid block header with value less than a target value.

What is the proof of work problem? Think of mining as a massive competition. A huge game of hungry, hungry hippos (a random process). The faster a hippo chomps the more likely it is to win. Calculating hashes faster increases chances of winning.

What are the Mining incentives - Block rewards? Miner receives BTC for every confirmed block. Miner includes special transaction to self. An profit incentive for honest behavior. Mining halves every 210,000 blocks. The BTC supply cap is 21,000,000

What are Mining incentives – transaction fees? Tx creator sets tx fee (Voluntary but necessary). Extra income for miners on top of block reward (Incentivizing as reward diminishes). Higher transaction fee => faster confirmation time. TX_Fee = Input – Output When block reward becomes 0, TX fees will become primary source of revenue for miners

How are Mining incentives done? GPU mining, Field Programming Gate Arrays (FPGA mining) and Application-Specific Integrated Circuit (ASIC mining).

What are the Variable Operating Costs with Mining incentives? The energy consumed in mining comes from powering your hardware and cooling to maintain your hardware.

What are Mining pools? Mining pools allow individual miners to combine, or ‘pool’, their computational power together to reduce variance in mining rewards, are run by pool managers or pool operators and the pool manager usually takes a cut of the mining rewards.

What is censorship resistance and why is it important? Discuss how China and other nations have done this well.

What are the mining scalability problems? Bitcoin blocks are created every 10 minutes and can only hold 1 MB of transactions. In 2015, blocks begin to run out of space, transactions go unconfirmed. The Block Size Debate raises questions about decentralized governance

ECONOMIC INCENTIVES

How to align Blockchain Incentives? Early adopters usually benefit while late adopters usually lose. No VCs or investors to demand their money back. No obligation to pay back token holders.

  • Tokens: units of a protocol-defined cryptocurrency given out to miners (i.e. block rewards)

  • Privileges: decision making rights they can charge for (i.e. transaction fees)

What is the Double edged sword?

Lack of regulatory oversight leads to fraud

Imbalanced power between founders and investors

Control premium results in centralization

What are public and private keys? Security is hidden in plain sight. Nearly impossible to guess with random generation. Each entity is represented with a unique public key. A corresponding private key acts as a key to “unlock” the public key. Private key chosen at random, public key generated from private key. Public key for receiving, private key for redeeming.

Identity in daily life:

  • Houses have addresses and mailbox keys

  • Emails have aliases and passwords

  • Bitcoin has public keys and private keys

How do Proofs work in Blockchain Economics?

  • The Proof of Work (PoW) algorithm used by Bitcoin. It has slow throughput and is expensive to mine. Reward subsidies will eventually end and miners will eventually receive only subsidies. System limited to 21m bitcoins.

  • The Proof of Stake (PoS) algorithm now used by Ethereum]]. Attacks are more expensive, more decentralized, energy efficient

Other Types of Proofs

Proof-of-History

  • Proof of History is a sequence of computation that can provide a way to cryptographically verify passage of time between two events. It uses a cryptographically secure function written so that output cannot be predicted from the input, and must be completely executed to generate the output.

Proof-of-Activity

  • Resources Consumed: POW and POS resources (hybrid)

POW: Miners solve and submit block headers to the system (with or without transactions, depending on implementation)

POS: validators vote on valid blocks, receiving awards for signing

Proof-of-Burn

  • Overview:

  • Send coins to irretrievable address (more coins burned, higher likelihood of voting)

  • Like Proof-of-Stake Bootstrapping mechanism:

  • Tie a coin’s value to some other coin (i.e. Bitcoin) by demonstrating that users willingly burn Bitcoin to receive this other coin

  • Resources consumed: currency (potentially not native)

Proof-of-Space

  • Use disk space to solve challenge

  • Nowadays used for file storage

  • Resources consumed: storage space

  • Variations on implementations:

  • Let users receive coins after voting on blocks (a.ka. Proof-of-Capacity)

  • Give rewards directly for storing information (i.e. Filecoin)

Proof-of-Elapsed-Time

  • Overview:

  • Spend time instead of mining power

  • Ask to wait some amount of time decided by the machine, wait for an attribution that it has indeed waited that much time. Let the first person to complete this random wait wain.

  • Based on assumptions of randomness and trust in manufacturers Resources consumed: time

GAME THEORY

What is the Game theory behind competitive mining? Bitcoin miners competing, proving they produced a specific block, as well as half the network having to agree on the next block, is the first game theory to work in the context of digital currencies. This has worked well for seven years. Three components: players, strategies, payoff. Two types of games: zero sum and non-zero sum

How does the Prisoner’s Dilemma work?

The two suspects are questioned in different rooms. The police offer the following deal to each of the suspects:

  • 1. If none of the suspects confesses his crime then they both will be sentenced to just 1 month in jail for a lesser crime.

  • 2. If one suspect confesses his crime and other doesn’t then the one who confesses will get 0 months jail while the one who denies will get 12-month jail.

  • 3. If both the suspects confess their crime, then both will get 8 months jail each.

What would you like to do?

  • 1. Collude

  • 2. Defect

What is Nash's equilibrium?

  • If pool A choose to attack pool B, pool A gains revenue, pool B loses revenue

  • Pool B can retaliate by attacking pool A and gaining more revenue

  • Attacking is the dominant strategy in each iteration

  • Therefore if both pool A and pool B attack each other, they will at a Nash Equilibrium

  • Both will earn less than they would have if neither of them attacked.

You now know the resiliency of a blockchain proof. Good actors will outperform bad ones. A well made system is designed to be fail safe.

Back to blog