Blockchain: why have permissionless blockchains to be slow?

This post is the seventh post in a series dedicated to demystifying blockchains. The previous post introduced the permissioned blockchains based on Byzantine Fault Tolerant (BFT) protocols.  BFT-consensus is faster than lottery-based consensus.  This post attempts to explain why lottery-based consensuses are inherently slow.

First, we need to explain the concept of a temporary fork.  As the consensus is a lottery, sometimes two nodes win at the same time.  The consequence is that at a given time there are two different instances of the blockchain, so-called forks.  Depending on the spreading speed of the information of the wining block and the number of validation nodes, the two forks may grow independently.  The consensus algorithm decides that the longest fork wins.  The shorter fork is abandoned.  The transactions of the “abandoned” fork that were not approved in the winning fork are returned to the pool of “not yet approved” transactions.  This strategy is the reason why Bitcoin approves a transaction finally only once there are at least five following blocks in the chain.

Temporary forks are inherent to lottery-based consensus protocols.  BFT-based consensus protocols do not have temporary forks because their consensus is deterministic.  All the faithful nodes take the same decision.

For the sake of completion, there are two other types of fork: hard forks and soft forks.  They are due to the evolution of the protocol and are independent of the consensus mechanism.  Thus, they are out of the scope of this post.  We will come back to these forks in a future post.

Let us use Bitcoin to illustrate the issue.  Bitcoin validates a block every ten minutes (average time).  The difficulty of the Proof of Work drives this duration.  (see Blockchain: Proof of Work).  As a block holds a maximum number of transactions, this average time of validation defines the blockchain’s throughput.  Currently, Bitcoin’s capacity is about seven transactions per second.

How to increase this throughput?  An “obvious” solution seems to decrease the average validation time by reducing the difficulty of the cryptographic puzzle.  For instance, should the average duration be halved, then the throughput would double, isn’t it?  Unfortunately, it is not so simple.  If we reduce the difficulty of the challenge at constant global calculation power, the duration is reduced.  Another consequence is that the probability to solve the puzzle rises.  The easier the challenge, the higher the likelihood to solve it.  Thus, the probability to create a temporary fork increases also.  The shorter the average validation time, the more temporary forks.  Should the duration decrease drastically, the system would end up with a pandemonium of temporary forks.  Unfortunately, abandoned temporary forks inject back many “seemingly-approved” transactions in the pool of “not-yet-approved” transactions.  In other words, the global throughput is decreasing, and the predictability of when a transaction may be approved finally is impacted.  Of course, Sato understood this crucial aspect of his protocol and chose ten minutes rather than one minute.

Conclusion: Lottery-based consensus protocols need to have “long” approval duration to avoid the proliferation of temporary forks.

Blockchain: Consensus based on Byzantine Fault Tolerance

This post is the sixth post in a series dedicated to demystifying blockchains. The last post introduced the Proof of Stake. The third post introduced the concept of consensus and classified them into two categories: lottery-based and Byzantine Fault agreement-based protocols. This post presents the most known Byzantine Fault agreement-based consensus: Proof of Work

Byzantine Fault Tolerance is known for about four decades. A Byzantine Fault is any failure of a system. The failure may be that the system is not working, or the system is behaving maliciously. An f-Byzantine Fault-Tolerant network is a network that operates correctly under the assumption that f nodes may be concurrently experiencing Byzantine faults. The nodes are either well-behaving or ill-behaving, i.e., suffering from Byzantine Faults. The network comprises n nodes with a maximum of f ill-behaving nodes.

The best implementation of Byzantine Fault Tolerant system for asynchronous protocols is the Practical Byzantine Fault Tolerance (PBFT).  PBFT assumes that the size of the network is at least 3 f + 1.

At any given moment, one node is the primary and the other nodes are replicas. The simplified algorithm operates as follows:

  • A client sends a request to the primary node for an operation
  • The primary node broadcasts this request to all replicas
  • Replicas perform the requested operation and return an answer to the client
  • The client waits for at least replicas answering the same result. This is the consensus outcome of the operation.

The figure illustrates the basic behavior for the case of a 1-node fault tolerant system. It has at least four nodes. The client sends its request to the primary node. The primary node broadcasts the request of operation to the three other nodes: replica 1, replica 2 and replica 3. As soon as the client receives an affirmative answer from two replicas (in the figure, replica 1 and replica 3), it considers the operation successful. If one replica is failing, the client will still receive proper answers from the two other replicas. If the primary node is failing, then the protocol switches the role of primary to one replica.

The PBFT is well adapted for fixed networks with known trusted entities. As such, it may be useful for blockchain validated by a set of known, approved authorities, i.e., permissioned blockchains. The number of authorities defines the system’s resilience. The minimal size is four which allows for one ill-behaving authority. Each authority has the same level of trust and shares trust with the other authorities.

PBFT is straightforward and robust. The theoretical background is well studied, known for a long time and used by the community. The trust is not linked to the ownership of resources or stakes. It is fast and computationally efficient. There is no notion of mining. It is deterministic. A permissioned blockchain using PBFT can have a very high through output of transactions.

Unfortunately, the trust is not flexible. PBFT requires a pre-established list of collaborating trusted participants. Any new participant requires new settings. This adjustment cannot be dynamic. The consensus of permissionless blockchains cannot employ PBFT. By participants, we mean the validation nodes and not the users of the blockchain. The system does not need to trust the users. It is the role of the validation and consensus to screen bad transactions.

A known weakness is that the system is prone to Sybil attacks. An entity may attempt to register several participants that it controls stealthily. If the cheating entity controls at least f sibyls, then it controls the full system. Therefore, the initial trust in the participants is paramount. In most permissioned blockchains, the likelihood of Sybil attacks is negligible.

Conclusion: PBFT is a well-known, well-studied protocol. For decades, distributed systems have taken their decisions using it. It is fast and computationally efficient. PBFT is an interesting consensus protocol for permissioned blockchain. Some variants of BFT are also available.

Bibliography

M. Castro and B. Liskov, “Practical Byzantine Fault Tolerance,” in Proceedings of the Third Symposium on Operating Systems Design and Implementation, New Orleans, USA, 1999

Blockchain: Proof of Stake

This post is the fifth post in a series dedicated to demystifying blockchains. The fourth post introduced the most famous lottery-based consensus: Proof of Work (PoW). This post presents another type of lottery-based consensus: Proof of Stake (PoS).

With PoW, the entity that has the highest calculation power has the highest probability to be the elected validator of the block. With PoS, the entity that has the highest stake in the system has the highest probability to become the elected validator of the block. Each potential validator, so called forger, puts some collaterals in escrow. Would the elected validator cheat, then thisvalidator would loose the escrowed collateral. Obviously, the collaterals cannot exceed the validator’s owned wealth.

The security assumption is that the more stakes you have in a system, the more you are expected to defend this system in order not to lose your stake. Thus, the underlying hypothesis is that the highest stakeholders are acting rationally. Furthermore, like with PoW, there is an incentive to forge. The forger who validated a block is rewarded with some coins (and the transaction fees).

PPCoin implemented the first PoS. It uses the concept of coin age. The coin age is the number of coins multiplied by the holding period of these coins. If Alice holds ten coins for 20 days, then her coin age is 200 coin.day. The collateral is a number of coin ages bidded for validating the block. Indeed, Alice bids n coin ages by paying herself these n coins. The forger must solve a kind of PoW, but the challenge is inversely proportional to the collateral, i.e., the space of search decreases when the collateral increases. Thus, the winning forger is most probably the forger with the highest collateral.

Buterin Vitalik, the founder of Ethereum, introduced Casper: a PoS based on Byzantine Fault Tolerance. Casper has a set of volunteer validators. Each validator commits a number of ethers as a deposit. Casper defines an epoch as a set of 100 consecutive blocks. Roughly, Casper validates an epoch once a number of validators whose aggregated deposit represents more than 2/3 of the total deposit validated the epoch. The assumption is that if less than 1/3 of the validators cheated, they would be detected and identified. In that case, the community would remove their escrowed collateral through a hard fork.

The main advantage of PoS is that it consumes far less computing power than a PoW-based system. Usually, PoS has also less latency than PoW. The main latency is for the minimal time that collaterals are escrowed.

According to me, the trust model of PoS is weaker than the trust model of PoW. An ill-behaving majority stakeholder may attempt to use her advantage to take an even more significant stake. Furthermore, the stakeholders are expected to act rationally. This assumption may be weak in case of a state-funded attack which may have objectives other than monetary ones.

Even with a rationale player, there may be some weakness. If the gain of a misapproved transaction exceeds the loss of the escrowed collaterals, it may be a rationale decision to cheat. For instance, PoS is prone to “Nothing at Stake” attack. Alice may have a significant stake at time T0. She cashes all her stake at time T1. At T1, she has no more stake but has gained some benefits. As she has nothing to lose anymore, she may attempt to attack the blockchain at T0 when she had the majority of the stake. She attempts to start a new branch from T0, using all her then available-stake as collateral. Having a higher stake, she has good chance to win and thus begin a new branch where she would own again the already spent stakes. Would she not win, she would have lost nothing. Of course, T0 and T1 must be close.

Conclusions

PoS is faster and less power consuming than PoW. As such, this type of consensus may overcome some of the issues of PoW. Nevertheless, its trust model seems weaker. I foresee that we may see new PoS protocols in the coming years.

Blockchain: Proof of Work

This post is the fourth post in a series dedicated to demystifying blockchains. The third post introduced the major concept of consensus and classified them into two categories: lottery-based and Byzantine Fault agreement-based protocols. This post presents the most known lottery-based consensus: Proof of Work (PoW). Most permissionless blockchains, such as Bitcoin or Ethereum use PoW.

The aim of a PoW algorithm is to validate a block in a distributed environment. The PoW is designed so that ill-behaved actors would need to possess the majority of the system’s computing power to validate a forged block, for instance, for implementing double spending. The PoW is information, so called the nonce in Bitcoin parlance, present inside the block to be validated. The verification of the PoW is simple and fast whereas its construction is difficult to compute. The validator must solve the cryptographic puzzle described in the first picture where

  • Bi is the block to be validated.
  • Target is a value determined by an authority to adjust the difficulty of the challenge. Hash is a cryptographic one-way function. One characteristic of a cryptographic one-way function is collision resistance. Hash is easy to calculate, but given a number y, it is computationally difficult to find x such as Hash(x)=y.

All validators (so-called miners in the Bitcoin ecosystem) try to solve this equation. Currently, for approved cryptographic one-way functions, such as SHA-3, the only known method to solve this equation is brute force attack, i.e., exploring all the possible values systematically. The first miner to solve the equation, i.e., to find a suitable PoW, is elected the winning validator. Furthermore, the winner receives a reward (currently 25 Bitcoins). The parameter Target defines the difficulty of the challenge. Bitcoin defines the minimum number of trailing zeros that the hash must have. This number depends on the expected average solving time and the global mining computation power. In the Bitcoin ecosystem, this value is adjusted every fortnight to have an average validation time of ten minutes. If the global processing power increases too much, the difficulty target adds some zeros. Two interesting consequences:

  • For a given block, there is not one unique PoW as the equation as many solutions. The winner is the first to find one solution. Two validators may find “simultaneously” a solution. The branch validation strategy addresses the corresponding problem. A future post will address branching.
  • The validation time of one block is not fixed. It depends on the “luck” of the miners.

PoW works because it requires a huge calculation power. An ill-behaving actor that would control the majority of computing power would also control the system. The higher the total processing power of an entity is, the larger the probability of this entity to become the validator of a block is. With the rise of mining pools, the assumption that no entity controls a significant calculation power weakens. A mining pool is an association that federates many miners who share the reward and transaction fees. Mining pools unbalance the odds. It is not unrealistic in tiny environments that a mining pool could control more than half of the total processing power. Therefore, PoW may only be suitable for large deployments.

Finding the PoW is, by construction, computing intensive. To gain efficiency mining usually employs either Graphical Processing Units (GPU) or specialized gear based on custom developed ASICs. This calculation implies a lot of energy consumption. There is no sanctioned estimation of the total energy consumption due to mining. Nevertheless, it is commonly accepted that it is huge. Many mining farms are located in regions where energy is cheap.

PoW is surely not green nor fast.

Conclusions

PoW favors the entity that invests the most effort into validation.

PoW is a robust consensus protocol under the assumption that no entity would control more than 50% of the total mining power. At the time of Bitcoin’s creation, this assumption was valid. Currently, six mining pools have each more than 10% of the total mining power (see https://en.bitcoin.it/wiki/Comparison_of_mining_pools). This assumption may become less robust.

PoW is a huge energy consuming protocol.

 

Blockchain: consensus protocols

This post is the third post in a series dedicated to demystifying blockchains. The second post described the difference between permissionless and permissioned blockchains. It also introduced the concept of validator. This post will present the notion of consensus.

A consensus mechanism or protocol ensures that all nodes of the system agree to a shared, approved state. In the framework of a distributed ledger or blockchain, a consensus mechanism enforces that all participants use the same state or version of the ledger. In other words, it is the mechanism that ensures that every entity agreed on the same transactions and that all copies of the ledger are identical. Consensus protocols are not new. They at the core of many distributed systems and mirroring systems. Nevertheless, permissionless blockchains introduced a new challenge. Not all participants to a permissionless blockchain may be behaving properly.

Two kinds of consensus

There are mainly two categories of consensus mechanism:

  • Lottery-based
  • Byzantine Fault agreement-based

The first category is sometimes called the Nakamoto-consensus in honor of the pseudonym of Bitcoin’s founder Nakamoto. The consensus mechanism elects the validator, i.e., the node that decides which is the next block to be appended to the ledger. The election is a lottery draw. The winner is the validator. Each new block requires a new draw. The selection through a lottery reduces the likelihood of an ill-behaving node to validate a forged block. The lottery does not necessarily follow an equiprobable distribution. Each mechanism has its own probability distribution favoring one given characteristics of the winner. Thus, each lottery-consensus has a different trust model. The Proof of Work (PoW), used by Bitcoin, is the most well-known mechanism. There are many other types such as Proof of Stake (PoS), Proof of Space, or Proof of Elapsed Time (PoET). Future posts will explore in details PoW and PoS.

The second category is based on Byzantine Fault Tolerant (BFT) system. BFT systems are designed to operate even if some participants in the protocol are failing. Failure may be involuntary (for instance, a participating node is out of order) or voluntary (for instance, an attacker controls the failing node). BFT employs voting mechanisms to decide the consensus. The used mechanism defines the trust model. It is usually well defined. The Practical Byzantine Fault Tolerant (PBFT) mechanism is the most well-known mechanism. A future post will explore PBFT.

Hybrid consensus mechanisms seem to appear mixing lottery with a pinch of BFT. Casper, the next consensus mechanism of Ethereum is a PoS with some BFT in it.

Lottery or Byzantine Fault Tolerant?

Lottery-based mechanisms are more complex and slower than BFT-mechanisms. Lottery-based consensuses are well fitted for permissionless blockchains. There is no control on the validators. Anybody may participate in the validation. Therefore, the lottery reduces the risk. In a permissioned blockchain, the validators are known. Thus, BFT-consensuses are adequate. Depending on the mechanism, the designer knows how many validators must be compromised or must collude to validate a forged block successfully.

Research on consensus is currently an extremely active field. Unfortunately, many consensus mechanisms are (too) young and their security has not been enough studied. The following chart illustrates relative age of major consensus mechanisms.

Conclusions

If you want to manage your blockchain, then you need to understand the corresponding consensus mechanism. It participates to the trust model of your solution.

Blockchain: Permissionless versus Permissioned

This post is the second one in a series dedicated to demystifying blockchains. The first post proposed a definition of blockchain. I intended that the topic of this second post would be consensus. The consensus is the cornerstone of blockchain. While starting to write it, I discovered that I needed first to introduce a fundamental characteristic of blockchain: permission.

Entities decide whether a block is valid and appended to the blockchain. They may be called blockchain nodes or validators. Validators are the pieces of software that determine which is the new block on the chain. In Sato’s vision, everybody could/should be a validator. Thus, his blockchain has no central authority. It is claimed that the blockchain is ruled by everybody (or nobody depending on your point of view). Bitcoin is a permissionless blockchain. This is the case for most cryptocurrencies and many other systems. Ethereum is another example of a permissionless blockchain. In a permissionless blockchain, users delegate their trust to uncontrolled, unknown validators under the assumption that the consensus mechanism does not allow a bad acting validator to cheat.

This delegation of trust is not always possible or desirable. Therefore, there is a second breed of blockchains that operate with a different configuration: permissioned blockchains. The validators are a set of finite known servers. A consortium manages this list following some defined governance rules. You may have noticed that the validators were not necessarily trusted. Depending on the chosen consensus mechanism, the level of expected trust may vary. The open source projects of Hyperledger offer many such permissioned architectures.

Which one is the best?

The advantage of the permissionless blockchain is that there is no (at least claimed) central authority. There is not a single point of failure that may be attacked. This advantage comes at a price: the consensus mechanism is complicated and/or extremely power consuming. It will have to be slow. Furthermore, it requires that the nodes have a robust method to validate a transaction. When managing financial ledgers, it is easy. Checking that Alice currently has the number of tokens she asks to transfer to Bob is straightforward. With more complex transactions, it may be less obvious. Would you trust an unknown validator to check whether your land deed belongs to you and to register it on a land registry blockchain? Or a copyright right? Smart contracts are not the golden answer to that issue.

The advantage of permissioned blockchain is that a set of entities that share a common interest in the fulfillment of the transactions can manage it efficiently. The validators have the authority and implement the complex validation rules that some use cases may be requiring. The consensus mechanisms are simpler and faster than the ones used by permissionless blockchains.

Many “purists” claim that permissionless blockchains are more secure than permissioned ones due to the absence of a central authority, arguing that the management of the validators is a weak point. As usual, the answer is more balanced. It mainly depends on the use cases. Some industrial use cases may benefit from permissioned blockchains. Personally, I would argue that the trust model of a permissioned blockchain can usually be more accurately defined than the trust model of permissionless blockchain. I have not yet read a convincing complete
convincing trust model of a permissionless blockchain.

Conclusion

Thus, a hyper-simplified definition: A permissionless blockchain does not trust nor know its validators whereas a permissioned blockchain knows all its validators but does not need to trust all of them.