Simple fuzzing goes a long way, even for critical blockchain software

Blockchain technology is used to store and transmit billions in value. Security issues in blockchain software puts this value at risk. One software underpinning billions in value is Parity, an Ethereum client written in Rust.

Through fuzz-testing Parity, the SRLabs team discovered a vulnerability that could be used to remotely crash Parity nodes prior to version 2.2.10. For a high-level description of the vulnerability and its implications, refer to this parallel blog post. Herein, we provide a deep technical description of the discovered vulnerability and how we found it.

Ethereum nodes ask other nodes for new blocks in the chain. Due to the decentralized nature of the Ethereum network, nodes receive information about the current state of the blockchain from other nodes. During this process, a node asks its peers if they have a “longer” version of the chain and, if so, to send the newer blocks. The receiving node then verifies each block (e.g., by confirming the proof of work is correct) and – only if valid – adds the block to its blockchain. Since synchronization requests can happen between any two nodes, verifying a block requires processing untrusted data – making the verification process an attractive hacking target.

The verification process contained a DoS vulnerability. The block verification function in the Parity Ethereum client contained an unsigned integer overflow vulnerability, which allows for a remote denial of service attack. Each block has a timestamp property and Parity Ethereum nodes only accept blocks with a timestamp in the past. To compare the block header timestamp with the actual system time, the function “verify_header_params” in the file “verification.rs” contains the following code:

let max_time = SystemTime::now() + ACCEPTABLE_DRIFT;
let timestamp = UNIX_EPOCH + Duration::from_secs(header.timestamp());
[…]
if timestamp > max_time {
return Err(From::from(BlockError::TemporarilyInvalid(OutOfBounds { max: Some(max_time), min: None, found: timestamp })))
}

The vulnerability in this code is located in the second line: When adding up UNIX_EPOCH and the attacker-controlled timestamp, an unhandled integer overflow allows for a denial of service attack on Parity Ethereum nodes since Rust’s standard library panics when the addition of two timestamps would overflow.

An attacker can send a malicious block with a very large timestamp value to trigger the integer overflow during chain synchronization. This will crash the node while it is verifying the malicious block.

The vulnerability can be abused via the Ethereum wire sync protocol. To trick the victim node into processing a malicious block, an attacker can use the Ethereum wire synchronization protocol to initiate a synchronization request. The attacker then announces the malicious block as a new block to the victim, triggering the verification process and crashing the node. Figure 1 describes this attack in detail.

Figure 1. A hacker can cause unpatched Parity Ethereum nodes to crash via Ethereum wire sync protocol.
Figure 1. A hacker can cause unpatched Parity Ethereum nodes to crash via Ethereum wire sync protocol.

Vulnerable Parity Ethereum nodes can be crashed remotely. Since every node must accept connections to stay synchronized with the network, the vulnerability allows an attacker to launch an easily scalable denial of service attack against vulnerable Parity Ethereum nodes currently active in the Ethereum network. Note that the crash occurs before the proof of work is checked, so there is no need for the attacker to mine a block to launch the attack.

The vulnerability is fixed by explicitly checking if the timestamp value overflows. The exploitation of this vulnerability is quite simple. And so is the fix: Simply check if the addition of the two timestamps UNIX_EPOCH and the header timestamp would result in an overflow and, if this is the case, reject the block. The fix is on Github.

The vulnerability can be identified by simple fuzzing. We found this vulnerability while auditing the block verification process in the Parity Ethereum client using a fuzzing harness for the block validation, which can be downloaded here. The fuzzing harness, which is written to work with honggfuzz, performs three steps:

1. Populate a blockchain object with a custom chain consisting of a few blocks
2. Try to decode the data output from honggfuzz as an rlp-encoded block. To exchange data with other nodes, Ethereum nodes encode the data using rlp – Recursive Length Prefix encoding
3. If the decoding was successful: Tries to insert the decoded block into the custom blockchain we initialized the blockchain object with. This triggers the verification of the block

We seeded this rudimentary fuzzing setup with a single rlp-encoded block, and after a few seconds, it found the crash.

Establishing more and continuous fuzz-testing for Ethereum clients will help increase their resilience and robustness by identifying vulnerabilities early in the software development process. For the moment, we should recall that blockchain technology is built on software and that software has bugs.

Note: For a more high-level overview of the implications of this vulnerability, take a look at the corresponding blog post. As explained over there, a significant number of Parity Ethereum nodes are still unpatched and vulnerable to the DoS attack described herein. (For this reason, we are not releasing a proof-of-concept at this point.) The patch gap raises further questions on Ethereum’s resistance to software bugs. It’s only a matter of time until an even more serious bug is found in some Ethereum implementation, putting value and trust at stake.