Fantom Virtual Machine & Storage: The Roadmap

Quantum Miami was a blast filled with exciting insights into the bright future of Fantom, including promising upgrades that will elevate the platform to new heights, such as the Fantom Virtual Machine.

In his presentation, Michael Kong, the CEO of the Fantom Foundation, outlined the projects that will lead to faster block processing on Fantom, and hence a more scalable network.

Let’s unwrap the presentation to see what’s in store for Fantom!

Blockchain’s scalability problem

Before we continue, let’s look at the inherent scalability issues with current blockchain-based systems.

The United States sees a staggering 100 million credit card transactions daily, with VISA reportedly handling 24,000 transactions per second, around 2 billion daily transactions. To reach these speeds, blockchain systems often trade scalability, decentralization, and security. However, the long-term goal for Fantom is to achieve all three goals at the same time.

The Fantom mainnet is a decentralized network run by nodes and uses an asynchronous and leaderless consensus mechanism; similar to other smart contract platforms, Fantom verifies blocks and transactions, executes smart contracts, and stores their states. The state of a smart contract is its current data at a particular point in time, such as its balance of FTM.

Smart contracts on Fantom store their state in a StateDB database containing the contract states. The Ethereum Virtual Machine executes the code of these contracts and updates their states in the StateDB.

Given that the current release of Fantom leverages Ethereum technology, including its storage system and virtual machine, the network is not optimized for scalability. Hence, identifying the exact bottlenecks that slow down Fantom is critical to upgrading the platform, which is the focus of the testing framework named Project Aida.

Aida: Identifying the bottlenecks

Aida is a testing framework for finding bottlenecks and testing. As part of this effort, we conducted an experiment for the first 40 million blocks of the mainnet and successfully identified block processing components dominating the runtime.

As illustrated in the chart, the EVM consumed 13% of the time on processing and executing smart contracts. The StateDB consumed 84%, spent on the EVM accessing and updating the state of the smart contracts.

The excessive time taken by the EVM to access the storage in the StateDB necessitates an improved storage system, which is the goal of Project Carmen.

Carmen: New storage system

Currently, the StateDB stores the current state of the blockchain in a key-value database (e.g., LevelDB, PebbleDB). The state is encoded with a complex data structure called Merkle Patricia Trie (MPT) that is used by Ethereum and Fantom’s mainnets. For example, a Fantom contract address could have a specific key that accesses the database entry for the contract’s total FTM balance, which is the value.

The currently-used key-value database with MPT results in slow StateDB accesses. To address this, Fantom came up with a radical new design for StateDB and has implemented a new file-based StateDB using ideas of address/key value translations via an index.

The file-based StateDB allows for a more flexible and scalable way of storing state information compared to key-value stores as it eliminates the need to map the information to tries that are later mapped to key-value pairs in the StateDB. It also eliminates the need for complex traversals and management of the key-value data, making it easier to manage and update the state information more efficiently.

Let’s look at an example. Currently, looking up the totalSupply variable (which is a 256-bit address) of a smart contract in the StateDB requires the following steps:

(contract address, storage key) -> storage value = (0x0DE2f6ff6887579593702a0bC573be0bC6BB0f80, "totalSupply") -> 1000

The first part, contract address and storage key, will be encoded as paths in the Merkel Patricia Trie. The trie will be fragmented into various key-value pairs. As a consequence, up to approximately 100 key-value pairs could be accessed in the key-value database for this example.

(contract address, storage key) -> storage value = (123, 456) -> 1000

In this example, the contract address and storage key are both represented by numbers in a compact range or rather, ordinal numbers. The use of ordinal numbers in the set of addresses/keys instead of storing addresses/keys verbatim reduces the amount of storage required and ensures that there are no gaps in the address space.

The new storage system will also not use RLP encoding, which saves space, and will not actively prune MPT, which saves processing time and works similarly to virtual memory in modern operating systems.

MPT, short for Merkle Patricia Tree, is the data structure used by Ethereum and the current Fantom mainnet to store the states of smart contracts. However, it consumes a significant amount of storage space and requires many operations to read and write data.

An improved storage system is great, but it also requires a high-speed virtual machine to execute smart contracts and update their state in storage, which is the focus of Project Tosca, or the Fantom Virtual Machine.

Tosca: Fantom Virtual Machine

The Fantom Virtual Machine (FVM) is the upgrade that will replace the EVM and increase the speed of execution on Fantom.

For our developer community, there's no need to worry! The FVM will be fully Solidity and Vyper compatible, allowing you to continue using the same development tools.

The Fantom Virtual Machine uses dynamic translation, where the code is translated into a more efficient instruction format inside the client, allowing for more efficient execution of the smart contract. This is achieved through more efficient implementation techniques and "super-instructions" which are efficient representations of frequently occurring patterns in code.

The super-instructions consist of multiple instructions that can be executed in one instance. For example, three instructions can be merged into one, so the FVM would only have to execute one instruction, rather than three. They are adaptable and can change over time to reflect changing patterns in the code.

This results in an efficient dispatch and execution of smart contracts, meaning the system is optimized for running the smart contract as efficiently as possible, reducing the processing time and resources required.

Experiment

Outlining the theory is fine, but proving it in practice is better. We conducted an experiment on the Fantom mainnet data, using a fully pruned version of the current Fantom mainnet (i.e. what validators use) compared to the new FVM and fully pruned storage system.

The experiment used the first 40 million mainnet blocks and focused solely on block processing. The findings are shown in the charts below:

As depicted above, there were significant improvements in transactions per second, averaging approximately 4500 per second in our experiment alone. The transactions were 8.1x faster and the blocks used 98% less storage.

This speed could theoretically allow for approximately 400 million daily transactions, four times the number processed daily by VISA in the United States.

The results speak for themselves. The implementation of a new storage system and virtual machine could significantly improve the performance of the Fantom network and make it easier for people to run nodes at a significantly lower cost with much faster syncing times.