Benchmarks
As of August 2024
Abstract
The Valida zk-VM will offer industry leading speed and efficiency in succinctly proving program executions. The preliminary benchmarks in this report provide evidence for this claim. The following table summarizes how many times faster Valida is compared to RISC Zero, SP1, and Jolt for the examples we studied:
Iterative Fibonacci (n=25)
CPU efficiency (single-core Valida)
1024
3461
66
CPU efficiency (Multi-core Valida)
205
1130
13
Wall clock time (single-core Valida)
37
349
15
Wall clock time (Multi-core Valida)
56
423
23
SHA-256
CPU efficiency (single-core Valida)
66
304
6.4
CPU efficiency (Multi-core Valida)
26.6
123
2.6
Wall clock time (single-core Valida)
2.5
12
1.2
Wall clock time (Multi-core Valida)
12
53.5
5.4
Status of Valida and its compiler toolchain
The Valida ISA is a work in progress. The working ISA spec forms the core of it. There are ongoing efforts to figure out what would be a sufficient set of extensions to the Valida ISA to support efficient execution of most programs. The working ISA spec does not include all instructions currently supported by the Valida zk-VM.
The Valida ISA has a modular architecture which allows for application developers to choose which subsets of the ISA (called “chips”) they require for their use case. There is a default selection of chips known as the basic Valida ISA, which is supposed to support efficient execution of most programs. Application developers can add specialized chips to support their particular use cases; e.g., specialized chips may compute hash permutations, elliptic curve group operations, or other expensive, frequently used functions.
The Valida zk-VM implementation is available to the public. There are some known issues with the Valida zk-VM implementation affecting correctness and completeness of the prover / verifier pair:
The memory argument is not fully sound.
The following arithmetic operations are unconstrained or wrongly constrained:
DIV32 (32-bit unsigned division),
MULHU32 (32-bit unsigned multiply, high bits only), and
MULHS32 (32-bit signed multiply, high bits only).
All of the known non-working and missing constraints are in the process of being added or replaced, or they are planned to be added or replaced in the near future. None of these appear particularly risky, either from a timeline or performance standpoint, in that workable approaches are known for all of them and we as Lita Foundation expect some of these to be reasonably performant.
We as Lita Foundation are internally auditing the Valida zk-VM. We will report any issues we find, as we find them. We also plan to submit Valida for a third party external audit.
There is an LLVM-based compiler toolchain for Valida, which can compile some simple C programs into Valida programs. Broader language support is also planned for the Valida compiler toolchain, including Rust and Go. The alpha version of our C compiler has been released. See here to get started!
Future directions
Future studies should compare the performance of Valida using more example programs, against more provers, on more platforms, and with larger problem sizes.
Ongoing work on the Valida prover / verifier pair will address the known issues with its correctness and completeness, as well as further auditing and verifying Valida to flush out any unknown issues.
The Valida prover contains bottlenecks which result in inconsistent CPU utilization throughout the proving process. Better usage of the available parallelism may be able to get rid of these bottlenecks, resulting in better prover wall clock time. Reducing the memory footprint of the Valida prover could also be useful.
Ongoing work on the Valida LLVM backend will allow it to compile a wider variety of source programs in a wider variety of source languages. Future work on the Valida LLVM backend will also improve the efficiency of the generated code.
Future work should run the Valida prover in hardware accelerated settings, using coprocessors like GPUs or Fabric Cryptography’s VPUs. This may result in exponential improvements to the speed and efficiency of proving.
Other choices of proving algorithms may provide further breakthroughs in speed and efficiency of proving. The relevance of upcoming research in zero knowledge cryptography to Valida proving should be studied.
Last updated