In this article, we introduce GoLibAFL, a fuzzer for Go code built on top of LibAFL. GoLibAFL provides state-of-the-art fuzzing techniques and offers great customizability for advanced users. We dive into the motivation behind this project, the architectural decisions, and the challenges we faced during development. We also provide an overview of existing Go fuzzing solutions and compare them to GoLibAFL, presenting benchmark results that demonstrate GoLibAFL’s superior performance.
If you’re just looking to get started with fuzzing Go code using GoLibAFL, refer to the Introduction section or visit the repository for setup instructions.
During a recent engagement, our team set out to fuzz a Go codebase and explored existing fuzzers. In the process, we discovered that available Go fuzzing solutions lack key features, such as the ability to solve path constraints, like string or integer comparisons. Some Go fuzzers were even incompatible with the latest Go toolchain.
To overcome these limitations, we explored more effective methods for fuzzing Go. This research led us to discover that Go code can also be fuzzed using LibAFL by leveraging Go’s native libFuzzer-compatible instrumentation.
In this article, we describe how you can fuzz Go code using LibAFL using this approach.
An inherent challenge of fuzzing Go is that it uses its own toolchain, independent of widely used toolchains like GCC and LLVM. As a result, much of the existing fuzzing tooling that relies on these toolchains cannot be used with Go code. While alternative compiler toolchains like gccgo (a Go frontend for GCC) and llgo (an LLVM-based Go compiler) exist, neither achieve full feature parity with the official Go compiler. This makes compiling existing Go projects with these compilers often impossible.
Due to these limitations, the open-source community has turned to developing Go-specific fuzzing tooling. The three main approaches for fuzzing Go binaries today are:
Go-fuzz was one of the earliest fuzzing tools for Go, implementing its own fuzzing logic. It also has support for integrating libFuzzer. However, go-fuzz is only compatible with Go versions up to 1.18 and is no longer actively maintained, as discussed here. Projects leveraging newer Go features cannot be compiled using the go-fuzz toolchain.
Native support for coverage-guided fuzzing is included in the standard toolchain since Go v1.18. Unfortunately, this built-in fuzzing support is limited and lacks advanced features such as tracing comparisons for guided mutations and dictionary-based input generation. These are essential to improve fuzzing efficiency and discover deeper bugs. The Go team has taken initiatives to enhance native fuzzing support, as highlighted in this issue. However, progress on these efforts appears to have stalled, with no updates since 2022.
Go-118-fuzz-build transforms native Go fuzzing harnesses into archive files that can then be linked with libFuzzer to leverage advanced fuzzing features while maintaining compatibility with Go v1.18 and beyond. Despite its name, Go-118-fuzz-build is not limited to Go v1.18; compatibility with newer versions can be enabled by updating the Go version in the project’s go.mod file. While this approach improves the fuzzing performance of native Go fuzzers, it remains dependent on libFuzzer, which is no longer actively maintained and lacks support for recent advancements in fuzzing.
In summary, the current tooling landscape for fuzzing Go code lacks support for modern fuzzing features, and some tools are incompatible with recent Go versions. None of the available Go fuzzing solutions are actively maintained.
Tools for fuzzing Go code should support industry standards and the latest fuzzing features implemented by AFL++ and LibAFL, while remaining independent of Go versions to ensure compatibility across updates without requiring extensive modifications.
GoLibAFL is a fuzzer designed to provide more powerful and flexible fuzzing capabilities for Go code than existing tooling. It maintains compatibility across Go versions through seamless interoperability via the Foreign Function Interface (FFI). It is built on top of LibAFL, a Rust-based fuzzing library providing state-of-the-art fuzzing capabilities. Despite its Rust foundation, using GoLibAFL requires no knowledge of Rust.
Getting started is straightforward — simply follow these steps:
1. Clone the repository
git clone https://github.com/srlabs/golibafl/
cd golibafl
2. Implement your harness in the LLVMFuzzerTestOneInput function, which will be used by LibAFL
func LLVMFuzzerTestOneInput(data *C.char, size C.size_t) C.int {
// `data` should be converted to the type you need and passed as input
input := C.GoBytes(unsafe.Pointer(data), C.int(size))
// function to be fuzzed, i.e `parse`
mylib.parse(input)
}
3. Define the harness location with the environment variable HARNESS
export "HARNESS=harnesses/mylib"
4. Build and run the fuzzer
cargo build --release
./target/release/golibafl fuzz
or, if you're using Docker:
// Build the docker container & run it
docker build --build-arg HARNESS="harnesses/mylib" -t golibafl
docker run -v ./output:/golibafl/output golibafl
If everything succeeds, fuzzing should start immediately, and the output should look similar to this:
And just like that, you are running high-performance fuzzing on your Go code!
If you’re curious about how GoLibAFL works behind the scenes, how it stacks up against other Go fuzzing tools, and the challenges we encountered along the way, stick around for our deep dive below.
Once we realized that the Go compiler includes support for libFuzzer compatible instrumentation, we decided to explore an alternative approach for fuzzing Go binaries. Particularly, Go includes support for SanitizerCoverage inline 8-bit counters (sancov_8bit), one of several coverage instrumentation methods provided by LLVM. Sancov_8bit instrumentation is also supported by LibAFL, a Rust-based library for building customizable fuzzers.
Unlike traditional fuzzing tools that come as standalone solutions, LibAFL provides a modular framework that enables the creation of tailored fuzzing setups for various targets, including native binaries, virtual machines, and even scripting languages. Additionally, since it is a library, LibAFL enables extensive customization of fuzzers to suit specific project needs, such as custom mutators or grammars.
With support for state-of-the-art fuzzing techniques and a track record of outperforming existing fuzzers in benchmarks (see FuzzBench’s latest report), LibAFL presents a powerful alternative to libFuzzer for fuzzing Go code. Leveraging LibAFL can significantly improve performance and introduce greater diversity in fuzzing campaigns.
With this in mind, we set out to leverage Go’s sancov_8bit instrumentation to enable fuzzing Go code with LibAFL.
The following table shows an overview of the Go fuzzing landscape, comparing LibAFL with other fuzzing approaches for Go code:
* This comparison assumes Go-fuzz’s native fuzzing logic and does not consider its libFuzzer integration.
** The tool itself and libFuzzer are no longer maintained.
Although LibAFL is written in Rust, it’s not limited to fuzzing Rust code and can target many other languages, including Go. Go targets can be integrated with LibAFL-based fuzzers with minimal effort, allowing for efficient fuzzing.
In-process fuzzing is the optimal approach for fuzzing if there is no target state or if the state can be reset. In-process fuzzing harnesses and fuzzers are part of the same program and run within the same process. This enables the fuzzer to repeatedly execute the harness without the overhead of forking a new process for each fuzzing iteration, significantly improving performance as a result. All the existing Go fuzzing solutions make use of in-process fuzzing.
To utilize in-process fuzzing with LibAFL, we can use the InprocessExecutor. This executor requires defining a harness function, which the fuzzer will repeatedly call during the fuzzing campaign. Here’s an example harness for an in-process fuzzing campaign:
let mut harness = |input: &PacketData| {
let target = input.target_bytes();
let buf = target.as_slice();
unsafe {
libfuzzer_test_one_input(buf);
}
ExitKind::Ok
};
This harness provides the fuzzer’s byte input to libfuzzer_test_one_input, a function following the libFuzzer-style naming convention (LLVMFuzzerTestOneInput). For a complete example of an in-process fuzzer using LibAFL, refer to their baby fuzzer example.
To call a Go-based harness function from our Rust-based fuzzer, we must use the Foreign Function Interface (FFI) to bridge the two languages. By exposing a C-compatible interface as an intermediary layer, Go and Rust can communicate using C-style primitives for function arguments and return values.
To expose Go functions through FFI, we can use the cgo package and write standard Go code that is C-compatible. Here is an example of an exported function invoking a harness function:
// #include <stdint.h>
import "C"
func harness(data []byte) {
if len(data) < 4 {
return
}
if data[0] == 'F' {
if data[1] == 'U' {
if data[2] == 'Z' {
if data[3] == 'Z' {
panic("FUZZ")
}
}
}
}
}
//export LLVMFuzzerTestOneInput
func LLVMFuzzerTestOneInput(data *C.char, size C.size_t) C.int {
s := C.GoBytes(unsafe.Pointer(data), C.int(size))
harness(s)
return 0
}
The LLVMFuzzerTestOneInput function is exported and can be called by C-compatible code. Inside this function, the data buffer is converted into a Go slice using C.GoBytes and then passed to the harness function. The harness itself is standard Go code implementing the actual harness.
The exported LLVMFuzzerTestOneInput function can directly be invoked using Rust’s FFI interface, which allows Rust code to call foreign functions written in C-compatible languages. By declaring LLVMFuzzerTestOneInput as an extern "C" function in Rust, we enable seamless interoperability, allowing our Rust harness to invoke the Go-based fuzzing logic as if it were a native Rust function. Since LLVMFuzzerTestOneInput is the industry standard for libFuzzer-style harnesses, LibAFL provides a convenient wrapper for it. Below is an example of how this function is declared and used in Rust:
extern "C" {
fn LLVMFuzzerTestOneInput(data: *const u8, size: usize) -> i32;
}
pub unsafe fn libfuzzer_test_one_input(buf: &[u8]) -> i32 {
unsafe { LLVMFuzzerTestOneInput(buf.as_ptr(), buf.len()) }
}
To perform in-process fuzzing of the Go code with our Rust-based fuzzer, we must compile both the fuzzer and the target into a single binary. We achieve this using a Rust build script (build.rs), which compiles the Go target as a static archive and then links it with our Rust-based fuzzer. Note the specific Go compiler flags we are setting to activate the libFuzzer compatible instrumentation:
use std::env;
use std::path::PathBuf;
use std::process::{exit, Command};
fn main() {
// Enable cgo
env::set_var("CGO_ENABLED", "1");
let harness_path = env::var("HARNESS")
.unwrap_or_else(|_| String::from("./examples/prometheus"));
// Define the output directory for the Go library
let out_dir = match env::var("OUT_DIR") {
Ok(out) => PathBuf::from(out),
Err(err) => {
eprintln!("Failed to get OUT_DIR: {err}");
exit(1);
}
};
// Build the Go code as a static library
let status = Command::new("go")
.args([
"build",
"-buildmode=c-archive",
"-tags=libfuzzer,gofuzz",
// Enable coverage instrumentation for libFuzzer
"-gcflags=all=-d=libfuzzer",
// avoid instrumenting unnecessary packages
"-gcflags=runtime/cgo=-d=libfuzzer=0",
"-gcflags=runtime/pprof=-d=libfuzzer=0",
"-gcflags=runtime/race=-d=libfuzzer=0",
"-gcflags=syscall=-d=libfuzzer=0",
"-o",
])
.arg(out_dir.join("libharness.a"))
.current_dir(harness_path)
.status();
match status {
Ok(status) if status.success() => (),
Ok(exit_code) => {
eprintln!("Go build failed");
match exit_code.code() {
Some(code) => exit(code),
None => exit(2),
}
}
Err(err) => {
eprintln!("Failed to execute Go build: {err}");
exit(3);
}
}
// Tell cargo to look for the library in the output directory
println!("cargo:rustc-link-search=native={}", out_dir.display());
// Tell cargo to link the static Go library
println!("cargo:rustc-link-lib=static=harness");
}While developing GoLibAFL, we encountered Go runtime initialization deadlocks and Go specific performance issues. This section explains how to solve these.
The biggest challenge we initially encountered was a deadlock that completely blocked our fuzzing attempts. Whenever we called the Go harness function from within our LibAFL harness function, it would get stuck inside _cgo_wait_runtime_init_done, waiting for the Go runtime to be initialized. For reasons unknown to us, the Go runtime failed to initialize when invoked solely from our Rust harness. We suspect that compiling the Go binary with -buildmode=c-archive, as well as Go’s garbage collector (see below), has something to do with this. Compiling the binary with -buildmode=c-shared might resolve it, but this seems to be incompatible with -gcflags=all=-d=libfuzzer.
We managed to resolve this issue by defining and exporting an additional Go function whose sole purpose is to explicitly trigger the Go runtime’s initialization. In keeping with LibFuzzer-style harness naming conventions, we used another standard function name (LLVMFuzzerInitialize) and ensured it was invoked at the very start of our Rust fuzzer’s main function, before the harness is called:
//export LLVMFuzzerInitialize
func LLVMFuzzerInitialize(argc *C.int, argv ***C.char) C.int {
fmt.Println("LLVMFuzzerInitialize")
return 0
}After successfully getting in-process fuzzing of Go code to work, we encountered an unexpected challenge: our approach initially performed worse than the libFuzzer-based setups. Specifically, it achieved roughly three times fewer executions per second across test targets, leading to lower code coverage.
Given that LibAFL-based fuzzers have previously demonstrated performance parity or better performance than libFuzzer for in-process fuzzing, we suspected an issue with our setup rather than an inherent limitation of LibAFL. With the assistance of the helpful LibAFL maintainers, we tracked down the issue to the Go garbage collector (GC).
Using Linux perf, we realized the fuzzer was spending most of its time waiting for the garbage collector. Something about the interaction between LibAFL and the Go runtime caused the GC to stall, resulting in the fuzzing run timing out. Specifically, the execution seemed to get stuck inside gcBgMarkStartWorkers. These timeouts caused the fuzzer to restart the target and restore its state, negatively impacting performance. Notably, this issue only occurred when calling the Go harness function from the LibAFL harness. Executing the Go harness outside of the fuzzing loop worked fine, even when manually triggering garbage collection with runtime.GC().
The libFuzzer-based approaches did not encounter this issue, allowing them to achieve higher performance on the test targets.
To resolve the issue, we initially adjusted the Go garbage collector’s behavior using the GOGC environment variable, which controls how frequently the GC runs. By default, GOGC=100 triggers garbage collection when the heap grows by 100% since the last collection, while setting GOGC=off disables it entirely.
Another way to modify GC behavior dynamically is by calling the SetGCPercent function at runtime, which allows for GC tuning without relying on environment variables. Calling SetGCPercent(-1) disables the garbage collector. Moreover, the SetMemoryLimit function can be used to adjust the Go runtime’s soft memory limit, which triggers garbage collection upon reaching the limit, even if the garbage collector has been disabled.
For our benchmarks, we settled on disabling the garbage collector by calling SetGCPercent(-1) and setting the soft memory limit to 1 GiB using SetMemoryLimit(1024 * 1024 * 1024). With this configuration, garbage collection is triggered when any fuzzing process exceeds 1 GiB of memory usage. As an example, when fuzzing with multiple processes (e.g., using 4 cores), each process has its own 1 GiB memory limit, meaning total memory usage can reach up to 4 GiB.
This approach significantly improved performance, with our setup even outperforming the libFuzzer-based approach — despite GC also being disabled for it.
//export LLVMFuzzerInitialize
func LLVMFuzzerInitialize(argc *C.int, argv ***C.char) C.int {
debug.SetGCPercent(-1)
// set a max of 1G RAM usage per process
debug.SetMemoryLimit(1024*1024*1024)
return 0
}
We hope to find a better solution to this in the future and fix the problem completely. PRs are welcome!
Note: Reducing the frequency of garbage collection — or disabling it entirely — can significantly increase memory usage. When GOGC is set to a higher value, the Go garbage collector runs less often, which may result in temporary memory spikes. Setting GOGC=off disables garbage collection altogether, causing memory to grow continuously throughout the fuzzing campaign since the Go runtime no longer reclaims unused memory. This behaviour is especially critical during in-process fuzzing, where the target process isn’t restarted, preventing any implicit memory cleanup.In this section, we explain how to solve performance issues with CGO in Go v1.22 and LibAFL’s MultiMapObserver.
While searching for further optimizations for our fuzzing setup, we found that the cgo function x_cgo_getstackbound was consuming a significant portion of execution time — up to 30% for the targets we tested. A quick search led us to this issue, which covers the same function. As it turns out, this is a cgo-specific performance issue related to stack bound retrieval, introduced in Go v1.22 and fixed in Go v1.23. To optimize performance, upgrade to at least Go v1.23.
For sancov_8bit instrumentation, each dynamically shared object (DSO) has its own counters_map, set up by LLVM and accessible via the __sanitizer_cov_8bit_counters_init function. As a result, a fuzzer must track multiple counters_map instances to obtain a complete coverage view.
LibAFL addresses this by using a MultiMapObserver, which can observe multiple maps simultaneously. However, this observer is quite inefficient and lacks the optimizations used by other observers in LibAFL. Fortunately, since our setup compiles the Go target as a static library, there is only a single DSO, meaning only one counters_map exists. This allows us to use StdMapObserver and take advantage of its optimizations.
Note that the nightly Rust toolchain is required to enable these optimizations. For this reason, we have fixed the toolchain using a rust-toolchain.toml file.
To utilize the StdMapObserver, we simply select the first (and only) map from the COUNTERS_MAP vector:
let counters_map_len = unsafe { COUNTERS_MAPS.len() };
if counters_map_len != 1 {
panic!(
"{}",
format!("Unexpected COUNTERS_MAPS length: {counters_map_len}")
);
}
let edges = unsafe { extra_counters() };
let edges_observer =
StdMapObserver::from_mut_slice(
"edges",
edges
.into_iter()
.next()
.unwrap()
)
.track_indices();
This optimization results in another significant performance boost.
With the fixes and optimizations applied, and both the build script and the Go harness FFI configured, the setup is ready for fuzzing. You can now implement your fuzzer in Rust using LibAFL, just as you would for any other LibAFL-based fuzzer and kick off your fuzzing campaign! 🚀
For the full code of an example Go fuzzer using our approach, please refer to our repository.
To evaluate the performance of our fuzzer, we ran benchmarks on three targets: prometheus, caddy, and burntsushi-toml. These targets were selected arbitrarily from the list of Go-based projects supported by oss-fuzz. Our harnesses for the individual targets are largely based on the corresponding OSS-Fuzz harnesses and fuzz the same functionality. Each target was fuzzed without a prior corpus of valid or well-formed inputs.
Every target was fuzzed for 24 hours, and we compared the code coverage achieved within their respective Go packages. We couldn’t compile any of the targets with go-fuzz, as it only supports Go code up to v1.18. So, it is not included in the comparison.
Below are the results of our benchmarking for the selected targets:
As we can see, GoLibAFL outperforms both Go-118-fuzz-build and the native Go fuzzing infrastructure on all targets except Caddy, where both Go-118-fuzz-build and GoLibAFL maintain a stagnant coverage of 5.2% throughout the entire run.
We included an example target in the repository that was specifically designed to test the fuzzer’s ability to solve path constraints involving string and big integer comparisons. This target highlights challenges typically addressed by comparison logging (CMPLOG) in fuzzers like AFL++ or LibAFL. As we use fuzzers extensively in our daily work, we frequently encounter these specific path constraints, and as such this is a common test we include to see what can or cannot be found by our fuzzers.
When running this example, GoLibAFL immediately (before the speed counters are updated) finds the crash.
Running the crash shows the correct input was found:
On the Go-118-fuzz-build / libFuzzer side, there were still no finds after one minute of runtime. It eventually finds the crash after a long time, with a big variance in time every run.
By leveraging Go’s native libFuzzer-compatible instrumentation and LibAFL’s support for it, we created an effective and versatile fuzzing tool for Go code that outperformed existing Go fuzzers in the chosen benchmarks. That said, our solution does come with certain limitations.
For future improvements, we aim to:
We would like to thank our colleagues Aarnav, Marc, Kevin, and Constantin for their help with the fuzzer, as well as the LibAFL maintainers for their support in resolving the deadlock issue we encountered. Thanks also to the reviewers Linus, Laura, and Maria.
The Go Gopher mascot in the post cover was designed by Renee French and is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0).
