Fuzz Testing Go HTTP Services

A guide on fuzz testing in Go.

As a developer, you can't envision all of the possible inputs your programs or functions could receive. Even though you can define the major edge cases, you still can't predict how your program will behave in the case of some weird unexpected input. In other words, you can only find bugs you expect to find.

That's where fuzz testing or fuzzing comes to the rescue.

What is Fuzz Testing

Fuzzing is an automated software testing technique that involves inputting a large amount of valid, nearly-valid or invalid random data into a computer program and observing its behavior and output. So the goal of fuzzing is to reveal bugs, crashes and security vulnerabilities in source code you might not find through traditional testing methods.

The Fuzz Testing in Go video which I made a few years ago shows a very simple example of the Go code that may work well unless you provide a certain input:

func Equal(a []byte, b []byte) bool {
  for i := range a {
    // can panic with runtime error: index out of range.
    if a[i] != b[i] {
      return false
    }
  }

  return true
}

Fuzzing technique would easily spot this bug by bombarding this function with various inputs.

It's a good practice to integrate fuzzing into your team’s software development lifecycle (SDLC) as well. For example, Microsoft uses fuzzing as one of the stages in its SDLC, to find potential bugs and vulnerabilities.

Fuzz Testing in Go

There are many fuzzing tools that have been available for a while such as oss-fuzz for example, but since Go 1.18 fuzzing was added to Go's standard library and is part of the regular testing package since it is a kind of test. It can also be used together with the other testing primitives which is nice.

The steps to create a fuzz test in Go are the following:

1. In a _test.go file create a function that starts with Fuzz which accepts *testing.F

2. Add corpus seeds using f.Add() to allow fuzzer to generate the data based on it.

3. Call fuzz target using f.Fuzz() by passing fuzzing arguments which our target function accepts.

4. Start the fuzzer using regular go test command, but with the –fuzz=Fuzz flag

Note, the fuzzing arguments can only be the following types:

• string, byte, []byte

• int, int8, int16, int32/rune, int64

• uint, uint8, uint16, uint32, uint64

• float32, float64

• bool

A simple fuzz test for the Equal function above may look like this:

// Fuzz test
func FuzzEqual(f *testing.F) {

  // Seed corpus addition
  f.Add([]byte{'f', 'u', 'z', 'z'}, []byte{'t', 'e', 's', 't'})

  // Fuzz target with fuzzing arguments
  f.Fuzz(func(t *testing.T, a []byte, b []byte) {
    // Call our target function and pass fuzzing arguments
    Equal(a, b)
  })
}

By default, fuzz tests run forever, so you either need to specify the time limit or wait for fuzz tests to fail. You can specify which tests to run using --fuzz argument.

go test --fuzz=Fuzz -fuzztime=10s

If there are any errors during the execution, the output should look similar to this:

go test --fuzz=Fuzz -fuzztime=30s
--- FAIL: FuzzEqual (0.02s)
    --- FAIL: FuzzEqual (0.00s)
        testing.go:1591: panic: runtime error: index out of range
    Failing input written to testdata/fuzz/FuzzEqual/84ed65595ad05a58
    To re-run:
    go test -run=FuzzEqual/84ed65595ad05a58

Notice that the input for which fuzz test has failed are written into a file in testdata folder and can be re-played by using that input identifier.

go test -run=FuzzEqual/84ed65595ad05a58

The testdata folder can be checked into the repository and be used for regular tests, because fuzz tests can also act as regular tests when executed without --fuzz flag.

Fuzzing HTTP Services

It's also possible to fuzz test the http services by writing a test for your HandlerFunc and using the httptest package. Which can be very useful to test the whole HTTP service, not only the underlying functions.

Let's now introduce a more real example such as an HTTP Handler that accepts some user input in the request body and then write a fuzz test for it.

Our handler accepts JSON request with limit and offset fields to paginate some static mocked data. Let's define the types first.

type Request struct {
  Limit  int `json:"limit"`
  Offset int `json:"offset"`
}

type Response struct {
  Results    []int `json:"items"`
  PagesCount int   `json:"pagesCount"`
}

Our handler function then parses the JSON, paginates the static slice and returns a new JSON in response.

func ProcessRequest(w http.ResponseWriter, r *http.Request) {
  var req Request

  // Decode JSON request
  if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
    http.Error(w, err.Error(), http.StatusBadRequest)
    return
  }

  // Apply offset and limit to some static data
  all := make([]int, 1000)
  start := req.Offset
  end := req.Offset + req.Limit
  res := Response{
    Results:    all[start:end],
    PagesCount: len(all) / req.Limit,
  }

  // Send JSON response
  if err := json.NewEncoder(w).Encode(res); err != nil {
    http.Error(w, err.Error(), http.StatusInternalServerError)
    return
  }

  w.WriteHeader(http.StatusOK)
}

As you may have already noticed, this function doesn't handle slice operations quite well and can easily panic. Also it can panic if it tries to divide by 0. It's great if we can spot it during the development or using only unit tests, but sometimes not everything is visible to our eye, and our handler may pass the input to other functions and so forth.

Following our FuzzEqual example above, let's implement a fuzz test for the ProcessRequest handler. The first thing we need to do is to provide the sample inputs for the fuzzer. This is the data that the fuzzer will use and modify into new inputs that are tried. We can craft some sample JSON request and use f.Add() with []byte type.

func FuzzProcessRequest(f *testing.F) {
  // Create sample inputs for the fuzzer
  testRequests := []Request{
    {Limit: -10, Offset: -10},
    {Limit: 0, Offset: 0},
    {Limit: 100, Offset: 100},
    {Limit: 200, Offset: 200},
  }

  // Add to the seed corpus
  for _, r := range testRequests {
    if data, err := json.Marshal(r); err == nil {
      f.Add(data)
    }
  }

  // ...
}

After that we can use httptest package to create a test HTTP server and make requests to it.

Note: Since our fuzzer can generate invalid non-JSON requests, it's better just to skip them and ignore with t.Skip(). We can also skip BadRequest errors.

func FuzzProcessRequest(f *testing.F) {
  // ...

  // Create a test server
  srv := httptest.NewServer(http.HandlerFunc(ProcessRequest))
  defer srv.Close()

  // Fuzz target with a single []byte argument
  f.Fuzz(func(t *testing.T, data []byte) {
    var req Request
    if err := json.Unmarshal(data, &req); err != nil {
      // Skip invalid JSON requests that may be generated during fuzz
      t.Skip("invalid json")
    }

    // Pass data to the server
    resp, err := http.DefaultClient.Post(srv.URL, "application/json", bytes.NewBuffer(data))
    if err != nil {
      t.Fatalf("unable to call server: %v, data: %s", err, string(data))
    }

    defer resp.Body.Close()

    // Skip BadRequest errors
    if resp.StatusCode == http.StatusBadRequest {
      t.Skip("invalid json")
    }

    // Check status code
    if resp.StatusCode != http.StatusOK {
      t.Fatalf("non-200 status code %d", resp.StatusCode)
    }
  })
}

Our fuzz target has a single argument with a type []byte that contains the full JSON request, however you can change it to have multiple arguments.

Everything is ready now to run our fuzz tests. When fuzzing http servers, you may need to adjust the amount of parallel workers, otherwise the load may overwhelm the test server. You can do that by setting -parallel=1 flag.

go test --fuzz=Fuzz -fuzztime=10s -parallel=1

And as expected we will see the following errors uncovered.

go test --fuzz=Fuzz -fuzztime=30s
--- FAIL: FuzzProcessRequest (0.02s)
    --- FAIL: FuzzProcessRequest (0.00s)
        runtime error: integer divide by zero
        runtime error: slice bounds out of range

We can also see the fuzz inputs in the testdata folder to see which JSON contributed to this failure. Here is a sample content of the file:

go test fuzz v1
[]byte("{"limit":0,"offset":0}")

To fix that issue we can introduce input validation and default settings:

if req.Limit <= 0 {
  req.Limit = 1
}

if req.Offset < 0 {
  req.Offset = 0
}

if req.Offset > len(all) {
  start = len(all) - 1
}

if end > len(all) {
  end = len(all)
}

With this change the fuzz tests will run for 10 seconds and exit without an error.

Conclusion

Writing fuzz tests for your HTTP services or any other methods is a great way to detect hard-to-find bugs. Fuzzers can detect hard-to-spot bugs that happen for only some weird unexpected input.

It's amazing to see that fuzzing is a part of built-in testing library, making it easy to combine with regular tests. Note: prior to Go 1.18 developers used go-fuzz, which is a great tool for fuzzing as well.

Resources

• Source Code

• Fuzz Testing in Go

• Go Fuzzing