f-tests as a replacement for table-driven tests in Go
Table-driven tests in Go is the officially recommended way to write multiple tests for some function, which may produce different outputs for different inputs. See this article, which explains the advantages of table-driven tests over function-based tests. The main advantage is that you don’t need to copy the same code, which prepares inputs for the tested function, calls the tested function and then checks the returned outputs against the expected outputs. Table-driven tests allow writing this code once and then executing it in the loop across prepared test cases. For example, strings.Index function can be tested with the following functions:
func TestStringsIndex_FirstCharMatch(t *testing.T) {
n := strings.Index("foobar", "foo")
if n != 0 {
t.Fatalf("unexpected n; got %d; want %d", n, 0)
}
}
func TestStringsIndex_MiddleCharMatch(t *testing.T) {
n := strings.Index("foobar", "bar")
if n != 3 {
t.Fatalf("unexpected n; got %d; want %d", n, 3)
}
}
func TestStringsIndex_Mismatch(t *testing.T) {
n := strings.Index("foobar", "baz")
if n != -1 {
t.Fatalf("unexpected n; got %d; want %d", n, -1)
}
}Let’s convert these functions to canonical table-driven test:
func TestStringsIndex(t *testing.T) {
tests := []stuct{
name string
s string
substr string
want int
}{
{
name: "firstCharMatch",
s: "foobar",
substr: "foo",
want: 0,
},
{
name: "middleCharMatch",
s: "foobar",
substr: "bar",
want: 3,
},
{
name: "mismatch",
s: "foobar",
substr: "baz",
want: -1,
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
got := strings.Index(tc.s, tc.substr)
if got != tc.want {
t.Fatalf("unexepcted n; got %d; want %d", got, tc.want)
}
}
}Problems with table-driven tests
Look at the code above. Which code is easier to read and understand? Obviously, function-based tests are easier to read, since every such test contains short, clear self-contained code without unnecessary indirections and abstractions. The table-driven test, on the other hand, is more convoluted:
- Test cases are split from the actual code, which runs these test cases. This complicates reading and understanding the code — you need to jump between tests cases and the actual test code inside the loop in order to understand how every test case works.
- When you read some test function, the first thing you want to do is to read the actual test code in order to understand what it does. This code is located at the end of the table-driven test function. So you need to scroll over all the test cases (which may occupy hundreds or thousands of lines for non-trivial tests) before reaching the actual test code.
- When some test case fails, Go prints the line of the corresponding t.Fatalf call in the error log. This line is located inside the actual test code at the end of function with table-driven tests. This line is located in completely different place comparing to the actual test case. So, you need to figure out the location of the actual test case in order to understand which parameters this particular test case has. The recommended workaround is to put some name of the test into
namefield of every test case. This simplifies locating the actual test case for the failed test at the cost that you need to figure out good names per each test case (computer science has two hard things — cache invalidation and naming). - The code, which runs actual tests, works with fields of the
testcasestructure, which is usually defined at the beginning of the function with table-driven tests. This introduces additional level of indirection — you need to be aware of thetestcasestructure definition when reading the code, which runs table-driven tests.
Is there a solution, which allows solving all the issues of table-driven tests, while allowing re-using the code, which prepares input args, passes them to the tested function and then checks the returned results against the expected output results? Yes — f-tests!
f-tests
Let’s put the main test code into an anonymous function at the beginning of test function. This function should accept args with the input for the tested function, plus the expected output results. Let’s name this function f. Let’s call this function with the inputs and outputs, which must be tested:
func TestStringsIndex(t *testing.T) {
f := func(s, substr string, nExpected int) {
t.Helper()
n := strings.Index(s, substr)
if n != nExpected {
t.Fatalf("unexpected n; got %d; want %d", n, nExpected)
}
}
// first char match
f("foobar", "foo", 0)
// middle char match
f("foobar", "bar", 3)
// mismatch
f("foobar", "baz", -1)
}Two important notes:
f()doesn’t acceptt *testing.Targ, since it can use the corresponding arg from the outer test function.- The first line inside
f()is t.Helper() call. It is needed for printing the line with the correspondingf()call when some test fails. This instantly gives you the full context about the failed test — just navigate to the corresponding line Go prints in the error message. You don’t need giving some artificial names per each test case (such as thenamefield in the table-driven tests). Instead, you can put arbitrary complex explanations for the particular test case inside regular comments in front of everyf()call. You don’t need to put all the test case context intot.Fatalf()call, since the full context is available just at the line Go prints in the error message.
Every test case inside f-test is completely isolated of each other — you can perform arbitrary complex setup and tear-down per each f() call directly inside the f() function.
Other benefits of f-tests over table-driven tests:
- The actual test code is located at the beginning of f-test, so it is easy to read and understand it before going to test cases.
- The actual test code doesn’t depend on some
testcasestruct fields — all the inputs and the expected outputs are passed as regular args tof()function. This allows avoiding unnecessary level of indirection and writing simpler code (thetestcasestruct still can be defined in rare cases iff()accepts too many args — then these args can be defined in thetestcasestruct, which is then passed tof()).
Common questions about f-tests
- How to test functions, which may return error? It is recommended writing two test functions — one for testing failure cases with the
_Failuresuffix in its name, and another one for testing success cases with the_Successsuffix in its name. This is better than mixing success and failure cases in a single function, since failure test cases are usually more clear to test in a separate function. For example:
func TestSomeFunc_Failure(t *testing.T) {
f := func(input string) {
t.Helper()
_, err := SomeFunc(input)
if err == nil {
t.Fatalf("expecting non-nil error")
}
}
f("broken_input_1")
f("broken_input_2")
}
func TestSomeFunc_Success(t *testing.T) {
f := func(input, resultExpected string) {
t.Helper()
result, err := SomeFunc(input)
if err != nil {
t.Fatalf("unexpected error: %s", err)
}
if result != resultExpected {
t.Fatalf("unexpected result; got %q; want %q", result, resultExpected)
}
}
f("input_1", "result_1")
f("input_2", "result_2")
}- How to test functions which accept and/or produce deeply nested non-trivial structs? The straightforward approach — to prepare these non-trivial structs before every
f()call — may result in code bloat and duplication. The better way is to figure out some simple args forf()function, which then could be converted to the needed non-trivial structs byf()itself before calling the tested function:
func TestFuncWithNonTrivialArgs(t *testing.T) {
f := func(inputMarshaled, resultMarshaledExpected string) {
t.Helper()
input := unmarshalInputToComplexStruct(inputMarshaled)
result := FuncWithNonTrivialArgs(input)
resultMarshaled := marshalComplexResult(result)
if resultMarshaled != resultMarshaledExpected {
t.Fatalf("unexpected result; got %q; want %q", resultMarshaled, resultMarshaledExpected)
}
}
f("foo", "bar")
f("abc", "def")
}- How to use subtests in f-tests? It isn’t recommend doing this in general case, since this may unnecessarily complicate the test code without giving any practical benefits. But if you really need subtests (for example, you need executing sub-tests separately by passing their names to
go test -run=…command for some reason), then just wrap everyf()call into t.Run() call and pass subtest’starg as the first arg tof():
func TestSomeFuncWithSubtests(t *testing.T) {
f := func(t *testing.T, input, outputExpected string) {
t.Helper()
output := SomeFunc(input)
if output != outputExpected {
t.Fatalf("unexpected output; got %q; want %q", output, outputExpected)
}
}
t.Run("first_subtest", func(t *testing.T) {
f(t, "foo", "bar")
}
t.Run("second_subtest", func(t *testing.T) {
f(t, "baz", "abc")
}
}- In which cases it isn’t recommended to use f-tests? If test results depend on the order of
f()calls, then it is bad idea to use f-tests. In general, f-tests are good for classical unit testing of multiple cases, while they may be not so good for other test types. - Are there practical examples for f-tests? Sure — they are successfully used in VictoriaMetrics source code and in other VictoriaMetrics-related open source projects since 2018. Links to some of these examples: one, two, three.
- Are there drawbacks for f-tests? Yes — they aren’t so good comparing to table-driven tests in code obfuscation contests.
Conclusions
Table-driven tests in Go are good, but f-tests are much better. Simplify your table-driven tests by converting them to f-tests. We successfully use f-tests in VictoriaMetrics (open-source time series database and observability tools written in Go) for more than 5 years, and aren’t going to return back to table-driven tests.
P.S. See also my thoughts about generics and iterators in Go1.23.
