Customizing how tests run

Chapter 13.4.

Customizing how tests run

Before we continue adding more tests to our application, I want to take a quick break and talk about a few of the useful flags and options that are available to customize how your tests run.

Controlling which tests are run

So far in this book we’ve been running the tests in a specific package (the cmd/web package) like so:

$ go test ./cmd/web

But it’s also possible to run all the tests in your current project by using the ./... wildcard pattern. In our case, we can use it to run all tests in our project like this:

$ go test ./...
ok      snippetbox.alexedwards.net/cmd/web      0.007s
?       snippetbox.alexedwards.net/internal/models      [no test files]
?       snippetbox.alexedwards.net/internal/validator   [no test files]
?       snippetbox.alexedwards.net/ui   [no test files]

Or going in the other direction, it’s possible to only run specific tests by using the -run flag. This allows you to specify a regular expression — and only tests with a name that matches the regular expression will be run.

For example, we could opt to run only the TestPing test as follows:

$ go test -v -run="^TestPing$" ./cmd/web/
=== RUN   TestPing
--- PASS: TestPing (0.00s)
PASS
ok      snippetbox.alexedwards.net/cmd/web      0.008s

And you can even use the -run flag to limit testing to some specific sub-tests using the format {test regexp}/{sub-test regexp}. For example to run the UTC sub-test of our TestHumanDate test we could do this:

$ go test -v -run="^TestHumanDate$/^UTC$" ./cmd/web
=== RUN   TestHumanDate
=== RUN   TestHumanDate/UTC
--- PASS: TestHumanDate (0.00s)
    --- PASS: TestHumanDate/UTC (0.00s)
PASS
ok      snippetbox.alexedwards.net/cmd/web    0.003s

In contrast, you can prevent specific tests from running by using the -skip flag. Like the -run flag we just looked at, this allows you to specify a regular expression and any tests with a name that matches the regular expression won’t be run. For example, to skip the TestHumanDate test:

$ go test -v -skip="^TestHumanDate$" ./cmd/web/
=== RUN   TestPing
--- PASS: TestPing (0.00s)
=== RUN   TestCommonHeaders
--- PASS: TestCommonHeaders (0.00s)
PASS
ok      snippetbox.alexedwards.net/cmd/web      0.006s

Test caching

You’ve perhaps noticed by now that if you run exactly the same test twice — without making any changes to the package that you’re testing — then a cached version of the test result is shown (indicated by the (cached) annotation next to the package name).

$ go test ./cmd/web
ok      snippetbox.alexedwards.net/cmd/web      (cached)

In most cases, the caching of test results is really useful (especially for large codebases) because it helps reduce the total test runtime. But if you want force your tests to run in full (and avoid the cache) you can use the -count=1 flag:

$ go test -count=1 ./cmd/web

Alternatively, you can clear cached results for all tests with the go clean command:

$ go clean -testcache

Fast failure

As I mentioned briefly a couple of chapters ago, when you use the t.Errorf() function to mark a test as failed, it doesn’t cause go test to immediately exit. All your other tests (and sub-tests) will continue to be run after a failure.

If you would prefer to terminate the tests immediately after the first failure you can use the -failfast flag:

$ go test -failfast ./cmd/web

It’s important to note that the -failfast flag only stops tests in the package that had the failure. If you are running tests in multiple packages (for example by using go test ./...), then the tests in the other packages will continue to run.

Parallel testing

By default, the go test command executes all tests in a serial manner, one after another. When you have a small number of tests (like we do) and the runtime is very fast, this is absolutely fine.

But if you have hundreds or thousands of tests the total run time can start adding up to something more meaningful. And in that scenario, you may save yourself some time by running the tests in parallel.

You can indicate that it’s OK for a test to be run concurrently alongside other tests by calling the t.Parallel() function at the start of the test. For example:

func TestPing(t *testing.T) {
    t.Parallel()

    ...
}

It’s important to note here that:

Tests marked using t.Parallel() will be run in parallel with — and only with — other parallel tests.
By default, the maximum number of tests that will be run simultaneously is the current value of GOMAXPROCS. You can override this by setting a specific value via the -parallel flag. For example:
```
$ go test -parallel=4 ./...
```
Not all tests are suitable to be run in parallel. For example, if you have an integration test which requires a database table to be in a specific known state, then you wouldn’t want to run it in parallel with other tests that manipulate the same database table.

Enabling the race detector

The go test command includes a -race flag which enables Go’s race detector when running tests.

If the code you’re testing leverages concurrency, or you’re running tests in parallel, enabling this can be a good idea to help to flag up race conditions that exist in your application. You can use it like so:

$ go test -race ./cmd/web/

You should be aware that the race detector is limited in its usefulness… it’s just a tool that flags data races if and when they are identified at runtime during testing. It doesn’t carry out static analysis of your codebase, and a clear run doesn’t ensure that your code is free of race conditions.

Enabling the race detector will also increase the overall running time of your tests. So if you’re running tests very frequently as part of a TDD workflow, you may prefer to use the -race flag during pre-commit test runs only.