Go Testing: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(77 intermediate revisions by the same user not shown)
Line 1: Line 1:
=External=
=External=
* https://go.dev/doc/code#Testing
* https://go.dev/doc/code#Testing
* https://pkg.go.dev/testing
=Internal=
=Internal=
* [[Testing]]
* [[Go Engineering#Subjects|Go Engineering]]
* [[Go Engineering#Subjects|Go Engineering]]
* [[Go_Tool#test|<code>go test</code> command]]
* [[Go_test_Command#Overview|<code>go test</code> command]]
=TO DO=
<font color=darkkhaki>
* A system to allow testing with real temporary files in <code>./build</code> in Go.
* Integrate see [[Go_Packages#External_Test_Packages|external test packages]].
* Testing idiom "Introducing Go" page 96.
* Investigate gotest.tools module.
* Running go test immediately after cloning should succeed without any configuration or any running database. If a database is required, those are integration tests.
* https://about.sourcegraph.com/blog/go/advanced-testing-in-go
* Understand unit test lifecycle. Is package initialization performed once per test package?
* Try Testify set up and tear down test suite fixtures, which can save you time and reduce code duplication: https://blog.stackademic.com/using-testify-in-golang-a-comprehensive-guide-8e0417529669
* Try gotest.tools/v3 v3.0.3.
* A mock framework for Go: https://pkg.go.dev/github.com/golang/mock/gomock
* Testable examples in Go: https://go.dev/blog/examples
* https://blog.jetbrains.com/go/2022/11/22/comprehensive-guide-to-testing-in-go/
</font>


=Overview=
=Overview=
Go comes with a lightweight test framework that includes the [[Go_Tool#test|<code>go test</code>]] command and the <code>[[Go Package testing#Overview|testing]]</code> package. The tests live in <code>*_test.go</code> files.
Go comes with a lightweight test framework that includes the [[Go_test_Command#Overview|<code>go test</code>]] command and the <code>[[Go Package testing#Overview|testing]]</code> package. The tests live in <code>*_test.go</code> files, located in the package directory.
=<span id='Helper_Packages'></span>Packages=
* [[gotest.tools#Overview|gotest.tools]]
* [[github.com/stretchr/testify#Overview|github.com/stretchr/testify]]
=Test Types in Go=
==Unit Tests==
<code>somepkg_test.go</code> files should contain exclusively unit tests, not integration or system tests. Integration tests should be stored in <code>*_integration_test.go</code> files and system tests in <code>*_system_test.go</code> files. <span id='unit_test_Build_Tag'></span>Optionally, unit test files may include the <code>unit_test</code> [[Go_Build_Tags#Controlling_Test_Execution_with_Build_Tags|build tag]], which allows controlling test execution based on their type (unit, functional and system).


=<span id='Writing_a_Unit_Test'></span>Write a Unit Test=
<syntaxhighlight lang='go'>
//go:build unit_test
 
package somepkg
</syntaxhighlight>
 
If no tag is declared,  the tests will run fine, but we lose the possibility to control when we don't want them to run. They will '''always''' run.
Also see: {{Internal|Software_Testing_Concepts#Unit_Test|Software Testing Concepts &#124; Unit Tests}}
===<tt>testing.T</tt>===
 
<code>t.Cleanup(func())</code>
 
===<span id='Write_a_Unit_Test'></span>Writing a Unit Test===
 
<font color=darkkhaki>TODO: revisit for Testify</font>.


Write a module as shown here: {{Internal|Go_Modules#Declaring_Modules|Declaring a Module}}
Write a module as shown here: {{Internal|Go_Modules#Declaring_Modules|Declaring a Module}}


For each file containing the behavior to test (<code>a.go</code>):
For each file containing behavior to test (<code>a.go</code>)
<syntaxhighlight lang='go'>
<syntaxhighlight lang='go'>
package a
package a
Line 26: Line 64:
</syntaxhighlight>
</syntaxhighlight>


add a <code><file-name>_test.go</code> test file. In this case <code>a_test.go</code>.
add a <code><file-name>_test.go</code> test file. In this case <code>a_test.go</code>.  These files are ignored by the compiler and only compiled and executed when <code>go test</code> is run. These files will be excluded from regular package builds. For more details on how <code>go test</code> command handles different files, see: {{Internal|Go_test_Command#File_Selection|<tt>go test</tt> File Selection}}
 
The test files should be part of the same package. If that is the case, the test has access to unexpected identifiers from the package being tested.  It is also possible to declare the test files into a corresponding package with the suffix <code>_test</code>. In this case, the package being tested must be imported explicitly in the test. This is known as "black box" testing. Also see [[Go_Packages#External_Test_Packages|external test packages]].


The test file should be part of the same package.
The test files should import <code>testing</code>.


The test file should import "testing":
Add individual tests, as functions starting with <code>TestX..</code>, where <code>X...</code> does not start with a lowercase letter, and taking an argument <code>t *testing.T</code>. The function name serves to identify the test routine. Within these functions, use <code>Error</code>, <code>Fail</code> or related methods to signal failure.
 
<syntaxhighlight lang='go'>
func TestX...(t *testing.T) {
  ...
  t.Error("expected this: %q, got that: %q ", ...)
}
</syntaxhighlight>


<syntaxhighlight lang='go'>
<syntaxhighlight lang='go'>
Line 38: Line 85:


func TestReverseEmptyString(t *testing.T) {
func TestReverseEmptyString(t *testing.T) {
expected := ""
expected := ""
result := Reverse("")
result := Reverse("")
if result != expected {
if result != expected {
t.Errorf("expected %q, got %q", expected, result)
  t.Errorf("expected %q, got %q", expected, result)
}
}
}
}


func TestReverseOneCharString(t *testing.T) {
func TestReverseOneCharString(t *testing.T) {
expected := "a"
expected := "a"
result := Reverse("a")
result := Reverse("a")
if result != expected {
if result != expected {
t.Errorf("expected %q, got %q", expected, result)
  t.Errorf("expected %q, got %q", expected, result)
}
}
}
}


func TestReverseTwoCharString(t *testing.T) {
func TestReverseTwoCharString(t *testing.T) {
expected := "ba"
expected := "ba"
result := Reverse("ab")
result := Reverse("ab")
if result != expected {
if result != expected {
t.Errorf("expected %q, got %q", expected, result)
  t.Errorf("expected %q, got %q", expected, result)
}
}
</syntaxhighlight>
 
From the module directory, run the tests:
<syntaxhighlight lang='bash'>
go test
</syntaxhighlight>
<font size=-1>
PASS
ok  example.com/a 0.116s
</font>
 
===Table (Tabular) Tests===
 
The "table" ("tabular") format is useful when we need to test a function with a variety of input parameters. The "table" consists in a list of structures (hence the tabular format). Each structure instance provides the test name, the input parameters to be tested, and an assertion function that takes the function result and error and perform assertions correlated to the input. If we simply test for expected results, the assert function can be replaced with the expected results.
 
Assuming that <code>SomeFunction()</code> is a function to test:
 
<syntaxhighlight lang='go'>
func SomeFunction(arg string) (string, error) {
return strings.ToUpper(arg), nil
}
</syntaxhighlight>
 
then a tabular test would be:
 
<syntaxhighlight lang='go'>
import (
"testing"
"github.com/stretchr/testify/require"
)
 
func TestSomeFunction(t *testing.T) {
testCases := []struct {
name  string
input  string
assert func(require *require.Assertions, result string, err error) // can be replaced with 'expectedResult'
                                                                          // if no error is expected
        debug  bool // used to stop the debugger on a specific case, by setting a conditional breakpoint on tc.debug
}{
{
name:  "invocation with empty string",
input: "",
assert: func(require *require.Assertions, result string, err error) {
require.NoError(err)
require.Equal("", result)
},
},
{
name:  "invocation with non-empty string",
input: "something",
assert: func(require *require.Assertions, result string, err error) {
require.NoError(err)
require.Equal("SOMETHING", result)
},
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
result, err := SomeFunction(tc.input)
tc.assert(require.New(t), result, err)
})
}
}
}
}
</syntaxhighlight>
</syntaxhighlight>


=TO DEPLETE=
This is the template:
<font color=darkkhaki>
 
==Writing a Unit Test==
<syntaxhighlight lang='go'>
import (
testifyrequire "github.com/stretchr/testify/require"
"testing"
)


* Write a <tt>_test.go</tt> test file. They should belong to the same package as the tested code. These files are ignored by the compiler and only compiled and executed when <tt>go test</tt> is run.
func TestSomeMethod(t *testing.T) {
testCases := []struct {
name  string
input  any
require func(require *testifyrequire.Assertions, result any, err error)
}{
{},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
            require := testifyrequire.New(t)
result, err := SomeMethod(tc.input)
tc.require(require, result, err)
})
}
}
</syntaxhighlight>


* Import "testing"
==<span id='Integration_Testing'><span>Integration Tests==
We document implementation of integration tests in go. We use this definition of '''integration test''': {{Internal|Software_Testing_Concepts#Integration_Test|Software Testing Concepts &#124; Integration Test}}


* Tests are identified as functions starting with <tt>Test_...</tt> and taking an argument <tt>(t *testing.T)</tt>.
Integration tests, with or without mocks, should be placed in files named <code>*_integration_test.go</code> files. The name of the file, except the <code>..._test.go</code> part that is mandated by the Go test compiler, has no influence over whether the tests contained in it are executed or not. We control the conditional execution with <span id='integration_test_Build_Tag'></span>[[Go_Build_Tags#Controlling_Test_Execution_with_Build_Tags|build tags]]:


<pre>
<syntaxhighlight lang='go'>
package blue
//go:build integration_test


import "testing"
package somepkg
</syntaxhighlight>


func TestBlue(t *testing.T) {
<font size=-2>
    ...
.
    t.Error("expected this, got ", ...)
└─ internal
}
     └── mypkg
</pre>
         ├── mypkg.go
         ├── mypkg_test.go
         └── mypkg_integration_test.go
</font>
===Code Coverage for Integration Tests===
<font color=darkkhaki>TO PROCESS: https://go.dev/blog/integration-test-coverage</font>
Also see: {{Internal|Go_test_Command#Test_Coverage|<tt>go test</tt> Test Coverage}}


Then
==System Tests==
We use this definition of '''system test''': {{Internal|Software_Testing_Concepts#System_Test|Software Testing Concepts &#124; System Test}}


<pre>
System tests should be placed in files conventionally named <code>*_system_test.go</code> files and should declare a <span id='system_test_Build_Tag'></span><code>system_test</code> [[Go_Build_Tags#Controlling_Test_Execution_with_Build_Tags|build tag]] at the top of the file:
go test
</pre>


The command will look for any tests in any of the files in the current folder and run them.
<syntaxhighlight lang='go'>
//go:build system_test


Also see [[Go_Packages#External_Test_Packages|external test packages]].
package somepkg
</syntaxhighlight>


<font color=red>'''TODO''' testing idiom "Introducing Go" page 96.</font>
=Benchmarks=
<font color=darkkhaki>Process this: https://pkg.go.dev/testing#hdr-Benchmarks</font>.
=Fuzzing=
<font color=darkkhaki>Process this: https://pkg.go.dev/testing#hdr-Fuzzing</font>.
=Skipping=
<font color=darkkhaki>Process this: https://pkg.go.dev/testing#hdr-Skipping</font>.
=Subtests and Sub-benchmarks=
<font color=darkkhaki>Process this: https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks</font>.
=Main=
<font color=darkkhaki>Process this: https://pkg.go.dev/testing#hdr-Main</font>.
=Mocks=
Mock support in Go:
{{Internal|Github.com/stretchr/testify#Mocks|Mocks with Testify}}
{{Internal|github.com/uber/mock#Overview|Uber gomock}}

Latest revision as of 22:07, 3 September 2024

External

Internal

TO DO

Overview

Go comes with a lightweight test framework that includes the go test command and the testing package. The tests live in *_test.go files, located in the package directory.

Packages

Test Types in Go

Unit Tests

somepkg_test.go files should contain exclusively unit tests, not integration or system tests. Integration tests should be stored in *_integration_test.go files and system tests in *_system_test.go files. Optionally, unit test files may include the unit_test build tag, which allows controlling test execution based on their type (unit, functional and system).

//go:build unit_test

package somepkg

If no tag is declared, the tests will run fine, but we lose the possibility to control when we don't want them to run. They will always run.

Also see:

Software Testing Concepts | Unit Tests

testing.T

t.Cleanup(func())

Writing a Unit Test

TODO: revisit for Testify.

Write a module as shown here:

Declaring a Module

For each file containing behavior to test (a.go)

package a

func Reverse(s string) string {
 rs := []rune(s)
 var result []rune
 for i := len(rs) - 1; i >= 0; i-- {
  result = append(result, rs[i])
 }
 return string(result)
}

add a <file-name>_test.go test file. In this case a_test.go. These files are ignored by the compiler and only compiled and executed when go test is run. These files will be excluded from regular package builds. For more details on how go test command handles different files, see:

go test File Selection

The test files should be part of the same package. If that is the case, the test has access to unexpected identifiers from the package being tested. It is also possible to declare the test files into a corresponding package with the suffix _test. In this case, the package being tested must be imported explicitly in the test. This is known as "black box" testing. Also see external test packages.

The test files should import testing.

Add individual tests, as functions starting with TestX.., where X... does not start with a lowercase letter, and taking an argument t *testing.T. The function name serves to identify the test routine. Within these functions, use Error, Fail or related methods to signal failure.

func TestX...(t *testing.T) {
  ...
  t.Error("expected this: %q, got that: %q ", ...)
}
package a

import "testing"

func TestReverseEmptyString(t *testing.T) {
 expected := ""
 result := Reverse("")
 if result != expected {
  t.Errorf("expected %q, got %q", expected, result)
 }
}

func TestReverseOneCharString(t *testing.T) {
 expected := "a"
 result := Reverse("a")
 if result != expected {
  t.Errorf("expected %q, got %q", expected, result)
 }
}

func TestReverseTwoCharString(t *testing.T) {
 expected := "ba"
 result := Reverse("ab")
 if result != expected {
  t.Errorf("expected %q, got %q", expected, result)
 }
}

From the module directory, run the tests:

go test

PASS
ok  	example.com/a	0.116s

Table (Tabular) Tests

The "table" ("tabular") format is useful when we need to test a function with a variety of input parameters. The "table" consists in a list of structures (hence the tabular format). Each structure instance provides the test name, the input parameters to be tested, and an assertion function that takes the function result and error and perform assertions correlated to the input. If we simply test for expected results, the assert function can be replaced with the expected results.

Assuming that SomeFunction() is a function to test:

func SomeFunction(arg string) (string, error) {
	return strings.ToUpper(arg), nil
}

then a tabular test would be:

import (
	"testing"
	"github.com/stretchr/testify/require"
)

func TestSomeFunction(t *testing.T) {
	testCases := []struct {
		name   string
		input  string
		assert func(require *require.Assertions, result string, err error) // can be replaced with 'expectedResult'
                                                                           // if no error is expected
        debug  bool // used to stop the debugger on a specific case, by setting a conditional breakpoint on tc.debug
	}{
		{
			name:  "invocation with empty string",
			input: "",
			assert: func(require *require.Assertions, result string, err error) {
				require.NoError(err)
				require.Equal("", result)
			},
		},
		{
			name:  "invocation with non-empty string",
			input: "something",
			assert: func(require *require.Assertions, result string, err error) {
				require.NoError(err)
				require.Equal("SOMETHING", result)
			},
		},
	}
	for _, tc := range testCases {
		t.Run(tc.name, func(t *testing.T) {
			result, err := SomeFunction(tc.input)
			tc.assert(require.New(t), result, err)
		})
	}
}

This is the template:

import (
	testifyrequire "github.com/stretchr/testify/require"
	"testing"
)

func TestSomeMethod(t *testing.T) {
	testCases := []struct {
		name   string
		input  any
		require func(require *testifyrequire.Assertions, result any, err error)
	}{
		{},
	}
	for _, tc := range testCases {
		t.Run(tc.name, func(t *testing.T) {
            require := testifyrequire.New(t)
			result, err := SomeMethod(tc.input)
			tc.require(require, result, err)
		})
	}
}

Integration Tests

We document implementation of integration tests in go. We use this definition of integration test:

Software Testing Concepts | Integration Test

Integration tests, with or without mocks, should be placed in files named *_integration_test.go files. The name of the file, except the ..._test.go part that is mandated by the Go test compiler, has no influence over whether the tests contained in it are executed or not. We control the conditional execution with build tags:

//go:build integration_test

package somepkg

.
└─ internal
    └── mypkg
        ├── mypkg.go
        ├── mypkg_test.go
        └── mypkg_integration_test.go

Code Coverage for Integration Tests

TO PROCESS: https://go.dev/blog/integration-test-coverage

Also see:

go test Test Coverage

System Tests

We use this definition of system test:

Software Testing Concepts | System Test

System tests should be placed in files conventionally named *_system_test.go files and should declare a system_test build tag at the top of the file:

//go:build system_test

package somepkg

Benchmarks

Process this: https://pkg.go.dev/testing#hdr-Benchmarks.

Fuzzing

Process this: https://pkg.go.dev/testing#hdr-Fuzzing.

Skipping

Process this: https://pkg.go.dev/testing#hdr-Skipping.

Subtests and Sub-benchmarks

Process this: https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks.

Main

Process this: https://pkg.go.dev/testing#hdr-Main.

Mocks

Mock support in Go:

Mocks with Testify
Uber gomock