Go Testing: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(88 intermediate revisions by the same user not shown)
Line 1: Line 1:
=External=
=External=
* https://go.dev/doc/code#Testing
* https://go.dev/doc/code#Testing
* https://pkg.go.dev/testing
=Internal=
=Internal=
* [[Testing]]
* [[Go Engineering#Subjects|Go Engineering]]
* [[Go Engineering#Subjects|Go Engineering]]
* [[Go_Tool#test|<code>go test</code> command]]
* [[Go_test_Command#Overview|<code>go test</code> command]]
=TO DO=
<font color=darkkhaki>
* A system to allow testing with real temporary files in <code>./build</code> in Go.
* Integrate see [[Go_Packages#External_Test_Packages|external test packages]].
* Testing idiom "Introducing Go" page 96.
* Investigate gotest.tools module.
* Running go test immediately after cloning should succeed without any configuration or any running database. If a database is required, those are integration tests.
* https://about.sourcegraph.com/blog/go/advanced-testing-in-go
* Understand unit test lifecycle. Is package initialization performed once per test package?
* Try Testify set up and tear down test suite fixtures, which can save you time and reduce code duplication: https://blog.stackademic.com/using-testify-in-golang-a-comprehensive-guide-8e0417529669
* Try gotest.tools/v3 v3.0.3.
* A mock framework for Go: https://pkg.go.dev/github.com/golang/mock/gomock
* Testable examples in Go: https://go.dev/blog/examples
* https://blog.jetbrains.com/go/2022/11/22/comprehensive-guide-to-testing-in-go/
</font>


=Overview=
=Overview=
Go comes with a lightweight test framework that includes the [[Go_Tool#test|<code>go test</code>]] command and the <code>[[Go Package testing#Overview|testing]]</code> package.
Go comes with a lightweight test framework that includes the [[Go_test_Command#Overview|<code>go test</code>]] command and the <code>[[Go Package testing#Overview|testing]]</code> package. The tests live in <code>*_test.go</code> files, located in the package directory.
=<span id='Helper_Packages'></span>Packages=
* [[gotest.tools#Overview|gotest.tools]]
* [[github.com/stretchr/testify#Overview|github.com/stretchr/testify]]
=Test Types in Go=
==Unit Tests==
<code>somepkg_test.go</code> files should contain exclusively unit tests, not integration or system tests. Integration tests should be stored in <code>*_integration_test.go</code> files and system tests in <code>*_system_test.go</code> files. <span id='unit_test_Build_Tag'></span>Optionally, unit test files may include the <code>unit_test</code> [[Go_Build_Tags#Controlling_Test_Execution_with_Build_Tags|build tag]], which allows controlling test execution based on their type (unit, functional and system).


The tests live in <code>*_test.go</code> files.
<syntaxhighlight lang='go'>
//go:build unit_test


Also see [[Go_Packages#External_Test_Packages|external test packages]].
package somepkg
</syntaxhighlight>


=TO DEPLETE=
If no tag is declared,  the tests will run fine, but we lose the possibility to control when we don't want them to run. They will '''always''' run.
<font color=darkkhaki>
Also see: {{Internal|Software_Testing_Concepts#Unit_Test|Software Testing Concepts &#124; Unit Tests}}
===<tt>testing.T</tt>===
 
<code>t.Cleanup(func())</code>
 
===<span id='Write_a_Unit_Test'></span>Writing a Unit Test===
 
<font color=darkkhaki>TODO: revisit for Testify</font>.
 
Write a module as shown here: {{Internal|Go_Modules#Declaring_Modules|Declaring a Module}}
 
For each file containing behavior to test (<code>a.go</code>)
<syntaxhighlight lang='go'>
package a
 
func Reverse(s string) string {
rs := []rune(s)
var result []rune
for i := len(rs) - 1; i >= 0; i-- {
  result = append(result, rs[i])
}
return string(result)
}
</syntaxhighlight>


add a <code><file-name>_test.go</code> test file. In this case <code>a_test.go</code>.  These files are ignored by the compiler and only compiled and executed when <code>go test</code> is run. These files will be excluded from regular package builds. For more details on how <code>go test</code> command handles different files, see: {{Internal|Go_test_Command#File_Selection|<tt>go test</tt> File Selection}}


==Writing a Unit Test==
The test files should be part of the same package. If that is the case, the test has access to unexpected identifiers from the package being tested.  It is also possible to declare the test files into a corresponding package with the suffix <code>_test</code>. In this case, the package being tested must be imported explicitly in the test. This is known as "black box" testing. Also see [[Go_Packages#External_Test_Packages|external test packages]].


* Write a <tt>_test.go</tt> test file. They should belong to the same package as the tested code. These files are ignored by the compiler and only compiled and executed when <tt>go test</tt> is run.
The test files should import <code>testing</code>.


* Import "testing"
Add individual tests, as functions starting with <code>TestX..</code>, where <code>X...</code> does not start with a lowercase letter, and taking an argument <code>t *testing.T</code>. The function name serves to identify the test routine. Within these functions, use <code>Error</code>, <code>Fail</code> or related methods to signal failure.


* Tests are identified as functions starting with <tt>Test_...</tt> and taking an argument <tt>(t *testing.T)</tt>.
<syntaxhighlight lang='go'>
func TestX...(t *testing.T) {
  ...
  t.Error("expected this: %q, got that: %q ", ...)
}
</syntaxhighlight>


<pre>
<syntaxhighlight lang='go'>
package blue
package a


import "testing"
import "testing"


func TestBlue(t *testing.T) {
func TestReverseEmptyString(t *testing.T) {
    ...
expected := ""
    t.Error("expected this, got ", ...)
result := Reverse("")
if result != expected {
  t.Errorf("expected %q, got %q", expected, result)
}
}
}
</pre>


Then
func TestReverseOneCharString(t *testing.T) {
expected := "a"
result := Reverse("a")
if result != expected {
  t.Errorf("expected %q, got %q", expected, result)
}
}


<pre>
func TestReverseTwoCharString(t *testing.T) {
expected := "ba"
result := Reverse("ab")
if result != expected {
  t.Errorf("expected %q, got %q", expected, result)
}
}
</syntaxhighlight>
 
From the module directory, run the tests:
<syntaxhighlight lang='bash'>
go test
go test
</pre>
</syntaxhighlight>
<font size=-1>
PASS
ok  example.com/a 0.116s
</font>
 
===Table (Tabular) Tests===
 
The "table" ("tabular") format is useful when we need to test a function with a variety of input parameters. The "table" consists in a list of structures (hence the tabular format). Each structure instance provides the test name, the input parameters to be tested, and an assertion function that takes the function result and error and perform assertions correlated to the input. If we simply test for expected results, the assert function can be replaced with the expected results.
 
Assuming that <code>SomeFunction()</code> is a function to test:
 
<syntaxhighlight lang='go'>
func SomeFunction(arg string) (string, error) {
return strings.ToUpper(arg), nil
}
</syntaxhighlight>
 
then a tabular test would be:
 
<syntaxhighlight lang='go'>
import (
"testing"
"github.com/stretchr/testify/require"
)
 
func TestSomeFunction(t *testing.T) {
testCases := []struct {
name  string
input  string
assert func(require *require.Assertions, result string, err error) // can be replaced with 'expectedResult'
                                                                          // if no error is expected
        debug  bool // used to stop the debugger on a specific case, by setting a conditional breakpoint on tc.debug
}{
{
name:  "invocation with empty string",
input: "",
assert: func(require *require.Assertions, result string, err error) {
require.NoError(err)
require.Equal("", result)
},
},
{
name:  "invocation with non-empty string",
input: "something",
assert: func(require *require.Assertions, result string, err error) {
require.NoError(err)
require.Equal("SOMETHING", result)
},
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
result, err := SomeFunction(tc.input)
tc.assert(require.New(t), result, err)
})
}
}
</syntaxhighlight>
 
This is the template:
 
<syntaxhighlight lang='go'>
import (
testifyrequire "github.com/stretchr/testify/require"
"testing"
)
 
func TestSomeMethod(t *testing.T) {
testCases := []struct {
name  string
input  any
require func(require *testifyrequire.Assertions, result any, err error)
}{
{},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
            require := testifyrequire.New(t)
result, err := SomeMethod(tc.input)
tc.require(require, result, err)
})
}
}
</syntaxhighlight>
 
==<span id='Integration_Testing'><span>Integration Tests==
We document implementation of integration tests in go. We use this definition of '''integration test''': {{Internal|Software_Testing_Concepts#Integration_Test|Software Testing Concepts &#124; Integration Test}}
 
Integration tests, with or without mocks, should be placed in files named <code>*_integration_test.go</code> files. The name of the file, except the <code>..._test.go</code> part that is mandated by the Go test compiler, has no influence over whether the tests contained in it are executed or not. We control the conditional execution with <span id='integration_test_Build_Tag'></span>[[Go_Build_Tags#Controlling_Test_Execution_with_Build_Tags|build tags]]:
 
<syntaxhighlight lang='go'>
//go:build integration_test
 
package somepkg
</syntaxhighlight>
 
<font size=-2>
.
└─ internal
     └── mypkg
         ├── mypkg.go
         ├── mypkg_test.go
         └── mypkg_integration_test.go
</font>
===Code Coverage for Integration Tests===
<font color=darkkhaki>TO PROCESS: https://go.dev/blog/integration-test-coverage</font>
Also see: {{Internal|Go_test_Command#Test_Coverage|<tt>go test</tt> Test Coverage}}
 
==System Tests==
We use this definition of '''system test''': {{Internal|Software_Testing_Concepts#System_Test|Software Testing Concepts &#124; System Test}}
 
System tests should be placed in files conventionally named <code>*_system_test.go</code> files and should declare a <span id='system_test_Build_Tag'></span><code>system_test</code> [[Go_Build_Tags#Controlling_Test_Execution_with_Build_Tags|build tag]] at the top of the file:
 
<syntaxhighlight lang='go'>
//go:build system_test


The command will look for any tests in any of the files in the current folder and run them.
package somepkg
</syntaxhighlight>


<font color=red>'''TODO''' testing idiom "Introducing Go" page 96.</font>
=Benchmarks=
<font color=darkkhaki>Process this: https://pkg.go.dev/testing#hdr-Benchmarks</font>.
=Fuzzing=
<font color=darkkhaki>Process this: https://pkg.go.dev/testing#hdr-Fuzzing</font>.
=Skipping=
<font color=darkkhaki>Process this: https://pkg.go.dev/testing#hdr-Skipping</font>.
=Subtests and Sub-benchmarks=
<font color=darkkhaki>Process this: https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks</font>.
=Main=
<font color=darkkhaki>Process this: https://pkg.go.dev/testing#hdr-Main</font>.
=Mocks=
Mock support in Go:
{{Internal|Github.com/stretchr/testify#Mocks|Mocks with Testify}}
{{Internal|github.com/uber/mock#Overview|Uber gomock}}

Latest revision as of 22:07, 3 September 2024

External

Internal

TO DO

Overview

Go comes with a lightweight test framework that includes the go test command and the testing package. The tests live in *_test.go files, located in the package directory.

Packages

Test Types in Go

Unit Tests

somepkg_test.go files should contain exclusively unit tests, not integration or system tests. Integration tests should be stored in *_integration_test.go files and system tests in *_system_test.go files. Optionally, unit test files may include the unit_test build tag, which allows controlling test execution based on their type (unit, functional and system).

//go:build unit_test

package somepkg

If no tag is declared, the tests will run fine, but we lose the possibility to control when we don't want them to run. They will always run.

Also see:

Software Testing Concepts | Unit Tests

testing.T

t.Cleanup(func())

Writing a Unit Test

TODO: revisit for Testify.

Write a module as shown here:

Declaring a Module

For each file containing behavior to test (a.go)

package a

func Reverse(s string) string {
 rs := []rune(s)
 var result []rune
 for i := len(rs) - 1; i >= 0; i-- {
  result = append(result, rs[i])
 }
 return string(result)
}

add a <file-name>_test.go test file. In this case a_test.go. These files are ignored by the compiler and only compiled and executed when go test is run. These files will be excluded from regular package builds. For more details on how go test command handles different files, see:

go test File Selection

The test files should be part of the same package. If that is the case, the test has access to unexpected identifiers from the package being tested. It is also possible to declare the test files into a corresponding package with the suffix _test. In this case, the package being tested must be imported explicitly in the test. This is known as "black box" testing. Also see external test packages.

The test files should import testing.

Add individual tests, as functions starting with TestX.., where X... does not start with a lowercase letter, and taking an argument t *testing.T. The function name serves to identify the test routine. Within these functions, use Error, Fail or related methods to signal failure.

func TestX...(t *testing.T) {
  ...
  t.Error("expected this: %q, got that: %q ", ...)
}
package a

import "testing"

func TestReverseEmptyString(t *testing.T) {
 expected := ""
 result := Reverse("")
 if result != expected {
  t.Errorf("expected %q, got %q", expected, result)
 }
}

func TestReverseOneCharString(t *testing.T) {
 expected := "a"
 result := Reverse("a")
 if result != expected {
  t.Errorf("expected %q, got %q", expected, result)
 }
}

func TestReverseTwoCharString(t *testing.T) {
 expected := "ba"
 result := Reverse("ab")
 if result != expected {
  t.Errorf("expected %q, got %q", expected, result)
 }
}

From the module directory, run the tests:

go test

PASS
ok  	example.com/a	0.116s

Table (Tabular) Tests

The "table" ("tabular") format is useful when we need to test a function with a variety of input parameters. The "table" consists in a list of structures (hence the tabular format). Each structure instance provides the test name, the input parameters to be tested, and an assertion function that takes the function result and error and perform assertions correlated to the input. If we simply test for expected results, the assert function can be replaced with the expected results.

Assuming that SomeFunction() is a function to test:

func SomeFunction(arg string) (string, error) {
	return strings.ToUpper(arg), nil
}

then a tabular test would be:

import (
	"testing"
	"github.com/stretchr/testify/require"
)

func TestSomeFunction(t *testing.T) {
	testCases := []struct {
		name   string
		input  string
		assert func(require *require.Assertions, result string, err error) // can be replaced with 'expectedResult'
                                                                           // if no error is expected
        debug  bool // used to stop the debugger on a specific case, by setting a conditional breakpoint on tc.debug
	}{
		{
			name:  "invocation with empty string",
			input: "",
			assert: func(require *require.Assertions, result string, err error) {
				require.NoError(err)
				require.Equal("", result)
			},
		},
		{
			name:  "invocation with non-empty string",
			input: "something",
			assert: func(require *require.Assertions, result string, err error) {
				require.NoError(err)
				require.Equal("SOMETHING", result)
			},
		},
	}
	for _, tc := range testCases {
		t.Run(tc.name, func(t *testing.T) {
			result, err := SomeFunction(tc.input)
			tc.assert(require.New(t), result, err)
		})
	}
}

This is the template:

import (
	testifyrequire "github.com/stretchr/testify/require"
	"testing"
)

func TestSomeMethod(t *testing.T) {
	testCases := []struct {
		name   string
		input  any
		require func(require *testifyrequire.Assertions, result any, err error)
	}{
		{},
	}
	for _, tc := range testCases {
		t.Run(tc.name, func(t *testing.T) {
            require := testifyrequire.New(t)
			result, err := SomeMethod(tc.input)
			tc.require(require, result, err)
		})
	}
}

Integration Tests

We document implementation of integration tests in go. We use this definition of integration test:

Software Testing Concepts | Integration Test

Integration tests, with or without mocks, should be placed in files named *_integration_test.go files. The name of the file, except the ..._test.go part that is mandated by the Go test compiler, has no influence over whether the tests contained in it are executed or not. We control the conditional execution with build tags:

//go:build integration_test

package somepkg

.
└─ internal
    └── mypkg
        ├── mypkg.go
        ├── mypkg_test.go
        └── mypkg_integration_test.go

Code Coverage for Integration Tests

TO PROCESS: https://go.dev/blog/integration-test-coverage

Also see:

go test Test Coverage

System Tests

We use this definition of system test:

Software Testing Concepts | System Test

System tests should be placed in files conventionally named *_system_test.go files and should declare a system_test build tag at the top of the file:

//go:build system_test

package somepkg

Benchmarks

Process this: https://pkg.go.dev/testing#hdr-Benchmarks.

Fuzzing

Process this: https://pkg.go.dev/testing#hdr-Fuzzing.

Skipping

Process this: https://pkg.go.dev/testing#hdr-Skipping.

Subtests and Sub-benchmarks

Process this: https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks.

Main

Process this: https://pkg.go.dev/testing#hdr-Main.

Mocks

Mock support in Go:

Mocks with Testify
Uber gomock