Testify require and assert: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(45 intermediate revisions by the same user not shown)
Line 2: Line 2:
* https://pkg.go.dev/github.com/stretchr/testify#readme-assert-package
* https://pkg.go.dev/github.com/stretchr/testify#readme-assert-package
=Internal=
=Internal=
* [[Github.com/stretchr/testify#Assertions|Testify]]
* [[Github.com/stretchr/testify#require_and_assert|Testify]]
 
=Overview=
=Overview=
<code>require</code> and <code>assert</code> packages provide functions to evaluate arguments and establish whether the test passes or fails.
Experience shows that it's a good idea to use the "f" form of the functions (<code>Equalf()</code>, <code>Nilf()</code>, <code>Containsf()</code>, etc.) to provide messages that serve as documentation for the code and also explicit messages on failure.
<syntaxhighlight lang='go'>
result, err := DoSomething(model)
require.NotNilf(err, "failed to DoSomething() on model %v, error: ", model, err)
</syntaxhighlight>
=Calling <tt>assert</tt> and <tt>require</tt> from Goroutines=
The major difference between <code>require</code> and <code>assert</code> is that <code>assert</code> means '''continue on error'''. If an assertion fails, the test reports it and the goroutine the asserting failure happened on '''continues execution'''. In the end, the overall test is counted as "failed", but while executing, the test keeps going on failed assertions. This is different from the behavior of <code>assert</code> in other languages and frameworks, where the first failed assertion stops the current test. This does not make too much sense - if an assertion is broken, I am not that interested in what comes after that, they are likely to be broken too, or at least, not to be trusted. <code>assert</code> exit behavior is internally implemented by calling <code>[[#Fail()|Fail()]]</code>.
If you want the test to exit the '''current goroutine''' after the first failed assertion, use <code>require</code>. <code>require</code> means '''exit the goroutine on failed assertion'''. However, <code>require</code> does not exit the test. If the test is multi-threaded, you need to account for what other goroutines are doing. For example, if you use <code>require</code> from a goroutine that is supposed to release a semaphore, and <code>require</code> fails, the test might get deadlocked. <code>require</code> exit behavior is internally implemented by calling <code>[[#FailNow()|FailNow()]]</code>.
and <code>require</code> calling <code>FailNow()</code>.
<font color=darkkhaki>What happens when an <code>assert</code> fails? How is the test state impacted? Debug and find out.</font>
=Equality and Non-Equality=
=Equality and Non-Equality=
In case the result is failure, <code>Equal()</code> and <code>NotEqual()</code> delegate to <code>Fail()</code>.


=Nil and Non-Nil=
It works with structs, slices.
<syntaxhighlight lang='go'>
package somepkg


=Failing the Test=
import (
==Failing a Test from a Goroutine==
"testing"


testifyassert "github.com/stretchr/testify/assert"
    testifyrequire "github.com/stretchr/testify/require"
)


func TestSomething(t *testing.T) {
assert := testifyassert.New(t)
    require := testifyrequire.New(t)
    assert.Equal(123, 123, "this message is displayed when the assertion fails, but the test continues %s", "because of inequality")
    require.Equal(123, 123, "this message is displayed when the test fails and exists %s", "because of inequality")
assert.NotEqual(123, 321, "this message is displayed when the test fails %s", "because of equality")
    require.NotEqual(123, 321, "this message is displayed when the test fails and exists %s", "because of inequality")
}
</syntaxhighlight>


=Nil and Non-Nil=
In case the result is failure, <code>Nil()</code> and <code>NotNil()</code> delegate to <code>Fail()</code>.
<syntaxhighlight lang='go'>
<syntaxhighlight lang='go'>
package yours
package somepkg


import (
import (
  "testing"
"testing"
  tassert "github.com/stretchr/testify/assert"
 
testifyassert "github.com/stretchr/testify/assert"
    testifyrequire "github.com/stretchr/testify/require"
)
)


func TestSomething(t *testing.T) {
func TestSomething(t *testing.T) {
assert := testifyassert.New(t)
    require := testifyrequire.New(t)
someObj := &struct{}{}
assert.NotNil(someObj, "someObj should not have been nil but it is %v, and the test continues", someObj)
require.NotNil(someObj, "someObj should not have been nil but it is %v, and the test exits", someObj)
someObj = nil
assert.Nil(someObj, "someObj should have been nil but it is %v, and the test continues", someObj)
    require.Nil(someObj, "someObj should have been nil but it is %v, and the test exits", someObj)
}
</syntaxhighlight>


  assert := tassert.New(t)
=Other Test Functions=
==<tt>Contains()</tt>==
Applies to strings, slices and maps.
<syntaxhighlight lang='go'>
Contains(whole, fragment)
</syntaxhighlight>
==<tt>ErrorContains()</tt>==
<syntaxhighlight lang='go'>
err := fmt.Error("something")
require.ErrorContains(err, "something")
</syntaxhighlight>


  // assert equality
=Failing the Test=
  assert.Equal(123, 123, "they should be equal")
==<tt>Fail()</tt>==


  // assert inequality
Does not exit the current goroutine, so it does not stop the test. It annotates its internal state that there was a failure, and let the test continue. <code>require.Fail()</code> and <code>assert.Fail()</code> behavior is identical.
  assert.NotEqual(123, 456, "they should not be equal")


  // assert for nil (good for errors)
==<tt>Failf()</tt>==
  assert.Nil(object)
Same as <code>[[#Failf()|Fail()]]</code>, but provide additional message features.


  // assert for not nil (good when you expect something)
==<tt>FailNow()</tt>==
  if assert.NotNil(object) {
Annotate the test with failure and exit the current goroutine. This does not necessarily mean the test exits, if there are other goroutine going. <code>require.FailNow()</code> and <code>assert.FailNow()</code> behavior is identical.


    // now we know that object isn't nil, we are safe to make
==<tt>FailNowf()</tt>==
    // further assertions without causing any errors
Same as <code>[[#FailNow()|FailNow()]]</code>, but provide additional message features.
    assert.Equal("Something", object.Value)
  }
}
</syntaxhighlight>


To check that an error has the expected message:
=Checking that an Error Has Expected Message=
<syntaxhighlight lang='go'>
<syntaxhighlight lang='go'>
err := ...
err := ...
Line 50: Line 104:
assert.Equal(err.Error(), "expected message")
assert.Equal(err.Error(), "expected message")
</syntaxhighlight>
</syntaxhighlight>
===Failing the Test===

Latest revision as of 02:03, 15 March 2024

External

Internal

Overview

require and assert packages provide functions to evaluate arguments and establish whether the test passes or fails.

Experience shows that it's a good idea to use the "f" form of the functions (Equalf(), Nilf(), Containsf(), etc.) to provide messages that serve as documentation for the code and also explicit messages on failure.

result, err := DoSomething(model)
require.NotNilf(err, "failed to DoSomething() on model %v, error: ", model, err)

Calling assert and require from Goroutines

The major difference between require and assert is that assert means continue on error. If an assertion fails, the test reports it and the goroutine the asserting failure happened on continues execution. In the end, the overall test is counted as "failed", but while executing, the test keeps going on failed assertions. This is different from the behavior of assert in other languages and frameworks, where the first failed assertion stops the current test. This does not make too much sense - if an assertion is broken, I am not that interested in what comes after that, they are likely to be broken too, or at least, not to be trusted. assert exit behavior is internally implemented by calling Fail().

If you want the test to exit the current goroutine after the first failed assertion, use require. require means exit the goroutine on failed assertion. However, require does not exit the test. If the test is multi-threaded, you need to account for what other goroutines are doing. For example, if you use require from a goroutine that is supposed to release a semaphore, and require fails, the test might get deadlocked. require exit behavior is internally implemented by calling FailNow().

and require calling FailNow().

What happens when an assert fails? How is the test state impacted? Debug and find out.

Equality and Non-Equality

In case the result is failure, Equal() and NotEqual() delegate to Fail().

It works with structs, slices.

package somepkg

import (
	"testing"

	testifyassert "github.com/stretchr/testify/assert"
    testifyrequire "github.com/stretchr/testify/require"
)

func TestSomething(t *testing.T) {
	assert := testifyassert.New(t)
    require := testifyrequire.New(t)
    assert.Equal(123, 123, "this message is displayed when the assertion fails, but the test continues %s", "because of inequality")
    require.Equal(123, 123, "this message is displayed when the test fails and exists %s", "because of inequality") 
	assert.NotEqual(123, 321, "this message is displayed when the test fails %s", "because of equality")
    require.NotEqual(123, 321, "this message is displayed when the test fails and exists %s", "because of inequality") 
}

Nil and Non-Nil

In case the result is failure, Nil() and NotNil() delegate to Fail().

package somepkg

import (
	"testing"

	testifyassert "github.com/stretchr/testify/assert"
    testifyrequire "github.com/stretchr/testify/require"
)

func TestSomething(t *testing.T) {
	assert := testifyassert.New(t)
    require := testifyrequire.New(t)
	someObj := &struct{}{}
	assert.NotNil(someObj, "someObj should not have been nil but it is %v, and the test continues", someObj)
	require.NotNil(someObj, "someObj should not have been nil but it is %v, and the test exits", someObj)
	someObj = nil
	assert.Nil(someObj, "someObj should have been nil but it is %v, and the test continues", someObj)
    require.Nil(someObj, "someObj should have been nil but it is %v, and the test exits", someObj)
}

Other Test Functions

Contains()

Applies to strings, slices and maps.

Contains(whole, fragment)

ErrorContains()

err := fmt.Error("something")
require.ErrorContains(err, "something")

Failing the Test

Fail()

Does not exit the current goroutine, so it does not stop the test. It annotates its internal state that there was a failure, and let the test continue. require.Fail() and assert.Fail() behavior is identical.

Failf()

Same as Fail(), but provide additional message features.

FailNow()

Annotate the test with failure and exit the current goroutine. This does not necessarily mean the test exits, if there are other goroutine going. require.FailNow() and assert.FailNow() behavior is identical.

FailNowf()

Same as FailNow(), but provide additional message features.

Checking that an Error Has Expected Message

err := ...
assert.NotNil(err)
assert.Equal(err.Error(), "expected message")