Software Testing Concepts

Internal

• Testing

Overview

Automated testing provides an effective mechanism for catching regressions, especially when combined with test-driven development.

Automated Test

An automated test verifies an assumption about the behavior of the system and provides a safety mesh that is exercised continuously, again and again, in an automated fashion, in most cases on each commit in the repository. The benefits of automated testing and that the software is continuously verified, maintaining its quality. Another benefit of tests is that they serve as documentation for code.

Unit Test

Unit testing is the testing of the smallest possible part of software, such a single method, a small set of related methods or a class. In reality we test logical units - which can extend to a method, a single class or multiple classes. A unit test has the following characteristics:

• It should be automated.
• It should be fast - not more that a few milliseconds to finish execution.
• It should be self-contained and runnable in isolation, and then in any order as part of the test suite. The unit test should not depend on the result of another test or on the execution order.One should understand what is going on in a unit test without having to look at other parts of the code.
• It should not depend on database access, file access or any long running task. If the presence of external dependencies are necessary to test the logic, they should be provided as test doubles.
• It should be time (time of the day, timezone) and location independent.
• It should be meaningful. Getter or setter testing is not meaningful.
• It should be usable as documentation: readable and expressive.

External Dependency

A database, an external service accessible over the network.

Integration Test

Integration test depend on real external dependencies, such databases, and they are inherently slower than the unit tests. Integration tests should be automated, but they should run outside the continuous unit test feedback loop.

Test Double

A test double is meant to replace a real external dependency in the unit test cycle, isolating it from the real dependency. This may be necessary either because the external dependency is unavailable, or the interaction with it is slow. The term was introduced by Gerard Meszaros in his xUnit Test Patterns book.

Dummy

A dummy object is passed as a mandatory parameter object but is not directly used in the test code or the code under test. The dummy object is required for the creation of another object required in the code under test. When implemented as a class, all methods should throw a "Not Implemented" runtime exception.

Stub

A stub delivers indirect inputs to the caller when the stub's methods are invoked. Stubs are programmed only for the test scope. They may record other information such as how many times are invoked, etc. Stubs' methods can be programmed to return hardcoded results or to throw specific exceptions.

Stubs are useful in impersonating error conditions in external dependencies. The same result can be achieve with a mock.

Spy

A spy is a variation of a stub but instead of only setting the expectation, a spy records the method calls made to the collaborator. A spy can act as an indirect output of the unit under test and can also act as an audit log.

Mock

A mock object is a combination of a stub and a spy. It stubs methods to return values and throw exceptions, like a stub, but it acts as an indirect output for code under test, as a spy. A mock object fails a test if an expected method is not invoked, or if the parameter of the method do not match. The Mockito framework provides an API to mock objects.