Bats Operations
Internal
Best Practices
Script to Run all .bats Tests from a Directory or Multiple Directories
Single Directory
#!/usr/bin/env bash
for i in $(dirname $0)/*.bats; do
echo "running $(basename ${i}) tests ..."
bats $i
done
Multiple Directories
#!/usr/bin/env bash
cd ${dir}
for d in $(find . -type d -not -name '.'); do
(
d=${d#./}
echo "running all tests from directory '${d}' ..."
cd ${d};
for i in *.bats; do
echo "running $(basename ${i}) tests ..."
bats ${i}
done
)
done
Loading a Library to be Tested into a .bats Test
Assuming that the library you want to test is called 'blue.shlib', create a 'blue-library.bash' file in the directory that contain the .bats tests. If the library is located at a fixed relative location to the .bats test files, use this arrangement:
source ${BATS_TEST_DIRNAME}/[...]/blue.shlib
Otherwise, you can use an absolute path, but that is more fragile.
From each .bats test that wants to test functionality from that library:
load blue-library
@test "something" {
...
}
For more details, see:
Loading an Executable Script as Library to be Tested into a .bats Test
It is possible to test functions from an executable script by loading the executable script as library, even if the script loads external libraries itself, and invokes a main
function. We use the same idea described above in the Loading a Library to be Tested into a .bats Test section, with a few additions. Let's assume there is a script called blue that is executable (contains a main()
invocation), sources the bash.shlib library, and contains functions that we would like to unit test:
#!/usr/bin/env bash
#
# this is blue
#
[ -f $(dirname $0)/lib/bash.shlib ] && source $(dirname $0)/lib/bash.shlib || { echo "[error]: $(dirname $0)/lib/bash.shlib not found" 1>&2; exit 1; }
function main() {
...
}
function auxiliary() {
...
}
main "$@"
We would like to test auxiliary()
by loading blue as a library.
For that, the following changes need to be introduced in blue:
1. Guard library loading with a BATS_TESTING
variable:
...
[ -z "${BATS_TESTING}" ] && { [ -f $(dirname $0)/lib/bash.shlib ] && source $(dirname $0)/lib/bash.shlib || { echo "[error]: $(dirname $0)/lib/bash.shlib not found" 1>&2; exit 1; } }
...
2. Guard main()
invocation with the same BATS_TESTING
variable:
...
if [ -z "${BATS_TESTING}" ]; then
main "$@"
fi
...
Interestingly enough, this does not work, it breaks BATS:
...
[ -z "${BATS_TESTING}" ] && main "$@"
...
3. Create the all-required-libraries.bash library loader in the directory that contains the .bats tests. The library loader is similar to the one described in the section Loading a Library to be Tested into a .bats Test above. The difference is that it must set the BATS_TESTING
to true, load the executable script as library, and also load all libraries required by the executable script:
export BATS_TESTING=true
source ${BATS_TEST_DIRNAME}/../../lib/bash.shlib
source ${BATS_TEST_DIRNAME}/../../blue
4. For each .bats test that wants to test functionality from the executable script, load "all-required-libraries", similarly to how is described in Loading a Library to be Tested into a .bats Test:
load all-required-libraries
@test "something" {
...
}
Handling stdout and stderr
function teardown() {
rm -f ${BATS_TEST_DIRNAME}/tmp/stdout
rm -f ${BATS_TEST_DIRNAME}/tmp/stderr
}
@test "stderr" {
run $(fail "blah" 1>${BATS_TEST_DIRNAME}/tmp/stdout 2>${BATS_TEST_DIRNAME}/tmp/stderr )
[[ -z $(cat ${BATS_TEST_DIRNAME}/tmp/stdout) ]]
[[ $(cat ${BATS_TEST_DIRNAME}/tmp/stderr) = "[failure]: blah" ]]
}
This is a proposed solution, I need to do more research:
@test "test all outputl" {
local stdoutPath="${BATS_TMPDIR}/${BATS_TEST_NAME}.stdout"
local stderrPath="${BATS_TMPDIR}/${BATS_TEST_NAME}.stderr"
myCommandOrFunction 1>${stdoutPath} 2>${stderrPath}
grep "What Im looking for in stdout" ${stdoutPath}
! grep "something it cannot appear in stdout" ${stdoutPath}
grep "What I'm looking for in stderr" ${stderrPath}
! grep "Something it cannot appear in stderr" ${stderrPath}
}
Handling data
A useful pattern is to create a "data" subdirectory in the directory that holds the .bats tests, and then access it from the tests with ${BATS_TEST_DIRNAME}/data/...
"Override" Functions
A useful technique when testing a layer is to "override" (declare) the functions called from that layer right before the test, so we can control input and output in the layer.
function setup() {
function some-function() {
echo "some-function($@)"
}
}
...
@test "test invocation through layer to some-function()" {
run caller-function A B C
[[ ${status} -eq 0 ]]
[[ "${output}" =~ some-function\(.*\) ]]
args=${output#some-function(}
args=${args%)}
arg0=${args%% *}
[[ ${arg0} = "A" ]]
args=${args#${arg0} }
arg1=${args%% *}
[[ ${arg1} = "B" ]]
...
}
The function do not need to be "overridden" in setup(), they can be overridden in the test itself:
@test "test invocation through layer to some-function()" {
function some-function-that-is-invoked-inside-caller-function() {
exit 1
}
run caller-function A B C
[[ ${status} -eq 1 ]]
# ...
}
Debugging Failures
BATS does not handle error logging gracefully. A useful technique is to turn verbosity in bash.shlib debug()/error()/warn()/info() functions and to configure them to send output into an alternate file, for problematic tests:
VERBOSE=true; DEBUG_OUTPUT=~/bats.out
Run the tests as follows:
rm ~/tmp/bats.out ; clear; bats ...test/some-test.bats; cat ~/tmp/bats.out