Bats Operations

Internal

• bats

Best Practices

Script to Run all .bats Tests from a Directory or Multiple Directories

Single Directory

#!/usr/bin/env bash

for i in $(dirname $0)/*.bats; do
    echo "running $(basename ${i}) tests ..."
    bats $i
done

Multiple Directories

#!/usr/bin/env bash

cd ${dir}

for d in $(find . -type d -not -name '.'); do
    (
        d=${d#./}
        echo "running all tests from directory '${d}' ..."
        cd ${d};
        for i in *.bats; do
            echo "running $(basename ${i}) tests ..."
            bats ${i}
        done
    )
done

Loading a Library to be Tested into a .bats Test

Assuming that the library you want to test is called 'blue.shlib', create a 'blue-library.bash' file in the directory that contain the .bats tests. If the library is located at a fixed relative location to the .bats test files, use this arrangement:

source ${BATS_TEST_DIRNAME}/[...]/blue.shlib

Otherwise, you can use an absolute path, but that is more fragile.

From each .bats test that wants to test functionality from that library:

load blue-library

@test "something" {
...
}

For more details, see:

Libraries being Tested and Helpers

Loading an Executable Script as Library to be Tested into a .bats Test

It is possible to test functions from an executable script by loading the executable script as library, even if the script loads external libraries itself, and invokes a main function. We use the same idea described above in the Loading a Library to be Tested into a .bats Test section, with a few additions. Let's assume there is a script called blue that is executable (contains a main() invocation), sources the bash.shlib library, and contains functions that we would like to unit test:

#!/usr/bin/env bash

#
# this is blue
#
[ -f $(dirname $0)/lib/bash.shlib ] && source $(dirname $0)/lib/bash.shlib || { echo "[error]: $(dirname $0)/lib/bash.shlib not found" 1>&2; exit 1; }

function main() {
...
}

function auxiliary() {
...
}

main "$@"

We would like to test auxiliary() by loading blue as a library.

For that, the following changes need to be introduced in blue:

1. Guard library loading with a BATS_TESTING variable:

...
[ -z "${BATS_TESTING}" ]  && { [ -f $(dirname $0)/lib/bash.shlib ] && source $(dirname $0)/lib/bash.shlib || { echo "[error]: $(dirname $0)/lib/bash.shlib not found" 1>&2; exit 1; } }
...

2. Guard main() invocation with the same BATS_TESTING variable:

...
if [ -z "${BATS_TESTING}" ]; then
    main "$@"
fi
...

Interestingly enough, this does not work, it breaks BATS:

...
[ -z "${BATS_TESTING}" ] && main "$@"
...

3. Create the all-required-libraries.bash library loader in the directory that contains the .bats tests. The library loader is similar to the one described in the section Loading a Library to be Tested into a .bats Test above. The difference is that it must set the BATS_TESTING to true, load the executable script as library, and also load all libraries required by the executable script:

export BATS_TESTING=true
source ${BATS_TEST_DIRNAME}/../../lib/bash.shlib
source ${BATS_TEST_DIRNAME}/../../blue

4. For each .bats test that wants to test functionality from the executable script, load "all-required-libraries", similarly to how is described in Loading a Library to be Tested into a .bats Test:

load all-required-libraries

@test "something" {
...
}

Handling stdout and stderr

function teardown() {

    rm -f ${BATS_TEST_DIRNAME}/tmp/stdout
    rm -f ${BATS_TEST_DIRNAME}/tmp/stderr
}

@test "stderr" {

    run $(fail "blah" 1>${BATS_TEST_DIRNAME}/tmp/stdout 2>${BATS_TEST_DIRNAME}/tmp/stderr )

    [[ -z $(cat ${BATS_TEST_DIRNAME}/tmp/stdout) ]]
    [[ $(cat ${BATS_TEST_DIRNAME}/tmp/stderr) = "[failure]: blah" ]]
}

This is a proposed solution, I need to do more research:

@test "test all outputl" {
  local stdoutPath="${BATS_TMPDIR}/${BATS_TEST_NAME}.stdout"
  local stderrPath="${BATS_TMPDIR}/${BATS_TEST_NAME}.stderr"

  myCommandOrFunction 1>${stdoutPath} 2>${stderrPath}

  grep "What Im looking for in stdout"  ${stdoutPath}
  ! grep "something it cannot appear in stdout"  ${stdoutPath}

  grep "What I'm looking for in stderr" ${stderrPath}
  ! grep "Something it cannot appear in stderr" ${stderrPath}
}

Handling data

A useful pattern is to create a "data" subdirectory in the directory that holds the .bats tests, and then access it from the tests with ${BATS_TEST_DIRNAME}/data/...

"Override" Functions

A useful technique when testing a layer is to "override" (declare) the functions called from that layer right before the test, so we can control input and output in the layer.

function setup() {

    function some-function() {
        echo "some-function($@)"
    }
}

...

@test "test invocation through layer to some-function()" {

    run caller-function A B C

    [[ ${status} -eq 0 ]]
    [[ "${output}" =~ some-function\(.*\) ]]

    args=${output#some-function(}
    args=${args%)}

    arg0=${args%% *}
    [[ ${arg0} = "A" ]]

    args=${args#${arg0} }
    arg1=${args%% *}
    [[ ${arg1} = "B" ]]

    ...
}

The function do not need to be "overridden" in setup(), they can be overridden in the test itself:

@test "test invocation through layer to some-function()" {

    function some-function-that-is-invoked-inside-caller-function() {
       exit 1
    }

    run caller-function A B C

    [[ ${status} -eq 1 ]]

    # ...
}

Debugging Failures

BATS does not handle error logging gracefully. A useful technique is to turn verbosity in bash.shlib debug()/error()/warn()/info() functions and to configure them to send output into an alternate file, for problematic tests:

VERBOSE=true; DEBUG_OUTPUT=~/bats.out

Run the tests as follows:

rm ~/tmp/bats.out ; clear; bats ...test/some-test.bats; cat ~/tmp/bats.out