Jq Usage

External

• https://gist.github.com/olih/f7437fb6962fb3ee9fe95bda8d2c8fa4
• https://github.com/stedolan/jq/wiki/Cookbook

Internal

• jq
• jq Concepts

Command Line Options

-r

Output raw strings, not JSON content. This is very useful when jq is used as part of shell pipelines and the results feed other shell commands.

Universal Filters

select() Filter

select(boolean-expression)

The select() filter lets its input pass through unchanged if the boolean expression evaluates to true, and produces no output (swallows the input) otherwise. The result consists of the same kind of pipeline items that are the input of select().

For more details about expressions, see:

jq Concepts - Expressions

Example:

jq '... | select(.Name == "something")'

Spaces are optional:

jq '... | select(.Name=="something")'

.Name extracts the value of a map key value pair whose key is "Name" and the filter lets pass only those maps for which the "Name" key is equals to "something". Note that "==" should be used, it is semantically different from "=".

Also see Double Quotes in select() Expressions scripting tips.

select() Expressions

Equality

select(.name == "something")

String Matching

select(.name | contains("something"))

select(.name | startswith("somet"))

select(.name | endswith("ething"))

Functions

https://stedolan.github.io/jq/manual/#Builtinoperatorsandfunctions

Map Filters

Object Identifier-Index Filter .key

.key

The filter expects a map and produces the value associated with the key given as argument of the filter, or null if there is no such object. The value, if exists, can be a primitive, a map or an array.

The key is case sensitive.

cat example.json | jq '.color'

This syntax only works for "identifier-like" keys: keys that are all made of alphanumeric characters and underscore, and which do not start with a digit.

The .key syntax is actually an alias for the more generic syntax:

.["key"]

If the key contains special characters, the .key alias cannot be used, and the complete .["key"] syntax should be used:

jq '.["complex::key"]'

For:

{
  "complex::color": "something"
}

jq '.complex::color'

will fail:

jq: error: syntax error, unexpected ':', expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.complex::color

while

jq '.["complex::color"]' 

will work.

This syntax is particularly useful if the key contains dashes:

.["my-color"]

Display Two or More Map Keys

jq '... | .name, .color'

Concatenate Two or More Map Keys

With a dot:

jq '... | (.name + "." + .color)'

With a space:

jq '... | (.name + " " + .color)'

Updating a File

test.json has to exist and be a valid JSON file.

jq '.something="something else"' test.json

Note that the file is not updated "in-place", jq sends the updated result to stdout.

The |= operator is used to update a value. What is the difference?

Array Filters

Array Index Filter .[index]

.[zero-based-index]

selects the corresponding array element, or null if the index does not designate an element that exists.

For:

[
  "blue",
  "red",
  "green"
]

jq '.[0]'

return "blue"

while

jq '.[5]'

returns null.

If the input is not an array, the filter fails:

echo \"blah\" | jq '.[]'
jq: error (at <stdin>:1): Cannot iterate over string ("blah")

Array Slices Filters

Index Range Filter .[index:index]

jq '.[4:6]'

First Elements

First 4 elements:

jq '.[:4]'

Last Elements

Last element:

jq '.[-1]'

Second element from the end:

jq '.[-2]'

Last 2 elements:

jq '.[-2:]'

Before Last Element

jq '.[-2]'

Array-to-Iterator Filter []

The filter expects and array and produces the array's elements as an iteration.

echo "[\"red\", \"blue\"]" | jq '.[]'

has the same result as:

echo "\"red\"" "\"blue\"" | jq '.'

Interestingly enough, and somewhat counterintuitively, the filter also works on maps. The effect is equivalent with Java's Map#values().

echo "{\"a\": 1, \"b\": 2}" | jq '.[]'

produces:

1 
2

The filter does not work on primitives:

echo \"blah\" | jq '.[]'
jq: error (at <stdin>:1): Cannot iterate over string ("blah")

Array-to-Iterator Filter applied on a Object value

If an Object value is an Array, then the elements of that Array can be turned into a stream with the Array-to-iterator filter. The key the Array is associated with should be specified:

For:

{
  "items": [
    {
      "color": "red"
    },
    {
      "color": "blue"
    }
  ]
}

the following expression:

jq ".items[]"

produces the following stream:

{
  "color": "red"
}
{
   "color": "blue"
}

Stream to Array Filter

A stream can collected into a single array by wrapping the underlying filter in square brackets.

... | jq '[ .[] | select(.color=="red") ]'

A simple no-op transformation from array to stream and back to array is:

... | jq '[.[]]'

Array Length

cat ... | jq -r '.items | length'

Use Cases

Select the Array Element which is a Map and Contains a Specific Key

... | jq -r '.clusters[] | select(.name | contains("docker")) | .cluster.server'

Selects the array element whose "name" key contains "docker" and displays the .cluster.server path of the selected map.

jq Scripting Tips

Double Quotes in select() Expressions

Use a shell variable to hold the value we want to select against. Node that the value may contain spaces.

local select_value="some value that may or may not contain spaces"
local jq_result
jq_result=$(${command_that_generates_json} | jq 'select(.keyA=="'"${select_value}"'"')

Note that ${select_value} is enclosed in double quotes, and the resulted value is "introduced" in jq enclosed in single quotes.

Generating JSON with jq

https://spin.atomicobject.com/2021/06/08/jq-creating-updating-json/

jq --null-input --arg arg1 "A" --arg arg2 "B" '{"a": $arg1, "b": $arg2}'

produces:

{
  "a": "A",
  "b": "B"
}

How to use jq to filter a larger JSON and only select a few fields to be emitted as a smaller JSON, in a pipeline?