Yq: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(23 intermediate revisions by the same user not shown)
Line 5: Line 5:
* [[jq]]
* [[jq]]


=DEPLETE=
=Overview=
{{Internal|YqTODELETE| YqTODELETE}}
 
<font color=darkkhaki size=+3>Obsolete</font>


=Overview=
yq is a lightweight and portable command-line YAML processor. It is written in Go and it consists in a single portable binary. It can be used to query a YAML file using a YAML path, to update a YAML file for a YAML path or using a script file, to create a YAML file or merge YAML files. The update operation creates any missing path elements. yq can also be used to convert YAML files to JSON and JSON files to YAML.
yq is a lightweight and portable command-line YAML processor. It is written in Go and it consists in a single portable binary. It can be used to query a YAML file using a YAML path, to update a YAML file for a YAML path or using a script file, to create a YAML file or merge YAML files. The update operation creates any missing path elements. yq can also be used to convert YAML files to JSON and JSON files to YAML.


Line 21: Line 21:
==Mac==
==Mac==
  brew install yq
  brew install yq
===Upgrade===
brew upgrade yq
===Install a Specific Version===
{{Internal|Brew_Operations#Install_a_Specific_Package_.28Formula.29_Version|Install a Specific Package (Formula) Version}}
==Linux==
==Linux==
  yq_version=2.4.1
Go to https://github.com/mikefarah/yq/releases to figure out what the latest release is.
 
  yq_version=3.4.1
  cd /tmp
  cd /tmp
  wget https&#58;//github.com/mikefarah/yq/releases/download/${yq_version}/yq_linux_amd64
  wget https&#58;//github.com/mikefarah/yq/releases/download/${yq_version}/yq_linux_amd64
  mv /tmp/yq_linux_amd64 yq /bin/yq
  mv /tmp/yq_linux_amd64 /bin/yq
  chmod a+x /bin/yq
  chmod a+x /bin/yq
For 4.x, the version has a preceding "v":
yq_version=v4.33.3
cd /tmp
wget https&#58;//github.com/mikefarah/yq/releases/download/${yq_version}/yq_linux_amd64
mv /tmp/yq_linux_amd64 /bin/yq
chmod a+x /bin/yq
==Maintain Co-existing yq 3 and yq 4 Versions==
<syntaxhighlight lang='bash'>
wget -O /usr/local/bin/yq3 https://github.com/mikefarah/yq/releases/download/3.4.1/yq_darwin_amd64
chmod +x /usr/local/bin/yq3
</syntaxhighlight>
In scripts:
<syntaxhighlight lang='bash'>
shopt -s expand_aliases
alias yq=yq3
</syntaxhighlight>


=Concepts=
=Concepts=
Line 36: Line 66:


=Operations=
=Operations=
==Conversion to and from JSON==
Also see {{Internal|Conversion between YAML and JSON|Conversion between YAML and JSON}}
===YAML to JSON===
yq r <file> -j
===JSON to YAML===
yq r <file.json>
==Read==
==Read==
{{External|https://mikefarah.github.io/yq/read/}}
{{External|https://mikefarah.github.io/yq/read/}}
Line 53: Line 90:
     echo "file.yaml invalid"
     echo "file.yaml invalid"
   fi
   fi
===Expressions===
====Arrays====
Select elements from an array.
All elements:
...<''enclosing-element-name''>[*]
First element:
...<''enclosing-element-name''>[0]
Specific keys from all elements:
...<''enclosing-element-name''>[*].<''some-key''>
====Maps====
====Organizatorium====
If an array contains single key maps, selecting one of them with:
yq -r ./file.yaml spec.template.spec.volumes[*].configMap


==Create a New YAML Document==
==Create a New YAML Document==
{{External|https://mikefarah.github.io/yq/create/}}
{{External|https://mikefarah.gitbook.io/yq/commands/create}}


  yq n <path> <value>
  yq n <path> <value>
Line 70: Line 129:
   color: blue
   color: blue
</syntaxhighlight>
</syntaxhighlight>
==Write/Update a YAML Document==
{{External|https://mikefarah.gitbook.io/yq/commands/write-update}}
The command takes the file specified as a base and writes/updates the specified paths. It then sends the entire resulted YAML file at stdout.
If the base is empty, or only contains comments, yq w fails with:
Error: asked to process document index 0 but there are only 0 document(s)
"yq-validity" can be checked with [[#Checking_yq-validity|yq r]] command.
Starting from test.yaml:
<syntaxhighlight lang='yaml'>
item:
  color: blue
</syntaxhighlight>
yq w test.yaml item.color red
produces:
<syntaxhighlight lang='yaml'>
item:
  color: red
</syntaxhighlight>
The command can add a new path, creating all required path elements:
yq w test.yaml item.size large
produces:
<syntaxhighlight lang='yaml'>
item:
  color: blue
  size: large
</syntaxhighlight>
The command may update the file '''in-place''' by specifying the "-i" option:
yq w -i test.yaml ...
'''write Idiosyncrasies''':
* write strips off the comments from the base file
==Merge YAML Documents==
{{External|https://mikefarah.github.io/yq/merge/}}
YAML files can be merged using the 'merge' (m) command. Each additional file merged with the first file will set values for any key '''not existing already''' or where the key has no value.
==Delete Paths==
{{External|https://mikefarah.gitbook.io/yq/commands/delete}}
yq d ./some-file.yaml a.b.c
Deletes the subtree identified by the right-most path element, including the element. The deletion happens even if the subtree is empty.
If the document does not contain an element identified by the right-most path element, the operation is a noop.

Latest revision as of 23:47, 16 May 2023

External

Internal

Overview

Obsolete

yq is a lightweight and portable command-line YAML processor. It is written in Go and it consists in a single portable binary. It can be used to query a YAML file using a YAML path, to update a YAML file for a YAML path or using a script file, to create a YAML file or merge YAML files. The update operation creates any missing path elements. yq can also be used to convert YAML files to JSON and JSON files to YAML.

The default output is YAML, unless -j option is used.

Version

yq --version
yq version 2.4.1

Installation

Mac

brew install yq

Upgrade

brew upgrade yq

Install a Specific Version

Install a Specific Package (Formula) Version

Linux

Go to https://github.com/mikefarah/yq/releases to figure out what the latest release is.

yq_version=3.4.1
cd /tmp
wget https://github.com/mikefarah/yq/releases/download/${yq_version}/yq_linux_amd64
mv /tmp/yq_linux_amd64 /bin/yq
chmod a+x /bin/yq

For 4.x, the version has a preceding "v":

yq_version=v4.33.3
cd /tmp
wget https://github.com/mikefarah/yq/releases/download/${yq_version}/yq_linux_amd64
mv /tmp/yq_linux_amd64 /bin/yq
chmod a+x /bin/yq

Maintain Co-existing yq 3 and yq 4 Versions

wget -O /usr/local/bin/yq3 https://github.com/mikefarah/yq/releases/download/3.4.1/yq_darwin_amd64
chmod +x /usr/local/bin/yq3

In scripts:

shopt -s expand_aliases
alias yq=yq3

Concepts

Command Line Options

-v

Provides additional details on parsing.

-j

Output formatted as JSON.

Operations

Conversion to and from JSON

Also see

Conversion between YAML and JSON

YAML to JSON

yq r <file> -j

JSON to YAML

yq r <file.json>

Read

https://mikefarah.github.io/yq/read/
yq r <file.yaml> <path>
cat <file.yaml> | yq r - <path>

If the path does not exist - either the final element, or intermediary elements - , yq returns literally "null", but the return code is 0.

If the path exists, but the content is empty, yq returns "null".

Checking yq-validity

An empty file or a file that only contains commands is invalid for for yq. This can be checked as follows:

 if ! yq r file.yaml 1>/dev/null 2>/dev/null; then
   echo "file.yaml invalid"
 fi

Expressions

Arrays

Select elements from an array.

All elements:

...<enclosing-element-name>[*]

First element:

...<enclosing-element-name>[0]

Specific keys from all elements:

...<enclosing-element-name>[*].<some-key>

Maps

Organizatorium

If an array contains single key maps, selecting one of them with:

yq -r ./file.yaml spec.template.spec.volumes[*].configMap

Create a New YAML Document

https://mikefarah.gitbook.io/yq/commands/create
yq n <path> <value>

The content is sent to stdout.

Example:

yq n test.color blue > test.yaml

Result:

test:
  color: blue

Write/Update a YAML Document

https://mikefarah.gitbook.io/yq/commands/write-update

The command takes the file specified as a base and writes/updates the specified paths. It then sends the entire resulted YAML file at stdout.

If the base is empty, or only contains comments, yq w fails with:

Error: asked to process document index 0 but there are only 0 document(s)

"yq-validity" can be checked with yq r command.

Starting from test.yaml:

item:
  color: blue
yq w test.yaml item.color red

produces:

item:
  color: red

The command can add a new path, creating all required path elements:

yq w test.yaml item.size large

produces:

item:
  color: blue
  size: large

The command may update the file in-place by specifying the "-i" option:

yq w -i test.yaml ...

write Idiosyncrasies:

  • write strips off the comments from the base file

Merge YAML Documents

https://mikefarah.github.io/yq/merge/

YAML files can be merged using the 'merge' (m) command. Each additional file merged with the first file will set values for any key not existing already or where the key has no value.

Delete Paths

https://mikefarah.gitbook.io/yq/commands/delete
yq d ./some-file.yaml a.b.c

Deletes the subtree identified by the right-most path element, including the element. The deletion happens even if the subtree is empty.

If the document does not contain an element identified by the right-most path element, the operation is a noop.