Ansible Concepts
External
- https://docs.ansible.com/ansible/latest/reference_appendices/playbooks_keywords.html
- http://docs.ansible.com/ansible/glossary.html
Internal
Overview
Ansible is a configuration management and provisioning Infrastructure as Code tool, similar to Chef, Puppet or Salt. What sets it aside is that it does not require agents on the hosts it configures - it only requires ssh access and sudo. The target hosts are specified in an inventory file, located on the Ansible host, the machine that is use to conduct operations from. Ansible executes on the Ansible host. During the execution, Ansible connects via SSH to the target hosts, gets the context and executes tasks on the target hosts, while being cognizant of the context. Being context-aware allows tasks to be idempotent. In the beginning, the inventory can be probed for availability for running a ping module:
ansible -i ./hosts.yaml <group-name|all> -m ping
The idempotence of tasks comes from the idempotence of their components, the modules. Modules are aware of the facts of the context.
Multiple tasks are grouped together in playbooks. A playbook is executed with the ansible-playbook command.
Ansible is one of the tools that can be used to manage generic Infrastructure as Code stacks.
Playground Example
Hosts
Ansible Host
The host that executes the ansible or ansible-playbook command and that connects via SSH to other hosts to be managed with Ansible.
Target Host
The target hosts are specified in the inventory file. If there is no inventory file, localhost is implied. This document refers to target hosts as "systems".
Delegate Host
localhost
The local host can be explicitly referred to as localhost.
Ansible Files
Ansible works with the following files, which are in YAML format:
- Inventory file
- Playbook file
- To be continued.
"#" can be used to comment in files.
Inventory File
Ansible works against multiple target hosts at the same time. It does this by selecting all or some of the target hosts listed in Ansible’s inventory file. The default location of the inventory file is /etc/ansible/host, but it can be placed anywhere and referred from the command line with -i <path-to-inventory-file>.
The conventional name is hosts.yaml.
Inventory File Structure
host1.example.com
[webservers]
web1.example.com
web2.example.com
[dbservers]
db1.example.com
db2.example.com
The inventory file can also be used to define inventory file variables, as group variables or host variables.
Groups
The headings in brackets are group names, which are arbitrary strings used in classifying systems and deciding what systems you are controlling at what times and for what purpose. A host can be part of more than one group.
Also see group variables below.
Default Groups
There are two default groups: "all" and "ungrouped". "all" contains every host. "ungrouped" contains all hosts that don’t have another group aside from "all".
Recursive Groups
Recursive groups are declared with the [<group-name>:children]:
[A]
host1
host2
[B]
host3
host4
[AandB:children]
A
B
[AandB:vars]
something=something-else
Dynamic Inventory
Playbooks and Plays
Playbooks are YAML files that contain one or more plays in an ordered list. Each play executes part of the overall goal of the playbook. Each play includes at minimum two things:
- the managed hosts to target, declared with the keyword
hosts:.
Optionally, a playbook may include:
- One or more tasks to be executed against the target hosts, listed under the
tasks:keyword. A noop playbook may include no task and it will still execute. The reason for a playbook to exists is to declare tasks, which are executed in sequence at runtime, as part of the playbook. - Variables, under the
vars:subtree. - Roles, under the
roles:subtree. All roles specified here are are assigned to the configured hosts. If a list of roles is provided in a play, it is said that the roles are used "at the play" level, meaning that their tasks are added to the play. The name of roles specified underroles:keyword represent names of directories that contain the corresponding role's components. If a role name is specified in the play, the corresponding role directory, whose name is identical with the role name, is expected to be present in the following locations:- A
rolessubdirectory of the directory that contains the playbook. - In the directory that contains the playbook.
$HOME/.ansible/roles/usr/share/ansible/roles/etc/ansible/roles
- A
- 'become' keywords.
remote_userspre_tasks- Other playbooks, imported with
import_playbook.
Playbook File
---
- name: play one
hosts: all # For the local host, use 'localhost'
become: yes
become_user: root
vars:
some_var: something
pre_tasks:
- import_tasks: tasks/some_tasks.yml
roles:
- kubernetes-local-storage
tasks:
- import_tasks: tasks/some_other_tasks.yml
- name: Install Nginx
apt:
name: nginx
state: installed
update_cache: true
post_tasks:
- ...
- name: play two
hosts: ...
tasks:
- ...
pre_tasks
The pre_tasks keyword allows listing tasks to be executed before the play's roles are called and the play's tasks listed under tasks keyword are executed. It could be used to load variables declared in files, for example.
import_playbook
A playbook can include other playbooks:
---
- import_playbook: some-other-playbook.yml
ansible-playbook
A playbook is executed with the ansible-playbook command. For more details see:
Task
A task gives a name to an action, which consists of a module and its arguments.
The task may optionally specify an execution condition.
The task may also optionally specify that the action should be executed in a loop, if looping directives are present.
The task may register a variable which, upon execution, contains the task status and module execution result data.
- name: some task
<module-name>: #
<module-arg-1>: value 1 #
<module-arg-2>: value 2 # this is the action
<module-arg-3>: value 3 #
... #
when: <conditional>
register: <variable-name>
<loop-construct>
Example:
- name: Create some directories
file:
path: "{{ item }}"
state: directory
owner: ec2-user
when: inventory_hostname in groups['blue']
become: true
register: some_var
with_items:
- "/tmp/a"
- "/tmp/b"
- "{{ some_dir_variable }}"
Handlers are also tasks, but they do not run unless they are notified by name when a task reports an underlying change on a target host.
Playbooks exist to declare tasks, which are then executed at runtime as part of the playbook.
Task Idempotence
Ansible tasks are idempotent, they can be safely run repeatedly. Idempotence is achieved by the fact that ansible first gathers the context, which consists of facts, before running the tasks. Ansible "checks the facts" first to decide whether it needs to change anything. If the outcome it seeks is already achieved, the task is not executed.
Importing Tasks
Wherever we use tasks, we can import tasks from other files. import_tasks module can be used in the task list of a play, or in a task file, or wherever a tasks is declared. The task file that is imported may contain an embedded import_tasks, so tasks can be imported recursively.
---
- import_tasks: some-file-that-contains-tasks.yaml
- import_tasks: some-dir/some-other-file-that-contains-tasks.yaml
Task Configuration
name
An optional task identifier that can be used for documentation. Gives a user-friendly name for the action executed as part of this task. If the string does not contain variable references, it can be left unquoted:
- name: Install the Flux Capacitor
...
The string must be quoted if it includes variable references; the references will be resolved:
- name: "Install {{ flux_capacitor_name }} {{ flux_capacitor_version }}"
...
If the name value is not specified, it will default to the module name. However, it is good practice to provide good descriptive names, it has been proven that this helps with understanding of configuration logic.
state
register
Specifies the name of the variable that will reference the module return value. See:
vars
A map of variables.
args
A secondary way to add arguments into a task. Takes a dictionary in which the keys are the underlying module's valid argument names, while the values are the corresponding argument values. For example, if the module car has color as an argument, it can be configured this way:
- name: an example
car:
args:
color: blue
This is equivalent with:
- name: an example
car:
color: blue
If both args and the module's argument are specified, the module's argument takes precedence. In this example:
- name: an example
car:
color: red
args:
color: blue
the "car" is configured to "red".
changed_when
Loops
Looping directives: with_items, loop, loop_control, with_<lookup_plugin>:
when - Conditional Task Execution
become Keywords
become
Boolean that controls if privilege escalation is used or not on task execution. Implemented by the become plugin.
- name: Some task
become: true
[...]
become_exe
The path of the executable used to elevate privileges. Implemented by the become plugin.
become_flags
A string of flag(s) to pass to the privilege escalation program when become is true.
become_method
Which method of privilege escalation to use (such as sudo or su).
- name: Some task
become_method: sudo
[...]
become_user
User that you ‘become’ after using privilege escalation. The remote/login user must have permissions to become this user.
notify
Action
An action is a module and its arguments. The action is part of a task. A task can have one and only one action.
Module
Modules are the "verbs" in Ansible, they do things, like install software, copy files, use templates and so on. Modules can use available context (facts) in order to determine what actions, if any, need to be done to accomplish a task. Thus, they insure idempotence of the tasks that execute them.
A module can be executed individually, on command line, in the same way as it would be executed as part of a playbook.
ansible -m <module-name> -a "arg1 arg2 ..."
Example:
ansible -m shell -a "echo {{ a_global_variable }} > /tmp/test.txt"
A module is declared as follows:
- name: This is a descriptive name of what the module execution will achieve
<module-name>: ...
...
Example:
- name: This is a descriptive name of what the module execution will achieve
shell: echo {{ openshift_dns_ip }} > /etc/origin/node/openshift-dns-ip
args:
executable: /bin/bash
For more details on module and task syntax, see Task above.
Modules by Category
| Category | Modules |
|---|---|
| Setting facts | |
| Troubleshooting and diagnostics | |
| Voluntary failure | |
| Dynamic loading of various Ansible elements | |
| Arbitrary command execution | |
| Installers | |
| File manipulation | |
| Text File Editing | |
| Archive handling | |
| Download from an URL | |
| XML handling | |
| Low-level filesystem manipulation |
|
| Miscellanea |
Module Return Value
Block
Tasks can be grouped together in logical groups with the block keyword. Blocks also offer ways to handle task errors, similar to exception handling in many programming languages.
tasks:
- name: A block containing a logically-related group of tasks
block:
- name: Task 1
<module-name>: ...
- name: Task 2
<module-name>: ...
...
when: ...
ignore_errors: yes
Blocks can be executed conditionally the same way a task is, and accept other task configuration elements. From this perspective, a block can be thought of as a task that is made of sub-tasks.
It is apparently fine to skip providing names to tasks inside a block. The following syntax does not raise any warning:
- name: Some block
block:
- name:
debug:
msg: something
- name:
debug:
msg: something else
Handler
A handler is similar to a regular tasks, in that it can do anything a task can, but it will only run when called by another tasks: handles are only run if the task contains a notify directive and also indicates that it changed something. They can be though as part of an event-based system: a handler will take an action when called by an event it listens for. This is useful for "secondary" actions that might be required after running a task, such as starting a new service after installation or reloading a service after a configuration change. For example, if a config file is changed, then the task referencing the config file templating operation may notify a service restart handler. This means services can be bounced only if they need to be restarted. Handlers can be used for things other than service restarts, but service restarts are the most common usage.
- hosts: all
become: yes
become_user: root
tasks:
- name: Install Nginx
apt:
name: nginx
state: installed
update_cache: true
notify:
- Start Nginx
handlers:
- name: Start Nginx
service:
name: nginx
state: started
Role
Roles are units of organization that are good for organizing multiple, related tasks and encapsulating data needed to accomplish those tasks. Assigning a role to a group of hosts implies that they should implement a specific behavior. A role may include applying certain variable values, certain tasks, and certain handlers. Roles are redistributable units that allow you to share behavior among playbooks. A playbook includes its roles typically in a 'roles' directory. Conventionally, the elements associated with a role are stored in a directory named after the role:
roles ├─ role1 │ ├─ defaults │ │ └─ main.yaml │ ├─ files │ │ └─ main.yaml │ ├─ handlers │ │ └─ main.yaml │ ├─ meta │ │ └─ main.yaml │ ├─ tasks │ │ └─ main.yaml │ ├─ templates │ │ └─ main.yaml │ └─ vars │ └─ main.yaml │ ├─ role2 ├─ role3 ...
Within each directory, Ansible will search for and automatically read any Yaml file called main.yml.
Roles can be created with the following command:
cd .../roles ansible-galaxy init <role-name>
Role Structure
Role Defaults
The default directory contains files, usually main.yaml, which can be used to provide defaults for variables. For more details see "Undefined Variables".
Role Files
The files directory contains files that we want copied into target hosts via the copy module. There is no main.yaml file here.
Role Handlers
Example of handlers/main.yaml:
---
- name: Start Nginx
service:
name: nginx
state: started
- name: Reload Nginx
service:
name: nginx
state: reloaded
Role Meta
The main.yml file within the meta directory contains role metadata, including dependencies on another role:
---
dependencies:
- { role: ssl }
Role Tasks
tasks/main.yml contains the role's tasks.
---
#
# tasks file for kubernetes-local-storage
#
- name: format ESB block storage device
filesystem:
fstype: xfs
dev: /dev/xvdb
force: no
- name: create local storage mount point
file:
path: "{{ item }}"
state: directory
owner: root
with_items:
- "/mnt/ebs0"
when:
- inventory_hostname in groups['kube-node']
- name: mount storage and update /etc/fstab
mount:
state: mounted
path: /mnt/ebs0
src: /dev/xvdb
fstype: xfs
- name: create volume directories
file:
path: "/mnt/ebs0/{{ item }}"
state: directory
owner: root
with_items:
- "local-pv0"
- "local-pv1"
- "local-pv2"
Role Templates
Template files can contain template variables, based on Python's Jinja2 template engine. Files in here should end in .j2, but can otherwise have any name. Similar to files, we won't find a main.yml file within the templates directory.
Role Vars
The vars directory contains a main.yaml file that lists variables to use. This provides a convenient place for us to change configuration-wide settings.
---
domain: example.com
ssl_key: /etc/ssl/sfh/sfh.key
ssl_crt: /etc/ssl/sfh/sfh.crt
Using Roles
Using Roles at Play Level
If a list of roles is provided in a play, it is said that the roles are used "at the play" level. This is the classic way to use roles. If a role is specified in the play's roles: list, then the tasks from roles/<role-name>/tasks/main.yaml are added to the play. Note that usually main.yaml imports other tasks, coming from other files, for modularization.
- hosts: servers
roles:
- common
- webservers
- hosts: servers
roles:
- { role: username.rolename, x: 42 }
In what order are the tasks executed? Is the listing order honored?
Variables
Places where to Define Variables
Variables can be defined in:
- Inventory.
- Playbooks, as playbook variables, declared under the
vars:subtree in the playbook. - Reusable variable files that can be loaded explicitly with the
include_varsmodule. A typical way to load variable this way is to invoke theinclude_varsmodule in the playbook'spre_tasks:section. - Roles (as role variables).
- On the the command line.
Variable Scopes
A playbook variable declared in the vars: section of a playbook is available to all tasks executing as part of that playbook.
Undefined Variables
If a variable is not defined, and it is referenced, by default Ansible raises an "undefined variable" error and fails. When appropriate, this behavior can be avoided by providing a default value for the missing variable with the default() filter. If working with role, the same effect can be achieved by declaring the default variable value in the role's defaults/main.yaml. Beginning in version 2.8, attempting to access an attribute of an undefined value in Jinja will return another undefined value, rather than throwing an error immediately. This means that you can now simply use a default() filter with a value in a nested data structure ({{ a.b.c | default('DEFAULT') }}) when you do not know if the intermediate values are defined.
In case there are a lot of undefined variables, Ansible can be configured to ignore them and only some specific variables can be made mandatory with the mandatory() filter.
Configuring Ansible to Ignore Undefined Variables
See DEFAULT_UNDEFINED_VAR_BEHAVIOR below.
Making a Variable Optional
By default Ansible requires values for all variables in a templated expression. However, specific variables can be made optional by setting its default value to the special variable omit, using the default() filter:
{{ item.mode | default(omit) }}
If you are chaining additional filters after the default(omit) filter, you should instead do something like this:
{{ foo | default(None) | some_filter or omit }}
In this example, the default None (Python null) value will cause the later filters to fail, which will trigger the or omit portion of the logic. Using omit in this manner is very specific to the later filters you are chaining though, so be prepared for some trial and error if you do this.
Making a Variable Mandatory
If Ansible was configured to ignore undefined variables, specific variables can be made mandatory and cause Ansible to fail if they are not declared with the mandatory filter:
{{ some_variable | mandatory }}
The variable value will be used as is, but the template evaluation will raise an error if it is undefined.
Expressions involving Undefined Variables
Variable Precedence
TODO.
Variable Types
Simple Variables
Simple variable can be declared as such:
amazon_corretto_package_name: amazon-corretto-11-x64-macos-jdk.pkg
jdk:
version: 11
parameterized_amazon_corretto_package_name: amazon-corretto-{{ jdk.version }}-x64-macos-jdk.pkg
my_shell: "{{ lookup('env','SHELL') }}"
and can be used by enclosing them in double curly braces {{ ... }}:
- name: Download the latest Amazon Corretto {{ jdk.version }}
get_url:
url: https://corretto.aws/downloads/latest/{{ amazon_corretto_package_name }}
dest: /tmp/{{ amazon_corretto_package_name }}
Variables Loaded with include_vars
Registered Variables
Variables can be created from the output of Ansible tasks. The entire output of a task can be referred to from a variable. The association of the output with a variable is done with the register: keyword, followed by the name of the variable. Note that the variables registered as such point to complex maps, and can be used by any task that executes later in the play. Registered variables may be simple variables, list variables, dictionary variables, or complex nested data structures. Registered variables are stored in memory. Registered variables cannot be cached for use in future plays. Registered variables are only valid on the host on which the playbook runs, for the rest of the current playbook run and this is why they're called host-level variables.
Examples:
Host File Variables
Inventory File Variables
These variables are declared in the inventory file.
Group Variables
Group variables are declared under [<group-name>:vars] tag in the inventory file. If they are declared this way, the variables apply to an entire group at once.
[group-A]
host1
host2
[group-A:vars]
something=something-else
Host Variables
Variables that apply to a specific host are declared in the inventory file after the host name:
[group1]
host1 http_port=80 maxRequestsPerChild=808
host2 http_port=303 maxRequestsPerChild=909
Defining Variables on the Command Line
ansible-playbook --extra-vars "{\"some_var\": \"some value\", \"some_other_var\": \"${SOME_OTHER_ENV_VAR}\"}" playbook-file.yaml
Data Types
The data types in Ansible are borrowed from Python. See:
Data types can be managed in Ansible with the following data type management filters:
Fact
Before running any tasks, Ansible will gather information about the system it is provisioning. These are called facts. Facts are pieces of information about target hosts: system and environment information. Facts are used in playbooks, tasks and templates just like variables, but they are inferred, rather than set, during automatic discovery when running plays, by executing the internal setup module on the remote host. Ansible facts all start with anisble_ and are globally available for use any place variables can be used: variable files, tasks, and templates.
Fact-Gathering
Ansible must be configured to gather facts automatically. This is controlled by gather_facts, which is set to "true" by default.explicitly.
Explicitly Setting Facts
Facts can also be explicitly with the set_fact module:
Filters
Template
A template is a file to be installed on the target host after the declared variables are substituted with actual values. Variable values may come from the inventory file, host variables, group variables, or facts. Templates use the Jinja2 template engine and can also include logical constructs like loops and if statements.
In case of a role declaration, template files are stored into a fixed location, the role's templates directory.
Jinja2 Templating
Vault
Privilege Escalation
Keywords: become, sudo, su, root:
Plugins
Define the relationship between plugins and modules.
Become Plugins
Ansible Configuration Settings
The ansible.cfg Configuration File
Configuration Options
DEFAULT_UNDEFINED_VAR_BEHAVIOR
When True, this causes ansible templating to fail steps that reference variable that do not exist. Variables do not exist most likely because they have been mistyped in code. Changing DEFAULT_UNDEFINED_VAR_BEHAVIOR to False will cause any {{ template_expression }} that contains undefined variables will be rendered in a template or ansible action line exactly as written.
Environment Variables
External environment variables can be obtained in Ansible in the following ways:
Local Environment Variables
lookup() Plugin
This. prints the local server HOME environment variable.
- name: Get HOME Environment Variable
debug:
msg: "{{ lookup('env','HOME') }}"
The value can be stored in a playbook environment variable:
- hosts: all
vars:
local_shell: "{{ lookup('env','SHELL') }}"
Target Host Environment Variables via Fact-Gathering
The environment variables of the remote servers can be accessed via facts. There is a list called ansible_env which stores all the environment variables. This only works if fact-gathering is "on", which is the default value.
- name: HOME Environment Variable
debug:
msg: "{{ ansible_env.HOME }}"
Version
ansible --version
ansible [core 2.11.0]
config file = None
configured module search path = ['/Users/ovidiu/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/Cellar/ansible/4.0.0/libexec/lib/python3.9/site-packages/ansible
ansible collection location = /Users/ovidiu/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.9.5 (default, May 4 2021, 03:33:11) [Clang 12.0.0 (clang-1200.0.32.29)]
jinja version = 3.0.1
libyaml = True
Version in Brew
brew list ansible
/usr/local/Cellar/ansible/4.0.0/bin/ansible
...
does not aways reflect the value returned by ansible --version.