Bash Concepts: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(95 intermediate revisions by the same user not shown)
Line 5: Line 5:
=Metacharacters=
=Metacharacters=


''Metacharacters'' are characters that have a special meaning to bash and are interpreted and possibly modified before the modified strings are sent to the command as arguments. Also see [[Regular Expressions Concepts#Metacharacters|regular expressions metacharacters]].
''Metacharacters'' are characters that, when unquoted, separate words. These characters have a special meaning to bash and are interpreted and possibly modified before the modified strings are sent to the command as arguments. Also see [[Regular Expressions Concepts#Metacharacters|regular expressions metacharacters]]. For more details on how bash processes a command line, see "[[Bash_Command_Line_Expansion|bash Command Line Expansion]]"
 
Also see {{External|http://tldp.org/LDP/abs/html/special-chars.html}}


==Space==
==Space==
Line 13: Line 15:
==Single Quotes==
==Single Quotes==


Enclosing a string in single quotes tells the shell NOT to interpret other shell metacharacters in the text between quotes.
Enclosing a string in single quotes tells the shell NOT to interpret any shell metacharacters in the text between quotes: single quotes suppress ''all'' [[Bash_Command_Line_Expansion#Command_Line_Expansion|types of expansion]].


==Double Quotes==
==Double Quotes==


===Difference Between Single Quotes and Double Quotes===
Double quotes permit [[Bash_Command_Line_Expansion#Parameter_and_Variable_Expansion|parameter and variable expansion]] but suppress [[Bash_Command_Line_Expansion#Command_Line_Expansion|other types of expansion]].


==Dollar Sign ($)==
==Dollar Sign ($)==
===<tt>$?</tt>===
===<span id='$@'></span><tt>"$@"</tt>===
<code>"$@"</code> means "all command line arguments received by the program at command line, where all arguments that contain spaces are preserved as such". The <code>"$@"</code> syntax (enclosed in double quotes) instructs the parser to preserve the spaces in all arguments that contain inner spaces.  For example, the following script <code>example</code>:
<syntaxhighlight lang='bash'>
#!/usr/bin/env bash
for i in "$@"; do
  echo "${i}"
done
</syntaxhighlight>
will display
<syntaxhighlight lang='text'>
a
b
c
</syntaxhighlight>
when invoked as <code>example a b c</code>, and:
<syntaxhighlight lang='text'>
a b
c
</syntaxhighlight>
when invoked as <code>example "a b" c</code>.
However, <code>example2</code>:
<syntaxhighlight lang='bash'>
#!/usr/bin/env bash
for i in $@; do
  echo "${i}"
done
</syntaxhighlight>
will display:
<syntaxhighlight lang='text'>
a
b
c
</syntaxhighlight>
in both cases: <code>example2 a b c</code> and <code>example2 "a b" c</code>.
Note that $@ is singled out as static syntax check error by [[Bash_*_and_@_Relative_to_Array_Elements#@|some checkers]].
Also see: {{Internal|Bash_*_and_@_Relative_to_Array_Elements|<tt>*</tt> and <tt>@</tt> Relative to Array Elements}}
===<tt>$*</tt>===
<code>"$*"</code> will hand over exactly one argument, containing all  original arguments, separated by single spaces.
<code>$*</code> will join all arguments by single spaces as well and will then split the string as the shell does on the command line, thus it will split an argument containing spaces into several arguments.
Also see: {{Internal|Bash_*_and_@_Relative_to_Array_Elements|<tt>*</tt> and <tt>@</tt> Relative to Array Elements}}


==Exclamation Mark (!)==
==Exclamation Mark (!)==


==Globbing==
==|==
 
'|' is the "pipe" character, and it is interpreted by bash as a request to set up a [[#Pipeline|pipeline]].
 
==&==
 
==;==
 
==(...)==
{{Internal|bash $() and ()|$(...) and (...)}}
 
=={...}==
 
Executed the enclosed commands in the same shell. There must be a space between '{' and the command and a space between the command and '}'. Also, the last command must be terminated with ';':
 
<syntaxhighlight lang='bash'>
{ set a=1; echo ${a}; }
</syntaxhighlight>
 
==<==


Bash cannot recognize regular expressions but it can do ''filename expansion''. The process is also known as ''globbing''. The process recognizes and expands ''wild cards''. Globbing interprets the following characters (for more details on metacharacters, see [[Regular Expressions Concepts#Metacharacters|Regular Expressions Concepts - Metacharacters]]:
==>==
{{Internal|Bash_Input/Output#Redirection_Operator_.28.3E.29|Redirection Operator (>)}}


====The Wild Card (*)====
==Space==


The wild card (<tt>*</tt>) matches every name in a given directory. Strings containing * will not match filenames that start with a dot.
==Tab==


====The Question Mark (?)====
==<span id='Globbing'></span>Globbing (*, ?)==


The question mark (<tt>?</tt>) serves as a single-character "wild card" for filename expansion.
{{Internal|Bash_Command_Line_Expansion#Pathname_Expansion|Pathname Expansion}}


====The Character Class [...]====


The character class matches just one of the characters listed among square brackets. One of those characters must exist in the file name, on the specified position, for the name to match.
==<tt><<< </tt>==
This is a "here-string". In passes the string specified at the right of the operator to the standard input of the command specified on the right.


====Negation (^)====
<syntaxhighlight lang='bash'>
cat <<< "something"
</syntaxhighlight>


<tt>^</tt> negates the sense of match for the above.
=Control Operators=
 
A control operator is a sequence of characters that performs a control function. It is one of the following characters or sequences:
 
==|==
 
See [[#Pipeline|Pipelines]].
 
==|&==
 
See [[#Pipeline|Pipelines]].
 
==||==
 
==&==
 
==&&==
 
==;==
 
==;;==
 
==(...)==
{{Internal|bash $() and ()|$(...) and (...)}}
 
==The New Line==
 
=Other Operators=
==&#61;~==
{{Internal|bash &#61;~|&#61;~}}
==+&#61; and -&#61;==
{{Internal|bash +&#61; and -&#61;|+&#61; and -&#61;}}


=Login Shell=
=Login Shell=
Line 81: Line 183:
If any of the files exist but cannot be read, bash reports an error.
If any of the files exist but cannot be read, bash reports an error.


=Variables=
=<span id='bash_Built-In_Variables'></span><span id='Variables'></span>Parameters and Variables=


{{Internal|bash Variables|bash Variables}}
{{Internal|bash Parameters and Variables|bash Parameters and Variables}}


=bash Environment Variables=
=<span id='Command_Substitution'></span>Command Substitution $(...)=


<blockquote style="background-color: #f9f9f9; border: solid thin lightgrey;">
{{Internal|bash Command Substitution|Command Substitution}}
:[[bash Environment Variables]]
</blockquote>


=Command Substitution=
=<span id='Command_Line_Evaluation_Process'></span>Command Line Expansion=


{{Internal|bash Command Substitution|Command Substitution}}
{{Internal|Bash_Command_Line_Expansion|Command Line Expansion}}
=Functions=
 
{{Internal|bash Functions|Functions}}


=Process=
=<span id='Process'></span>Processes=


==Foreground Process==
==Foreground Process==


Unless otherwise specified, a process launched by the shell will run in foreground, blocking use of the shell for the duration of the process, because the shells stdin, stdout and stderr are redirected to the process. A foreground process can be typically started by sending SIGINT signal into it, and the shell sends SIGINT when Ctrl-C is pressed.
Unless otherwise specified, a process launched by the shell will run in foreground, blocking use of the shell for the duration of the process, because the shells stdin, stdout and stderr are redirected to the process.  
 
A foreground process can be typically stopped by sending [[Linux_Signals#SIGINT_.282.29|SIGINT]] signal into it, and the shell sends [[Linux_Signals#SIGINT_.282.29|SIGINT]] when Ctrl-C is pressed.
 
A foreground process can be typically suspended by sending [[Linux_Signals#SIGTSTP_.2820.29|SIGTSTP]] signal into it. The shell sends [[Linux_Signals#SIGTSTP_.2820.29|SIGTSTP]] when Ctrl-Z is pressed. A suspended process can be resumed by executing
 
fg
 
in the shell that suspended it.


==Background Process==
==Background Process==
A background process is associated with the specific terminal that started it (the shell process is the parent of the background process), but does not block access to the shell. Instead, it executes in the background, leaving the user able to interact with the system while the command runs. Many background processes can be started by the same shell and can run at the same time.
A process can be launched in background by appending "&" at the end of the command line.The stdout, stderr of the background process are directed to the shell's stdout/stderr, unless they are explicitly redirected.
All background and stopped processes can be listed with:
jobs
=Pipeline=
A pipeline is a sequence of commands separated by one of the control operators '|' or '|&'
When '|' is used, the stdout of the command is connected via a pipe to the stdin of the next command, and stderr goes to the controlling terminal.
When '|&' is used, '''both''' stdout and stderr of the command is connected via a pipe to the stdin of the next command, and nothing goes to the controlling terminal.  The effect is equivalent with:
2>&1 |
When the pipeline execution status is evaluated, the default behavior is to consider only the exit code of the last command of the pipeline. To change this behavior to use the exit status of the rightmost command that exits with a non-zero status, execute [[Bash_set#set_-o_pipefail|set -o pipefail]].
The exit status of individual components of the pipeline can be accessed with:
<syntaxhighlight lang='bash'>
${PIPESTATUS[<command-index>]}
</syntaxhighlight>
where the command-index is zero based, the left-most command has the index 0:
<syntaxhighlight lang='bash'>
${PIPESTATUS[0]}
</syntaxhighlight>
=Tests [...] &#91;[...]]=
[...] is part of the shell built-in command [[Bash_test#Overview|test]], and it is a synonym for it. It tests the expression between [ ]. The &#91;[...]] construct is the more versatile version of [...], also called the ''extended test command''. For more details see {{Internal|Bash_Expressions#.5B.5B....5D.5D|&#91;[...]]}}
=Subshell=
{{External|https://www.tldp.org/LDP/abs/html/subshells.html}}
A subshell is a child process launched by a shell - a separate instance of the command processor.
A subshell is launched with:
<syntaxhighlight lang='bash'>
(...)
</syntaxhighlight>
[[Bash_Command_Substitution|Command substitution]] executes the command in a subshell and replaces it with its stdout:
<syntaxhighlight lang='bash'>
$(command)
</syntaxhighlight>
A subshell's PID is provided by the [[Bash_Built-In_Variables#BASHPID|BASHPID]] built-in variable. The subshell level is indicated by the [[Bash_Built-In_Variables#BASH_SUBSHELL|BASH_SUBSHELL]] built-in variable.
==External Commands, Built-ins and Subshells==
In general, an external command in a script forks off a subprocess, whereas a bash [[#Builtin|builtin]] does not. For this reason, builtins execute more quickly and use fewer system resources than their external command equivalents.
=Builtin=
{{External|https://www.tldp.org/LDP/abs/html/internal.html#BUILTINREF}}
{{External|https://bash.cyberciti.biz/bash-reference-manual/Bash-Builtins.html}}
A builtin is a command contained within the bash tool set, literally built in. This is either for performance reasons, as builtins execute faster than external commands, which usually require forking off a separate process, or because a particular builtin needs direct access to the shell internals.
List of Builtins: {{Internal|Bash#Built-In_Commands|Built-In Commands}}
=((...))=
{{Internal|((...))|((...))}}
=$((...))=
{{Internal|Numbers_and_Arithmetic_in_bash#.24.28.28....29.29|$((...)) and Arithmetic in bash}}
=<span id='Alias'></span>Aliases=
{{External|https://fog.ccsf.edu/~gboyd/cs160b/online/8-advanced/shopt.html}}
An alias is a textual substitution performed by the shell on a command. It is defined as such:
<syntaxhighlight lang='bash'>
alias something='ls -l'
</syntaxhighlight>
Aliases cannot be recursive, so it can be self-referencing without ambiguity:
<syntaxhighlight lang='bash'>
alias ls='ls -F'
</syntaxhighlight>
The already defined aliases can be displayed with the alias command:
<syntaxhighlight lang='bash'>
alias
</syntaxhighlight>
An alias can be removed by using the unalias command followed by the name of the alias. Aliases are local to the shell. When you start a new shell, aliases must be recreated. Thus, if you create an alias and start a new shell, that new shell does not have the new alias. They are more like local variables than environment variables, and there is no command for aliases that is equivalent to export.
In bash, aliases are not expanded by default in shell scripts. alias expansion must be enabled by using [[shopt]] at the beginning of the script:
<syntaxhighlight lang='bash'>
shopt -s expand_aliases
</syntaxhighlight>

Latest revision as of 19:09, 1 April 2024

Internal

Metacharacters

Metacharacters are characters that, when unquoted, separate words. These characters have a special meaning to bash and are interpreted and possibly modified before the modified strings are sent to the command as arguments. Also see regular expressions metacharacters. For more details on how bash processes a command line, see "bash Command Line Expansion"

Also see

http://tldp.org/LDP/abs/html/special-chars.html

Space

For bash, a space is a metacharacter that separates a command from its arguments, and arguments from each other.

Single Quotes

Enclosing a string in single quotes tells the shell NOT to interpret any shell metacharacters in the text between quotes: single quotes suppress all types of expansion.

Double Quotes

Double quotes permit parameter and variable expansion but suppress other types of expansion.

Dollar Sign ($)

$?

"$@"

"$@" means "all command line arguments received by the program at command line, where all arguments that contain spaces are preserved as such". The "$@" syntax (enclosed in double quotes) instructs the parser to preserve the spaces in all arguments that contain inner spaces. For example, the following script example:

#!/usr/bin/env bash
for i in "$@"; do
  echo "${i}"
done

will display

a
b
c

when invoked as example a b c, and:

a b
c

when invoked as example "a b" c.

However, example2:

#!/usr/bin/env bash
for i in $@; do
  echo "${i}"
done

will display:

a
b
c

in both cases: example2 a b c and example2 "a b" c.

Note that $@ is singled out as static syntax check error by some checkers.

Also see:

* and @ Relative to Array Elements

$*

"$*" will hand over exactly one argument, containing all original arguments, separated by single spaces.

$* will join all arguments by single spaces as well and will then split the string as the shell does on the command line, thus it will split an argument containing spaces into several arguments.

Also see:

* and @ Relative to Array Elements

Exclamation Mark (!)

|

'|' is the "pipe" character, and it is interpreted by bash as a request to set up a pipeline.

&

;

(...)

$(...) and (...)

{...}

Executed the enclosed commands in the same shell. There must be a space between '{' and the command and a space between the command and '}'. Also, the last command must be terminated with ';':

{ set a=1; echo ${a}; }

<

>

Redirection Operator (>)

Space

Tab

Globbing (*, ?)

Pathname Expansion


<<<

This is a "here-string". In passes the string specified at the right of the operator to the standard input of the command specified on the right.

cat <<< "something"

Control Operators

A control operator is a sequence of characters that performs a control function. It is one of the following characters or sequences:

|

See Pipelines.

|&

See Pipelines.

||

&

&&

;

;;

(...)

$(...) and (...)

The New Line

Other Operators

=~

=~

+= and -=

+= and -=

Login Shell

A login shell is a shell whose first character of its argument zero is a -, or it was started with the --login option.

Interactive Shell

An interactive shell is a shell started:

  • without the -c option
  • without non-option arguments
  • with the -i option

The interactive shell's standard input and error are both connected to terminals.

$- includes i if the shell is interactive, allowing a script or a startup file to test this state.

Non-interactive Shell

A shell started with -c option.

bash Startup Files

When bash is invoked as an interactive login shell, or as a non-interactive shell with the --login option, it first executes commands from /etc/profile, if exists.

Them it looks for ~/.bash_profile, ~/.bash_login and ~/.profile in that order. It executes commands from the first one that exists and is readable. The --noprofile option may be used when the shell is started to inhibit this behavior.

When a login shell exits, bash executes commands from ~/.bash_logout, if it exists.

When an interactive shell that is not a login shell is started, bash executes commands from ~/.bashrc, if it exists. This may be inhibited by using the --norc option. The --rcfile file option will force bash to execute commands from the specified file instead of ~/.bashrc.

When bash is started in non-interactive mode, to run a shell script, for example, it looks for the variable BASH_ENV in the environment, expands its value if it appears there, and uses the expanded value as the name of a file execute. Bash behaves as if the following command were executed:

if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi

but the value of the PATH variable is not used to search for the file name.

If any of the files exist but cannot be read, bash reports an error.

Parameters and Variables

bash Parameters and Variables

Command Substitution $(...)

Command Substitution

Command Line Expansion

Command Line Expansion

Functions

Functions

Processes

Foreground Process

Unless otherwise specified, a process launched by the shell will run in foreground, blocking use of the shell for the duration of the process, because the shells stdin, stdout and stderr are redirected to the process.

A foreground process can be typically stopped by sending SIGINT signal into it, and the shell sends SIGINT when Ctrl-C is pressed.

A foreground process can be typically suspended by sending SIGTSTP signal into it. The shell sends SIGTSTP when Ctrl-Z is pressed. A suspended process can be resumed by executing

fg

in the shell that suspended it.

Background Process

A background process is associated with the specific terminal that started it (the shell process is the parent of the background process), but does not block access to the shell. Instead, it executes in the background, leaving the user able to interact with the system while the command runs. Many background processes can be started by the same shell and can run at the same time.

A process can be launched in background by appending "&" at the end of the command line.The stdout, stderr of the background process are directed to the shell's stdout/stderr, unless they are explicitly redirected.

All background and stopped processes can be listed with:

jobs

Pipeline

A pipeline is a sequence of commands separated by one of the control operators '|' or '|&'

When '|' is used, the stdout of the command is connected via a pipe to the stdin of the next command, and stderr goes to the controlling terminal.

When '|&' is used, both stdout and stderr of the command is connected via a pipe to the stdin of the next command, and nothing goes to the controlling terminal. The effect is equivalent with:

2>&1 |

When the pipeline execution status is evaluated, the default behavior is to consider only the exit code of the last command of the pipeline. To change this behavior to use the exit status of the rightmost command that exits with a non-zero status, execute set -o pipefail.

The exit status of individual components of the pipeline can be accessed with:

${PIPESTATUS[<command-index>]}

where the command-index is zero based, the left-most command has the index 0:

${PIPESTATUS[0]}

Tests [...] [[...]]

[...] is part of the shell built-in command test, and it is a synonym for it. It tests the expression between [ ]. The [[...]] construct is the more versatile version of [...], also called the extended test command. For more details see

[[...]]

Subshell

https://www.tldp.org/LDP/abs/html/subshells.html

A subshell is a child process launched by a shell - a separate instance of the command processor.

A subshell is launched with:

(...)

Command substitution executes the command in a subshell and replaces it with its stdout:

$(command)

A subshell's PID is provided by the BASHPID built-in variable. The subshell level is indicated by the BASH_SUBSHELL built-in variable.

External Commands, Built-ins and Subshells

In general, an external command in a script forks off a subprocess, whereas a bash builtin does not. For this reason, builtins execute more quickly and use fewer system resources than their external command equivalents.

Builtin

https://www.tldp.org/LDP/abs/html/internal.html#BUILTINREF
https://bash.cyberciti.biz/bash-reference-manual/Bash-Builtins.html

A builtin is a command contained within the bash tool set, literally built in. This is either for performance reasons, as builtins execute faster than external commands, which usually require forking off a separate process, or because a particular builtin needs direct access to the shell internals.

List of Builtins:

Built-In Commands

((...))

((...))

$((...))

$((...)) and Arithmetic in bash

Aliases

https://fog.ccsf.edu/~gboyd/cs160b/online/8-advanced/shopt.html

An alias is a textual substitution performed by the shell on a command. It is defined as such:

alias something='ls -l'

Aliases cannot be recursive, so it can be self-referencing without ambiguity:

alias ls='ls -F'

The already defined aliases can be displayed with the alias command:

alias

An alias can be removed by using the unalias command followed by the name of the alias. Aliases are local to the shell. When you start a new shell, aliases must be recreated. Thus, if you create an alias and start a new shell, that new shell does not have the new alias. They are more like local variables than environment variables, and there is no command for aliases that is equivalent to export.

In bash, aliases are not expanded by default in shell scripts. alias expansion must be enabled by using shopt at the beginning of the script:

shopt -s expand_aliases