Bash Input/Output: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
(Created page with "=Internal= * bash")
 
 
(51 intermediate revisions by the same user not shown)
Line 2: Line 2:


* [[bash#Subjects|bash]]
* [[bash#Subjects|bash]]
=Here-Docs=
{{External|http://tldp.org/LDP/abs/html/here-docs.html}}
==Echo Multiple Lines to <tt>stdout</tt>==
<pre>
cat <<EOF
  blah
  ${some_var}
  $(some-command)
  blah
EOF
</pre>
* It will perform variable substitution. If you don't want that, escape <tt>\${some_var}</tt>.
* <tt>`...`</tt> sequences should be escaped as shown here: <tt>\`...\`</tt> otherwise they will executed before the output is sent to stdout.
* \ will join lines. If you want "\" in the output, then you should escape it:
<pre>
      ...
      blah blah \\
      ...
</pre>
==Echo Multiple Lines into a File==
<syntaxhighlight lang='bash'>
cat > /tmp/test.txt << EOF
blah
EOF
</syntaxhighlight>
or
<syntaxhighlight lang='bash'>
(cat << EOF
blah
EOF
) > /tmp/test.txt
</syntaxhighlight>
To append:
<syntaxhighlight lang='bash'>
cat >> /tmp/test.txt << EOF
blah
EOF
</syntaxhighlight>
=Here-String=
<code><<<</code> is known as here-string. Instead of typing in text, you give a pre-made string of text to a program.
=Iterating over a Multi-Line Command Output=
<syntaxhighlight lang='bash'>
local f=$(mktemp)
(....) > ${f}
# iterate through file as shown below
rm -f ${f}
</syntaxhighlight>
=Iterating over Lines from a File=
==Line Processing Takes Place in a Subprocess==
The following sequence is simple, but the line processing is done in a different sub-shell than the one that invoked <code>cat</code>:
<syntaxhighlight lang='bash'>
cat ./file | while read line; do
    echo ">>> ${line}"
done
</syntaxhighlight>
==Iterating over Lines from a File in the Same bash Process==
===With <tt>while</tt>===
<syntaxhighlight lang='bash'>
while IFS= read -r line; do
  echo ${line}
done <<< "$(cat ./myfile.txt)"
</syntaxhighlight>
This was tested and indeed the top level [[Bash_Built-In_Variables#BASHPID|BASHPID]] has the same value with [[Bash_Built-In_Variables#BASHPID|BASHPID]] resolved inside the loop.
⚠️ Note this will work with '''ny command that generates multi-line output'''.
===With <tt>for</tt>===
Note that the following approach works better for small files. If the file is large, the content will be first cached in memory, and it may look like the command is irresponsive:
<blockquote style="background-color: #f9f9f9; border: solid thin lightgrey;">
:[[Bash_for#Iterating_over_Lines_in_the_Same_bash_Process|Iterating over Lines in the Same bash Process with <tt>for</tt>]]
</blockquote>
===With <tt>sed</tt>===
This approach has the advantage that lines are read one by one and not buffered:
<pre>
filename=...
line_count=$(wc -l ${filename}) || exit 1
line_count=${line_count% *}
line_count=$(echo ${line_count} | sed -e 's/ *//')
line_number=1
while [ ${line_number} -le ${line_count} ]; do
    local line
    line=$(sed -n ${line_number}p ${filename})
    # process line ...
    ((line_number++))
done
</pre>
=Read Patterns=
==Read First Line from a File==
head -i <''file''>
==Read the First Line of a Command's Output==
bash --version | head -n 1
==Read Confirmation to Proceed==
{{Internal|Bash_read#Confirmation_to_Proceed|Confirmation to Proceed}}
==Read a Password with Confirmation==
{{Internal|Bash_read#Read_a_Password_with_Confirmation|Read a Password with Confirmation}}
=Extracting a Line Specified by Its Number from a File=
==With head and tail==
Extract line 15 (line numbers are 1-based) with [[bash head|head]] and [[bash tail|tail]]:
<syntaxhighlight lang='bash'>
cat ./test.txt | head -15 | tail -1
</syntaxhighlight>
==With sed==
{{Internal|Sed#Print_a_Line_Specified_by_Its_Number|Print a Line Specified by Its Number with sed}}
=echo Formatting=
Bold:
echo -e "\033[1msomething bold \033[0m"
Red:
echo -e "\033[31msomething red\033[0m"
=Quoted String Expansion=
$'\012' # Octal value
$'\x0a' # Hexadecimal value
More details: {{Internal|Bash_Built-In_Variables#Quoted_String_Expansion_.24.27....27|Quoted String Expansion}}
=<span id='Reading_from_stdin'></span>Interactively Reading from stdin=
{{Internal|Bash_read#Confirmation_to_Proceed|Interactively Read Input from stdin}}
=Redirection Operator (>)=
{{External|http://tldp.org/LDP/abs/html/io-redirection.html#IOREDIRREF}}
">" is the output redirection operator.
By itself, the ">" redirects the stdout of the command to a file, resetting it to zero length before writing:
<syntaxhighlight lang='bash'>
command > /tmp/some-file
</syntaxhighlight>
The above command is equivalent to:
<syntaxhighlight lang='bash'>
command 1> /tmp/some-file
</syntaxhighlight>
To redirect and append stdout only to a file:
<syntaxhighlight lang='bash'>
command 1>> /tmp/some-file
</syntaxhighlight>
To redirect stderr only to a file, resetting it to zero length before writing:
<syntaxhighlight lang='bash'>
command 2> /tmp/some-file
</syntaxhighlight>
To redirect and append stderr only to a file:
<syntaxhighlight lang='bash'>
command 2>> /tmp/some-file
</syntaxhighlight>
To redirect both stdout and stderr to a file, resetting it to zero length before writing:
<syntaxhighlight lang='bash'>
command &> /tmp/some-file
</syntaxhighlight>
To redirect stderr to stdout, for example to send error messages to the same place as standard output:
<syntaxhighlight lang='bash'>
command 2>&1
</syntaxhighlight>
The following syntax is useful to append both stdout and stderr to the same file:
<syntaxhighlight lang='bash'>
command 2>&1 >>/tmp/some-file
</syntaxhighlight>
The following syntax is useful to process in-line both stdout and stderr
<syntaxhighlight lang='bash'>
command 2>&1 | some-processor
</syntaxhighlight>
Bash 4 has the following abbreviation:
<syntaxhighlight lang='bash'>
|&
</syntaxhighlight> for "2>&1 |"
==Opening File Descriptors for Reading and Writing==
The redirection operator causes the file whose name results from the expansion of <code>word</code> to be opened for both reading and writing on file descriptor <code>n</code> or on file descriptor 0 if <code>n</code> is not specified. If the file does not exists, it is created:
<syntaxhighlight lang='bash'>
n<>word
</syntaxhighlight>
<syntaxhighlight lang='bash'>
5<>/dev/null
</syntaxhighlight>
=Terminal Colors=
<font color=darkgray>TODO: [[tput]]</font>
=Temporary Files and Directories=
{{Internal|mktemp#Overview|<tt>mktemp</tt>}}

Latest revision as of 17:40, 11 May 2023

Internal

Here-Docs

http://tldp.org/LDP/abs/html/here-docs.html

Echo Multiple Lines to stdout

cat <<EOF
  blah
  ${some_var}
  $(some-command)
  blah
EOF
  • It will perform variable substitution. If you don't want that, escape \${some_var}.
  • `...` sequences should be escaped as shown here: \`...\` otherwise they will executed before the output is sent to stdout.
  • \ will join lines. If you want "\" in the output, then you should escape it:
      ...
      blah blah \\
      ...

Echo Multiple Lines into a File

cat > /tmp/test.txt << EOF
blah
EOF

or

(cat << EOF
blah
EOF
) > /tmp/test.txt

To append:

cat >> /tmp/test.txt << EOF
blah
EOF

Here-String

<<< is known as here-string. Instead of typing in text, you give a pre-made string of text to a program.

Iterating over a Multi-Line Command Output

local f=$(mktemp)
(....) > ${f}
# iterate through file as shown below
rm -f ${f}

Iterating over Lines from a File

Line Processing Takes Place in a Subprocess

The following sequence is simple, but the line processing is done in a different sub-shell than the one that invoked cat:

cat ./file | while read line; do
    echo ">>> ${line}"
done

Iterating over Lines from a File in the Same bash Process

With while

while IFS= read -r line; do
  echo ${line}
done <<< "$(cat ./myfile.txt)"

This was tested and indeed the top level BASHPID has the same value with BASHPID resolved inside the loop.

⚠️ Note this will work with ny command that generates multi-line output.

With for

Note that the following approach works better for small files. If the file is large, the content will be first cached in memory, and it may look like the command is irresponsive:

Iterating over Lines in the Same bash Process with for

With sed

This approach has the advantage that lines are read one by one and not buffered:

filename=...
line_count=$(wc -l ${filename}) || exit 1
line_count=${line_count% *}
line_count=$(echo ${line_count} | sed -e 's/ *//')

line_number=1

while [ ${line_number} -le ${line_count} ]; do

    local line
    line=$(sed -n ${line_number}p ${filename})

    # process line ...

    ((line_number++))
done

Read Patterns

Read First Line from a File

head -i <file>

Read the First Line of a Command's Output

bash --version | head -n 1

Read Confirmation to Proceed

Confirmation to Proceed

Read a Password with Confirmation

Read a Password with Confirmation

Extracting a Line Specified by Its Number from a File

With head and tail

Extract line 15 (line numbers are 1-based) with head and tail:

cat ./test.txt | head -15 | tail -1

With sed

Print a Line Specified by Its Number with sed

echo Formatting

Bold:

echo -e "\033[1msomething bold \033[0m"

Red:

echo -e "\033[31msomething red\033[0m"

Quoted String Expansion

$'\012' # Octal value
$'\x0a' # Hexadecimal value

More details:

Quoted String Expansion

Interactively Reading from stdin

Interactively Read Input from stdin

Redirection Operator (>)

http://tldp.org/LDP/abs/html/io-redirection.html#IOREDIRREF

">" is the output redirection operator.

By itself, the ">" redirects the stdout of the command to a file, resetting it to zero length before writing:

command > /tmp/some-file

The above command is equivalent to:

command 1> /tmp/some-file

To redirect and append stdout only to a file:

command 1>> /tmp/some-file

To redirect stderr only to a file, resetting it to zero length before writing:

command 2> /tmp/some-file

To redirect and append stderr only to a file:

command 2>> /tmp/some-file

To redirect both stdout and stderr to a file, resetting it to zero length before writing:

command &> /tmp/some-file

To redirect stderr to stdout, for example to send error messages to the same place as standard output:

command 2>&1

The following syntax is useful to append both stdout and stderr to the same file:

command 2>&1 >>/tmp/some-file

The following syntax is useful to process in-line both stdout and stderr

command 2>&1 | some-processor

Bash 4 has the following abbreviation:

|&

for "2>&1 |"

Opening File Descriptors for Reading and Writing

The redirection operator causes the file whose name results from the expansion of word to be opened for both reading and writing on file descriptor n or on file descriptor 0 if n is not specified. If the file does not exists, it is created:

n<>word
5<>/dev/null

Terminal Colors

TODO: tput

Temporary Files and Directories

mktemp