Sed

External

• https://www.gnu.org/software/sed/manual/sed.html

Internal

• Linux
• sed Regular Expressions
• awk

GNU sed vs. BSD sed

sed [options] -e [command] file1 file2 ...

GNU sed supports extra functionality relative to the BDS sed - for example \L and \U functions to turn a character to lower and upper cases. macOS comes with the BSD sed, so that is why it's probably a good idea to make upgrading to GNU sed part of the configuring a new Mac.

This is how you know you're using a BSD sed: if no GNU sed was installed, 'man sed' displays in the header "BSD General Commands Manual". The default location is

This is how you know you're using a GNU sed: 'man sed' lists Free Software Foundations in copyright. Also, 'which sed' that returns /usr/local/bin/sed is a good indicator.

Programmatic Test

echo a | sed -e 's/\(.*\)/\U\1/'

will return "Ua" for BDS sed and "A" for GNU sed.

Install

Install gnu sed on Mac

Command Line Options

-i

sed -i <extension> <expression> <filename>

Edit the file in-place, saving backups with the specified extension. If a zero-length extension is given, no backup will be saved:

sed -i -e <expression> <filename>

Example:

sed -i -e 's/a/b/' ./somefile.txt

-e

Provides the command expression:

sed -e 's/a/b/g'

Multiple -e are allowed:

sed -e 's/a/b/g' -e 's/x/y/g' -e ...

Commands

s

Substitute according to regex.

The return code of the command is not influenced by whether the regex matched or not.

p

Print the current pattern space. Note that the pattern space is printed automatically, unless -n is used.

sed Regular Expressions

sed Regular Expressions

Insert a Line/Append in a Specific Position (line number) in a file

Figure out the line number:

# determine the last line that contains ^JAVA_OPTS
local n
n=$(cat ${f} | grep -n "^JAVA_OPTS=" | tail -1) || { echo "failed to determine the line number" 1>&2; exit 1; }
n=${n%%:*}

Insert a line at line 'n':

# insert at line "n":
cat ${f} | sed -e ${n}'a\
This line will be inserted at line number '"${n}"', and this '"${variable}"' will be substituted' > ${dest}

To append at a specific line number, determine the line number as before and effectively "substitute" (s) the line end with your addition:

cat ${f} | sed -e ${n}'s/$/this text will be appended at line number, and this '"${variable}"' will be substituted\n' > ${dest}

Insert Multiple Lines at a Specific Position in a File

Use the above recipe, and in the last phase, use:

cat ${f} | sed -e ${n}'a\
blah\
blah\
blah with variable substitution, etc' > ${dest}

Insert on a New Line

Assuming that blah is the last sequence on the line:

cat ... | sed -e 's/blah/blah\n    on a new line/'

Deleting with sed

Delete a Line that Matches a Certain Pattern

sed will delete a line identified by line number or if the line matches a regular expression pattern:

sed '{<n>|/<regex>/}d' <file-name>

where:

• n is the 1-based line number.
• /<regex>/ is the pattern.

Examples:

Delete the third line of the file

sed -e '3d' a.txt 

Delete lines 5 through 10 and 12:

sed -e '5,10d;12d' a.txt

Delete all lines below line 10:

sed -e '10,$d' a.txt

Delete all lines that match a certain pattern

sed '/b..h/d' a.txt > b.txt

Delete all lines that match a certain patter and then the next lines

The solution uses pattern-matching deletion command 'd', and the command 'N' that loads the next pattern space.

This removes the pattern line and the next line:

sed -e '/something/{N;d}' ...

This removes the pattern line and the next two lines:

sed -e '/something/{N;N;d}' ...

Delete only the next line containing the pattern, but not the very line:

sed '/something/{N;s/\n.*//;}' file

Delete only the line containing the pattern, and the line before the pattern:

 sed -n '/something/{s/.*//;x;d;};x;p;${x;p;}' file | sed '/^$/d'

Delete empty lines

sed '/^$/d' file

Delete all lines which are empty or which contains just some blank spaces

sed '/^ *$/d' file

Delete the last line

$ means "the last line"

sed -e '$d' a.txt

Delete Lines Between Certain "Addresses"

sed '<n1>,<n2>d' <file-name>

sed '/<regex1/,/regex2/d' <file-name>

sed '<n1>,/regex1/d' <file-name>

sed '/regex1/,<n2>d' <file-name>

The addresses can be line numbers or regular expressions. Line numbers are 1-based.

Line numbers and regular expressions can be combined, they're both treated as addresses.

Examples:

Delete all lines between line 2 and line 5 (inclusively)

sed '2,5d' a.txt > b.txt

Delete all lines between the occurrences of two regular expressions

sed '/red/,/blue/d' a.txt > b.txt

Delete Negation

sed can be used to delete all lines that do not match a certain pattern.

sed '/<regex>/!d' <file-name>

Printing with sed

Print a Line Specified by Its Number

Line numbering is 1-based. The following command suppresses the automatic display of the pattern space by specifying -n, then it loads the specified line in the pattern space and prints it:

sed -n <line_number>p <file>

The folding example prints line 12:

sed -n 12p a.txt

Replacing First Occurrence In a File

On Mac, use 1. I read that on other sed version I have to use 0:

    sed -e '1,/pattern/s/pattern/replacement/' ${file} > ./tmp

Convert a Character to Upper or Lower Case

This functionality is only available in the GNU sed.

Use \L and \U, as follows:

sed -e 's/^\(.\)/\U\1/'

Numbers are not changed.

To convert the entire string:

sed -e 's/^\(.*\)$/\U\1/')