Sed
External
Internal
GNU sed vs. BSD sed
GNU sed supports extra functionality relative to the BDS sed - for example \L and \U functions to turn a character to lower and upper cases. macOS comes with the BSD sed, so that is why it's probably a good idea to make upgrading to GNU sed part of the configuring a new Mac.
This is how you know you're using a BSD sed: if no GNU sed was installed, 'man sed' displays in the header "BSD General Commands Manual". The default location is
This is how you know you're using a GNU sed: 'man sed' lists Free Software Foundations in copyright. Also, 'which sed' that returns /usr/local/bin/sed is a good indicator.
Programmatic Test
echo a | sed -e 's/\(.*\)/\U\1/'
will return "Ua" for BDS sed and "A" for GNU sed.
Install
Command Line Options
-i
sed -i extension 'expression' filename
Edit the file in-place, saving backups with the specified extension. If a zero-length extension is given, no backup will be saved:
sed -i '' 'expression' filename
Commands
s
Substitute according to regex.
p
Print the current pattern space. Note that the pattern space is printed automatically, unless -n is used.
sed Regular Expressions
Insert a Line/Append in a Specific Position (line number) in a file
Figure out the line number:
# determine the last line that contains ^JAVA_OPTS
local n
n=$(cat ${f} | grep -n "^JAVA_OPTS=" | tail -1) || { echo "failed to determine the line number" 1>&2; exit 1; }
n=${n%%:*}
Insert a line at line 'n':
# insert at line "n":
cat ${f} | sed -e ${n}'a\
This line will be inserted at line number '"${n}"', and this '"${variable}"' will be substituted' > ${dest}
To append at a specific line number, determine the line number as before and effectively "substitute" (s) the line end with your addition:
cat ${f} | sed -e ${n}'s/$/this text will be appended at line number, and this '"${variable}"' will be substituted\n' > ${dest}
Insert Multiple Lines at a Specific Position in a File
Use the above recipe, and in the last phase, use:
cat ${f} | sed -e ${n}'a\
blah\
blah\
blah with variable substitution, etc' > ${dest}
Insert on a New Line
Assuming that blah is the last sequence on the line:
cat ... | sed -e 's/blah/blah\n on a new line/'
Deleting with sed
Delete a Line that Matches a Certain Pattern
sed will delete a line identified by line number or if the line matches a regular expression pattern:
sed '{<n>|/<regex>/}d' <file-name>
where:
- n is the 1-based line number.
- /<regex>/ is the pattern.
Examples:
Delete the third line of the file
sed -e '3d' a.txt
Delete lines 5 through 10 and 12:
sed -e '5,10d;12d' a.txt
Delete all lines below line 10:
sed -e '10,$d' a.txt
Delete all lines that match a certain pattern
sed '/b..h/d' a.txt > b.txt
Delete all lines that match a certain patter and then the next lines
The solution uses pattern-matching deletion command 'd', and the command 'N' that loads the next pattern space.
This removes the pattern line and the next line:
sed -e '/something/{N;d}/ ...
This removes the pattern line and the next two lines:
sed -e '/something/{N;N;d}/ ...
Delete only the next line containing the pattern, but not the very line:
sed '/something/{N;s/\n.*//;}' file
Delete only the line containing the pattern, and the line before the pattern:
sed -n '/something/{s/.*//;x;d;};x;p;${x;p;}' file | sed '/^$/d'
Delete empty lines
sed '/^$/d' file
Delete all lines which are empty or which contains just some blank spaces
sed '/^ *$/d' file
Delete the last line
$ means "the last line"
sed -e '$d' a.txt
Delete Lines Between Certain "Addresses"
sed '<n1>,<n2>d' <file-name> sed '/<regex1/,/regex2/d' <file-name> sed '<n1>,/regex1/d' <file-name> sed '/regex1/,<n2>d' <file-name>
The addresses can be line numbers or regular expressions. Line numbers are 1-based.
- Line numbers and regular expressions can be combined, they're both treated as addresses.
Examples:
Delete all lines between line 2 and line 5 (inclusively)
sed '2,5d' a.txt > b.txt
Delete all lines between the occurrences of two regular expressions
sed '/red/,/blue/d' a.txt > b.txt
Delete Negation
sed can be used to delete all lines that do not match a certain pattern.
sed '/<regex>/!d' <file-name>
Printing with sed
Print a Line Specified by Its Number
Line numbering is 1-based. The following command suppresses the automatic display of the pattern space by specifying -n, then it loads the specified line in the pattern space and prints it:
sed -n <line_number>p <file>
The folding example prints line 12:
sed -n 12p a.txt
Replacing First Occurrence In a File
On Mac, use 1. I read that on other sed version I have to use 0:
sed -e '1,/pattern/s/pattern/replacement/' ${file} > ./tmp
Convert a Character to Upper or Lower Case
This functionality is only available in the GNU sed.
Use \L and \U, as follows:
sed -e 's/^\(.\)/\U\1/'
Numbers are not changed.
To convert the entire string:
sed -e 's/^\(.*\)$/\U\1/')