Ade Malsasa Akbar contact
Senior author, Open Source enthusiast.
Tuesday, July 19, 2016 at 23:01

Previously, we provided the first and second articles about GNU sed command line examples. Now it is the third article covering combination between sed and bash to do daily life jobs. We begin to focus in daily life jobs in this episode. This episode is a continuation of our previous articles. So if you miss something here, we suggest you to read the two previous episodes.


Text Examples


text6.txt:
gnu gnu gnu gnu unix
bsd bsd bsd bsd unix
unix unix unix unix
minix minix minix unix

As previous articles go, save this file in your /tmp and perform all sed commands there. 

21. Run Multiple Commands


Command Example:

sed -e 's/gnu/\U&/g' -e 's/bsd/\U&/g' text6.txt

Output Example:

master@master:/tmp$ sed -e 's/gnu/\U&/g' -e 's/bsd/\U&/g' text6.txt
GNU GNU GNU GNU unix
BSD BSD BSD BSD unix
unix unix unix unix
minix minix minix unix
master@master:/tmp$

Explanation:

We see the only different here compared to our previous examples, is, the `-e` option. This option handles one command or script of sed, so putting two of this options means doing two sed commands simultaneously. You may put more `-e` option to do more commands. See, in this example there are two `s///g` command sequences. See the result, there are two lines result (uppercased strings) from the two commands. 

22. Run Multiple Commands (Simpler)


Command Example:

sed 's/gnu/\U&/g; s/bsd/\U&/g' text6.txt

Output Example:

master@master:/tmp$ sed 's/gnu/\U&/g; s/bsd/\U&/g' text6.txt
GNU GNU GNU GNU unix
BSD BSD BSD BSD unix
unix unix unix unix
minix minix minix unix
master@master:/tmp$

Explanation: 

This example is simpler compared the number 21 example. It has the same result, but without the `-e` option. Instead, this commands make use of (`;`) character to separate two commands inside a pair of single quotes (`’ ’`). 

23. Combine sed with bash Looping Commands


Command Examples:

for i in {6..8}; do sed '/bsd/d' "text$i.txt"; echo ""; done;

Output Examples:

(1)
master@master:/tmp$ for i in {6..8}; do sed '/bsd/d' "text$i.txt"; echo ""; done;
gnu gnu gnu gnu unix
unix unix unix unix
minix minix minix unix

vms vms vms

unix dunix ultrix irix tru64
hp-ux sunos solaris
xnu hurd linux

(2)
master@master:/tmp$ for i in {6..8}; do sed '/bsd/!d' "text$i.txt"; echo ""; done;
bsd bsd bsd bsd unix

unix bsd unix unix
gnu gnu gnu bsd
minix minix minix minix bsd

bsd darwin

master@master:/tmp$

Explanation:

This is the first time within all episodes we use combination of GNU sed and GNU bash. Here, we make use of bash `for` command for looping. For the first timer, those two examples above perhaps makes no sense. But if you look one command in a programming manner you will understand it easier. For example, the first command above can be written like this:

for i in {6..8};
do
sed '/bsd/d' "text$i.txt";
echo "";
done;

This code is GNU bash scripting language. To read it properly, we can analyze it one by one:

  1. The first line sets a variable named `i` to contain numbers from 6 until 8 (hence, there would be 3 loops). Every loop will have `i` to contain one number, so the first loop i = 6, second loop i = 7, and third loop i = 8.
  2. The second line is a starting point of loop, to wrap the actual command line to do the actual job.
  3. The third line is the actual code. This is your sed command to do deleting any line containing “bsd” string. This is the actual job.
  4. The fourth line is also the command line to do the job, like the sed line, but it is just secondary. The most important command is still the third line. This fourth line does the job to give newlines so the final output will be easier to read.
  5. The fifth line is an ending point of loop, pairing with the second line (so there are do and done).
  6. Conclusion: this code means do the sed command line, but do loop it 3 times, for the number 6 until number 8, towards the file text6.txt then text7.txt then text8.txt. The `i` variable does the job for substituting number in every file name.
  7. These five lines can be written in a single line like the first command example here. 
     

24. Delete Contents of Multiple Files’ (bash Looping)


Command Example:

for i in {6..8}; do sed -i 'd' text$i.txt; done

Output Examples:

master@master:/tmp$ for i in {6..8}; do sed -i 'd' text$i.txt; done
master@master:/tmp$ for i in {6..8}; do cat text$i.txt; done
master@master:/tmp$

Explanation:

This example is just the same with the example number 23. It is also a bash looping code in single line. This code means do the sed command line, but do loop it three times, for the number 6 until 8, towards the file text6.txt, text7.txt, text8.txt. The sed command is just ‘d’, it means delete the whole content of file. So this single line command do erase all the contents of text6.txt, text7.txt, text8.txt. 

WARNING: this sed `-i` option and 'd' command result in deleting contents of the files. Be careful, make sure you don't delete important contents.

25. Rename Multiple Files (bash Looping)


Command Examples:

  1. for i in *.txt; do mv "$i" "`echo $i | sed "s/text/flext/g"`"; done
  2. for i in *.txt; do mv -v "$i" "`echo $i | sed "s/^[0-9]*[0-9]//g"`"; done

Output Examples:

(1)
master@master:/tmp$ ls
text6.txt
text7.txt
text8.txt
master@master:/tmp$ for i in *.txt; do mv "$i" "`echo $i | sed "s/text/flext/g"`"; done
master@master:/tmp$ ls
flext6.txt
flext7.txt
flext8.txt
master@master:/tmp$

(2)
master@master:/tmp$ touch 1lion.txt 2tiger.txt 3giraffe.txt 4monkey.txt 5eagle.txt 6deer.txt
master@master:/tmp$ ls
1lion.txt
2tiger.txt
3giraffe.txt
4monkey.txt
5eagle.txt
6deer.txt
master@master:/tmp$ for i in *.txt; do mv -v "$i" "`echo $i | sed "s/^[0-9]*[0-9]//g"`"; done
'1lion.txt' -> 'lion.txt'
'2tiger.txt' -> 'tiger.txt'
'3giraffe.txt' -> 'giraffe.txt'
'4monkey.txt' -> 'monkey.txt'
'5eagle.txt' -> 'eagle.txt'
'6deer.txt' -> 'deer.txt'
mv: 'flext.txt' and 'flext.txt' are the same file
master@master:/tmp$

Explanation:

First example do the rename loops towards some .txt files with number in every beginning of their file names.

  • The loop is basically mv command, but the argument of this mv is handled by echo and sed.
  • The echo command print the $i. This $i is a notation of the `i` variable content. Command echo print this in every loop.
  • The sed command receive the echo output via the `|` (pipe). So sed processes it, find and replace the “text” string into “flext” string.
  • This loop repeated until the filename text6.txt become flext6.txt, so on until text8.txt become flext8.txt.
  • You should pay attention to the usage or quotation marks (“ ”) and the apostrophe marks (` `). Place them properly just like the example. 

     

26. Delete Multiple Files (bash Looping)


Command Examples:

  1. for i in *.txt; do rm -v "`echo $i | sed "s/^[0-9]*[0-9]//g"`"; done
  2. for i in *.txt; do rm -v "`echo $i | sed -n "/^[0-9]*[0-9]/p"`"; done

Output Examples:

(1)
master@master:/tmp$ ls -w 1 *.txt
1lion.txt
2tiger.txt
3giraffe.txt
4monkey.txt
5eagle.txt
6deer.txt
deer.txt
eagle.txt
flext.txt
giraffe.txt
lion.txt
monkey.txt
tiger.txt
master@master:/tmp$ for i in *.txt; do rm -v "`echo $i | sed "s/^[0-9]*[0-9]//g"`"; done
removed 'lion.txt'
removed 'tiger.txt'
removed 'giraffe.txt'
removed 'monkey.txt'
removed 'eagle.txt'
removed 'deer.txt'
master@master:/tmp$ ls
1lion.txt
2tiger.txt
3giraffe.txt
4monkey.txt
5eagle.txt
6deer.txt

(2)
master@master:/tmp$ ls -w 1 *.txt
1lion.txt
2tiger.txt
3giraffe.txt
4monkey.txt
5eagle.txt
6deer.txt
elephant2.txt
giraffe4.txt
lion1.txt
monkey3.txt
tiger5.txt
master@master:/tmp$ for i in *.txt; do rm -v "`echo $i | sed -n "/^[0-9]*[0-9]/p"`"; done
removed '1lion.txt'
removed '2tiger.txt'
removed '3giraffe.txt'
removed '4monkey.txt'
removed '5eagle.txt'
removed '6deer.txt'
master@master:/tmp$ ls -w 1 *.txt
elephant2.txt
giraffe4.txt
lion1.txt
monkey3.txt
tiger5.txt
master@master:/tmp$

Explanation:

This example number 26 basically the same with number 25 except mv being rm. Here, we try to delete beginning-numbered file names and non-beginning numbered file names. We use substitution at the first example, and deletion at the second example. 

WARNING: this rm command is already dangerous, combined with bash looping commands it would be extremely dangerous. Be careful, make sure you don't make any regex or syntax mistake. Please avoid using sudo if you don't know what will happen.

27. Change Slash Delimiters to Another Characters


Command Examples:

  1. sed 's/unix/CHANGED/g' text6.txt
  2. sed 's@unix@CHANGED@g' text6.txt
  3. sed 's!unix!CHANGED!g' text6.txt
  4. sed 's?unix?CHANGED?g' text6.txt

Output Examples:

master@master:/tmp$ sed 's/unix/CHANGED/g' text6.txt
gnu gnu gnu gnu CHANGED
bsd bsd bsd bsd CHANGED
CHANGED CHANGED CHANGED CHANGED
minix minix minix CHANGED
master@master:/tmp$ sed 's@unix@CHANGED@g' text6.txt
gnu gnu gnu gnu CHANGED
bsd bsd bsd bsd CHANGED
CHANGED CHANGED CHANGED CHANGED
minix minix minix CHANGED
master@master:/tmp$ sed 's!unix!CHANGED!g' text6.txt
gnu gnu gnu gnu CHANGED
bsd bsd bsd bsd CHANGED
CHANGED CHANGED CHANGED CHANGED
minix minix minix CHANGED
master@master:/tmp$ sed 's?unix?CHANGED?g' text6.txt
gnu gnu gnu gnu CHANGED
bsd bsd bsd bsd CHANGED
CHANGED CHANGED CHANGED CHANGED
minix minix minix CHANGED
master@master:/tmp$

Explanation:

Using slash for sed command regex delimiters is sometimes confusing. Especially, if your regex contains many slashes too. The solution is you must use delimiters other than slash. As long as they are used consistently, you may use characters like at, exclamation mark, question mark, or underscore to replace the slash. The examples shown in this number 27 have the same commands and the same results. The only difference is the delimiter (`@`, `!`, and `?`). 

28. Adjust Line Spacing (Single)


Command Examples:



Output Examples:

master@master:/tmp$ cat text6.txt
gnu gnu gnu gnu unix
bsd bsd bsd bsd unix
unix unix unix unix
minix minix minix unix
master@master:/tmp$ sed ‘G’ text6.txt
gnu gnu gnu gnu unix

bsd bsd bsd bsd unix

unix unix unix unix

minix minix minix unix

Explanation:

This single ‘G’ command of sed causes addition one blank line for every line. 

29. Adjust Line Spacing (Double)


Command Example:

Output Example:

master@master:/tmp$ sed 'G;G' text6.txt
gnu gnu gnu gnu unix


bsd bsd bsd bsd unix


unix unix unix unix


minix minix minix unix


master@master:/tmp$

Explanation:

This double `G` command (separated with a (`;`) character) gives every line addition of two blank lines. 

30. Find & Replace (Lines Range)


Command Examples:

Output Examples:

master@master:/tmp$ sed '1,3 s/unix/CHANGED/g' text6.txt
gnu gnu gnu gnu CHANGED
bsd bsd bsd bsd CHANGED
CHANGED CHANGED CHANGED CHANGED
minix minix minix unix
master@master:/tmp$

Explanation:

You may determine the range of lines for sed command, not only “the coordinate”. See the two numbers separated with comma (`,`) above. That command means do substitution command only for line number 1 until line number 3.