The Power of sed – Tweak Your Terminal

In the world of Unix-like systems, sed (Stream Editor) is an influential tool that allows for manipulation of text streams: either files or input piped from other commands. From simple text substitutions to complex transformations, sed is a versatile tool that can make processing and manipulating text easier and more efficient. This article aims to explore the essential aspects of the sed command.

This article makes use of a collection of random files. These files were put together to help you “Tweak Your Terminal”. Checkout our guide to setting up the random files.

Basic Structure of a sed Command

The basic structure of a sed command is as follows:

sed 'command' file_name

Here, ‘command’ is the instruction you want sed to perform, and ‘file_name’ is the file you want the command to act on.

Substituting Text: s Command

The most common use of sed is substituting text, which is performed with the s (substitute) command. Here’s the basic syntax:

sed 's/search_pattern/replacement_text/' file_name

This command will replace the first occurrence of ‘search_pattern’ with ‘replacement_text’ on each line of the file.

To replace every occurrence of the pattern on each line, not just the first, you can add the g (global) flag at the end:

sed 's/search_pattern/replacement_text/g' file_name

Let’s try changing the word bar to barred with the strandberg example file.

sed 's/bar/barred/' strandberg.txt
Foo? Hoo? Baz. Foo barred, Foo Foo Baz! Foo bar, baz? Bar? Foo! Hoo? Baz!
Foo Foo Baz. Foo barred, baz. Bar; Foo. Hoo! Baz! Foo bar, Foo Foo Baz!
baz; Bar! Foo? Hoo? Baz. Foo barred, Foo Foo Baz! Foo bar, baz. Bar. Foo;

You can see that command sort of worked, but not completely. It seems to have only made a change once per line. We need to use the global flag in the regular expression to tell sed to keep replacing every occurrence.

sed 's/bar/barred/g' strandberg.txt
Foo? Hoo? Baz. Foo barred, Foo Foo Baz! Foo barred, baz? Bar? Foo! Hoo? Baz!
Foo Foo Baz. Foo barred, baz. Bar; Foo. Hoo! Baz! Foo barred, Foo Foo Baz! 
baz; Bar! Foo? Hoo? Baz. Foo barred, Foo Foo Baz! Foo barred, baz. Bar. Foo;

We still have some occurrences of Bar in our text. The installed version of sed is case sensitive. We will need to provide sed with both versions of the regular expression to preserve the case. We will also use the -e option to explicitly tell sed we are using more than one command.

sed -e 's/bar/barred/g' -e 's/Bar/Barred/g' strandberg.txt
Foo? Hoo? Baz. Foo barred, Foo Foo Baz! Foo barred, baz? Barred? Foo! Hoo?
Foo Foo Baz. Foo barred, baz. Barred; Foo. Hoo! Baz! Foo barred, Foo Foo Baz! 
baz; Barred! Foo? Hoo? Baz. Foo barred, Foo Foo Baz! Foo barred, baz. Barred.

Deleting Lines: d Command

The d command is used to delete lines in a file that match a specific pattern:

sed '/pattern_to_match/d' file_name

This command will delete all lines containing ‘pattern_to_match’ from the file.

Print Command: p

The p command is used to print lines in a file that match a specific pattern:

sed -n '/pattern_to_match/p' file_name

This command prints all lines containing ‘pattern_to_match’ from the file. The -n option suppresses automatic printing, which ensures only the lines you specify are printed.

Working with Multiple Files

sed can process multiple files at once:

sed 's/search_pattern/replacement_text/g' file1.txt file2.txt

This command will perform the text substitution on both ‘file1.txt’ and ‘file2.txt’.

Editing Files In-Place

By default, sed outputs the result to standard output (usually the terminal), and the original files remain unchanged. If you want sed to edit files in-place (i.e., change the original files), you can use the -i option:

sed -i 's/search_pattern/replacement_text/g' file_name

Be careful with this option, as it overwrites the original files without any prompt.

Hopefully you can see the power of the sed command, but I gets better than editing simple text files. Imagine using sed to edit configuration files.

Imagine this small configuration snippet from php.ini.
; Maximum amount of memory a script may consume (128MB)
; http://php.net/memory-limit
memory_limit = 128M

Now imagine that the value, 128M, might be different on different systems, but we need to change it to 1G.

sed -E -e 's/^memory_limit = [0-9]+[MB]/memory_limit = 1G/' php.ini
; Maximum amount of memory a script may consume (128MB)
; http://php.net/memory-limit
memory_limit = 1G

The -E option is to enable the extended regular expressions.

Conclusion

sed is a potent tool in the Unix-like systems. The command’s ability to filter and transform text makes it indispensable for text processing tasks. While we have touched upon some basic and commonly used sed commands, there’s a lot more to learn. The man sed command will give you a detailed overview of sed and its capabilities. Learning to use sed effectively can save you a lot of time and make your terminal work more efficient.