07. Pipelining |, sort, uniq, grep, wc, tee

Pipelining is a very powerful feature that allows you to send the output of one command as the input to another.

The notation used is a vertical bar |.

$ ls /usr/bin | less 

The output of the ls /usr/bin command will be passed in as an argument to the command on the right. Thus, we will be able to view the contents in our bin file through the less file reader.

We can have multiple pipes, which makes pipelineing not only useful, but fun!

Sorting

To sort the outputs alphabetically, use the sort command.

$ ls .. | sort

Sort Options

-f
Ignore case.
-M
Month sort with JAN < ... < DEC
-r
Reverse sort.

Omitting repeated lines with uniq

To grab just the unique names of within a file, we can use uniq.

$ cat employeeNames.txt | sort | uniq 

Uniq options

-c
Precede each line with the count of the occurences in the input.
-d
Only output the repeated lines.
-u
Output lines that are not repeated.
-i
Case insensitive comparison.

Using grep to find text patterns

To find text patterns, we can use grep, followed by a string.

Grep actually stands for "global expression pattern." If you know regex, you can use regex syntax to capture a specific string pattern.

$ ls .. | grep .html

This would return all outputs that have the .html extention in them.

Checking the number of lines, words and bytes

To check the number of lines, words and bytes in a file, we can use the wc command.

$ less fileName.txt | wc
5 7 61

This means we have 5 lines, 7 words and 61 bytes.

Options

-c
Number of bytes in each input file to standard out.
-l
Number of lines in each input file to standard out.
-m
Number of characters in each input files to standard out.
-w
Number of words in each input file to standard out.

Outputting to a file and standard out

We can use the tee command to output to standard out as well as a file. In this way, we can save the results of our pipeline.

$ ls /usr/bin | tee test.txt | grep zip

This command lists all files in /usr/bin, sending it off to the next pipe, then writes it to test.txt.

Using man to check for options

Remember to check each command's man page to learn more options!

$ man sort

Aching back from coding all day?

Self-Massage Tool

Aching back from coding all day? Try Back Problems

Relieve spasms, tight muscles, trigger points and pressure points with the Body Back Buddy! This trigger point massage is designed to help you self-message any area of your body - especially those that are hard to reach. Keeping your muscles relaxes and out of contraction is importan in helping to reduce pain and prevent muscle injury.

$ Check price
29.9529.95Amazon 4.5 logo(3,443+ reviews)

More Back Problems resources

Take your Linux skills to the next level!

System Admin Handbook

Take your Linux skills to the next level! Try Linux & UNIX

This book approaches system administration in a practical way and is an invaluable reference for both new administrators and experienced professionals. It details best practices for every facet of system administration, including storage management, network design and administration, email, web hosting, scripting, and much more.

$ Check price
74.9974.99Amazon 4.5 logo(142+ reviews)

More Linux & UNIX resources

Ad