07. Pipelining |, sort, uniq, grep, wc, tee

Pipelining is a very powerful feature that allows you to send the output of one command as the input to another.

The notation used is a vertical bar |.

$ ls /usr/bin | less 

The output of the ls /usr/bin command will be passed in as an argument to the command on the right. Thus, we will be able to view the contents in our bin file through the less file reader.

We can have multiple pipes, which makes pipelineing not only useful, but fun!

Sorting

To sort the outputs alphabetically, use the sort command.

$ ls .. | sort

Sort Options

-f
Ignore case.
-M
Month sort with JAN < ... < DEC
-r
Reverse sort.

Omitting repeated lines with uniq

To grab just the unique names of within a file, we can use uniq.

$ cat employeeNames.txt | sort | uniq 

Uniq options

-c
Precede each line with the count of the occurences in the input.
-d
Only output the repeated lines.
-u
Output lines that are not repeated.
-i
Case insensitive comparison.

Using grep to find text patterns

To find text patterns, we can use grep, followed by a string.

Grep actually stands for "global expression pattern." If you know regex, you can use regex syntax to capture a specific string pattern.

$ ls .. | grep .html

This would return all outputs that have the .html extention in them.

Checking the number of lines, words and bytes

To check the number of lines, words and bytes in a file, we can use the wc command.

$ less fileName.txt | wc
5 7 61

This means we have 5 lines, 7 words and 61 bytes.

Options

-c
Number of bytes in each input file to standard out.
-l
Number of lines in each input file to standard out.
-m
Number of characters in each input files to standard out.
-w
Number of words in each input file to standard out.

Outputting to a file and standard out

We can use the tee command to output to standard out as well as a file. In this way, we can save the results of our pipeline.

$ ls /usr/bin | tee test.txt | grep zip

This command lists all files in /usr/bin, sending it off to the next pipe, then writes it to test.txt.

Using man to check for options

Remember to check each command's man page to learn more options!

$ man sort

Take your Linux skills to the next level!

How Linux Works

Take your Linux skills to the next level! Try Linux & UNIX

In this completely revised second edition of the perennial best seller How Linux Works, author Brian Ward makes the concepts behind Linux internals accessible to anyone curious about the inner workings of the operating system. Inside, you'll find the kind of knowledge that normally comes from years of experience doing things the hard way.

$ Check price
39.9539.95Amazon 5 logo(114+ reviews)

More Linux & UNIX resources

Aching back from coding all day?

Foam Seat Cushion

Aching back from coding all day? Try Back Problems

This foam seat cushion relieves lowerback pain, numbness and pressure sores by promoting healthy weight distribution, posture and spine alignment. Furthermore, it reduces pressure on the tailbone and hip bones while sitting. Perfect for sitting on the computer desk for long periods of time.

$ Check price
99.9599.95Amazon 4.5 logo(9,445+ reviews)

More Back Problems resources

Ad