The Secrets of File Redirection
a file-redirection command in a regular command line:
< redirects input
> redirects output
>> redirects output and appends the information to the existing file
$ cat -v < datafile > file
Counting Words and Lines Using wc
$ wc file
$ wc < file
$ cat file | wc -l
$ wc -l lines
$ wc -w words
$ wc -c characters
How many files you have in your home directory?
$ ls | wc -l
How about a quick gauge of how many users are on the system?
$ who | wc -l
How many accounts are on your computer?
$ cat /etc/passwd | wc -l
Removing Extraneous Lines Using uniq
-u lists only lines that are not repeated,
-d lists only lines that are repeated (the exact opposite of -u)
-c adds a count of how many times each line occurred.
$ cat line1 line2 line1 line2 line1 line2 >file2
$ more file2
i am line 1.
i am line 2.
i am line 1.
i am line 2.
i am line 1.
i am line 2.
$ wc file2
6 24 78 file2
$ uniq file2|wc
6 24 78
$ uniq file2
i am line 1.
i am line 2.
i am line 1.
i am line 2.
i am line 1.
i am line 2.
$ cat line1 line1 line1 line1 >file
$ cat line2 >>file
$ more file
i am line 1.
i am line 1.
i am line 1.
i am line 1.
i am line 2.
Remember, the uniq command removes duplicate lines only if they’re adjacent.
$ uniq -c file2
1 i am line 1.
1 i am line 2.
1 i am line 1.
1 i am line 2.
1 i am line 1.
1 i am line 2.
$ uniq -d file2
$ uniq -u file2
i am line 1.
i am line 2.
i am line 1.
i am line 2.
i am line 1.
i am line 2.
$ cat line1 line1 line1 line1 line2 >file1
$ more file1
i am line 1.
i am line 1.
i am line 1.
i am line 1.
i am line 2.
$ wc file1
5 20 65 file1
$ uniq file1|wc
2 8 26
$ uniq file1
i am line 1.
i am line 2.
$ uniq -c file1
4 i am line 1.
1 i am line 2.
$ uniq -d file1
i am line 1.
$ uniq -u file1
i am line 2.
Sorting Information in a File Using sort
Flags for the sort command.
Flag Function
-b Ignore leading blanks.
-d Sort in dictionary order (only letters, digits, and blanks are significant).
-f Fold uppercase into lowercase; that is, ignore the case of words.
-n Sort in numerical order.
-r Reverse order of the sort.
$ ls -1F
A/
B/
C
D
E
a/
b/
c
d
e
$ ls -1F|sort -f
A/
a/
B/
b/
C
c
D
d
E
e
$ ls -sF|sort -n
0 C
0 D
0 E
0 e
total 22
2 A/
2 B/
2 a/
2 b/
2 f
4 d
8 c
$ ls -lF|sort -n
-rw-r--r-- 1 miscall miscall 0 Jun 16 11:21 C
-rw-r--r-- 1 miscall miscall 0 Jun 16 11:21 D
-rw-r--r-- 1 miscall miscall 0 Jun 16 11:21 E
-rw-r--r-- 1 miscall miscall 0 Jun 16 11:21 e
-rw-r--r-- 1 miscall miscall 260 Jun 16 12:12 f
-rw-r--r-- 1 miscall miscall 1560 Jun 16 12:13 d
-rw-r--r-- 1 miscall miscall 3120 Jun 16 12:14 c
drwxr-xr-x 2 miscall miscall 512 Jun 16 11:21 A/
drwxr-xr-x 2 miscall miscall 512 Jun 16 11:21 B/
drwxr-xr-x 2 miscall miscall 512 Jun 16 11:21 a/
drwxr-xr-x 2 miscall miscall 512 Jun 16 11:21 b/
total 22
$ ls -lF|sort -nr
total 22
drwxr-xr-x 2 miscall miscall 512 Jun 16 11:21 b/
drwxr-xr-x 2 miscall miscall 512 Jun 16 11:21 a/
drwxr-xr-x 2 miscall miscall 512 Jun 16 11:21 B/
drwxr-xr-x 2 miscall miscall 512 Jun 16 11:21 A/
-rw-r--r-- 1 miscall miscall 3120 Jun 16 12:14 c
-rw-r--r-- 1 miscall miscall 1560 Jun 16 12:13 d
-rw-r--r-- 1 miscall miscall 260 Jun 16 12:12 f
-rw-r--r-- 1 miscall miscall 0 Jun 16 11:21 e
-rw-r--r-- 1 miscall miscall 0 Jun 16 11:21 E
-rw-r--r-- 1 miscall miscall 0 Jun 16 11:21 D
-rw-r--r-- 1 miscall miscall 0 Jun 16 11:21 C
$ ls -lF|sort -n|head -5
-rw-r--r-- 1 miscall miscall 0 Jun 16 11:21 C
-rw-r--r-- 1 miscall miscall 0 Jun 16 11:21 D
-rw-r--r-- 1 miscall miscall 0 Jun 16 11:21 E
-rw-r--r-- 1 miscall miscall 0 Jun 16 11:21 e
-rw-r--r-- 1 miscall miscall 260 Jun 16 12:12 f
Number Lines in Files Using cat -n and nl
$ ls -l>test
$ more test
total 347004
-rw-r--r-- 1 miscall miscall 354949120 Apr 22 15:54 a.tar
drwxr-xr-x 2 miscall miscall 4096 Feb 18 10:25 Desktop
drwxrwxrwx 5 miscall miscall 4096 Mar 16 12:56 mkchadata
drwxr-xr-x 2 miscall miscall 4096 Mar 26 18:02 shell
-rw-r--r-- 1 miscall miscall 0 Jun 16 12:22 test
drwxr-xr-x 3 miscall miscall 4096 Mar 9 11:21 test_compare
drwxr-xr-x 2 miscall miscall 4096 Feb 23 10:13 testshell
drwxr-xr-x 8 miscall miscall 4096 Mar 31 15:04 vgop
drwxrwxrwx 5 vgopftp vgop 4096 Mar 14 16:23 vgopdata
$ cat -n test
1 total 347004
2 -rw-r--r-- 1 miscall miscall 354949120 Apr 22 15:54 a.tar
3 drwxr-xr-x 2 miscall miscall 4096 Feb 18 10:25 Desktop
4 drwxrwxrwx 5 miscall miscall 4096 Mar 16 12:56 mkchadata
5 drwxr-xr-x 2 miscall miscall 4096 Mar 26 18:02 shell
6 -rw-r--r-- 1 miscall miscall 0 Jun 16 12:22 test
7 drwxr-xr-x 3 miscall miscall 4096 Mar 9 11:21 test_compare
8 drwxr-xr-x 2 miscall miscall 4096 Feb 23 10:13 testshell
9 drwxr-xr-x 8 miscall miscall 4096 Mar 31 15:04 vgop
10 drwxrwxrwx 5 vgopftp vgop 4096 Mar 14 16:23 vgopdata
$ nl test
$ ls -CF|cat -n
$ ls -CF|nl
Cool nl Tricks and Capabilities
-ba which numbers all lines
-bt which numbers printable text only
-bn which results in no numbering
-bppattern for numbering lines that contain the specified pattern
$ ls -CF ..>test
$ echo "">>test
$ echo "">>test
$ ls -CF ..>>test
$ nl test
1 bin/ games/ include/ lib64/ man/ sbin/ src/
2 etc/ hp/ lib/ libexec/ miscall/ share/ tools/
3 bin/ games/ include/ lib64/ man/ sbin/ src/
4 etc/ hp/ lib/ libexec/ miscall/ share/ tools/
$ nl -ba test
1 bin/ games/ include/ lib64/ man/ sbin/ src/
2 etc/ hp/ lib/ libexec/ miscall/ share/ tools/
3
4
5 bin/ games/ include/ lib64/ man/ sbin/ src/
6 etc/ hp/ lib/ libexec/ miscall/ share/ tools/
$ nl -bpgames test
1 bin/ games/ include/ lib64/ man/ sbin/ src/
etc/ hp/ lib/ libexec/ miscall/ share/ tools/
2 bin/ games/ include/ lib64/ man/ sbin/ src/
etc/ hp/ lib/ libexec/ miscall/ share/ tools/
$ nl -bpgames -snl -bpgames -s#^^# test
1#^^#bin/ games/ include/ lib64/ man/ sbin/ src/
etc/ hp/ lib/ libexec/ miscall/ share/ tools/
2#^^#bin/ games/ include/ lib64/ man/ sbin/ src/
etc/ hp/ lib/ libexec/ miscall/ share/ tools/
$ nl -s' line is: ' test
1 line is: bin/ games/ include/ lib64/ man/ sbin/ src/
2 line is: etc/ hp/ lib/ libexec/ miscall/ share/ tools/
3 line is: bin/ games/ include/ lib64/ man/ sbin/ src/
4 line is: etc/ hp/ lib/ libexec/ miscall/ share/ tools/
file redirection Most UNIX programs expect to read their input from the user (that is, standard input) and write their output to the screen (standard output). By use of file redirection, however, input can come from a previously created file, and output can be saved to a file instead of being displayed on the screen.
filter Filters are a particular type of UNIX program that expects to work either with file redirection or as part of a pipeline. These programs read input from standard input, write output to standard output, and often don’t have any starting arguments.