Useful bash one-liners¶

Replace text in template¶

Replace TEMPLATE_TEXT with $BUILD_NUMBER var inside filename.json.template and save it into filename.json

sed "s/TEMPLATE_TEXT/1.$BUILD_NUMBER/g" \
    filename.json.template > filename.json

A few examples of piecing together commands:¶

It is remarkably helpful sometimes that you can do set intersection, union, and difference of text files via sort/uniq. Suppose a and b are text files that are already uniqued. This is fast, and works on files of arbitrary size, up to many gigabytes. (Sort is not limited by memory, though you may need to use the -T option if /tmp is on a small root partition.) See also the note about LC_ALL above and sort's -u option (left out for clarity below).
```
sort a b | uniq > c   # c is a union b
sort a b | uniq -d > c   # c is a intersect b
sort a b b | uniq -u > c   # c is set difference a - b
```
Pretty-print two JSON files, normalizing their syntax, then coloring and paginating the result:
```
diff <(jq --sort-keys . < file1.json) <(jq --sort-keys . < file2.json) | colordiff | less -R
```
Use grep . * to quickly examine the contents of all files in a directory (so each line is paired with the filename), or head -100 * (so each file has a heading). This can be useful for directories filled with config settings like those in /sys, /proc, /etc.
Summing all numbers in the third column of a text file (this is probably 3X faster and 3X less code than equivalent Python):
```
awk '{ x += $3 } END { print x }' myfile
```
To see sizes/dates on a tree of files, this is like a recursive ls -l but is easier to read than ls -lR:
```
find . -type f -ls
```
Say you have a text file, like a web server log, and a certain value that appears on some lines, such as an acct_id parameter that is present in the URL. If you want a tally of how many requests for each acct_id:
```
egrep -o 'acct_id=[0-9]+' access.log | cut -d= -f2 | sort | uniq -c | sort -rn
```
To continuously monitor changes, use watch, e.g. check changes to files in a directory with watch -d -n 2 'ls -rtlh | tail' or to network settings while troubleshooting your wifi settings with watch -d -n 2 ifconfig.

Run this function to get a random tip from this document (parses Markdown and extracts an item):

function taocl() {
curl -s https://raw.githubusercontent.com/jlevy/the-art-of-command-line/master/README.md |
    sed '/cowsay[.]png/d' |
    pandoc -f markdown -t html |
    xmlstarlet fo --html --dropdtd |
    xmlstarlet sel -t -v "(html/body/ul/li[count(p)>0])[$RANDOM mod last()+1]" |
    xmlstarlet unesc | fmt -80 | iconv -t US
}

Useful bash one-liners¶

Replace text in template¶

A few examples of piecing together commands:¶

Comments