1. for beginners 2. getting into the details 3. bash scripts

1. for beginners

On Macs, launch Bash with Terminal, located in the Applications/Utilities folder.

Some easy one-liners to begin with

Use Ctrl-C to quit current command

bash$ echo Hello world!
Hello world!
bash$ look zoon #print words starting with 'zoon'
zoon
zoonal
zoonerythrin
zoonic
 ...
bash$ tr a-z A-Z #turns input to upper case
hello
HELLO
bash$ rev #writes input backwards
hola
aloh
bash$ echo a* #print names of all files in current directory starting with 'a'

To make a quick text file:

bash$ cat > quick.txt
this is 
my quick file 
#press ^D (Ctrl-D) to end text entry
bash$ cat quick.txt
this is
my quick file

Basic shell navigation, commands, features

pwd = print working directory (your current location)
cd directory = move into that directory
ls = list contents of current directory
echo = print something onscreen- a string, variable, filename etc etc
cat myfile = display contents of a text file
open . = open current directory in Finder
Ctrl-A = go to start of line
Ctrl-E = go to end of line
Completion: Press [TAB] while typing a command/filename to auto-complete it; it beeps if ambiguous – press [TAB] again and the alternatives are shown.
Drag n drop: For commands using/requiring files/folders with their full path, drag and drop the file/folder onto the command line. e.g. type "cd " then drag a folder from Finder to make that the current directory.
Permissions: If you aren’t permitted to execute a command, put sudo (super-user do) at the start of the command, which will ask for the admin password before continuing.
Inspector: [CMD]-I = launch the Terminal Inspector to change settings for shell fonts, colours, cursor, etc.

Help!

Type help, or to see all available commands, hold ESC until prompted, then answer y.
man command = display manual pages on a command.
Multi-page results: Navigate the manual pages, the output of less etc a page at a time with : SPACE = see next page, b = back one page, q = quit back to terminal prompt, /string = search – highlights occurrences of string. Cycle forwards through the highlighted strings with n, or backwards with N.
info bash = many pages of info about bash. [SPACE] = next page, B = back, U = Up a level, N = Next, P = Prev. Move cursor to any section title and press ENTER to go to that section.
man bash = exhaustive bash documentation
man man = display more info on the man command
Help: help command. Some commands also have short help messages, use command -h
help * = help on all builtin commands – very useful to read!
help = nice 1-page summary of builtin commands + options
apropos blang or man -k blang = search manual pages for blang.
type blang = searches the environment for matching commands & displays them. use option -a to display all matches, not just the first.
locate blang = find a file or command

Controlling input/output

Pipes: A concept central to Unix. Input can be taken from, and output to, files or other commands, with pipelines |, which make the output of one command the input of the next. E.g. cat * | wc -l counts the total number of lines in all files in current directory – this takes the output of cat and feeds it to wc. The idea is to use a range of simple tools together, rather than try to do everything with one program.
Less: Use | less at the end of a command line to display results one page at a time. Or less myfile to display a text file. q=quit, SPACE=next page, ENTER=next line, b=prev page, /=search mode: Type /string to find string in the less output, and n to cycle forward, N to cycle back, through occurrences of string.

bash$ a=5;echo "$a" '$a'
5 $a

Escaping: use backslash \ before a special character to use it literally, e.g. before a space in a file/folder name.

File/directory commands

df -hl = displays space used/available on hard disk and all available drives
open filename = open a file in its default application
file myfile.sxj = displays info on what type of file it is

mkdir – create a new folder/directory
cp – copy files/folders
mv – move/rename files/folders
rm – remove (delete) files/folders. NB Be careful with this!

Shell pattern matching (Globbing)

Shell pattern matching occurs with: filename globbing (pathname expansion), [[ = and != ]], case statements, help, string parameter operators (e.g. ${var#word}, etc. Regular expressions are only used within bash by [[ string =~ regex ]].
● ALWAYS use quotes around arguments to simple commands that could be interpreted as globs.

echo ? = print list of all one-letter files in current directory
echo ????? = all files whose name has 5 letters
ls * = list contents of directories (and subdirectories)
echo * = print names of files and directories

Image commands

Internet commands

ifconfig = display network information

host URL/IPaddress = look up IP address/host name e.g. host blang.com
Use host -a blang.com to display more information.
whois blang.com = displays info about who owns the domain name blang.com
wget – download multiple web pages/files
wget -r -A .pdf http://url-to-webpage-with-pdfs/ = download all pdfs from a website.

Other commands

date

bash$ date "+%a %-d %b %Y"
Wed 1 Mar 2017
bash$ date +%-d/%-m/%y     # day, month, year
1/3/17

rs – reshape a data array

bash$ jot -r 25 | rs 5 #print 5x5 array of random integers between 0-100
50  75  95  99  97
5   49  55  86  46
79  86  79  2   14
45  42  75  98  19
12  32  33  78  93
bash$ jot -s ''  -c 26 z a
zyxwvutsrqponmlkjihgfedcba

csplit – split a file into multiple files, based on regexes, number of lines etc.

2. getting into the details

More file/directory commands

stat /path/filename = displays info about the file
strings filename = displays text strings from a binary file
xxd filename = hex dump of a file
stat -x myfile = display important information about a file
du -h = size of current directory/files, in human-readable units. (disk usage)
find * -size +5000000c = show files larger than 5,000,000 bytes in the current directory & subdirectories (in newer bash versions, -size +5M works)
ls -l | awk '$5/1000000>5' = show info for files larger than 5MB
od -c – display files/stream with backslash-style escape characters.

bash$ echo "Hi there! How's things?" | od -c
0000000    H   i       t   h   e   r   e   !       H   o   w   '   s    
0000020    t   h   i   n   g   s   ?  \n                                
0000030

More shell navigation commands

Double (left) mouse click = select word
Triple (left) mouse click = select line

bash$ echo t u v w x | sed 's/ /        /g'
t       u       v       w       x

There are many more, and they’re customizable. See GNU bash manual
See a list of your current shell key bindings with:

bind -p | grep -Ev 'not bound|self-|do-' | sed 's/"//g' | less

More shell history commands

bash$ ecco hello
-bash: ecco: command not found
bash$ ^cc^ch
echo hello
hello

Operations on text/text files

echo -e = enable backslash-escaped characters like \n and \t. e.g.:

echo -e Hi "\t" There "\n\n\n"

diff -y file1 file2 = displays all differences between 2 similar files, -y means ‘format in 2 columns’. Also can be used to display the differences between 2 similar directories.
vimdiff file1 file2 [file3] = colourful visual display of the differences between 2 or 3 text files.

bash$ echo '12ag 25r4 qw 6z' | grep -o [0-9] | wc -l
6

bash$ grep ^[^aeiou]*a[^aeiou]*e[^aeiou]*i[^aeiou]*o[^aeiou]*u[^aeiou]*$ /usr/share/dict/words |
> tr "\n" " "     # Find words containing all the vowels in order
abstemious abstemiously abstentious acheilous acheirous acleistous affectious annelidous arsenious arterious bacterious caesious facetious facetiously fracedinous majestious

grep $(printf "%s" {^,a,e,i,o,u}"[^aeiou]*")$ /usr/share/dict/words |
> tr "\n" " " #or:
c=[^aeiou];grep ^$c*a$c*e$c*i$c*o$c*u$c*$  /usr/share/dict/words | tr "\n" " "

sed 's/this//g' #delete all occurrences of 'this'
sed 's/this/that/g' < myfile.txt #replaces all occurrences of 'this' with 'that'

paste – pastes columns of text together, side-by-side

bash$ echo "1 2 3 4 5 6
> 7 8 9 10
> 11 12" > example.txt
bash$ cat example.txt | xargs
1 2 3 4 5 6 7 8 9 10 11 12
bash$ cat example.txt | xargs -n 3
1 2 3
4 5 6
7 8 9
10 11 12
bash$ echo "splitXsplitXsplitXsplit" | xargs -d X 
split split split split
bash$ echo "splitXsplitXsplitXsplit" | xargs -d X -n 2 
split split 
split split 
bash$ cat example.txt | xargs -n 1 -I {} echo -n "{}, "
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 

bash$ tr ' ' '\t' < example.txt       #Convert spaces to tabs
1        2        3        4        5       6
7        8        9        10
11        12

bash$ echo 'Lots       of    spaces' | tr -s ' '
Lots of spaces

bash$ cat example.txt | tr -d 1
 2 3 4 5 6
7 8 9 0
 2

bash$ echo hello 1 char 2 next 4 | tr -dc '0-9 \n' 
 1  2  4
bash$ echo 1456243526545342 | tr 123456 EADGBE
 EGBEAGDBAEBGBDGA
bash$ echo 'This is ROT-13 encrypted.' | tr 'A-Za-z' 'N-ZA-Mn-za-m'
Guvf vf EBG-13 rapelcgrq.

awk '{print $2, $4}' myfile # print the 2nd and 4th word from each line.
awk 'length < 60' myfile # print all lines shorter than 60 characters.
awk 'NR>=3{print} NR==7{exit}' myfile.txt # print from the 3rd to the 7th line.

Note on quoting

bash$ echo 'Don\'t quote me' # This doesn't work.

Bruce Barnett says “I bet many of you programmers are confused by this. All of us are very familiar with strings in programming languages like C. This is why we get confused. [In bash] the quotes turn substitution on and off. They are not used to indicate the starting and ending of a string.”

bash$ echo 'Don'\''t do that' # This works.

e.g. shell script printcol:

#!/bin/bash
# printcol
# e.g. printcol 3
awk '{print $'$1'}' 

Notice how the 1st command-line argument $1 isn’t single-quoted, but is expanded by the shell, to become in the awk program $n, meaning in awkese the nth word/field on the input line.

More on redirecting input/output

<< label = Here-document. Accepts multi-line input from a file or keyboard, which starts and ends with an arbitrary string label.
● Use brackets to redirect the output from a group of commands. This actually runs the commands in a subshell. e.g. (pwd; ls; cd ../; pwd; ls) > outputfile
< <( ) = process substitution e.g. for reading lines from output of commands
e.g. To make the shell scroll down, with command line at top:

(while read line;do tput home;tput ri;echo "$line";done;tput home;tput ri)< <(ls -l)

echo “$(<myfile)” works like cat myfile.

Other commands

defaults write com.apple.finder AppleShowAllFiles TRUE ; killall Finder = show hidden files in Finder
defaults write com.apple.finder AppleShowAllFiles FALSE ; killall Finder = don’t show hidden files in Finder (the default)
export PS1="\w" = changes the prompt to the full path of the current directory.
++\d = current date, \t = current time, \h = host name, \# = command number, \u = user name, \W = current (working) directory
last -l $USER = displays how long you’ve been logged in
cal = displays calendar for the current month
cal 1967 = displays calendar for 1967.
echo $HOME = display the value of HOME

Doing maths:

bash$ var1=$(echo "scale=4; 3.44 / 5" | bc);echo The answer is $var1
.6880

Let is the old way, (( )) the new.

let Y=(X+2)*10 X=Y+1 # no spaces allowed within 'words'
Y=$(( ( X +2 ) * 10, X = Y+1 )) # any spacing is allowed
bash$ g=5
bash$ ((g+=2));echo $g
7
bash$ ((g++));echo $g
8

If the value of the expression is non-zero, the return status is 0; otherwise the return status is 1:

bash$ ((1));echo $?
0
bash$ ((0));echo $?
1

Or use declare:

bash$ declare -i n #declare the variable n as an integer type
bash$ n=6/4; echo $n
1

Two ways of doing floating point maths easily on the bash command line
Put these functions in your .bash_profile:

calc() { awk "BEGIN{print $*}"; } # awk calc
pc() #python calc
{ python -c "print($*)"; }
#then:
bash$ calc 2.4^3.1
15.0888
bash$ pc 2.4**3.1
15.0888051157

Dozens more ways here.

Variables

set = display all variables (system or user-defined) and their values

bash$ echo -n "Enter name: " ; read name ; echo Name is $name
Enter name: Snurgel
Name is Snurgel

bash$ read message < quick.txt;echo $message
this is

bash$ while read message; do echo $message;done < quick.txt
this is
my quick file

bash$ d=$(date) #old style: d=`date`

bash$ read etad < <(date | rev) ; echo $etad
7102 TSE 33:52:61 5  beF nuS

bash$ read a b c <<< $(echo "d e f" | awk '{ print $3, $2, $1 }')
bash$ echo $a $b $c
f e d

ls -l | while read a b c d; do echo owner is $c; done

typeset = declare local variables in a function
typeset -i x = declare local integer variable x
Setting variables with typeset makes them much faster. [test this]
declare -p = prints all parameters, with appropriate set instructions
declare -l mystring = string automatically converted to lower case
declare -u mystring = string automatically converted to upper case
declare -r myvar = creates read-only variable
declare -a myarray = indexed array (indexed by integers)
declare -A myarray = associative array (indexed by strings)

Environment variables

bash$ set one two three four # set some fake positional parameters
bash$ echo ${!#} # get the last ('nth', not 'previous') argument
four

(# stores the number of arguments, so !# will reference the last argument)

for v in $(compgen -v);do echo $v=${!v};done # Print environment variables

String variable operators

bash$ string=abcdefghijklmnopqrstuvwxyz 
bash$ echo ${#string} = print the length of the variable/string
26
bash$ echo ${string:4} = substring from character 4 to end of string
efghijklmnopqrstuvwxyz 
bash$ echo ${string:4:8} = substring from char 4 with length 8 
efghijkl 
bash$ echo ${string:(-1)} = substring from last character to the end 
z 
bash$ echo ${string:(-2):2} = substring from 2nd last char, length 2. 
yz 
bash$ var="This is a line of text" 
bash$ echo ${var/line/REPLACED}    #Replacing some text in a variable
This is a REPLACED of text

bash$ f=/home/my/path/thing.txt
bash$ echo ${f#*.}   # get extension - remove pattern from beginning of string
txt

bash$ n=${f##*/};echo "$n"   # Get filename
thing.txt

bash$ echo ${n%.*}   #remove filename extension
thing
bash$ echo ${f%/*}     #Get directory path
/home/my/path/
for f in *.foo; do mv $f ${f%foo}bar; done # Rename *.foo to *.bar

● These string operations using the ${name … } notation can also be applied to all string elements in an array, with the ${name[@] …} or ${name[*] …} notation.
Indirection
${!param} = the referenced parameter is not param itself, but the parameter whose name is stored as the value of param, e.g.

bash$ read -rep 'Which variable to inspect? ' v; \
> printf 'The value of "%s" is: "%s"\n' "$v" "${!v}" 

Arrays

bash$ a[6]=iurhi  # set an element
bash$ a[blah]=4  # associative arrays are allowed, i.e. the index can be a string
bash$ a=(z 1 2 3 haha hi "&^%")   # set an array. NB the first element is a[0].
bash$ echo ${a[@]}   # expands to all array elements.
z 1 2 3 haha hi &^%
bash$ echo ${a[6]} 
&^%
bash$ echo ${#a[@]}     # number of elements
7
bash$ echo ${a[@]:2:3}   # slicing - 3 elements starting at a[2]
2 3 haha
bash$ echo ${#a[4]}   # gives the length of a[4], i.e."haha"
4
bash$ echo "${a[@]%a}"   # substring removal - remove trailing "a"
z 1 2 3 hah hi &^%
bash$ echo "${a[@]#h}"   # remove leading "h"
z 1 2 3 aha i &^%
bash$ echo "${a[@]/h/j}"   # replace first "h" in each word with "j"
z 1 2 3 jaha ji &^%
bash$ echo "${a[@]//h/j}"   # replace all occurrences
z 1 2 3 jaja ji &^%

bash$ c=(0 1 1 1 0 1 1 1) #set array
bash$ q=([1]=2 [4]=7 [5]=8) # another way
bash$ echo ${#q[@]}
3
bash$ echo ${q[@]}
2 7 8
bash$ echo ${!q[@]} # ${!name[@]} expands to the array indices
1 4 5
bash$ files=(*) # make array of names of files in current dir
bash$ c=(${b[@]}) #copy one array into another

myarray+=(newmember) = add to array
array=( “${array[@]// /_}” ) = Recreate array with spaces in elements as underscores
array=( “${array[@]:2:3}” ) = Recreate array only with elements from index 2 to 4

Brace expansion

bash$ echo test.{txt,jpg,png}
test.txt test.jpg test.png

mv info{,.old} = mv info info.old
for i in {1..$n} = WRONG – the Bash parser performs brace expansion before any other expansions or substitutions. Use a C-style for loop.

bash$ echo {1..20..2} #now has inc, since bash 4.0 (2009)
1 3 5 7 9 11 13 15 17 19
bash$ echo {009..012} # also leading zeros
009 010 011 012

● Single quotes inhibit all expansions—the characters enclosed by the quotes pass through the expansions unscathed.
● Double quotes permit some expansions and inhibit others. Word expansion and command, arithmetic, and process substitution take place—the double quotes only affect how the result is handled—but brace and tilde expansion do not.

Compound statements/flow control

Test/If-then statements

Use [[ to compare strings, files, (( for integers. (The only benefit of [ is compatibility with some other and ancient shells.)

Integer/integer variable tests:
if [[ n1 –test n2 ]]; then do_if_true; [else do_if_false;] fi
e.g. if [[ $v -eq 5 ]]; then echo Yup; else echo Nope; fi
NB The spaces inside the square brackets are necessary.
-eq ++-ge ++-gt ++-le ++-lt ++-ne

String tests: =++ !=++ <++ >
[[ -n str1 ]] – checks if str1 has length > 0
[[ -z str1 ]] – checks if str1 has length 0.
NB The and symbols must be escaped (e.g. \>), or the shell uses them as redirection symbols, with the string values as filenames. Also, what is considered greater or less than is different from the order used by sort.

For testing commands use if; for numerical tests use (( )).
Numeric: val++/–– = post-increment/-decrement, ++/––val = pre-increment/-decrement, ! = logical NOT, ~ = bitwise NOT, ** = exponentiation, <</>> = bitwise shift, & = bitwise AND, | = bitwise OR

if

The form is : if list; then list; [ elif list; then list; ] … [ else list; ] fi
Unlike the usual programming if-then, these are lists of commands, and the list in if list then takes the return value of the last command in list, e.g.

bash$ if ((x = 4 , x += 5 , x -= 3)); echo $x, 5;((x > 5 ))
> then echo Greater
> else echo Not greater
> fi
6, 5
Greater

for .. in

for w in $(<myfile.txt) = loops over words
for f in *.txt = loops over all .txt files in current dir

bash$ for letter in a b c; do echo -n ..$letter; done
..a..b..c

If ‘in list’ is not present, the commands are executed once for each positional parameter, as if ‘in “$@”’ had been specified.

bash$ sum() { g=0;for f;do ((g=g+f));done;echo $g; }
bash$ sum 1 2 3
6

Using a command to generate a list:

for f in $(find . -name *.txt) # loops over all .txt files in dir and subdirs

for (( ; ; )) (C-style for loop)

bash$ for ((i=0;i<10;i++)); do echo -n $i", "; done
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 

More complex for loop:

for (( i=0, j=0 ; i+j < 10 ; i++, j++ )) 
do 
    echo $((i*j)) 
done 

Floating point loop:

for fp in $(seq 1.0 .01 1.1) 
do 
    echo $fp 
done 

Or:

seq 1.0 .01 1.1 | \
while read fp 
do
    echo $fp 
done 

while

Do stuff as long as test-commands has a 0 exit status.

until

Do stuff as long as test-commands has a non-0 exit status.

case

You can put more than one pattern in a case using |.

The patterns are file glob type patterns, not regexes. e.g. * always matches.

Aliases & functions

● Functions, unlike aliases, can take arguments.
● Putting alias definitions in single quotes means variables won’t be evaluated until the command is run. Inside double quotes, they’re evaluated immediately, and never change after that – which is probably not what you want.
type blang = print info on an alias/function/builtin
builtin blang = ignore shell functions and aliases to run the actual built-in command
command blang = ignore shell functions and aliases to run the actual external command
\blang = prevent alias expansion
declare -f = display all function definitions
declare -F = display all functions, name only
declare or set = display all shell parameters, aliases and functions

Sample functions

rep () # repeat - looper from Brian Fox
# e.g. rep 10 echo hello
{ 
    local count="$1" i;
    shift;
    for i in $(seq 1 "$count");
    do
        eval "$@";
    done
}

Int to char (via hex)

bash$ chr() { echo -ne "\x$(printf "%x" $1)"; }
bash$ chr 65
A

flocate – A useful function that locates files by filename. (locate is fast but often returns hundreds of entries with the search string in the path. Flocate searches only on the filename.)

flocate() { locate "*/$1"; }

words – prints 1 argument on each line – note that { } isn’t needed.

bash$ words()
> for word
> do
> echo "$word"
> done
bash$ words of love
of
love
bash$ declare -f words
words () 
{ 
    for word in "$@";
    do
        echo "$word";
    done
}

hr () # print a horizontal line the width of the window
{ 
    T=''
    read r r < <(stty size)
    while ((r-->0))
    do
        T=$T-
    done
    echo $T
}

d – used when downloading a lot of files from online links. Copy the first file link to clipboard, then run function. Return to browser, copy more links one at a time, and the files will be automatically downloaded each time the clipboard changes.

d() { 
c=$(pbpaste)
curl -kLO $c
while True
do
	sleep 3
	f=$(pbpaste)
	[[ $c = $f ]] || curl -kLO $f
	c=$f
done
}

Functions can have redirection on their first line, which then always applies, e.g. myfunc() { } < tmp

Processes & job control

kill 123 = kill process number 123 (stop it instantly)
killall Blang = kill the application Blang
top = display running processes
bg = put job in background
jobs = list background jobs
fg = end background job in foreground
use nohup before a command to keep running it after the shell is closed: nohup command &

Exit Codes & Control Operators

Every command returns an exit code on completion, 0 if successful, some other number 1-255 if not, depending on the command.

Displaying text in colour

I have this in my bash profile, so I can always use these commands:

alias red='tput setaf 1'
alias green='tput setaf 2'
alias yellow='tput setaf 3'
alias blue='tput setaf 4'
alias magenta='tput setaf 5'
alias cyan='tput setaf 6'
alias white='tput setaf 7'
alias bred='tput setab 1'
alias bgreen='tput setab 2'
alias byellow='tput setab 3'
alias bblue='tput setab 4'
alias bmagenta='tput setab 5'
alias bcyan='tput setab 6'
alias bwhite='tput setab 7'
alias normal='tput sgr0' # back to normal text

Put this in scripts to return altered colours/fonts to normal in the event of a Ctrl-C quit:

trap control_c SIGINT
control_c() {
    tput sgr0
    exit 3 #for c=3...
}

Timing commands

time for ((i=0;i<5000;i++));do result=$(($i%2)); done = get times for 5000 loops, on my machine:
expression++++++++++++time
result=$(($i%2))++++++0.249s
result=`expr "$i%2"`+24.389s
let result=$i%2+++++++0.216s
result=$((i%2))+++++++0.141s
result=$[i%2]++++++++0.156s

Bash scripts

Use: #!/usr/bin/env bash as the shebang in all bash scripts. This finds the bash in the $PATH.
(#!/bin/sh or #!/bin/bash have been more common. #!/bin/sh/ is fine if bash isn’t needed, if bash features aren’t used.)
bash filename = load a shell script/program from a text file in the current directory, and run it in a new subshell.
Run a user-written script like a shell command (just by typing its name) by adding its directory to the command search path, and giving it execute permission with chmod +x scriptname. OR add to the .bash_profile file as a function – e.g. myfunc() { blahbla; blah; }
.bash_profile is read when Bash is invoked as a login shell.
.bashrc is executed every time a new shell starts.
aliases and functions should be in .bashrc, otherwise they’re not available to subshells. But DONT put things that will grow in .bashrc, e.g. PATH=$PATH:blah/blah/blah would be expanded with each new shell.
Running/executing vs sourcing a script
source script.sh or . script doesn’t start a new process, so variables can be shared e.g.

bash$ echo x=22 > setx.sh
bash$ chmod +x setx.sh
bash$ setx.sh
bash$ echo $x

bash$ source ./setx.sh # same as . ./setx.sh
bash$ echo $x
22

set = list all shell variables
env = list all exported variables, available to subshells – not many!
export -f myfunc = exports a function so subshells can see it
exit = exits current shell
Debugging
set -v = switch on verbose mode++++set -x = switch on echo mode
set +v = switch off++++++++++++++++set +x = switch off
Make xtrace (-x) more useful with source file + line number

export PS4='+(${BASH_SOURCE}:${LINENO}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }'

Hmm this seems to give weird line numbers sometimes…
Dealing with errors, interrupts, exiting
Use || for error messages:
command || echo “command didnt work!”
trap -l = list trappable signals
To do stuff in a script after a Ctrl-c, before quitting:

trap 'echo INT signal received;exit' INT

${:?} – use for parameter errors, e.g.

DIR=${1:?"Error. Must give a directory."}
SRC=${2:?"Error. Must give a source file."}
GV=${3:?"Error. Must give a giblet vendor."}

Some sample scripts

Download something, if interrupted keep resuming until download is completed
I added this to my .bash_profile:

dl () # usage: dl myfilename "fileURL"
{ 
    until curl -kLC - -o "$1" "$2"; do
        :;
    done
}

Print the number of files with each extension, sorted

for i in *.*;do echo ${i#*.};done | awk '{a[$0]+=1} END{print NR" files :";
for (j in a) print a[j]"\t"j}' | sort -n -r

or this, which also gets the files with no extension:

ls -1 | awk '{split($0,e,".");a[e[2]]+=1} END{print NR" files :";
for (j in a) print a[j]"\t"j}' | sort -n -r

Strip html from a file
sed 's|<[^>]*>||g' myfile = remove the pattern “<, then 1 or more occurrences of any character but >, then >” anywhere that pattern’s found. Can’t just use <*> because it will always just find the last > in the line, not the next.

Reverse the numbering on sequentially numbered files

num_of_files=1034;for ((f=$num_of_files;f>-1;f--));do
cp /my/path/0$f.png /my/path2/0$(($num_of_files-f)).png; done

This copies images numbered 00.png – 01034.png so that /my/path/00.png becomes /my/path2/01034.png, 01 becomes 01033 etc etc.

Replace text in all files in a directory

find . -name *.cpp -print0 |  xargs -I{} -0 sed -i 's/Copyright/Copyleft/g' {}

Convert programs/scripts to HTML-safe and QuickLatex-safe for pasting online
It takes the contents of the clipboard and transforms < into &lt;, > into &gt; and $ into \$.
pbpaste | sed -e 's/</\&lt;/g' -e 's/>/\&gt;/g' -e 's/\$/\\$/g' | pbcopy

(This script doesn’t work applied to itself. I was going to paste here the HTML needed to produce that line on the page, but the HTML needed to display that properly would be much longer again… etc. Very like something out of Hofstadter’s Gödel, Escher, Bach, which features such things as a record that can’t be played by a certain record player, in illustration of Gödel’s ideas.)

Scrolling ad banner

a='SHEEP SUITS 50% OFF!! BAAGAIN!! ';while 
    printf '\r%s' "${a:i}${a::i++}";do 
    ((i>${#a})) && i=0;sleep 0.1;done

Find out the frequency of words in a file

#!/bin/bash 
# word_freq.sh 
if [ $# -ne 1 ]; 
then 
    echo "Usage: $0 filename"; #see how the pros do it
    exit -1 
fi 
filename=$1 
egrep -o "\b[[:alpha:]]+\b" "$filename" | awk '{ count[$0]++ } 
END{ printf("%-14s%s\n","Word","Count") ; 
for(ind in count) 
    {  printf("%-14s%d\n",ind,count[ind]);  } 
}'

then call the script perhaps with: bash word_freq.sh myfile.txt | sort -n +1 | less

Function to convert downloaded youtube mp4 video to audio only:

vtoa () { ffmpeg -i "$1" -vn -acodec copy "${1%.*}".aac; }

I like having scripts operate silently on the contents of the clipboard.
Convert SPACEs in clipboard to underscores, to make filenames more unix-friendly:

uscore() { pbpaste | tr " " _ | pbcopy; }

Convert newlines to spaces – useful to turn a copied paragraph of text with unwanted newlines into one long line:

oneline() { pbpaste | tr '\n' ' ' | pbcopy; } # OR :
oneline() { pbpaste | awk 1 ORS=' ' | pbcopy; }

Thue-Morse function

bash$ xnotx() { echo -n "$* ";for a in $*;do echo -n $((1-a))" ";done;echo; }
bash$ tm() { s=1; for ((c=1;c<=$1;c++)); do
> s=$(xnotx $s); done; echo $s | xargs -n 64 | tr -d ' ' | tr 1 ' '; }
bash$ tm 10
 00 0  00  0 00 0  0 00  00 0  00  0 00  00 0  0 00 0  00  0 00 
0  0 00  00 0  0 00 0  00  0 00  00 0  00  0 00 0  0 00  00 0  0
0  0 00  00 0  0 00 0  00  0 00  00 0  00  0 00 0  0 00  00 0  0
 00 0  00  0 00 0  0 00  00 0  00  0 00  00 0  0 00 0  00  0 00 
0  0 00  00 0  0 00 0  00  0 00  00 0  00  0 00 0  0 00  00 0  0
 00 0  00  0 00 0  0 00  00 0  00  0 00  00 0  0 00 0  00  0 00 
 00 0  00  0 00 0  0 00  00 0  00  0 00  00 0  0 00 0  00  0 00 
0  0 00  00 0  0 00 0  00  0 00  00 0  00  0 00 0  0 00  00 0  0
0  0 00  00 0  0 00 0  00  0 00  00 0  00  0 00 0  0 00  00 0  0
 00 0  00  0 00 0  0 00  00 0  00  0 00  00 0  0 00 0  00  0 00 
 00 0  00  0 00 0  0 00  00 0  00  0 00  00 0  0 00 0  00  0 00 
0  0 00  00 0  0 00 0  00  0 00  00 0  00  0 00 0  0 00  00 0  0
 00 0  00  0 00 0  0 00  00 0  00  0 00  00 0  0 00 0  00  0 00 
0  0 00  00 0  0 00 0  00  0 00  00 0  00  0 00 0  0 00  00 0  0
0  0 00  00 0  0 00 0  00  0 00  00 0  00  0 00 0  0 00  00 0  0
 00 0  00  0 00 0  0 00  00 0  00  0 00  00 0  0 00 0  00  0 00 

Customizing your bash prompt

Example 1: Put in .bash_profile :

export PS1='\![$?]\T \W$ '

i.e. history number – return status – time – current directory
Example 2: I put in .bash_profile :

PROMPT_COMMAND=prompt
prompt () 
{ 
    s=$?;
    PS1='$(bold;yellow;hr;reset)';
    if ((SHLVL>1)); then
        prstring='';
        for ((i=1; i<SHLVL; i++))
        do
            prstring+="+";
        done;
        PS1+="$(bold;cyan;echo -n $prstring;reset)";
    fi;
    if [[ s -eq '0' ]]; then
        PS1+="\!$(green;echo -n [0];reset)";
    else
        PS1+="\!$(bold;red;echo -n [$s];reset)";
    fi;
    PS1+="\T \W$ "
}

Which looks like:

i.e. yellow line (much easier to find when scrolling back up to prev command) – shell level – history num – red/green return status – time – current directory.
Prompt string format codes

Miscellaneous

stty size = returns height & width of current shell window

Try disabling Unicode support; bash may run 2x faster: export LC_ALL=C

See also

Books

WWW

osxdaily Terminal tips Add colour to the terminal
Nate Landau: shell scripting utilities my bash profile shell script template

Doug McIlroy thinks up pipes in 1964.

Output of tree, which I adapted from a program on Mendel Cooper’s site (I think). This version shows files, not just directories, and is in colour:

#!/bin/bash
# tree.sh
NUM_OF_COLOURS=7
trap control_c SIGINT
control_c() {
    tput sgr0
    echo
    exit 0
}
search () {
for dir in *; do
    if [[ -d "$dir" ]] ; then # ==> If it is a directory (-d)...
        declare -i zz=0 # ==> keeps track of directory level.
        tput setab 0 #black background
        while ((zz!=$1)); do     # Keep track of inner nested loop.
            tput setaf $((zz++ % $NUM_OF_COLOURS +1))
            echo -n "| "        # ==> Display vertical connector symbol,
        done
        if [[ -L "$dir" ]] ; then # ==> If directory is a symbolic link...
            g=$(ls -l "$dir")
            tput setaf $((zz % $NUM_OF_COLOURS+1))
            echo -n "L__> $dir "
            tput setab 2
            tput setaf 0 #green
            echo -n "${g#*$dir} " #just show the link at end of ls -l line.
            tput setab 0
            echo
        else #so it's a normal directory
            tput setaf $((zz % $NUM_OF_COLOURS+1))
            echo -n "+==="
            tput setaf 0
            tput setab $((zz % $NUM_OF_COLOURS+1))  
            echo -n " $dir "
            tput setab 0
            echo
            ((numdirs+=1)) #==> Increment directory count.
            if cd "$dir" ; then         # ==> If can move to subdirectory...
                search $(( $1 + 1))      # ==> Function calls itself.
                cd .. # return up one level
            fi
        fi
#delete these lines before the FI to show just directories.
    else
        if [[ -f $dir ]] ; then # continue; fi
            for ((j=0;j<$1;j++)); do
                tput setaf $((j % $NUM_OF_COLOURS +1))
                echo -n "| "
            done
            tput setaf $(($1 % $NUM_OF_COLOURS +1))
            echo -n 'L__ '
            tput sgr0 #set normal - all text printed in bold except fnames
            tput setaf $(($1 % $NUM_OF_COLOURS +1))
            echo "$dir" 
            tput bold #set back to bold  #i.e. delete up to HERE.
        fi
    fi
done
}

if [[ $# != 0 ]] ; then
    cd $1   # Move to indicated directory.
  #else   # stay in current directory
fi
tput bold
echo 'Coloured Tree v1.'
echo 'Ctrl-S = pause. SPACE = continue. Ctrl-C = quit'
echo "Initial directory = $PWD"
numdirs=0
search 0
tput sgr0 #=back to normal, i think
echo "Total directories = $numdirs"

exit 0