Bash Scripting Basics: Automate Repetitive Tasks on Linux

A bash script is a text file that holds a sequence of Linux commands so you can run them with one command instead of typing them by hand. Start the file with #!/usr/bin/env bash, write the commands below it, make it executable with chmod +x script.sh and run it with ./script.sh. Anything you do on a server by hand more than two or three times (backups, cleaning up old logs, checking that sites are up, deploying a new release) is a good candidate for a script.

But there's a gap between a script that works and a script you can trust. By default, bash keeps going after an error, silently accepts empty variables and splits file names containing spaces into two. This article covers the basics with a focus on exactly those traps, and ends with a complete script that checks the HTTP status of several sites. It assumes you're familiar with basic Linux commands.

Short answer: A bash script is a text file that runs a sequence of Linux commands so repetitive tasks take a single command. Start it with #!/usr/bin/env bash and set -euo pipefail, make it executable with chmod +x and run it with ./script.sh. Always wrap variables in double quotes, and check the script with shellcheck before relying on it.

The shebang and running a script#

#!/usr/bin/env bash
echo "Hello from $(hostname)"

The first line (the shebang) tells the system which interpreter to run the file with. #!/usr/bin/env bash finds the first bash on your PATH; #!/bin/bash is perfectly fine on Ubuntu too. Just don't write #!/bin/sh if you use bash features: on Ubuntu, sh is actually dash, which doesn't understand things like [[ ]] or arrays.

chmod +x hello.sh
./hello.sh

set -euo pipefail: three lines of insurance#

Start almost every script with this line:

set -euo pipefail

• -e: if a command fails, the script stops right there. Without it, if cd /var/www/app fails, the next command (say, rm -rf storage/cache/*) runs in the wrong directory.
• -u: using an undefined variable is an error. Without it, a typo like $BACKUP_DRI silently becomes an empty string and rm -rf "$BACKUP_DRI/"* becomes rm -rf /*.
• -o pipefail: in a pipe like mysqldump ... | gzip > db.sql.gz, the exit code is by default that of the last command (gzip). So if the dump fails, the script thinks everything went fine. With pipefail, a failure anywhere fails the whole pipe. The backup article shows how critical this one is.

Exceptions to -e you should know#

-e doesn't apply inside an if condition, a while condition, or && and || expressions, because there a failure is part of the logic. It also has a well-known trap:

set -e
count=0
((count++))   # the script silently exits right here

The value of count++ before incrementing is zero, and (( )) returns exit code 1 when the result is zero. Write count=$((count + 1)) instead.

If you deliberately want to ignore a command's failure, say so explicitly: grep -q pattern file || true.

Variables and quoting#

name="app"            # no spaces around =
dir="/var/www/$name"
today=$(date +%F)     # capture command output in a variable
echo "deploying to ${dir}/releases/${today}"

The golden rule: always put variables in double quotes: "$file", not $file. Without quotes, bash splits the value on whitespace and expands * too:

file="my report.pdf"
rm $file      # tries to delete two files: my and report.pdf
rm "$file"    # correct

Single quotes ('...') expand nothing; '$HOME' is the literal text $HOME. For default values use ${VAR:-default}, which also works with set -u.

Arguments#

#!/usr/bin/env bash
set -euo pipefail

if (( $# < 1 )); then
    echo "usage: $0 <domain> [days]" >&2
    exit 2
fi

domain="$1"
days="${2:-30}"
echo "checking $domain for $days days"

With set -u, reading an optional argument directly as $2 when it wasn't passed is an error; hence ${2:-30}.

Conditionals: if and test#

if [[ -f "$config" ]]; then
    echo "config exists"
elif [[ -d "$config" ]]; then
    echo "it is a directory" >&2
    exit 1
else
    echo "no config, using defaults"
fi

In bash, use [[ ]]. It is less error-prone than [ ] (the test command) and supports &&, || and =~ pattern matching. The most common tests:

For numeric comparisons use (( )): if (( size > 1000 )); then.

if actually tests a command's exit code, so this works too: if grep -q "ERROR" app.log; then.

Loops#

# Over a list
for site in example.com example.org; do
    echo "$site"
done

# Over files (no ls; globs handle spaces in names fine)
for f in /var/log/app/*.log; do
    [[ -e "$f" ]] || continue
    gzip "$f"
done

# Until a condition holds
attempt=1
until curl -sf http://localhost:8080/health > /dev/null; do
    (( attempt >= 10 )) && { echo "service did not start" >&2; exit 1; }
    attempt=$((attempt + 1))
    sleep 3
done

[[ -e "$f" ]] || continue handles the case where no file matches the pattern; bash then puts the pattern itself into $f as a literal string.

Reading a file line by line#

while IFS= read -r line || [[ -n "$line" ]]; do
    echo "line: $line"
done < servers.txt

IFS= preserves leading and trailing whitespace, -r stops backslashes from being interpreted, and || [[ -n "$line" ]] reads the last line even if the file doesn't end with a newline. Don't use for line in $(cat file); it splits on whitespace, not on lines.

Functions and exit codes#

backup_dir() {
    local src="$1"
    local dest="$2"
    tar -czf "$dest" -C "$(dirname "$src")" "$(basename "$src")"
}

if backup_dir /var/www/app "/var/backups/app-$(date +%F).tar.gz"; then
    echo "backup ok"
fi

• Declare variables inside functions with local so they don't overwrite variables outside.
• Function arguments are also read with $1, $2.
• return sets the function's exit code (zero means success, 1 to 255 mean failure), not a return value. To return data, echo it and capture it with $(...).
• End the script with exit 0 or exit 1; cron, systemd and CI read that code to decide whether the job succeeded.

For cleanup that runs in every case (success or failure), set a trap:

tmp=$(mktemp -d)
trap 'rm -rf "$tmp"' EXIT

Logging#

A script run by cron has nobody watching its output. A small function is enough:

log() {
    printf '%s  %s\n' "$(date '+%F %T')" "$*" | tee -a "$LOG_FILE"
}
log "starting backup"

Send error messages to stderr (echo "..." >&2) to keep them separate from normal output.

Full example: checking the HTTP status of several sites#

This script reads a list of URLs from a file, gets each one's HTTP status code, logs the result, and exits with code 1 if even one is down:

#!/usr/bin/env bash
# Check the HTTP status of several sites from a list file
set -euo pipefail

readonly SITES_FILE="${1:-/etc/check-sites/sites.txt}"
readonly LOG_FILE="${LOG_FILE:-/var/log/check-sites.log}"
readonly TIMEOUT=10

log() {
    printf '%s  %s\n' "$(date '+%F %T')" "$*" | tee -a "$LOG_FILE"
}

check_site() {
    local url="$1"
    local code
    code=$(curl -s -o /dev/null -w '%{http_code}' --max-time "$TIMEOUT" "$url") || true

    if [[ "$code" =~ ^(2|3)[0-9][0-9]$ ]]; then
        log "OK    $code  $url"
        return 0
    else
        log "FAIL  $code  $url"
        return 1
    fi
}

main() {
    if [[ ! -r "$SITES_FILE" ]]; then
        echo "list file not found: $SITES_FILE" >&2
        exit 2
    fi

    local total=0 failed=0
    while IFS= read -r url || [[ -n "$url" ]]; do
        # Skip blank lines and comments
        if [[ -z "$url" || "$url" == \#* ]]; then
            continue
        fi
        total=$((total + 1))
        if ! check_site "$url"; then
            failed=$((failed + 1))
        fi
    done < "$SITES_FILE"

    log "done: $total checked, $failed failed"
    if (( failed > 0 )); then
        exit 1
    fi
}

main "$@"

The list file, one URL per line:

# Main sites
https://example.com
https://example.com/api/health

Install and run:

sudo install -m 755 check-sites /usr/local/bin/check-sites
sudo mkdir -p /etc/check-sites
check-sites /etc/check-sites/sites.txt
echo "exit code: $?"

A few points about this script:

• curl -w '%{http_code}' prints only the status code, and 000 if the connection fails. Because curl then exits non-zero and set -e is active, || true keeps the script from stopping.
• All the logic lives in main, which is called with main "$@" at the end of the file, so the order in which functions are defined doesn't matter.
• Exit code 2 means "bad input" and 1 means "a site is down," so cron or monitoring can tell them apart.

To run it automatically every five minutes, one crontab line is enough; scheduling details and the pitfalls of the cron environment are in the cron jobs article. If you want notifications, the FAIL branch is the natural place to send an alert. For more serious monitoring, see Linux server monitoring.

shellcheck: an automatic script reviewer#

shellcheck analyzes bash scripts and catches most of the mistakes above before they run: missing quotes, unused variables, cd without error handling and dozens more.

sudo apt install -y shellcheck
shellcheck check-sites

Every warning has a code, such as SC2086 for an unquoted variable, with a full explanation on the shellcheck wiki¹. If you deliberately want to silence a warning, put # shellcheck disable=SC2086 on the line above. Add shellcheck as a CI step too, so no unreviewed script gets into the repository; setting up CI itself is covered in CI/CD with GitHub Actions.

Frequently asked questions#

What is the difference between bash and sh?#

sh is the standard POSIX shell, and on Ubuntu it actually points to dash, which lacks features like [[ ]] and arrays. If your script uses bash features, set the shebang to bash, not /bin/sh.

How do I check if the previous command succeeded in bash?#

The exit code of the last command is in $?: zero means success and anything else means failure. It's usually simpler to put the command directly in an if, such as if grep -q ERROR app.log, because if tests that exit code.

Why does my bash script work manually but not in cron?#

cron's environment differs from your terminal: PATH is more limited, your environment variables aren't loaded and the working directory is different. Use full paths for files and commands or set PATH explicitly, and redirect output to a log file so errors are visible.

Do bash scripts run on macOS and Windows?#

macOS ships with the old bash 3.2, which runs most simple scripts but lacks newer features such as associative arrays. On Windows, the easiest route is WSL, which gives you a real Linux environment.

When should I use Python instead of bash?#

When a script grows past roughly a hundred lines, needs complex data structures, or its logic is more than gluing system commands together. Bash is the simplest tool for combining Linux commands, but large programs in it become hard to read and test.

Wrap-up#

A checklist for every bash script:

• First line #!/usr/bin/env bash, immediately followed by set -euo pipefail.
• Every variable inside "..."; optional arguments via ${2:-default}.
• Validate arguments and print a usage message.
• Put logic in functions with local variables.
• Read files with while IFS= read -r, not for over cat.
• Timestamped logs, errors to stderr, meaningful exit codes.
• Run shellcheck before committing.

Once a script grows past about a hundred lines or needs complex data structures, it's time to consider Python or the project's main language. But for gluing system commands together, bash is still the simplest and most universally available tool.