Linux Command Reference

time

Synopsis

time [options] command [arguments]
/usr/bin/time [options] command [arguments]

Description

The time command measures and reports how long a command takes to execute. It provides three key timing measurements: real time (wall-clock time from start to finish), user time (CPU time spent in user mode), and system time (CPU time spent in kernel mode). These metrics are essential for performance analysis, benchmarking, and optimization.

There are actually two versions of time: the shell builtin (available in bash, zsh, etc.) and the standalone /usr/bin/time command. The shell builtin is simpler and faster, providing basic timing. The standalone version (GNU time) offers extensive formatting options, memory statistics, I/O metrics, and detailed resource usage reporting. Understanding which version you're using and when to use each is crucial for effective performance measurement.

Key Concepts

Real Time: Wall-clock time from start to finish. Includes time waiting for I/O, other processes, and any delays.
User Time: CPU time spent executing user-mode code. This is what your program does (excluding kernel calls).
System Time: CPU time spent in kernel mode on behalf of the process (system calls, I/O operations).
CPU Percentage: (user + sys) / real. Shows how much CPU was actually used. Low % indicates I/O-bound or waiting.
Shell Builtin vs GNU time: Builtin is simpler (real/user/sys only). GNU time (/usr/bin/time) provides memory, I/O, and extensive formatting.
Output to stderr: Time statistics go to stderr (file descriptor 2), allowing you to separate program output from timing data.

Detailed Examples

Example 1

Basic Timing with Shell Builtin

Use the shell builtin time command to get basic execution time statistics for a command. This is the simplest and most common usage pattern for quick performance checks.

                    # Time a simple command
time ls -lR /usr/lib > /dev/null

# Time a script
time ./backup_script.sh

# Time a pipeline
time find /var/log -name "*.log" | wc -l

# Time a compound command
time { sleep 2; echo "done"; }

# Check which time you're using
type time
# Output: time is a shell keyword (builtin)

# Compare different operations
echo "=== Timing CPU-intensive task ==="
time python3 -c "sum(range(10000000))"

echo "=== Timing I/O-intensive task ==="
time find / -name "*.conf" 2>/dev/null | wc -l
                

Output:

# For ls command:
real    0m2.341s
user    0m0.145s
sys     0m0.234s

# For CPU-intensive task:
real    0m0.856s
user    0m0.842s
sys     0m0.012s
# Note: user+sys ≈ real (CPU-bound, little waiting)

# For I/O-intensive task:
real    0m15.234s
user    0m0.234s
sys     0m2.145s
# Note: user+sys << real (lots of I/O waiting)

Note:

Real time is always >= user + sys time. If real is much larger, the process spent significant time waiting (I/O, network, or blocked by other processes). If they're close, the process was CPU-bound.

Example 2

GNU time with Verbose Output

Use the standalone GNU time (/usr/bin/time) with the -v flag to get comprehensive resource usage statistics including memory, I/O, context switches, and page faults.

                    # Use GNU time explicitly (bypass shell builtin)
/usr/bin/time -v ls -lR /usr/lib > /dev/null

# Time a memory-intensive application
/usr/bin/time -v python3 -c "x = [i for i in range(10000000)]"

# Compare with shell builtin for contrast
echo "=== Shell builtin time ==="
time sleep 1

echo "=== GNU time with verbose ==="
/usr/bin/time -v sleep 1

# Practical example: time a compilation
/usr/bin/time -v make clean && /usr/bin/time -v make all
                

GNU time -v output:

        Command being timed: "ls -lR /usr/lib"
        User time (seconds): 0.14
        System time (seconds): 0.23
        Percent of CPU this job got: 87%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.34
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 3456
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 12
        Minor (reclaiming a frame) page faults: 234
        Voluntary context switches: 45
        Involuntary context switches: 23
        Swaps: 0
        File system inputs: 256
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

Note:

Maximum resident set size shows peak memory usage. Major page faults indicate disk I/O for memory pages. Context switches show how often the process was interrupted. This data is invaluable for performance optimization.

Example 3

Custom Output Format

Use GNU time's -f flag to create custom output formats. This is essential for scripts, logging, and automated performance monitoring where you need specific metrics in a particular format.

                    # Custom format for specific metrics
/usr/bin/time -f "Elapsed: %E\nCPU: %P\nMax Memory: %M KB" ls -lR /usr

# Format for CSV output (scripting/logging)
/usr/bin/time -f "%C,%e,%U,%S,%P,%M" python3 script.py

# Detailed format for performance analysis
/usr/bin/time -f "\
Command: %C
Real time: %e seconds
User time: %U seconds
System time: %S seconds
CPU usage: %P
Max memory: %M KB
Page faults: %F major, %R minor
Context switches: %c voluntary, %w involuntary" \
./benchmark_test

# Create a timing function for scripts
time_cmd() {
    /usr/bin/time -f "[$1] Time: %es | CPU: %P | Memory: %MKB" "$@"
}

# Use it
time_cmd "Database Query" mysql -e "SELECT COUNT(*) FROM large_table;"
time_cmd "File Processing" grep -r "error" /var/log/

# Log timing to file
/usr/bin/time -f "%C: real=%e user=%U sys=%S mem=%MKB" \
    -o performance.log \
    ./production_task.sh
                

Custom format output:

# Simple format:
Elapsed: 0:02.45
CPU: 87%
Max Memory: 3456 KB

# CSV format:
ls -lR /usr,2.45,0.14,0.23,87%,3456

# Detailed format:
Command: ./benchmark_test
Real time: 5.234 seconds
User time: 4.123 seconds
System time: 0.892 seconds
CPU usage: 95%
Max memory: 125678 KB
Page faults: 5 major, 1234 minor
Context switches: 23 voluntary, 8 involuntary

# In performance.log:
mysql -e SELECT COUNT(*) FROM large_table;: real=3.45 user=0.12 sys=0.08 mem=12345KB
grep -r error /var/log/: real=8.92 user=1.23 sys=2.45 mem=2345KB

Note:

Available format specifiers: %E (elapsed), %e (elapsed seconds), %U (user), %S (system), %P (CPU %), %M (max memory KB), %F (major faults), %R (minor faults), %c (voluntary switches), %w (involuntary switches), %C (command). See man time for complete list.

Example 4

Benchmark and Compare Commands

Use time to benchmark different implementations or approaches to solve the same problem. This is critical for optimization decisions and choosing the most efficient algorithm or tool.

                    # Compare different grep implementations
echo "=== Standard grep ==="
time grep -r "error" /var/log/ > /dev/null

echo "=== ripgrep (rg) ==="
time rg "error" /var/log/ > /dev/null

echo "=== ag (silver searcher) ==="
time ag "error" /var/log/ > /dev/null

# Compare sorting methods
dd if=/dev/urandom of=/tmp/random_data bs=1M count=100 2>/dev/null

echo "=== GNU sort ==="
time sort /tmp/random_data > /dev/null

echo "=== Sort with LC_ALL=C (faster) ==="
time LC_ALL=C sort /tmp/random_data > /dev/null

# Compare compression algorithms
echo "=== gzip compression ==="
time gzip -c /tmp/random_data > /tmp/test.gz

echo "=== bzip2 compression ==="
time bzip2 -c /tmp/random_data > /tmp/test.bz2

echo "=== xz compression ==="
time xz -c /tmp/random_data > /tmp/test.xz

# Check compression ratios
ls -lh /tmp/test.* | awk '{print $9, $5}'

# Compare script implementations
cat > /tmp/test1.sh << 'EOF'
#!/bin/bash
for i in {1..10000}; do
    echo "Line $i" >> /tmp/output1.txt
done
EOF

cat > /tmp/test2.sh << 'EOF'
#!/bin/bash
for i in {1..10000}; do
    echo "Line $i"
done > /tmp/output2.txt
EOF

chmod +x /tmp/test1.sh /tmp/test2.sh

echo "=== Appending in loop (slower) ==="
time /tmp/test1.sh

echo "=== Single redirection (faster) ==="
time /tmp/test2.sh
                

Benchmark results:

=== Standard grep ===
real    0m8.234s
user    0m1.234s
sys     0m2.456s

=== ripgrep (rg) ===
real    0m0.892s
user    0m0.567s
sys     0m0.234s

=== ag (silver searcher) ===
real    0m1.234s
user    0m0.678s
sys     0m0.345s

=== gzip compression ===
real    0m2.345s (6.2MB)

=== bzip2 compression ===
real    0m8.901s (5.8MB - better compression, slower)

=== xz compression ===
real    0m12.456s (5.1MB - best compression, slowest)

=== Appending in loop ===
real    0m15.678s

=== Single redirection ===
real    0m0.234s

Note:

These benchmarks show ripgrep is ~9x faster than grep, single redirection is ~67x faster than appending in a loop, and xz provides best compression at cost of speed. Real-world optimization decisions need this data.

Example 5

Timing with Output Separation

Separate program output from timing statistics. Since time writes to stderr, you can redirect stdout and stderr independently to keep timing data separate from program output.

                    # Redirect program output, keep timing visible
time ls -lR /usr > file_list.txt
# Timing goes to screen, ls output goes to file

# Redirect timing to file, keep program output visible
time ls -lR /usr 2> timing.txt
# Program output to screen, timing to file

# Redirect both separately
time ls -lR /usr > output.txt 2> timing.txt

# GNU time: redirect timing and program output separately
/usr/bin/time -o timing.log ls -lR /usr > output.txt

# Append timing to log file
/usr/bin/time -a -o performance.log \
    -f "[%C] %e seconds, %M KB memory" \
    ./script.sh > script_output.log

# Complex example: separate streams for analysis
{
    /usr/bin/time -f "TIMING: %e,%U,%S,%M" \
        python3 data_processor.py > results.csv
} 2> timing_stats.log

# Parse timing separately
grep "^TIMING:" timing_stats.log | cut -d: -f2

# Script that logs both output and timing
run_and_log() {
    local cmd="$1"
    local timestamp=$(date +%Y%m%d_%H%M%S)
    
    /usr/bin/time -f "Runtime: %e seconds\nMemory: %M KB" \
        -o "logs/timing_${timestamp}.log" \
        bash -c "$cmd" > "logs/output_${timestamp}.log" 2>&1
    
    echo "Command completed. Check logs/ directory."
}

run_and_log "find /var -name '*.log' | xargs wc -l"
                

File contents after execution:

# output.txt contains:
total 12345
drwxr-xr-x  142 root root  12288 Dec 14 14:00 .
drwxr-xr-x   20 root root   4096 Nov 28 09:12 ..
...

# timing.txt contains:
real    0m2.341s
user    0m0.145s
sys     0m0.234s

# timing.log (GNU time with -o):
        Command being timed: "ls -lR /usr"
        User time (seconds): 0.14
        System time (seconds): 0.23
        ...

# performance.log (appended format):
[./script.sh] 5.234 seconds, 12345 KB memory
[python3 data_processor.py] 8.901 seconds, 98765 KB memory

Note:

The -o flag (GNU time) writes timing to a file instead of stderr. Use -a to append instead of overwrite. This is essential for building performance monitoring systems and keeping logs organized.

Example 6

Timing Multiple Iterations

Run a command multiple times to get average timing and identify performance variability. This provides more reliable benchmarks by accounting for system load variations and cache effects.

                    # Simple loop timing
echo "Running 5 iterations..."
for i in {1..5}; do
    echo "Iteration $i:"
    time python3 script.py
done

# Collect timing data for analysis
cat > benchmark.sh << 'EOF'
#!/bin/bash

COMMAND="$1"
ITERATIONS="${2:-10}"
OUTPUT_FILE="benchmark_results.txt"

echo "Benchmarking: $COMMAND"
echo "Iterations: $ITERATIONS"
echo "Results saved to: $OUTPUT_FILE"
echo ""

# Clear previous results
> "$OUTPUT_FILE"

for i in $(seq 1 $ITERATIONS); do
    echo -n "Run $i/$ITERATIONS... "
    
    # Run and capture timing
    { time $COMMAND > /dev/null; } 2>&1 | \
        grep real | \
        awk '{print $2}' >> "$OUTPUT_FILE"
    
    echo "done"
done

# Calculate statistics
echo ""
echo "=== Results ==="
echo "All times (seconds):"
cat "$OUTPUT_FILE"

echo ""
echo "=== Statistics ==="

# Average
AVG=$(awk '{sum+=$1; count++} END {print sum/count}' "$OUTPUT_FILE")
echo "Average: ${AVG}s"

# Min/Max
MIN=$(sort -n "$OUTPUT_FILE" | head -1)
MAX=$(sort -n "$OUTPUT_FILE" | tail -1)
echo "Minimum: ${MIN}s"
echo "Maximum: ${MAX}s"

# Median
MEDIAN=$(sort -n "$OUTPUT_FILE" | awk '{arr[NR]=$1} END {
    if (NR % 2) print arr[(NR+1)/2]
    else print (arr[NR/2] + arr[NR/2+1])/2
}')
echo "Median: ${MEDIAN}s"
EOF

chmod +x benchmark.sh

# Use it
./benchmark.sh "grep -r 'error' /var/log" 10
./benchmark.sh "python3 -c 'sum(range(1000000))'" 20

# Advanced: GNU time with iterations
for i in {1..5}; do
    /usr/bin/time -f "Run $i: %e seconds, %M KB" \
        -a -o detailed_benchmark.log \
        ./test_program
done

# Statistical analysis with awk
cat detailed_benchmark.log | \
    awk '{print $3}' | \
    awk '{sum+=$1; sumsq+=$1*$1} END {
        print "Mean:", sum/NR
        print "StdDev:", sqrt(sumsq/NR - (sum/NR)^2)
    }'
                

Benchmark output:

Benchmarking: grep -r 'error' /var/log
Iterations: 10
Results saved to: benchmark_results.txt

Run 1/10... done
Run 2/10... done
...
Run 10/10... done

=== Results ===
All times (seconds):
0m8.234s
0m7.891s
0m8.456s
0m7.723s
0m8.901s
0m7.654s
0m8.123s
0m8.345s
0m7.987s
0m8.234s

=== Statistics ===
Average: 8.1548s
Minimum: 7.654s
Maximum: 8.901s
Median: 8.1785s

Mean: 8.155
StdDev: 0.347

Note:

Multiple iterations reveal performance consistency. High standard deviation indicates variable performance (system load, caching effects, I/O contention). Always benchmark multiple times for reliable data.

Example 7

Profile Script Sections

Time individual sections of a complex script to identify bottlenecks. This is essential for optimization - knowing which parts are slow lets you focus improvement efforts effectively.

                    # Create a script with timed sections
cat > profiled_script.sh << 'EOF'
#!/bin/bash

# Function to time sections
time_section() {
    local section_name="$1"
    shift
    echo "=== $section_name ==="
    /usr/bin/time -f "  Time: %es | Memory: %MKB" "$@"
    echo ""
}

echo "Starting data processing pipeline..."
echo ""

time_section "Data Download" \
    curl -s https://example.com/data.json -o /tmp/data.json

time_section "Data Parsing" \
    python3 -c "import json; data = json.load(open('/tmp/data.json')); print(len(data))"

time_section "Database Import" \
    mysql mydb < /tmp/import.sql

time_section "Index Creation" \
    mysql mydb -e "CREATE INDEX idx_name ON table_name(column)"

time_section "Report Generation" \
    python3 generate_report.py > report.html

echo "Pipeline complete!"
EOF

chmod +x profiled_script.sh
./profiled_script.sh

# Alternative: inline timing with SECONDS variable
cat > timing_with_seconds.sh << 'EOF'
#!/bin/bash

# Reset timer
SECONDS=0

echo "Step 1: Processing..."
sleep 2
step1_time=$SECONDS

echo "Step 2: Analysis..."
sleep 3
step2_time=$((SECONDS - step1_time))

echo "Step 3: Output..."
sleep 1
step3_time=$((SECONDS - SECONDS + step1_time + step2_time))

total_time=$SECONDS

echo ""
echo "=== Timing Breakdown ==="
echo "Step 1: ${step1_time}s"
echo "Step 2: ${step2_time}s"
echo "Step 3: ${step3_time}s"
echo "Total: ${total_time}s"
EOF

# Detailed profiling with timestamps
cat > detailed_profile.sh << 'EOF'
#!/bin/bash

log_time() {
    local label="$1"
    local timestamp=$(date +%s.%N)
    echo "$timestamp|$label" >> /tmp/profile.log
}

> /tmp/profile.log  # Clear log

log_time "START"

log_time "DB_QUERY_START"
mysql -e "SELECT COUNT(*) FROM large_table" > /dev/null
log_time "DB_QUERY_END"

log_time "FILE_PROCESS_START"
find /var/log -name "*.log" -exec wc -l {} \; > /dev/null
log_time "FILE_PROCESS_END"

log_time "END"

# Analyze timing
echo "=== Profile Results ==="
awk -F'|' '
    NR==1 {start=$1; prev=$1}
    NR>1 {
        duration = $1 - prev
        total = $1 - start
        printf "%-20s: %6.3fs (total: %6.3fs)\n", $2, duration, total
        prev = $1
    }
' /tmp/profile.log
EOF

chmod +x detailed_profile.sh
./detailed_profile.sh
                

Profiled script output:

Starting data processing pipeline...

=== Data Download ===
  Time: 1.234s | Memory: 2345KB

=== Data Parsing ===
  Time: 0.567s | Memory: 45678KB

=== Database Import ===
  Time: 8.901s | Memory: 12345KB

=== Index Creation ===
  Time: 3.456s | Memory: 8901KB

=== Report Generation ===
  Time: 2.345s | Memory: 23456KB

Pipeline complete!

# Timing breakdown shows database import is the bottleneck (8.9s)

=== Profile Results ===
DB_QUERY_START      :  0.000s (total:  0.000s)
DB_QUERY_END        :  5.234s (total:  5.234s)
FILE_PROCESS_START  :  0.001s (total:  5.235s)
FILE_PROCESS_END    :  3.456s (total:  8.691s)
END                 :  0.000s (total:  8.691s)

Note:

Profiling reveals that database operations take 5.2s and file processing takes 3.5s - together they account for the entire runtime. This data-driven insight directs optimization efforts to these specific sections.

Example 8

Memory Leak Detection

Use GNU time to track memory usage over time and detect memory leaks. By monitoring maximum resident set size across iterations, you can identify if a program is leaking memory.

                    # Monitor memory usage over iterations
cat > memory_monitor.sh << 'EOF'
#!/bin/bash

PROGRAM="$1"
ITERATIONS="${2:-10}"

echo "Monitoring memory usage: $PROGRAM"
echo "Iterations: $ITERATIONS"
echo ""

for i in $(seq 1 $ITERATIONS); do
    MEM=$(/usr/bin/time -f "%M" $PROGRAM 2>&1 | tail -1)
    echo "Iteration $i: ${MEM} KB"
    sleep 1
done

echo ""
echo "If memory increases with each iteration, suspect a memory leak."
EOF

chmod +x memory_monitor.sh

# Test with a leaky program
cat > leaky.py << 'EOF'
# Simulated memory leak
import sys
leaked = []
for i in range(1000000):
    leaked.append(i)
    if i % 100000 == 0:
        print(f"Iteration {i}", file=sys.stderr)
EOF

./memory_monitor.sh "python3 leaky.py" 5

# Compare with non-leaky program
cat > clean.py << 'EOF'
# Clean program
for i in range(1000000):
    if i % 100000 == 0:
        print(f"Iteration {i}")
EOF

./memory_monitor.sh "python3 clean.py" 5

# Detailed memory tracking
cat > track_memory.sh << 'EOF'
#!/bin/bash

COMMAND="$1"
DURATION="${2:-60}"  # seconds
INTERVAL="${3:-5}"   # seconds

OUTPUT="memory_tracking_$(date +%Y%m%d_%H%M%S).csv"

echo "timestamp,memory_kb" > "$OUTPUT"

# Start the command in background
$COMMAND &
PID=$!

echo "Tracking PID $PID for ${DURATION} seconds..."

START_TIME=$(date +%s)
while kill -0 $PID 2>/dev/null; do
    CURRENT_TIME=$(date +%s)
    ELAPSED=$((CURRENT_TIME - START_TIME))
    
    if [ $ELAPSED -ge $DURATION ]; then
        kill $PID 2>/dev/null
        break
    fi
    
    # Get current memory usage
    MEM=$(ps -o rss= -p $PID 2>/dev/null)
    if [ -n "$MEM" ]; then
        echo "$ELAPSED,$MEM" >> "$OUTPUT"
    fi
    
    sleep $INTERVAL
done

echo "Tracking complete. Data saved to $OUTPUT"

# Generate simple report
echo ""
echo "=== Memory Usage Summary ==="
awk -F',' 'NR>1 {
    if (NR==2) min=max=$2
    if ($2 < min) min=$2
    if ($2 > max) max=$2
    sum+=$2; count++
}
END {
    print "Minimum:", min, "KB"
    print "Maximum:", max, "KB"
    print "Average:", sum/count, "KB"
    print "Growth:", max-min, "KB"
}' "$OUTPUT"
EOF

chmod +x track_memory.sh

# Use it
./track_memory.sh "python3 long_running_app.py" 120 10
                

Memory monitoring output:

Monitoring memory usage: python3 leaky.py
Iterations: 5

Iteration 1: 45678 KB
Iteration 2: 67890 KB
Iteration 3: 89012 KB
Iteration 4: 110234 KB
Iteration 5: 131456 KB

If memory increases with each iteration, suspect a memory leak.

# Clean program:
Iteration 1: 12345 KB
Iteration 2: 12456 KB
Iteration 3: 12398 KB
Iteration 4: 12423 KB
Iteration 5: 12411 KB

# Leaky program shows steady increase, clean program stays constant

=== Memory Usage Summary ===
Minimum: 45678 KB
Maximum: 131456 KB
Average: 88854 KB
Growth: 85778 KB  # Significant growth indicates leak

Warning:

Memory growth over iterations is a strong indicator of memory leaks. Use tools like valgrind for detailed leak analysis, but time's -v flag provides a quick first check. Growing maximum resident set size across identical runs suggests leaks.

Example 9

Performance Regression Testing

Create automated tests that verify performance doesn't degrade over time. This is essential for CI/CD pipelines to catch performance regressions before they reach production.

                    # Performance regression test framework
cat > perf_test.sh << 'EOF'
#!/bin/bash

# Configuration
BASELINE_FILE="performance_baseline.txt"
THRESHOLD_PERCENT=20  # Fail if >20% slower

# Test cases
declare -A TESTS=(
    ["db_query"]="mysql -e 'SELECT COUNT(*) FROM users'"
    ["file_search"]="find /var/log -name '*.log'"
    ["compression"]="gzip -c /tmp/testfile.dat > /dev/null"
)

run_test() {
    local test_name="$1"
    local test_cmd="$2"
    
    # Run test and capture timing
    local runtime=$(/usr/bin/time -f "%e" bash -c "$test_cmd" 2>&1 | tail -1)
    
    echo "$runtime"
}

initialize_baseline() {
    echo "Creating performance baseline..."
    > "$BASELINE_FILE"
    
    for test_name in "${!TESTS[@]}"; do
        test_cmd="${TESTS[$test_name]}"
        echo -n "  $test_name... "
        
        # Run 3 times, take median
        times=()
        for i in {1..3}; do
            runtime=$(run_test "$test_name" "$test_cmd")
            times+=($runtime)
        done
        
        median=$(printf '%s\n' "${times[@]}" | sort -n | sed -n '2p')
        echo "$test_name=$median" >> "$BASELINE_FILE"
        echo "${median}s"
    done
    
    echo "Baseline created: $BASELINE_FILE"
}

run_regression_tests() {
    if [ ! -f "$BASELINE_FILE" ]; then
        echo "Error: No baseline found. Run with --init first."
        exit 1
    fi
    
    echo "Running performance regression tests..."
    echo ""
    
    local failed=0
    local total=0
    
    for test_name in "${!TESTS[@]}"; do
        test_cmd="${TESTS[$test_name]}"
        ((total++))
        
        # Get baseline
        baseline=$(grep "^$test_name=" "$BASELINE_FILE" | cut -d= -f2)
        
        if [ -z "$baseline" ]; then
            echo "❌ $test_name: No baseline found"
            ((failed++))
            continue
        fi
        
        # Run current test
        echo -n "Testing $test_name... "
        current=$(run_test "$test_name" "$test_cmd")
        
        # Calculate difference
        diff=$(awk "BEGIN {print ($current - $baseline) / $baseline * 100}")
        diff_int=${diff%.*}
        
        # Check threshold
        if [ "${diff_int#-}" -gt "$THRESHOLD_PERCENT" ]; then
            echo "❌ FAIL (${current}s vs ${baseline}s baseline, ${diff_int}% slower)"
            ((failed++))
        else
            echo "✓ PASS (${current}s vs ${baseline}s baseline, ${diff_int}% difference)"
        fi
    done
    
    echo ""
    echo "=== Results ==="
    echo "Total tests: $total"
    echo "Passed: $((total - failed))"
    echo "Failed: $failed"
    
    [ $failed -eq 0 ] && exit 0 || exit 1
}

# Main
case "$1" in
    --init)
        initialize_baseline
        ;;
    "")
        run_regression_tests
        ;;
    *)
        echo "Usage: $0 [--init]"
        echo "  --init: Create performance baseline"
        echo "  (no args): Run regression tests"
        exit 1
        ;;
esac
EOF

chmod +x perf_test.sh

# Initialize baseline
./perf_test.sh --init

# Run tests (in CI/CD pipeline)
./perf_test.sh

# Integration with CI
cat > .gitlab-ci.yml << 'EOF'
performance_test:
  stage: test
  script:
    - ./perf_test.sh
  only:
    - merge_requests
  artifacts:
    when: always
    reports:
      junit: performance_results.xml
EOF
                

Regression test output:

# Initialization:
Creating performance baseline...
  db_query... 0.234s
  file_search... 2.345s
  compression... 1.123s
Baseline created: performance_baseline.txt

# Running tests:
Running performance regression tests...

Testing db_query... ✓ PASS (0.245s vs 0.234s baseline, 4% difference)
Testing file_search... ❌ FAIL (3.012s vs 2.345s baseline, 28% slower)
Testing compression... ✓ PASS (1.156s vs 1.123s baseline, 2% difference)

=== Results ===
Total tests: 3
Passed: 2
Failed: 1

# Exit code 1 - CI pipeline fails

Note:

This framework catches performance regressions automatically. The file_search test shows 28% degradation, triggering a CI failure. Investigate before merging. Update baseline when intentional changes affect performance.

Example 10

Complete Performance Monitoring System

A comprehensive example combining timing, memory monitoring, logging, alerting, and trending analysis. This production-ready system demonstrates best practices for continuous performance monitoring.

                    #!/bin/bash
# File: /usr/local/bin/perf-monitor
# Production performance monitoring system

set -euo pipefail

# ============================================
# Configuration
# ============================================
readonly MONITOR_DIR="/var/lib/perf-monitor"
readonly LOG_DIR="/var/log/perf-monitor"
readonly DATA_DIR="$MONITOR_DIR/data"
readonly ALERT_THRESHOLD_SECONDS=10
readonly ALERT_MEMORY_MB=1024
readonly RETENTION_DAYS=30

# Ensure directories exist
mkdir -p "$MONITOR_DIR" "$LOG_DIR" "$DATA_DIR"

# ============================================
# Functions
# ============================================

log_message() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "$LOG_DIR/monitor.log"
}

record_metrics() {
    local job_name="$1"
    local command="$2"
    local timestamp=$(date +%Y%m%d_%H%M%S)
    local metrics_file="$DATA_DIR/${job_name}_${timestamp}.json"
    
    log_message "Starting monitoring: $job_name"
    
    # Run command with comprehensive timing
    /usr/bin/time -f '{
  "job": "%C",
  "timestamp": "'$(date -Iseconds)'",
  "elapsed_seconds": %e,
  "user_seconds": %U,
  "system_seconds": %S,
  "cpu_percent": "%P",
  "max_memory_kb": %M,
  "avg_memory_kb": %K,
  "page_faults_major": %F,
  "page_faults_minor": %R,
  "io_inputs": %I,
  "io_outputs": %O,
  "context_switches_voluntary": %c,
  "context_switches_involuntary": %w,
  "exit_status": %x
}' \
        -o "$metrics_file" \
        bash -c "$command" >  "$LOG_DIR/${job_name}_output_${timestamp}.log" 2>&1
    
    local exit_code=$?
    
    log_message "Completed: $job_name (exit: $exit_code)"
    
    # Check for alerts
    check_alerts "$job_name" "$metrics_file"
    
    return $exit_code
}

check_alerts() {
    local job_name="$1"
    local metrics_file="$2"
    
    # Extract metrics (requires jq)
    if ! command -v jq &>/dev/null; then
        log_message "Warning: jq not installed, skipping alert checks"
        return
    fi
    
    local elapsed=$(jq -r '.elapsed_seconds' "$metrics_file")
    local memory_kb=$(jq -r '.max_memory_kb' "$metrics_file")
    local memory_mb=$((memory_kb / 1024))
    local exit_status=$(jq -r '.exit_status' "$metrics_file")
    
    # Check exit status
    if [ "$exit_status" != "0" ]; then
        send_alert "FAILURE" "$job_name failed with exit code $exit_status"
    fi
    
    # Check elapsed time
    if (( $(echo "$elapsed > $ALERT_THRESHOLD_SECONDS" | bc -l) )); then
        send_alert "SLOW" "$job_name took ${elapsed}s (threshold: ${ALERT_THRESHOLD_SECONDS}s)"
    fi
    
    # Check memory usage
    if [ "$memory_mb" -gt "$ALERT_MEMORY_MB" ]; then
        send_alert "HIGH_MEMORY" "$job_name used ${memory_mb}MB (threshold: ${ALERT_MEMORY_MB}MB)"
    fi
}

send_alert() {
    local severity="$1"
    local message="$2"
    
    log_message "ALERT [$severity]: $message"
    
    # Send email (if configured)
    if [ -n "${ALERT_EMAIL:-}" ]; then
        echo "$message" | mail -s "Performance Alert: $severity" "$ALERT_EMAIL"
    fi
    
    # Log to syslog
    logger -t perf-monitor -p user.warning "[$severity] $message"
}

generate_report() {
    local job_name="${1:-all}"
    local days="${2:-7}"
    
    log_message "Generating report for $job_name (last $days days)"
    
    if ! command -v jq &> /dev/null; then
        log_message "Error: jq required for reports"
        return 1
    fi
    
    echo "=== Performance Report: $job_name ==="
    echo "Period: Last $days days"
    echo ""
    
    # Find relevant metrics files
    local cutoff_date=$(date -d "$days days ago" +%Y%m%d)
    local files=()
    
    if [ "$job_name" = "all" ]; then
        files=($(find "$DATA_DIR" -name "*.json" -type f))
    else
        files=($(find "$DATA_DIR" -name "${job_name}_*.json" -type f))
    fi
    
    # Filter by date
    local recent_files=()
    for file in "${files[@]}"; do
        local file_date=$(basename "$file" | grep -o '[0-9]\{8\}' | head -1)
        if [ "$file_date" -ge "$cutoff_date" ]; then
            recent_files+=("$file")
        fi
    done
    
    if [ ${#recent_files[@]} -eq 0 ]; then
        echo "No data found for specified period"
        return
    fi
    
    # Aggregate statistics
    jq -s '
        group_by(.job) | 
        map({
            job: .[0].job,
            runs: length,
            avg_elapsed: (map(.elapsed_seconds) | add / length),
            min_elapsed: (map(.elapsed_seconds) | min),
            max_elapsed: (map(.elapsed_seconds) | max),
            avg_memory_mb: (map(.max_memory_kb) | add / length / 1024),
            max_memory_mb: (map(.max_memory_kb) | max / 1024),
            failures: (map(select(.exit_status != 0)) | length)
        })
    ' "${recent_files[@]}" | \
    jq -r '
        (["Job", "Runs", "Avg Time", "Min Time", "Max Time", "Avg Mem", "Max Mem", "Failures"] | @tsv),
        (.[] | [.job, .runs, (.avg_elapsed|tostring+"s"), (.min_elapsed|tostring+"s"), 
                (.max_elapsed|tostring+"s"), (.avg_memory_mb|tostring+"MB"), 
                (.max_memory_mb|tostring+"MB"), .failures] | @tsv)
    ' | column -t
}

cleanup_old_data() {
    log_message "Cleaning up data older than $RETENTION_DAYS days"
    
    find "$DATA_DIR" -name "*.json" -type f -mtime +$RETENTION_DAYS -delete
    find "$LOG_DIR" -name "*_output_*.log" -type f -mtime +$RETENTION_DAYS -delete
    
    log_message "Cleanup complete"
}

# ============================================
# Main CLI
# ============================================

usage() {
    cat << EOF
Usage: $(basename $0) <command> [options]

Commands:
    run <job-name> <command>    Monitor and time a command
    report [job] [days]         Generate performance report
    cleanup                     Remove old data
    
Examples:
    $(basename $0) run backup "tar czf /backup/data.tar.gz /data"
    $(basename $0) report backup 30
    $(basename $0) report all 7
    $(basename $0) cleanup
EOF
}

main() {
    case "${1:-}" in
        run)
            [ $# -lt 3 ] && { usage; exit 1; }
            record_metrics "$2" "$3"
            ;;
        report)
            generate_report "${2:-all}" "${3:-7}"
            ;;
        cleanup)
            cleanup_old_data
            ;;
        *)
            usage
            exit 1
            ;;
    esac
}

main "$@"

                

System usage output:

# Record a job:
$ perf-monitor run database_backup "mysqldump --all-databases > /backup/db.sql"
[2025-12-14 16:30:00] Starting monitoring: database_backup
[2025-12-14 16:32:15] Completed: database_backup (exit: 0)

# Generate report:
$ perf-monitor report all 7

=== Performance Report: all ===
Period: Last 7 days

Job                 Runs  Avg Time  Min Time  Max Time  Avg Mem   Max Mem   Failures
database_backup     14    135.2s    128.4s    145.7s    234.5MB   267.8MB   0
log_rotation        28    3.4s      2.9s      4.1s      12.3MB    15.6MB    0
report_generation   7     45.6s     42.1s     52.3s     456.7MB   523.1MB   1

# Alert example:
[2025-12-14 16:32:15] ALERT [SLOW]: database_backup took 145.7s (threshold: 10s)

# Cleanup:
$ perf-monitor cleanup
[2025-12-14 16:35:00] Cleaning up data older than 30 days
[2025-12-14 16:35:01] Cleanup complete

Note:

This production system provides comprehensive performance monitoring: automatic timing, memory tracking, alerting on thresholds, historical reporting, and data retention management. Perfect for production cron jobs and critical batch processes.

Tips & Best Practices

Understand Shell Builtin vs GNU time

Use type time to check which version you're using. Shell builtin provides basic timing. Use /usr/bin/time explicitly for detailed metrics, memory stats, and custom formatting.

Interpret Real vs User+System Time

If real ≈ user+sys, the process is CPU-bound. If real >> user+sys, it's I/O-bound or waiting. This tells you whether to optimize CPU usage or I/O operations.

Use -v for Comprehensive Analysis

/usr/bin/time -v provides memory usage, page faults, context switches, and I/O stats. This is invaluable for identifying performance bottlenecks beyond just execution time.

Run Multiple Iterations

Never benchmark once. Run at least 3-5 iterations to account for system load, caching, and variability. Calculate average, min, max, and standard deviation for reliable data.

Separate Output Streams

Time statistics go to stderr. Use command > output.txt to separate program output from timing. Or use /usr/bin/time -o timing.log to save timing separately.

Custom Formatting for Automation

Use -f flag with format specifiers for scripts and logging: /usr/bin/time -f "%e,%U,%S,%M" produces CSV-parseable output perfect for automation.

Watch for Memory Leaks

Monitor maximum resident set size (%M) across repeated runs. Growing memory usage indicates potential memory leaks. Use -v to track page faults and swapping.

Consider System Load

Benchmarks on loaded systems aren't reliable. Check system load with uptime before benchmarking. For consistent results, benchmark on idle systems or use nice/ionice to isolate tests.

Use for Regression Testing

Establish performance baselines with time, then run automated tests in CI/CD to catch regressions. Fail builds if performance degrades beyond acceptable thresholds.

Synopsis

Description

Key Concepts

Detailed Examples

Basic Timing with Shell Builtin

GNU time with Verbose Output

Custom Output Format

Benchmark and Compare Commands

Timing with Output Separation

Timing Multiple Iterations

Profile Script Sections

Memory Leak Detection

Performance Regression Testing

Complete Performance Monitoring System

Tips & Best Practices

Related Commands