Linux Command Reference

paste

Synopsis

paste [OPTION]... [FILE]...
paste -s [OPTION]... [FILE]...

Description

The paste command merges lines from multiple files side-by-side, separating them with tabs (by default) or a custom delimiter. Think of it as the horizontal equivalent of cat - while cat stacks files vertically (appending one after another), paste combines them horizontally (joining corresponding lines across files).

This makes paste essential for data manipulation tasks: combining columns from different files, creating CSV data from separate sources, reformatting structured text, and general column-oriented data processing. The -s (serial) flag changes the behavior to merge all lines from each file into a single line, useful for converting multi-line data into single-line comma-separated or tab-separated values.

Key Concepts

Side-by-Side Merging: Combines corresponding lines from files horizontally, creating columns.
Default Delimiter: Uses tab character between columns. Change with -d option.
Serial Mode (-s): Joins all lines from each file into a single line instead of merging across files.
Stdin Support: Use - to read from stdin, allowing paste in pipelines.
Line Matching: Processes files line-by-line. Shorter files result in empty columns for missing lines.
Multiple Delimiters: Can specify multiple delimiters that cycle through columns.

Detailed Examples

Example 1

Basic Side-by-Side Merge

Combine two files side-by-side with default tab delimiter. This is the fundamental use case for paste - taking columns from separate files and creating a unified table.

                    # Create sample files
cat > names.txt << 'EOF'
Alice
Bob
Charlie
David
EOF

cat > ages.txt << 'EOF'
25
30
35
40
EOF

cat > cities.txt << 'EOF'
NYC
LA
Chicago
Houston
EOF

# Basic paste - merge two files
paste names.txt ages.txt

# Merge three files
paste names.txt ages.txt cities.txt

# Paste with file viewing
echo "=== Names ==="
cat names.txt
echo "=== Ages ==="
cat ages.txt
echo "=== Combined ==="
paste names.txt ages.txt
                

Output:

# Two files:
Alice   25
Bob     30
Charlie 35
David   40

# Three files:
Alice   25      NYC
Bob     30      LA
Charlie 35      Chicago
David   40      Houston

# The tab character separates columns (shows as spaces in display)

Note:

By default, paste uses tab as delimiter. The output shown uses spaces for readability, but actual output has tab characters. Use paste -d',' for comma-separated output if needed.

Example 2

Custom Delimiters

Use the -d option to specify custom delimiters. This is essential for creating CSV files, formatted reports, or any specific column separator requirement.

                    # Comma-separated (CSV)
paste -d',' names.txt ages.txt cities.txt

# Pipe-delimited
paste -d'|' names.txt ages.txt cities.txt

# Space-delimited
paste -d' ' names.txt ages.txt cities.txt

# Colon-delimited (like /etc/passwd format)
paste -d':' names.txt ages.txt cities.txt

# Multiple character delimiter (uses first char)
paste -d', ' names.txt ages.txt cities.txt

# Special characters: newline in delimiter
paste -d$'\n' names.txt ages.txt

# Create a properly formatted CSV with headers
echo "Name,Age,City" > data.csv
paste -d',' names.txt ages.txt cities.txt >> data.csv
cat data.csv

# Format as a table with custom spacing
paste -d' | ' names.txt ages.txt cities.txt | \
    awk '{printf "| %-10s | %-5s | %-10s |\n", $1, $3, $5}'
                

Output:

# Comma-separated:
Alice,25,NYC
Bob,30,LA
Charlie,35,Chicago
David,40,Houston

# Pipe-delimited:
Alice|25|NYC
Bob|30|LA
Charlie|35|Chicago
David|40|Houston

# Colon-delimited:
Alice:25:NYC
Bob:30:LA
Charlie:35:Chicago
David:40:Houston

# CSV file:
Name,Age,City
Alice,25,NYC
Bob,30,LA
Charlie,35,Chicago
David,40,Houston

# Formatted table:
| Alice      | 25    | NYC        |
| Bob        | 30    | LA         |
| Charlie    | 35    | Chicago    |
| David      | 40    | Houston    |

Note:

The -d option accepts a single character or a list of characters. With multiple characters, paste cycles through them for each column separator.

Example 3

Serial Mode (-s) - Convert Rows to Columns

Use -s to join all lines from a file into a single line. This converts multi-line data into a single comma-separated or tab-separated line, useful for creating horizontal lists.

                    # Create a list file
cat > fruits.txt << 'EOF'
Apple
Banana
Cherry
Date
Elderberry
EOF

# Serial mode with default tab delimiter
paste -s fruits.txt

# Serial mode with comma delimiter
paste -s -d',' fruits.txt

# Serial mode with space delimiter
paste -s -d' ' fruits.txt

# Multiple files in serial mode
cat > vegetables.txt << 'EOF'
Carrot
Broccoli
Spinach
EOF

# Each file becomes one line
paste -s -d',' fruits.txt vegetables.txt

# Create a formatted list with "and"
paste -s -d',' fruits.txt | sed 's/,/, /g; s/\(.*\),/\1 and/'

# Convert a column to SQL IN clause
echo "SELECT * FROM products WHERE name IN ("
paste -s -d',' fruits.txt | sed "s/^/'/; s/$/'/; s/,/', '/g"
echo ");"

# Practical: get all unique values in one line
cat access.log | awk '{print $1}' | sort -u | paste -s -d','
                

Output:

# Default tab delimiter:
Apple   Banana  Cherry  Date    Elderberry

# Comma delimiter:
Apple,Banana,Cherry,Date,Elderberry

# Space delimiter:
Apple Banana Cherry Date Elderberry

# Multiple files (each file one line):
Apple,Banana,Cherry,Date,Elderberry
Carrot,Broccoli,Spinach

# Formatted with "and":
Apple, Banana, Cherry, Date and Elderberry

# SQL IN clause:
SELECT * FROM products WHERE name IN (
'Apple', 'Banana', 'Cherry', 'Date', 'Elderberry'
);

Note:

The -s flag is perfect for converting vertical lists to horizontal format. Common use cases include creating SQL IN clauses, formatting comma-separated lists, and aggregating data for reports.

Example 4

Using stdin with Paste

Use - to represent stdin, allowing paste to work in pipelines. This enables dynamic data merging where one or more inputs come from command output rather than files.

                    # Merge command output with a file
ls -1 /tmp | paste - ages.txt

# Use stdin multiple times
seq 1 5 | paste - -

# Combine file and command output
paste names.txt <(echo "Status"; tail -n +2 names.txt | sed 's/.*/Active/')

# Number lines from a file
cat names.txt | nl | paste - ages.txt

# Add line numbers to paste output
paste names.txt ages.txt | nl

# Merge three sources: file, command, file
paste names.txt <(date +%Y | xargs -n1) cities.txt

# Complex pipeline: merge sorted unique values
cat file1.txt file2.txt | sort -u | paste -s -d','

# Combine cut columns with paste
cut -f1 data.tsv | paste - <(cut -f3 data.tsv)

# Create a quick lookup table
echo "1 2 3 4 5" | tr ' ' '\n' | \
    paste - <(echo "one two three four five" | tr ' ' '\n')
                

Output:

# ls with ages:
file1.txt       25
file2.txt       30
file3.txt       35

# seq with itself (two columns):
1       2
3       4
5

# File with status:
Alice   Active
Bob     Active
Charlie Active
David   Active

# Numbered with ages:
     1  Alice   25
     2  Bob     30
     3  Charlie 35
     4  David   40

# Lookup table:
1       one
2       two
3       three
4       four
5       five

Note:

Using process substitution <(command) with paste is powerful - it treats command output as if it were a file. This allows complex data merging from multiple sources in a single pipeline.

Example 5

Handling Files of Different Lengths

When files have different numbers of lines, paste handles this by outputting empty fields for the shorter files. Understanding this behavior is important for data integrity.

                    # Create files of different lengths
cat > short.txt << 'EOF'
A
B
C
EOF

cat > medium.txt << 'EOF'
1
2
3
4
5
EOF

cat > long.txt << 'EOF'
X
Y
Z
W
Q
R
S
EOF

# Paste files of different lengths
paste short.txt medium.txt long.txt

# With comma delimiter to see empty fields clearly
paste -d',' short.txt medium.txt long.txt

# Identify where files end
paste -d'|' short.txt medium.txt long.txt | nl

# Fill missing values with placeholder
paste -d',' short.txt medium.txt long.txt | \
    sed 's/^,/MISSING,/; s/,,/,MISSING,/g; s/,$/,MISSING/'

# Count lines per file
echo "short.txt: $(wc -l < short.txt) lines"
echo "medium.txt: $(wc -l < medium.txt) lines"
echo "long.txt: $(wc -l < long.txt) lines"
echo "paste output: $(paste short.txt medium.txt long.txt | wc -l) lines"

# Use awk to handle missing data
paste -d',' short.txt medium.txt long.txt | \
    awk -F',' '{
        if ($1 == "") $1 = "N/A"
        if ($2 == "") $2 = "N/A"  
        if ($3 == "") $3 = "N/A"
        print $1 "," $2 "," $3
    }'
                

Output:

# Tab-delimited:
A       1       X
B       2       Y
C       3       Z
        4       W
        5       Q
                R
                S

# Comma-delimited (empty fields visible):
A,1,X
B,2,Y
C,3,Z
,4,W
,5,Q
,,R
,,S

# With placeholders:
A,1,X
B,2,Y
C,3,Z
MISSING,4,W
MISSING,5,Q
MISSING,MISSING,R
MISSING,MISSING,S

# Output line count:
short.txt: 3 lines
medium.txt: 5 lines
long.txt: 7 lines
paste output: 7 lines  (longest file determines output length)

Warning:

When merging files of different lengths, paste continues until all files are exhausted. Shorter files contribute empty fields. Always verify your data integrity when files might have different lengths.

Example 6

Multiple Delimiter Cycling

When you provide multiple characters to -d, paste cycles through them for each column separator. This creates sophisticated formatting patterns for reports and structured output.

                    # Create sample data
cat > col1.txt << 'EOF'
Name
Alice
Bob
Charlie
EOF

cat > col2.txt << 'EOF'
Age
25
30
35
EOF

cat > col3.txt << 'EOF'
City
NYC
LA
Chicago
EOF

# Cycle through delimiters: | then ,
paste -d'|,' col1.txt col2.txt col3.txt

# Create a table with | and spaces
paste -d'| ' col1.txt col2.txt col3.txt

# Complex delimiter pattern
paste -d':;,' col1.txt col2.txt col3.txt col1.txt

# Create markdown-style table
echo "| Column1 | Column2 | Column3 |"
echo "|---------|---------|---------|"
paste col1.txt col2.txt col3.txt | \
    awk -F'\t' '{printf "| %-8s| %-8s| %-8s|\n", $1, $2, $3}'

# Pattern for key-value pairs
paste -d'=' col1.txt col2.txt | sed 's/^/export /'

# Create INI-style config
echo "[Settings]"
paste -d'=' col1.txt col2.txt
                

Output:

# Cycling | and , :
Name|Age,City
Alice|25,NYC
Bob|30,LA
Charlie|35,Chicago

# Delimiter pattern cycles through characters:
Name:Age;City,Name
Alice:25;NYC,Alice
Bob:30;LA,Bob
Charlie:35;Chicago,Charlie

# Markdown table:
| Column1 | Column2 | Column3 |
|---------|---------|---------|
| Name    | Age     | City    |
| Alice   | 25      | NYC     |
| Bob     | 30      | LA      |
| Charlie | 35      | Chicago |

# Key-value exports:
export Name=Age
export Alice=25
export Bob=30
export Charlie=35

# INI format:
[Settings]
Name=Age
Alice=25
Bob=30
Charlie=35

Note:

Delimiter cycling allows creative formatting. Each separator position uses the next character in the delimiter string, then wraps back to the beginning. This is powerful for creating structured formats like tables and config files.

Example 7

Data Transformation and Reformatting

Use paste to transform data formats - converting between row and column layouts, creating paired data, and restructuring text files for different purposes.

                    # Convert single column to multiple columns (4 columns)
seq 1 20 | paste - - - -

# Convert to 3 columns with serial mode
seq 1 12 | paste - - -

# Create key-value pairs from two lists
cat > keys.txt << 'EOF'
user
pass
host
port
EOF

cat > values.txt << 'EOF'
admin
secret123
localhost
5432
EOF

paste -d'=' keys.txt values.txt

# Transpose a matrix (swap rows and columns)
cat > matrix.txt << 'EOF'
1 2 3
4 5 6
7 8 9
EOF

# Simple transpose for small data
paste <(awk '{print $1}' matrix.txt) \
      <(awk '{print $2}' matrix.txt) \
      <(awk '{print $3}' matrix.txt)

# Create pairs from a single list
cat > items.txt << 'EOF'
A
B
C
D
E
F
EOF

# Pair consecutive items
paste - - < items.txt

# Create overlapping pairs (sliding window)
paste items.txt <(tail -n +2 items.txt)

# Interleave two files
cat > odd.txt << 'EOF'
1
3
5
EOF

cat > even.txt << 'EOF'
2
4
6
EOF

paste odd.txt even.txt | tr '\t' '\n'

# Convert space-separated to newline-separated
echo "one two three four five" | tr ' ' '\n' | paste -s -d' '

# Reverse operation: split line to multiple lines
echo "a,b,c,d,e" | tr ',' '\n'
                

Output:

# 4 columns from sequence:
1       2       3       4
5       6       7       8
9       10      11      12
13      14      15      16
17      18      19      20

# Key-value pairs:
user=admin
pass=secret123
host=localhost
port=5432

# Transposed matrix:
1       4       7
2       5       8
3       6       9

# Consecutive pairs:
A       B
C       D
E       F

# Sliding window pairs:
A       B
B       C
C       D
D       E
E       F

# Interleaved:
1
2
3
4
5
6

Note:

These transformations are essential for data wrangling. The pattern paste - - - creates 3 columns by reading stdin three times. This technique is widely used for reformatting data into different layouts.

Example 8

Combining paste with Other Commands

Paste is powerful in pipelines. Combine it with awk, sed, cut, grep, and other text processing tools to create sophisticated data manipulation workflows.

                    # Add line numbers to existing data
paste <(seq 1 4) names.txt ages.txt

# Create a columnar report with headers
{
    echo -e "ID\tName\tAge"
    paste <(seq 1 4) names.txt ages.txt
} | column -t

# Merge sorted files with indicators
paste <(sort file1.txt) <(sort file2.txt) | \
    awk -F'\t' '{
        if ($1 == $2) print $1 " [BOTH]"
        else if ($1 != "") print $1 " [FILE1]"
        else print $2 " [FILE2]"
    }'

# Calculate differences between columns
cat > jan.txt << 'EOF'
100
150
200
250
EOF

cat > feb.txt << 'EOF'
110
145
210
260
EOF

paste jan.txt feb.txt | \
    awk '{diff=$2-$1; pct=($2-$1)/$1*100; 
          printf "%d\t%d\t%+d\t%+.1f%%\n", $1, $2, diff, pct}'

# Create a lookup table from two files
paste -d':' \
    <(cut -d',' -f1 users.csv) \
    <(cut -d',' -f3 users.csv)

# Merge multiple log files with timestamps
paste \
    <(grep "ERROR" app1.log | awk '{print $1, $2}') \
    <(grep "ERROR" app2.log | awk '{print $3}') \
    | sort

# Generate SQL INSERT statements
paste names.txt ages.txt cities.txt | \
    awk -F'\t' '{
        printf "INSERT INTO users (name, age, city) VALUES ('\''%s'\'', %s, '\''%s'\'');\n", 
               $1, $2, $3
    }'

# Create a comparison table
paste -d'|' \
    <(ls -1 dir1/) \
    <(ls -1 dir2/) | \
    column -t -s'|'
                

Output:

# Numbered report with headers:
ID  Name     Age
1   Alice    25
2   Bob      30
3   Charlie  35
4   David    40

# Difference calculation:
100     110     +10     +10.0%
150     145     -5      -3.3%
200     210     +10     +5.0%
250     260     +10     +4.0%

# SQL INSERT statements:
INSERT INTO users (name, age, city) VALUES ('Alice', 25, 'NYC');
INSERT INTO users (name, age, city) VALUES ('Bob', 30, 'LA');
INSERT INTO users (name, age, city) VALUES ('Charlie', 35, 'Chicago');
INSERT INTO users (name, age, city) VALUES ('David', 40, 'Houston');

# Directory comparison:
file1.txt    file1.txt
file2.txt    
file3.txt    file3.txt
             file4.txt

Note:

Combining paste with awk is especially powerful for calculations and formatting. The ability to process multiple columns simultaneously enables complex data transformations in compact pipelines.

Example 9

Real-World Log Processing

Practical examples of using paste for log file analysis, combining data from multiple sources, and creating reports from disparate log files.

                    # Combine access and error logs by timestamp
cat > access.log << 'EOF'
2025-12-14 10:00 GET /api/users 200
2025-12-14 10:01 POST /api/login 200
2025-12-14 10:02 GET /api/data 500
2025-12-14 10:03 GET /api/status 200
EOF

cat > error.log << 'EOF'
2025-12-14 10:00 -
2025-12-14 10:01 -
2025-12-14 10:02 Database connection failed
2025-12-14 10:03 -
EOF

# Merge logs side by side
paste -d'|' access.log error.log | \
    grep -v '|-$' | \
    awk -F'|' '{print $1 " [ERROR] " $2}'

# Extract and compare metrics from two servers
cat > server1_metrics.txt << 'EOF'
CPU: 45%
Memory: 2.3GB
Disk: 67%
EOF

cat > server2_metrics.txt << 'EOF'
CPU: 52%
Memory: 3.1GB
Disk: 71%
EOF

echo "Metric     Server1    Server2"
echo "--------------------------------"
paste server1_metrics.txt server2_metrics.txt | \
    awk -F'\t' '{
        split($1, a, ": ")
        split($2, b, ": ")
        printf "%-10s %-10s %-10s\n", a[1], a[2], b[2]
    }'

# Create hourly report from logs
cat > hourly_requests.log << 'EOF'
00:00 234
01:00 189
02:00 156
03:00 142
04:00 167
EOF

cat > hourly_errors.log << 'EOF'
00:00 3
01:00 2
02:00 1
03:00 0
04:00 2
EOF

echo "Hour   Requests  Errors  Error%"
echo "================================"
paste hourly_requests.log hourly_errors.log | \
    awk '{
        hour=$1
        requests=$2
        errors=$4
        if (requests > 0) {
            pct = (errors/requests)*100
        } else {
            pct = 0
        }
        printf "%s  %-9d %-7d %.2f%%\n", hour, requests, errors, pct
    }'

# Merge user activity from multiple sources
paste \
    <(cut -d',' -f1,2 web_activity.csv) \
    <(cut -d',' -f2 mobile_activity.csv) \
    <(cut -d',' -f2 api_activity.csv) | \
    awk -F'\t' '{
        total = $2 + $3 + $4
        printf "%s: Web=%d Mobile=%d API=%d Total=%d\n", 
               $1, $2, $3, $4, total
    }'
                

Output:

# Merged error log:
2025-12-14 10:00 GET /api/users 200 [ERROR] 2025-12-14 10:02 Database connection failed

# Server comparison:
Metric     Server1    Server2
--------------------------------
CPU        45%        52%
Memory     2.3GB      3.1GB
Disk       67%        71%

# Hourly report:
Hour   Requests  Errors  Error%
================================
00:00  234       3       1.28%
01:00  189       2       1.06%
02:00  156       1       0.64%
03:00  142       0       0.00%
04:00  167       2       1.20%

# User activity:
user1: Web=145 Mobile=67 API=23 Total=235
user2: Web=89 Mobile=102 API=45 Total=236
user3: Web=201 Mobile=34 API=12 Total=247

Note:

These patterns are production-ready for log analysis. Paste excels at correlating data from multiple log sources, creating unified reports, and calculating metrics across disparate data files.

Example 10

Advanced Data Correlation Script

A comprehensive example showing a production-ready script that uses paste for data correlation, validation, and report generation. This demonstrates best practices for complex data processing.

                    #!/bin/bash
# File: data_correlator.sh
# Correlate and validate data from multiple sources

set -euo pipefail

# ============================================
# Configuration
# ============================================
readonly DATA_DIR="${1:-.}"
readonly OUTPUT_DIR="${2:-./output}"
readonly LOG_FILE="$OUTPUT_DIR/correlation.log"

mkdir -p "$OUTPUT_DIR"

# ============================================
# Logging
# ============================================
log() {
    echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" | tee -a "$LOG_FILE"
}

# ============================================
# Data Validation
# ============================================
validate_files() {
    local -a files=("$@")
    
    log "Validating input files..."
    
    for file in "${files[@]}"; do
        if [ ! -f "$file" ]; then
            log "ERROR: File not found: $file"
            exit 1
        fi
        
        local lines=$(wc -l < "$file")
        log "  $file: $lines lines"
    done
    
    # Check if all files have same length
    local first_len=$(wc -l < "${files[0]}")
    for file in "${files[@]:1}"; do
        local file_len=$(wc -l < "$file")
        if [ "$file_len" != "$first_len" ]; then
            log "WARNING: File length mismatch: ${files[0]}($first_len) vs $file($file_len)"
        fi
    done
}

# ============================================
# Correlation Functions
# ============================================
correlate_data() {
    local id_file="$1"
    local data1_file="$2"
    local data2_file="$3"
    local output_file="$4"
    
    log "Correlating data..."
    log "  IDs: $id_file"
    log "  Data1: $data1_file"
    log "  Data2: $data2_file"
    log "  Output: $output_file"
    
    # Create header
    echo -e "ID\tData1\tData2\tDifference\tPercent_Change" > "$output_file"
    
    # Correlate and calculate
    paste "$id_file" "$data1_file" "$data2_file" | \
        awk -F'\t' '
        NR > 1 {
            id = $1
            val1 = $2
            val2 = $3
            
            diff = val2 - val1
            if (val1 != 0) {
                pct = (diff / val1) * 100
            } else {
                pct = 0
            }
            
            printf "%s\t%.2f\t%.2f\t%+.2f\t%+.1f%%\n", 
                   id, val1, val2, diff, pct
        }' >> "$output_file"
    
    local rows=$(($(wc -l < "$output_file") - 1))
    log "Processed $rows data rows"
}

# ============================================
# Anomaly Detection
# ============================================
detect_anomalies() {
    local data_file="$1"
    local threshold="$2"
    local output_file="$3"
    
    log "Detecting anomalies (threshold: ${threshold}%)..."
    
    echo -e "ID\tData1\tData2\tChange%" > "$output_file"
    
    awk -F'\t' -v thresh="$threshold" '
    NR > 1 {
        # Extract percent change (remove % sign)
        gsub(/%/, "", $5)
        pct = $5
        
        # Remove + sign for comparison
        gsub(/\+/, "", pct)
        
        if (pct > thresh || pct < -thresh) {
            print $0
        }
    }' "$data_file" >> "$output_file"
    
    local anomalies=$(($(wc -l < "$output_file") - 1))
    log "Found $anomalies anomalies"
}

# ============================================
# Report Generation
# ============================================
generate_summary() {
    local data_file="$1"
    local output_file="$2"
    
    log "Generating summary report..."
    
    {
        echo "======================================="
        echo "Data Correlation Summary Report"
        echo "Generated: $(date)"
        echo "======================================="
        echo ""
        
        # Statistics
        awk -F'\t' '
        NR > 1 {
            sum_d1 += $2
            sum_d2 += $3
            sum_diff += $4
            count++
            
            if ($2 > max_d1 || NR == 2) max_d1 = $2
            if ($2 < min_d1 || NR == 2) min_d1 = $2
            if ($3 > max_d2 || NR == 2) max_d2 = $3
            if ($3 < min_d2 || NR == 2) min_d2 = $3
        }
        END {
            printf "Total Records: %d\n\n", count
            
            printf "Data1 Statistics:\n"
            printf "  Average: %.2f\n", sum_d1/count
            printf "  Min: %.2f\n", min_d1
            printf "  Max: %.2f\n\n", max_d1
            
            printf "Data2 Statistics:\n"
            printf "  Average: %.2f\n", sum_d2/count
            printf "  Min: %.2f\n", min_d2
            printf "  Max: %.2f\n\n", max_d2
            
            printf "Change Statistics:\n"
            printf "  Average Difference: %+.2f\n", sum_diff/count
            printf "  Total Change: %+.2f\n", sum_diff
        }' "$data_file"
        
        echo ""
        echo "======================================="
        
    } > "$output_file"
    
    log "Summary saved to: $output_file"
}

# ============================================
# Main Processing
# ============================================
main() {
    log "========================================="
    log "Data Correlation Tool"
    log "========================================="
    
    # Sample data creation (in real use, these would exist)
    cat > "$DATA_DIR/ids.txt" << 'EOF'
ID
USER001
USER002
USER003
USER004
USER005
EOF
    
    cat > "$DATA_DIR/baseline.txt" << 'EOF'
Baseline
100.50
150.25
200.00
175.75
225.50
EOF
    
    cat > "$DATA_DIR/current.txt" << 'EOF'
Current
105.25
148.00
210.50
175.80
250.00
EOF
    
    # Validate inputs
    validate_files \
        "$DATA_DIR/ids.txt" \
        "$DATA_DIR/baseline.txt" \
        "$DATA_DIR/current.txt"
    
    # Correlate data
    correlate_data \
        "$DATA_DIR/ids.txt" \
        "$DATA_DIR/baseline.txt" \
        "$DATA_DIR/current.txt" \
        "$OUTPUT_DIR/correlation.tsv"
    
    # Detect anomalies (>10% change)
    detect_anomalies \
        "$OUTPUT_DIR/correlation.tsv" \
        10 \
        "$OUTPUT_DIR/anomalies.tsv"
    
    # Generate summary
    generate_summary \
        "$OUTPUT_DIR/correlation.tsv" \
        "$OUTPUT_DIR/summary.txt"
    
    # Display results
    log ""
    log "Results:"
    log "  Correlation: $OUTPUT_DIR/correlation.tsv"
    log "  Anomalies: $OUTPUT_DIR/anomalies.tsv"
    log "  Summary: $OUTPUT_DIR/summary.txt"
    log ""
    
    cat "$OUTPUT_DIR/summary.txt"
    
    log "========================================="
    log "Processing complete"
}

main "$@"
                

Script output:

[2025-12-14 16:45:00] =========================================
[2025-12-14 16:45:00] Data Correlation Tool
[2025-12-14 16:45:00] =========================================
[2025-12-14 16:45:00] Validating input files...
[2025-12-14 16:45:00]   ./ids.txt: 6 lines
[2025-12-14 16:45:00]   ./baseline.txt: 6 lines
[2025-12-14 16:45:00]   ./current.txt: 6 lines
[2025-12-14 16:45:00] Correlating data...
[2025-12-14 16:45:00] Processed 5 data rows
[2025-12-14 16:45:00] Detecting anomalies (threshold: 10%)...
[2025-12-14 16:45:00] Found 2 anomalies
[2025-12-14 16:45:00] Generating summary report...

=======================================
Data Correlation Summary Report
Generated: Sun Dec 14 16:45:00 EST 2025
=======================================

Total Records: 5

Data1 Statistics:
  Average: 170.40
  Min: 100.50
  Max: 225.50

Data2 Statistics:
  Average: 177.91
  Min: 105.25
  Max: 250.00

Change Statistics:
  Average Difference: +7.51
  Total Change: +37.55

=======================================

# correlation.tsv:
ID       Data1   Data2   Difference  Percent_Change
USER001  100.50  105.25  +4.75       +4.7%
USER002  150.25  148.00  -2.25       -1.5%
USER003  200.00  210.50  +10.50      +5.3%
USER004  175.75  175.80  +0.05       +0.0%
USER005  225.50  250.00  +24.50      +10.9%

# anomalies.tsv shows USER005 (10.9% change)

Note:

This production script demonstrates best practices: input validation, logging, error handling, modular functions, and comprehensive reporting. It uses paste as the foundation for correlating multiple data sources and performing analysis.

Tips & Best Practices

Default Delimiter is Tab

Paste uses tab by default, which may not be visible in some contexts. Use -d',' for CSV output or cat -A to see tabs as ^I in output for verification.

Serial Mode for Row-to-Column

Use paste -s to convert multiple lines into a single line. Perfect for creating comma-separated lists from vertical data: paste -s -d','

Handle Different File Lengths

When files have different lengths, paste outputs empty fields for shorter files. Always validate line counts when data integrity matters: wc -l file1 file2

Use - for stdin in Pipelines

The dash represents stdin, allowing paste in pipelines. Combine with process substitution <(command) for powerful data merging from multiple sources.

Delimiter Cycling for Patterns

Multiple delimiter characters cycle through columns. paste -d':;,' uses : for first separator, ; for second, , for third, then wraps back to :

Combine with awk for Calculations

Paste creates columns, awk processes them. This combo is powerful: paste file1 file2 | awk '{print $1, $2, $1+$2}'

Create Multiple Columns from One File

Use multiple stdin references to create columns: cat file | paste - - - creates 3 columns from sequential lines. Great for reformatting data.

Preserve Tabs in Output

When viewing paste output, tabs may appear as spaces. Redirect to a file or pipe to cat -A to verify actual delimiters used.

Consider column for Alignment

Pipe paste output through column -t for human-readable aligned columns. Perfect for reports: paste file1 file2 | column -t

Synopsis

Description

Key Concepts

Detailed Examples

Basic Side-by-Side Merge

Custom Delimiters

Serial Mode (-s) - Convert Rows to Columns

Using stdin with Paste

Handling Files of Different Lengths

Multiple Delimiter Cycling

Data Transformation and Reformatting

Combining paste with Other Commands

Real-World Log Processing

Advanced Data Correlation Script

Tips & Best Practices

Related Commands