LearnLinux › LearnLinux › CMDS

sed Command Examples

Stream Editor for Filtering and Transforming Text

About sed

The sed (stream editor) command is a powerful text processing utility that performs basic text transformations on an input stream (a file or input from a pipeline). Unlike interactive text editors, sed processes text in a single pass, making it extremely efficient for automated text manipulation tasks. It's one of the fundamental tools in Unix/Linux text processing alongside grep and awk.

Key Capabilities:

Note: sed is part of the GNU coreutils and is available on all Linux/Unix systems by default. It uses regular expressions (regex) for pattern matching, similar to grep and awk.

Command Syntax

sed [OPTIONS] 'command' file(s)
sed [OPTIONS] -e 'command1' -e 'command2' file(s)
sed [OPTIONS] -f script.sed file(s)

Common Options

Option Description
-n Suppress automatic output (quiet mode)
-e Add script command (allows multiple commands)
-f Read commands from file
-i Edit files in-place (modify original file)
-i.bak Edit in-place and create backup with .bak extension
-r or -E Use extended regular expressions
--version Display version information

Basic sed Commands

Command Description Example
s/pattern/replacement/ Substitute (replace) text s/old/new/
d Delete lines /pattern/d
p Print lines /pattern/p
a Append text after line /pattern/a\text
i Insert text before line /pattern/i\text
c Change (replace) entire line /pattern/c\text
y/src/dst/ Transliterate (character substitution) y/abc/ABC/

Detailed Examples

Example 1: Basic Search and Replace

echo "Hello World" | sed 's/World/Universe/'
Hello Universe
The most basic sed operation: substitute (replace) text. The syntax is s/pattern/replacement/:
  • s - substitute command
  • /World/ - pattern to search for
  • /Universe/ - replacement text
  • By default, replaces only the first occurrence per line
This is the most commonly used sed operation, essential for text transformations in scripts.
Tip: You can use any delimiter instead of /, such as s|World|Universe| or s#World#Universe#. This is useful when your pattern contains slashes (like file paths).

Example 2: Global Replace (All Occurrences)

echo "The cat in the hat sat on the mat" | sed 's/at/AT/g'
The cAT in the hAT sAT on the mAT
The g flag (global) replaces all occurrences on each line, not just the first:
  • Without g: only first match per line is replaced
  • With g: all matches on line are replaced
  • Essential for comprehensive text transformations
Compare without g: sed 's/at/AT/' would output "The cAT in the hat sat on the mat"
Other flags:
  • i - Case-insensitive matching
  • 2 - Replace only second occurrence
  • p - Print if substitution made
  • w file - Write changed lines to file

Example 3: In-Place File Editing

# Create sample file
echo "DEBUG: Starting process" > app.log
echo "ERROR: Connection failed" >> app.log
echo "DEBUG: Retrying connection" >> app.log

# Replace in-place with backup
sed -i.bak 's/DEBUG/INFO/g' app.log

# View result
cat app.log
INFO: Starting process
ERROR: Connection failed
INFO: Retrying connection
The -i option enables in-place editing, modifying the original file:
  • -i - Edit file directly (no backup)
  • -i.bak - Edit file and save original as filename.bak
  • -i.backup - Use custom backup extension
  • Creates app.log.bak before modifying app.log
This is crucial for automated configuration updates and log processing.
Warning: Always create a backup when using -i in production! Use -i.bak or test on copies first. In-place editing is irreversible without backups.

Example 4: Delete Lines Matching Pattern

cat << EOF | sed '/^#/d'
# This is a comment
ServerName localhost
# Another comment
Port 8080
EOF
ServerName localhost
Port 8080
The d command deletes lines matching the pattern:
  • /^#/d - Delete lines starting with # (comments)
  • /pattern/d - Delete lines containing pattern
  • /^$/d - Delete empty lines
  • /^\\s*$/d - Delete blank lines (including whitespace)
Common uses: removing comments, cleaning log files, filtering configuration files.

More delete examples:

# Delete lines 2-4
sed '2,4d' file.txt

# Delete last line
sed '$d' file.txt

# Delete lines NOT matching pattern
sed '/pattern/!d' file.txt

Example 5: Print Specific Lines

cat << EOF | sed -n '2,3p'
Line 1
Line 2
Line 3
Line 4
EOF
Line 2
Line 3
The -n option suppresses automatic output, and p prints specific lines:
  • sed -n '2p' - Print only line 2
  • sed -n '2,5p' - Print lines 2 through 5
  • sed -n '$p' - Print last line
  • sed -n '/pattern/p' - Print lines matching pattern (like grep)
  • sed -n '1~2p' - Print odd-numbered lines (every 2nd line starting at 1)
This provides powerful line extraction capabilities beyond basic grep functionality.
Tip: sed -n '2,5p' file.txt is similar to head -n 5 file.txt | tail -n 4 but more efficient and flexible.

Example 6: Multiple Commands with -e

echo "Hello World From Linux" | sed -e 's/Hello/Hi/' -e 's/World/Universe/' -e 's/Linux/Unix/'
Hi Universe From Unix
Use -e to chain multiple sed commands in sequence:
  • Commands execute in order from left to right
  • Each command operates on the output of the previous one
  • Useful for complex multi-step transformations
  • Alternative: separate commands with semicolon: sed 's/Hello/Hi/; s/World/Universe/'

Practical example - Log sanitization:

sed -e 's/password=[^&]*/password=REDACTED/' \
-e 's/token=[^&]*/token=REDACTED/' \
-e 's/ssn=[0-9-]*/ssn=XXX-XX-XXXX/' \
access.log
This removes sensitive information from logs before sharing with support teams.

Example 7: Using Regular Expressions and Backreferences

echo "Date: 2024-11-13" | sed -E 's/([0-9]{4})-([0-9]{2})-([0-9]{2})/\3\/\2\/\1/'
Date: 13/11/2024
Advanced pattern matching with capture groups and backreferences:
  • -E - Use extended regex (makes syntax cleaner)
  • ([0-9]{4}) - Capture 4 digits (year) as \1
  • ([0-9]{2}) - Capture 2 digits (month) as \2
  • ([0-9]{2}) - Capture 2 digits (day) as \3
  • \3\/\2\/\1 - Reorder captured groups (day/month/year)
Backreferences allow you to capture parts of patterns and rearrange them in the replacement.
Tip: Without -E, you need to escape parentheses: s/\([0-9]\{4\}\)-\([0-9]\{2\}\)/\2\/\1/. Use -E for more readable patterns!

Example 8: Insert and Append Text

cat << EOF | sed '/ServerName/a\ ServerAlias www.example.com'
<VirtualHost *:80>
ServerName example.com
DocumentRoot /var/www/html
</VirtualHost>
EOF
<VirtualHost *:80>
ServerName example.com
ServerAlias www.example.com
DocumentRoot /var/www/html
</VirtualHost>
Insert or append text at specific locations:
  • a\text - Append text after matching line
  • i\text - Insert text before matching line
  • c\text - Change (replace) entire matching line
  • Useful for automated configuration file updates

More insertion examples:

# Insert header at beginning of file
sed '1i\# Configuration File' config.txt

# Append footer at end of file
sed '$a\# End of Configuration' config.txt

# Change matching line
sed '/old_setting/c\new_setting=value' config.txt

Example 9: Address Ranges and Complex Patterns

cat << EOF | sed '/START/,/END/s/^/ /'
Normal line
START section
Line 1 in section
Line 2 in section
END section
Normal line
EOF
Normal line
START section
Line 1 in section
Line 2 in section
END section
Normal line
Address ranges allow operations on sections of text:
  • /START/,/END/ - From line matching START to line matching END
  • 10,20 - Lines 10 through 20
  • 10,+5 - Line 10 plus next 5 lines
  • /pattern/,+3 - Matching line plus next 3 lines
In this example, we indent all lines between START and END markers by prepending 4 spaces.

More range examples:

# Delete section between markers
sed '/BEGIN/,/END/d' file.txt

# Comment out lines 10-20
sed '10,20s/^/# /' file.txt

# Process every line from pattern to end of file
sed '/START/,$s/old/new/' file.txt

Example 10: Complete Text Processing Script

#!/bin/bash
# Comprehensive log processing and sanitization script

LOG_FILE="application.log"
OUTPUT_FILE="sanitized.log"

cat "$LOG_FILE" | sed '
# Remove timestamp prefix (format: [YYYY-MM-DD HH:MM:SS])
s/^\[[0-9-]* [0-9:]*\] //;

# Standardize log levels
s/\[DEBUG\]/[INFO]/g;
s/\[WARN\]/[WARNING]/g;

# Redact sensitive information
s/password=[^ ]*/password=REDACTED/gi;
s/api_key=[^ ]*/api_key=REDACTED/gi;
s/token=[^ ]*/token=REDACTED/gi;
s/\b[0-9]{3}-[0-9]{2}-[0-9]{4}\b/XXX-XX-XXXX/g;

# Remove empty lines
/^$/d;

# Remove comment lines
/^#/d;

# Normalize spacing
s/ */ /g;
s/^ *//;
s/ *$//;

# Convert specific error codes
s/ERR_1001/Database Connection Error/g;
s/ERR_2001/Authentication Failure/g;

# Add line numbers to remaining lines
=' | sed 'N;s/\n/: /' > "$OUTPUT_FILE"

echo "Sanitized log saved to $OUTPUT_FILE"

# Show summary statistics
echo "Total lines: $(wc -l < "$OUTPUT_FILE")"
echo "Warnings: $(grep -c WARNING "$OUTPUT_FILE")"
echo "Errors: $(grep -c ERROR "$OUTPUT_FILE")"
This comprehensive script demonstrates real-world sed usage:
  • Multiple substitution patterns for data normalization
  • Security-focused sensitive data redaction
  • Pattern-based line deletion for cleaning
  • Whitespace normalization and formatting
  • Error code translation for readability
  • Line numbering for reference
  • Pipelined with other Unix tools (grep, wc)
This pattern is typical in DevOps, security auditing, and log analysis workflows.

Regular Expression Patterns in sed

Common Regex Metacharacters

Pattern Meaning Example
^ Start of line ^Hello matches lines starting with "Hello"
$ End of line World$ matches lines ending with "World"
. Any single character a.c matches "abc", "aXc", etc.
* Zero or more of preceding ab*c matches "ac", "abc", "abbc"
\+ One or more of preceding ab\+c matches "abc", "abbc" but not "ac"
\? Zero or one of preceding ab\?c matches "ac", "abc"
[abc] Any character in set [aeiou] matches any vowel
[^abc] Any character NOT in set [^0-9] matches non-digits
\| Alternation (OR) cat\|dog matches "cat" or "dog"
\( \) Grouping & capture \([0-9]\{3\}\) captures 3 digits

Character Classes

Class Meaning
[[:alnum:]] Alphanumeric characters (letters and digits)
[[:alpha:]] Alphabetic characters
[[:digit:]] Digits (0-9)
[[:space:]] Whitespace (space, tab, newline)
[[:upper:]] Uppercase letters
[[:lower:]] Lowercase letters

Real-World Use Cases

Use Case 1: Configuration File Updates

# Update database port in configuration
sed -i.bak 's/^port=.*/port=5432/' database.conf

# Enable debug mode
sed -i 's/^#\(debug=true\)/\1/' app.conf

# Update multiple settings
sed -i -e 's/^server=.*/server=prod.example.com/' \
-e 's/^timeout=.*/timeout=30/' \
-e 's/^retries=.*/retries=5/' \
config.ini

Use Case 2: CSV Data Processing

# Remove header line
sed '1d' data.csv > data_no_header.csv

# Extract specific columns (1st and 3rd)
sed 's/^\([^,]*\),[^,]*,\([^,]*\).*/\1,\2/' data.csv

# Replace commas with pipes
sed 's/,/|/g' data.csv > data.psv

Use Case 3: HTML/XML Processing

# Remove HTML tags
sed 's/<[^>]*>//g' webpage.html > plain_text.txt

# Extract URLs from HTML
sed -n 's/.*href="\([^"]*\)".*/\1/p' webpage.html

# Add target="_blank" to all links
sed 's/<a href=/<a target="_blank" href=/g' page.html

Use Case 4: Code Refactoring

# Rename function calls across project
find . -name "*.py" -exec sed -i 's/old_function/new_function/g' {} +

# Update import statements
sed -i 's/from old_module import/from new_module import/' *.py

# Convert tabs to spaces
sed -i 's/\t/ /g' *.py

Advanced Techniques

Technique 1: Hold Space and Pattern Space

# Reverse file (print lines in reverse order)
sed '1!G;h;$!d' file.txt

# Remove duplicate consecutive lines
sed '$!N; /^\(.*\)\n\1$/!P; D' file.txt
sed maintains two buffers: pattern space (current line) and hold space (temporary storage). Advanced sed users leverage these for complex transformations like reversing files or removing duplicates.

Technique 2: Conditional Execution

# Apply substitution only if line contains pattern
sed '/ERROR/s/^/*** /' logfile.txt

# Apply substitution if line does NOT contain pattern
sed '/SUCCESS/!s/^/[FAILED] /' results.txt

Technique 3: Multi-line Processing

# Join lines ending with backslash
sed -e :a -e '/\\$/N; s/\\\n//; ta' file.txt

# Replace newlines with spaces (entire file on one line)
sed ':a;N;$!ba;s/\n/ /g' file.txt

Performance Considerations

Performance Tips:

Common Pitfalls and Solutions

Pitfall 1: Special Characters in Patterns

# Problem: Trying to replace dots (matches any character)
sed 's/192.168.1.1/10.0.0.1/' file.txt # WRONG

# Solution: Escape special characters
sed 's/192\.168\.1\.1/10.0.0.1/' file.txt # CORRECT

Pitfall 2: Greedy Matching

# Problem: Removing HTML tags removes too much
echo "<b>Hello</b> <i>World</i>" | sed 's/<.*>//'
# Output: (empty - matched from first < to last >)

# Solution: Use non-greedy pattern
echo "<b>Hello</b> <i>World</i>" | sed 's/<[^>]*>//g'
# Output: Hello World

Pitfall 3: In-Place Editing Without Backup

Never do this in production:
sed -i 's/old/new/g' critical_config.conf # Dangerous!
Always create backups:
sed -i.backup 's/old/new/g' critical_config.conf # Safe

Troubleshooting

Testing sed Commands Safely

# Always test without -i first
sed 's/old/new/g' file.txt | less

# Create a test file
cp important.conf test.conf
sed -i 's/old/new/g' test.conf
diff important.conf test.conf

# Use -i with backup
sed -i.bak 's/old/new/g' important.conf
diff important.conf important.conf.bak

sed vs. Other Tools

Task sed Alternative
Simple search/replace ✅ Best choice awk, perl
Pattern matching only ❌ Use grep grep (faster, simpler)
Column/field processing ❌ Use awk awk (more powerful)
Complex logic ❌ Use scripting language Python, Perl
In-place file editing ✅ Excellent perl -i, Python
Stream processing ✅ Perfect awk, Perl

Related Commands

Quick Reference Card

Most Common sed Commands

# Basic substitution
sed 's/old/new/' file.txt

# Global substitution
sed 's/old/new/g' file.txt

# In-place editing with backup
sed -i.bak 's/old/new/g' file.txt

# Delete lines matching pattern
sed '/pattern/d' file.txt

# Print specific lines
sed -n '10,20p' file.txt

# Multiple commands
sed -e 's/a/A/' -e 's/b/B/' file.txt

# Comment out lines
sed 's/^/# /' file.txt

# Remove comments
sed '/^#/d' file.txt

# Remove empty lines
sed '/^$/d' file.txt