Regular Expressions (regex) Reference

About Regular Expressions

Regular expressions (regex) are patterns used to match character combinations in strings. They're fundamental to text processing in Linux, used by tools like grep, sed, awk, perl, and many programming languages.

Common flavors: Basic Regular Expressions (BRE), Extended Regular Expressions (ERE), Perl Compatible Regular Expressions (PCRE)

Primary tools: grep (searching), sed (editing), awk (processing), perl (advanced manipulation)

Metacharacter	Description	Example
`.`	Matches any single character	`a.c` matches "abc", "a5c", "a c"
`*`	Matches 0 or more of preceding	`ab*c` matches "ac", "abc", "abbc"
`+`	Matches 1 or more of preceding (ERE)	`ab+c` matches "abc", "abbc" (not "ac")
`?`	Matches 0 or 1 of preceding (ERE)	`ab?c` matches "ac", "abc"
`^`	Anchors to start of line	`^The` matches "The" at line start
`$`	Anchors to end of line	`end$` matches "end" at line end
`[]`	Character class (matches any inside)	`[aeiou]` matches any vowel
`[^]`	Negated character class	`[^0-9]` matches non-digits
`\`	Escapes special characters	`\.` matches literal period
`\|`	Alternation (OR) (ERE)	`cat\|dog` matches "cat" or "dog"
`()`	Grouping and capturing	`(ab)+` matches "ab", "abab", etc.

Example 1: Basic Pattern Matching with grep

Use simple regex patterns to search for text in files. Grep is the most common tool for regex searching in Linux.

                    # Sample file: users.txt

john.doe@email.com

mary_smith@company.org

bob123@test.net

invalid-email

# Find lines containing "@"

grep "@" users.txt

# Find lines ending with ".com"

grep "\.com$" users.txt

# Find lines starting with "john"

grep "^john" users.txt

# Case-insensitive search

grep -i "JOHN" users.txt

Output (ending with .com):

john.doe@email.com

Note:

The dot (.) is a special character, so we escape it with backslash (\.) to match a literal period. The $ anchors the pattern to the end of the line.

Example 2: Character Classes and Ranges

Character classes let you match any character from a set. Ranges like [a-z] and [0-9] are commonly used.

                    # Sample data file:

abc123

xyz789

test

12345

# Match lines with any digit

grep "[0-9]" data.txt

# Match lines with only digits

grep "^[0-9]*$" data.txt

# Match lines starting with lowercase letter

grep "^[a-z]" data.txt

# Match lines with uppercase OR lowercase

grep "[A-Za-z]" data.txt

# Match lines with vowels

grep "[aeiou]" data.txt

# Match lines WITHOUT digits (negated class)

grep "^[^0-9]*$" data.txt

Output (only digits):

12345

Note:

[^0-9] means "not a digit" - the caret inside brackets negates the class. [a-z] matches lowercase, [A-Z] uppercase, [0-9] digits.

Example 3: Quantifiers (*, +, ?, {n,m})

Quantifiers specify how many times a pattern should match. Use -E flag for extended regex in grep.

                    # Sample file: patterns.txt

ac

abc

abbc

abbbc

# Match "a" followed by zero or more "b" then "c"

grep -E "ab*c" patterns.txt

# Match "a" followed by one or more "b" then "c"

grep -E "ab+c" patterns.txt

# Match "a" followed by optional "b" then "c"

grep -E "ab?c" patterns.txt

# Match "a" followed by exactly 2 "b" then "c"

grep -E "ab{2}c" patterns.txt

# Match "a" followed by 2 to 4 "b" then "c"

grep -E "ab{2,4}c" patterns.txt

# Find repeated digits (3 or more)

grep -E "[0-9]{3,}" numbers.txt

Output (ab+c matches):

abc

abbc

abbbc

Note:

Without -E, grep uses basic regex where +, ?, {}, and () need backslashes: \+, \?, \{2\}, . Extended regex (-E) is cleaner for complex patterns.

Example 4: Anchors and Word Boundaries

Anchors like ^ and $ match positions, not characters. Word boundaries \b match between word and non-word characters.

                    # Sample file: text.txt

The cat sat on the mat

catch the ball

category of items

# Match "cat" only at start of line

grep "^cat" text.txt

# Match "mat" only at end of line

grep "mat$" text.txt

# Match entire line (start to end)

grep "^The.*mat$" text.txt

# Match "cat" as a complete word (not in catch/category)

grep -w "cat" text.txt

# Or using word boundaries:

grep -E "\bcat\b" text.txt

# Match empty lines

grep "^$" file.txt

# Match lines with only whitespace

grep "^[[:space:]]*$" file.txt

Output (word "cat" only):

The cat sat on the mat

Note:

-w flag is a grep shortcut for word boundaries. \b matches between \w (alphanumeric) and \W (non-alphanumeric). ^ and $ are zero-width assertions.

Example 5: Alternation and Grouping

Use | for OR conditions and parentheses for grouping patterns together.

                    # Sample log file:

ERROR: disk full

WARNING: low memory

INFO: system started

ERROR: connection timeout

# Match lines with ERROR or WARNING

grep -E "ERROR|WARNING" log.txt

# Match lines with cat or dog

grep -E "cat|dog" animals.txt

# Group patterns: match gray or grey

grep -E "gr(a|e)y" text.txt

# Match file extensions

grep -E "\.(jpg|png|gif)$" filelist.txt

# Match repeated groups

echo "abcabc" | grep -E "(abc){2}"

# Complex: match error levels with codes

grep -E "(ERROR|WARN|INFO):\s+[0-9]{3}" log.txt

Output (ERROR or WARNING):

ERROR: disk full

WARNING: low memory

ERROR: connection timeout

Note:

Parentheses create capture groups (used in replacements) and control precedence. Without grouping, "cat|dog" matches "cat" or "dog", while "c(at|og)" matches "cat" or "cog".

Example 6: Validating Email Addresses and URLs

Practical regex for validating common formats. Note that perfect email validation is complex; these are practical patterns.

                    # Validate basic email format

grep -E "^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}$" emails.txt

# Extract email addresses from text

grep -Eo "[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}" document.txt

# Match URLs starting with http/https

grep -E "https?://[A-Za-z0-9./-]+" urls.txt

# More complete URL pattern

grep -E "https?://([A-Za-z0-9-]+\.)*[A-Za-z0-9-]+(/[A-Za-z0-9-._~:/?#@!$&'()*+,;=%]*)?" urls.txt

# Match IP addresses (simple pattern)

grep -E "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" network.txt

# Better IP address validation (0-255 range)

grep -E "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)" ips.txt

Valid Emails:

user@example.com

john.doe@company.co.uk

test_user123@mail-server.org

Note:

These patterns work for most cases but aren't RFC-compliant. The -o flag extracts only the matching portion. Use validation libraries for production code.

Example 7: Search and Replace with sed

Sed uses regex for powerful search-and-replace operations on text streams.

                    # Sample file: data.txt

John Smith, 555-1234

Mary Jones, 555-5678

# Replace first occurrence per line

sed 's/555/XXX/' data.txt

# Replace all occurrences (global flag)

sed 's/555/XXX/g' data.txt

# Replace only on lines matching pattern

sed '/John/s/555/XXX/' data.txt

# Use capture groups for reordering

# Change "FirstName LastName" to "LastName, FirstName"

echo "John Smith" | sed -E 's/([A-Za-z]+) ([A-Za-z]+)/\2, \1/'

# Extract phone numbers and format them

sed -E 's/.*([0-9]{3})-([0-9]{4}).*/(\1) \2/' data.txt

# Remove all digits

sed 's/[0-9]//g' data.txt

# Add prefix to lines matching pattern

sed '/ERROR/s/^/CRITICAL: /' log.txt

Output (global replacement):

John Smith, XXX-1234

Mary Jones, XXX-5678

Note:

Capture groups \1, \2, etc. reference matched portions. Use -E for extended regex in sed. Add -i flag to modify files in-place: sed -i 's/old/new/g' file.txt

Example 8: Advanced Pattern Matching with Perl

Perl offers the most powerful regex implementation with features like lookahead, lookbehind, and non-capturing groups.

                    # Non-capturing groups (?: ... )

echo "gray grey" | perl -ne 'print if /gr(?:a|e)y/'

# Positive lookahead (?= ... )

# Match "test" only if followed by "123"

echo "test123 test456" | perl -ne 'print if /test(?=123)/'

# Negative lookahead (?! ... )

# Match "test" only if NOT followed by "123"

echo "test123 test456" | perl -ne 'print if /test(?!123)/'

# Positive lookbehind (?<= ... )

# Match digits preceded by "$"

echo "Price: $100" | perl -ne 'print "$&\n" if /(?<=\$)[0-9]+/'

# Named capture groups

echo "2025-11-12" | perl -ne '/

    /(?<year>\d{4})-(?<month>\d{2})-(?<day>\d{2})/;

    print "Year: $+{year}, Month: $+{month}, Day: $+{day}\n"

'

# Case-insensitive inline modifier

perl -ne 'print if /(?i)error/' log.txt

# Multiline mode

perl -ne 'print if /(?m)^ERROR/' log.txt

Output (named groups):

Year: 2025, Month: 11, Day: 12

Note:

Perl regex is most powerful but less portable. Use for complex tasks. $& contains the matched text. -n processes input line by line, -e executes the code.

Example 9: Working with Log Files

Practical regex patterns for parsing and analyzing log files, extracting timestamps, IPs, and status codes.

                    # Sample Apache access log:

# 192.168.1.1 - - [12/Nov/2025:10:30:45 +0000] "GET /index.html HTTP/1.1" 200 1234

# Extract IP addresses

grep -Eo "^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" access.log

# Find all 404 errors

grep -E '" 404 ' access.log

# Extract timestamps

grep -Eo "\[[0-9]{2}/[A-Z][a-z]{2}/[0-9]{4}:[0-9]{2}:[0-9]{2}:[0-9]{2}" access.log

# Find requests from specific IP range

grep -E "^192\.168\.1\." access.log

# Extract URLs from GET requests

grep -Eo 'GET [^ ]+' access.log | sed 's/GET //'

# Find errors by status code (4xx and 5xx)

grep -E '" [45][0-9]{2} ' access.log

# Count requests by hour

grep -Eo "[0-9]{2}:[0-9]{2}:[0-9]{2}" access.log | \

    cut -d: -f1 | sort | uniq -c

# Extract failed SSH login attempts

grep -E "Failed password.*from [0-9.]+" /var/log/auth.log

Output (IP addresses):

192.168.1.1

192.168.1.15

10.0.0.50

203.0.113.42

Note:

Use -o flag to extract only matching parts. Combine with other tools (sort, uniq, awk) for analysis. Consider tools like logwatch or goaccess for comprehensive log analysis.

Example 10: Data Validation and Extraction

Use regex for validating structured data like phone numbers, dates, credit cards, and extracting specific fields.

                    # Validate US phone numbers (multiple formats)

grep -E "^(\+1[-.]?)?\(?\d{3}\)?[-.]?\d{3}[-.]?\d{4}$" phones.txt

# Matches: 555-123-4567, (555) 123-4567, 5551234567, +1-555-123-4567

# Validate dates (MM/DD/YYYY or MM-DD-YYYY)

grep -E "^(0[1-9]|1[0-2])[-/](0[1-9]|[12][0-9]|3[01])[-/][0-9]{4}$" dates.txt

# Extract dates from text in various formats

grep -Eo "[0-9]{4}-[0-9]{2}-[0-9]{2}" text.txt  # YYYY-MM-DD

grep -Eo "[0-9]{2}/[0-9]{2}/[0-9]{4}" text.txt  # MM/DD/YYYY

# Validate credit card numbers (basic Luhn check not included)

grep -E "^[0-9]{4}[-]?[0-9]{4}[-]?[0-9]{4}[-]?[0-9]{4}$" cards.txt

# Extract hashtags from social media text

grep -Eo "#[A-Za-z0-9_]+" tweets.txt

# Validate MAC addresses

grep -E "^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2})$" macs.txt

# Extract dollar amounts

grep -Eo "\$[0-9]{1,3}(,[0-9]{3})*(\.[0-9]{2})?" prices.txt

# Validate strong passwords (8+ chars, upper, lower, digit, special)

grep -E "^(?=.*[a-z])(?=.*[A-Z])(?=.*[0-9])(?=.*[!@#$%^&*]).{8,}$" passwords.txt

Valid Phone Numbers:

555-123-4567

(555) 123-4567

5551234567

+1-555-123-4567

Note:

These are practical patterns for common formats. For production validation, consider locale-specific rules and use established validation libraries. The password regex uses lookaheads to check multiple conditions.

Additional Tips & Best Practices

Escaping special characters: In basic regex: . * [ ] ^ $ \. In extended: + ? { } | ( ) also need escaping if used literally
POSIX character classes: [:alnum:], [:alpha:], [:digit:], [:lower:], [:upper:], [:space:], [:punct:] are portable across systems
Greedy vs non-greedy: .* is greedy (matches maximum), .*? is non-greedy (matches minimum) in Perl/PCRE
Testing tools: Use regex101.com or regexr.com for testing and debugging patterns with explanations
Performance: Avoid catastrophic backtracking with nested quantifiers. Pattern ^.*abc is slower than ^[^a]*abc
Common shortcuts: \d (digit), \w (word char), \s (whitespace), \D \W \S (negated versions) in Perl/PCRE
Multi-line matching: Use -z flag in grep for null-terminated records, allows matching across lines
Case sensitivity: Use -i flag in grep, /i modifier in Perl, I flag in sed for case-insensitive matching
Raw strings: In scripts, use single quotes to avoid shell interpretation of backslashes and special chars
Debugging regex: Build patterns incrementally, test each component separately before combining
Documentation: Comment complex regex patterns. Future you will thank present you!
Alternatives: For complex parsing (JSON, XML, HTML), use specialized parsers instead of regex