Example 1
Trace a Simple Command
$ strace ls
execve("/usr/bin/ls", ["ls"], 0x7ffd... /* 45 vars */) = 0
brk(NULL) = 0x55d123456000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=98765, ...}) = 0
close(3) = 0
openat(AT_FDCWD, ".", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3
getdents64(3, /* 15 entries */, 32768) = 480
write(1, "file1.txt\nfile2.txt\n", 20) = 20
close(3) = 0
exit_group(0) = ?
Traces all system calls made by the ls command. Shows every interaction with the kernel including file opens, reads, writes, and memory operations. Each line shows the system call name, arguments, and return value. Essential for understanding what a program is actually doing at the system level.
Understanding Output: Format is syscall(args) = return_value
Example 2
Attach to Running Process
$ strace -p 1234
strace: Process 1234 attached
select(5, [4], NULL, NULL, {tv_sec=30, tv_usec=0}) = 1 (in [4], left {tv_sec=25, tv_usec=123456})
read(4, "GET /index.html HTTP/1.1\r\n", 4096) = 26
write(4, "HTTP/1.1 200 OK\r\n", 17) = 17
close(4) = 0
Attaches strace to an already running process using its PID with the '-p' option. Extremely useful for debugging live applications without restarting them. You can watch system calls in real-time to diagnose issues in production environments.
Permission: Usually requires root/sudo to attach to processes owned by other users
Example 3
Save Output to File
$ strace -o trace.log ls
# Output written to trace.log instead of stderr
$ cat trace.log
execve("/usr/bin/ls", ["ls"], 0x7ffd...) = 0
brk(NULL) = 0x55d123456000
...
Redirects strace output to a file using the '-o' option instead of displaying on stderr. Essential for analyzing long traces or when you need to save diagnostic information for later review. Makes output easier to search and share with others.
Analysis: Use grep, awk, or other text tools to analyze saved traces
Example 4
Trace Specific System Calls
$ strace -e open,read,write cat file.txt
openat(AT_FDCWD, "file.txt", O_RDONLY) = 3
read(3, "Hello World\n", 131072) = 12
write(1, "Hello World\n", 12) = 12
read(3, "", 131072) = 0
+++ exited with 0 +++
Filters output to show only specific system calls using the '-e' option. This dramatically reduces noise when you're investigating particular operations. You can specify multiple calls separated by commas. Perfect for focusing on file I/O, network operations, or memory management.
Common Filters: -e file (all file ops), -e network (network calls), -e signal (signals)
Example 5
Show Timestamps
$ strace -t ls
10:45:23 execve("/usr/bin/ls", ["ls"], 0x7ffd...) = 0
10:45:23 brk(NULL) = 0x55d123456000
10:45:23 openat(AT_FDCWD, ".", O_RDONLY) = 3
10:45:23 getdents64(3, /* 15 entries */, 32768) = 480
10:45:23 write(1, "file1.txt\n", 10) = 10
Adds timestamps to each system call using the '-t' option. Shows the time each system call occurs, which is invaluable for performance analysis and understanding timing issues. Use '-tt' for microsecond precision or '-ttt' for Unix timestamp format.
Time Options: -t (HH:MM:SS), -tt (HH:MM:SS.microseconds), -ttt (Unix timestamp)
Example 6
Show Time Spent in Each System Call
$ strace -c ls
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
45.23 0.000234 23 10 read
30.15 0.000156 15 10 write
15.42 0.000080 10 8 openat
9.20 0.000048 6 8 close
------ ----------- ----------- --------- --------- ----------------
100.00 0.000518 36 total
Generates a summary report showing time statistics for each system call using the '-c' option. Shows percentage of time, total time, average time per call, number of calls, and errors. Perfect for performance profiling and identifying bottlenecks in your application.
Performance Analysis: Use this to identify which syscalls are taking the most time
Example 7
Follow Child Processes
$ strace -f ./script.sh
[pid 1234] execve("./script.sh", ...) = 0
[pid 1234] clone(child_stack=NULL, ...) = 1235
[pid 1235] execve("/usr/bin/grep", ["grep", "error"], ...) = 0
[pid 1235] read(0, "line1\nline2\n", 4096) = 12
[pid 1234] wait4(1235, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 1235
Follows child processes created by fork/clone using the '-f' option. Essential for tracing shell scripts, daemon processes, or any application that spawns child processes. Each process is labeled with its PID, making it easy to follow the execution flow.
Complex Apps: Many applications fork children, so -f is often necessary for complete tracing
Example 8
Trace Network System Calls
$ strace -e trace=network curl http://example.com
socket(AF_INET, SOCK_STREAM, IPPROTO_TCP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("93.184.216.34")}, 16) = 0
sendto(3, "GET / HTTP/1.1\r\nHost: example.com\r\n", 37, 0, NULL, 0) = 37
recvfrom(3, "HTTP/1.1 200 OK\r\n...", 16384, 0, NULL, NULL) = 1256
close(3) = 0
Traces only network-related system calls using '-e trace=network'. Shows socket creation, connections, data transfers, and closures. Invaluable for debugging network applications, API calls, or connectivity issues. Helps understand exactly what network operations your program performs.
Network Debug: Perfect for diagnosing connection problems, timeout issues, and data transfer
Example 9
Show String Arguments in Detail
$ strace -s 200 cat file.txt
openat(AT_FDCWD, "file.txt", O_RDONLY) = 3
read(3, "This is a very long string that would normally be truncated but now we can see the entire content because we increased the string length limit with the -s option", 131072) = 156
write(1, "This is a very long string that would normally be truncated but now we can see the entire content...", 156) = 156
Increases the maximum string size displayed using the '-s' option. Default is 32 characters, but increasing it reveals full command arguments, file contents, and buffer data. Essential when you need to see complete data being read, written, or passed to system calls.
Default: Without -s, strings are truncated to 32 chars with "..." appended
Example 10
Comprehensive Debugging Setup
$ strace -ff -tt -T -o /tmp/trace -s 1024 -p 1234
# Creates multiple files: /tmp/trace.1234, /tmp/trace.1235, etc.
# Each file contains full trace for one process/thread
$ cat /tmp/trace.1234
10:45:23.123456 read(3, "data...", 4096) <0.000234> = 4096
10:45:23.456789 write(4, "response...", 512) <0.000089< = 512
Combines multiple options for comprehensive debugging: '-ff' (follow forks, separate files), '-tt' (microsecond timestamps), '-T' (show time spent per call), '-o' (output to file), '-s 1024' (longer strings), '-p' (attach to PID). This setup is ideal for production debugging of complex applications.
Performance Impact: Extensive tracing can significantly slow down the traced process
Bonus
Common strace Options Quick Reference
Basic Usage:
strace command Trace a command
strace -p PID Attach to running process
strace -o file command Save output to file
Filtering:
strace -e syscall Trace specific syscall
strace -e trace=set Trace syscall set (file, network, process, signal, memory)
strace -e open,read Trace multiple specific calls
Timing:
strace -t Show time (HH:MM:SS)
strace -tt Show time with microseconds
strace -ttt Show Unix timestamp
strace -T Show time spent in each syscall
strace -c Summary statistics
Process Control:
strace -f Follow child processes
strace -ff -o file Follow forks, separate output files
Output Control:
strace -s SIZE Set max string size (default 32)
strace -v Verbose mode (no abbreviations)
strace -x Print non-ASCII strings in hex
strace -xx Print all strings in hex
Performance:
strace -c Count time/calls/errors per syscall
strace -C Like -c but also continue normal tracing
Common Combinations:
strace -e trace=file ls All file-related syscalls
strace -tt -T -o log cmd Timestamped trace with durations
strace -ff -o trace -p PID Full trace of process and children
A comprehensive reference of the most useful strace options for system debugging, performance analysis, and understanding application behavior at the kernel interface level.