📓 journald & journalctl

Logging & Log Management Series: Part 1 — Basics  |  Part 2 — journald  |  Part 3 — logrotate  |  Part 4 — Centralized Logging

The Systemd Journal

The systemd journal (journald) is a structured binary logging system that collects messages from the kernel, services, and applications into a single queryable store. Unlike text log files, the journal stores rich metadata with every entry — unit name, PID, UID, priority — making filtering far more powerful.

journalctl is the tool for reading the journal. Learning it well replaces most grep /var/log/messages workflows with something faster, more precise, and with full timestamps including the year.

Examples

1
Basic journalctl Usage
# Show all journal entries (paginated)
journalctl

# Show newest entries first
journalctl -r

# Show last N lines
journalctl -n 50

# Follow live (like tail -f)
journalctl -f

# No pager -- full output to stdout
journalctl --no-pager

# Current boot only
journalctl -b

# Kernel messages only (like dmesg)
journalctl -k
journalctl -k -b    # kernel messages this boot
journalctl output format:
Apr 25 10:22:15 bambustudio sshd[12345]: Accepted publickey for craig from 192.168.1.5 Apr 25 10:25:33 bambustudio sudo[12400]: craig : TTY=pts/0 ; PWD=/home/craig ; USER=root ; COMMAND=/usr/bin/dnf update
💡 journalctl includes the year. Unlike traditional syslog which omits the year, journal entries have full timestamps. This makes it reliable for forensic work and cross-year log searches.
2
Filter by Service Unit
# Logs for a specific systemd unit
journalctl -u sshd
journalctl -u nginx
journalctl -u postgresql

# Follow a service log live
journalctl -u sshd -f

# Last 50 lines for a service
journalctl -u sshd -n 50

# Multiple units at once
journalctl -u nginx -u php-fpm

# Since last boot
journalctl -u sshd -b
journalctl -u sshd -n 4:
Apr 25 10:22:15 server sshd[12345]: Server listening on 0.0.0.0 port 22. Apr 25 10:22:18 server sshd[12346]: Accepted publickey for craig from 192.168.1.5 port 54400 ssh2 Apr 25 10:22:18 server sshd[12346]: pam_unix(sshd:session): session opened for user craig Apr 25 10:35:01 server sshd[12380]: Failed password for root from 203.0.113.42 port 39211 ssh2
3
Filter by Time
# Since a specific time
journalctl --since "2026-04-25 10:00:00"
journalctl --since today
journalctl --since yesterday
journalctl --since "1 hour ago"
journalctl --since "2 days ago"

# Between two times
journalctl --since "2026-04-25 08:00" --until "2026-04-25 09:00"

# Previous boot
journalctl -b -1

# List all boots
journalctl --list-boots
journalctl --list-boots:
-3 def456abc789 Fri 2026-04-22 08:55:12 EDT--Fri 2026-04-22 18:01:44 EDT -2 ghi789jkl012 Mon 2026-04-25 08:30:05 EDT--Mon 2026-04-25 10:15:33 EDT -1 jkl012mno345 Mon 2026-04-25 10:16:01 EDT--Mon 2026-04-25 23:59:58 EDT 0 mno345pqr678 Tue 2026-04-26 08:00:01 EDT--Tue 2026-04-26 10:22:15 EDT
--list-boots reveals reboot history. Use journalctl -b -2 to read logs from two boots ago. Invaluable when troubleshooting a problem that started after a reboot.
4
Filter by Priority and Program
# Filter by priority level
journalctl -p err           # errors and above
journalctl -p warning       # warnings and above
journalctl -p crit          # critical and above
journalctl -p 0..3          # emerg through err (range)

# Errors since boot -- great morning check
journalctl -b -p err

# Filter by program/tag
journalctl -t sshd
journalctl -t sudo
journalctl -t myapp         # matches logger -t myapp

# Filter by UID (all logs from a user's processes)
journalctl _UID=1001

# Combine filters
journalctl -u sshd -p err --since today
journalctl -u nginx -p warning --since "1 hour ago"
💡 journalctl -b -p err is your morning check. Every error-level or worse message since last boot — in seconds you know if anything serious happened overnight.
5
Output Formats
# Full timestamps with timezone
journalctl -u sshd --output=short-full

# ISO 8601 timestamps
journalctl -u sshd --output=short-iso

# JSON output -- one entry per line (great for scripting)
journalctl -u sshd -n 5 -o json
journalctl -u sshd -n 5 -o json-pretty

# Verbose -- shows ALL metadata fields
journalctl -u sshd -n 2 -o verbose

# Cat format -- just the message, no metadata
journalctl -u sshd -n 10 -o cat

# Export for archiving
journalctl -u sshd --output=export > sshd-logs.export
💡 JSON output for scripting. Pipe journalctl -o json to jq for powerful filtering and analysis, or feed it to a centralized log system.
6
Persistent Journal Configuration

By default the journal may only be in memory and lost on reboot. Make it persistent:

# Check current storage
sudo journalctl --disk-usage
ls /var/log/journal/ 2>/dev/null || echo "No persistent journal"

# Enable persistent journal
sudo mkdir -p /var/log/journal
sudo systemd-tmpfiles --create --prefix /var/log/journal
sudo systemctl restart systemd-journald

# Verify
journalctl --list-boots   # should show multiple boots now
Key settings in /etc/systemd/journald.conf:
[Journal] Storage=persistent # auto, persistent, volatile, none SystemMaxUse=500M # max disk space for journal SystemKeepFree=100M # keep this much free on filesystem SystemMaxFileSize=50M # max size of individual journal files MaxRetentionSec=3month # how long to keep entries Compress=yes # compress journal files ForwardToSyslog=yes # also send to rsyslog
# Apply journald config changes
sudo systemctl restart systemd-journald

# Manually vacuum old journal entries
sudo journalctl --vacuum-size=200M     # keep only 200MB
sudo journalctl --vacuum-time=2weeks   # keep only last 2 weeks
sudo journalctl --vacuum-files=5       # keep only 5 journal files
⚠️ Default journal is volatile. On many systems Storage=auto means the journal is only persistent if /var/log/journal/ exists. If it doesn't, logs are lost on reboot. Always verify with journalctl --list-boots — if you only see one boot, the journal is not persistent.
7
Disk Usage and Maintenance
# How much disk space is the journal using?
sudo journalctl --disk-usage

# Where are journal files stored?
ls -lh /var/log/journal/*/
ls -lh /run/log/journal/*/   # volatile storage location

# Vacuum to reclaim space
sudo journalctl --vacuum-size=500M     # trim to 500MB total
sudo journalctl --vacuum-time=30days   # remove entries older than 30 days

# Verify after vacuum
sudo journalctl --disk-usage

# Rotate journal files now (instead of waiting)
sudo journalctl --rotate

# Verify journald is running
sudo systemctl status systemd-journald
journalctl --disk-usage output:
Archived and active journals take up 312.0M in the file system.
💡 Set SystemMaxUse in journald.conf. Without a size limit, the journal grows until the filesystem fills. A reasonable limit for a production server is 500M–1G. The journal automatically removes old entries when the limit is reached.
8
Practical journalctl Workflows
# Morning health check -- errors since last boot
journalctl -b -p err --no-pager

# Why did a service fail?
journalctl -u myapp -n 50 --no-pager
journalctl -u myapp --since "30 minutes ago"

# What happened right before a crash?
journalctl -b -1 -n 100 --no-pager   # last 100 lines of previous boot

# Find all sudo commands run today
journalctl -t sudo --since today | grep COMMAND

# Who logged in via SSH today?
journalctl -u sshd --since today | grep "Accepted"

# Count errors per service
journalctl -p err --since today --no-pager | \
    awk '{print $5}' | sed 's/\[.*//' | sort | uniq -c | sort -rn | head -10

# Export last week's logs for a service
journalctl -u nginx --since "7 days ago" -o json > nginx-week.json

# Search journal like grep
journalctl --grep="Failed password"
journalctl -u sshd --grep="Failed" --since today
journalctl --grep vs grep: journalctl --grep="pattern" searches only the MESSAGE field of journal entries. Plain journalctl | grep "pattern" searches the entire formatted output including timestamps and hostnames. Use --grep for precision, pipe grep for broader matching.

Quick Reference

CommandWhat it does
journalctl -fFollow journal live
journalctl -bCurrent boot only
journalctl -b -1Previous boot
journalctl --list-bootsShow all boot sessions
journalctl -u SERVICELogs for a specific unit
journalctl -u SERVICE -fFollow a service live
journalctl -p errErrors and above only
journalctl -b -p errErrors this boot (morning check)
journalctl --since todayToday's entries
journalctl --since "1 hour ago"Last hour
journalctl -o jsonJSON output for scripting
journalctl --grep="pattern"Search journal messages
journalctl --disk-usageJournal disk usage
journalctl --vacuum-size=500MTrim journal to 500MB
journalctl --vacuum-time=30daysRemove entries older than 30 days

journalctl vs grep /var/log

TaskOld wayjournalctl way
Service logsgrep sshd /var/log/securejournalctl -u sshd
Live followtail -f /var/log/messagesjournalctl -f
Errors onlygrep -i error /var/log/messagesjournalctl -p err
Time rangegrep "Apr 25" /var/log/messagesjournalctl --since "2026-04-25"
Boot logscat /var/log/boot.logjournalctl -b
Previous bootNot easily possiblejournalctl -b -1
By PIDgrep "\[12345\]" /var/log/messagesjournalctl _PID=12345

← Back to Logging Index ↑ Back to EXPANDED