jq Command Examples - Linux Reference

About jq

jq is a lightweight and flexible command-line JSON processor. It's like sed, awk, and grep combined, but specifically designed for JSON data. In the modern world of APIs, cloud services, and configuration management, jq is an essential tool for every system administrator and developer.

Why jq is Essential:

Parse JSON: Extract data from complex JSON structures
Filter: Select specific fields or array elements
Transform: Reshape JSON into different structures
Format: Pretty-print or compact JSON output
Pipeline: Chain operations for complex queries
Everywhere: Works with APIs, AWS CLI, kubectl, Docker, config files

Common Use Cases:

Parsing AWS CLI output (EC2, S3, CloudWatch)
Working with Kubernetes (kubectl) JSON responses
Processing API responses from REST services
Extracting data from Docker inspect output
Filtering and transforming configuration files
Converting JSON to CSV or other formats
Building complex queries in shell scripts
Debugging API responses and troubleshooting

Installation

# Install on Ubuntu/Debian $ sudo apt update && sudo apt install jq # Install on RHEL/CentOS $ sudo yum install jq # Install on macOS $ brew install jq # Verify installation $ jq --version jq-1.6

Basic jq Syntax

Operator	Description
`.`	Identity - outputs input unchanged (pretty-prints)
`.field`	Access object field
`.[0]`	Access array element by index
`.[]`	Iterate over array elements
`.[].field`	Access field in each array element
`\|`	Pipe - pass output to next filter
`,`	Output multiple values
`select()`	Filter elements matching condition
`map()`	Apply expression to each element
`keys`	Get object keys or array indices
`length`	Length of array, object, or string
`@csv, @json, @base64`	Format output in specific format

Common Options

Option	Description
`-r`	Raw output (no quotes on strings)
`-c`	Compact output (no pretty-printing)
`-S`	Sort object keys
`-M`	Monochrome (no colored output)
`-s`	Slurp - read entire input as single array
`-n`	Null input (don't read from stdin)
`-e`	Exit with status based on output
`--arg name value`	Pass variable to jq expression
`-f file`	Read jq program from file

Detailed Examples

Example 1Pretty-Print and Basic Access

Format JSON and extract simple fields:

$ cat > user.json << 'EOF' {"name":"Alice","age":30,"email":"alice@example.com","active":true} EOF # Pretty-print JSON $ jq '.' user.json

{ "name": "Alice", "age": 30, "email": "alice@example.com", "active": true }

# Extract specific field $ jq '.name' user.json

"Alice"

# Raw output (no quotes) $ jq -r '.name' user.json

Alice

# Multiple fields $ jq '.name, .email' user.json

"Alice" "alice@example.com"

Explanation:

.: Identity filter - pretty-prints JSON
.name: Access "name" field
-r: Raw output without JSON quotes
, operator: Output multiple values
Default output includes quotes for strings
Raw mode useful for scripting and pipes

Tip: Use jq . as a quick JSON validator and formatter. Pipe any JSON through it to check syntax and make it readable: curl api.example.com/data | jq .

Example 2Working with Arrays

Extract data from JSON arrays:

$ cat > servers.json << 'EOF' { "servers": [ {"hostname": "web01", "ip": "192.168.1.10", "status": "running"}, {"hostname": "web02", "ip": "192.168.1.11", "status": "stopped"}, {"hostname": "db01", "ip": "192.168.1.20", "status": "running"} ] } EOF # Access array directly $ jq '.servers' servers.json

[ {"hostname": "web01", "ip": "192.168.1.10", "status": "running"}, {"hostname": "web02", "ip": "192.168.1.11", "status": "stopped"}, {"hostname": "db01", "ip": "192.168.1.20", "status": "running"} ]

# Get first element $ jq '.servers[0]' servers.json

{"hostname": "web01", "ip": "192.168.1.10", "status": "running"}

# Iterate over array elements $ jq '.servers[].hostname' servers.json

"web01" "web02" "db01"

# Get all IPs as raw output $ jq -r '.servers[].ip' servers.json

192.168.1.10 192.168.1.11 192.168.1.20

Explanation:

[0]: Access first array element (zero-indexed)
[]: Iterate over all array elements
.servers[].hostname: Chain operations
Each array element produces separate output line
Perfect for feeding to shell loops
Combine with -r for clean output

Note: The .[] operator "unwraps" arrays, outputting each element separately. This is crucial for processing collections in shell scripts: for ip in $(jq -r '.servers[].ip' servers.json); do ping -c1 $ip; done

Example 3Filtering with select()

Filter array elements based on conditions:

$ # Filter running servers only $ jq '.servers[] | select(.status == "running")' servers.json

{"hostname": "web01", "ip": "192.168.1.10", "status": "running"} {"hostname": "db01", "ip": "192.168.1.20", "status": "running"}

# Get hostnames of running servers $ jq -r '.servers[] | select(.status == "running") | .hostname' servers.json

web01 db01

# Multiple conditions with 'and' $ jq -r '.servers[] | select(.status == "running" and .hostname | startswith("web")) | .ip' servers.json

192.168.1.10

# Filter servers NOT in running state $ jq -r '.servers[] | select(.status != "running") | .hostname' servers.json

web02

Explanation:

|: Pipe operator passes data to next filter
select(): Filter elements where condition is true
==: Equality comparison
and, or: Logical operators
startswith(): String function
Chain multiple filters with pipes

Tip: Common select() patterns: select(.field) (field exists and is truthy), select(.field | not) (field is false/null), select(.num > 10) (numeric comparison). Use contains() for substring matching.

Example 4Building New Objects

Transform JSON into new structures:

$ # Create simplified object $ jq '.servers[] | {name: .hostname, address: .ip}' servers.json

{"name": "web01", "address": "192.168.1.10"} {"name": "web02", "address": "192.168.1.11"} {"name": "db01", "address": "192.168.1.20"}

# Add computed fields $ jq '.servers[] | {hostname, ip, port: 22, url: ("http://" + .ip)}' servers.json

{"hostname": "web01", "ip": "192.168.1.10", "port": 22, "url": "http://192.168.1.10"} {"hostname": "web02", "ip": "192.168.1.11", "port": 22, "url": "http://192.168.1.11"} {"hostname": "db01", "ip": "192.168.1.20", "port": 22, "url": "http://192.168.1.20"}

# Build array of simplified objects $ jq '[.servers[] | {name: .hostname, ip}]' servers.json

[ {"name": "web01", "ip": "192.168.1.10"}, {"name": "web02", "ip": "192.168.1.11"}, {"name": "db01", "ip": "192.168.1.20"} ]

Explanation:

{name: .hostname}: Create new object with renamed field
{hostname, ip}: Shorthand - keeps field names
+: String concatenation
[...]: Collect results into array
Computed fields can use expressions
Perfect for reshaping API responses

Best Practice: When integrating with other tools, reshape JSON to exactly what you need. Don't pass entire objects when you only need 2 fields. Smaller, focused data structures are easier to process and debug.

Example 5map() and Advanced Transformations

Apply transformations to array elements:

$ # Use map() to transform array $ jq '.servers | map(.hostname)' servers.json

["web01", "web02", "db01"]

# Map with select for filtering $ jq '.servers | map(select(.status == "running") | .hostname)' servers.json

["web01", "db01"]

# Convert to uppercase $ jq -r '.servers | map(.hostname | ascii_upcase) | .[]' servers.json

WEB01 WEB02 DB01

# Add index to each element $ jq '.servers | to_entries | map({id: .key, server: .value.hostname})' servers.json

[ {"id": 0, "server": "web01"}, {"id": 1, "server": "web02"}, {"id": 2, "server": "db01"} ]

Explanation:

map(expr): Apply expression to each array element
map() with select(): Filter and transform
ascii_upcase: String function (also: ascii_downcase)
to_entries: Convert array to key-value pairs
.key gets index, .value gets element
Powerful for batch transformations

Tip: map() is cleaner than [] | when you want to keep results in an array. Compare: .servers | map(.hostname) returns array, .servers[].hostname returns separate values.

Example 6Working with AWS CLI Output

Real-world example: Parse AWS EC2 instance data:

$ # Simulate AWS EC2 describe-instances output $ cat > ec2.json << 'EOF' { "Reservations": [ { "Instances": [ { "InstanceId": "i-1234567890abcdef0", "InstanceType": "t3.micro", "State": {"Name": "running"}, "PrivateIpAddress": "10.0.1.10", "Tags": [ {"Key": "Name", "Value": "web-server-01"}, {"Key": "Environment", "Value": "production"} ] }, { "InstanceId": "i-0987654321fedcba0", "InstanceType": "t3.small", "State": {"Name": "stopped"}, "PrivateIpAddress": "10.0.1.11", "Tags": [ {"Key": "Name", "Value": "web-server-02"}, {"Key": "Environment", "Value": "staging"} ] } ] } ] } EOF # Get all running instances $ jq '.Reservations[].Instances[] | select(.State.Name == "running") | .InstanceId' ec2.json

"i-1234567890abcdef0"

# Extract instance info with Name tag $ jq -r '.Reservations[].Instances[] | { id: .InstanceId, type: .InstanceType, ip: .PrivateIpAddress, name: (.Tags[] | select(.Key == "Name") | .Value), state: .State.Name } | "\(.name) (\(.id)): \(.ip) - \(.state)"' ec2.json

web-server-01 (i-1234567890abcdef0): 10.0.1.10 - running web-server-02 (i-0987654321fedcba0): 10.0.1.11 - stopped

# Get production instances only $ jq -r '.Reservations[].Instances[] | select(.Tags[] | select(.Key == "Environment" and .Value == "production")) | .PrivateIpAddress' ec2.json

10.0.1.10

Explanation:

AWS uses nested array structures
Chain [] to navigate nested arrays
Tags are array - use select() to find specific tag
\(): String interpolation in raw output
Complex filters for tag-based selection
Essential for AWS automation scripts

Real-World Usage: This is how you build dynamic Ansible inventories, SSH connection scripts, or monitoring configs from AWS: aws ec2 describe-instances | jq -r '.Reservations[].Instances[] | select(.State.Name=="running") | .PrivateIpAddress'

Example 7Aggregation and Statistics

Calculate statistics from JSON data:

$ cat > metrics.json << 'EOF' { "metrics": [ {"server": "web01", "cpu": 45, "memory": 2048, "disk": 50}, {"server": "web02", "cpu": 78, "memory": 4096, "disk": 75}, {"server": "db01", "cpu": 23, "memory": 8192, "disk": 40} ] } EOF # Get total memory across all servers $ jq '[.metrics[].memory] | add' metrics.json

14336

# Calculate average CPU usage $ jq '[.metrics[].cpu] | add / length' metrics.json

48.666666666666664

# Find server with highest CPU $ jq '.metrics | max_by(.cpu)' metrics.json

{"server": "web02", "cpu": 78, "memory": 4096, "disk": 75}

# Get min and max CPU $ jq '[.metrics[].cpu] | "Min: \(min), Max: \(max), Avg: \(add/length)"' metrics.json

"Min: 23, Max: 78, Avg: 48.666666666666664"

# Count servers over 50% disk usage $ jq '[.metrics[] | select(.disk > 50)] | length' metrics.json

Explanation:

add: Sum all array elements
length: Count elements in array
max_by(expr): Find element with max value
min, max: Min/max of array
add / length: Calculate average
Combine with select() for conditional counts

Tip: For grouped aggregations, use group_by(): jq 'group_by(.type) | map({type: .[0].type, count: length, total: map(.value) | add})' - groups by field and aggregates within groups.

Example 8Converting to CSV and Other Formats

Export JSON data to CSV for spreadsheets:

$ # Convert to CSV format $ jq -r '.servers[] | [.hostname, .ip, .status] | @csv' servers.json

"web01","192.168.1.10","running" "web02","192.168.1.11","stopped" "db01","192.168.1.20","running"

# Add CSV header $ jq -r '["Hostname","IP Address","Status"], (.servers[] | [.hostname, .ip, .status]) | @csv' servers.json

"Hostname","IP Address","Status" "web01","192.168.1.10","running" "web02","192.168.1.11","stopped" "db01","192.168.1.20","running"

# Tab-separated values (TSV) $ jq -r '.servers[] | [.hostname, .ip, .status] | @tsv' servers.json

web01 192.168.1.10 running web02 192.168.1.11 stopped db01 192.168.1.20 running

# Base64 encode $ echo '{"password":"secret123"}' | jq -r '.password | @base64'

c2VjcmV0MTIz

# URL encode $ echo '{"url":"hello world"}' | jq -r '.url | @uri'

hello%20world

Explanation:

@csv: Format array as CSV
@tsv: Format array as tab-separated
@base64: Base64 encode string
@uri: URL encode string
@json: Format as JSON (useful in strings)
Headers added manually with array

Note: CSV output is perfect for importing into Excel, Google Sheets, or databases: kubectl get pods -o json | jq -r '.items[] | [.metadata.name, .status.phase] | @csv' > pods.csv

Example 9Using Variables and Arguments

Pass variables to jq expressions for dynamic queries:

$ # Pass variable from command line $ jq --arg status "running" '.servers[] | select(.status == $status)' servers.json

{"hostname": "web01", "ip": "192.168.1.10", "status": "running"} {"hostname": "db01", "ip": "192.168.1.20", "status": "running"}

# Multiple variables $ jq --arg name "web" --arg state "running" \ '.servers[] | select(.hostname | startswith($name) and .status == $state)' servers.json

{"hostname": "web01", "ip": "192.168.1.10", "status": "running"}

# Pass JSON object $ jq --argjson config '{"port":8080,"ssl":true}' \ '. + $config' user.json

{ "name": "Alice", "age": 30, "email": "alice@example.com", "active": true, "port": 8080, "ssl": true }

# Build dynamic filter from shell variable $ STATUS="running" $ jq --arg s "$STATUS" '.servers[] | select(.status == $s) | .hostname' servers.json

"web01" "db01"

Explanation:

--arg name value: Pass string variable
$name: Reference variable in expression
--argjson name json: Pass JSON object
Variables make jq expressions reusable
Essential for shell script integration
Prevents injection issues vs string concatenation

Tip: Always use --arg to pass shell variables to jq. Never concatenate into the jq expression: jq ".servers[] | select(.status == \"$STATUS\")" is dangerous (injection risk). Use: jq --arg s "$STATUS" '.servers[] | select(.status == $s)'

Example 10Complex Real-World Pipeline

Combine jq with other tools in production scripts:

$ # Create sample log data $ cat > logs.json << 'EOF' { "logs": [ {"timestamp": "2024-12-16T10:00:00Z", "level": "ERROR", "service": "api", "message": "Database connection failed"}, {"timestamp": "2024-12-16T10:01:00Z", "level": "INFO", "service": "api", "message": "Request completed"}, {"timestamp": "2024-12-16T10:02:00Z", "level": "ERROR", "service": "worker", "message": "Queue timeout"}, {"timestamp": "2024-12-16T10:03:00Z", "level": "WARN", "service": "api", "message": "High memory usage"}, {"timestamp": "2024-12-16T10:04:00Z", "level": "ERROR", "service": "api", "message": "Authentication failed"} ] } EOF # Count errors by service $ jq '.logs | map(select(.level == "ERROR")) | group_by(.service) | map({service: .[0].service, errors: length})' logs.json

[ {"service": "api", "errors": 2}, {"service": "worker", "errors": 1} ]

# Generate alert emails for errors $ jq -r '.logs[] | select(.level == "ERROR") | "ALERT: \(.service) - \(.message) at \(.timestamp)"' logs.json

ALERT: api - Database connection failed at 2024-12-16T10:00:00Z ALERT: worker - Queue timeout at 2024-12-16T10:02:00Z ALERT: api - Authentication failed at 2024-12-16T10:04:00Z

# Complex pipeline: fetch, filter, transform, save # Simulated: Get running instances, extract IPs, generate config $ jq -r '.Reservations[].Instances[] | select(.State.Name == "running") | .PrivateIpAddress' ec2.json | \ while read ip; do echo " server $ip:80;" done > nginx-upstream.conf $ cat nginx-upstream.conf

server 10.0.1.10:80;

# Build Ansible inventory from JSON $ jq -r '.servers | group_by(.status) | map("[" + .[0].status + "]\n" + (map(.hostname) | join("\n"))) | join("\n\n")' servers.json

[running] web01 db01 [stopped] web02

Explanation:

group_by(): Group array elements by field
join(): Join array elements with separator
Multi-line jq expressions with backslash
Combine jq with while loops for automation
Generate config files dynamically
Real-world log analysis and alerting

Production Pattern: This demonstrates how jq integrates into automation pipelines: (1) API/CLI returns JSON, (2) jq parses and filters, (3) Shell script processes results, (4) Generate configs or trigger actions. Essential for cloud automation, monitoring, and CI/CD.

Related Commands and Tools

Common Pitfalls and Solutions

Pitfall 1: Forgetting -r for Shell Scripts

Output includes quotes, breaking shell variable assignments and loops.

Solution: Use -r (raw output) when feeding jq output to shell: for ip in $(jq -r '.servers[].ip' file.json); do .... Without -r, you get "10.0.1.10" instead of 10.0.1.10.

Pitfall 2: Confusing [] vs map()

Using .[] when you want to keep results in array.

Solution: .servers[].name outputs separate values. .servers | map(.name) outputs single array. Use map() when next step needs an array, use [] when processing individually.

Pitfall 3: Not Handling Null/Missing Fields

Accessing non-existent fields causes errors or unexpected output.

Solution: Use // default for null handling: .field // "N/A". Or select(.field) to skip null entries. Or try-catch: try .field catch "error".

Pitfall 4: Complex Expressions Without Comments

Long jq one-liners become unmaintainable.

Solution: Save complex queries to .jq files with comments: # Extract running instances\n.Reservations[].Instances[] | select(.State.Name == "running"). Run with: jq -f query.jq data.json. Document your expressions!

Pitfall 5: Performance with Large Files

Processing multi-GB JSON files is slow or runs out of memory.

Solution: Use --stream for streaming parse of large files. Or use jq -c to process line-delimited JSON. For huge datasets, consider specialized tools or databases instead of JSON files.

Pro Tips and Best Practices

Tip 1: Test Expressions Incrementally

Build complex queries step by step:

# Start simple $ jq '.servers' file.json # Add array iteration $ jq '.servers[]' file.json # Add filtering $ jq '.servers[] | select(.status == "running")' file.json # Add field selection $ jq '.servers[] | select(.status == "running") | .hostname' file.json

Test each stage before adding next filter. Easier debugging.

Tip 2: Use jqplay.org for Learning

Interactive jq playground in browser:

# Visit https://jqplay.org # Paste JSON # Try expressions interactively # Share queries with others via URL

Great for learning, testing expressions, and getting help from community.

Tip 3: Combine with Other Tools

jq shines in pipelines:

# With curl $ curl -s api.example.com/users | jq '.[] | select(.active)' # With AWS CLI $ aws ec2 describe-instances | jq '.Reservations[].Instances[].InstanceId' # With kubectl $ kubectl get pods -o json | jq '.items[] | select(.status.phase=="Running")' # With docker $ docker inspect container | jq '.[0].NetworkSettings.IPAddress'

jq is the glue between JSON-producing tools and shell scripts.

Tip 4: Save Reusable Queries

Build library of common jq queries:

$ mkdir -p ~/.jq $ cat > ~/.jq/aws-running-instances.jq << 'EOF' # Get running EC2 instance IPs .Reservations[].Instances[] | select(.State.Name == "running") | .PrivateIpAddress EOF $ aws ec2 describe-instances | jq -rf ~/.jq/aws-running-instances.jq

Reusable, documented, version-controlled queries.

Tip 5: Validate JSON Before Processing

Check JSON validity to avoid cryptic errors:

$ jq empty file.json # No output = valid JSON # Error message = invalid JSON # In scripts if jq empty "$JSON_FILE" 2>/dev/null; then # Process file jq '.data' "$JSON_FILE" else echo "Invalid JSON" exit 1 fi

jq empty parses JSON but produces no output. Perfect for validation.

Tip 6: Debug with Pretty-Print and Type Info

When queries fail, examine data structure:

# Pretty-print to understand structure $ jq . file.json # Show types $ jq '. | type' file.json $ jq '.field | type' file.json # Examine keys $ jq 'keys' file.json # Check if field exists $ jq 'has("fieldname")' file.json

Understanding data structure is key to writing correct queries.

Historical Note: jq was created by Stephen Dolan in 2012 and has become the de facto standard for JSON processing on the command line. The name "jq" follows Unix naming tradition (like "awk", "sed") - short, memorable, and related to the task (JSON Query). Its C implementation and efficient algorithms make it suitable for both small scripts and large-scale data processing.

Quick Reference Cheat Sheet

Task	Command
Pretty-print JSON	`jq . file.json`
Access field	`jq '.field' file.json`
Raw output (no quotes)	`jq -r '.field' file.json`
Array element	`jq '.[0]' file.json`
Iterate array	`jq '.[]' file.json`
Filter array	`jq '.[] \| select(.status == "active")' file.json`
Map transformation	`jq 'map(.field)' file.json`
Create new object	`jq '{name: .field1, value: .field2}' file.json`
Count elements	`jq '. \| length' file.json`
Sum values	`jq '[.[].value] \| add' file.json`
Convert to CSV	`jq -r '.[] \| [.f1, .f2] \| @csv' file.json`
Pass variable	`jq --arg var "value" '.[] \| select(.f == $var)' file.json`
Validate JSON	`jq empty file.json`

Common Use Case Patterns

Ready-to-Use jq Patterns

Scenario	Command Pattern
AWS: Running instance IPs	`aws ec2 describe-instances \| jq -r '.Reservations[].Instances[] \| select(.State.Name=="running") \| .PrivateIpAddress'`
Docker: Container IPs	`docker inspect $(docker ps -q) \| jq -r '.[].NetworkSettings.IPAddress'`
Kubernetes: Pod names	`kubectl get pods -o json \| jq -r '.items[].metadata.name'`
Extract nested field	`jq '.data.users[].profile.email' file.json`
Count by category	`jq 'group_by(.category) \| map({category: .[0].category, count: length})'`
Find duplicates	`jq 'group_by(.id) \| map(select(length > 1))'`
Merge JSON files	`jq -s add file1.json file2.json`
Deep value search	`jq '.. \| select(type == "string" and contains("search"))'`

About jq

Example 1Pretty-Print and Basic Access

Example 2Working with Arrays

Example 3Filtering with select()

Example 4Building New Objects

Example 5map() and Advanced Transformations

Example 6Working with AWS CLI Output

Example 7Aggregation and Statistics

Example 8Converting to CSV and Other Formats

Example 9Using Variables and Arguments

Example 10Complex Real-World Pipeline

Similar Commands Worth Knowing

Ready-to-Use jq Patterns