Consuming Web API JSON Data Using curl and jq: Complete Guide 2026

Whether you are testing endpoints, building shell scripts, or debugging production APIs, the combination of curl and jq is the most powerful two-tool stack in a developer's terminal. This guide takes you from zero to confidently querying real APIs, parsing nested JSON, transforming data pipelines, and automating workflows — entirely from the command line.

1B+

curl downloads per year

~50ms

typical jq parse time on 1MB JSON

200+

built-in jq filter functions

0

runtime dependencies for jq

1

What Are curl and jq?

curl (Client URL) is a command-line tool and library for transferring data with URLs. It supports HTTP, HTTPS, FTP, and dozens of other protocols. Developers use it daily to send GET, POST, PUT, and DELETE requests to REST APIs.

jq is a lightweight, portable command-line JSON processor written in C. It reads JSON from stdin or a file, applies a filter expression, and outputs transformed JSON or plain text. Think of it as sed and awk but designed specifically for JSON.

curl strengths

HTTP requests, custom headers, authentication, file uploads, cookie handling, redirect following, TLS certificates.

jq strengths

Pretty-printing, field extraction, array iteration, conditional logic, grouping, sorting, math, and string interpolation.

Why pipe them together

curl delivers raw JSON; jq shapes it. The Unix pipe model means zero temporary files and instant feedback.

Where they run

macOS, Linux, Windows (WSL / Git Bash), CI/CD pipelines, Docker containers — anywhere bash runs.

2

Installation

1

Install curl

curl ships with macOS and most Linux distros. On Ubuntu/Debian run: sudo apt install curl. On macOS with Homebrew: brew install curl. On Windows install WSL2 or Git for Windows.

# Ubuntu / Debian
sudo apt update && sudo apt install -y curl

# macOS
brew install curl

# Verify
curl --version
2

Install jq

jq is a single static binary — no dependencies. Download from jqlang.github.io/jq or use a package manager.

# Ubuntu / Debian
sudo apt install -y jq

# macOS
brew install jq

# Alpine (Docker)
apk add jq

# Verify
jq --version
3

Test the pipeline

Run a quick sanity check against a public API.

curl -s "https://jsonplaceholder.typicode.com/todos/1" | jq
# Expected pretty-printed output:
# {
#   "userId": 1,
#   "id": 1,
#   "title": "delectus aut autem",
#   "completed": false
# }
3

Essential curl Flags for API Work

Raw curl https://api.example.com works, but the following flags turn it into a proper API client.

ItemFlagWhat it does
-s-sSilent mode — suppresses progress bar. Essential when piping to jq.
-S-sSShow errors even in silent mode. Combine with -s as -sS.
-X-X POSTSet HTTP method. Default is GET.
-H-H "Accept: application/json"Add a request header. Repeat for multiple headers.
-d-d '{"key":"val"}'Send request body (implies POST).
-o-o response.jsonSave output to file instead of stdout.
-w-w "%{http_code}"Print extra info like HTTP status code after transfer.
-L-LFollow redirects (3xx responses).
-u-u user:passBasic authentication.
--fail--failExit with non-zero code on HTTP errors (4xx, 5xx). Great for scripts.
bashapi-request.sh
# Full production-grade GET request
curl -sS \
  -H "Authorization: Bearer $TOKEN" \
  -H "Accept: application/json" \
  -L --fail \
  "https://api.example.com/v2/users" \
  | jq

# POST with JSON body
curl -sS \
  -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"name":"Alice","role":"admin"}' \
  "https://api.example.com/v2/users" \
  | jq
4

jq Basics: Filters, Fields, and Arrays

Quick fact

The identity filter . in jq means "pass input through unchanged" — it is how you pretty-print any JSON.

Every jq expression is a filter. Filters read input, transform it, and produce output. You chain filters with | (the pipe operator inside jq).

bashjq-basics.sh
# Pretty-print (identity filter)
curl -s "https://jsonplaceholder.typicode.com/users/1" | jq '.'

# Extract a single field
curl -s "https://jsonplaceholder.typicode.com/users/1" | jq '.name'
# "Leanne Graham"

# Nested field
curl -s "https://jsonplaceholder.typicode.com/users/1" | jq '.address.city'
# "Gwenborough"

# Array element by index
curl -s "https://jsonplaceholder.typicode.com/users" | jq '.[0].name'
# "Leanne Graham"

# All elements of an array (iterator)
curl -s "https://jsonplaceholder.typicode.com/users" | jq '.[].name'
# Prints each name on a separate line

# Array length
curl -s "https://jsonplaceholder.typicode.com/users" | jq '. | length'
# 10

Remove string quotes with -r

By default jq wraps string values in double quotes. Add the -r flag (raw output) to strip them: jq -r '.name'. This is essential when piping jq output into other shell commands.
5

Filtering and Selecting Data

The select() function filters array elements based on a condition. Combined with the iterator .[] it is the jq equivalent of WHERE in SQL.

bashjq-select.sh
# Filter todos that are completed
curl -s "https://jsonplaceholder.typicode.com/todos" \
  | jq '[.[] | select(.completed == true)]'

# Filter users by userId
curl -s "https://jsonplaceholder.typicode.com/todos" \
  | jq '[.[] | select(.userId == 1)]'

# Filter posts where title contains "qui"
curl -s "https://jsonplaceholder.typicode.com/posts" \
  | jq '[.[] | select(.title | contains("qui"))]'

# Count matching items
curl -s "https://jsonplaceholder.typicode.com/todos" \
  | jq '[.[] | select(.completed == true)] | length'
6

Transforming and Reshaping JSON

jq can reconstruct JSON into any shape using object construction {} and array construction [].

bashjq-transform.sh
# Pick specific fields (projection)
curl -s "https://jsonplaceholder.typicode.com/users" \
  | jq '[.[] | {id, name, email}]'

# Rename fields
curl -s "https://jsonplaceholder.typicode.com/users" \
  | jq '[.[] | {userId: .id, fullName: .name, contactEmail: .email}]'

# Add computed fields
curl -s "https://jsonplaceholder.typicode.com/posts" \
  | jq '[.[] | . + {titleLength: (.title | length)}]'

# Flatten nested structure
curl -s "https://jsonplaceholder.typicode.com/users" \
  | jq '[.[] | {id, name, city: .address.city, lat: .address.geo.lat}]'
7

Aggregation: group_by, sort_by, unique_by

bashjq-aggregate.sh
# Group todos by userId
curl -s "https://jsonplaceholder.typicode.com/todos" \
  | jq 'group_by(.userId)'

# Count todos per user
curl -s "https://jsonplaceholder.typicode.com/todos" \
  | jq 'group_by(.userId) | map({userId: .[0].userId, count: length})'

# Sort posts by title length
curl -s "https://jsonplaceholder.typicode.com/posts" \
  | jq 'sort_by(.title | length)'

# Get unique userIds
curl -s "https://jsonplaceholder.typicode.com/todos" \
  | jq '[.[].userId] | unique | sort'

# Min and max
curl -s "https://jsonplaceholder.typicode.com/todos" \
  | jq 'group_by(.userId) | map({userId: .[0].userId, count: length}) | (max_by(.count), min_by(.count))'
8

Working with Headers and HTTP Status Codes

Quick fact

Always check HTTP status codes in scripts. A 200 OK with an error body is the most common API gotcha.

bashjq-headers.sh
# Print HTTP status code alongside response
curl -sS -w "\nHTTP_STATUS: %{http_code}\n" \
  "https://jsonplaceholder.typicode.com/posts/1" | jq '.'

# Store status code and body separately
HTTP_BODY=$(curl -sS -w "%{http_code}" -o /tmp/api_response.json \
  "https://api.example.com/data")
HTTP_CODE=$?
cat /tmp/api_response.json | jq '.'
echo "Status: $HTTP_BODY"

# Exit script on non-2xx (using --fail)
curl -sS --fail \
  "https://api.example.com/endpoint" \
  | jq '.' || { echo "API request failed"; exit 1; }

# Capture response headers with -D
curl -sS -D /tmp/headers.txt "https://api.example.com/data" | jq '.'
cat /tmp/headers.txt
9

Authentication Patterns

1

Bearer Token (OAuth 2.0 / JWT)

Most modern REST APIs use Bearer tokens in the Authorization header.

export TOKEN="eyJhbGciOiJSUzI1NiJ9..."
curl -sS \
  -H "Authorization: Bearer $TOKEN" \
  "https://api.example.com/me" | jq
2

API Key in Header

Some APIs use a custom header like X-API-Key.

export API_KEY="sk-live-abc123"
curl -sS \
  -H "X-API-Key: $API_KEY" \
  "https://api.example.com/data" | jq
3

Basic Authentication

Legacy APIs or internal tools sometimes use HTTP Basic auth.

curl -sS -u "$USERNAME:$PASSWORD" \
  "https://api.internal.com/v1/status" | jq
4

API Key in Query String

Some public APIs embed the key in the URL. Less secure — avoid for production.

curl -sS "https://api.example.com/data?api_key=$API_KEY" | jq
10

Real-World Examples

Let us walk through five common real-world scenarios.

bashexample-github.sh
# 1. List your GitHub repos and show name + star count
curl -sS \
  -H "Authorization: Bearer $GITHUB_TOKEN" \
  -H "Accept: application/vnd.github+json" \
  "https://api.github.com/user/repos?per_page=100" \
  | jq '[.[] | {name, stars: .stargazers_count}] | sort_by(-.stars)'
bashexample-weather.sh
# 2. Get current temperature from OpenWeatherMap
curl -sS "https://api.openweathermap.org/data/2.5/weather?q=London&appid=$OWM_KEY&units=metric" \
  | jq '{city: .name, temp: .main.temp, feels_like: .main.feels_like, description: .weather[0].description}'
bashexample-pagination.sh
# 3. Handle paginated APIs (fetch all pages)
PAGE=1
ALL_DATA="[]"
while true; do
  RESPONSE=$(curl -sS "https://api.example.com/items?page=$PAGE&limit=100")
  ITEMS=$(echo "$RESPONSE" | jq '.items')
  COUNT=$(echo "$ITEMS" | jq 'length')

  if [ "$COUNT" -eq 0 ]; then break; fi

  ALL_DATA=$(echo "$ALL_DATA $ITEMS" | jq -s '.[0] + .[1]')
  PAGE=$((PAGE + 1))
done

echo "$ALL_DATA" | jq 'length'
bashexample-post.sh
# 4. Create a resource and capture the new ID
NEW_ID=$(curl -sS -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"title":"New Post","body":"Content here","userId":1}' \
  "https://jsonplaceholder.typicode.com/posts" \
  | jq -r '.id')
echo "Created post with ID: $NEW_ID"
bashexample-jq-env.sh
# 5. Use shell variables inside jq with --arg
USER_ID=3
curl -s "https://jsonplaceholder.typicode.com/posts" \
  | jq --argjson uid "$USER_ID" '[.[] | select(.userId == $uid)]'
11

Error Handling in Scripts

Silent failures are dangerous

Without --fail or explicit status checking, curl returns exit code 0 even on 404 or 500 responses. Your script will happily process an error body as if it were valid data.

Unsafe (no error handling)

❌ Bad
# BAD: no error handling
curl -s "https://api.example.com/data" | jq '.users[]'
# If API returns {"error":"unauthorized"}, jq silently outputs nothing

Safe (proper error handling)

✅ Good
# GOOD: check exit codes and validate response
RESPONSE=$(curl -sS --fail "https://api.example.com/data" 2>&1)
if [ $? -ne 0 ]; then
  echo "curl failed: $RESPONSE" >&2
  exit 1
fi

ERROR=$(echo "$RESPONSE" | jq -r '.error // empty')
if [ -n "$ERROR" ]; then
  echo "API error: $ERROR" >&2
  exit 1
fi

echo "$RESPONSE" | jq '.users[]'
12

Performance Tips

Use -s flag always

Suppress progress output when piping to jq to avoid corrupting the JSON stream.

Stream large responses

For very large JSON arrays use jq --stream to process elements one at a time without loading the whole file into memory.

Save to file first

When applying multiple jq filters to the same API response, save to a temp file and run jq multiple times rather than making repeated API calls.

Use -c for compact output

jq -c outputs compact (minified) JSON, useful when passing output to another tool or saving space in logs.

Parallel requests with xargs

Combine curl with xargs -P to make multiple API requests in parallel: cat ids.txt | xargs -P 10 -I{} curl -s "https://api.example.com/item/{}"

Cache responses during dev

Save API responses to files during development and jq against the file: jq '.users' response.json. Saves rate limit quota.

13

Debugging Tips

1

Verbose mode for request inspection

Use -v to see request headers, TLS handshake, and response headers.

curl -v "https://api.example.com/data" 2>&1 | head -50
2

Check what jq receives

If jq gives unexpected output, first inspect the raw curl response.

# Step 1: see raw response
curl -sS "https://api.example.com/data"

# Step 2: validate it is JSON
curl -sS "https://api.example.com/data" | jq 'type'

# Step 3: apply your filter
curl -sS "https://api.example.com/data" | jq '.data.items'
3

Test jq expressions interactively

Use jqplay.org or echo a sample payload to test filters before scripting.

echo '{"users":[{"id":1,"name":"Alice"},{"id":2,"name":"Bob"}]}' \
  | jq '.users[] | select(.id == 2) | .name'
# "Bob"
14

curl + jq vs Alternatives

Itemcurl + jqPython requests + json
SetupZero deps, works in any shellNeeds Python + requests installed
SpeedFastest for one-linersFaster for complex logic
ReadabilityTerse, powerfulMore readable for complex transformations
Error handlingManual with exit codesStructured exceptions
CI/CD suitabilityExcellent — minimal footprintGood — more setup required
Learning curvejq syntax takes timeFamiliar Python syntax

Frequently Asked Questions

You are now ready to consume any REST API from the terminal

With curl and jq mastered, you can script API workflows, build lightweight data pipelines, write CI/CD health checks, and debug production issues — all without leaving your terminal. The next step is combining these into reusable shell functions and adding them to your dotfiles.