Offered by Dead Brands
HTTP semantics for agents (RFC 9110 and 9111): methods, status codes, caching and safe retries, so you call and debug APIs correctly.
Use it over MCP: connect to https://mcp.deadbrands.co/hub/mcp and call skills_get with slug http-and-rest-api-craft.
# HTTP and REST API Craft
Credit: SteTy — published free in the Skills Hub
HTTP is a contract, not a transport detail. This skill teaches the semantics defined in RFC 9110 (HTTP Semantics) and RFC 9111 (Caching) that every agent needs to call APIs correctly, retry safely, and design or debug endpoints: method safety and idempotency, the status codes that actually matter, headers that carry real meaning, pagination, filtering, versioning, and the RFC 9457 error envelope.
## Overview
An HTTP exchange is a request (method + target URI + headers + optional body) and a response (status code + headers + optional body). REST APIs expose *resources* at URIs and manipulate them with the uniform method set. Two properties from RFC 9110 §9.2 govern almost every correct decision:
- **Safe** — the method is read-only; the client does not request any state change (GET, HEAD, OPTIONS, TRACE).
- **Idempotent** — repeating the identical request has the same server-side effect as doing it once (all safe methods, plus PUT and DELETE).
Idempotency is your retry license: after a timeout, you may blindly retry an idempotent request. A non-idempotent POST may already have succeeded server-side, so retrying can create duplicates — unless the API supports an `Idempotency-Key` header (a client-generated unique value the server deduplicates on, popularized by Stripe).
## Core Principles
1. **The status code is the contract.** Never signal errors with `200 OK` plus an error body; clients, caches, and monitors key off the code first.
2. **Use the method for what it means.** GET retrieves, POST creates/processes, PUT replaces in full, PATCH modifies partially, DELETE removes. Intermediaries and clients assume these semantics.
3. **Requests are self-contained.** REST is stateless: auth and context travel with each request (typically `Authorization: Bearer <token>`), never in server-side session memory.
4. **Design for retries.** Timeouts are ambiguous — the request may have landed. Prefer idempotent operations, honor `Retry-After`, back off exponentially with jitter.
5. **Evolve additively.** Within a version, only add optional fields/parameters. Removing or renaming fields is a breaking change and needs a new version.
## Techniques
**Method semantics (RFC 9110 §9.3):**
| Method | Purpose | Safe | Idempotent | Notes |
|---|---|---|---|---|
| GET | Retrieve a representation | Yes | Yes | Body on GET has no defined semantics; don't send one |
| HEAD | GET without the body | Yes | Yes | Cheap existence/freshness checks |
| POST | Create subordinate resource / process | No | No | Returns 201 for creation |
| PUT | Replace resource in full (or create at a known URI) | No | Yes | Missing fields are *cleared*, not kept |
| PATCH | Partial modification | No | Not guaranteed | JSON Merge Patch (RFC 7386) or JSON Patch (RFC 6902) |
| DELETE | Remove resource | No | Yes | Second DELETE may 404 — the *effect* is still identical |
| OPTIONS | Discover allowed methods/capabilities | Yes | Yes | Answer carries `Allow`; also used for CORS preflight |
**Status codes that matter:**
- **2xx:** `200 OK` (read/update), `201 Created` (POST created something — include a `Location` header pointing at the new resource), `202 Accepted` (queued for async processing; poll a status URL), `204 No Content` (success, empty body — typical for DELETE), `206 Partial Content` (range responses).
- **3xx:** `301 Moved Permanently` vs `302 Found` (historically ambiguous — browsers may rewrite POST to GET), `303 See Other` (explicitly follow with GET), `307 Temporary Redirect` / `308 Permanent Redirect` (method and body are preserved — prefer these), `304 Not Modified` (answer to a conditional GET; no body).
- **4xx — the client's fault:** `400 Bad Request` (malformed), `401 Unauthorized` (actually means *unauthenticated* — the response MUST carry a `WWW-Authenticate` header), `403 Forbidden` (authenticated but not allowed; re-authenticating won't help), `404 Not Found` (also used to avoid leaking existence), `405 Method Not Allowed` (MUST include `Allow`), `406 Not Acceptable` (no representation matches `Accept`), `409 Conflict` (version/state clash — the optimistic-locking failure code), `410 Gone` (permanently deleted, unlike 404), `412 Precondition Failed` (an `If-Match` failed), `413 Content Too Large`, `415 Unsupported Media Type` (wrong `Content-Type`), `422 Unprocessable Content` (syntax fine, semantics invalid — RFC 9110 §15.5.21), `429 Too Many Requests` (send `Retry-After`).
- **5xx — the server's fault:** `500 Internal Server Error` (generic), `501 Not Implemented`, `502 Bad Gateway` (upstream gave an invalid response), `503 Service Unavailable` (overload/maintenance — send `Retry-After`), `504 Gateway Timeout` (upstream didn't answer in time). Retry 502/503/504 with backoff; retrying 500 rarely helps.
**Headers worth knowing cold:**
- `Content-Type` / `Accept` — media type negotiation; versioned types look like `application/vnd.example.v2+json`.
- `ETag` with `If-None-Match` → conditional GET (`304` if unchanged) and `If-Match` → optimistic locking (`412` if someone else changed it first). `Last-Modified` / `If-Modified-Since` are the weaker, timestamp-based pair.
- `Cache-Control` — `max-age`, `no-cache` (revalidate before reuse), `no-store` (never persist), `private` vs `public`.
- `Link` — typed relationships; pagination uses `rel="next"` / `rel="prev"` (RFC 8288).
- `Idempotency-Key` — makes POST safely retryable.
- Rate-limit signals: `Retry-After` plus `RateLimit-Limit` / `RateLimit-Remaining` / `RateLimit-Reset` (or vendor `X-RateLimit-*`).
**Pagination, three patterns:**
1. **Offset/limit** — `?offset=100&limit=25`. Simple, supports "jump to page N", but costs grow with depth (the database walks and discards `offset` rows) and results *drift*: a row inserted between page fetches shifts every position, causing duplicates and silent skips.
2. **Cursor / keyset** — `?cursor=<opaque>&limit=25`; the cursor encodes the last row's sort key and the query becomes `WHERE (created_at, id) > (?, ?) ORDER BY created_at, id`. Constant cost at any depth, immune to insert/delete drift. Costs: no random page jumps, cursor must be treated as opaque, and the sort key needs a unique tiebreaker (timestamps alone collide). Follow `Link: <...>; rel="next"` or a `next_cursor` field until absent. Reset the cursor whenever filters or sort change.
3. **Page number** — offset in disguise; same trade-offs.
**Filtering and sorting:** query parameters, exact match by default (`?status=active&tag=ops`), explicit operators where the API offers them (`?created_after=2026-01-01`, `?price_gte=10`), `?sort=-created_at` (minus = descending). Keep parameter names consistent across endpoints; reject unknown parameters rather than silently ignoring them (a typo'd filter that returns *everything* is a data-leak bug).
**Versioning:** URL prefix (`/v1/users`) is the most visible and debuggable; media-type versioning (`Accept: application/vnd.example.v2+json`) is cleaner REST but harder to explore in a browser. Either way: version from day one, run versions in parallel during migration, announce deprecations (the `Sunset` header exists for this), never silently change behavior inside a version.
**Error envelope — RFC 9457 Problem Details:** respond with `Content-Type: application/problem+json` and members `type` (a URI identifying the error kind; `"about:blank"` means "just the status title"), `title` (human-readable summary), `status` (the code, repeated in the body), `detail` (this occurrence's specifics), `instance` (URI of this occurrence). Extension members are allowed — add a machine-stable `code` and per-field validation errors. The golden rule: clients branch on `type`/`code`, never on parsing `detail` text.
**Reading API docs like a pro:** find, in this order — base URL and versioning scheme, auth mechanism and token lifetime, rate limits and their headers, one complete worked example per endpoint family, the error catalog, pagination convention, idempotency support, changelog/deprecation policy. Then verify docs against reality with a real request; docs drift, wire behavior doesn't lie. Use `curl -i` to see raw status and headers, not just the body.
## Worked Examples
Retry-safe creation:
```
POST /v1/orders Idempotency-Key: 7f3a-…-c21e
→ timeout (unknown state)
POST /v1/orders Idempotency-Key: 7f3a-…-c21e (identical retry)
→ 201 Created Location: /v1/orders/8412
```
The server stored the first result under the key; the retry returns it instead of creating order #2.
Optimistic locking update:
```
GET /v1/items/9 → 200, ETag: "v17"
PATCH /v1/items/9 If-Match: "v17" { "price": 42 }
→ 412 Precondition Failed (someone saved v18 first — re-read, merge, retry)
```
Problem Details response:
```json
HTTP/1.1 422 Unprocessable Content
Content-Type: application/problem+json
{ "type": "https://api.example.com/problems/validation",
"title": "Validation failed", "status": 422,
"detail": "email is not a valid address",
"instance": "/v1/users/signup/req-8f31",
"code": "VALIDATION_ERROR",
"errors": [ { "field": "email", "message": "not a valid address" } ] }
```
## Common Pitfalls
- **200-with-error-body** — breaks every generic client, cache, and monitor.
- **401/403 confusion** — 401 = "who are you?" (send `WWW-Authenticate`), 403 = "I know who you are; no."
- **Retrying POST blindly** after a timeout → duplicate charges/orders. Use idempotency keys or reconcile by querying first.
- **Trusting offset pagination on live data** — duplicates and skipped rows under concurrent writes.
- **PUT used as PATCH** — a full-replace that silently nulls every field the client didn't send.
- **Ignoring `Retry-After` on 429/503** — instant retries deepen the overload and earn longer bans.
- **Tokens in query strings** — URLs land in logs, proxies, and browser history. Use the `Authorization` header.
- **Breaking changes inside a version** — renaming a field is a new version, full stop.
## Practice Drills
1. Take any public API and classify every endpoint by method safety and idempotency; write down the safe retry strategy for each.
2. With `curl -i`, trigger a 401, a 404, and a 405 against a test API and identify the mandatory headers in each response.
3. Page through a 1,000-row offset-paginated list while inserting rows at the front; count duplicates/skips, then repeat with cursor pagination.
4. Convert three ad-hoc JSON error bodies from real APIs into RFC 9457 problem+json without losing information.
5. Read an unfamiliar API's docs and, before writing any code, list: auth flow, rate limits, pagination style, error format, idempotency support, version policy.
## When to Use and Limits
- **Use it when:** a task calls for http and rest api craft, as laid out in the Techniques and Worked Examples above.
- **Watch first for:** 200-with-error-body; 401/403 confusion; Retrying POST blindly (see Common Pitfalls).
- **Limits:** this is a general reference drawn from public sources; check exact values against the current standard or vendor docs before relying on them, and get expert review for production security decisions.
## Quick Reference
- Safe + idempotent: GET, HEAD, OPTIONS · Idempotent only: PUT, DELETE · Neither: POST · PATCH: neither guaranteed.
- 201 + `Location` for creation · 202 = accepted/async · 204 = success, no body.
- 301/308 permanent, 302/307 temporary; 307/308 preserve method+body, 303 forces GET.
- 401 unauthenticated (+ `WWW-Authenticate`) · 403 unauthorized · 404 missing · 409 conflict · 412 precondition · 422 semantic error · 429 rate-limited (+ `Retry-After`).
- 502 bad upstream response · 503 unavailable · 504 upstream timeout — all retryable with backoff.
- Errors: `application/problem+json` with `type`/`title`/`status`/`detail`/`instance` (RFC 9457).
- Pagination: offset = simple but drifts; cursor/keyset = stable and flat-cost, no page jumps.
---
*Trademarks and copyrights mentioned belong to their respective owners. Educational reference only; not affiliated with or endorsed by them.*