Offered by Dead Brands
JSON, CSV, and YAML look trivial and fail silently: numbers lose digits, countries become booleans, timestamps shift by hours.
Use it over MCP: connect to https://mcp.deadbrands.co/hub/mcp and call skills_get with slug data-formats-and-serialization.
# Data Formats and Serialization
Credit: SteTy — published free in the Skills Hub
JSON, CSV, and YAML look trivial and fail silently: numbers lose digits, countries become booleans, timestamps shift by hours. This skill teaches the traps at the byte level — number precision, duplicate keys, quoting rules, YAML's type-guessing, time formats, Unicode, and the base encodings — so the data you receive is the data that was sent.
## Overview
Every format is a contract with sharp edges the spec either leaves undefined or defines in ways parsers implement differently. JSON has one number type and no dates. CSV has no types, no standard encoding, and a quoting rule everyone half-implements. YAML guesses types from spelling — aggressively, and differently across versions. None of them carry timezone, charset, or precision information unless you put it there. The professional skill is not writing these formats; it's knowing exactly where two correct-looking implementations disagree, and choosing representations that survive the trip.
## Core Principles
- **Strict on send, explicit on receive.** Emit the most boring, canonical form of a format; on input, never let a parser guess what you can specify (encoding, delimiter, schema).
- **Text formats don't have your types.** If a value must stay a string ("007", "1.10", a 20-digit ID), represent it as a string. The wire is not your type system.
- **Round-trip test everything.** Serialize, parse, compare. If it doesn't come back identical, the format choice — not the data — is the bug.
- **An identifier is not a number.** IDs, codes, and tokens are strings that happen to be made of digits. Numbers are for arithmetic; the moment you don't plan to add it, quote it.
- **Bytes ≠ characters ≠ glyphs.** Most "string bugs" are actually layer confusion between these three.
## Techniques
**JSON numbers.** JSON (RFC 8259) defines a single numeric type with no precision guarantee; in practice almost every parser lands on an IEEE-754 double. Doubles represent integers exactly only up to **2⁵³ − 1 = 9,007,199,254,740,991**. Beyond that, values silently round: `JSON.parse('{"id": 9007199254740993}').id` returns `9007199254740992` — no error, a *different number*. Snowflake IDs, 64-bit database keys, and nanosecond timestamps all live past that line. Rule: **serialize large integers as strings**, convert at the edges. Also illegal in JSON, full stop: `NaN`, `Infinity`, `undefined`, trailing commas, comments, single-quoted strings. Money is not a float either — send integer minor units (cents) or a decimal string.
**JSON duplicate keys.** `{"role": "user", "role": "admin"}` — the spec says names *SHOULD* be unique and behavior is otherwise unpredictable; most parsers keep the last value, some keep the first, some can be configured to throw. This is a security hole, not a curiosity: if a gateway validates the first `role` and the backend honors the last, an attacker who can inject a duplicate walks through. Reject duplicates at trust boundaries; never emit them.
**CSV done right (RFC 4180).** Records are lines ending in **CRLF**; a field containing a comma, a double quote, or a line break **must** be wrapped in double quotes; a quote inside a quoted field is escaped by **doubling** it: `"He said ""hi"""` is the value `He said "hi"`. A newline inside quotes is data, not a record boundary — which is why `split('\n')` then `split(',')` is never a CSV parser; use a real library with a state machine. Beyond the RFC, the real world adds: no agreed encoding (UTF-8 vs windows-1252), no types (everything is text; `007` becomes `7` only if *your code* converts it), delimiter dialects (semicolons in European Excel exports, tabs, pipes), and **spreadsheet formula injection** — a cell starting with `=`, `+`, `-`, or `@` executes as a formula when opened in Excel, so sanitize user-supplied fields on export. Agree explicitly on empty-string vs null; CSV cannot express the difference.
**YAML's type-guessing (the Norway problem).** YAML resolves unquoted scalars by pattern, and YAML 1.1 (PyYAML, Ruby) and 1.2 (most JS/Go parsers) disagree. Under 1.1 rules: `country: NO` → the **boolean false** (yes/no/on/off/y/n are booleans — Norway's ISO code is the famous casualty); `version: 1.10` → the float `1.1` in *both* versions; `zip: 02134` → octal 1116 under 1.1; `mode: 0755` → 493 (octal) under 1.1, 755 under 1.2; `window: 22:30` → the sexagesimal (base-60) integer **1350** under 1.1; `2026-08-02` → a date object, not a string; `~` and empty values → null. Defenses: quote anything that must stay text (`"NO"`, `"1.10"`, `"02134"`), know which spec version your parser implements, validate against a schema, and always load with the safe loader (`yaml.safe_load` — the unsafe one instantiates arbitrary objects). **Anchors and aliases** (`&defaults` / `*defaults`, with the `<<:` merge key) deduplicate repeated blocks, but they're also a hazard: alias expansion bombs ("billion laughs") can blow a tiny file up exponentially in a naive parser, and merge-key behavior varies. Use anchors for genuinely shared config; never accept untrusted YAML without limits.
**Timestamps.** Three sane choices, one rule: **store and transmit in UTC, with the offset explicit.** (1) *Unix epoch* — seconds (or milliseconds — confusing the two is a classic 1000× bug; 1.7 billion means seconds, 1.7 trillion means ms) since 1970-01-01 UTC. Compact and sortable, zero readability. Beware the **2038 problem**: signed 32-bit epoch overflows on 2038-01-19; use 64-bit. (2) *RFC 3339* — the internet profile of ISO 8601 and what you should actually name in API contracts: `2026-07-22T14:30:00Z`. ISO 8601 alone permits far more forms (week dates, ordinal dates, missing components) than your parser may accept; RFC 3339 pins it down: `T` separator, `Z` or a numeric `±hh:mm` offset, no week dates. Quirk: RFC 3339 allows `-00:00` meaning "offset unknown," which ISO forbids. (3) *Naive local strings* like `07/22/2026 2:30 PM` — ambiguous by construction; reject on sight. Store the instant in UTC; if the user's local zone matters, store the IANA zone name (`America/New_York`) alongside — a bare offset like `-05:00` cannot tell you DST rules. Leap seconds: most systems ignore or smear them; don't design as if epoch seconds were perfectly uniform.
**Unicode and encodings.** A *code point* (U+00E9), its *encoding* in bytes (UTF-8: `C3 A9`, two bytes), and a *grapheme* (what a human calls one character) are three different units; string "length" functions report different ones in different languages. UTF-8 encodes every code point in 1–4 bytes, is ASCII-compatible, and is the default everywhere on the modern web — always declare it. Traps: the **BOM** (UTF-8 BOM is the 3 bytes `EF BB BF`) glued to the start of a file becomes an invisible character in your first CSV header or JSON parse error; UTF-16 files *require* a BOM to signal endianness. **Normalization:** `é` can be one code point (U+00E9, NFC) or two (`e` + U+0301 combining accent, NFD) — visually identical, byte-wise different, so equality checks, hashes, and database lookups fail. macOS filesystems favor NFD, the web favors NFC: normalize (usually to NFC) before comparing or storing. Invalid byte sequences decode to U+FFFD (�) — data already destroyed, silently.
**Base encodings (for moving bytes through text).** **Hex:** 2 characters per byte, +100% size, trivially readable. **Base64:** packs 3 bytes into 4 characters from a 64-char alphabet (+33% size), padded with `=` to a multiple of 4; the **base64url** variant swaps `+`/`/` for `-`/`_` and usually drops padding so values survive URLs and filenames. **Base32:** 5 bits per character (A–Z, 2–7), 8 characters per 5 bytes, case-insensitive and human-dictatable — hence its use in TOTP secrets and license keys. Encoding is *not* encryption and *not* hashing: anyone can decode it, instantly.
## Worked Examples
*The shrinking ID.* A service returns `{"order_id": 9223372036854775807}` (int64 max). A JavaScript consumer parses it into a double and now holds 9223372036854775808 — an ID that doesn't exist. Support looks it up, finds nothing. Fix on the wire: `"order_id": "9223372036854775807"`.
*The CSV record that ate a table.* `name,note` / `Asha,"line one` ⏎ `line two"` is **one** record. A split-on-newline importer produces three rows, misaligns every subsequent record, and imports garbage with no error raised.
*Norway vanishes.* `regions: [dk, fi, is, no, se]` parsed by a YAML 1.1 loader returns `[dk, fi, is, false, se]`. The geo-filter then silently stops matching Norway — nothing crashed, the config just lied. `regions: ["dk", "fi", "is", "no", "se"]` parses identically everywhere.
## Common Pitfalls
- **Writing your own CSV parser** with split. You are re-implementing a state machine, badly. Use the standard library.
- **Floats for money or IDs.** Doubles are for measurements, not for exact values.
- **Trusting YAML defaults in CI/config.** Unquoted scalars are the single largest source of "the file says X but the program read Y." Quote, lint, schema-validate.
- **Timestamps without offsets**, or epoch values whose unit (s/ms/µs) is undocumented. State the unit in the field name if you must: `created_at_ms`.
- **Counting characters by bytes** (truncating UTF-8 mid-sequence produces mojibake or U+FFFD) — slice on code-point/grapheme boundaries.
- **Padding-blind base64.** Some decoders reject missing `=` padding, others require it; know which variant you're holding before debugging "corrupt data" that isn't.
## Practice Drills
1. Parse `{"a": 9007199254740993, "a": 1}` in two languages you use. Record what each returns for both the big number and the duplicate key; then configure or wrap the parser to reject duplicates.
2. Build the nastiest legal CSV you can: quoted commas, doubled quotes, an embedded CRLF inside a field, a leading-zero code, a formula-injection cell. Round-trip it through your standard library and through Excel/a spreadsheet; fix what diverges.
3. Write a YAML file containing `NO`, `1.10`, `0755`, `22:30`, and an anchored block. Load it with two different parsers, print the types, then fix the file so both parsers agree exactly.
4. Take the string `café` in NFC and NFD. Compare bytes, lengths, and equality before and after normalizing; then decode a base64url token by hand to prove variant and padding handling.
## When to Use and Limits
- **Use it when:** a task calls for data formats and serialization, as laid out in the Techniques and Worked Examples above.
- **Watch first for:** Writing your own CSV parser; Floats for money or IDs.; Trusting YAML defaults in CI/config. (see Common Pitfalls).
- **Limits:** this is a general reference drawn from public sources; check exact values against the current standard or vendor docs before relying on them, and get expert review for production security decisions.
## Quick Reference
| Format | Signature trap | One-line defense |
|---|---|---|
| JSON | Integers > 2⁵³−1 silently round | Big integers as strings |
| JSON | Duplicate keys = undefined behavior | Reject at trust boundaries |
| CSV | Newlines/commas live inside quoted fields | Real parser, RFC 4180 quoting |
| YAML | `NO`→false, `1.10`→1.1, `0755`→octal (1.1) | Quote strings; know your spec version |
| Timestamps | Naive local time is ambiguous | UTC + RFC 3339 or epoch (64-bit, unit named) |
| Unicode | NFC ≠ NFD bytes; BOM glues to first field | UTF-8, normalize NFC, strip BOM |
| Base64 | Padding/alphabet variants break decoders | Specify standard vs URL-safe explicitly |