Compression

Codecs and negotiation

flowchart TB
  classDef c fill:#13203a,stroke:#818cf8,color:#fff
  A["response body"] --> B("compressible?<br/>type in compressibleTypes · > 200 B · not already encoded · not 206")
  B -- no --> ID["identity"]
  B -- yes --> T("trusted proxy peer and --cf-send-zstd?")
  T -- yes --> Z["zstd (forced)"]:::c
  T -- no --> N("first server-priority codec the client's<br/>Accept-Encoding allows")
  N -- zstd --> Z2["zstd (level 3)"]:::c
  N -- br --> BR["brotli (level 5)"]:::c
  N -- gzip --> GZ["gzip (level 6)"]:::c
  N -- none --> ID
  Z & Z2 & BR & GZ --> S("body shape")
  S -- "Body::Full" --> E1["encode in one shot"]
  S -- "Body::Stream (proxy, SSE, LSAPI)" --> E2["CompressStream: per-chunk, never buffered"]
  S -- "static file with .br/.gz sibling" --> E3["serve the precompressed sibling (br preferred)"]

zstd encoding uses a per-thread ZSTD_CCTX pool so no context is allocated per response. Streamed bodies (Body::Stream — LSAPI, proxy, SSE) wrap in a lazy per-chunk CompressStream: each inbound chunk is encoded as it arrives, so SSE stays real-time and a slow backend never holds the first byte hostage.

Trusted-proxy zstd egress

--cf-send-zstd (or CF_SEND_ZSTD=1) forces zstd to a trusted-proxy peer regardless of the forwarded Accept-Encoding. Cloudflare asks the origin for br/gzip but decodes an origin Content-Encoding: zstd and re-encodes per browser at its edge, so this hands the CDN the cheapest-to-produce form (zstd ≪ brotli to compress) without changing what browsers receive. Off by default, officially unsupported CDN↔origin, enabled in production; it never applies to untrusted or direct clients.

Precompressed siblings

If a .br or .gz sibling of a static file exists in the same directory, httpjet serves it directly with the matching Content-Encoding and Vary: Accept-Encoding — gzip_static/brotli_static behavior, brotli preferred. Ranged requests skip the swap so byte ranges stay exact; same-directory placement keeps accessDenyDir and symlink verdicts unchanged; a missing sibling falls back to on-the-fly compression at the cost of one stat.

Interaction with the page cache

The page cache keeps identity bytes plus one best precompressed variant chosen under the same priority; a client whose acceptable set differs is re-encoded per serve. Internal dictionary compression is a storage optimization only — dictionaries never reach the wire, and a hit decodes to identity before standard client compression is selected.

Request decompression

httpjet decodes a single supported Content-Encoding after the transport has fully framed the request body and before rewrite, cache, WAF, proxy or application dispatch — the same decoder for HTTP/1.1, HTTP/2, h2c and HTTP/3. Gzip is on by default for compatibility; brotli and zstd are opt-in:

httpjet --root /abs/root serve … --request-decompression-extra br,zstd     # accepts br, brotli, zstd; "gzip" is rejected as redundant

flowchart TB
  classDef bad fill:#2a1420,stroke:#e879f9,color:#fff
  A["framed request body<br/>(H1 · H2 · h2c · H3)"] --> B("exactly one Content-Encoding field<br/>with exactly one coding?")
  B -- "no (list, repeated)" --> F["forward unchanged"]
  B -- gzip --> D
  B -- "br / zstd" --> C("listed in --request-decompression-extra?")
  C -- no --> F
  C -- yes --> R("reserve 17 MiB codec workspace<br/>+ encoded + decoded bytes in the body budget")
  R -- "cannot" --> E503[503]:::bad
  R -- ok --> D["decode (buffered)<br/>zstd window ≤ 2^24 · no brotli large-window"]
  D -- "malformed" --> F
  D -- "> maxReqBodySize" --> E413[413]:::bad
  D -- "hits 16 MiB output ceiling" --> F
  D -- ok --> G["drop Content-Encoding, set decoded Content-Length<br/>→ rewrite · WAF · cache · backend"]

• A successful decode removes Content-Encoding and replaces Content-Length with the decoded length before any handler sees the request.
• Decoded output may not exceed the effective maxReqBodySize (413). The absolute decoder ceiling is 16 MiB — at it, httpjet forwards the encoded body rather than exposing a truncated one.
• Brotli large-window extensions are disabled; zstd frames are limited to a 2²⁴-byte window; deflate is not decoded.
• Brotli/zstd reserve 17 MiB for decoder tables before constructing the codec; if that plus both body forms cannot be reserved, the request gets 503.

cargo test --locked -p httpjet uring::request_body --no-fail-fast     # all three codecs, bombs, malformed input, stacked encodings, workspace exhaustion

Decode safety

Response-side decoding (page-cache variants, dictionary frames) caps windowLogMax and bounds decoded output size; a hostile or corrupt frame fails the decode rather than ballooning memory. Compression cost depends on body size, codec, cache state and backend latency — compare representative hits and misses before trading CPU for bytes. In production the origin's compression phase is sampled as httpjet_phase_compress_seconds.