The origin page cache

Overview

Two crates split the work. hj-pagecache is the transport-agnostic store: a sharded byte-weighted LRU index built on hj-cache, W-TinyLFU admission, key + vary + tag indexes, TTL and stale windows, the LSCache header classifier, and the HJPC file tier. The binary's lscache/ module is the request-aware glue: cache_lookup runs after rewrite and access control but before any backend; cache_store runs after each terminal's finalize_response. Both are inert and near-zero-cost without --page-cache.

Opt-in by the app

The cache never guesses. A response is stored only when the application says so. Per-vhost <cache><enableCache> (and the six LSWS gates on a context's cachePolicy), the .htaccess CacheLookup directive, and the [E=no-cache] env all gate lookups; the response header gates storage.

X-LiteSpeed-Cache-Control: public,max-age=3600            # public entry, one hour
X-LiteSpeed-Cache-Control: private,max-age=120            # per-session entry (needs --page-cache-private; TTL capped 300 s)
X-LiteSpeed-Cache-Control: public,max-age=60,stale-while-revalidate=600,stale-if-error=3600
X-LiteSpeed-Cache-Control: no-cache                       # never store this response
X-LiteSpeed-Tag: T1234,forum7                             # purge tags
X-LiteSpeed-Vary: cookie=xf_style_id                      # vary the public entry on a declared cookie value
X-LiteSpeed-Purge: tag=T1234                              # drop every entry carrying this tag (also: * for all)

Standards mode. --page-cache-standard-vhosts a.example,b.example puts listed vhosts into OpenLiteSpeed's checkPublicCache shape: a plain Cache-Control: public, max-age=N is an opt-in, a default public policy applies when no <cache> block exists, and CacheLookup defaults on. It is an allowlist, so a status/admin vhost cannot start caching because it merely marked something public. All other guards still apply.

The lookup path

flowchart TB
  classDef hit fill:#0f2a24,stroke:#34d399,color:#fff
  classDef miss fill:#2a2314,stroke:#fbbf73,color:#fff
  classDef key fill:#13203a,stroke:#818cf8,color:#fff
  A["request after rewrite + ACL<br/>(--page-cache on)"] --> B("GET/HEAD?<br/>vhost enableCache · .htaccess CacheLookup<br/>no [E=no-cache] env")
  B -- no --> BY["bypass → backend"]:::miss
  B -- yes --> C("member cookie present<br/>(xf_user)?")
  C -- "yes, and --page-cache-private" --> P["PRIVATE tier<br/>key += session cookie value"]:::key
  C -- "yes, path in --page-cache-shared-paths" --> S["PUBLIC tier (visitor-invariant endpoint)<br/>sticky canary %"]:::key
  C -- no --> S2["PUBLIC tier"]:::key
  C -- "yes, no private tier" --> BY
  P & S & S2 --> K["key = epoch·generation · vhost · scheme · host · ORIGINAL path?query<br/>· declared vary-cookie values · Origin (if the chain reads it)"]:::key
  K --> L("entry found and identity guard matches<br/>(host + raw path stored with the entry)?")
  L -- "no / collision" --> M("single-flight: another render in progress?")
  M -- yes --> W["wait on leader"] --> H
  M -- no --> R["render backend (leader)"] --> ST["cache store gates"]
  L -- "fresh" --> H["HIT<br/>X-LiteSpeed-Cache: hit"]:::hit
  L -- "stale, within stale-while-revalidate" --> SW["serve stale + background refresh"]:::hit
  L -- "expired" --> M
  R -- "5xx and stale-if-error window" --> SIE["serve retained stale<br/>hit,stale-if-error · max-age=30 egress"]:::hit
  click ST "page-cache.html#store"
  click K "page-cache.html#keys"

Keys, tiers and the identity guard

Two more dimensions were added later and are part of the key today: Origin when the vhost's .htaccess chain reads it (2026-09-16 — a reflected Access-Control-Allow-Origin no longer replays the first storer's CORS verdict; an absent Origin keeps the historic base key so persisted entries survive), and the epoch + generation namespace introduced by the first API publication so an old in-flight render can never overwrite a new generation's entry.

The store path and admission

flowchart TB
  classDef bad fill:#2a1420,stroke:#e879f9,color:#fff
  classDef ok fill:#0f2a24,stroke:#34d399,color:#fff
  A["terminal response (LSAPI · proxy · static)"] --> B("app opted in?<br/>X-LiteSpeed-Cache-Control: public|private,max-age=N<br/>(or standard Cache-Control on a --page-cache-standard-vhosts vhost)")
  B -- no --> N["not stored"]:::bad
  B -- yes --> C("guards")
  C -- "status not in cacheStatusCode · Set-Cookie of a private cookie<br/>· proxied response asserting X-LiteSpeed-* · self-redirect Location<br/>· purge received after render start · object > 100 MiB" --> N
  C -- ok --> D("W-TinyLFU admission<br/>frequency ≥ --page-cache-admit-threshold (+1 per 256 KiB)")
  D -- "not yet" --> N2["counted, not stored (miss-miss-hit)"]:::bad
  D -- admitted --> E["strip Set-Cookie + X-LiteSpeed-* from the public copy<br/>record identity (host + raw path), tags, vary set, TTL, stale windows"]
  E --> F("file tier on?")
  F -- no --> G["in-RAM body (charged to --page-cache-mem)"]:::ok
  F -- yes --> H["HJPC file at birth (identity form, tmpfs)<br/>metadata-only charge to RAM index; footprint to --page-cache-disk-mem"]:::ok
  H -- "after first real hit" --> I["background dict recompress in place<br/>(per-vhost zstd dictionary, 3.5–9× smaller)"]:::ok
  click D "page-cache.html#admission"

W-TinyLFU admission

flowchart LR
  A["cacheable miss for key K"] --> B["count K in the frequency sketch<br/>(fixed memory, W-TinyLFU)"]
  B --> C("freq(K) ≥ threshold + size bonus?<br/>threshold default 2 · +1 per 256 KiB")
  C -- "no" --> D["render, serve, do not store"]
  C -- "yes" --> E["render, serve, STORE"]
  D -- "next request" --> A

The self-redirect guard declines an app-marked-cacheable 3xx whose Location is the request's own URL (hosts normalized without port, because an h2 Host carries :443 and a generated absolute Location may not); it is re-checked at serve time, and the XF add-on refuses at a third layer. A cacheable origin redirect is also de-cached from the CDN's perspective via the deny-CDN-cache transform, so a canonical loop cannot pin itself at the edge.

Storage tiers & the file tier

flowchart LR
  classDef ram fill:#13203a,stroke:#818cf8,color:#fff
  classDef disk fill:#0e1424,stroke:#22d3ee,color:#e8ecf6
  subgraph RAM ["process RAM"]
    IDX["sharded index<br/>metadata · tags · variants · TTL heap<br/>--page-cache-mem (128 MiB)"]:::ram
    HOT["hot tier<br/>hottest bodies, zero-syscall serve<br/>--page-cache-hot-mem (192 MiB)"]:::ram
  end
  subgraph TMPFS ["--page-cache-store-path (tmpfs, e.g. /dev/shm/jetcache)"]
    F["one immutable HJPC file per body version<br/>self-describing · integrity key · survives restarts (boot scan, 8 threads)<br/>--page-cache-disk-mem (2 GiB) disk-LRU"]:::disk
  end
  IDX -- "lookup" --> HOT
  HOT -- "cold" --> F
  F -- "~µs read + decode" --> HOT
  IDX -- "teardown (evict · expire · purge · replace)<br/>synchronous under the shard lock: index + file + hot + tags" --> F

By default the cache is in-RAM only and flushes on restart. --page-cache-store-path /dev/shm/jetcache (or default/jetcache) makes it tmpfs-primary in the LiteSpeed Enterprise style: PageStore::store offloads every body to a file at birth in identity form, off the process heap; one immutable file per body version; self-describing HJPC containers authenticated with the --page-cache-integrity-key. The LiteSpeed cacheStorePath is parsed but deliberately unused so httpjet never collides with a resurrected LiteSpeed.

• Survives restarts. A background boot scan (up to 8 threads over the fan-out dirs) rebuilds the index from the files — ~300k entries in seconds. /__hj_cache_ready returns 503 while scanning and 200 + x-hj-cache-ready: 1 + x-hj-cache-loaded: N when done; probing a URL mid-scan re-primes it and unlinks the persisted copy, so wait for readiness.
• Reclaim invariant. Live entries resolve to live bodies; a missing file fails closed to a miss and counts disk_read_err. Orphan reconciliation reclaims superseded files it can prove are safe to delete and defers the rest.
• Durable purges. Purges unlink eagerly and retain a purge-all stamp, so old data cannot be resurrected on restart.
• Dictionaries compress at store. The birth copy is identity; a bounded background task then dict-recompresses it in place against the vhost's own zstd dictionary (~3.5–9× smaller) at store time — never inline on a response, and never triggered by a first hit. An admission filter rejects one-hit public content before it reaches this work, and incompressible Content-Types are never encoded. Judge by httpjet_pagecache_dict_recompress_{total,skipped_total{reason}}; the served bytes stay standard zstd/br/gzip either way.

Variants, compression and egress

The store keeps identity bytes plus one best precompressed variant chosen by the request's Accept-Encoding under the server priority zstd > br > gzip; a client whose accepted set differs gets a per-serve compress of the identity form. Egress is therefore always standard zstd/brotli/gzip/identity, and a hit re-negotiates encoding — stored responses are never tied to the first client. Hits from the hot tier are zero-copy Bytes on h2. See Compression.

Purge

Invalidation is tag-based. The app emits X-LiteSpeed-Purge: tag=T1234 (or *) on a response — every such header line is honored (0.0.5) — or a local process POSTs the loopback endpoint on the main listener. The endpoint answers POST only (2026-10-07); a loopback GET or other method gets 405 + Allow, and the ready probe takes GET/HEAD. A store that started before a relevant purge and finished after it is rejected (httpjet_pagecache_store_purge_rejections_total), so a stale render cannot win a race against the purge.

curl -s -X POST -H 'x-litespeed-purge: tag=T1234' -o /dev/null -w '%{http_code}\n' http://127.0.0.1/__hj_cache_purge   # 204
curl -s -X POST -H 'x-litespeed-purge: *' http://127.0.0.1/__hj_cache_purge                                          # everything

The endpoint is gated by the raw TCP source being loopback — no shared secret, no per-URL purge, and a CDN-relayed or LAN request gets 404. The former --cache-peer fan-out and --cache-purge-secret-file are gone. A purge with the file tier unlinks eagerly; a restart cannot resurrect purged entries.

Stale policies & single flight

XenForo public-shell capsules application-specific

A second, XenForo-shaped tier behind --xf-capsule (requires --page-cache): a public HTML shell for member requests, with the member-specific bits filled client-side by the SessionValidator/CacheOptimizer add-on. It is fail-closed by allow-list — --xf-capsule-vhosts empty disables it for every vhost — with route coverage by --xf-capsule-paths prefixes or the all-get-classified conservative classifier, deterministic canaries for guests and members, and its own stale window (--xf-capsule-stale-secs, production 3600 so a missed purge self-heals within an hour). Since 2026-10-04 members are served from the shell-capable public entry — --xf-capsule-dedicated-store (default off) restores the second dedicated copy. The capsule purge gate (scripts/cache_capsule_purge_test.sh) proves a tag purge invalidates the member-served shell. Metrics: httpjet_xf_capsule_*.

Generation-scoped namespaces

The first successful API publication introduces a response-cache epoch derived from the process-bound revision; every later generation (including SIGHUP) keys public, private and capsule entries by epoch + generation while preserving request identity and purge labels. Old in-flight renders finish in their own namespace; restart returns to the on-disk configuration's namespace, and volatile API entries cannot replay there. Obsolete entries age out under the ordinary budgets. The finished-response memo is generation-keyed for the same reason.

Microcaching today, and what is missing

Short TTLs already work: a backend that sends X-LiteSpeed-Cache-Control: public,max-age=2 gets every guard above and a two-second hit window (scripts/microcache_policy_example.sh proves miss→hit→expiry→fresh render, distinct-query isolation and no-store respect on an isolated instance). What does not exist yet is a first-class route-level policy: named profiles with deterministic precedence, route TTL floors/ceilings, per-profile eligibility, bypass matchers, declarative key dimensions, profile budgets, and bounded collapse/refresh controls. The proposed <microcachePolicyList> contract is specified in the source tree (docs/microcache-policy.md) and the parser intentionally rejects it until implemented.

Observing the cache

$ curl -sI -H 'Host: example.test' http://127.0.0.1:8080/ | grep -i x-litespeed-cache
x-litespeed-cache: miss            # ran the backend; stored if admitted
$ curl -sI -H 'Host: example.test' http://127.0.0.1:8080/ | grep -i x-litespeed-cache
x-litespeed-cache: hit             # served from cache (also: hit,stale-if-error)

$ curl -s http://127.0.0.1:9090/cache-entries      # totals, per-URL-class counts, top-N by size, ram/disk line, health line
$ curl -s http://127.0.0.1:9090/metrics | grep -E '^httpjet_(pagecache|cache)_'
$ curl -sk https://127.0.0.1/__hj_cache_ready -o /dev/null -w '%{http_code}\n'   # 200 once the boot scan finished

Judge health by httpjet_cache_hit_ratio_real (non-loopback lookups), httpjet_pagecache_{hits,misses,stores,purges,evictions}_total, the per-vhost httpjet_cache_{hits,misses}_by_vhost{vhost} split (2026-10-04), disk_{read,write,full}_errors_total, missing_file_invalidations_total, poisoned_locks_total (nonzero = indexes may be inconsistent), and direct content checks — never by a transient raw file count. The full metric list is on Observability.

Deploy gates

These scripts must pass before --page-cache ships anywhere; they run against a loopback instance and each catches a class of bug that has actually happened:

1. cache_xurl_test.sh

   Three distinct URLs keep their own content — the front-controller poisoning class.

2. cache_scheme_test.sh

   An HTTP-origin canonical 301 must not poison the HTTPS entry — the "Too Many Redirects" loop.

3. cache_xhost_test.sh

   Foreign-host isolation.

4. cache_private_test.sh

   Private-tier session isolation, plus the shared-paths case when configured.

5. cache_persist_test.sh

   With a file tier: waits on /__hj_cache_ready and asserts the boot scan restored entries. It restarts the service — run once (five restarts a minute trip systemd's start limit).

6. cache_sie_test.sh · cache_capsule_purge_test.sh

   Stale-if-error (kills its own lsphp mid-test) and the capsule purge invariant.