Rewrite engine & .htaccess

Overview

The rewrite stage runs the vhost's inline <rewrite> rules first, then the .htaccess chain for the resolved directory — OpenLiteSpeed's order. One entry point, evaluate(), returns exactly one outcome per pass: Rewritten, Redirect, Proxy, Forbidden, Gone or Unchanged.

flowchart TB
  classDef out fill:#13203a,stroke:#818cf8,color:#fff
  classDef bad fill:#2a1420,stroke:#e879f9,color:#fff
  A["request path (original URI kept aside)"] --> B["inline <rewrite> rules (vhost XML)"]
  B --> C["HtaccessCache: load chain docroot → dir<br/>mtime-invalidated · depth ≤ 64 · unreadable file ⇒ deny-all sentinel"]
  C --> D["SetEnvIf / SetEnvIfNoCase<br/>(lazy header access)"]
  D --> E["pre-rewrite access<br/>Order/Allow/Deny · Require · scoped auth"]
  E --> F["dense-DFA prefilter<br/>which RewriteRules could match?"]
  F --> G["evaluate candidates<br/>linear regex_automata, fancy-regex only for lookaround/backrefs"]
  G --> H("outcome")
  H --> R1["Rewritten<br/>(front controller: REQUEST_URI preserved)"]:::out
  H --> R2["Redirect [R]"]:::out
  H --> R3["Proxy [P]<br/>(loopback/link-local/unix targets screened)"]:::out
  H --> R4["Forbidden [F]"]:::bad
  H --> R5["Gone [G]"]:::bad
  H --> R6["Unchanged → suffix routing"]:::out
  R1 -- "crossed a directory" --> C
  click C "rewrite.html#loading"
  click F "rewrite.html#prefilter"
  click E "rewrite.html#access-control"

Per-directory loading

An HtaccessCache loads each directory's file (name from accessFileName, default .htaccess) with mtime invalidation, so an edit takes effect on the next request without a reload. The parsed form holds the directive list, compiled regexes, scoping blocks, the auth realm map and the memo-eligibility verdict. The chain from docroot to the request's directory is bounded at 64 levels; past that a deny-all sentinel applies. Since 2026-09-16 an unreadable or non-UTF-8 .htaccess also fails closed to that sentinel rather than being skipped.

Vhost-level switches: <htAccess><allowOverride>0 disables the whole chain (even with autoLoadHtaccess), while autoLoadHtaccess alone loads files without enabling general overrides. Candidate configuration generations own their own htaccess, rewrite-outcome and UA caches, so a reload never reintroduces stale results.

The dense-DFA prefilter

Rather than test every rule's regex per request, the eligible patterns compile once into one fully-determinized dense DFA (regex_automata::dfa::dense) searched a single time to learn which rules could match. Only those run for real. A dense DFA holds no lazy transition state, so the search is cacheless: no per-search cache allocation and no cross-thread pool lock — one mutex fewer on the request path at high concurrency.

let hits = prefilter.search(path);          // which rules *could* match — one DFA pass
for rule in hits {
    if let Some(m) = rule.regex.find(path) { return rule.apply(m); }   // substitute + flags
}
Outcome::Unchanged

Linear vs backtracking matching

A flagged rule runs through a CompiledRegex that prefers regex_automata::meta::Regex with an explicit thread-local cache keyed per pattern, and falls back to fancy-regex only for constructs the linear engine cannot express. The split is exact, not approximate: a pattern the linear engine accepts has identical match semantics under both, so the choice is invisible.

A regex that errors at match time fails closed toward the rule's polarity: a deny that cannot evaluate counts as matched; an allow that cannot evaluate is withheld (hardened 2026-09-06 alongside the spoofing checks).

Rewrite-outcome cache & UA classification

For chains whose inputs are path-cacheable, the whole rewrite decision is memoized for --rewrite-outcome-ttl-ms (default 1000 ms — which also bounds -f/-d filesystem staleness and lets .htaccess edits land within a second; 0 disables). The key is vhost/scheme/method/host/path/query plus any keyable header variables the chain reads. Counters: httpjet_rewrite_outcome_{hits,misses,uncacheable}_total.

--rewrite-ua-classify (default off, a deploy-time decision) keys User-Agent-reading chains by the bitmap of matching UA RewriteConds instead of the raw UA string, when parse-time analysis proves the outcome depends on the UA only through those matches. Real-world UA diversity collapses onto a handful of entries (httpjet_rewrite_ua_classify_entries). Since 2026-09-06 header-dependency collection continues past the UA classification so a chain that also reads Origin or Accept keys correctly.

The fast-path memo and chain eligibility

The on-core finished-response memo (default on; --no-fast-memo / HJ_FAST_MEMO=0) serves repeat cookieless static GETs without running the pipeline at all — per thread, 1 s TTL, 64 MiB/worker, only for 200 full bodies without Set-Cookie, and only after each distinct key has passed the full pipeline once. Whether a docroot's chain is eligible is a parse-time verdict (MemoClass, fail-closed) that httpjet check prints per vhost.

flowchart TB
  classDef ok fill:#0f2a24,stroke:#34d399,color:#fff
  classDef bad fill:#2a1420,stroke:#e879f9,color:#fff
  A["static GET/HEAD on the on-core path"] --> B("docroot chain MemoClass<br/>(parse-time verdict, shown by httpjet check)")
  B -- "reads Remote_Addr / Server_Addr / Request_Protocol,<br/>Allow/Deny from net|env|host, REDIRECT_STATUS,<br/>%{HTTP_COOKIE} %{REMOTE_ADDR} %{TIME*} …" --> X["ineligible → full pipeline every time<br/>httpjet_fast_memo_ineligible_total"]:::bad
  B -- "path-cacheable + keyable inputs only" --> C("request shape")
  C -- "cookie · Range · If-* · Authorization" --> X
  C -- "cookieless GET, 200, full body, no Set-Cookie" --> D["key = listener · scheme · trusted-proxy bit · vhost · Host · path?query · Accept-Encoding<br/>+ VARY-SET: raw value of every header the chain reads (Origin, UA bitmap …)"]
  D --> E("memo hit within 1 s TTL?")
  E -- yes --> H["serve finished bytes<br/>zero pipeline work"]:::ok
  E -- no --> P["full pipeline once → store (64 MiB/worker)"]
  P --> H

SetEnvIf & lazy headers

SetEnvIf / SetEnvIfNoCase evaluate with lazy header access: the engine hands each rule a callback that materializes headers only if the rule references one, and the attribute compare is case-insensitive without allocating a lowercased copy. A request matching no rule pays nothing. Since 2026-09-16, SetEnvIf/[E=] cannot set server-computed CGI names (REMOTE_ADDR, HTTPS, CONTENT_LENGTH, …) — those writes are denied.

SetEnvIfNoCase User-Agent "bot|crawler" is_bot=1
SetEnvIf Request_URI "\.(?:png|jpg|css|js)$" is_asset=1
RewriteCond %{ENV:is_bot} =1
RewriteRule ^/expensive/ - [E=no-cache:1]      # [E=no-cache] also bypasses the page cache

Front-controller routing

XenForo, WordPress and most PHP frameworks rewrite every URL to /index.php. The engine keeps the original REQUEST_URI and pins the resolved script separately (SCRIPT_FILENAME via the LsapiScript extension, SCRIPT_NAME/QUERY_STRING into the env). PHP sees the front controller; everything downstream still sees the URL the visitor asked for — which is why the page cache keys by the original URI.

Access control

<Files> / <FilesMatch> / <Directory> / <DirectoryMatch> / <If> scoping carries Order / Allow from / Deny from and Require, ANDed across every enclosing block. A <Directory> deny applies to the directory and everything beneath it. Require all granted grants; Require all denied and every other predicate (ip, host, env, expr, method, …) fails closed to deny rather than being approximated — a shipped auth file that httpjet cannot honor 403s rather than opening up. Directory patterns accept mapped filesystem paths as well as the docroot-relative convention.

HTTP Basic auth

AuthType Basic + AuthName + AuthUserFile + Require valid-user | user … protect a subtree. Since the 2026-09-06 review the model separates parsed policy from the effective realm: requirements resolve in order across scopes, a metadata-only section cannot erase an inherited requirement, an explicit Require replaces it, exact usernames are preserved, and the realm is enforced against the original, rewritten, index, script/PATH_INFO and canonical filesystem targets before cache lookup, with a served-target backstop. Unsupported or incomplete active auth fails closed. Valid credentials set REMOTE_USER; failures return 401 with the challenge.

AuthType Basic
AuthName "Staging"
AuthUserFile /srv/example.test/conf/htpasswd
Require valid-user

htpasswd hashes: bcrypt ($2y$/$2b$/$2a$, htpasswd -B, cost capped at 14 — a higher cost is refused before hashing) and {SHA}, verified in constant time; an unknown user pays one bcrypt at the file's recorded cost, so user enumeration costs nothing. Plaintext entries, $apr1$ and crypt() hashes verify false (matching Apache on Unix; legacy schemes get a parse-time warning). The AuthUserFile must resolve inside the vhost docroot — or a directory listed in --auth-user-file-roots — checked on the opened descriptor's real path (symlinks followed, ./.. rejected in relative paths, read capped at 1 MiB, a planted FIFO cannot block the open); an unreadable or out-of-root file is a 500 with an error-log entry, never a silent 401, and check warns about out-of-root paths. Authenticated static responses get Cache-Control: private, no-store from automatic expiry when no explicit policy exists (#414).

Directives reference

Unrecognized directives are ignored leniently so vendor .htaccess files remain loadable. The planned first step for the header family is making unsupported forms visible in check --strict before adding more mutation behavior.

Per-context overrides & sub_filter

Alongside the per-directory chain, XML contexts rewrite responses and override limits for a URI prefix (reference: Configuration → Contexts):

• subFilter — literal SEARCH => REPLACEMENT substitution between the expires and compress stages, 200 responses only, un-encoded bodies of matching subFilterTypes (default text/html). Buffered bodies filter in place; a stream past subFilterMaxBody (4 MiB) passes through raw, never half-filtered. Content-Length is recomputed, ETag dropped, Last-Modified off unless kept. sub_filter paths are memo-ineligible and bypass the page cache (double-substitution safety).
• maxReqBodySize — per-context body cap enforced at dispatch for all terminals (413).
• responseTimeout (proxy paths) and bandwidthLimit — override the backend deadline and the server-wide egress rate.

Reproducible checks

cargo test -p hj-rewrite                                   # parser, evaluator, access, auth, cache directives
cargo test -p hj-rewrite --test header_compat              # characterization fixtures for the Header boundary
cargo test -p hj-rewrite --test redirect_repro
bash scripts/fast_memo_parity_test.sh                      # memo-on vs memo-off byte identity on a real chain