Skip to content

Normalise header names so non-SAPI sources work - #617

Merged
JayBizzle merged 2 commits into
masterfrom
normalise-header-names
Jul 30, 2026
Merged

Normalise header names so non-SAPI sources work#617
JayBizzle merged 2 commits into
masterfrom
normalise-header-names

Conversation

@JayBizzle

Copy link
Copy Markdown
Owner

The problem

setHttpHeaders() filtered incoming keys on the HTTP_ prefix, so it only understood the _SERVER-style names PHP's SAPI produces. Headers arriving from anywhere else carry their real names, get dropped by that filter, and leave the user agent null.

That means the idiomatic integration for a PSR-7 framework silently classified every request as human. Verified against master before this change:

$cd = new CrawlerDetect($request->getHeaders());
$cd->isCrawler();      // false — for a self-identifying bingbot
$cd->getUserAgent();   // NULL

No exception, no warning, not even an Array to string conversion notice — the prefix filter discards the keys before their values are ever read, so isCrawler() exits down the empty-string path.

The failure mode is what makes this worth fixing. A developer wires up the obvious call, tests it in a browser, gets a correct false, and ships. Crawler traffic is then indistinguishable from human traffic, and if CrawlerDetect is gating a rate limiter, Googlebot starts collecting 429s. That surfaces weeks later in Search Console rather than as any error in the application.

A second, quieter loss

Callers who worked around this with isCrawler($request->getHeaderLine('User-Agent')) get the User-Agent checked, but silently lose the rest of the chain in Fixtures\Headers — 1 of 10 signals. That includes From and Sec-CH-UA, the two headers that exist precisely to catch crawlers sending a genuine browser User-Agent. Googlebot's documented habit of doing exactly that is unreachable from a PSR-7 source today:

Same request, four ways Before After
new CrawlerDetect($request->getHeaders()) false true
From + genuine Chrome UA, PSR-7 shape false true
isCrawler($request->getHeaderLine('User-Agent')) false false (inherent — one header, one signal)
forged ['HTTP_FROM' => ..., 'HTTP_USER_AGENT' => ...] true true

The change

Header names are reduced to a canonical form — uppercased, underscore separated, HTTP_ prefix stripped — then stored back under HTTP_. HTTP_USER_AGENT, User-Agent and user-agent all land on the same key, which is the key Fixtures\Headers already lists, so setUserAgent() needs no change at all.

Values are also flattened, because ServerRequestInterface::getHeaders() and HeaderBag::all() both return array<string, string[]> where the SAPI gives a pre-joined string. Without that, the naive integration stringifies to "Array" — which matches no pattern, so it too fails closed.

The round-trip is lossless for every entry in the fixture:

Fixture name Real header Canonical form
HTTP_X_OPERAMINI_PHONE_UA X-OperaMini-Phone-UA X_OPERAMINI_PHONE_UA
HTTP_X_UCBROWSER_DEVICE_UA X-UCBrowser-Device-UA X_UCBROWSER_DEVICE_UA
HTTP_SEC_CH_UA Sec-CH-UA SEC_CH_UA
HTTP_DEVICE_STOCK_UA Device-Stock-UA DEVICE_STOCK_UA

Why not PSR-7 support instead

Typehinting ServerRequestInterface would mean depending on an HTTP abstraction — URI, body streams, uploaded files, immutability semantics — to call getHeaderLine() once, and would cost the zero-dependency property. It also only helps one ecosystem. Normalising header names is dependency-free and fixes Swoole, HttpFoundation, RoadRunner and Lambda at the same time.

It also makes a PSR-15 middleware a few lines in a satellite package, with no knowledge of this library's internal key format — consistent with how the Laravel, Symfony and Yii2 integrations already live in their own repos.

Backwards compatibility

Nothing public changes. getUaHttpHeaders() still returns prefixed names, setUserAgent() and isCrawler() are untouched, and $this->httpHeaders keeps its HTTP_ key shape for any subclass reaching into it. Purely additive, so a minor release.

One accepted edge case, flagged for review: a bare $_SERVER['FROM'] or $_SERVER['USER_AGENT'] — inherited as an environment variable rather than a real header — will now be read where before it was ignored. To change an outcome its value would also have to match a crawler pattern.

The tighter alternative is to require a dash in the original key, but that breaks From, the one single-word header in the fixture and also the most valuable Googlebot signal. I took the narrow env-var risk over losing From; happy to flip it if you'd rather.

Who this does not affect

Worth stating plainly: classic Apache/nginx with mod_php or FPM, using the no-argument constructor, sees no behavioural change — likely still most installs. This is a correctness fix for PSR-7 frameworks, async workers and serverless runtimes, not a headline feature.

Tests

Eight new cases in UserAgentTest.php: PSR-7 shape with array values, lowercase dashed names, scalar values, From honoured from a non-SAPI source (asserting the UA alone does not match), Sec-CH-UA from a non-SAPI source, mixed-case SAPI names, a guard that non-header _SERVER vars such as REQUEST_URI are still excluded even when their values would match, and a guard that getUaHttpHeaders() keeps its prefixed names.

Full suite green — 27 tests, 170,870 assertions. PHPStan level 5 at the 7.2 floor reports no errors.

setHttpHeaders() filtered on the HTTP_ prefix, so it only ever understood
the _SERVER-style names PHP's SAPI produces. Headers from anywhere else —
a PSR-7 request, Symfony's HeaderBag, Swoole, a Lambda event — carry their
real names ('User-Agent'), were dropped by that filter, and left the user
agent null. The idiomatic call for those sources therefore reported every
request as human, with no error to hint otherwise:

    $cd = new CrawlerDetect($request->getHeaders());
    $cd->isCrawler();  // always false

Callers who worked around it with isCrawler($request->getHeaderLine(...))
got the User-Agent checked but silently lost the rest of the chain in
Fixtures\Headers — including From and Sec-CH-UA, the two headers that catch
crawlers sending a genuine browser User-Agent.

Header names are now reduced to a canonical form (uppercased, underscore
separated, HTTP_ prefix stripped) and stored back under HTTP_, so both
naming styles land on the same key and setUserAgent() is unchanged. Values
may now be arrays too, since PSR-7 and HttpFoundation both expose them that
way.

Nothing public changes: getUaHttpHeaders() still returns prefixed names,
$this->httpHeaders keeps its HTTP_ key shape, and the _SERVER prefix filter
still excludes non-header vars such as REQUEST_URI.
normaliseHeaderName() folded hyphens to underscores before testing for the
prefix, so a real header named 'Http-User-Agent' became HTTP_USER_AGENT,
had the prefix stripped, and was read as the actual user agent. Same for
'Http-From' and any other Http-* custom header, giving an attacker-supplied
false positive.

Testing the prefix before folding hyphens means only an underscore
separated SAPI key can lose it. Nothing is given up: the SAPI never emits
hyphenated keys, so no real source needs the conversion to happen first.
@JayBizzle
JayBizzle merged commit df91437 into master Jul 30, 2026
12 checks passed
@JayBizzle
JayBizzle deleted the normalise-header-names branch July 30, 2026 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant