Normalise header names so non-SAPI sources work - #617
Merged
Conversation
setHttpHeaders() filtered on the HTTP_ prefix, so it only ever understood
the _SERVER-style names PHP's SAPI produces. Headers from anywhere else —
a PSR-7 request, Symfony's HeaderBag, Swoole, a Lambda event — carry their
real names ('User-Agent'), were dropped by that filter, and left the user
agent null. The idiomatic call for those sources therefore reported every
request as human, with no error to hint otherwise:
$cd = new CrawlerDetect($request->getHeaders());
$cd->isCrawler(); // always false
Callers who worked around it with isCrawler($request->getHeaderLine(...))
got the User-Agent checked but silently lost the rest of the chain in
Fixtures\Headers — including From and Sec-CH-UA, the two headers that catch
crawlers sending a genuine browser User-Agent.
Header names are now reduced to a canonical form (uppercased, underscore
separated, HTTP_ prefix stripped) and stored back under HTTP_, so both
naming styles land on the same key and setUserAgent() is unchanged. Values
may now be arrays too, since PSR-7 and HttpFoundation both expose them that
way.
Nothing public changes: getUaHttpHeaders() still returns prefixed names,
$this->httpHeaders keeps its HTTP_ key shape, and the _SERVER prefix filter
still excludes non-header vars such as REQUEST_URI.
normaliseHeaderName() folded hyphens to underscores before testing for the prefix, so a real header named 'Http-User-Agent' became HTTP_USER_AGENT, had the prefix stripped, and was read as the actual user agent. Same for 'Http-From' and any other Http-* custom header, giving an attacker-supplied false positive. Testing the prefix before folding hyphens means only an underscore separated SAPI key can lose it. Nothing is given up: the SAPI never emits hyphenated keys, so no real source needs the conversion to happen first.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
setHttpHeaders()filtered incoming keys on theHTTP_prefix, so it only understood the_SERVER-style names PHP's SAPI produces. Headers arriving from anywhere else carry their real names, get dropped by that filter, and leave the user agentnull.That means the idiomatic integration for a PSR-7 framework silently classified every request as human. Verified against
masterbefore this change:No exception, no warning, not even an
Array to string conversionnotice — the prefix filter discards the keys before their values are ever read, soisCrawler()exits down the empty-string path.The failure mode is what makes this worth fixing. A developer wires up the obvious call, tests it in a browser, gets a correct
false, and ships. Crawler traffic is then indistinguishable from human traffic, and if CrawlerDetect is gating a rate limiter, Googlebot starts collecting 429s. That surfaces weeks later in Search Console rather than as any error in the application.A second, quieter loss
Callers who worked around this with
isCrawler($request->getHeaderLine('User-Agent'))get theUser-Agentchecked, but silently lose the rest of the chain inFixtures\Headers— 1 of 10 signals. That includesFromandSec-CH-UA, the two headers that exist precisely to catch crawlers sending a genuine browserUser-Agent. Googlebot's documented habit of doing exactly that is unreachable from a PSR-7 source today:new CrawlerDetect($request->getHeaders())false✗true✓From+ genuine Chrome UA, PSR-7 shapefalse✗true✓isCrawler($request->getHeaderLine('User-Agent'))falsefalse(inherent — one header, one signal)['HTTP_FROM' => ..., 'HTTP_USER_AGENT' => ...]true✓true✓The change
Header names are reduced to a canonical form — uppercased, underscore separated,
HTTP_prefix stripped — then stored back underHTTP_.HTTP_USER_AGENT,User-Agentanduser-agentall land on the same key, which is the keyFixtures\Headersalready lists, sosetUserAgent()needs no change at all.Values are also flattened, because
ServerRequestInterface::getHeaders()andHeaderBag::all()both returnarray<string, string[]>where the SAPI gives a pre-joined string. Without that, the naive integration stringifies to"Array"— which matches no pattern, so it too fails closed.The round-trip is lossless for every entry in the fixture:
HTTP_X_OPERAMINI_PHONE_UAX-OperaMini-Phone-UAX_OPERAMINI_PHONE_UAHTTP_X_UCBROWSER_DEVICE_UAX-UCBrowser-Device-UAX_UCBROWSER_DEVICE_UAHTTP_SEC_CH_UASec-CH-UASEC_CH_UAHTTP_DEVICE_STOCK_UADevice-Stock-UADEVICE_STOCK_UAWhy not PSR-7 support instead
Typehinting
ServerRequestInterfacewould mean depending on an HTTP abstraction — URI, body streams, uploaded files, immutability semantics — to callgetHeaderLine()once, and would cost the zero-dependency property. It also only helps one ecosystem. Normalising header names is dependency-free and fixes Swoole, HttpFoundation, RoadRunner and Lambda at the same time.It also makes a PSR-15 middleware a few lines in a satellite package, with no knowledge of this library's internal key format — consistent with how the Laravel, Symfony and Yii2 integrations already live in their own repos.
Backwards compatibility
Nothing public changes.
getUaHttpHeaders()still returns prefixed names,setUserAgent()andisCrawler()are untouched, and$this->httpHeaderskeeps itsHTTP_key shape for any subclass reaching into it. Purely additive, so a minor release.One accepted edge case, flagged for review: a bare
$_SERVER['FROM']or$_SERVER['USER_AGENT']— inherited as an environment variable rather than a real header — will now be read where before it was ignored. To change an outcome its value would also have to match a crawler pattern.The tighter alternative is to require a dash in the original key, but that breaks
From, the one single-word header in the fixture and also the most valuable Googlebot signal. I took the narrow env-var risk over losingFrom; happy to flip it if you'd rather.Who this does not affect
Worth stating plainly: classic Apache/nginx with mod_php or FPM, using the no-argument constructor, sees no behavioural change — likely still most installs. This is a correctness fix for PSR-7 frameworks, async workers and serverless runtimes, not a headline feature.
Tests
Eight new cases in
UserAgentTest.php: PSR-7 shape with array values, lowercase dashed names, scalar values,Fromhonoured from a non-SAPI source (asserting the UA alone does not match),Sec-CH-UAfrom a non-SAPI source, mixed-case SAPI names, a guard that non-header_SERVERvars such asREQUEST_URIare still excluded even when their values would match, and a guard thatgetUaHttpHeaders()keeps its prefixed names.Full suite green — 27 tests, 170,870 assertions. PHPStan level 5 at the 7.2 floor reports no errors.