TLS fingerprinting identifies a client from the shape of its TLS handshake — cipher order, extensions, supported groups — all of which differ between a real browser and a scripted HTTP client. It happens before your request line is ever sent.
You cannot fix it with a User-Agent header. The header is in the request; the fingerprint is in the connection.
The failure mode is not a block
This is the part that catches people. A fingerprint mismatch often does not produce a 403. It produces a successful response with the data missing.
Measured on one large platform:
| Client | Response | Contains the counter object |
|---|---|---|
Plain curl |
200, 268 KB | No |
| Browser-equivalent fingerprint | 200, 960 KB | Yes |
Both are HTTP 200. Both return a real page. One of them is missing the only thing you wanted. Nothing in your code errors — which is why this specific case is documented on the route page rather than buried in a troubleshooting doc.
Why it is done this way
A hard block is easy to detect and easy to route around. Serving a lighter page to non-browser clients is cheaper for the platform, harder to notice, and degrades scrapers quietly rather than provoking them. From the platform's side it is the better design.
What it means for your stack
- A header-only HTTP client will not do. You need something that reproduces a browser's handshake, or a headless browser.
- Bandwidth goes up, not down. The page that contains the data is often three or four times larger than the one that does not — 960 KB against 268 KB in the case above. That is the real cost of collection.
- Your health checks must look for the data, not the status. A monitor that asserts
200will stay green through this entire failure.
The general rule
Anything that can return a page without the data needs an anchor field — a value that must be present for the response to count as an answer. Status codes are not a completeness check, and treating them as one is the most common version of a silent failure.