A headless browser is a real browser engine — usually Chromium — driven by code, with no visible window. It executes JavaScript, holds cookies, and negotiates TLS exactly as a browser does, because it is one.
It is the heaviest tool available for reading a page, and sometimes the only one that works.
When you actually need it
| Situation | Plain HTTP request | Headless browser |
|---|---|---|
| Data is in the served HTML | Works | Overkill |
| Data is in an embedded JSON blob | Works | Overkill |
| Data is fetched by JavaScript after load | Fails | Needed |
| Client is filtered by TLS fingerprint | Returns a page without the data | Needed |
The first two cover more cases than people expect. A very large share of pages ship their own data inside the HTML as structured data or a hydration blob — see web scraping — and reaching for a browser before checking is how a cheap job becomes an expensive one.
What it costs
- Memory and CPU, per concurrent page, measured in hundreds of megabytes rather than kilobytes.
- Latency, because you wait for a real page load rather than a single response.
- Bandwidth, because a browser fetches the images, fonts and scripts a parser would have skipped.
- Operational surface — a browser pool that leaks processes is its own class of incident.
The middle option
Between a plain client and a full browser sits an HTTP client that reproduces a browser's TLS handshake without running a rendering engine. For pages whose data is in the HTML but whose client is being filtered, that is usually the right answer: you get the complete page at a fraction of the cost.
That is the route we take, and it is why our payload figures are measured per platform — under a kilobyte on an open protocol, hundreds of kilobytes on a large commercial page.
The check that matters either way
Whatever fetches the page, assert on the data, not the status. A headless browser will happily load an "account not found" page and report success, so the anchor field requirement does not go away just because you brought a heavier tool.