HonestHook

Sign in

Blog ·

A disabled source and a broken source were the same row in my collector log

I run an hourly collector that reads public developer platforms and writes what it finds into an archive. Which platforms it should read is data, not code: one row per platform in a Postgres table, with a boolean column ativo meaning "collecting right now". I counted the collector's own ledger against that table. They did not agree, and the ledger had no way to say so.

All numbers below were read on 2026-09-10, between 18:00 and 18:10 UTC, from the production database, and re-counted against the same rows on 2026-09-22 before publishing.

The count

The ledger (coleta) holds one row per run × platform × topic. 3,038 rows. Three states:

Six errors in 3,038 attempts. 99.8% not-error, on a system where I already knew three sources had never collected anything.

Here is where the 619 empties came from. The ativo column is as it read on 2026-09-10:

platform ativo empty rows
reddit false 162
instagram false 162
tiktok false 162
bluesky false 131
lobsters true 2

Those 619 rows cover three different situations:

Off, blocked, and quiet. All three are written as the same string. In all 619 vazio rows, erro_tipo is NULL and erro_detalhe is NULL. There is no column that separates them.

42 hours of calling sources that were marked off

Reddit's row was created with ativo = false on 2026-09-06 at 19:07 UTC. TikTok and Instagram were created false at 22:18 UTC the same day. No later migration flips any of them to true.

The collector logged attempts against all three anyway: 162 each, 486 in total, across 42 distinct hourly windows, first at 2026-09-07 21:07:18 UTC and last at 2026-09-09 15:07:18 UTC. The arithmetic is exact and boring: 54 runs × 3 topics = 162 per platform.

So the table said off, the scheduler kept going, and the ledger recorded 486 rows that look identical to a quiet hour on a healthy source. The config and the code had diverged, and the log was the wrong instrument to notice it with — it recorded faithfully that nothing came back, which was true and useless.

The calls stop at 2026-09-09 15:07:18 UTC. I have not verified from the database what changed at that moment; there is no migration at that timestamp.

The one source that failed loudly

Bluesky is the interesting counterexample. It was genuinely being called, and it is the only platform in the entire ledger that ever produced an erro row — all six of them:

erro_tipo:    bloqueio
erro_detalhe: tentativas_esgotadas apos 3: http 403; 10/10 termos falharam;
              30 tentativas gastas

That row tells me what happened, how many terms failed, and how many retries it burned. Its 162 attempts split into 25 ok, 131 vazio, 6 erro — a source alternating between working and not, which is worse than one that is plainly down, because the ok windows are real and the archive keeps them.

It also cost the most. Total recorded duration across Bluesky's 162 attempts: 6,668,359 ms, about 1 hour 51 minutes of collector wall-clock, averaging 41 seconds per attempt — mostly waiting. Reddit, Instagram and TikTok together spent 148,724 ms across their 486 attempts, about 306 ms each (305, 305 and 307 respectively). The expensive failure and the cheap one are both vazio.

The alerting did not help, by design

I have an alert table for stopped sources. On 2026-09-10 it had zero rows. That is not a bug in the alerter: a gap is defined as "a pair that was producing and stopped", and a pair that never produced anything cannot stop. The definition is right. The consequence is that a source which never worked is invisible to it forever, and the three sources with the most empty rows in my ledger are exactly the three that never produced.

It is no longer empty. Re-reading the same table on 2026-09-22, it holds 8 incidents, none of them still open — every one opened by a mechanism built after this reading, none by the gap definition above.

What I am not claiming

Reproduce it on your own collector

The whole finding is two queries. If your collector writes a per-attempt ledger:

-- 1. how many states does "nothing came back" actually cover?
select estado, count(*),
       count(*) filter (where erro_tipo is not null) as com_tipo
from coleta group by 1 order by 2 desc;

-- 2. which of those empties belong to sources you switched off?
select c.plataforma, p.ativo, count(*) as vazios,
       min(c.janela) as primeira, max(c.janela) as ultima
from coleta c join plataforma p on p.id = c.plataforma
where c.estado = 'vazio'
group by 1, 2 order by 3 desc;

If the second query returns rows with ativo = false, your scheduler and your configuration are two different lists, and your log cannot tell you which one is lying. The cheap fix is not a new alert. It is making the empty case carry a reason, so that "off by choice", "blocked", and "quiet hour" stop sharing a column.

Happy to share the rest of the ledger schema, or the per-platform duration numbers, if that is useful to anyone measuring the same thing.

Trend data with a memory

Every social API answers what’s trending now, then throws it away. HonestHook keeps the hourly archive, so you can ask what gained traction.

Free key, 1,000 credits a month, no card →