How we compute referring domains

Backlink edges come from monthly Common Crawl WARC ingest (CC0-licensed public data) filtered against the per-workspace target-host allowlist, plus focused re-fetches of source pages for edges whose target host is tracked.

Referring-domain counts and referring-page counts are computed by ClickHouse `uniqMerge` and `sum` against the `backlink_referring_domains` materialized view.

Coverage

We do not aim to match Ahrefs' ~35 T link universe. Realistic coverage is 10–20 % of their footprint for any given host; the trade-off is a fully self-hosted pipeline with transparent methodology and no third-party API dependency.