{"sku":"crawler-policy-sentinel","label":"AI-crawler policy sentinel","amount_cents":null,"cadence":null,"for_sale":false,"withdrawn":{"since":"2026-08-24","reason":"The 100,000-domain panel this feed was built on was withdrawn, so daily robots collection stopped on that date. The sealed archive we already collected stays readable."},"summary":"A site's robots.txt is free to read and impossible to look back at: publishers overwrite it in place, so the day a publisher started blocking a crawler is simply gone. We have held a dated daily copy across a 100,000-domain panel since 2026-06-09. Name the domains you care about and we tell you who started blocking which AI crawler, who stopped, and on which day — with the sealed snapshot behind every row.","delivers":["A monthly change feed for the domains you name, across the AI crawler product tokens we track","Who moved to blocked, who moved back, and the exact snapshot date the change first appeared in","The wildcard-block case counted separately — a site that blocks every crawler made no decision about AI in particular","Every row carries the two snapshot dates it was derived from","A domain we could not reach that day is reported unreachable, never as a policy we did not observe"],"checkout_armed":false,"request_url":null,"contact_attribution":{"contract":"product-specific contact to L0 Opportunity","path_id":"permits-offers:crawler-policy-sentinel:contact","hypothesis_id":"h-crawler-policy-sentinel-priced-offer-2026-08","artifact_id":"permits-offer-crawler-policy-sentinel-v1","cutover_at":"2026-08-02T07:10:00Z","archetype":"diff-sentinel","data_type":"tabular-registry","geo_level":"global","cta_contract_id":"permits-offer:crawler-policy-sentinel:primary","proof_url":"https://permits.ustechautomations.com/public/offers/crawler-policy-sentinel","proof_occurred_at":"2026-08-02T07:10:00Z"},"sample":null,"sample_status":{"state":"UNKNOWN","reason":"The catalog points to a specimen, but its compiled artifact cannot be read. No current proof is asserted."}}