AI Crawler Configuration Review & Draft Pack
A dated observation of one customer-controlled domain's robots.txt, page-level crawler directives, and optional llms.txt, followed by edit-ready drafts. It does not deploy changes or claim crawler conduct, access control, search inclusion, training opt-out, or legal effect.
$249
one-time
Request this →Checkout for this service opens by emailed invoice link. Tell us where to send it and we reply the same working day.
What you get
- Dated PDF and matching CSV for one controlled domain
- Observed robots.txt groups by named crawler and up to 50 public URL findings
- Existing llms.txt classified as genuine plaintext, soft-200 HTML, empty, or other
- Edit-ready robots.txt and optional llms.txt drafts for the customer to apply
- Official-source appendix, fetch dates, and SHA-256 delivery manifest
Specimen status UNKNOWN
The catalog points to a specimen, but its compiled artifact cannot be read. No current proof is asserted.
View the machine-readable offer and no-specimen status
US Tech Automations is a data and automation company. These are our own service terms, not a government fee, filing cost, or permit charge. Nothing on this page is legal, engineering, or financial advice.