← All endpoints
api.trustsource.cc robots
https://api.trustsource.cc/robots
payable
robots.txt and AI crawler policy check. Tells an agent whether a website permits crawling and whether it blocks AI bots such as GPTBot, ClaudeBot, Google-Extended, PerplexityBot and CCBot, before scraping, RAG ingestion, training data collection or archiving. Parses robots.txt and returns a crawl-policy tier with per-bot allow and disallow rules and sitemap URLs.
We sent a real
GET request to this endpoint and it
answered 402 with payment requirements we could parse.
- Price
- $0.002
- Network
- base
- Pay to
- 0xC1FE5Ce97fB6038527D9660cF290b716a264e874
- Spec dialect
- v2 (declares x402Version 2)
- Transport
- PAYMENT-REQUIRED header
- HTTP method
- GET
- Response latency
- 305 ms
- Payment options
- 1 usable of 1
- Last checked
- 2026-08-10T09:13:51.330Z (1d ago)
- Category
- finance-crypto
Re-check it yourself:
curl "https://api.the402.dev/validate?url=https%3A%2F%2Fapi.trustsource.cc%2Frobots"