Discover / crawl-preflight.postnov01.workers.dev

Crawl Pre-Flight

□ No challenge detected

Crawl pre-flight for a domain: will you be blocked, is content behind a paywall, and how much is actually there. Returns per-bot robots.txt verdicts for 16 AI crawlers, crawlable page count from sitemaps, detected CDN and paid-access signals (x402, TollBit), plus a fetch/skip recommendation. Call before spending requests on an unknown domain.

Listed price
$0.02
Challenge detected
0 of the last 7 checks
Last checked
17 Oct, 18:25 UTC
In our index since
26 Sept 2026

Endpoint

curl -i https://crawl-preflight.postnov01.workers.dev/crawl-check/:domain
# 402 Payment Required: pay with an x402 client (USDC on Arbitrum),
# then send the same request again with the payment attached.
1 listed endpoints · on x402 Bazaar

Checks and payment terms on this page refer to the endpoint above. Other listed endpoints have not been checked individually.

  • GEThttps://crawl-preflight.postnov01.workers.dev/crawl-check/:domainNo price listedCrawl pre-flight for a domain: will you be blocked, is content behind a paywall, and how much is actually there. Returns per-bot robots.txt verdicts for 16 AI crawlers, crawlable page count from sitemaps, detected CDN and paid-access signals (x402, TollBit), plus a fetch/skip recommendation. Call before spending requests on an unknown domain.

A dash means the listing gives no price or method. Copy buttons copy the full URL.

Showing 1 of 1 listed endpoints.

Agents can find it through the Superstables MCP server and JSON API: GET /api/v1/services/crawl-preflight.postnov01.workers.dev

Recent checks

29 Sept 2026One bar per check, showing up to 20 recent checks. Coloured bars show relative response time, with a minimum visible height; full-height grey bars mean no challenge was detected. “Challenge detected” means HTTP 402 or another detected payment-challenge signal.17 Oct 2026
All 7 checks
When (UTC)ResultHTTPResponse
17 Oct, 18:25 UTCNo challenge detected400735ms
11 Oct, 19:34 UTCNo challenge detected400226ms
10 Oct, 03:32 UTCNo challenge detected400188ms
8 Oct, 21:32 UTCNo challenge detected400234ms
7 Oct, 15:42 UTCNo challenge detected40093ms
6 Oct, 15:31 UTCNo challenge detected400431ms
29 Sept, 05:02 UTCNo challenge detected400165ms

Questions

Did our last probe detect a payment challenge?

No. At our last probe on 17 Oct, 18:25 UTC, we detected no payment challenge.

What price is recorded for Crawl Pre-Flight?

Its listed price is $0.02.

What payment terms does Crawl Pre-Flight advertise?

The listing advertises x402, Arbitrum, Base, Polygon and USDC. It names cdp as its facilitator. Check the seller's current payment challenge and authentication requirements before paying.

Where is Crawl Pre-Flight listed?

In x402 Bazaar. Superstables first indexed it on 26 Sept 2026.

Something wrong on this page? Tell us.