⌘K
Change language Switch ThemeSign In
Narrow Mode
Cloudflare Launches Crawl API: Supporting Full-Site Crawling and Structured Output ==================================================================================
Cloudflare Launches Crawl API: Supporting Full-Site Crawling and Structured Output ==================================================================================  ### 小互
@xiaohu
科技圈最大的背叛😂
做反爬虫起家的 Cloudflare，转头发布了一个爬虫 API
一次调用爬完一整个网站
核心卖点：
• 给一个网址，自动爬完整站，返回HTML/Markdown/JSON
• 支持 JS 渲染（React/Vue 动态站也能爬）
• 增量爬取：只爬更新过的页面，省时省钱要什么数据，直接返回结构化 JSON
• 增量爬取：只爬更新过的页面，省时省钱 Show More
Mar 12, 2026, 2:34 AM View on X
25 Replies
30 Retweets
249 Likes
26.7K Views  小互 @xiaohu
One Sentence Summary
Cloudflare introduces a new crawler API featuring JS rendering, incremental crawling, and direct structured data output optimized for AI.
Summary
Cloudflare, a company built on anti-bot technology, has made a surprising move by launching its official crawling tool, the Crawl API. This powerful tool supports automated full-site crawling, dynamic web rendering (for React/Vue sites), incremental updates (crawling only modified pages), and direct output in HTML, Markdown, or structured JSON. It's a game-changer for developers who need to acquire high-quality AI training data at scale.
AI Score
88
Influence Score 124
Published At Today
Language
Chinese
Tags
Cloudflare
Crawl API
Web Crawling
Data Acquisition
AI Development HomeArticlesPodcastsVideosTweets
Cloudflare Launches Crawl API: Supporting Full-Site Crawl... ===============