Bot crawls

Crawl activity from AI agents and search bots. Which bots read which paths, and how often.

By bot

• ClaudeBot264
• Googlebot6
• CCBot6
• bingbot5
• ChatGPT-User4
• YandexBot2
• Claude-User1
• anthropic-ai1
• GPTBot1
• PerplexityBot1
• OAI-SearchBot1
• Perplexity-User1

By path

• /sources10
• /crawls6
• /asml6
• /apple6
• /tsmc6
• /roadmap6
• /schema6
• /asmpt5
• /broadcom5
• /amd5
• /intel5
• /5

Data dependency by AI bot

Pages, .md and .json merged per node. More distinct-day (d) revisits signal that the AI depends on that data.

• ClaudeBot264 visits · 61 nodes · 26 recurring/asmpt 9× · 2d/sources 9× · 2d/marvell 8× · 2d/asml 8× · 2d/apple 8× · 2d/meta 8× · 2d
• ChatGPT-User4 visits · 4 nodes · 0 recurring/hanmi-semiconductor 1× · 1d/tsmc 1× · 1d/asml 1× · 1d/service-account 1× · 1d
• Claude-User1 visits · 1 nodes · 0 recurring/crawls 1× · 1d
• anthropic-ai1 visits · 1 nodes · 0 recurring/google-services 1× · 1d
• GPTBot1 visits · 1 nodes · 0 recurring/asml 1× · 1d
• PerplexityBot1 visits · 1 nodes · 0 recurring/schema 1× · 1d
• OAI-SearchBot1 visits · 1 nodes · 0 recurring/sitemap.xml 1× · 1d
• Perplexity-User1 visits · 1 nodes · 0 recurring/application.properties 1× · 1d
• Bytespider1 visits · 1 nodes · 0 recurring/google-services 1× · 1d

Most-fetched data

• 10×/asmlClaudeBot, GPTBot, ChatGPT-User
• 9×/asmptClaudeBot
• 9×/sourcesClaudeBot
• 9×/tsmcClaudeBot, ChatGPT-User
• 8×/marvellClaudeBot
• 8×/appleClaudeBot
• 8×/metaClaudeBot
• 8×/micronClaudeBot
• 8×/amazonClaudeBot
• 8×/broadcomClaudeBot
• 8×/amdClaudeBot
• 8×/nvidiaClaudeBot
• 8×/intelClaudeBot
• 7×/hanwha-semitechClaudeBot
• 7×/hanmi-semiconductorClaudeBot, ChatGPT-User

Weekly trend by bot

AI crawl volume over the last 8 weeks. Rising bars signal growing dependency.

• ClaudeBotlast 8 weeks · 264 visits
• ChatGPT-Userlast 8 weeks · 4 visits
• Claude-Userlast 8 weeks · 1 visits
• anthropic-ailast 8 weeks · 1 visits
• GPTBotlast 8 weeks · 1 visits
• PerplexityBotlast 8 weeks · 1 visits
• OAI-SearchBotlast 8 weeks · 1 visits
• Perplexity-Userlast 8 weeks · 1 visits

Recent crawls

• 2026-06-21 10:28ClaudeBot/crawls
• 2026-06-21 10:28ClaudeBot/samsung-electro-mechanics.json
• 2026-06-21 10:28ClaudeBot/wonik-ips
• 2026-06-21 10:28ClaudeBot/marvell
• 2026-06-21 10:28ClaudeBot/qualcomm
• 2026-06-21 10:28ClaudeBot/hanwha-semitech.md
• 2026-06-21 10:28ClaudeBot/murata
• 2026-06-21 10:28ClaudeBot/advantest
• 2026-06-21 10:28ClaudeBot/kla
• 2026-06-21 10:28ClaudeBot/sk-siltron.md
• 2026-06-21 10:28ClaudeBot/asml
• 2026-06-21 10:28ClaudeBot/hanwha-semitech.json
• 2026-06-21 10:28ClaudeBot/apple
• 2026-06-21 10:28ClaudeBot/arm
• 2026-06-21 10:27ClaudeBot/asmpt
• 2026-06-21 10:27ClaudeBot/meta
• 2026-06-21 10:27ClaudeBot/synopsys
• 2026-06-21 10:27ClaudeBot/ase
• 2026-06-21 10:27ClaudeBot/dongjin-semichem.md
• 2026-06-21 10:27ClaudeBot/sources
• 2026-06-21 10:27ClaudeBot/sumco
• 2026-06-21 10:27ClaudeBot/tsmc
• 2026-06-21 10:27ClaudeBot/micron
• 2026-06-21 10:27ClaudeBot/amazon
• 2026-06-21 10:27ClaudeBot/globalfoundries.md