| Googlebot Smartphone | Google | Googlebot | Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html) |
| Googlebot | Google | Googlebot | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/131.0.0.0 Safari/537.36 |
| Googlebot News | Google | Googlebot-News | Googlebot-News |
| Googlebot Images | Google | Googlebot-Image | Googlebot-Image/1.0 |
| Googlebot Video | Google | Googlebot-Video | Googlebot-Video/1.0 |
| Google StoreBot (Mobile) | Google | Storebot-Google | Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012; Storebot-Google/1.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Mobile Safari/537.36 |
| Google StoreBot (Desktop) | Google | Storebot-Google | Mozilla/5.0 (X11; Linux x86_64; Storebot-Google/1.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 |
| Google-InspectionTool (Mobile) | Google | Google-InspectionTool | Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Mobile Safari/537.36 (compatible; Google-InspectionTool/1.0;) |
| Google-InspectionTool (Desktop) | Google | Google-InspectionTool | Mozilla/5.0 (compatible; Google-InspectionTool/1.0;) |
| GoogleOther | Google | GoogleOther | GoogleOther |
| GoogleOther-Image | Google | GoogleOther-Image | GoogleOther-Image/1.0 |
| GoogleOther-Video | Google | GoogleOther-Video | GoogleOther-Video/1.0 |
| Google-Extended | Google | Google-Extended | Google-Extended |
| AdsBot Mobile Web | Google | AdsBot-Google-Mobile | Mozilla/5.0 (iPhone; CPU iPhone OS 14_7_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.2 Mobile/15E148 Safari/604.1 (compatible; AdsBot-Google-Mobile; +http://www.google.com/mobile/adsbot.html) |
| AdsBot | Google | AdsBot-Google | AdsBot-Google (+http://www.google.com/adsbot.html) |
| AdSense | Google | Mediapartners-Google | Mediapartners-Google |
| Google-CloudVertexBot | Google | Google-CloudVertexBot | Google-CloudVertexBot |
| Bingbot | Bing | Bingbot | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/131.0.0.0 Safari/537.36 |
| Bingbot Smartphone | Bing | Bingbot | Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Mobile Safari/537.36 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) |
| MSNBot | Bing | MSNBot | msnbot/2.0b (+http://search.msn.com/msnbot.htm) |
| MSNBot-Media | Bing | MSNBot-Media | msnbot-media/1.1 (+http://search.msn.com/msnbot.htm) |
| AdIdxBot | Bing | AdIdxBot | Mozilla/5.0 (compatible; adidxbot/2.0; +http://www.bing.com/bingbot.htm) |
| BingPreview | Bing | BingPreview | Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 BingPreview/1.0b |
| Yahoo! | Other search engines | Slurp | Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) |
| DuckDuckGo | Other search engines | DuckDuckBot | DuckDuckBot/1.1; (+http://duckduckgo.com/duckduckbot.html) |
| Baidu | Other search engines | Baiduspider | Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) |
| Baidu - Render (Desktop) | Other search engines | Baiduspider | Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html) |
| Baidu - Render (Mobile) | Other search engines | Baiduspider | Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13B143 Safari/601.1 (compatible; Baiduspider-render/2.0; +http://www.baidu.com/search/spider.html) |
| Yandex | Other search engines | Yandex | Mozilla/5.0 (compatible; YandexBot/3.0; +http://yandex.com/bots) |
| Applebot | Other search engines | Applebot | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.1 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot) |
| Applebot Smartphone | Other search engines | Applebot | Mozilla/5.0 (iPhone; CPU iPhone OS 17_5 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.5 Mobile/15E148 Safari/604.1 (Applebot/0.1; +http://www.apple.com/go/applebot) |
| Facebook | Social media | facebookexternalhit | facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php) |
| Twitter / X | Social media | Twitterbot | Twitterbot/1.0 |
| LinkedIn | Social media | LinkedInBot | LinkedInBot/1.0 (compatible; Mozilla/5.0; Apache-HttpClient +http://www.linkedin.com) |
| Screaming Frog | SEO tools | Screaming Frog SEO Spider | Screaming Frog SEO Spider/21.0 |
| Screaming Frog (Mobile) | SEO tools | Screaming Frog SEO Spider | Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Mobile Safari/537.36 (compatible; Screaming Frog SEO Spider/21.0) |
| Botify | SEO tools | Botify | Mozilla/5.0 (compatible; botify; http://botify.com) |
| OnCrawl | SEO tools | OnCrawl | Mozilla/5.0 (compatible; OnCrawl/1.0; +http://www.oncrawl.com/) |
| Moz (Campaign/Diagnostics) | SEO tools | rogerbot | rogerbot/1.2 (https://moz.com/help/guides/moz-procedures/what-is-rogerbot) |
| Moz (Mozscape/Freshscape) | SEO tools | DotBot | Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com) |
| Majestic | SEO tools | MJ12bot | Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/) |
| Ahrefs | SEO tools | AhrefsBot | Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/) |
| Semrush | SEO tools | SemrushBot | Mozilla/5.0 (compatible; SemrushBot/7~bl; +http://www.semrush.com/bot.html) |
| Internet Archive | SEO tools | ia_archiver | ia_archiver (+http://www.alexa.com/site/help/webmasters; crawler@alexa.com) |
| PageSpeed Insights (Mobile) | SEO tools | * | Mozilla/5.0 (Linux; Android 11; moto g power (2022)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Mobile Safari/537.36 Chrome-Lighthouse |
| PageSpeed Insights (Desktop) | SEO tools | * | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36 Chrome-Lighthouse |
| OpenAI | AI crawlers | GPTBot | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot) |
| SearchGPT | AI crawlers | OAI-SearchBot | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; OAI-SearchBot/1.0; +https://openai.com/searchbot |
| ChatGPT-User | AI crawlers | ChatGPT-User | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot |
| Perplexity | AI crawlers | PerplexityBot | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot) |