Have you ever wanted to surf the web as a bot?
Ever wondered what a bot gets to see online that you don't?
Refuse to be Human lets you pretend to be a bot from the most popular search engines Google, Baidu (China) or Yandex (Russia) to find out.
It is a simple web extension which changes your browser’s user agent to that of one of the three bots.
While Google is the most popular web search engine in the world, Yandex is number one in Russia and Baidu the number one in China. All three companies use web crawlers which surf the web extensively to scrape the contents for their search engines. Only what the bots see is indexed and can later be found by other users through their search. By changing your settings to match those of the bot, you become one of thousands of bots browsing the web to index its contents.
Surfing the web as a search engine bot gives you access to what is referred to as the grey web, a layer of content only visible to bots. In some cases it might give you access to websites and archives that are usually hidden behind a paywall. While the website owner will present the regular user with a login page, they will give web crawlers access to their full archive in order for it to be represented in Yandex’s, Baidu's or Google's search results.
1000 Bots - Surf the Web as Googlebot
To surf the internet as Googlebot – the most influential Internet user of all – download the browser extension for Firefox or check out the 1000 Bots website to find instructions for other browsers.
The Yandex edition
Download the browser extension to surf the net as a Yandex bot – the number one search engine in Russia – for Firefox and Chrome.
The Baidu edition
Download the browser extension to surf the net as a Baidu bot – the number one search engine in China – for Firefox or Chrome.
