Update, 11/16/2006: See this comment from Pingdom on the issue. They claim they are tweaking the GIGRIB bot for better behavior and will soon have a working monitoring page on the other end of the referrer. (Preview of that here.)
Update, 2007: Well, the public monitoring page never happened, and Pingdom’s spoofed referrers still forward to their main page. So yeah, they’re still referrer spammers.
My entry on Pingdom’s referrer spam gets a response in the comments from Pingdom themselves:
What Pingdom says:
Crawling the Internet for statistics and information is nothing new. Search engines and other statistical tools have been doing this since the start of the Internet. The referrer link you mention will be leading to a stats page as soon as the service goes live. The page will contain very basic statistics about your site, such as IP address, country of your server, etc. We are not collecting any visitor statistics (this is impossible for us or anyone else to do anyway) or any private information.
I respond to that in the comments, but here it is again just for purposes of clarity.
My response:
The issue, “Pingdom,” isn’t the crawling, it’s the spoofed referrer in the headers. You say that pingdom.com/monitor/whatever.com will eventually point to a working page, but right now it just rewrites to the front page of Pingdom, a page which does not actually link to the site being crawled. So the name of Pingdom and the name of the crawled site are being search-engine optimized in Pingdom.com’s favor in public referrer logs without any current reciprocal link.
That’s referrer spam, and that’s unethical. I would not be complaining about this if the referrer field in the uptime crawler were blank, or led to a current, working page with an actual link to the site, the way the Whois.sc crawler does it. As it is, Pingdom’s in my blacklist, until such time that the monitor link actually returns a real page.
I must add that it’s even worse for Pingdom’s reputation that their monitor bot is working from a server with a questionable history, and hosted by EV1, a company which Michael Pollitt has previously complained about as being lax on spammer activity.
Guys at Pingdom, I’m trying to help you out here. I’m not hating or trying to be an irritant; I’m pointing out where your bot practices are falling short of what people would expect of an ethical company. Uptime monitoring is fine, crawling web content for legitimate reasons is fine, in every way Pingdom appears to be a fine and legitimate service, except for those spoofed referrers. It’s the same reason I’m blocking RSSMicro’s bot.
Other complaints about the Pingdom bot’s behavior on various forums and weblogs follow. A lot of people are having trouble figuring out the purpose of the bot because of the referrer redirect, and are getting suspicious:
- Textdrive
- iHelpYou.com SEO Best Practices
- Silicon Vision (scroll down to below the photograph)
- DigitalPoint Forums
- WebMasterWorld
Also check out this Google search for the Pingdom referrer string, showing how the spoofed referrer is coming up on public logs everywhere.
On a related note, the original entry on Pingdom is a huge magnet for all sorts of comment spam now, mostly of the standard pharmaceutical and pornographic variety. MT’s spam filters have been catching all of it so far, but it’s interesting that the overwhelming majority of link spam I’ve been receiving is now targeted at that single entry, even more than at my older entries on referrer, comment, and trackback spam attacks.