Referrer Spam – What it is and How and Why to Stop It
Referrer Spam: What It Is, Why It’s Bad, and How to Stop It
Ah, spam. Once upon a time, this was merely the name for a faintly distasteful loaf of canned meat, but with the rise of the Internet, it became something else altogether. Spam, as it is officially known, is any type of inappropriate, irrelevant, fake, or otherwise obtrusive and unwelcome message sent via mass communication channels (with the Internet being far and away the most popular avenue). Of course, in the infancy of cyber culture, it wasn’t such a problem, if only because in the old days – that is, the brief period between an almost total lack of a civilian Internet and the world we live in today, wherein kids who aren’t even old enough to shave are making money on YouTube – there wasn’t much of a user base for spammers to infect. Gradually, however, the Internet’s popularity and user pool grew larger, and before long, email spam became the flagship example of this phenomenon. Now, with the new millennium well underway, email spam has mutated, and the result is referrer spam. Read on to learn what it is, why it’s bad for you, and how to defeat it.
Referrer Spam: The What
Unlike email spam, referrer spam isn’t really designed to be read or engaged with by people. Instead, its focus is on machines and software, particularly the search engines. To be succinct, every website has a URL, or address – facebook.com, for example. When someone visits your website, they’re coming from somewhere, whether it be a Google search or from another website they’ve finished interacting with. When those visits happen, the site your visitor was on immediately before they came to you gets recorded in your website’s traffic logs as the referrer or referral URL. There’s nothing nefarious about it – a real, live person is behind it, and technically speaking, it’s just how the software underpinning the Internet works and speaks, as it were.
On the other hand, referrer spam is the generation of fake HTTP requests by specially written robots. For any number of reasons, these bots roam the Web in search of promising websites, and when they find one, they pay it a visit, sending its stat logs fake information about who they are and where they’re coming from in the process.
Referrer Spam: The Why
There are actually two parts to this section. As for the motivations of the spammers themselves, it usually boils down to simple self-interest. Their agendas are varied, and they feel that this is the best way to advance them. As for the technical aspects, and why it’s bad for legitimate website owners and operators, that’s a bit more complex. Here, however, are a few reasons why referrer spam is bad news for anyone targeted by it:
Referrer spam can seriously skew the data output from your analytics package, whether it be Google Analytics or something else, giving you a false picture of your website(s)’ visitation and usage statistics. This makes it much more difficult to understand what’s going on with your site or sites, which in turn makes it harder to form any sort of long-term strategy about how you want to use your website, the direction in which you’d like to take it in the future, and so forth. A foundation in bad information can kill a website before it even really has a chance to get off the ground.
If you used these numbers to gauge a price for buying or selling a domain or website you would be making decisions based on a false set of numbers. And possibly ruining your reputation at the same time
The Cash Crunch
Eventually, most successful website owners / operators get to a point where they feel comfortable extending invitations to advertisers in their market. The idea, of course, is that the advertisers in question get more exposure to targeted traffic, but they pay the website owner for the privilege. Now, if your website is beset by referral spam, you may be getting ten visitors a month instead of ten thousand, and if you’ve been paid on the basis of false claims, you can quickly find yourself in some very hot water.
Even if you avoid the spam issue for as long as you yourself run a website, what happens when you want to sell it? The same rotten metrics that make it hard or impossible to draw and retain reputable advertisers will make it a Sisyphean task to sell your site(s), and that’s a pity, because a well-made, highly functioning website can turn quite a pretty penny on the auction block. And let’s not forget that referrer spam is among the biggest reasons / pretexts for Google to unilaterally de-index a site from their search results altogether. When it comes to websites, the word of Google (or the equivalent) is law, and you can’t use it or profit by it if they make sure it doesn’t exist!
Additionally, referrer spam allows the spammer to steal website content from the rightful author / owner, which devalues it and makes it harder to gain any appreciable money from advertising or a sale. This is all separate, of course, from the simple fact that referrer spam allows those behind it to spread malware, viruses, and other nasty things all over the Internet, making it less safe for everyone, not just the targets of a spam attack, but to bring things back around to revenue, referrer spam can serve as quite the fertile seed bed for other types of spam, including email spam, click fraud, identity theft, the creation of fake website user profiles, and so on – ever try to make money from a jail cell?
To reiterate, the above is just a general overview of the deleterious effects of referrer spam, but it should do a fair job of illustrating the point that this new type of spam is bad news. Left unchecked, it can cost you money and time while you attempt to act upon bad information, and eventually, it can mean the obliteration of your entire online presence and reputation. This is entirely separate from the physical, real world ramifications of referrer spam; identity theft, for instance, can quite literally ruin lives both in and out of cyberspace.
Referrer Spam: The How
Knowing that a threat exists and what it can do if it catches you unprepared is always a good foundation, but it’s useless without follow-up action. While it may be impossible at this point to ever truly eradicate spam from the face of the Internet, it’s quite possible to deter and defend against it. After all, cockroaches may be nearly impossible to destroy, but it still isn’t any fun getting squished! A short list of anti-spam defense strategies follows.
Check Your Analytics
There’s no point in installing a website analytics suite if you never bother to look at what it’s trying to tell you! To that end, make sure to check your analytics software on a semi-monthly basis at a minimum; weekly checks are optimal. Taking a peek under the hood every couple of weeks will give you great insight not only into potential threats, but also to the nuts and bolts of how your website works and what you can do to improve it. As for the spam itself, there’s no substitute for combing through your traffic logs and reporting fake URLs, so do it!
Muster the Troops
Your website has a bevy of technological defenses at its disposal as a function of its very existence. First, all of your website(s) should have a robots.txt file in them someplace, which is basically a permission slip for how visitors to your website must behave, including the search engines. Obviously, the bad guys of the Internet will scoff and ignore it, but it needs to be there anyway, and with such a fast and easy creation process, there’s no excuse not to include it. Additional lines of defense include referral blockers, IP blockers, user agent blockers, and the like, while utilities from the search engines themselves – Google Tag Manager – for instance – comprise another. The best defense is a good offense, so find, create, and/or install these digital fortifications ASAP!
Bring Out The Big Guns
In addition to using firewalls and secure web browsers like Firefox or Google Chrome in lieu of less secure alternatives, there are a few really heavy-hitting options a website owner can employ to protect himself or herself, including paying for the services of an outside penetration testing firm, getting your network’s system administrator on board with your protection efforts if it’s a corporate network you’re trying to guard, and creating custom analytics alerts to give you real time alerts to possible attacks / spam.
The Web’s Most Wanted
While it might be nice to have a “worst offenders” list for this sort of thing, the simple fact is that spam and the community of computer users that authors and disseminates it is simply too amorphous, rapid in its evolution, and mobile to pin down like that. The best anyone connected to the Internet can do is to avoid suspicious sites, follow the instructions / advice given in this article, and cross their fingers – lists are for Pinterest.
Conclusion / Closing Thoughts
Referrer spam is a pernicious, potentially damning phenomenon that seems to be almost as widespread and hard to kill as its elder brother, the email chain letter from Nigeria. Happily, with this little primer in hand, it should be much easier for Internet users everywhere to recognize and defend against it.
Next post will show ways to stop referrer spam