The Downside of the Decline of Search Engine Impersonator Bots, and What it means for DDoS Attacks.
On the surface, the news that the use of search engine impersonator bots is down from 57% of all DDoS bot traffic in 2014 to a miniscule 0.9% in 2015 seems like good news. However, if life hasn’t yet taught you to always look for the downside, then welcome to lesson number one.
In order to understand what botnet operators’ abandonment of search engine impersonator bots actually means for your website security, imagine being the owner of a cheese store. Every morning for a week you arrive at work to find several wheels of cheese have been nibbled on. Your search of the store finally leads you to a hole in the frame of your back door, which you immediately patch. Take that, mice! No more cheese buffet for you!
But mice love cheese and your store has a lot of it. Those mice want in. Patching that hole was a good step, and it had to be done, but all they’re going to do is find new ways in. It’s the nature of these mini beasts.
Essentially, botnet operators have largely given up on search engine impersonator bots because they were being blocked, so now they’ve found other ways to cause havoc.
Trust the Googlebots, fear the other user-agent variants
It used to be that 1 in 25 bots that appeared to be Googlebots was actually an impersonator. The logic behind this, from an attacker’s viewpoint, was simple but sensible. Nearly every website in existence welcomes Google’s search engine crawlers. Therefore, Googlebots (and convincing impersonators) were being given a backstage pass to any and every site, almost without exception.
Once they had that access, search engine impersonator bots were using it for DDoS attacks, spam, content theft, hacking and other malicious activities. Then a great thing happened: people caught on, internet security caught up, and IP-based mitigation techniques that can identify and bounce these bots was widely adopted.
Alas, since you now know to look for the downside in every situation, botnet operators didn’t exactly pick up their balls and go home when fake Googlebots stopped being effective. They just started using a wider variety of so-called user-agent variants, which are bots that pretend to be something they aren’t, like a browser, in order to gain access and instigate malicious activities such as DDoS attacks, which render a website or online service unusable for legitimate users.
So instead of 57% of DDoS bot traffic coming from one identifiable type of bot, search engine impersonator bots, it’s now being spread out over a diverse number of user-agent variants. In fact, while the top ten fabricated user-agent variants accounted for 90% of application layer DDoS attacks from November 2013 to February 2014, the top ten fabricated user-agent variants accounted for just 43% of application layer DDoS attacks from March to May of 2015.
Surveying the rest of the DDoS threat landscape
All of the above information on search engine impersonator bots and user-agent variants comes from the Incapsula Q2 2015 DDoS Global Threat Landscape Report. The same report has also identified the two of the biggest DDoS attack trends, which happen to be oddly opposite.
The first trend is the quick hits: rudimentary attacks that use a single attack vector and typically last no more than 30 minutes. These are coming from what’s called booters or stressors – DDoS attack for hire services that allow anyone to pay a set amount of money and unleash an attack on the website of their choosing.
The second trend is much more sophisticated: complex and long multiphase DDoS attacks that use multiple attack vectors that can last for anywhere from days to months. These are the handiwork of professional cyber-criminals.
Additionally, the report found that 47% of all DDoS attack targets are hit with another DDoS attack within 60 days, and 17% of targets are hit with more than five DDoS attacks. Incapsula has previously reported that unmitigated DDoS attacks have been shown to cost businesses a shocking $40,000 per hour, and that’s not taking into consideration the hardware and software damage that can be done, the loss of consumer trust that can result, or the theft of financial data or consumer information that can occur.
What needs to be learned from the DDoS threat landscape
If you want to be a pessimist about it, you could say that it all boils down to the fact that DDoS attacks are coming from absolutely everywhere. If you want to be a realist about it…it still pretty much boils down to the fact that DDoS attacks are coming from absolutely everywhere.
To summarize, there is now a diverse array of user-agent variants taking the place of predictable search engine impersonator bots, professional cybercriminals launching multi-vector DDoS attacks that can last for months and cost targeted companies millions of dollars, and quick-hit DDoS attacks available to anyone with a few dozen dollars to spare. It should no longer be a question of whether or not companies and websites should invest in professional DDoS protection, rather a question of what level of professional DDoS protection to acquire.
After all, DDoS attacks have proven to be both incredibly effective and, in some cases, incredibly lucrative. Attackers and cybercriminals aren’t going to be dissuaded or distracted. They need to be stopped. Much like a cheese store needs an exterminator on call, a website needs professional DDoS mitigation.
Featured Image Source: BigStock