Hello everyone. I’m pretty sure while reading the title you had only one question in mind “What’s the heck EDoS attack?” Well, congratulations, there is one more reason software startup founders and technology company owners don’t sleep well at night. If you are still sleeping like a baby - continue with the reading to get rid of that boring peace of mind state.
EDoS or Economic Denial of Sustainiability, is the kind of attack which targets usage-based resources of your cloud to inflate usage of such resource and make your cloud bill sky-high. In other words, the goal is not to make your product unavailable for end-users, and instead make it financially unbearable for you.
Because modern clouds are highly-available, and for average cloud provider customer are almost infinitely scalable, chances are connected bank account or a credit card will decline faster than your application will stop working.
It actually happens pretty often, you can tell it from various topics on forums like Reddit where people share their stories about waking up and seeing their cloud bill raise tenfold overnight.
Threat vector
Remember that feeling when thinking about the serverless computing that it can’t be so ideal and there is a catch? Well, hello there, here is the catch.
One of the biggest benefits of the serverless model, it’s high scalability. It’s actually relatively easy for the cloud provider to provide it, since serverless relies on the fact only a small fraction of cloud services consumers will reach peak usage. What’s important to understand is that serverless computing is actually very expensive, and it’s low utilization what makes it look so affordable.
This is where EDoS comes into play. By identifying components of your system which involve usage-based cloud computing services, the attacker can spin up an attack with a traditional DoS/DDoS tool and instead of bringing your app down, it will make your cloud bill go up.
It’s applicable for any cloud computing service, so I won’t be sticking to particular names here. Let’s look at average pricing of services instead:
-
Function as a service - $5 per 30 million requests a month. 30M invocation per month sounds like a huge amount of traffic. But is it? It’s 1M requests a day on average, meaning roughly 11 requests per second.
-
API Gateways - $100 per 30 million REST API requests a month.
-
NoSQL DB - $380 per 30 million write API requests a month
-
So for these three basic services, we have $485 per month with our 10 RPS as a baseline. Let’s imagine we have a eCommerce website with the following API calls under the hood:
GET /products/101 // REST API based on a serverless function behind the API gateway which returns a particular product details from the NoSQL DB to be displayed on the product details page for guest usersPOST /comments // REST API based on a serverless function behind the API gateway which allows guest user to post a comment under the blog post articleLet’s assume you became popular and the traffic increases by 1M requests a day. Now your cloud bill is $970/month. Now imagine your unfair competitor is upset with your popularity and decides to involve non-ethical methods of competitions and orders a DDoS attack with 100K RPS which is active for one hour. You are getting 10x monthly usage in one hour. And it is going to cost you exactly 10x your monthly bill which is $4850!
I bet at this moment you hope your cloud services provider will fail under such heavy load and this will be considered as a DDoS attack and you will get your money back. But it won’t, actually. It’s peanuts for a highly available cloud. And a fortune for you. Spooky!
Of course, this is oversimplified, but you got the general idea. Things become even worse when we are dealing with complex chains of internal API calls to expensive, highly-available, per-invocation services. For example, big data query services which can have an API call cost around $1/invocation! For sure, something needs to be done here.
How to deal with EDoS attacks?
Let’s see what we can do about EDoS attacks (and DDoS) in general. The best approach is to prevent the attack from happening. If it’s not possible, then we can at least contain consequences.
As we already understood, poorly designed software product is not simply a source of irritation for end users, it can also become a cash cow due to per-invocation billing.
The first seemingly obvious way of preventing EDoS on serverless stack is not using such stack at all. Reverting to good old virtual machines decreases a chance of such attacks, especially if no usage-based services are involved on the backend. Unfortunately, it doesn’t solve the problem with third-party APIs with per-invocation pricing used by your backend. For example, identity verification services, various quotation, pricing and tax calculation tools. So while this sentiment is strong, simply ditching all functions won’t help.
Let’s revert to a more pragmatic and realistic path of decreasing the probability of an EDoS attack. Recommendations will be similar to what should be done to avoid DoS/DDoS attacks as well.
Rate limiting is your friend
The main feature of any DDoS/DoS/EDoS attack is the scale. It will either be reflected in RPS (requests per second) or your cloud provider forecasted monthly bill/current usage bill. To avoid surprises, carefully evaluate what’s considered a “natural” activity of your application and add rate limiting on the application level. This will ensure third-party services and APIs used by your backend won’t be affected by the attack. All popular frameworks either have this functionality out of the box, or there is a popular plugin for it.
As the next step, add rate limiting on the level of your cloud infrastructure. Ensure your application is scalable, but not infinitely scalable. Pay-per-use services like FaaS, NoSQL databases and API gateways can be configured to limit the number of requests for a given period of time, concurrency limits, etc.
Finally, put a web application firewall like or DDoS protection tool to your stack, like Amazon WAF or CloudFront. This will offload the attack from your application infrastructure to the infrastructure of a tool, specifically crafted for the purposes of preventing DoS attacks.
Look for anomalies on multiple levels
Attackers can be sly. Your app can experience the same 10 RPS for a month and then you realize your cloud billing went nuts because one of those requests was a API call triggering a query to your huge data lake. Or the attack is happening on the user pool hosted on your third-party authentication provider infrastructure and you don’t see any load spikes. Instead, you get an invoice for reaching 100K users tier. Monitor at least the following metrics:
- RPS
- Number of registered users
- Number of requests to third-party services
- Month-to-date billing
Preferable, configure anomaly detection for all of the above if it’s present in your cloud services provider offerings.
Set up monitoring and alarms
The worst is when you figure out your app is under attack from angry users or because of the “Credit Card Declined” notifications from your banking app.
To have situation awareness, configure various alarms and notifications for detected load and billing anomalies and add all stakeholders as notification recipient to ensure the shortest incident response time possible.
Final words
In the good old days, only large enterprises could accidentally burn a bag of money because of pity mistake in their software architecture. (Un)luckily, times are changing, clouds become more accessible and everyone can go broke because of poor programming practices. Do not underestimate the importance of proper architecture design!
