WSS
On this page
Infrastructure & ProvisioningRecommendedUpdated

Autoscaling policies

Strategies for dynamically adjusting the number of active computing resources based on real-time demand, ensuring performance while minimizing costs.

Reference Card

  • Origin/Prior Art: Cloud computing elasticity.
  • Related Practices: Load balancing, Stateless architecture, Capacity planning.
  • Key Concepts: Scale out/in, Scale up/down, Thrashing, Cooldown periods.
  • Primary Failure Modes: Metric lag, Boot time delays, Resource exhaustion.

Introduction

Autoscaling is a cloud computing feature that automatically adjusts the amount of computational resources in a server pool based on the current load. When traffic spikes, the system provisions new instances (scaling out). When traffic subsides, it terminates idle instances (scaling in). This elasticity guarantees that an application has enough resources to maintain performance during peak hours without paying for idle capacity during off-peak hours.

Mental Model

Think of a busy supermarket checkout area. The store manager constantly monitors the length of the lines. If the lines stretch down the aisles (high CPU utilization), the manager calls for more cashiers to open registers (scaling out). If the lines disappear and cashiers are standing idle, the manager sends them on break to save labor costs (scaling in).

The manager needs a strict policy to avoid chaos: they shouldn’t call a cashier if a line gets long for exactly five seconds, and they need to give the cashier time to walk to the register and log in before assuming they are ready to help.

Operational Pattern

An autoscaling group manages a collection of identical instances. The group requires a defined Minimum, Maximum, and Desired capacity.

  1. A monitoring system (like CloudWatch or Prometheus) continuously gathers metrics from the instances, such as CPU utilization, memory usage, or active network connections.
  2. The autoscaling engine evaluates these metrics against predefined thresholds.
  3. If the average CPU utilization exceeds 80% for three consecutive minutes, the policy triggers a scale-out event.
  4. The engine requests a new instance from the cloud provider, waits for it to boot, and registers it with the load balancer.
  5. The engine enters a “cooldown period” to prevent further scaling actions while the new instance absorbs traffic.
  6. If the average CPU drops below 30% for ten minutes, a scale-in event triggers, safely draining and terminating an instance.

Failure Modes

Boot Time Delays

Autoscaling is reactive. By the time the system detects a CPU spike and requests a new instance, the application is already struggling. If the instance takes ten minutes to boot, install dependencies, and pass health checks, the traffic spike may have already crashed the existing servers. Autoscaling only works if instances can boot and serve traffic in seconds.

Thrashing (Scale Flapping)

If the scale-out and scale-in thresholds are too close together, the system will thrash. It will add a server, which drops the average CPU below the scale-in threshold. It will immediately terminate the server, which spikes the CPU back over the scale-out threshold. This loop destroys performance and racks up cloud billing charges.

Downstream Resource Exhaustion

Scaling web servers horizontally is easy. However, if you scale from 10 to 100 web servers, they will immediately open thousands of new connections to the backend database. If the database cannot handle that connection volume, the autoscaling event will cause a catastrophic database outage. Autoscaling policies must account for the weakest link in the architecture.

Security

Autoscaling instances must never rely on interactive human configuration or manual credential injection. Because they boot and terminate autonomously, they must retrieve all necessary secrets securely and automatically via machine identity roles (like AWS IAM Instance Profiles).

Attackers can weaponize autoscaling. A Denial of Service (DoS) attack might not crash your servers if autoscaling absorbs the traffic, but it will trigger a massive “Denial of Wallet” attack as the cloud provider provisions hundreds of expensive instances to handle the malicious load. Set strict Maximum instance limits to cap financial exposure.

Operational Guidance

Use Target Tracking policies instead of Step policies whenever possible. Instead of defining complex rules (“add 2 instances if CPU > 80%, remove 1 if CPU < 30%”), Target Tracking allows you to simply state: “Maintain average CPU utilization at 50%.” The engine calculates the exact number of instances required dynamically.

Define cooldown periods carefully. The cooldown prevents the engine from launching a third instance before the second instance has finished booting and begun taking load. Set the cooldown equal to your maximum expected instance boot time.

Consider predictive scaling for predictable workloads. If traffic always spikes at 9:00 AM on Monday, schedule the scaling event for 8:45 AM. This ensures capacity is warm and ready before the surge hits, bypassing the reactive boot time delay.

Diagnostics

When an application struggles under load, check the autoscaling group activity history. Verify if the scaling alarms triggered correctly. If they triggered but no instances were added, check cloud provider quotas (you may have hit a regional limit on EC2 instances).

If instances boot but immediately terminate, they are likely failing their load balancer health checks. The autoscaling engine assumes they are broken and replaces them in an infinite loop. Inspect the boot logs of a failed instance to determine why the application failed to start.

Related topics

Sources & further reading