ECS Scaling and Event-Driven Tasks

By the end of this lesson you will be able to

• Explain the two layers of scaling in ECS: the number of tasks in a service, and the capacity they run on (EC2 instances or Fargate).
• Configure ECS Service Auto Scaling with Application Auto Scaling: target tracking on CPU, memory or ALB requests per target, step scaling, scheduled scaling, and predictive scaling as a side note; calculate the desired count a target tracking policy will set.
• Explain why an Auto Scaling group that scales on CPU is the wrong tool for container instances, and how a capacity provider with managed scaling, target capacity and managed termination protection fixes it; calculate CapacityProviderReservation.
• Write a capacity provider strategy with base and weight, mix Fargate and Fargate Spot, and handle a Fargate Spot interruption (two-minute warning, SIGTERM, stopTimeout).
• Run tasks from events: an EventBridge rule on an S3 upload, an EventBridge Scheduler schedule, and a service of SQS workers that scales on backlog per task.
• Catch stopped tasks with an EventBridge rule on task state change events and send an alert through SNS.

#Two layers of scaling

In lesson 14.2 you ran the checkout API of a Dhaka online shop as an ECS service: a task definition, a desired count of tasks, an ALB in front. On an ordinary day two tasks are plenty. On the first night of the Eid sale they are not.

Scaling a container application happens on two layers, and the exam checks that you keep them apart:

On Fargate the second layer disappears: every task gets its own capacity from AWS, so you only scale tasks. That is why Fargate auto scaling is much easier to set up. On EC2, adding tasks is not enough: when the instances are full, new tasks have nowhere to go.

#ECS Service Auto Scaling

ECS Service Auto Scaling automatically increases and decreases the desired number of tasks in a service. ECS does not do this itself: it uses Application Auto Scaling, the same service that scales DynamoDB tables, Aurora replicas and Lambda provisioned concurrency.

You register the service as a scalable target with three numbers:

• the scalable dimension ecs:service:DesiredCount,
• a minimum capacity (for example 2 tasks, so the service survives one task failing; 0 is allowed for workers that may sleep),
• a maximum capacity (a cost and safety ceiling, for example 20 tasks).

Then you attach one or more scaling policies. ECS sends the service’s average CPU and memory to CloudWatch every minute, and the policies act on those or any other CloudWatch metric.

#Target tracking scaling

The notes call this the most used policy, and it is the one AWS recommends: you say “keep the average CPU of all the tasks at 70%”, and Application Auto Scaling does the rest, like a thermostat. It creates and manages two CloudWatch alarms for you (do not edit them): a scale-out alarm that reacts after 3 one-minute data points above the target, and a scale-in alarm that waits for 15 data points below it.

Three predefined metrics exist for ECS services:

You can also track a custom metric or a metric math expression (the SQS example later in this lesson), as long as it goes up and down in proportion to the number of tasks.

How many tasks does target tracking ask for? Roughly in proportion:

new desired count = current running tasks × current metric ÷ target, rounded up.

With the notes’ 70% CPU target: 2 tasks at 90% → 2 × 90 ÷ 70 = 2.57 → 3 tasks. If the load keeps climbing, 3 tasks at 85% → 3.6 → 4 tasks. It always rounds up when scaling out: a little too much capacity costs a little money, too little capacity costs customers.

A few rules about the behaviour:

• It starts from the running count, not the desired count. If tasks could not be placed, it does not keep stacking up the desired count (no runaway scaling).
• Scale-out is fast, scale-in is slow on purpose. Scale-in waits for the 15 low data points and removes a task only when the rest would stay safely below the target (usually more than 10% lower).
• Cooldowns default to 300 seconds for ECS services. During the scale-out cooldown only a bigger scale-out is allowed; during the scale-in cooldown no scale-in happens, but a scale-out still can.
• Several policies on one service (CPU and requests, say): it scales out if any policy wants more, and scales in only if all of them agree.
• You can disable scale-in on a policy and remove tasks some other way.
• On EC2, CPU can read above 100%. The metric compares used CPU with the CPU the tasks reserve; on EC2 a container without a hard limit can borrow idle CPU from the instance. On Fargate a task cannot use more than it asked for.

Try your own numbers. The last row, tasks per instance, connects this layer to the next one:

#Step scaling

With step scaling you create the CloudWatch alarm yourself (“average CPU above 70% for 2 minutes”) and define steps by how far the metric has gone past it: 70–85% → add 1 task, above 85% → add 2 tasks, or add a percentage of the running count. You need a second policy and a second, low alarm for scale-in.

Choose it when you want full control: very fast reactions to a burst (a high-resolution metric and a one-data-point alarm), large jumps for large breaches, or different rules for scaling out and in. The price is that you design and tune the alarms and steps yourself; that is why target tracking is the default answer. (Simple scaling, the legacy one-adjustment-then-wait policy from lesson 6.5, exists for EC2 Auto Scaling groups; for ECS use step scaling.)

#Scheduled scaling

A scheduled action changes the minimum and maximum capacity at a date and time, once or on a cron schedule with a time zone. It is for predictable changes: the Eid sale begins at 8 pm, a cricket final starts at 2 pm, office hours begin at 9 am. Set min to 10 at 7:45 pm, and the tasks are running and warm before the first customer arrives; set min back to 2 at 2 am. Target tracking keeps working inside those bounds, so if the sale is bigger than expected it still adds tasks up to max. Scheduled actions run at their time without waiting for a cooldown.

#Scaling and deployments

While an ECS deployment (a rolling update to a new task definition, lesson 14.2) is in progress, Application Auto Scaling turns off scale-in for the service, so it does not remove tasks the deployment is counting on. Scale-out continues, so a traffic spike during a deployment is still handled. If you want no scaling at all during a deployment, suspend dynamic scaling on the scalable target (DynamicScalingInSuspended and DynamicScalingOutSuspended) and resume it afterwards.

#Scaling the EC2 capacity underneath

On the EC2 launch type, the tasks need free CPU and memory reservations on registered container instances. When Service Auto Scaling asks for more tasks than the instances can hold, you need more instances.

#The old way: an Auto Scaling group on CPU

The slides show the first approach: put the container instances in an Auto Scaling group and give it a scaling policy on the instances’ CPU utilization, so that instances are added over time. It is better than nothing, but it measures the wrong thing:

• ECS places tasks by reservation: a task that reserves 1 vCPU blocks that vCPU even if it uses only 10% of it. An instance can be full (no room for another task) while its CPU is 40%. The ASG sees 40%, stays put, and the new tasks cannot be placed.
• The ASG does not know about tasks waiting to be placed, which is exactly the signal you need.
• On scale-in, the ASG picks instances by its own termination policy and can terminate an instance that is running tasks.

#Cluster auto scaling with capacity providers

A capacity provider links an ECS cluster to a source of capacity. For EC2 you create an Auto Scaling group capacity provider (one ASG per capacity provider), associate it with the cluster, and turn on two features.

Managed scaling (ECS cluster auto scaling). ECS creates a target tracking policy on the Auto Scaling group and drives it with a metric it computes itself:

CapacityProviderReservation = instances needed (M) ÷ instances running (N) × 100

M counts what all tasks need, including the ones that have not been placed yet: with a capacity provider, tasks that do not fit wait in the PROVISIONING state instead of failing. ECS publishes the metric to CloudWatch (namespace AWS/ECS/ManagedScaling) and keeps it at your target capacity:

• Target capacity 100% (the default): no spare instances. Above 100% the group scales out, below it scales in. Each scale-out waits for an instance to boot.
• Target capacity below 100%: spare capacity. At 80%, ECS keeps N = M ÷ 0.8, so with 8 instances needed it runs 10, and new tasks land on a ready instance at once. You pay for the idle ones.

Other managed scaling settings: the minimum and maximum scaling step size (instances added in one go; defaults 1 and 10,000), and the instance warm-up period (default 300 seconds before a new instance counts in the metric). When the group is at zero and tasks need capacity, ECS launches 2 instances to start. ECS never changes the group’s own min and max, so max must be high enough.

Managed termination protection. ECS turns on instance scale-in protection for every instance that runs tasks and removes it only from instances with no tasks (daemon tasks do not count). The ASG can then terminate only empty instances. It requires managed scaling and scale-in protection enabled on the Auto Scaling group. On top of that, managed instance draining (on by default) moves service tasks off an instance before the ASG terminates it for any reason.

Worked example: 10 tasks, 2 fit on each instance, so M = 5. Four instances are running: reservation = 5 ÷ 4 × 100 = 125%, above the 100% target, so the group scales out to 5. The service later shrinks to 6 tasks: M = 3, N = 5, reservation 60%, so the group scales in to 3, and with managed termination protection only empty instances go.

#Capacity provider strategies: base and weight

A capacity provider strategy tells ECS where to place the tasks of a service or a RunTask call. It replaces the older “launch type” choice and is the modern way to pick capacity. Fargate has two built-in capacity providers, FARGATE and FARGATE_SPOT, which you only need to associate with the cluster. Each item in a strategy has:

• base: how many tasks, at minimum, run on this provider. Only one provider in a strategy may have a base. Base is satisfied first.
• weight: after the base, the remaining tasks are shared in the ratio of the weights. A provider with weight 0 gets no share of the tasks after the base. AWS documents that a weight-0 provider cannot place tasks, so give the provider that holds the base a weight of at least 1, as in the example below.

Example: FARGATE base 2, weight 1; FARGATE_SPOT weight 3. With a desired count of 6: the first 2 tasks go to Fargate (base); the other 4 are split 1 : 3, so 1 more on Fargate and 3 on Fargate Spot. Result: 3 on Fargate, 3 on Fargate Spot. With 10 tasks: 2 + (8 split 1 : 3 = 2 and 6), so 4 on Fargate and 6 on Spot.

Rules worth remembering:

• A strategy can name up to 20 capacity providers, but it may contain either Fargate providers or Auto Scaling group providers, never both. A cluster, however, can have both kinds, used by different services.
• A cluster can have a default capacity provider strategy, used when a service or task names neither a strategy nor a launch type.
• Changing a service from a launch type to a capacity provider strategy needs a new deployment (force new deployment).
• On EC2, the same idea mixes two Auto Scaling groups: one of On-Demand instances with a base, one of Spot Instances with a larger weight (Spot Instances, lesson 4.4).

#Fargate Spot

Fargate Spot runs tasks on spare Fargate capacity at up to 70% off the Fargate price. The price follows long-term supply and demand; there is no bidding. In return, AWS can take the capacity back:

• Two-minute warning. ECS sends a task state change event to EventBridge (stop code SpotInterruption, reason “Your Spot Task was interrupted.”) and a SIGTERM to the containers.
• Graceful shutdown. The application must catch SIGTERM, stop taking new work, finish or checkpoint what it is doing and exit. After the container’s stopTimeout (default 30 seconds, maximum 120 seconds on Fargate), ECS sends SIGKILL; anything unfinished, including the task’s ephemeral storage, is lost.
• Replacement. A service notices the missing task and launches a new one on Fargate Spot, if Spot capacity is available. ECS does not fall back to regular Fargate by itself, so during a capacity crunch Spot tasks may stay missing; a service whose only task is on Spot is down until capacity returns. That is what the base on regular Fargate is for.
• Where it works. Linux tasks on x86 (platform version 1.3.0 or later) and 64-bit Arm, ARM64 (1.4.0 or later); not Windows containers. Fargate Spot is an ECS feature; EKS pods on Fargate cannot use it (lesson 14.4).

Good fits: stateless web tasks behind an ALB with an on-demand base, queue workers that are idempotent (a stopped worker’s message just comes back to the queue), CI build agents, batch jobs that checkpoint. Bad fits: a single task holding state in memory, long jobs that cannot resume, anything that must never lose capacity suddenly.

#Event-driven ECS tasks

Not every container is a long-running service behind a load balancer. Many jobs should start because something happened, run once, and stop: you pay only while the task runs. Here there is no service and no desired count; something calls the ECS RunTask API for you.

#An EventBridge rule runs a task

The slides’ example: a client uploads an object to an S3 bucket; S3 sends an event to Amazon EventBridge; a rule matches it and its target is “run an ECS task” on a Fargate cluster; the new task gets the object from S3, processes it, and saves the result in DynamoDB.

The pieces you configure:

1. Send S3 events to EventBridge

   Turn on EventBridge notifications for the bucket (lesson 10.7). Classic S3 event notifications go only to SQS, SNS and Lambda; an ECS task is not one of their destinations, which is why EventBridge sits in the middle.

2. Write the rule

   An event pattern on source: aws.s3, detail-type: Object Created, the bucket name and, for example, the prefix raw/. EventBridge itself is covered in depth in lesson 19.5.

3. Point the target at the cluster

   Target type ECS task: the cluster, the task definition, the launch type or capacity provider strategy, and for Fargate the network configuration (subnets, security groups), which is required because Fargate tasks use awsvpc. An input transformer can pass the bucket and key into the container as environment variables through container overrides. TaskCount can start several tasks per event.

4. Give each actor its own role

   EventBridge needs a role it can assume with ecs:RunTask and iam:PassRole for the task’s execution role and task role. The task itself uses its task role to read S3 and write DynamoDB (the three ECS roles, lesson 14.2). EventBridge’s role is not the task’s role.

#Scheduled tasks with EventBridge Scheduler

The slides’ second picture: a schedule (“every 1 hour”) runs an ECS task that does batch processing against S3. Two tools can do this:

AWS now recommends EventBridge Scheduler for new schedules (the ECS console can still create scheduled tasks, but the Scheduler console offers more options). For the exam, “run an ECS task on a schedule (cron)” → EventBridge (either form) with an ECS task target. You do not need a server running cron, or a service with a desired count of 1 that sleeps all day.

Example: a garment exporter computes payroll every night at 1 am Dhaka time. A schedule cron(0 1 * * ? *) in the Asia/Dhaka time zone runs the payroll task on Fargate; it reads the day’s attendance from S3, writes payslips back to S3, and exits after 20 minutes. Twenty minutes of Fargate a day instead of a server running for 24 hours.

#Queue workers: an ECS service that scales on SQS

The slides’ third pattern: messages arrive in an SQS queue, and an ECS service of worker tasks polls the queue; ECS Service Auto Scaling adds tasks when the queue grows. Unlike the two patterns above, this is a long-running service: workers stay up and pull messages (SQS does not start tasks; it is pull-based, lesson 13.1).

Which metric? Not CPU: workers waiting on the network show low CPU while the queue explodes. And not the raw queue length either: a target tracking policy on ApproximateNumberOfMessagesVisible does not know how many workers already exist, so it keeps adding tasks while the queue drains, far beyond what the work needs. Use backlog per task, exactly like backlog per instance in lesson 13.1:

• Backlog per task = ApproximateNumberOfMessagesVisible (from AWS/SQS) ÷ RunningTaskCount (from ECS/ContainerInsights).
• Target = longest acceptable wait ÷ average time to process one message.

The target tracking policy computes the division itself with metric math; no custom metric or code. One requirement: RunningTaskCount is published only when Container Insights is turned on for the cluster; without it the expression has no divisor and the service never scales.

AWS’s own example: acceptable latency 10 s, 0.1 s per message → target 100 messages per task. 1,500 messages and 10 tasks → 150 per task → the service scales out by 5 tasks, to 15.

For workers that run long jobs, use task scale-in protection: when a worker starts a job it sets ProtectionEnabled through the ECS agent endpoint ($ECS_AGENT_URI/task-protection/v1/state), and scale-in or a deployment will not stop it; it clears the flag when done. Protection lasts 2 hours by default and up to 48 hours. Even without protection no message is lost: a stopped worker never deleted its message, so it reappears after the visibility timeout.

#Intercepting stopped tasks with EventBridge

ECS reports every task’s life to EventBridge as ECS Task State Change events (and ECS Container Instance State Change and ECS Service Action events for the other resources). A task that stops produces an event with lastStatus: STOPPED, a stoppedReason, and a stop code:

The slides’ pattern: an EventBridge rule with an event pattern that matches stopped tasks (source aws.ecs, detail-type ECS Task State Change, lastStatus STOPPED, your cluster ARN) triggers an SNS topic, which emails the administrator. Instead of SNS, a Lambda function can post to a chat channel, open a ticket or restart a failed batch job.

{
  "source": ["aws.ecs"],
  "detail-type": ["ECS Task State Change"],
  "detail": {
    "clusterArn": ["arn:aws:ecs:ap-southeast-1:111122223333:cluster/payroll-example"],
    "lastStatus": ["STOPPED"],
    "stopCode": ["EssentialContainerExited", "TaskFailedToStart"]
  }
}

Filtering on the stop code matters: without it every scale-in and every deployment would page someone at 3 am.

#What goes wrong

#ECS scaling in system design

• Web and API services: target tracking on CPU or ALBRequestCountPerTarget, a minimum of 2 or more across AZs, a scheduled action before known events, and a max sized to what the database can take.
• Fargate first when you do not want to manage the capacity layer; EC2 with capacity providers when you need specific instance types, GPUs, very high density or Reserved Instances (lesson 14.2), and ECS Managed Instances when you want EC2 choice without patching.
• Cost: a base on regular capacity plus a weighted Spot share (Fargate Spot or Spot Instances); Compute Savings Plans for the steady base (lesson 4.4).
• Asynchronous work: producers enqueue in SQS, an ECS worker service scales on backlog per task, and Spot workers are fine because messages survive a stopped task.
• Event-driven jobs: S3 or business events start one task per event through EventBridge; schedules replace cron servers. Compare with Lambda (chapter 15): Lambda starts faster and needs no networking setup, ECS tasks run longer than 15 minutes, use more memory and any container image.
• Operations: alert on unexpected stops through EventBridge, watch Container Insights, and keep the ECS-managed scaling resources (policies, alarms, tags) untouched.

#Try it yourself

Register a service for auto scaling, add a target tracking policy, set up a capacity provider strategy with Fargate Spot, and create the event-driven pieces. The output is shortened.

# 1. Register the service as a scalable target: 2 to 20 tasks
aws application-autoscaling register-scalable-target \
  --service-namespace ecs --scalable-dimension ecs:service:DesiredCount \
  --resource-id service/shop-example/checkout --min-capacity 2 --max-capacity 20

# 2. Target tracking: keep average CPU at 70%
aws application-autoscaling put-scaling-policy --policy-name cpu70 \
  --service-namespace ecs --scalable-dimension ecs:service:DesiredCount \
  --resource-id service/shop-example/checkout --policy-type TargetTrackingScaling \
  --target-tracking-scaling-policy-configuration '{
    "TargetValue": 70.0,
    "PredefinedMetricSpecification": {"PredefinedMetricType": "ECSServiceAverageCPUUtilization"},
    "ScaleOutCooldown": 60, "ScaleInCooldown": 300 }'
# { "PolicyARN": "arn:aws:autoscaling:ap-southeast-1:111122223333:scalingPolicy:...",
#   "Alarms": [ { "AlarmName": "TargetTracking-service/shop-example/checkout-AlarmHigh-..." },
#               { "AlarmName": "TargetTracking-service/shop-example/checkout-AlarmLow-..." } ] }

# 3. Scheduled action: min 10 tasks before the Eid sale, Dhaka time
aws application-autoscaling put-scheduled-action --service-namespace ecs \
  --scalable-dimension ecs:service:DesiredCount --resource-id service/shop-example/checkout \
  --scheduled-action-name eid-sale --schedule "at(2026-03-19T19:45:00)" \
  --timezone "Asia/Dhaka" --scalable-target-action MinCapacity=10,MaxCapacity=40

# 4. Watch what it did
aws application-autoscaling describe-scaling-activities --service-namespace ecs \
  --resource-id service/shop-example/checkout --max-items 2
# "Description": "Setting desired count to 3.",
# "Cause": "monitor alarm TargetTracking-...-AlarmHigh-... in state ALARM triggered policy cpu70"

# 5. Capacity provider for EC2: an ASG with managed scaling and termination protection
aws ecs create-capacity-provider --name shop-asg-cp \
  --auto-scaling-group-provider 'autoScalingGroupArn=arn:aws:autoscaling:ap-southeast-1:111122223333:autoScalingGroup:...:autoScalingGroupName/shop-asg-example,managedScaling={status=ENABLED,targetCapacity=90},managedTerminationProtection=ENABLED'

# 6. A Fargate cluster's providers and default strategy: base 2 on Fargate, then 1 : 3 with Spot
aws ecs put-cluster-capacity-providers --cluster trips-example \
  --capacity-providers FARGATE FARGATE_SPOT \
  --default-capacity-provider-strategy capacityProvider=FARGATE,base=2,weight=1 \
      capacityProvider=FARGATE_SPOT,weight=3

aws ecs describe-tasks --cluster trips-example --tasks 0f1e2d3c4b5a69788796a5b4c3d2e1f0 \
  --query 'tasks[].{cp:capacityProviderName,status:lastStatus}'
# [ { "cp": "FARGATE_SPOT", "status": "RUNNING" } ]

# 7. Run the payroll task every night at 1 am Dhaka time
aws scheduler create-schedule --name payroll-nightly \
  --schedule-expression "cron(0 1 * * ? *)" --schedule-expression-timezone "Asia/Dhaka" \
  --flexible-time-window Mode=OFF --target file://payroll-target.json

# 8. Email the admin when a task stops unexpectedly
aws events put-rule --name ecs-task-stopped --event-pattern file://stopped-pattern.json
aws events put-targets --rule ecs-task-stopped \
  --targets Id=1,Arn=arn:aws:sns:ap-southeast-1:111122223333:ops-alerts-example

payroll-target.json holds the cluster ARN as Arn, a RoleArn that EventBridge Scheduler can assume (with ecs:RunTask and iam:PassRole), and EcsParameters with the task definition, LaunchType or a capacity provider strategy, and the subnets and security groups. stopped-pattern.json is the event pattern shown above. For the S3 trigger, aws s3api put-bucket-notification-configuration --bucket photos-example --notification-configuration '{"EventBridgeConfiguration": {}}' turns on EventBridge delivery, and a rule like step 8 with an ECS target runs the resize task.

#Summary

Key takeaways

• ECS scales on two layers: tasks (ECS Service Auto Scaling) and, on the EC2 launch type, instances (an Auto Scaling group, ideally driven by a capacity provider). On Fargate only the task layer exists, which is why Fargate scaling is easier.
• ECS Service Auto Scaling uses Application Auto Scaling on ecs:service:DesiredCount with a min and max. Policy types: target tracking (ECSServiceAverageCPUUtilization, ECSServiceAverageMemoryUtilization, ALBRequestCountPerTarget, or custom/metric math), step scaling (your alarms, your steps), scheduled scaling (known times); predictive scaling since November 2024.
• Target tracking asks for about running × metric ÷ target, rounded up; scales out fast, scales in slowly (default cooldown 300 s for ECS), never past max, and does not scale in during a deployment.
• An ASG on CPU misses tasks that cannot be placed. A capacity provider with managed scaling tracks CapacityProviderReservation = M ÷ N × 100 against a target capacity (100% = no spare; 80% = spare instances). Waiting tasks sit in PROVISIONING. Managed termination protection terminates only empty instances.
• A capacity provider strategy places tasks by base (first, one provider only) then weight (ratio). Fargate and ASG providers cannot share one strategy.
• Fargate Spot: up to 70% cheaper, two-minute warning by EventBridge event and SIGTERM, SIGKILL after stopTimeout (default 30 s, max 120 s), no automatic fallback to regular Fargate; Linux only.
• Event-driven tasks: an EventBridge rule runs a task on an event (S3 upload with EventBridge notifications on); EventBridge Scheduler runs tasks on a cron or rate schedule in any time zone. EventBridge’s role needs ecs:RunTask and iam:PassRole; the work uses the task role.
• SQS workers: an ECS service that scales on backlog per task = queue depth ÷ RunningTaskCount (needs Container Insights), with task scale-in protection for long jobs.
• Stopped tasks: an EventBridge rule on ECS Task State Change with lastStatus STOPPED (filter by stop code) → SNS email or Lambda.

Self-check: ECS scaling and event-driven tasks31 questions