Systems Manager

By the end of this lesson you will be able to

• Explain what AWS Systems Manager is for, and what makes a server a managed node: the SSM Agent, a permission (instance profile or Default Host Management Configuration, or a hybrid activation on-premises) and an outbound path on 443 (NAT or interface VPC endpoints).
• Troubleshoot a node that shows as Connection lost or does not appear at all.
• Use Session Manager beyond the basics: session logging, KMS, idle timeouts, port forwarding to a database, SSH through it, IAM by tag, and know that CloudTrail records StartSession but not the commands.
• Run a script on a whole fleet with Run Command, and choose max concurrency and max errors so a bad script cannot break everything.
• Patch Windows, Linux and macOS with Patch Manager: baselines, auto-approval delays, patch groups and patch policies, Scan vs Install, compliance.
• Schedule disruptive work with Maintenance Windows (schedule, time zone, duration, cutoff, tasks) and automate operations with Automation runbooks triggered by EventBridge, Config, a schedule or a person.
• Name the rest of the toolbox from an exam keyword: Parameter Store, State Manager, Inventory, Fleet Manager, Distributor, OpsCenter, Explorer, Change Calendar, Quick Setup, and the 2025–26 status of Change Manager and Incident Manager.

#One console for a whole fleet

A bank in Dhaka runs 40 servers in AWS and 300 more in its branches. Every month somebody has to patch them, install a new antivirus agent, check which software is on each, and occasionally log in to fix something. Doing that by hand over SSH and RDP means open ports, shared keys, a bastion host, a spreadsheet of who patched what, and a 3 a.m. phone call when a reboot goes wrong.

AWS Systems Manager (still abbreviated SSM, from its old name Simple Systems Manager) is one service that does this work for servers you do not want to touch by hand: EC2 instances, on-premises servers and virtual machines in other clouds. It is a toolbox, and the exam asks about the tools by name:

The notes call these the five Systems Manager topics of the exam (Session Manager, Run Command, Patch Manager, Maintenance Windows, Automation); Parameter Store was taught in lesson 21.3. This lesson starts with the foundation they all share, then takes each tool in turn, and ends with a tour of the smaller tools.

#The foundation: managed nodes

#What makes a node “managed”

Every tool above works only on a managed node: a machine Systems Manager can talk to. Three things make one:

1. The SSM Agent runs on it. It is preinstalled on Amazon Linux 2023 and 2, Ubuntu, macOS and the Windows Server AMIs AWS publishes. On other images you install it (in the AMI build or in user data).
2. A permission lets the agent call Systems Manager:
   • an instance profile (an IAM role for the instance) with the AWS managed policy AmazonSSMManagedInstanceCore, or
   • Default Host Management Configuration: a per-account, per-Region setting that gives every EC2 instance a default role (AWSSystemsManagerDefaultEC2InstanceManagementRole) without an instance profile. It requires IMDSv2 and SSM Agent 3.2.582.0 or later, and if an instance profile that already allows ssm:UpdateInstanceInformation is attached, the agent uses that first.
3. A path out on HTTPS 443 to the Systems Manager endpoints. The agent always opens the connection outward; nothing ever connects in. A public subnet uses the internet gateway, a private subnet a NAT gateway or interface VPC endpoints (lesson 22.2) for ssm and ssmmessages, plus ec2messages for older agents.

Once registered, the agent sends a heartbeat every 5 minutes. Fleet Manager shows the node as Online; if heartbeats stop it becomes Connection lost; a node that never registered is simply not listed.

For a private subnet with endpoints, three details decide whether it works:

• The endpoint's own security group must allow inbound TCP 443 from the instances (their security group or the VPC CIDR). That is the only inbound rule in the whole design, and it belongs on the endpoint, not on the instance.
• Private DNS must be on for the endpoints (with DNS hostnames and DNS resolution on the VPC), so the agent's normal service names resolve to the endpoints.
• Other features need their own endpoints: S3 (a gateway endpoint) for patch downloads, agent updates and S3 logs; logs for CloudWatch Logs; kms when sessions are encrypted with a KMS key.

#On-premises servers and other clouds: hybrid activations

A branch server has no instance profile and no metadata service. It joins through a hybrid activation:

1. An admin creates an activation: an activation code and activation ID, a registration limit, an expiry date, and an IAM service role the servers will use.
2. On each server the admin installs the SSM Agent and registers it with the code and ID.
3. The server appears with an ID starting mi- (managed instance) instead of i-. Everything else works as on EC2: sessions, Run Command, patching, inventory. It still needs outbound 443, over the internet, a VPN or Direct Connect.

Now try it yourself: set up a node the way you found it and see which status Systems Manager would show, and why.

#Documents: the recipes

Every action is described by an SSM document, a JSON or YAML file with parameters and steps. The ones that matter here:

AWS owns hundreds of documents (names start with AWS- or AWSSupport-). You can write your own, keep versions, and share them with other accounts. A document says what; the tool decides where and when.

#Targeting: tags, not lists

Tools pick their nodes by instance IDs (fine for a few, up to 50 in one Run Command call), by tags (Env=prod, Patch Group=branches), by a resource group, or all managed nodes. Tags are the normal answer: a new server tagged Env=prod is included the next time, with nobody editing a list.

#Session Manager in depth

Lesson 4.2 introduced Session Manager as the modern replacement for SSH and bastion hosts. In one line: the agent's outbound channel carries your shell, so the instance needs no inbound rule, no public IP, no SSH key and no bastion, and IAM decides who may connect. Here is what an architect needs on top of that.

How a session runs. You call start-session (CLI, with the Session Manager plugin) or click Connect in the console. IAM checks ssm:StartSession for that instance. Session Manager tells the agent to open a data channel, and keystrokes and output travel through it over TLS. Linux, Windows (PowerShell) and macOS are supported; Windows can also get a graphical RDP session in the browser through Fleet Manager.

Preferences (one set per account and Region):

• Session logging: stream commands and output to CloudWatch Logs, and/or upload the transcript to an S3 bucket when the session ends. This is how you “log all terminal commands”.
• KMS encryption of session data with your own key, on top of TLS.
• Idle timeout: 20 minutes by default, 1 to 60 allowed. A maximum session duration can cap even busy sessions.
• Run As on Linux: sessions start as a named OS user instead of the default ssm-user; shell profiles set the shell environment.

Port forwarding. AWS-StartPortForwardingSession forwards a local port to a port on the instance. AWS-StartPortForwardingSessionToRemoteHost forwards to a remote host the instance can reach, such as a private RDS database: your laptop's localhost:3306 reaches the database through the instance, with nothing open inbound. SSH and scp through Session Manager (AWS-StartSSHSession as an SSH ProxyCommand) let tools that insist on SSH work without port 22.

Access by tag. IAM conditions on resource tags (ssm:resourceTag/Env) give developers sessions on Env=dev only. Since April 2025, just-in-time node access in the unified console adds approval policies (automatic, manual with approvers, or deny) for production nodes; it is charged per node-hour after a free trial.

#Run Command

Run Command runs a Command document on a set of managed nodes now, without logging in to any of them: install software, restart a service, collect a log, change a configuration file. In the notes' words: run a script on 100 EC2 instances at once without SSH.

What you choose for each command:

• The document: AWS-RunShellScript or AWS-RunPowerShellScript with your commands, or any other Command document.
• Targets: tags, a resource group, instance IDs, or all nodes.
• Rate control:
  • Max concurrency: how many nodes run the command at the same time, a number (10) or a percentage (10%). Default 50.
  • Max errors: how many failures are tolerated. When the command fails once more than this value, Systems Manager stops sending it to new targets. Commands already running are allowed to finish, so the final count can end slightly above the limit. Default 0: the first failure stops the rollout.
• Timeouts: how long delivery may wait for a node that is offline, and how long the script may run.
• Output: the console shows only the beginning of long output; send the full output to an S3 bucket and/or CloudWatch Logs.
• Notifications: SNS messages on status changes (In progress, Success, Failed, Timed out…) for the whole command or per node.

Run Command is also an EventBridge target (lesson 19.5), so an event (a new instance running, a schedule) can trigger it. Every SendCommand call is recorded by CloudTrail, and IAM decides who may run which documents on which nodes. On EC2 it costs nothing; on hybrid nodes, since 30 September 2026, each invocation is charged.

Rate control is a safety catch. Try it: with a bad script on a large fleet, the right max concurrency and max errors keep the damage to a handful of servers.

#Patch Manager

Patch Manager automates patching for operating systems and some applications on managed nodes: Windows Server, Amazon Linux, Ubuntu, Red Hat Enterprise Linux, Debian, macOS and others, on EC2 and on-premises. It does three things: decide which patches are approved, scan or install, and report compliance.

#Patch baselines: what is approved

A patch baseline is the rule book. AWS provides a predefined default baseline for each operating system (for example, Windows critical and security updates approved 7 days after release). You can create your own:

• Approval rules: products, classifications (Security, Critical, Bugfix…) and severities, plus an auto-approval delay (ApproveAfterDays, 0 to 360 days after release) or an approve-until date for a fixed cutoff. A delay gives the rest of the world a week to discover a broken patch before it reaches you.
• Approved and rejected patch lists: force a specific patch in (an emergency fix) or keep one out (a patch known to break your software).
• Compliance severity for missing patches, and for Linux, whether non-security updates are allowed and which repositories to use.

A baseline decides what is allowed; the patches themselves still come from the normal sources (Windows Update or your own update server, the Linux distribution's repositories).

#Which nodes get which baseline

Two ways, old and new:

• Patch groups: tag nodes with Patch Group (or PatchGroup) = branches, and register that patch group with a baseline. A node in no patch group gets the default baseline for its OS.
• Patch policies (in Quick Setup, since 2022): one policy sets the baselines, the scan and install schedules and the targets for many accounts and Regions in an Organization at once. This is what AWS recommends for new setups.

#Scan, Install and compliance

Patching runs the Command document AWS-RunPatchBaseline through Run Command, usually inside a Maintenance Window:

• Operation=Scan: compare what is installed with the baseline and report what is missing. Nothing changes, nothing reboots. Run it daily.
• Operation=Install: install the approved missing patches, then reboot if needed (RebootIfNeeded, or NoReboot to reboot yourself later). Run it in a window.

After each run the node reports Compliant or Non-compliant with the list of missing patches. Compliance shows in the Patch Manager dashboard, in Systems Manager Compliance, and can feed AWS Config and Security Hub.

#Maintenance Windows

A maintenance window decides when disruptive work may happen. In the notes' exam words: “ensure disruptive updates only happen on weekends or off-peak hours”. A window has:

• A schedule: a cron or rate expression, with a time zone (Asia/Dhaka; without it the schedule is read as UTC) and an optional offset in days (for “two days after Patch Tuesday”).
• A duration: 1 to 24 hours.
• A cutoff: 0 to 23 hours before the end, after which no new task starts. Tasks already running continue.
• Registered targets: nodes by tag, resource group or ID.
• Registered tasks, each with a priority (lower runs first) and its own max concurrency and max errors. Four task types: Run Command, Automation, Lambda and Step Functions.

A task registered in a window runs only when the window is open. A disruptive task added on a Tuesday afternoon waits for Friday night. If you must act now, you run the command directly, outside the window, deliberately.

#Automation

Automation runs runbooks: Automation documents with several steps that act on AWS resources and on instances. Where Run Command runs one script on nodes, a runbook can stop an instance, wait, snapshot its volume, call an API, run a script, ask a human for approval, branch on a result and create an image, all with logging, retries and rate control.

Common step actions: aws:executeAwsApi (call any AWS API), aws:runCommand (run a Command document on instances), aws:executeScript (Python or PowerShell), aws:invokeLambdaFunction, aws:approve (pause until approvers answer, notified through SNS), aws:branch, aws:waitForAwsResourceProperty, aws:createImage.

AWS-owned runbooks cover the classic jobs from the notes and slides:

• Restart an instance: AWS-RestartEC2Instance.
• Create an AMI: AWS-CreateImage; a full golden-AMI pipeline (launch, patch, test, image, terminate): AWS-UpdateLinuxAmi and AWS-UpdateWindowsAmi.
• Snapshots of EBS volumes, and many AWSSupport- troubleshooting runbooks (reset a lost key, fix an unreachable instance).

How a runbook is started (slide p839):

• Manually from the console, CLI or SDK (start-automation-execution).
• By an EventBridge rule, on an event or on a schedule (EventBridge Scheduler).
• By a Maintenance Window task.
• By AWS Config remediation for a non-compliant resource (lesson 19.7), manually or automatically with retries.
• From a CloudWatch alarm, through an EventBridge rule on the alarm's state change (lesson 19.3), or from an OpsCenter OpsItem.

A runbook acts with an automation role (the AutomationAssumeRole parameter), so the caller needs iam:PassRole for it, and the role needs exactly what the steps call. With rate control a runbook can run on many targets (by tag) with max concurrency and max errors, and from a central account it can run across many accounts and Regions of an Organization. Pricing is per step (0.002 USD per step, plus script duration for aws:executeScript).

#The rest of the toolbox

The exam can name any of these in one answer option. One line each:

#Which Systems Manager tool?

#What goes wrong

• The instance is not listed. One of the three: no agent, no permission (no instance profile, Default Host Management Configuration off, or on with IMDSv1), or no path out (no NAT, no endpoints). Check them in that order; the unified console's diagnosis does it for you.
• The endpoints exist but nothing connects. The endpoint security group has no inbound 443 from the instances, or private DNS is off on the endpoints.
• Connection lost. The agent stopped, the disk is full, the clock is wrong (signatures fail), or the instance profile was removed (it can take up to an hour for the loss to show).
• Session Manager works, but no logs appear. Session logging is not turned on, the instance role cannot write to the bucket or log group, a logs or S3 endpoint is missing in a private subnet, or the session was a port-forwarding or SSH session, which session logging does not record.
• “CloudTrail should show what he typed.” It shows only StartSession. Turn on session logging.
• A command reached 30 servers and broke them all. Max concurrency was 50 (the default) or 100%. Use a small percentage and a low max errors.
• A command stopped after the first failure. Max errors defaults to 0. Set it on purpose.
• Patch compliance is green, but a server is unpatched. That server is not a managed node, so it never reported. Compare managed nodes with your EC2 list, and use the Config rule ec2-instance-managed-by-systems-manager.
• A patch released yesterday was not installed. The baseline's auto-approval delay has not passed. Add it to the approved list for an emergency.
• The window ran at 08:00 instead of 02:00. The schedule had no time zone and was read as UTC. Set --schedule-timezone Asia/Dhaka.
• A task never started. It was registered after the cutoff, or earlier tasks ran past the cutoff.
• An Automation runbook fails with AccessDenied. The automation role lacks a permission a step needs, or the caller lacks iam:PassRole for the role.
• A hybrid server will not register. The activation expired or its registration limit is used up, or the office firewall blocks outbound 443.

#Systems Manager in system design

• No inbound management ports anywhere. Session Manager for people, Run Command and Automation for machines; security groups have no 22 or 3389 rules; private subnets reach Systems Manager through interface endpoints. Bastion hosts disappear.
• Managed by default. Turn on Default Host Management Configuration (or a Quick Setup host management configuration) in every account and Region, so a new instance is managed without anyone remembering a role. Check with a Config rule.
• Patch in layers. Scan daily for visibility, install in maintenance windows, keep an auto-approval delay, test on a staging patch group first, take an AMI or snapshot before installing.
• Immutable where you can. For fleets behind load balancers, patch the golden AMI with Automation (or Image Builder) and replace instances, instead of patching servers in place.
• Prevent, detect, correct. SCPs and IAM prevent; Config and Inspector detect; Automation runbooks correct; EventBridge connects them (lesson 19.7).
• Many accounts. Quick Setup and patch policies across the Organization, Automation across accounts and Regions, Explorer and the unified console for one view.
• Configuration out of code. Parameter Store for settings and AMI IDs, Secrets Manager for rotating secrets (lesson 21.3).
• The same tools on-premises. Hybrid activations bring branch and data-center servers under the same sessions, patching and inventory, which also helps before a migration (lesson 23.4).

#Try it yourself

These commands use the AWS CLI v2 with made-up IDs. Session Manager and Run Command on EC2 cost nothing; Automation steps, hybrid sessions and invocations, NAT gateways and endpoints cost money.

# Which nodes are managed, and are they online?
aws ssm describe-instance-information \
  --query 'InstanceInformationList[].[InstanceId,PingStatus,PlatformName,AgentVersion]' --output text
# i-0a1b2c3d4e5f67890    Online           Amazon Linux    3.3.1802.0
# i-0b2c3d4e5f6789012    ConnectionLost   Ubuntu          3.2.2303.0
# mi-0c3d4e5f678901234   Online           Microsoft Windows Server 2022 Datacenter   3.3.1611.0

# Turn on Default Host Management Configuration for this account and Region
aws ssm update-service-setting \
  --setting-id arn:aws:ssm:ap-south-1:111122223333:servicesetting/ssm/managed-instance/default-ec2-instance-management-role \
  --setting-value service-role/AWSSystemsManagerDefaultEC2InstanceManagementRole

# A shell with no inbound port (needs the Session Manager plugin)
aws ssm start-session --target i-0a1b2c3d4e5f67890
# Starting session with SessionId: admin-0a1b2c3d4e5f6a7b8
# sh-5.2$

# Forward laptop port 3306 to a private RDS database through the instance
aws ssm start-session --target i-0a1b2c3d4e5f67890 \
  --document-name AWS-StartPortForwardingSessionToRemoteHost \
  --parameters '{"host":["accounts-db.abc123xyz.ap-south-1.rds.amazonaws.com"],"portNumber":["3306"],"localPortNumber":["3306"]}'
# Port 3306 opened for sessionId admin-1b2c3d4e5f6a7b8c9.
# Waiting for connections...

# Run a script on every prod server, 10% at a time, stop after 5 failures, full output to S3
aws ssm send-command --document-name AWS-RunShellScript \
  --targets Key=tag:Env,Values=prod \
  --parameters 'commands=["sudo dnf install -y amazon-cloudwatch-agent"]' \
  --max-concurrency 10% --max-errors 5 \
  --output-s3-bucket-name erp-command-output-example \
  --query 'Command.[CommandId,Status,TargetCount]' --output text
# 7b9c2e1a-3f4d-4c5b-9a8e-EXAMPLE11111    Pending    0

aws ssm list-command-invocations --command-id 7b9c2e1a-3f4d-4c5b-9a8e-EXAMPLE11111 \
  --query 'CommandInvocations[].[InstanceId,Status]' --output text
# i-0a1b2c3d4e5f67890    Success
# i-0d4e5f6789012345a    Failed

# A patch baseline: Windows security and critical updates, approved 7 days after release
aws ssm create-patch-baseline --name branches-windows --operating-system WINDOWS \
  --approval-rules 'PatchRules=[{PatchFilterGroup={PatchFilters=[{Key=CLASSIFICATION,Values=[SecurityUpdates,CriticalUpdates]}]},ApproveAfterDays=7}]'
# { "BaselineId": "pb-0123456789abcdef0" }
aws ssm register-patch-baseline-for-patch-group --baseline-id pb-0123456789abcdef0 --patch-group branches

# Patch state of each node after a Scan
aws ssm describe-instance-patch-states --instance-ids mi-0c3d4e5f678901234 \
  --query 'InstancePatchStates[].[InstanceId,Operation,InstalledCount,MissingCount,FailedCount]' --output text
# mi-0c3d4e5f678901234    Scan    214    7    0

# A maintenance window: every Friday 02:00 Dhaka time, 3 hours, cutoff 1 hour
aws ssm create-maintenance-window --name branch-patching \
  --schedule "cron(0 2 ? * FRI *)" --schedule-timezone Asia/Dhaka \
  --duration 3 --cutoff 1 --no-allow-unassociated-targets
# { "WindowId": "mw-0a1b2c3d4e5f67890" }
aws ssm register-target-with-maintenance-window --window-id mw-0a1b2c3d4e5f67890 \
  --resource-type INSTANCE --targets 'Key=tag:Patch Group,Values=branches'
# { "WindowTargetId": "e32eecb2-646c-4f4b-8ed1-EXAMPLE22222" }
aws ssm register-task-with-maintenance-window --window-id mw-0a1b2c3d4e5f67890 \
  --task-type RUN_COMMAND --task-arn AWS-RunPatchBaseline --priority 2 \
  --targets Key=WindowTargetIds,Values=e32eecb2-646c-4f4b-8ed1-EXAMPLE22222 \
  --max-concurrency 10% --max-errors 5 \
  --task-invocation-parameters '{"RunCommand":{"Parameters":{"Operation":["Install"],"RebootOption":["RebootIfNeeded"]}}}'

# Automation: create an AMI from an instance (the role is passed, so you need iam:PassRole)
aws ssm start-automation-execution --document-name AWS-CreateImage \
  --parameters InstanceId=i-0a1b2c3d4e5f67890,AutomationAssumeRole=arn:aws:iam::111122223333:role/ssm-automation-example
# { "AutomationExecutionId": "4105a4fc-f944-4ddc-a9fe-EXAMPLE33333" }
aws ssm get-automation-execution --automation-execution-id 4105a4fc-f944-4ddc-a9fe-EXAMPLE33333 \
  --query 'AutomationExecution.[AutomationExecutionStatus,Outputs]'
# [ "Success", { "createImage.ImageId": [ "ami-0123456789abcdef0" ] } ]

# Hybrid activation for 300 branch servers, valid for 7 days
aws ssm create-activation --default-instance-name branch-server \
  --iam-role SSMServiceRole-example --registration-limit 300 \
  --expiration-date 2026-11-08T00:00:00Z
# { "ActivationId": "e488f2f6-e686-4afb-8a04-EXAMPLE44444", "ActivationCode": "EXAMPLE-ACTIVATION-CODE" }
# On the server: amazon-ssm-agent -register -code "<code>" -id "<id>" -region ap-south-1

#Summary

Key takeaways

• Systems Manager manages EC2, on-premises and other-cloud servers as managed nodes: SSM Agent + permission (instance profile with AmazonSSMManagedInstanceCore, or Default Host Management Configuration with IMDSv2) + outbound 443 (NAT or interface endpoints ssm, ssmmessages, older agents also ec2messages; endpoint SG allows 443, private DNS on). Status: Online, Connection lost, or not listed.
• Hybrid activations register on-premises servers as mi- nodes. Since 30 June 2026 there is no advanced-instances tier; from 30 September 2026 Session Manager and Run Command on hybrid nodes are pay-as-you-go. On EC2 they are free.
• Session Manager: no inbound ports, no bastion, no SSH keys, IAM by tag; session logging to S3 / CloudWatch Logs records the commands; CloudTrail records only StartSession; port forwarding to a remote host such as RDS; SSH through it.
• Run Command: a Command document on many nodes now, targeted by tags; max concurrency (default 50) and max errors (default 0, then no new targets); output to S3 / CloudWatch Logs; SNS notifications; EventBridge can trigger it.
• Patch Manager: baselines (approval rules, auto-approval delay up to 360 days, approved/rejected lists), patch groups or patch policies; AWS-RunPatchBaseline with Scan (report only) or Install (and reboot); compliance reports. Inspector finds CVEs; Patch Manager installs fixes.
• Maintenance Windows: schedule (cron/rate, time zone, offset), duration 1–24 h, cutoff 0–23 h, targets, tasks (Run Command, Automation, Lambda, Step Functions) by priority. Tasks wait for the window. Exam: “only on weekends / off-peak”.
• Automation: multi-step runbooks (AWS-RestartEC2Instance, AWS-CreateImage, AWS-UpdateLinuxAmi), approvals, rate control, multi-account and multi-Region; started manually, by EventBridge (events, schedules, alarm state changes), Maintenance Windows or Config remediation; runs with an automation role (caller needs iam:PassRole).
• The rest: State Manager (desired state), Inventory, Fleet Manager, Distributor, OpsCenter, Explorer, Change Calendar, Quick Setup, Parameter Store. Change Manager and Incident Manager closed to new customers on 7 November 2025. Unified console since November 2024.

Self-check: Systems Manager32 questions