Security Groups in Practice, SSH and Session Manager

By the end of this lesson you will be able to

• Set up security groups for a real instance: the right ports, the right sources, and a separate group for admin access.
• Recognise the classic ports on sight, including the database and file-sharing ports the exam uses.
• Create and protect a key pair, and SSH into a Linux instance from Linux, Mac or Windows.
• Read an SSH failure and know where to look: timeout, connection refused, permission denied, key file too open, host key changed.
• Choose between direct SSH, a bastion, EC2 Instance Connect, the Instance Connect Endpoint and Session Manager.
• Give applications on EC2 access to AWS with an IAM role instead of access keys, and reach Windows instances over RDP or without it.

#From theory to a running instance

Lesson 3.5 explained how security groups work: allow rules only, stateful, attached to network interfaces, able to reference other groups. Lesson 3.4 showed bastion hosts and introduced Session Manager. This lesson is the hands-on side: you have just launched an instance, and now you need to let the right traffic in, log in to it, and keep everyone else out.

There are two different questions, and beginners often mix them up:

1. Who can reach the instance over the network? That is the security group (plus the network ACL, routes and a public IP).
2. Who can log in once they reach it? That is the key pair for SSH, the Windows password for RDP, or IAM for Instance Connect and Session Manager.

A login needs both. Most “I can’t connect” problems are solved by working out which of the two failed.

#Security groups on a real instance

When you launch an instance, the console asks for a security group. A few habits make life easier:

• One group per purpose. For example sg-web (80 and 443 from anywhere) and sg-admin (22 only from your office). An instance can carry up to 5 groups per network interface, and their rules are combined. To cut off admin access everywhere, you detach or empty sg-admin without touching the web rules.
• Use “My IP” carefully. The console can fill in your current address as a /32. Home connections often get a new public address after a router restart, so yesterday’s rule may stop matching today. Offices with fixed addresses can be grouped into a managed prefix list and referenced from every group.
• Changes are live. Editing a rule or attaching a group affects a running instance at once; no restart.
• Outbound is open by default. A new group allows all outbound traffic. Tighten it only if you know every destination the instance needs (updates, APIs, Systems Manager).

#Ports you must know

On the exam, the port is often the whole question: “cannot log in to a Linux instance” → check port 22 in the security group; “cannot log in to a Windows instance” → port 3389.

#A WordPress rule set

A small business runs WordPress on one instance. Visitors can be anywhere; only the owner maintains the server.

0.0.0.0/0 means any IPv4 address in the world; /32 means exactly one address. Two variations matter:

• MySQL on the same instance: WordPress talks to it over localhost, so port 3306 is not opened at all.
• MySQL on a separate database server: that server’s group allows 3306 from the WordPress server’s security group, never from 0.0.0.0/0.

Better still, drop the SSH rule entirely and use Session Manager (below): then the server has only 80 and 443 open.

#Key pairs

Linux instances on AWS do not use passwords for SSH. They use a key pair:

• the public key is placed on the instance, in the default user’s ~/.ssh/authorized_keys, by cloud-init at first boot;
• the private key stays with you, in a file on your laptop.

When you log in, ssh proves that you hold the private key by signing a challenge with it; the instance checks the signature against the public key. The private key itself never crosses the network.

What you need to know about EC2 key pairs:

#Protect the file: chmod 400

ssh refuses to use a private key that other users on your computer can read. If the file is too open, you see a warning that the private key file is unprotected, and the login fails. Fix it once:

chmod 400 my-key.pem          # Linux and Mac: read-only, for you only

On Windows, remove inherited permissions so only your account can read the file (in the file’s Properties → Security → Advanced, or with icacls).

#Losing the private key

AWS cannot give it back. For a Linux instance with an EBS root volume, the usual ways back in are:

• Session Manager or EC2 Instance Connect, if they were already working: log in and add a new public key to authorized_keys;
• stop the instance, detach its root volume, attach it to a helper instance, add a new public key to the file, then put the volume back and start the instance.

#SSH from Linux, Mac and Windows

The command is the same everywhere OpenSSH runs:

ssh -i my-key.pem ec2-user@198.51.100.10

Three parts: the private key (-i), the user name, and the instance’s public IP or DNS name. The user name depends on the image:

The first time you connect, ssh shows the server’s host key fingerprint and asks whether to trust it. Your answer is stored in ~/.ssh/known_hosts, so next time ssh can tell if a different machine is answering on that address.

#When SSH fails: read the error

Each message points at a different layer. Lesson 3.5 explained why timeouts mean a firewall and refusals mean the application; here is the full checklist for SSH.

#EC2 Instance Connect

EC2 Instance Connect lets you open an SSH session without keeping a key file. You click Connect in the console (a terminal opens in the browser) or use the CLI. Behind the scenes:

1. IAM checks that you may push a key to this instance for this OS user (ec2-instance-connect:SendSSHPublicKey).
2. The service pushes a temporary public key into the instance metadata. It is valid for 60 seconds, just long enough to log in.
3. A normal SSH connection on port 22 is made. The ec2-instance-connect package on the instance lets sshd accept the temporary key.

What it requires:

• a supported image: the package comes pre-installed on Amazon Linux 2023, Amazon Linux 2 and Ubuntu 20.04 and later; other Linux images need it installed;
• port 22 open in the security group. For the browser method, the connection comes from the EC2 Instance Connect service’s address range for that Region, not from your laptop, so allow it with the managed prefix list com.amazonaws.<region>.ec2-instance-connect;
• a public IP for the browser method (for private instances, use the endpoint below).

From the CLI, Instance Connect can push a key from your own computer, and your SSH then comes from your own address:

aws ec2-instance-connect ssh --instance-id i-0123456789abcdef0

#EC2 Instance Connect Endpoint: SSH and RDP to private instances

An EC2 Instance Connect Endpoint is an endpoint you create in a subnet of your VPC. AWS tunnels your connection from the console or CLI, through the endpoint, to an instance’s private IP:

• no public IP, no internet gateway, no bastion host on the path;
• IAM decides who may open a tunnel (ec2-instance-connect:OpenTunnel), and every tunnel is recorded in CloudTrail;
• it carries SSH (22) and RDP (3389); each tunnel lasts at most one hour;
• there is no extra charge for the endpoint;
• you still log in with a key, your own or a temporary one pushed by Instance Connect.

The endpoint is the client of your instances, so the security groups follow from that: the instance allows 22 (or 3389) from the endpoint’s security group, and the endpoint is allowed to send that port to the instances.

#Session Manager

AWS Systems Manager Session Manager gives you a shell on an instance with no inbound port at all. Lesson 3.4 introduced it as the modern replacement for a bastion; here is what makes it work.

1. The SSM Agent runs on the instance. It is pre-installed on Amazon Linux 2023, Amazon Linux 2, Ubuntu and the Windows Server images AWS publishes.
2. The instance has permission to talk to Systems Manager: usually an IAM role (instance profile) with the AmazonSSMManagedInstanceCore managed policy. Alternatively, Default Host Management Configuration grants it to every instance in the account and Region.
3. The agent has a path out on HTTPS 443: to the internet through a NAT gateway or internet gateway, or privately through interface VPC endpoints for ssm, ssmmessages and ec2messages. That is the safe set; since SSM Agent 3.3.40.0 the agent uses ssmmessages instead of ec2messages when its permissions allow it, so ssm and ssmmessages are the two it cannot do without (lesson 25.3).
4. You have IAM permission to start a session on that instance (ssm:StartSession), and open it from the console or with aws ssm start-session (the CLI needs the Session Manager plugin).

The agent keeps an outbound connection to Systems Manager, and your session travels down it. The instance’s security group needs no inbound rules, and there is no SSH key to create, share or lose.

What else Session Manager gives you:

• Logging. Session output can be streamed to CloudWatch Logs and stored in an S3 bucket, and sessions can be encrypted with your own key in AWS Key Management Service.
• Fine-grained access. IAM policies can limit sessions to instances with certain tags, for example env=dev.
• Port forwarding. Reach a port on the instance (or a database behind it) from your laptop, still without opening anything inbound.
• SSH and file copies (scp) through it, for tools that expect SSH, still with no port 22 open.
• Windows too: PowerShell sessions, and graphical RDP in the browser through Fleet Manager.
• No extra cost for EC2 instances.

#Comparing the ways in

#Never put access keys on an instance

Sooner or later an application on your instance needs AWS itself: reading from S3, writing to a queue, querying DynamoDB. The tempting shortcut is to run aws configure on the instance and paste in an access key. Never do this. The keys:

• are long-lived: they work from any computer in the world until someone deactivates them;
• travel with the disk: into images, snapshots, backups and, by mistake, code repositories;
• usually carry far more permissions than the application needs;
• must be rotated by hand on every instance.

Instead, attach an IAM role to the instance (through an instance profile):

• The AWS CLI and the AWS software libraries on the instance find the role’s temporary credentials automatically, through the instance metadata (lesson 4.3 covers how to read metadata safely).
• AWS rotates them automatically, hours before they expire. Nothing secret is stored on the disk.
• The role’s policy gives the application exactly what it needs, for example “read this one bucket”.
• An instance has one role at a time. You can attach, replace or remove it on a running instance, with no restart.

#Windows instances: the RDP path

Windows instances are administered with Remote Desktop (RDP, port 3389) and a user name and password, not an SSH key.

1. Allow 3389 in the security group, only from your address (never 0.0.0.0/0).
2. Wait a few minutes after launch, then choose Get Windows password in the console. At first boot, Windows generated a random Administrator password and encrypted it with your key pair’s public key; you provide the private key (.pem) to decrypt it. This is why Windows needs an RSA key pair.
3. Open your RDP client (Remote Desktop Connection on Windows, Windows App on Mac), connect to the public IP, and log in as Administrator with that password.

Ways in without opening 3389 to the internet:

• Fleet Manager Remote Desktop (part of Systems Manager): a graphical RDP session in the browser, carried by Session Manager.
• Session Manager for a PowerShell session.
• EC2 Instance Connect Endpoint for RDP to a private instance.

#How it is used in design

A production setup usually looks like this:

• No SSH from the internet. Application servers sit in private subnets; their security groups allow only the load balancer on the application port. Port 22 is not open anywhere.
• Session Manager for people, through VPC endpoints so even management traffic stays off the internet, with session logs to an S3 bucket that has a retention policy, and IAM rules by tag (developers on env=dev, a small group on production).
• IAM roles for software. Every instance gets a role with just the permissions its application needs. No access keys anywhere.
• Fewer logins at all. Instances are built from images and configured automatically (lesson 4.3), so nobody needs to log in to fix things by hand; a broken instance is replaced, not repaired.
• A break-glass path. Keep one documented emergency route, for example an Instance Connect Endpoint, with tight IAM permissions and alerts on every use.

#Try it yourself

# 1. Create a key pair and protect the private key
aws ec2 create-key-pair --key-name lab-key --key-type ed25519 \
  --query KeyMaterial --output text > lab-key.pem
chmod 400 lab-key.pem

# 2. An admin security group: SSH only from your current address
MYIP=$(curl -s https://checkip.amazonaws.com)
ADMIN=$(aws ec2 create-security-group --group-name admin --description "SSH from my IP" \
  --vpc-id $VPC --query GroupId --output text)
aws ec2 authorize-security-group-ingress --group-id $ADMIN --protocol tcp --port 22 --cidr $MYIP/32

# 3. Log in (debug output shows TCP, host key and authentication stages)
ssh -v -i lab-key.pem ec2-user@198.51.100.10

# Optional: a shortcut in ~/.ssh/config, then just "ssh lab"
#   Host lab
#     HostName 198.51.100.10
#     User ec2-user
#     IdentityFile ~/lab-key.pem

# 4. EC2 Instance Connect: push a temporary key and connect (no key file)
aws ec2-instance-connect ssh --instance-id i-0123456789abcdef0

# 5. Instance Connect Endpoint for a private instance
aws ec2 create-instance-connect-endpoint --subnet-id $PRIVATE_SUBNET --security-group-ids $EICE_SG
aws ec2-instance-connect ssh --instance-id i-0abcdef1234567890 --connection-type eice

# 6. Session Manager: give the instance a role, then open a shell (needs the Session Manager plugin)
aws ec2 associate-iam-instance-profile --instance-id i-0abcdef1234567890 \
  --iam-instance-profile Name=ssm-core-profile
aws ssm describe-instance-information --query "InstanceInformationList[].InstanceId"
aws ssm start-session --target i-0abcdef1234567890

# 7. Close SSH entirely once Session Manager works
aws ec2 revoke-security-group-ingress --group-id $ADMIN --protocol tcp --port 22 --cidr $MYIP/32

#Summary

Key takeaways

• Reaching an instance (security group, NACL, public IP, route) and logging in (key, password, IAM) are separate checks; a login needs both.
• Keep admin access in its own security group, open 22 or 3389 only from known addresses, and never from 0.0.0.0/0. Watch for home IPs that change.
• Ports: 21 FTP, 22 SSH and SFTP, 80 HTTP, 443 HTTPS, 3389 RDP, 3306 MySQL/Aurora, 5432 PostgreSQL, 1433 SQL Server, 1521 Oracle, 2049 NFS, 445 SMB.
• AWS keeps only the public key; the private key is downloaded once. .pem for OpenSSH (including Windows 10+), .ppk for PuTTY. chmod 400 the file. Default user: ec2-user on Amazon Linux, ubuntu on Ubuntu.
• Timeout → network path (often your IP changed). Refused → sshd not listening. Permission denied → wrong user or wrong key.
• Instance Connect pushes a 60-second key through IAM but still needs port 22 (from the service’s range). The Instance Connect Endpoint reaches private instances over SSH or RDP without public IPs or a bastion.
• Session Manager: SSM Agent + instance permissions + an outbound path (NAT or VPC endpoints) + IAM for users. No inbound port, no keys, sessions logged to S3/CloudWatch Logs (CloudTrail only records who started them).
• Applications on EC2 use an IAM role, never access keys stored on the instance.
• Windows: RDP on 3389, password decrypted with the key pair’s private key; Fleet Manager gives RDP without opening 3389.

Self-check: security groups in practice, SSH and Session Manager30 questions