Site-to-Site VPN and CloudHub

By the end of this lesson you will be able to

• Say when a Site-to-Site VPN is the right way to join an office or data centre to AWS, and when Direct Connect, Client VPN or a transit gateway is the better answer.
• Name the three resources a VPN needs (customer gateway, virtual private gateway, VPN connection) and what each one holds: public IP, BGP ASN, certificate, tunnels.
• Explain what travels on the internet inside a tunnel: IKE on UDP 500, ESP, and NAT traversal on UDP 4500 when your router sits behind NAT.
• Explain why every VPN connection has two tunnels in two AZs, what that does and does not protect, and design for maximum availability with two customer gateways and four tunnels.
• Choose static or dynamic (BGP) routing, turn on route propagation, open the security group for ICMP, and troubleshoot a VPN whose tunnels are up but whose ping fails.
• Pick where the VPN ends (virtual private gateway, transit gateway, Cloud WAN), when to use Accelerated Site-to-Site VPN, and what the 2025 features (large bandwidth tunnels, VPN Concentrator) change.
• Build a hub and spoke between branch offices with AWS VPN CloudHub, and know its three conditions.
• Tell Site-to-Site VPN apart from AWS Client VPN, and estimate the monthly bill of both.

#Why join your own network to AWS over the internet

Chapter 3 built a VPC that talks to the internet. Lessons 22.1 to 22.3 connected VPCs to each other and to AWS services. Now the other half of a hybrid cloud: your own network, an office, a factory or a data centre, has to reach private IPs inside a VPC as if they were in the next room.

There are two roads from your building into AWS:

• AWS Site-to-Site VPN (this lesson): an encrypted IPsec connection over the public internet you already pay for. It is ready in minutes, costs about $0.05 an hour per connection, and its speed and latency follow the internet’s mood.
• AWS Direct Connect (lesson 22.5): a private physical line to AWS. Steady bandwidth and latency, never the internet, but ordering it often takes more than a month and it costs more.

Both were introduced in lesson 10.10 as roads to S3. Here you build the VPN for real.

#The three pieces: customer gateway, virtual private gateway, VPN connection

A Site-to-Site VPN needs one end on each side and a connection between them. In AWS that is three resources.

#Virtual private gateway: the AWS end

A virtual private gateway (VGW) is the VPN concentrator on the AWS side.

• You create it and attach it to the VPC you want to reach. A VGW attaches to one VPC at a time; it can never serve several VPCs at once. For many VPCs, end the VPN on a transit gateway instead (lesson 22.6).
• It has a BGP ASN for the Amazon side. If you do not choose one, it gets the Amazon default 64512; you can set a custom private ASN (64512–65534, or a 32-bit private one) when you create it. You cannot change it afterwards: you would create a new VGW.
• It is highly available by design: AWS runs it on redundant endpoints in different AZs. You do not size it or patch it.
• Quotas worth knowing: 5 VGWs per Region and 10 VPN connections per VGW by default (both adjustable). A VGW carries IPv4 only; IPv6 VPNs need a transit gateway or Cloud WAN.

#Customer gateway: your end, described to AWS

Here the words are easy to mix up:

• The customer gateway device is your real router or firewall in the office: a physical box (Cisco, Fortinet, Palo Alto…) or a software appliance. AWS publishes a list of devices it has tested, but any device that speaks standard IPsec and, ideally, BGP will do.
• The customer gateway (CGW) resource is the record you create in AWS that describes that device. It does not configure the device; you do that yourself, using the configuration file AWS generates for your vendor.

What goes into the customer gateway resource:

• The device’s public IP address, which must be static. If the device sits behind a NAT device that supports NAT traversal (NAT-T), enter the NAT device’s public IP, because that is the address AWS will see.
• Its BGP ASN (for dynamic routing): your public ASN if you have one, otherwise a private one such as 65010.
• Optionally a private certificate from AWS Private Certificate Authority (issued through ACM) for certificate-based authentication instead of pre-shared keys. With a certificate, the customer gateway does not even need a fixed IP.

Your office firewall must let the tunnels through: UDP 500 (IKE), UDP 4500 (NAT-T) and ESP (IP protocol 50) between your public IP and the two AWS tunnel IPs.

#When the office’s public IP changes

A customer gateway resource cannot be edited: its IP address and ASN are fixed. If your ISP gives the router a new public IP:

1. Create a new customer gateway with the new IP.
2. Modify the existing VPN connection to use the new customer gateway (aws ec2 modify-vpn-connection --customer-gateway-id), or create a new VPN connection. The connection is briefly unavailable while AWS updates it; download the configuration again and update the router.
3. Delete the old customer gateway.

#The VPN connection: two tunnels

The VPN connection joins one customer gateway to one virtual private gateway (or transit gateway). When you create it, AWS automatically builds two IPsec tunnels, each to its own AWS public IP, in two different Availability Zones. It also generates a configuration file for your device (both tunnels, their pre-shared keys and inside addresses) that you download and apply.

#How the tunnel works: IKE, ESP and NAT traversal

An IPsec tunnel is built in two stages:

1. IKE (Internet Key Exchange, UDP 500). The two ends prove who they are, with a pre-shared key per tunnel or with certificates, and agree on encryption keys. AWS supports IKE version 1 and IKEv2; prefer IKEv2.
2. ESP (Encapsulating Security Payload). Each original packet, private addresses and all, is encrypted (AES), given an integrity check, and wrapped in a new IP header between the two public addresses. An ISP in the middle sees only “203.0.113.10 talks to an AWS address”.

If your router is behind a NAT device, plain ESP is a problem: it has no ports, so the NAT cannot keep track of it. IKE detects the NAT and both ends switch to NAT-T: ESP is wrapped inside UDP 4500, which the NAT treats like any other flow.

#Tunnel options you can set

AWS gives safe defaults, but you can choose per tunnel:

Two numbers that surprise people in production: the maximum MTU through a tunnel is 1446 bytes (MSS 1406), because the IPsec headers take space, and jumbo frames and Path MTU Discovery are not supported. Set the MSS on your router, or large transfers may hang while small pings work.

#Tunnels and high availability

Why two tunnels? Because AWS will take one down. AWS replaces tunnel endpoints for maintenance from time to time, and any hardware can fail. With two tunnels in two AZs, one tunnel can be away while the other carries traffic.

What the two tunnels do and do not do:

• From your office to AWS, your router may send over both tunnels.
• From AWS to your office, AWS picks one tunnel as the active path. On a VGW it is one tunnel across all the VPN connections on that gateway; the rest are standby. So more connections on a VGW add redundancy, not bandwidth. Spreading traffic over several tunnels (ECMP) is a transit gateway feature (lesson 22.6).
• Failover is automatic only if your device has both tunnels configured. A router set up with tunnel 1 only goes dark at the first maintenance window.
• Both tunnels end on your one router. If that router dies, both tunnels die with it.

#Routing: static or dynamic

Packets need routes on both sides and in both directions. There are two ways to give them:

• Dynamic routing (BGP): your router and AWS run a BGP session inside each tunnel. Your router advertises the office ranges; AWS advertises the VPC range. A new subnet in the office is learned without anyone touching AWS. BGP also detects a dead tunnel quickly and withdraws its routes, which makes failover clean. Best practice whenever your device supports BGP.
• Static routing: you type the office prefixes into the VPN connection (up to 100), and you add a route for the VPC range on your router yourself. Fine for a small site with a cheap router; painful when ranges change, and failover relies on the device’s own health checks.

A few limits matter for big networks: a customer gateway may advertise at most 100 routes to a VPN connection on a VGW (1,000 on a transit gateway), and this quota cannot be raised. Summarise your office ranges (for example one 10.0.0.0/8 instead of 300 small prefixes) or move to a transit gateway.

#How AWS chooses between routes

Inside the VPC route table, the most specific route wins (longest prefix match, lesson 3.3). For identical prefixes, the local route always wins, and a static route you added beats a propagated one.

Inside the VGW, for the same prefix arriving from several places, the order is:

1. Routes learned over BGP from Direct Connect,
2. Static routes of a VPN connection,
3. Routes learned over BGP from a VPN connection,
4. among BGP VPN routes, the shortest AS path, then the lowest multi-exit discriminator.

The first line is why a VPN makes a good backup for Direct Connect: while both carry the same prefix, AWS prefers Direct Connect (lesson 22.5). The health of a tunnel always comes before these attributes: a down tunnel is never chosen.

#Route propagation and the security group: the two classic traps

Two steps are easy to forget, and the exam loves both:

• Route propagation. Creating the VPN does not change any VPC route table. You must either enable route propagation for the VGW on every route table whose subnets talk to the office, or add a static route for the office range with the VGW as target. Propagation copies whatever the VGW knows (BGP routes or the connection’s static routes) into the table and keeps it up to date.
• Security groups. The VPN carries traffic to the instance, but the instance’s security group still decides. If you want to test with ping from the office, add an inbound rule for ICMP from the office range. Otherwise the tunnel is up, routing is perfect, and ping still times out.

The trap with propagation is that inbound traffic arrives fine (it only needs the VPC’s local route), so the tunnel looks healthy. It is the reply that has no route back.

The private subnet’s route table in the animation, before and after enabling propagation:

Try it: start from a VPN where everything is done except route propagation, then break and fix the other steps one by one.

#Bandwidth, limits and price

• Bandwidth: up to 1.25 Gbps per tunnel and about 140,000 packets per second. Real throughput depends on the internet path, packet size and your device. On a VGW, traffic from AWS to you uses one tunnel, so plan for 1.25 Gbps per VGW.
• More bandwidth: on a transit gateway, ECMP spreads flows over many tunnels (two connections = four tunnels ≈ 5 Gbps), but one flow still uses one tunnel (lesson 22.6).
• Price: about $0.05 per VPN connection-hour in most Regions, plus normal data transfer out from AWS to your network (data into AWS is free). The tunnels also use public IPv4 addresses, which AWS bills under its public IPv4 pricing; check the VPN pricing page for your Region.

Worked example for the Gazipur factory: one connection for a 730-hour month is 730 × $0.05 = $36.50; sending 200 GB of reports and backups back to the factory adds roughly 200 × $0.09 = $18 at the first internet-egress tier (less after the free allowance). The highly available design with two connections doubles the connection part to $73.

#Where the VPN ends: VGW, transit gateway or Cloud WAN

The office end is always your customer gateway. The AWS end can be:

• A virtual private gateway: one VPN reaches one VPC. Simple and the design most exam questions describe.
• A transit gateway: the VPN becomes an attachment of a regional hub and reaches every VPC (and account) the transit gateway’s route tables allow. Needed for ECMP, IPv6 VPNs, large bandwidth tunnels and Accelerated VPN (lesson 22.6).
• AWS Cloud WAN: the same idea on a global, policy-driven network; a side note for now.

You can move an existing VPN connection from a VGW to a transit gateway (or to another VGW) with modify-vpn-connection; the connection is down briefly while AWS provisions the new endpoints, and you update the route tables afterwards.

#Accelerated Site-to-Site VPN

With acceleration enabled, AWS creates two Global Accelerator accelerators for you, one per tunnel (lesson 11.4). Your tunnels enter the AWS network at the edge location nearest to your office and ride the AWS backbone to the Region, instead of crossing many congested internet hops. That gives steadier latency for distant or badly connected sites.

The rules:

• Only for VPN connections attached to a transit gateway. A VGW does not support accelerated VPN.
• It cannot be switched on or off for an existing connection: create a new one.
• NAT-T is required (on by default), and your device must start IKE.
• You pay the accelerator fees on top of the VPN hours.

#AWS VPN CloudHub: branch offices talking through AWS

That is AWS VPN CloudHub: a simple, low-cost hub-and-spoke network over the internet, for primary or backup connectivity between your sites, with or without anything in the VPC.

How it works:

1. Create one VGW (attached to a VPC or not).
2. Create one customer gateway per site, each with its own public IP and a unique BGP ASN.
3. Create a dynamically routed VPN connection from each customer gateway to the same VGW.
4. Each site advertises only its own prefix over BGP. The VGW re-advertises every route it learns to every other site, so every branch learns every other branch with the VGW as next hop.
5. If instances in the VPC must reach the sites too, enable route propagation (an aggregate such as 10.0.0.0/8 is fine there).

The three conditions:

• Dynamic routing (BGP) on every site.
• A unique BGP ASN for every customer gateway. If two sites share an ASN, BGP’s loop prevention makes each one reject the other’s routes, because it sees its own ASN in the AS path.
• Non-overlapping IP ranges. Two sites with the same range give the VGW two identical routes; traffic for one office ends up in the other.

Cost: the normal VPN connection-hours for each site (four sites = 4 × 730 × $0.05 = $146 a month), plus data transfer out for traffic the VGW sends down to a site. Data sent from a site up to the VGW is free.

#AWS Client VPN: one laptop instead of one site

Site-to-Site VPN connects networks. Sometimes the thing to connect is a person: a developer at home, an auditor at a client’s office. That is AWS Client VPN, a managed point-to-site VPN.

How it works: the endpoint is associated with one or more subnets (one per AZ for availability). Users connect, get an address from a client CIDR range you choose, and their traffic leaves through the endpoint’s network interface in the subnet, with that interface’s address as the source. Authorization rules say which groups may reach which ranges, and split tunnel decides whether only VPC traffic goes through the tunnel or everything.

Cost for the team above for one month: two subnet associations × 730 hours × $0.10 = $146, plus 6 users × 8 hours × 22 days × $0.05 = $52.80. The association part is charged even when nobody is connected, so delete endpoints you no longer need.

#What goes wrong

• Tunnels up, nothing works: no route propagation (or static route) in the subnets’ route tables. The request arrives, the reply goes to the NAT gateway or nowhere.
• Ping fails, applications work: the security group (or a network ACL, lesson 3.5) does not allow ICMP from the office range.
• Both tunnels down from the start: the customer gateway has the router’s private IP although it sits behind NAT; the firewall blocks UDP 500, UDP 4500 or ESP; the pre-shared key, IKE version or cryptography settings do not match the downloaded configuration.
• Outage during AWS maintenance: only one tunnel was configured on the router. Configure and monitor both (CloudWatch metric TunnelState, and the tunnel activity logs AWS can send to CloudWatch Logs).
• Both tunnels drop together: the single office router or its internet line failed. Add a second customer gateway on a second line.
• Large transfers hang, small requests work: MTU and MSS. The tunnel MTU is 1446 bytes; clamp the MSS to 1406 or lower on the router.
• Some office subnets unreachable: with static routing, a new office range was never added to the VPN connection; with BGP, the customer gateway advertises more than 100 prefixes to a VGW and the extra routes are refused.
• Office reaches VPC A but not peered VPC B: peering does not carry VPN traffic onward (no edge-to-edge routing, lesson 22.1). Use a transit gateway.
• CloudHub branch unreachable: a site with static routing, a duplicated BGP ASN, or an overlapping range.
• Throughput stuck at about 1.25 Gbps however many connections you add to a VGW: a VGW uses one tunnel toward you. Move to a transit gateway with ECMP or large bandwidth tunnels.

#How this is used in system design

• First step of a migration: a VPN is up the same day, so servers can move to AWS while still talking to systems left on premises; Direct Connect follows if traffic grows (migration and disaster recovery are chapter 23).
• Backup for Direct Connect: the cheapest second path. AWS prefers Direct Connect for the same prefix; you make your side prefer it with BGP local preference or AS path prepending (lesson 22.5).
• Hybrid DNS: on-premises servers resolve private hosted zones through Route 53 Resolver endpoints, and those queries travel over this VPN (lesson 8.5).
• Private access to AWS services from the office: the VPN brings traffic into the VPC; an interface endpoint carries it to S3 or other services. A gateway endpoint cannot be used from on-premises (lessons 10.10 and 22.2).
• Many VPCs and accounts: end every VPN on one transit gateway in a network account and share it with RAM (lesson 20.1, lesson 22.6).
• Branch networks: CloudHub for a handful of offices on one VGW; a transit gateway (or the newer VPN Concentrator) when there are many.
• Remote people: Client VPN with SAML and MFA for staff and contractors, with authorization rules per group.

#Try it yourself

# 1. Describe your office router to AWS (dynamic routing, private ASN)
aws ec2 create-customer-gateway --type ipsec.1 \
  --public-ip 203.0.113.10 --bgp-asn 65010 \
  --tag-specifications 'ResourceType=customer-gateway,Tags=[{Key=Name,Value=gazipur-fw}]'
# { "CustomerGateway": { "CustomerGatewayId": "cgw-0c1d2e3f4a5b6c7d8", "BgpAsn": "65010",
#   "IpAddress": "203.0.113.10", "State": "available", "Type": "ipsec.1" } }

# 2. Create the virtual private gateway (custom Amazon-side ASN) and attach it to the VPC
aws ec2 create-vpn-gateway --type ipsec.1 --amazon-side-asn 64512
aws ec2 attach-vpn-gateway --vpn-gateway-id vgw-0a1b2c3d4e5f67890 \
  --vpc-id vpc-0a1b2c3d4e5f67890

# 3. Create the VPN connection with dynamic routing and IKEv2 only
aws ec2 create-vpn-connection --type ipsec.1 \
  --customer-gateway-id cgw-0c1d2e3f4a5b6c7d8 --vpn-gateway-id vgw-0a1b2c3d4e5f67890 \
  --options '{"StaticRoutesOnly":false,
    "TunnelOptions":[{"IKEVersions":[{"Value":"ikev2"}],"DPDTimeoutSeconds":40},
                     {"IKEVersions":[{"Value":"ikev2"}],"DPDTimeoutSeconds":40}]}'

# 4. The step everyone forgets: route propagation on the private subnets' route table
aws ec2 enable-vgw-route-propagation \
  --route-table-id rtb-0d1e2f3a4b5c6d7e8 --gateway-id vgw-0a1b2c3d4e5f67890

# 5. Allow ping from the office in the instances' security group
aws ec2 authorize-security-group-ingress --group-id sg-0e1f2a3b4c5d6e7f8 \
  --ip-permissions 'IpProtocol=icmp,FromPort=-1,ToPort=-1,IpRanges=[{CidrIp=192.168.0.0/16}]'

# 6. Check the tunnels (after configuring the router from the downloaded file)
aws ec2 describe-vpn-connections --vpn-connection-ids vpn-0f1e2d3c4b5a69788 \
  --query "VpnConnections[].VgwTelemetry[].[OutsideIpAddress,Status,AcceptedRouteCount]" --output table
# |  198.51.100.21 |  UP   |  2 |
# |  198.51.100.87 |  UP   |  2 |

# 7. The router's public IP changed: new customer gateway, then point the connection at it
aws ec2 create-customer-gateway --type ipsec.1 --public-ip 203.0.113.44 --bgp-asn 65010
aws ec2 modify-vpn-connection --vpn-connection-id vpn-0f1e2d3c4b5a69788 \
  --customer-gateway-id cgw-0a9b8c7d6e5f43210

The configuration file for your device is in the console under the VPN connection, Download configuration, or from aws ec2 get-vpn-connection-device-sample-configuration. It contains <pre-shared-key> values for both tunnels: treat it as a secret. Then, from an office PC:

ping -c 3 10.0.1.20          # needs the ICMP rule from step 5
traceroute -n 10.0.1.20      # first hop is your router, then the tunnel: the internet hops are hidden

#Summary

Key takeaways

• Site-to-Site VPN = IPsec over the public internet: encrypted, ready in minutes, about $0.05 per connection-hour + data out, but inconsistent performance. Direct Connect is the private, steady, slow-to-order alternative.
• Three resources: the customer gateway (describes your router: static public IP, or the NAT device’s IP with NAT-T; BGP ASN; optional certificate), the virtual private gateway (AWS side, attached to one VPC, Amazon ASN 64512 by default or custom), and the VPN connection (two tunnels).
• A customer gateway cannot be edited: new IP → new customer gateway, then modify the VPN connection (or create a new one).
• The tunnel: IKE on UDP 500 (pre-shared key or certificate), ESP carrying the encrypted packet, UDP 4500 with NAT-T. MTU 1446 bytes.
• Two tunnels in two AZs protect against AWS-side failure and maintenance; configure both. Maximum availability: second customer gateway + second VPN connection = 4 tunnels.
• Dynamic routing (BGP) is best practice; static means typing prefixes on both sides. Enable route propagation on the subnets’ route tables and allow ICMP in security groups for ping: the two classic traps.
• 1.25 Gbps per tunnel; a VGW uses one tunnel toward you. More bandwidth: transit gateway with ECMP, or (late 2025) 5 Gbps large bandwidth tunnels on a transit gateway or Cloud WAN.
• End the VPN on a VGW (one VPC) or a transit gateway (many VPCs, ECMP, Accelerated VPN). Accelerated VPN uses Global Accelerator edge locations and works only on a transit gateway.
• VPN CloudHub: several sites, one VGW, sites talk to each other through it; needs BGP, unique ASNs and non-overlapping ranges; cheap hub and spoke. Many VPCs too → Transit Gateway.
• Client VPN connects people, not networks: TLS/OpenVPN client, AD, certificate or SAML authentication, authorization rules, split tunnel.

Self-check: Site-to-Site VPN and CloudHub30 questions