Paths to S3 (1/3): Private Network Routes

By the end of this lesson you will be able to

• Name all 13 ways data reaches S3 that this three-lesson block covers, and say which lesson teaches each one.
• Tell the two jobs apart: a last hop to S3 (gateway endpoint, interface endpoint, Direct Connect public virtual interface) and a road into a VPC (VPC peering, Transit Gateway, Site-to-Site VPN, Direct Connect private or transit virtual interface).
• Explain how traffic flows through a gateway endpoint (prefix list, route table) and an interface endpoint (ENI, private IP, security group, DNS), what each costs, and what each cannot do.
• Lock a bucket to an endpoint or a VPC with endpoint policies and the aws:sourceVpce and aws:sourceVpc condition keys.
• Explain why a peered VPC, a transit gateway spoke or an on-premises network cannot use a gateway endpoint, and what to build instead.
• Choose between the five network paths for a scenario, including cost, bandwidth, setup time and encryption, and calculate NAT, endpoint and Transit Gateway bills.

#The 13 paths to S3, and where each one lives

Data gets into and out of S3 in many ways, and learners mix them up because several of them sound alike: “endpoint”, “gateway”, “transfer”, “connect”. This lesson and the next two form one section that puts all 13 paths side by side, each with its own explanation, its own animation and a real example of what it is for. This first lesson covers the network paths: how packets get from a server to S3 without the public internet.

Paths 1–5 answer “how do my packets travel”. Paths 6–10 answer “which service moves my files”, and they all ride on paths 1–5 or the internet underneath. Paths 11–13 answer “how does my application talk to S3”. Lesson 10.12 ends with a chooser across all 13.

#Two different jobs: the road and the last hop

This is the confusion this lesson exists to clear up. The five network paths do two different jobs:

• The last hop to S3. Something must hand your packet to S3 itself. Only three things do that privately: a gateway endpoint, an interface endpoint, or, for Direct Connect, a public virtual interface.
• A road into a VPC. VPC peering, Transit Gateway, Site-to-Site VPN and Direct Connect (private or transit virtual interface) carry traffic from somewhere else into a VPC. They do not reach S3 by themselves. When the traffic arrives in the VPC, an interface endpoint must take it the rest of the way.

#Where we start: S3 through a NAT gateway

Without any endpoint, an instance in a private subnet reaches S3 the way lesson 3.4 taught: through a NAT gateway in a public subnet, then the internet gateway, to S3’s public endpoint. It works, but:

• every gigabyte is charged by the NAT gateway (about $0.045 per GB processed in many Regions, plus about $0.045 per hour);
• the subnet needs a path to the internet, which some security teams forbid;
• if the NAT gateway’s AZ has trouble, S3 access from the other AZs’ subnets that use it goes too.

S3 is a public service: its endpoints have public IP addresses. Every option below is a way to reach those endpoints, or a private copy of them, without that internet path.

#Path 1: the gateway endpoint

#What a gateway endpoint is

A gateway endpoint is a target you put in a route table. It is not a device and has no IP address. It exists for exactly two services: Amazon S3 and DynamoDB. You create it for a service name such as com.amazonaws.ap-south-1.s3, choose a VPC and the route tables of the subnets that should use it, and optionally attach an endpoint policy.

#How traffic flows through it

1. AWS adds a route to each chosen route table: the destination is the AWS-managed prefix list for S3 in this Region (pl-…, a named list of all S3 address ranges), the target is the endpoint (vpce-…).
2. Your application keeps using the normal S3 name, such as s3.ap-south-1.amazonaws.com, which still resolves to a public S3 address. The VPC must have DNS resolution and DNS hostnames enabled if it uses the Amazon DNS server.
3. The VPC router finds two matching routes, 0.0.0.0/0 (to the NAT gateway) and the S3 prefix list. The more specific one wins (longest prefix match, lesson 3.3), so S3 traffic goes to the endpoint and everything else still goes to NAT.
4. S3 sees the request coming from the instance’s private IP through endpoint vpce-…. Security groups must allow outbound HTTPS to S3; you can reference the S3 prefix list in a security group rule. Network ACLs cannot reference prefix lists, so they need the address ranges themselves.

The private subnet’s route table in the animation, before and after the endpoint:

#Cost and limits of a gateway endpoint

• Free. No hourly charge and no per-GB charge. Normal S3 request and storage charges still apply; data between S3 and EC2 in the same Region is not charged for transfer.
• Same Region only. The prefix list covers S3 in the endpoint’s own Region. A bucket in another Region is not reached through it.
• Quotas. 20 gateway endpoints per Region per account by default (adjustable), and at most 255 per VPC.
• Bandwidth. No endpoint-imposed limit: it is part of the VPC’s routing.
• IP versions. It started as IPv4 only; S3 gateway endpoints now also support IPv6 and dual-stack.
• When you add or change the route, existing TCP connections to S3 from that subnet are reset, because the path and the source address S3 sees change. Do it outside a backup window.

#What a gateway endpoint cannot do

• It cannot be extended out of its VPC. Traffic that arrives over Site-to-Site VPN, Direct Connect, VPC peering or a transit gateway cannot use it. Only traffic that starts inside the VPC can.
• It cannot reach another Region’s buckets.
• It has no security group and no IP address, so on-premises machines have nothing to send packets to.
• With an endpoint in the path, S3 sees private source addresses, so aws:SourceIp conditions do not work for these requests. Use aws:VpcSourceIp, aws:sourceVpc or aws:sourceVpce instead.

#Locking down access through an endpoint

Two policies work together, and both endpoint types support them:

• An endpoint policy on the endpoint says which buckets and actions may be used through this endpoint. The default is full access. Use it to stop a compromised instance from copying data to a bucket in someone else’s account.
• A bucket policy with a condition on the endpoint or the VPC says this bucket may only be used through that path (lesson 10.2).

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "OnlyThroughOurEndpoint",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": ["arn:aws:s3:::bank-backups-example", "arn:aws:s3:::bank-backups-example/*"],
      "Condition": { "StringNotEquals": { "aws:sourceVpce": "vpce-0a1b2c3d4e5f60718" } }
    }
  ]
}

A policy like this also blocks the console and anyone outside the VPC, including you. Keep an exception for an administrator role, or you may have to ask the account’s root user to delete the policy.

#Path 2: the interface endpoint (PrivateLink)

#What an interface endpoint is

An interface endpoint is built on AWS PrivateLink. It places one elastic network interface (ENI) in each subnet you choose, normally one per AZ, each with a private IP address from that subnet and a security group. To your network, S3 now has addresses inside your VPC. Interface endpoints exist for most AWS services (SQS, SNS, KMS, the EC2 API…); for S3 you can have both types in the same VPC.

#How traffic flows, and why DNS decides

Because the endpoint has real IP addresses, the question is only which name your application resolves:

• Endpoint-specific names. Every S3 interface endpoint gets names such as bucket.vpce-0a1b2c3d-xyz.s3.ap-south-1.vpce.amazonaws.com (Regional) and a zonal variant with the AZ in it. These names are in public DNS and resolve to the ENIs’ private IPs, so a server on-premises or in another VPC can use them without any DNS changes. With the AWS CLI you pass them as --endpoint-url.
• Private DNS. If you enable private DNS on the endpoint, the normal S3 names (s3.ap-south-1.amazonaws.com, the access point and control names) resolve to the endpoint’s private IPs inside this VPC. Applications need no change.
• Private DNS only for inbound endpoints. If the VPC also has an S3 gateway endpoint, you can choose this option (the default when you enable private DNS for S3): instances in the VPC keep resolving to public S3 addresses and use the free gateway endpoint, while queries arriving through a Route 53 Resolver inbound endpoint (from on-premises) get the interface endpoint’s private IPs. The gateway endpoint must exist for this option, and you cannot delete it while the option is on.

Then the packet goes to the ENI over HTTPS, the ENI’s security group must allow inbound TCP 443 from the clients, and PrivateLink carries the request to S3 on the AWS network.

#Cost and limits of an interface endpoint

• Hourly charge per AZ: about $0.01 per hour for each AZ the endpoint is in.
• Data processing: about $0.01 per GB for the first petabyte each month, cheaper above that.
• Example: an endpoint in 3 AZs for a 730-hour month costs 3 × 730 × $0.01 = $21.90, plus 5 TB processed = 5,120 GB × $0.01 = $51.20, so about $73 in total. A gateway endpoint for the same traffic would cost $0.
• Bandwidth: up to 10 Gbps per AZ by default, scaling automatically to 100 Gbps per AZ.
• Quotas: 50 interface (and Gateway Load Balancer) endpoints per VPC by default.

#What an interface endpoint cannot do

• It does not serve S3 website endpoints (lesson 10.3), and it does not support CopyObject or UploadPartCopy between buckets in different Regions.
• It costs money for in-VPC traffic that a gateway endpoint would carry free.
• It does nothing if nobody resolves to it: with private DNS off and the default name in use, traffic goes to public S3 addresses as before.
• A forgotten security group rule turns it into a black hole: requests time out instead of returning Access Denied.

#Gateway or interface endpoint for S3?

#Path 3: VPC peering

#What VPC peering is

VPC peering is a private network connection between two VPCs. Instances in each can talk to the other using private IPs, as if they were in one network, over the AWS backbone, never the public internet.

• It works across accounts and across Regions (inter-Region peering is encrypted).
• The two VPCs’ CIDR ranges must not overlap (lesson 3.1).
• After the peering is accepted you must add routes in both VPCs’ route tables (destination = the other VPC’s CIDR, target = pcx-…). Without them nothing flows.
• In the same Region, a security group can reference a security group in the peer VPC, even across accounts.
• Peering is not transitive. If A is peered with B and B with C, A cannot reach C through B. You need an A–C peering too. A full mesh of n VPCs needs n(n−1)/2 connections: 100 VPCs → 4,950.
• No edge-to-edge routing. A VPC cannot use its peer’s internet gateway, NAT gateway, VPN, Direct Connect or gateway endpoint.
• Cost: no hourly charge. Data that stays within one AZ is free; data that crosses AZs or Regions is charged as normal transfer.

#How a peered VPC reaches S3

Peering does not reach S3. It only lets VPC A reach addresses inside VPC B. So:

• VPC B’s gateway endpoint: not usable from VPC A (no edge-to-edge routing). The packet is dropped.
• VPC B’s interface endpoint: usable, because its ENIs are real addresses inside VPC B. VPC A uses the endpoint-specific DNS name (public DNS answers with VPC B’s private IPs), or associates a Route 53 private hosted zone with VPC A. The endpoint’s security group must allow VPC A’s range.
• Usually best: VPC A creates its own gateway endpoint, free, and peering is kept for what it is for, VPC-to-VPC traffic.

#Why choose peering, and when not to

Choose peering for a few VPCs that need low-latency, cheap private traffic between them: no hourly fee, no per-GB processing fee, no bandwidth limit of its own. Avoid it when the number of VPCs grows (mesh explosion), when you need transitive routing or a shared path to on-premises, or when the only goal is reaching S3 (use endpoints). The full peering story is in chapter 22.

#Path 4: Transit Gateway

#What Transit Gateway is

AWS Transit Gateway (TGW) is a regional hub. VPCs, Site-to-Site VPN connections, Direct Connect gateways and other transit gateways attach to it (hub and spoke), and it routes between them. Unlike peering, routing through it is transitive: A and C both attached means A can reach C.

• Regional resource; transit gateways in different Regions can be peered.
• Shared with other accounts through AWS Resource Access Manager (RAM).
• Transit gateway route tables control which attachments can reach which: for example, production and development VPCs both reach shared services but not each other.
• Supports IP multicast, which no other AWS networking service does.
• ECMP for Site-to-Site VPN: with several VPN connections attached, it spreads flows over all tunnels. Each tunnel carries up to 1.25 Gbps, so two VPN connections (four tunnels) give about 5 Gbps in total. A virtual private gateway cannot do this. One flow still uses one tunnel.
• Cost: about $0.05 per attachment per hour and $0.02 per GB processed.

#How a transit gateway reaches S3

Like peering, a transit gateway only carries traffic to addresses inside attached VPCs. A gateway endpoint in another VPC cannot be reached through it. The common design for many VPCs is a shared endpoints VPC:

1. Create the S3 interface endpoint in one central VPC attached to the transit gateway.
2. Private DNS of an interface endpoint works only inside its own VPC, so create a Route 53 private hosted zone for the S3 name that points at the endpoint, and associate it with every spoke VPC (or forward the name with Route 53 Resolver rules).
3. Spokes route the central VPC’s range to the transit gateway. Their S3 traffic goes spoke → TGW → endpoint ENI → S3.

#Transit Gateway with Direct Connect, and why you would pick it

To share one Direct Connect connection with many VPCs in many accounts: create a transit virtual interface to a Direct Connect gateway, associate the Direct Connect gateway with the transit gateway, and share the transit gateway with the other accounts through RAM.

Pick Transit Gateway when you have many VPCs, need transitive routing, want one shared path to on-premises (VPN or Direct Connect), or need more VPN bandwidth with ECMP. Do not pick it just to reach S3 from a VPC that could have its own free gateway endpoint: you would pay attachment and per-GB fees for nothing. More in chapter 22.

#Path 5: Site-to-Site VPN or Direct Connect

These connect your own network (an office, a data centre) to AWS. Both end in a VPC (or a transit gateway); neither reaches S3 by itself.

#Site-to-Site VPN

• An encrypted IPsec connection over the public internet between your network and AWS. Ready in minutes, cheap (about $0.05 per connection-hour plus data transfer out), but its speed and latency follow the internet’s mood.
• Virtual private gateway (VGW): the AWS end, attached to one VPC (you can set its ASN). Or terminate the VPN on a transit gateway instead.
• Customer gateway (CGW): the AWS resource describing your router or firewall: its public IP and BGP ASN. If the router sits behind a NAT device with NAT traversal (NAT-T), use the NAT device’s public IP. If that public IP changes, you create a new customer gateway; the VPN connection can then be modified to use it, or replaced.
• One VPN connection = two tunnels, ending in two different AZs on the AWS side. Each tunnel carries up to 1.25 Gbps. Since late 2025, VPN connections on a transit gateway or Cloud WAN can use 5 Gbps tunnels (large bandwidth tunnels); exam questions still assume 1.25 Gbps.
• Routing: dynamic (BGP), best practice, or static. Turn on route propagation for the VGW in the subnets’ route tables, and allow ICMP in security groups if you want to ping from on-premises.
• Maximum availability: two tunnels protect against an AWS-side failure, not against your one router failing. Add a second customer gateway (second router) and a second VPN connection: two CGWs, four tunnels.
• AWS VPN CloudHub: several branch offices each build a VPN to the same VGW, with BGP and non-overlapping ranges, and can then talk to each other through it: a low-cost hub and spoke over the internet. (Many VPCs as well as many VPNs → Transit Gateway.)

#Direct Connect

• A dedicated private line from your network to an AWS Direct Connect location, then to AWS: steady bandwidth and latency, never the internet. Good for daily terabytes, real-time feeds, hybrid systems and compliance.
• Dedicated connections: a physical port of 1, 10, 100 or 400 Gbps, requested from AWS. Hosted connections: capacity from a Direct Connect Partner, 50 Mbps to 25 Gbps; the partner can change the capacity if it supports upgrades, otherwise you order a new one. A need below 1 Gbps (for example 500 Mbps) → hosted.
• Lead time: a new connection often takes more than a month.
• Virtual interfaces (VIFs) on the line: a private VIF reaches private IPs in a VPC (through its VGW or a Direct Connect gateway); a public VIF reaches AWS public addresses, including S3, over the line; a transit VIF reaches transit gateways through a Direct Connect gateway.
• Direct Connect gateway: one virtual interface reaches VPCs in many Regions (China excluded) and accounts. It does not route between those VPCs.
• Not encrypted by default (private, but plain text). To encrypt: run an IPsec Site-to-Site VPN over Direct Connect (the usual exam answer), or use MACsec on dedicated 10, 100 or 400 Gbps connections at supported locations.
• Resiliency: development and test, one location with two connections on separate devices; high resiliency, one connection at each of two locations; maximum resiliency (mission-critical), two connections at each of two locations on separate devices, four in total.
• Cheapest backup: a Site-to-Site VPN next to Direct Connect. To make traffic prefer Direct Connect, use BGP: from AWS to you, AWS already prefers Direct Connect over VPN for the same prefix; from you to AWS, set local preference, or make the VPN path look longer with AS path prepending.
• Cost: a port-hour fee plus data transfer out at Direct Connect rates, lower than internet rates; data in is free.

#How on-premises traffic reaches S3

1. VPN or Direct Connect private VIF + interface endpoint: the road brings traffic into the VPC; the server sends to the interface endpoint’s private IP (endpoint-specific name, or private DNS through a Route 53 Resolver inbound endpoint, lesson 8.5); PrivateLink does the last hop. Lock the bucket with aws:sourceVpce.
2. Direct Connect public VIF: traffic goes straight to S3’s public endpoints over the private line, no VPC, no endpoint fee. The bucket sees your public IPs, so restrict with aws:SourceIp. Filter routes on your router, because a public VIF learns all AWS public prefixes.
3. Never the gateway endpoint: on-premises traffic cannot use it.

#Choosing between VPN and Direct Connect

Chapter 22 goes deeper into Site-to-Site VPN and Direct Connect.

#The five paths side by side

#A decision guide

1. Where does the request start?

   In a VPC, for a bucket in the same Region → gateway endpoint in that VPC. Stop here: nothing is cheaper.

2. Is it another VPC?

   Same Region → give that VPC its own gateway endpoint. Many VPCs and you want one control point → Transit Gateway + shared interface endpoint with a private hosted zone. Bucket in another Region → interface endpoint in the bucket’s Region over inter-Region peering or TGW peering (or replicate the data, lesson 10.4).

3. Is it on-premises?

   Need it this week or bandwidth is modest → Site-to-Site VPN + interface endpoint. Need steady, high bandwidth or no internet at all → Direct Connect private VIF + interface endpoint (transit VIF + Direct Connect gateway + TGW for many VPCs). Only need “not over the internet” and want the lowest running cost → Direct Connect public VIF.

4. Then add the controls

   Endpoint policy (which buckets), bucket policy with aws:sourceVpce or aws:sourceVpc (which path), encryption for Direct Connect if compliance asks (VPN over it or MACsec), and a backup path (VPN for Direct Connect, two customer gateways for VPN).

#What goes wrong

• Timeouts right after creating a gateway endpoint for long-running uploads: adding the route resets existing connections. Retry; schedule changes outside backup windows.
• Access Denied after adding a gateway endpoint: a bucket policy with aws:SourceIp (public NAT address) no longer matches, because S3 now sees private addresses. Switch to aws:sourceVpce or aws:sourceVpc. Or the endpoint policy does not include the bucket, for example a bucket AWS needs for agents or logs.
• The route table was not selected: the endpoint exists but one subnet’s route table has no prefix-list route, so that subnet still uses NAT (and pays) or fails.
• Bucket in another Region: requests silently go through NAT, or time out when there is no NAT.
• Interface endpoint timeouts: the endpoint’s security group does not allow 443 from the clients, or DNS still returns public addresses (private DNS off, or the client is outside the VPC and not using the endpoint-specific name).
• On-premises cannot reach S3 privately over VPN or Direct Connect: someone built only a gateway endpoint.
• Peered VPC or TGW spoke fails to reach S3: it is trying to use another VPC’s gateway endpoint (edge-to-edge), or the private hosted zone is not associated with the spoke.
• Locked out of a bucket: an aws:sourceVpce deny without an exception also blocks the console.
• Direct Connect “secure” but unencrypted: compliance fails audit. Add a VPN over Direct Connect or MACsec.

#How this is used in system design

• Private-only subnets: data, batch and backup tiers with no NAT at all, reaching S3 and DynamoDB through gateway endpoints and other services through interface endpoints.
• Data perimeter: an endpoint policy that allows only the organisation’s buckets, plus bucket policies that allow only the organisation’s endpoints or VPCs, so data can neither leave to a foreign bucket nor be read from outside.
• Hybrid data platform: on-premises producers write through Direct Connect + interface endpoint, in-VPC consumers read through the gateway endpoint, with “private DNS only for inbound endpoints” doing the split automatically.
• Hub and spoke with Transit Gateway and centralised interface endpoints for many accounts; heavy S3 users keep local gateway endpoints to avoid per-GB fees.
• Bulk movers such as DataSync and S3 File Gateway (lesson 10.11) usually run on top of these paths: for example a DataSync agent on-premises sending over Direct Connect to an interface endpoint.

#Try it yourself

# 1. Find the S3 prefix list for your Region (used by gateway endpoint routes and security groups)
aws ec2 describe-managed-prefix-lists \
  --filters Name=prefix-list-name,Values=com.amazonaws.ap-south-1.s3

# 2. Create a gateway endpoint and attach it to a private subnet's route table
aws ec2 create-vpc-endpoint --vpc-id vpc-0abc1234example \
  --vpc-endpoint-type Gateway --service-name com.amazonaws.ap-south-1.s3 \
  --route-table-ids rtb-0def5678example

# 3. See the route AWS added (destination = pl-..., target = vpce-...)
aws ec2 describe-route-tables --route-table-ids rtb-0def5678example \
  --query "RouteTables[].Routes[]"

# 4. Create an interface endpoint in two subnets with private DNS for on-premises only
aws ec2 create-vpc-endpoint --vpc-id vpc-0abc1234example \
  --vpc-endpoint-type Interface --service-name com.amazonaws.ap-south-1.s3 \
  --subnet-ids subnet-0aaa1111example subnet-0bbb2222example \
  --security-group-ids sg-0ccc3333example \
  --private-dns-enabled --dns-options PrivateDnsOnlyForInboundResolverEndpoint=true

# 5. Use the interface endpoint by its endpoint-specific name (works from on-premises too)
aws s3 ls s3://bank-backups-example/ --region ap-south-1 \
  --endpoint-url https://bucket.vpce-0a1b2c3d-xyz.s3.ap-south-1.vpce.amazonaws.com

# 6. Check the hybrid side: VPN tunnels and Direct Connect virtual interfaces
aws ec2 describe-vpn-connections --query "VpnConnections[].VgwTelemetry"
aws directconnect describe-virtual-interfaces \
  --query "virtualInterfaces[].[virtualInterfaceType,virtualInterfaceState]"

Then attach the bucket policy from the gateway endpoint section with aws s3api put-bucket-policy --bucket bank-backups-example --policy file://policy.json, and confirm that the same aws s3 ls from your laptop is now denied while it works from the instance.

#Summary

Key takeaways

• 13 paths to S3 in three lessons: network paths here (1–5), bulk movers in 10.11 (6–10), application techniques in 10.12 (11–13).
• Two jobs: the last hop to S3 (gateway endpoint, interface endpoint, Direct Connect public VIF) and roads into a VPC (peering, Transit Gateway, VPN, Direct Connect private/transit VIF). Roads never reach S3 by themselves.
• Gateway endpoint: S3 and DynamoDB only; route-table target with the S3 prefix list; normal DNS names; free; same Region; cannot be used from on-premises, peered VPCs or through a transit gateway; no security group. The cost-effective exam answer.
• Interface endpoint: PrivateLink ENIs with private IPs and a security group in each chosen subnet; endpoint-specific names or private DNS (optionally only for inbound Resolver endpoints, with a gateway endpoint in the VPC); ~$0.01 per AZ-hour + ~$0.01 per GB; reachable from on-premises, peered VPCs, Transit Gateway and other Regions.
• Lock it down with endpoint policies and bucket policies using aws:sourceVpce or aws:sourceVpc; aws:SourceIp does not work through endpoints.
• Peering: two VPCs, cross-account and cross-Region, no overlapping CIDRs, routes in both, not transitive, no edge-to-edge routing, n(n−1)/2 for a mesh, no hourly fee.
• Transit Gateway: regional hub, transitive, RAM sharing, route tables, multicast, ECMP for VPN bandwidth, shares one Direct Connect through a Direct Connect gateway and transit VIF; ~$0.05 per attachment-hour + $0.02 per GB; reaches S3 only through an interface endpoint in an attached VPC.
• Site-to-Site VPN: IPsec over the internet, minutes to set up, VGW (one VPC) or TGW, CGW = your router’s public IP, 2 tunnels in 2 AZs, 1.25 Gbps each, BGP or static, two CGWs for maximum availability, CloudHub for branch-to-branch.
• Direct Connect: private line, dedicated 1–400 Gbps or hosted 50 Mbps–25 Gbps, often over a month to set up, private/public/transit VIFs, Direct Connect gateway for many Regions, not encrypted (VPN over it or MACsec), two-location resiliency, VPN as the cheap backup.

Self-check: private network paths to S332 questions