Lambda in a VPC, RDS Proxy and Database Triggers

By the end of this lesson you will be able to

• Explain where a Lambda function runs by default (a VPC that AWS owns), what it can reach from there (the internet and public AWS endpoints) and what it cannot (your private RDS, ElastiCache, internal load balancers).
• Attach a function to your VPC: subnets in two or more AZs, a security group, the execution-role permissions in AWSLambdaVPCAccessExecutionRole, and how Hyperplane ENIs are created, shared and reclaimed.
• Explain why a VPC-attached function has no internet, even in a public subnet, and fix it with a NAT gateway (internet) or VPC endpoints (AWS services).
• Describe the connection storm that Lambda causes on a relational database and how RDS Proxy fixes it (pooling, faster failover, IAM authentication, Secrets Manager), and why the function must then be in the VPC.
• Tell apart invoking Lambda from inside the database (RDS for PostgreSQL, Aurora PostgreSQL, Aurora MySQL: data events) and RDS event notifications (news about the instance itself, no data), and pick the right one, or a change-data-capture tool, for a requirement.

#Where a Lambda function runs by default

Lesson 15.1 said “no servers to manage”. The servers still exist, and so does a network: every Lambda function runs inside a VPC that the Lambda service owns and manages. You never see that VPC, you cannot change its route tables, and nothing in your account points at it.

From there, a function has the same reach as any machine on the internet:

• The internet works. A call to a payment gateway, a weather API or GitHub goes straight out.
• Public AWS endpoints work. DynamoDB, S3, SQS, SNS, Secrets Manager and the rest are reached over HTTPS at their public Regional endpoints, and IAM (the execution role) decides what is allowed.
• Your private resources do not. An RDS database with Publicly accessible = No, an ElastiCache cluster, an internal ALB, an EC2 instance at 10.0.2.40: they only have private addresses inside your VPC, and the Lambda-owned VPC has no route there. The connection never arrives; the function waits until its timeout.

#Attaching a function to your VPC

You give the function a VPC configuration:

• Subnets: the private subnets where the resources live, at least two in different AZs so an AZ problem does not stop the function. (The VPC itself is derived from the subnets.)
• Security groups: one or more, for example lambda-sg. They control the function’s outgoing traffic and are what your database’s security group refers to.
• Optionally Ipv6AllowedForDualStack, to let the function open IPv6 connections in dual-stack subnets (off by default; IPv6-only subnets are not supported).

The execution role needs permission to create and manage network interfaces: ec2:CreateNetworkInterface, ec2:DescribeNetworkInterfaces, ec2:DescribeSubnets, ec2:DeleteNetworkInterface, ec2:AssignPrivateIpAddresses and ec2:UnassignPrivateIpAddresses. The AWS managed policy AWSLambdaVPCAccessExecutionRole contains them (plus the CloudWatch Logs permissions), and the console adds it for you. The person making the change also needs ec2:DescribeSecurityGroups, ec2:DescribeSubnets and ec2:DescribeVpcs.

#Hyperplane ENIs: one per subnet and security group pair

Lambda does not put one network interface per execution environment into your subnets. It creates a Hyperplane ENI for each combination of subnet and security groups, and every function in the account that uses the same combination shares it. One Hyperplane ENI handles up to 65,000 connections; Lambda adds more by itself when traffic needs them.

The life of that ENI explains several things you will see in the console:

The code still runs in the Lambda-owned VPC. Only its network traffic enters your VPC through the ENI, with the ENI’s private IP and your security group. The ENI behaves like a one-way door: the function can open connections into your VPC, but nothing in your VPC can open a connection to the function through it. You still invoke the function the normal way (API Gateway, an S3 event, the SDK), through the Lambda service.

#Security groups: refer to the group, not the address

The database’s security group must allow its port (5432 for PostgreSQL, 3306 for MySQL) from lambda-sg, not from an IP range: the ENI’s address can change when Lambda recreates or adds ENIs, and the group follows them automatically (security groups are in lesson 3.5). The function’s own group usually has no inbound rules at all, because nothing connects to the function; its outbound rules must allow the database port and 443 for AWS APIs.

A wrong security group does not produce an error message. The packet is silently dropped and the function ends with “Task timed out”. A timeout points to the network (route, security group, NACL); an error such as “password authentication failed” proves the network is fine and the credentials are not.

#Locking it down with IAM

Two IAM tricks come up in real accounts and in the exam:

• The function’s code also gets the EC2 permissions you gave its role, so a compromised function could create network interfaces. Add a deny for those EC2 actions with the condition key lambda:SourceFunctionArn: it matches only calls made by the function’s code, so the Lambda service can still manage the ENIs.
• Force every function into a VPC (or into approved VPCs, subnets and security groups) with the condition keys lambda:VpcIds, lambda:SubnetIds and lambda:SecurityGroupIds on lambda:CreateFunction and lambda:UpdateFunctionConfiguration. A deny with "Null": {"lambda:VpcIds": "true"} refuses any function without a VPC configuration; as an SCP it applies to the whole organization (chapter 20).

#A VPC function has no internet

Here is the price of the visitor pass. Once attached, all of the function’s outgoing traffic leaves through the ENI and follows your subnet’s route table. A typical private subnet has one route, 10.0.0.0/16 → local. So:

• the payment gateway on the internet: times out;
• DynamoDB, S3, SQS, Secrets Manager, even the Lambda API itself: also time out, because their endpoints are public addresses outside your VPC.

The second point is the one that surprises people. “I added a VPC so the function could read RDS, and now it hangs when it reads its database password from Secrets Manager” is a classic ticket.

#The public subnet trap

The obvious shortcut is to attach the function to a public subnet, whose route table sends 0.0.0.0/0 to the internet gateway. EC2 instances there reach the internet, so why not Lambda? Because an internet gateway only translates traffic for addresses that have a public IP (lesson 3.3), and Lambda never gives its ENI a public IP; you cannot attach an Elastic IP to it either. AWS states it directly: connecting a function to a public subnet does not give it internet access or a public IP address.

#Fix 1: a NAT gateway for the internet

The standard pattern from lesson 3.4:

1. Attach the function to private subnets.
2. Put a NAT gateway (with an Elastic IP) in a public subnet of the same AZ.
3. In the private subnet’s route table, add 0.0.0.0/0 → nat-gateway.

The NAT gateway swaps the private source address for its own public one, and the internet gateway sends the packet out. A side benefit: every call from every function leaves from one fixed public IP, which a bank or payment partner can put on its allow-list. For high availability, use one NAT gateway per AZ, each private subnet pointing at the one in its own AZ. NAT gateways are billed per hour and per GB processed, so pushing heavy S3 or DynamoDB traffic through them is money for nothing.

#Fix 2: VPC endpoints for AWS services

If the function only talks to AWS services, it may not need the internet at all. VPC endpoints (lesson 22.2) give private paths:

• Gateway endpoints for S3 and DynamoDB add a route (a prefix list) to your route tables. Free, no ENI.
• Interface endpoints (AWS PrivateLink) for almost everything else: SQS, SNS, Secrets Manager, KMS, STS, Step Functions, Lambda itself… Each is an ENI with a private IP in your subnets, and private DNS makes the normal service name (secretsmanager.ap-southeast-1.amazonaws.com) resolve to it, so the code does not change. Billed per hour per AZ and per GB.

Endpoints never reach the internet, and a NAT gateway never makes S3 or DynamoDB traffic cheaper. Most production VPCs use both: endpoints for AWS services, a NAT gateway for the rest.

#Do you need the VPC at all?

Try the combinations yourself. Pick where the function’s traffic leaves and what the VPC has, then check every path:

#Lambda and relational databases: the connection storm

A web server fleet talks to a database through a connection pool inside each server: 10 servers × 20 connections = 200, steady. Lambda breaks that model, as lesson 7.4 explained from the database side:

• Each execution environment handles one request at a time and is its own little process, so it has its own connection, normally opened in the init code (outside the handler) and reused while that environment lives.
• Concurrency is environments, and it rises fast: by up to 1,000 environments every 10 seconds per function, up to the account limit (1,000 per Region by default; lesson 15.2).
• A relational database has a fixed max_connections, set by its memory: a few hundred on a small instance.

So a traffic spike turns into a connection storm: 1,000 environments open 1,000 connections at the same moment, the database refuses everything above its limit with “too many connections”, and the requests that matter most fail. The CPU is often fine; the database simply has no connection slots left. Idle environments are frozen, not closed, so their connections stay open for a while after the rush, too.

A quick estimate: concurrent environments ≈ requests per second × average duration in seconds. At 2,500 requests per second and 400 ms per request, that is 1,000 environments, so 1,000 direct connections.

#RDS Proxy in front of the database

Amazon RDS Proxy is a fully managed, serverless, highly available connection pooler for RDS (MySQL, PostgreSQL, MariaDB, SQL Server) and Aurora (MySQL, PostgreSQL). The functions connect to the proxy endpoint instead of the database, with no other code change:

• Scalability (pooling). The proxy keeps a small pool of real database connections and lends one to a client only while its transaction runs (multiplexing). If a request takes 200 ms and spends 20 ms in a transaction, 1,000 concurrent functions need about 1,000 × 20 ÷ 200 = 100 database connections. Requests beyond the pool wait in a queue instead of being refused (the proxy sheds load only beyond the limits you set).
• Availability. The functions’ connections end at the proxy, so they survive a Multi-AZ failover; the proxy watches the instances directly instead of waiting for DNS and reconnects to the new primary. AWS quotes failover times up to 66% shorter.
• Security. You can require IAM authentication for clients: the function signs a short-lived token with its execution role (rds-db:connect), so no password lives in code or environment variables. The proxy reaches the database with credentials it reads from Secrets Manager (or, since September 2025 for MySQL and PostgreSQL engines, with IAM database authentication end to end, so no secret is needed at all).
• Network. The proxy lives in the same VPC as the database and can never be publicly accessible. So a function that uses it must be attached to that VPC (or a connected one), with the proxy’s security group allowing the port from the function’s group.

The details of multiplexing, pinning, endpoints and pricing are in lesson 7.4; here the point is the Lambda side.

#Other ways to keep the database alive

• Open the connection outside the handler, so a warm environment reuses it instead of opening one per request; close nothing per request.
• Cap the function’s concurrency (reserved concurrency, lesson 15.2) when protecting the database matters more than serving every request at once. It complements RDS Proxy; it does not replace it.
• RDS Data API (Aurora PostgreSQL and Aurora MySQL, provisioned and Serverless v2): run SQL over an HTTPS API with IAM and a Secrets Manager secret. There are no database connections to manage and the function does not even need to be in the VPC. Good for low-to-moderate traffic; it is a different API, so the code changes.
• DynamoDB (lessons 15.4 and 15.5) has no connections at all: every request is an HTTPS call. That is why serverless designs pair Lambda with DynamoDB by default.

#When the database calls Lambda

So far the function called the database. The arrows can also go the other way, and there are two features that look alike but answer different questions.

#Invoking Lambda from inside the database

RDS for PostgreSQL, Aurora PostgreSQL and Aurora MySQL can call a Lambda function from SQL, so you can react to data events inside the database, whichever application changed the data:

• PostgreSQL: install the aws_lambda extension, then SELECT aws_lambda.invoke(...), often from a trigger. The default invocation_type is RequestResponse (synchronous): the SQL statement waits for the function’s answer. Use 'Event' (asynchronous) when you do not need the result, so the insert is not held up.
• Aurora MySQL: the native functions lambda_sync and lambda_async (the old mysql.lambda_async stored procedure is deprecated and gone in version 3).

Two things are always required:

1. A network path from the database to the Lambda service. A publicly accessible instance in a public subnet goes out by itself (its security group must allow outbound 443). A private one needs a NAT gateway or a VPC interface endpoint for Lambda. RDS for PostgreSQL behind such an endpoint also needs the parameter rds.custom_dns_resolution = 1 (in a custom parameter group, then a reboot) before it uses the endpoint’s private DNS.
2. Permission. An IAM role that trusts rds.amazonaws.com and allows lambda:InvokeFunction, attached to the instance (aws rds add-role-to-db-instance --feature-name Lambda) or to the Aurora cluster. A Lambda resource-based policy is only needed when the function is in another account.

Typical uses: a welcome or confirmation email when a row is inserted, pushing a changed price to a search index, calling an external fraud check. Keep the work small and asynchronous; heavy logic in triggers makes every insert slower and more fragile.

#RDS event notifications

RDS event notifications tell you about the database resource itself, not about your rows: an instance was created, stopped, started, rebooted, failed over, ran low on storage, a backup or snapshot started or finished, a parameter group changed. You subscribe to event categories for these source types: DB instance, DB cluster, DB snapshot, DB cluster snapshot, DB parameter group, DB security group, RDS Proxy, custom engine version.

• Destinations: an SNS topic (email, SMS, HTTPS, SQS, Lambda), through an event subscription; and EventBridge, which receives every RDS event by default with no subscription, where rules can filter and send them to Lambda, SQS, Step Functions and more (chapter 19, lesson 19.5).
• Delivery is near real time: up to five minutes, and the order is not guaranteed.
• Each event has an ID, for example RDS-EVENT-0049, “A Multi-AZ failover has completed.”

#Capturing data changes elsewhere

If what you really want is “a stream of every change in the data” for analytics or other services, there are tools built for that, covered in their own lessons:

• DynamoDB Streams or Kinesis Data Streams for DynamoDB → Lambda (lesson 15.5): the serverless default for item-level changes.
• Zero-ETL integrations (Aurora MySQL and Aurora PostgreSQL, plus RDS for MySQL, RDS for PostgreSQL and RDS for Oracle) replicate changes to Amazon Redshift in near real time without pipelines (lesson 17.2).
• AWS DMS with change data capture (CDC) streams changes to Kinesis, S3 or another database (lesson 23.3).

#What goes wrong

#Lambda networking in system design

• Default to no VPC. Attach a function only when it needs a private resource or a fixed outbound IP. Fewer moving parts, no NAT bill.
• When attached: private subnets in at least two AZs, one security group per role of function (so they share ENIs), the database allowing that group by reference.
• Plan the way out on day one: gateway endpoints for S3 and DynamoDB in every VPC (they are free), interface endpoints for the services you call often, a NAT gateway per AZ only if you need the internet.
• Relational database + Lambda = RDS Proxy (or the Data API, or DynamoDB). Combine with reserved concurrency to set a hard upper bound.
• Secrets: IAM authentication to the proxy, the database secret in Secrets Manager with rotation (chapter 21), reached through an interface endpoint.
• Database-driven events: invoke Lambda asynchronously from SQL for small per-row actions; RDS event notifications through EventBridge for operations; DynamoDB Streams, zero-ETL or DMS for data pipelines.
• Govern it: SCPs with lambda:VpcIds where every function must be inside a VPC, and a deny on EC2 actions with lambda:SourceFunctionArn for least privilege.

#Try it yourself

The order function in ap-southeast-1, account 111122223333. IDs are made up and outputs shortened.

# 1. Give the execution role the VPC permissions, then attach the function to two private subnets
aws iam attach-role-policy --role-name order-fn-role-example \
  --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole

aws lambda update-function-configuration --function-name order-example \
  --vpc-config SubnetIds=subnet-0a1b2c3d4e5f60001,subnet-0a1b2c3d4e5f60002,SecurityGroupIds=sg-0lambda1234example
aws lambda get-function-configuration --function-name order-example \
  --query '{state:State,update:LastUpdateStatus,vpc:VpcConfig.VpcId}'
#   { "state": "Active", "update": "InProgress", "vpc": "vpc-0abc1234def567890" }   # ENI being prepared
#   ...a minute later: "update": "Successful"

# 2. See the Hyperplane ENIs (one per subnet + security group pair)
aws ec2 describe-network-interfaces --filters Name=interface-type,Values=lambda \
  --query 'NetworkInterfaces[].{id:NetworkInterfaceId,ip:PrivateIpAddress,subnet:SubnetId}'
#   [ { "id": "eni-0f1e2d3c4b5a60001", "ip": "10.0.2.15", "subnet": "subnet-0a1b2c3d4e5f60001" },
#     { "id": "eni-0f1e2d3c4b5a60002", "ip": "10.0.3.22", "subnet": "subnet-0a1b2c3d4e5f60002" } ]

# 3. Let the database accept the function's security group
aws ec2 authorize-security-group-ingress --group-id sg-0db12345example \
  --protocol tcp --port 5432 --source-group sg-0lambda1234example

# 4. Private paths to AWS services: a free gateway endpoint for DynamoDB, an interface endpoint for Secrets Manager
aws ec2 create-vpc-endpoint --vpc-id vpc-0abc1234def567890 \
  --service-name com.amazonaws.ap-southeast-1.dynamodb --route-table-ids rtb-0private1example
aws ec2 create-vpc-endpoint --vpc-id vpc-0abc1234def567890 --vpc-endpoint-type Interface \
  --service-name com.amazonaws.ap-southeast-1.secretsmanager --private-dns-enabled \
  --subnet-ids subnet-0a1b2c3d4e5f60001 subnet-0a1b2c3d4e5f60002 --security-group-ids sg-0endpoints1example

# 5. The internet through a NAT gateway (one per AZ in production)
aws ec2 create-nat-gateway --subnet-id subnet-0public1example --allocation-id eipalloc-0abc1234example
aws ec2 create-route --route-table-id rtb-0private1example \
  --destination-cidr-block 0.0.0.0/0 --nat-gateway-id nat-0abc1234example

# 6. An RDS Proxy in front of the orders database, IAM auth required, secret in Secrets Manager
aws rds create-db-proxy --db-proxy-name orders-proxy-example --engine-family POSTGRESQL \
  --auth '[{"AuthScheme":"SECRETS","SecretArn":"arn:aws:secretsmanager:ap-southeast-1:111122223333:secret:orders-db-example-AbCdEf","IAMAuth":"REQUIRED"}]' \
  --role-arn arn:aws:iam::111122223333:role/orders-proxy-secret-reader-example \
  --vpc-subnet-ids subnet-0a1b2c3d4e5f60001 subnet-0a1b2c3d4e5f60002 \
  --vpc-security-group-ids sg-0proxy1234example --require-tls
aws rds register-db-proxy-targets --db-proxy-name orders-proxy-example --db-instance-identifiers orders-db-example
aws rds describe-db-proxies --db-proxy-name orders-proxy-example --query 'DBProxies[0].[Status,Endpoint]'
#   [ "available", "orders-proxy-example.proxy-c1a2b3c4d5e6.ap-southeast-1.rds.amazonaws.com" ]

# 7. Let the database call Lambda (RDS for PostgreSQL): role with lambda:InvokeFunction, feature name Lambda
aws rds add-role-to-db-instance --db-instance-identifier orders-db-example --feature-name Lambda \
  --role-arn arn:aws:iam::111122223333:role/rds-invoke-welcome-example

# 8. Page the DBA on failovers
aws rds create-event-subscription --subscription-name orders-db-failover-example \
  --sns-topic-arn arn:aws:sns:ap-southeast-1:111122223333:dba-oncall-example \
  --source-type db-instance --source-ids orders-db-example --event-categories failover availability

Inside the database (psql), the trigger side of step 7:

CREATE EXTENSION IF NOT EXISTS aws_lambda CASCADE;

CREATE OR REPLACE FUNCTION notify_new_order() RETURNS trigger AS $$
BEGIN
  PERFORM aws_lambda.invoke(
    aws_commons.create_lambda_function_arn('arn:aws:lambda:ap-southeast-1:111122223333:function:welcome-email-example'),
    row_to_json(NEW)::json,
    'Event');               -- asynchronous: the INSERT does not wait for the email
  RETURN NEW;
END $$ LANGUAGE plpgsql;

CREATE TRIGGER orders_after_insert AFTER INSERT ON orders
  FOR EACH ROW EXECUTE FUNCTION notify_new_order();

Then detach the function again (--vpc-config SubnetIds=[],SecurityGroupIds=[]) and watch its ENIs disappear within about 20 minutes, unless another function shares them.

#Summary

Key takeaways

• Every Lambda function runs in a VPC owned by the Lambda service. By default it reaches the internet and public AWS endpoints, but not private resources in your VPC (RDS, ElastiCache, internal ALB, EC2).
• Attach to your VPC with subnets (2+ AZs) and security groups; the execution role needs the ENI permissions in AWSLambdaVPCAccessExecutionRole. Lambda creates a shared Hyperplane ENI per subnet + security group pair (Pending for minutes once; no extra cold start; reclaimed after 14 days idle → Inactive, first call fails).
• The database’s security group allows the port from the function’s security group. Timeouts mean network; error messages mean credentials.
• A VPC function has no internet and no public AWS endpoints, even in a public subnet (its ENI never gets a public IP). Use private subnets + a NAT gateway in a public subnet (one per AZ) for the internet, gateway endpoints (S3, DynamoDB, free) and interface endpoints (most other services) for AWS.
• Lambda + relational database = connection storm (one connection per environment). RDS Proxy pools and multiplexes connections, keeps them through failover (up to 66% faster), enforces IAM authentication with credentials in Secrets Manager, and is never public, so the function must be in the VPC. Alternatives: Data API, DynamoDB, reserved concurrency as a cap.
• RDS for PostgreSQL, Aurora PostgreSQL and Aurora MySQL can invoke Lambda from SQL (data events); they need an outbound path (public, NAT, or Lambda VPC endpoint) and an IAM role with lambda:InvokeFunction. aws_lambda.invoke is synchronous unless you pass 'Event'.
• RDS event notifications describe the resource (failover, backup, reboot), carry no data, go to SNS or EventBridge, arrive within about 5 minutes and unordered. For data changes use DynamoDB Streams, zero-ETL or DMS CDC.

Self-check: Lambda networking, RDS Proxy and database events31 questions