Course outline

22.3

VPC Flow Logs and Troubleshooting

Reading a flow log record, telling a NACL from a security group by the ACCEPT/REJECT pattern, and analysis pipelines.

20 min Reading + animations

Tip: tap or hover any dotted-underlined term to see what it means.

By the end of this lesson you will be able to

  • Explain what VPC Flow Logs capture (metadata of IP flows, never the payload), at which levels you create them (VPC, subnet, network interface, plus Transit Gateway flow logs), and which traffic they never see.
  • Read a default-format record field by field, tell inbound from outbound, and know the extra fields of a custom format (pkt-srcaddr, tcp-flags, flow-direction, traffic-path…).
  • Decide whether a security group or a network ACL dropped traffic from the ACCEPT/REJECT pattern, for connections that come in and connections the instance starts.
  • Choose a destination: CloudWatch Logs (metric filters, alarms, Logs Insights, Contributor Insights), S3 with Athena (months of history, cheapest), or Amazon Data Firehose (Splunk, OpenSearch), and set the right permissions: an IAM role for CloudWatch Logs, a bucket policy for S3.
  • Debug “my instance cannot reach the internet” with the IPv4 checklist, using flow logs as evidence, and know when Reachability Analyzer, Traffic Mirroring or GuardDuty is the better tool.

#The VPC’s call records

When the police investigate a phone, they rarely need the conversation. The operator’s call detail records are enough: which number called which, when, for how long, and whether the call connected. VPC Flow Logs are exactly that for your VPC. They record, for every network interface (ENI), which address talked to which, on which ports and protocol, how many packets and bytes, when, and whether the traffic was accepted or rejected. They never record what was said.

#Metadata only, never the payload

A flow log record holds the source and destination address, the ports, the protocol number, packet and byte counts, the start and end time and the action: ACCEPT or REJECT. It never holds the HTTP request, the SQL query, the file or the password inside the packet. That is the notes’ main exam trick: if a question needs the payload, flow logs are the wrong answer.

The action is the decision of the security groups and network ACLs on that network interface. Flow logs do not say why an application failed above the network, and they do not report routing dead ends as REJECT (the checklist section below shows what a missing route looks like instead).

#Three levels, and the ENIs of managed services

You create a flow log for one of three resources (slide p740):

LevelWhat it coversTypical use
VPCEvery network interface in the VPC, including ones created laterThe default for a production VPC
SubnetEvery network interface in that subnetA sensitive subnet, such as the database tier
Network interface (ENI)One interface, for example one EC2 instance’sDebugging one server cheaply

A flow log on a VPC or subnet monitors every ENI inside it, and an ENI launched later gets its own log stream as soon as it sends traffic. That includes the interfaces that AWS services create in your subnets: Elastic Load Balancing, RDS, ElastiCache, Redshift, WorkSpaces, NAT gateways, Transit Gateway attachments and interface VPC endpoints. You still create the flow log from the EC2/VPC side, whatever service owns the ENI.

Three more choices when you create one:

  • Traffic type: ALL, ACCEPT only, or REJECT only. REJECT only is a cheap way to watch blocked traffic; ALL is what you want for investigations.
  • Destination: CloudWatch Logs, S3 or Amazon Data Firehose (section below).
  • Record format and maximum aggregation interval (next section).

Transit Gateway Flow Logs are a separate resource type with their own fields (tgw-id, tgw-attachment-id, packets-lost-no-route, packets-lost-blackhole…), created on the transit gateway or one attachment (lesson 22.6). You can create a flow log for a peered VPC only if it is in your own account.

Flow logs are collected outside the path of the traffic, so turning them on or off never affects throughput or latency.

#Reading a flow log record

#The default format

The default format is version 2: fourteen fields, separated by spaces, always in this order (slide p742):

#FieldMeaning
1versionFormat version: 2 for the default format
2account-idThe account that owns the network interface
3interface-idThe ENI that saw the traffic
4srcaddrSource address (for outgoing traffic, the ENI’s private IPv4 or IPv6 address)
5dstaddrDestination address (for incoming traffic, the ENI’s private address)
6srcportSource port
7dstportDestination port
8protocolIANA protocol number: 6 = TCP, 17 = UDP, 1 = ICMP
9packetsPackets in this flow during the interval
10bytesBytes in this flow during the interval
11startUnix time of the first packet in the interval
12endUnix time of the last packet in the interval
13actionACCEPT or REJECT
14log-statusOK; NODATA (no traffic in the interval); SKIPDATA (records were skipped because of an internal capacity limit or error)

A field that does not apply shows -. A NODATA record has dashes in place of addresses and counts.

#A worked record

2 111122223333 eni-0a1b2c3d4e5f67890 203.0.113.50 10.0.1.25 51544 443 6 12 4310 1730448000 1730448060 ACCEPT OK

Read it left to right: version 2; account 111122223333; seen by ENI eni-0a1b2c3d4e5f67890; from 203.0.113.50 port 51544 (a browser’s ephemeral port) to 10.0.1.25 port 443; protocol 6, TCP; 12 packets, 4,310 bytes; between 08:00:00 and 08:01:00 UTC on 1 November 2024; accepted; logged normally.

The direction is not a field in the default format. You read it from where the ENI’s own private address sits: 10.0.1.25 is in dstaddr, so this is inbound to the ENI. The reply appears as a second record with the addresses and ports swapped. One TCP connection, two records, each from the point of view of the same interface.

Slide p742 lists the uses: srcaddr/dstaddr find a problem address, srcport/dstport a problem port, action tells success or failure caused by security groups or network ACLs, and the whole set feeds analytics on usage patterns or malicious behaviour, with Athena on S3 or CloudWatch Logs Insights.

#Aggregation interval and delivery delay

Flow logs are not real time. Packets of the same flow (same five-tuple on the same ENI) are added up during a capture window:

  • The maximum aggregation interval is 10 minutes by default; you can choose 1 minute for finer records (more records, so more cost).
  • On Nitro-based instances the interval is always 1 minute or less, whatever you chose.
  • After the window closes, delivery usually takes about 5 minutes to CloudWatch Logs and about 10 minutes to S3, best effort.

So the record of a problem appears several minutes after the problem. For seconds, use CloudWatch metrics or Traffic Mirroring, not flow logs.

#Custom format: extra fields

A custom format chooses which fields appear and in what order. The version of the record becomes the highest version among the fields you pick. The fields worth knowing:

Field (version)What it adds
vpc-id, subnet-id, instance-id (3)Where the ENI lives, without joining against an inventory
tcp-flags (3)Bitmask of FIN 1, SYN 2, RST 4, SYN-ACK 18 (OR-ed within the interval: 19 = SYN-ACK + FIN). Many SYN-only flows with no SYN-ACK suggest a port scan
type (3)IPv4, IPv6 or EFA
pkt-srcaddr, pkt-dstaddr (3)The original source and destination in the packet. On a NAT gateway’s ENI, srcaddr/dstaddr show the NAT’s own address; these show the instance and the real destination. Also EKS pod addresses
region, az-id, sublocation-type, sublocation-id (4)Region, AZ ID, Local Zone, Wavelength or Outpost
pkt-src-aws-service, pkt-dst-aws-service (5)Which AWS service an address belongs to (S3, DYNAMODB, EC2, ROUTE53…)
flow-direction (5)ingress or egress, so you no longer infer direction
traffic-path (5)For egress: 1 inside the VPC, 2 internet gateway or gateway endpoint, 3 virtual private gateway, 4 intra-Region peering, 5 inter-Region peering, 6 Local or Wavelength Zone, 7 gateway endpoint, 8 internet gateway
ecs-… fields (7)Cluster, service, task and container of ECS tasks in awsvpc mode
reject-reason (8)BPA when VPC Block Public Access rejected the flow, EC for encryption controls

Newer versions (9 to 11, 2025–2026) add regional NAT gateway IDs, the encryption status of a flow, EC2 and Auto Scaling group tag values, and next-hop fields. They are useful in production; the exam knows the default fields, pkt-srcaddr/pkt-dstaddr and the idea of a custom format.

#What flow logs never see

Some traffic is not logged at all:

  • Traffic to the Amazon DNS server (the Amazon-provided resolver, the VPC base address + 2, also reachable at 169.254.169.253). If you run your own DNS server, its traffic is ordinary traffic and is logged.
  • Traffic to and from 169.254.169.254, the instance metadata service.
  • Traffic to and from 169.254.169.123, the Amazon Time Sync Service.
  • DHCP traffic and ARP traffic.
  • Windows license activation traffic to Amazon’s activation servers.
  • Traffic to the reserved address of the default VPC router.
  • The source side of mirrored traffic (you see only the copies arriving at the mirror target).
  • Traffic between an endpoint network interface and a Network Load Balancer’s network interface.

Two address quirks complete the picture: traffic sent to a secondary private IPv4 address shows the primary address in dstaddr (use pkt-dstaddr), and with a NAT gateway or an EKS pod, the packet’s real addresses also live only in pkt-srcaddr/pkt-dstaddr.

Blind spots, NAT and a miner

A private instance talks to the Amazon DNS server, the metadata service and, through a NAT gateway, the internet. Some of this traffic never appears in flow logs; some appears twice.

Scenario

DNS to the Amazon resolver and metadata calls leave no record at all.

Instance10.0.2.40
Amazon DNS10.0.0.2no record
A? api.example.com

Flow log records

ENIsrcaddrdstaddrpkt-srcaddrpkt-dstaddraction
—————nothing
Step 1 of 3
1

The instance resolves a name with the Amazon-provided DNS server (the VPC base address plus two, here 10.0.0.2). The query and answer happen, yet the flow log writes nothing: traffic to the Amazon DNS server is not logged.

#Security group or network ACL?

This is the notes’ “ultimate trick” and the exam’s favourite flow log question. Everything rests on one difference from lesson 3.5: a security group is stateful (a reply to allowed traffic is allowed automatically, in either direction), a network ACL is stateless (the reply needs its own rule, usually for the ephemeral ports 1024–65535).

Look at the action field of the request and of its reply (slide p743):

The connection…Request recordReply recordCulprit
comes in (a client calls the instance)inbound REJECTnoneSecurity group or network ACL: check both
comes ininbound ACCEPToutbound REJECTNetwork ACL outbound rules
goes out (the instance is the client)outbound REJECTnoneSecurity group or network ACL (outbound rules)
goes outoutbound ACCEPTinbound REJECTNetwork ACL inbound rules

Why the “ACCEPT then REJECT” rows can only be the network ACL: the security group that accepted the request has already remembered the connection, and it has no power to block the reply. A network ACL remembers nothing; it judges the reply as a brand-new packet to port 51544, and if no rule allows 1024–65535, its final * rule denies it.

When the request itself is rejected, the pattern alone cannot separate the two. Look at the other layer: a default network ACL allows everything, so the security group is to blame; a security group that clearly allows the source points at a network ACL deny rule. Reachability Analyzer (below) names the exact rule.

Reading ACCEPT and REJECT

An e-commerce web server in a public subnet, with a flow log on its network interface. Each packet that crosses the network ACL and the security group leaves a line in the table below.

Scenario

One HTTPS connection produces two records on the ENI: the request in, the reply out.

VPC10.0.0.0/16
Public subnet10.0.1.0/24
Customer203.0.113.50
Network ACLstatelessin: 443 allowed
Web server10.0.1.25
Flow logon the ENI
:51544 → :443
Step 1 of 5
1

A customer opens the shop. Her browser picks a temporary (ephemeral) port, 51544, and connects to port 443. The network ACL is checked first, at the subnet edge: its inbound rule allows TCP 443.

Two details that sometimes confuse real investigations:

  • REJECT is also written for packets that arrive after the connection was closed (late retransmissions), so a few REJECT lines on an otherwise healthy server are normal.
  • The record is on the ENI, after the network ACL and security group have judged it. A packet the network ACL drops on the way in is still logged on the destination ENI as REJECT; you do not need a flow log “on the NACL”.

Practise on raw lines. Pick a case, click a line to label its fields, and name the culprit:

Flow log detective

Pick a case, read the raw flow log lines (default format), click a line to label its fields, then name the culprit.

Web server 10.0.1.25 in a public subnet. Customers say the shop “keeps loading” and then times out. The security group allows 443 from anywhere.

Raw records (click one to parse it)

Click a line above to see its fields.

Who is the culprit?

#Where flow logs go

Slide p744 shows the architectures. A flow log publishes to one destination; create two flow logs on the same resource if you need two (up to 250 per resource).

#CloudWatch Logs: watch and alert

The flow log writes into a log group, one log stream per ENI (lesson 19.2). From there:

  • Metric filters turn matching lines into a metric, for example rejected SSH (dstport 22, action REJECT) or any RDP attempt; a CloudWatch alarm on that metric notifies SNS (lesson 19.3).
  • Logs Insights queries the stored records; it discovers the flow log fields by name (srcAddr, dstPort, action, bytes…).
  • Contributor Insights keeps a live top-N: the “top 10 IP addresses” of slide p744, the busiest source–destination pairs, the most rejected sources (lesson 19.4).
  • Subscription filters can stream the group on to Lambda, Kinesis or Firehose.

Records arrive within minutes, which is the closest flow logs get to “live”. Storing many months in CloudWatch Logs costs more than in S3.

#S3: keep it long, query it with Athena

The flow log writes compressed log files to a bucket, under AWSLogs/<account-id>/vpcflowlogs/<region>/<year>/<month>/<day>/. Options at creation:

  • File format: plain text (default) or Apache Parquet, columnar, smaller and much cheaper to query.
  • Hive-compatible prefixes (year=2026/month=10/…) so Athena and Glue understand the partitions.
  • Hourly partitions instead of daily, so a query for one afternoon reads only those hours.

Athena then runs standard SQL straight on the files, charging by data scanned (lesson 17.1); QuickSight can chart the results. The VPC console even generates the Athena table and saved queries for you. S3 is the cheapest place for months or years of records, and lifecycle rules can move old files to colder classes.

#Amazon Data Firehose: to another tool

The flow log publishes to a Firehose stream (in the same or another account), which buffers, can transform, and delivers to Splunk, OpenSearch, an HTTP endpoint or S3, with a backup bucket for failures (lesson 13.6). Choose it when a third-party or central security tool must receive the records continuously. Amazon Data Firehose was called Kinesis Data Firehose until February 2024; the notes and the slides use the old name, and exam questions may use either.

You needDestination
Alarms on rejected traffic, top talkers, quick searches over recent daysCloudWatch Logs (metric filters, alarms, Logs Insights, Contributor Insights)
Months of history, SQL, the lowest costS3 + Athena
Splunk, OpenSearch, a partner SIEM, another account’s pipelineAmazon Data Firehose

#Permissions: a role for CloudWatch, a bucket policy for S3

Flow logs cannot write anywhere without permission (slide p745):

  • CloudWatch Logs needs an IAM role that you pass to the flow log. Its trust policy must allow the service principal vpc-flow-logs.amazonaws.com to assume it, and its permissions policy must allow logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents, logs:DescribeLogGroups and logs:DescribeLogStreams. The person creating the flow log needs iam:PassRole for that role. Adding aws:SourceAccount and aws:SourceArn conditions to the trust policy protects against the confused deputy problem.
  • S3 needs no IAM role. The bucket policy must allow the log delivery service delivery.logs.amazonaws.com to s3:PutObject (and s3:GetBucketAcl). If you own the bucket and may change its policy, AWS adds this policy automatically when you create the flow log; for a bucket in another account, its owner adds it.
  • Firehose delivery uses a service-linked role in your account (and, across accounts, a role the destination account trusts).

Three destinations, two permission models

A flow log on a VPC can publish to CloudWatch Logs, to S3 or to Amazon Data Firehose. Each destination answers a different question, and CloudWatch Logs is the only one that needs an IAM role.

Scenario

Records arrive in minutes; metric filters and Insights turn them into alarms and top-10 lists.

Flow logVPC levelIAM role ✓
CloudWatch Logslog group
Alarmmetric filter
SNSon-call phone
S3 bucketParquet, hourly
AthenaSQL
QuickSightdashboard
records
Step 1 of 3
1

The flow log assumes its IAM role, which trusts vpc-flow-logs.amazonaws.com and allows logs:CreateLogGroup, CreateLogStream, PutLogEvents and the Describe calls. It writes into the log group, one log stream per network interface.

#Debugging a VPC: the checklist and the evidence

Lesson 3.3 gave the checklist for “my instance cannot reach the internet”. Here is the same list with the evidence a flow log gives for each fault, because the evidence is what tells you which item to fix.

Public subnet:

  1. Public IPv4 address or Elastic IP on the instance.
  2. Internet gateway attached to the VPC.
  3. Route table of the subnet sends 0.0.0.0/0 to the IGW.
  4. Security group allows the traffic (inbound for the service port; outbound is open by default).
  5. Network ACL allows the traffic both ways, especially the ephemeral ports.

Private subnet:

  1. A NAT gateway exists in a public subnet, and that public subnet’s route table sends 0.0.0.0/0 to the IGW.
  2. The private subnet’s route table sends 0.0.0.0/0 to the NAT gateway.
  3. Then the same security group and network ACL checks.
FaultWhat the flow log showsFix
No public IP (public subnet)Outbound ACCEPT with a few packets (SYN retries), no inbound line, no REJECTElastic IP, or auto-assign public IPv4
No route 0.0.0.0/0 → igw-…ACCEPT lines only, nothing back; a route is not a firewallAdd the route to the subnet’s own route table
No IGW attachedSame as aboveAttach the IGW, then add the route
Security group blocks the portInbound REJECT, no reply lineAdd the inbound rule
Network ACL without ephemeral portsInbound ACCEPT, outbound REJECT (or outbound ACCEPT, inbound REJECT for calls the instance makes)Allow 1024–65535 in the right direction
Private route to NAT missingOutbound ACCEPT on the instance, nothing on the NAT gateway’s ENI0.0.0.0/0 → nat-… in the private route table
NAT gateway’s subnet has no IGW routeRecords on the NAT ENI going out, none coming backPut the NAT gateway in a real public subnet

The pattern to remember: REJECT means a firewall; ACCEPT with silence means routing or addressing.

Debug the VPC

A web server in a public subnet, an app server in a private subnet behind a NAT gateway. Each scenario breaks one item of the checklist and shows what the flow log says about it.

Scenario

Public IP, IGW attached, route to the IGW, security group, network ACL both ways.

VPC10.0.0.0/16
Public10.0.1.0/24
Private10.0.2.0/24
Internet
IGWattached ✓
Web server10.0.1.25public IP ✓
NAT gateway
App server10.0.2.40
:443
Step 1 of 3
1

Checks 1 and 2: the web server has a public IPv4 address (or an Elastic IP), and an internet gateway is attached to the VPC. Without both, no internet traffic can reach it.

#Other tools for the same question

Flow logs record what did happen. Three neighbours answer different questions:

  • VPC Reachability Analyzer analyses the configuration between a source and a destination (instance, ENI, IGW, VPN gateway, peering connection, transit gateway…) and reports either a reachable path or the exact component that blocks it: “security group sg-… has no rule for TCP 8080”. It sends no packets, so it works before any traffic exists and even when flow logs would show nothing. It is the checklist above, automated, at about 0.10 USD per analysis.
  • Network Access Analyzer checks intent across a whole network: “list every path from the internet to the database subnet” or “nothing in production may reach development”, and reports the paths that violate it. It is for audits, not for one broken connection.
  • Traffic Mirroring copies full packets, payload included, to an IDS or packet analyser (lesson 22.7). Use it when metadata is not enough.

GuardDuty analyses VPC flow log data too, but from its own independent feed: you do not have to enable flow logs for it, and enabling them changes nothing for GuardDuty (lesson 21.7). Your own flow logs are for your own investigations and audits.

#What goes wrong

  • No data in CloudWatch Logs: the IAM role lacks logs:PutLogEvents/CreateLogStream, or its trust policy does not name vpc-flow-logs.amazonaws.com. describe-flow-logs shows DeliverLogsStatus: FAILED with “Access error”.
  • “Flow log created but empty” after two minutes: nothing is wrong yet. The first records appear only after traffic, one aggregation interval and the delivery delay: 10 to 20 minutes is normal.
  • Creation to S3 fails: the bucket is in another account and its owner has not added the delivery.logs.amazonaws.com bucket policy, or you lack PutBucketPolicy on your own bucket. Note that the automatic policy replaces an existing bucket policy.
  • Needed a field you did not include: flow logs cannot be edited. Delete and recreate with a custom format; the past stays without that field.
  • NAT gateway logs show only the NAT’s own address: add pkt-srcaddr and pkt-dstaddr to see the instance and the real destination.
  • Assuming no line means no traffic: DNS to the Amazon resolver, metadata, Time Sync, DHCP and ARP are never logged.
  • Blaming a security group for a dropped reply: impossible; a dropped reply after an ACCEPT is the network ACL.
  • A routing fault hunted in REJECT lines: routes, missing public IPs and detached gateways leave ACCEPT and silence, not REJECT. Use Reachability Analyzer.
  • Huge bill from a VPC-wide ALL flow log at 1-minute interval into CloudWatch Logs: send bulk history to S3 in Parquet, keep CloudWatch Logs for the few things you alarm on, or log REJECT only for watching.
  • Many REJECT lines on a healthy server: late packets after a connection closed are rejected and logged. Look for patterns, not single lines.
  • Expecting payload, URLs or SQL: flow logs never have them. WAF logs, application logs or Traffic Mirroring do.

#Flow logs in system design

Patterns to remember:

  • Every production VPC has a flow log, created by the account baseline (CloudFormation stack sets or Control Tower customisations), with the AWS Config managed rule vpc-flow-logs-enabled reporting any VPC without one (lesson 19.7).
  • Central log archive: every account’s flow logs go to one S3 bucket in a log-archive account (Parquet, hourly partitions), queried with Athena by the security team. Bucket policies allow the log delivery service from the member accounts only, and S3 Object Lock or a deny-delete policy keeps the evidence safe.
  • Two flow logs, two jobs: a REJECT-only flow log into CloudWatch Logs for metric filters and alarms, and an ALL flow log into S3 for history.
  • SIEM feed: flow logs through Firehose into Splunk or OpenSearch, next to WAF logs and CloudTrail (lesson 13.6).
  • Troubleshooting toolkit: Reachability Analyzer before traffic exists, flow logs after, Traffic Mirroring when the payload matters, GuardDuty for threats.
  • Blocking what you find: a top talker found in flow logs is blocked with a network ACL deny rule, a WAF rule, or Network Firewall, depending on where the traffic enters (lesson 24.3 compares them).
  • Cost awareness: flow logs themselves are charged as vended logs (in us-east-1 about 0.50 USD per GB into CloudWatch Logs and less into S3 or Firehose, with tiered discounts at high volume; check the CloudWatch pricing page, prices checked November 2026), plus storage and Athena scans. The traffic they reveal is often the bigger saving: flow logs are how teams discover that NAT gateway or cross-AZ traffic dominates the bill (lesson 22.8).

#Try it yourself

These commands use the AWS CLI v2 in ap-southeast-1 with made-up IDs. Flow logs cost a little per GB; delete what you create.

# 1. The IAM role for CloudWatch Logs delivery: trust the flow logs service, allow the logs actions
cat > trust.json <<'EOF'
{ "Version": "2012-10-17",
  "Statement": [ { "Effect": "Allow",
    "Principal": { "Service": "vpc-flow-logs.amazonaws.com" },
    "Action": "sts:AssumeRole",
    "Condition": { "StringEquals": { "aws:SourceAccount": "111122223333" } } } ] }
EOF
cat > perms.json <<'EOF'
{ "Version": "2012-10-17",
  "Statement": [ { "Effect": "Allow",
    "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents",
                "logs:DescribeLogGroups", "logs:DescribeLogStreams" ],
    "Resource": "*" } ] }
EOF
aws iam create-role --role-name flow-logs-to-cwl --assume-role-policy-document file://trust.json
aws iam put-role-policy --role-name flow-logs-to-cwl --policy-name publish --policy-document file://perms.json

# 2. A REJECT-only flow log on the VPC into CloudWatch Logs, 1-minute interval
aws ec2 create-flow-logs --resource-type VPC --resource-ids vpc-0a1b2c3d4e5f67890 \
  --traffic-type REJECT --log-destination-type cloud-watch-logs \
  --log-group-name /vpc/shop-rejects \
  --deliver-logs-permission-arn arn:aws:iam::111122223333:role/flow-logs-to-cwl \
  --max-aggregation-interval 60
# { "ClientToken": "...", "FlowLogIds": [ "fl-0123456789abcdef0" ], "Unsuccessful": [] }

# 3. Is it delivering?
aws ec2 describe-flow-logs --flow-log-ids fl-0123456789abcdef0 \
  --query 'FlowLogs[].[FlowLogStatus,DeliverLogsStatus,DeliverLogsErrorMessage]'
# [ [ "ACTIVE", "SUCCESS", null ] ]
# with a role missing PutLogEvents:  [ [ "ACTIVE", "FAILED", "Access error" ] ]

# 4. ALL traffic into S3: custom format, Parquet, Hive-compatible hourly partitions (no role needed)
aws ec2 create-flow-logs --resource-type VPC --resource-ids vpc-0a1b2c3d4e5f67890 \
  --traffic-type ALL --log-destination-type s3 \
  --log-destination arn:aws:s3:::shop-flow-logs-example/vpc/ \
  --log-format '${version} ${vpc-id} ${subnet-id} ${instance-id} ${interface-id} ${srcaddr} ${dstaddr} ${srcport} ${dstport} ${protocol} ${tcp-flags} ${packets} ${bytes} ${start} ${end} ${action} ${log-status} ${pkt-srcaddr} ${pkt-dstaddr} ${flow-direction} ${traffic-path}' \
  --destination-options FileFormat=parquet,HiveCompatiblePartitions=true,PerHourPartition=true
aws s3api get-bucket-policy --bucket shop-flow-logs-example --query Policy --output text
# ... "Principal":{"Service":"delivery.logs.amazonaws.com"},"Action":"s3:PutObject" ...  (added automatically)

Watch a metric filter and an Insights query work on the CloudWatch copy:

# 5. Count rejected SSH attempts (space-delimited pattern over the 14 default fields) and alarm on them
aws logs put-metric-filter --log-group-name /vpc/shop-rejects --filter-name ssh-rejects \
  --filter-pattern '[version, account, eni, src, dst, srcport, dstport="22", protocol="6", packets, bytes, start, end, action="REJECT", status]' \
  --metric-transformations metricName=SshRejects,metricNamespace=Shop/Network,metricValue=1
aws cloudwatch put-metric-alarm --alarm-name ssh-scan --namespace Shop/Network --metric-name SshRejects \
  --statistic Sum --period 300 --evaluation-periods 1 --threshold 100 \
  --comparison-operator GreaterThanThreshold \
  --alarm-actions arn:aws:sns:ap-southeast-1:111122223333:netops-oncall

# 6. Top rejected sources in the last hour with Logs Insights
aws logs start-query --log-group-name /vpc/shop-rejects \
  --start-time $(date -d '-1 hour' +%s) --end-time $(date +%s) \
  --query-string 'filter action = "REJECT" | stats count(*) as rejects by srcAddr, dstPort | sort rejects desc | limit 10'
# { "queryId": "a1b2c3d4-5678-90ab-cdef-EXAMPLE33333" }
aws logs get-query-results --query-id a1b2c3d4-5678-90ab-cdef-EXAMPLE33333
# srcAddr 203.0.113.66   dstPort 22    rejects 4812
# srcAddr 198.51.100.9   dstPort 3389  rejects 1290

Query months of history in S3 with Athena (run in the Athena console or with aws athena start-query-execution):

-- 7. A table over the Parquet files (field names use underscores in Parquet)
CREATE EXTERNAL TABLE vpc_flow_logs (
  version int, vpc_id string, subnet_id string, instance_id string, interface_id string,
  srcaddr string, dstaddr string, srcport int, dstport int, protocol bigint, tcp_flags int,
  packets bigint, bytes bigint, start bigint, `end` bigint, action string, log_status string,
  pkt_srcaddr string, pkt_dstaddr string, flow_direction string, traffic_path int)
PARTITIONED BY (`aws-account-id` string, `aws-service` string, `aws-region` string,
  `year` string, `month` string, `day` string, `hour` string)
STORED AS PARQUET
LOCATION 's3://shop-flow-logs-example/vpc/AWSLogs/';
MSCK REPAIR TABLE vpc_flow_logs;

-- 8. The auditor's question: who reached MySQL last quarter, and how much data moved?
SELECT srcaddr, dstaddr, count(*) AS records, sum(bytes) AS total_bytes
FROM vpc_flow_logs
WHERE year = '2026' AND month IN ('07', '08', '09')
  AND dstport = 3306 AND action = 'ACCEPT'
GROUP BY srcaddr, dstaddr
ORDER BY total_bytes DESC;
-- srcaddr      dstaddr     records  total_bytes
-- 10.0.2.40    10.0.3.15   12960    88412339201
-- 10.0.2.41    10.0.3.15   12941    87990211554
-- 10.0.9.7     10.0.3.15   3        1840          <- the analytics subnet? ask why

Let Reachability Analyzer name the blocking rule for the “two subnets cannot talk” case:

# 9. Analyse the path web → app on TCP 8080 without sending a packet
aws ec2 create-network-insights-path --source i-0aaa1111bbbb22223 --destination i-0ccc3333dddd44445 \
  --protocol tcp --destination-port 8080
# { "NetworkInsightsPath": { "NetworkInsightsPathId": "nip-0a1b2c3d4e5f67890", ... } }
aws ec2 start-network-insights-analysis --network-insights-path-id nip-0a1b2c3d4e5f67890
aws ec2 describe-network-insights-analyses --network-insights-path-id nip-0a1b2c3d4e5f67890 \
  --query 'NetworkInsightsAnalyses[0].[NetworkPathFound,Explanations[0].ExplanationCode,Explanations[0].SecurityGroup.Id]'
# [ false, "ENI_SG_RULES_MISMATCH", "sg-0app1234567890abc" ]

# 10. Clean up
aws ec2 delete-flow-logs --flow-log-ids fl-0123456789abcdef0 fl-0fedcba9876543210

From an instance, curl -m 5 https://example.com that times out, followed a few minutes later by the matching records in Logs Insights, is the quickest way to feel the delay between an event and its flow log.

#Summary

Key takeaways

  • VPC Flow Logs record metadata of IP flows on every ENI (addresses, ports, protocol, packets, bytes, time, ACCEPT/REJECT), never the payload. Levels: VPC, subnet, ENI; managed-service ENIs (ELB, RDS, ElastiCache, Redshift, WorkSpaces, NAT gateway, Transit Gateway) included; Transit Gateway Flow Logs are their own type. No effect on performance.
  • Default format v2: version, account-id, interface-id, srcaddr, dstaddr, srcport, dstport, protocol, packets, bytes, start, end, action, log-status. Direction = where the ENI’s own address sits. Custom format adds vpc-id, subnet-id, instance-id, tcp-flags, pkt-srcaddr/pkt-dstaddr (NAT gateway), flow-direction, traffic-path….
  • Not real time: aggregation 10 minutes default or 1 minute (always ≤ 1 minute on Nitro), then ~5 minutes to CloudWatch Logs, ~10 to S3. A flow log cannot be edited: delete and recreate. Nothing is back-filled.
  • Not logged: Amazon DNS server, 169.254.169.254 metadata, Time Sync, DHCP, ARP, Windows activation, default router reserved IP, mirrored source traffic.
  • SG vs NACL: inbound REJECT → SG or NACL; inbound ACCEPT + outbound REJECT → NACL; outbound ACCEPT + inbound REJECT → NACL. A security group never drops a reply. ACCEPT and silence → routing or addressing.
  • Destinations: CloudWatch Logs (metric filters, alarms, Logs Insights, Contributor Insights top-10; needs an IAM role trusted by vpc-flow-logs.amazonaws.com with logs:CreateLogGroup/CreateLogStream/PutLogEvents); S3 (no role, bucket policy for delivery.logs.amazonaws.com, Parquet + hourly partitions, Athena = months of SQL, cheapest); Amazon Data Firehose (formerly Kinesis Data Firehose) → Splunk, OpenSearch.
  • Checklist: public IP, IGW, route to IGW, SG, NACL both ways; private: NAT in a public subnet that routes to the IGW, route to NAT. Reachability Analyzer names the blocking configuration without packets; Traffic Mirroring for payload; GuardDuty reads flow data from its own feed.