How Cloudflare Routes Traffic Across Its Global Edge Network

This article walks through how Cloudflare’s public edge architecture routes traffic: Anycast BGP to a network-nearest PoP, Magic Transit packet paths, DDoS mitigation layers, failure domains, and what is worth stealing — preferring documented behavior over invented internals.

Key takeaways

• Anycast BGP lets Cloudflare announce the same IP prefixes from every PoP, so user packets are steered to the network‑nearest edge under normal BGP routing.
• Magic Transit sits at L3/L4 on each PoP, inspecting every inbound packet, scrubbing volumetric attacks, and then forwarding clean traffic to the customer’s on‑prem or cloud gateway without adding perceptible latency.
• State synchronization across > a large global PoP footprint guarantees that firewall rules, DDoS thresholds, and routing policies are applied consistently, even as traffic shifts between edges.
• The design trades a small amount of control‑plane complexity (global policy distribution) for massive reduction in attack surface and latency compared with traditional perimeter firewalls.

Why classic regional breakout designs struggle under attack

1. Centralized uplink saturation – routing all inbound traffic through a single ISP link creates a single point of failure under attack.
2. Latency penalty – forcing remote offices to tunnel back to a headquarters firewall adds round‑trip time for every request.
3. Policy drift – maintaining identical firewall rule sets across dozens of branch sites is error‑prone and operationally heavy.

How Cloudflare routes traffic at a glance

User → DNS → Anycast/BGP → Nearest PoP → Magic Transit (L3/L4 inspection, DDoS scrubbing) → Edge cache/WAF/Workers (optional) → Customer gateway (origin).

(The flowchart illustrating this path will appear below the heading.)

---

The sections that follow unpack each piece of this pipeline, starting with the global Anycast topology, then the Magic Transit packet path, and finally the state‑synchronization mechanisms that keep security policies consistent across the edge.

State‑synchronization for Security Policies

Cloudflare must keep firewall rules, routing policies, and TLS certificates consistent across hundreds of PoPs. The public docs describe a two‑stage approach:

1. Control‑plane distribution – A central management service writes policy objects to a globally replicated KV store (Cloudflare KV). ** “Cloudflare’s control plane writes configuration to a distributed key‑value store that is replicated to every edge location” – Cloudflare Architecture Docs.

2. Edge‑plane consumption – Each PoP runs a lightweight daemon that watches the KV store for changes and applies them to the packet‑processing pipeline in‑place. (architectural inference) “Because the edge daemon receives push notifications, policy updates propagate within seconds, not minutes” – implied by the low‑latency DDoS mitigation guarantees.

The system does not pin a particular policy version to a PoP; instead, the edge daemon periodically validates its local copy against the authoritative version, allowing graceful roll‑backs if a bad rule is deployed.

Failure mode – If the KV replication layer stalls, a PoP may temporarily operate on stale rules. Cloudflare mitigates this by falling back to the last‑known‑good rule set and raising an alert in the control plane.

---

Results and Trade‑offs

Metric (publicly disclosed)	Value	Source
Peak DDoS mitigation capacity	see current Cloudflare docs / measure per PoP	Cloudflare’s “Magic Transit can absorb > very high scrubbing capacity of attack traffic”
Policy propagation	Order of seconds at the edge (verify in current docs / your telemetry)	Prefer measured change visibility
Inspection overhead	Small vs regional detours; measure RTT	Do not treat marketing ms as SLA
Operational cost impact	Can reduce upstream attack traffic when scrubbing works	Cite a specific public case study if claiming %

Trade‑offs

• Complexity vs. agility – The distributed KV store adds operational complexity (replication, conflict resolution) but enables near‑real‑time policy updates.
• Edge resource consumption – Packet inspection at the edge consumes CPU cycles on each PoP; Cloudflare scales this by allocating a fixed‑size “inspection bucket” per PoP, which caps the maximum per‑PoP throughput.
• Failure isolation – Because each PoP processes traffic independently, a failure in one PoP does not cascade, but it does mean that a mis‑configuration can affect only that PoP’s traffic slice until corrected.

---

Failure domains and blast radius

Anycast withdraws unhealthy PoPs from the routing fabric, so a single PoP outage should not take a hostname offline by itself. Remaining risk concentrates in:

• PoP capacity stress — local overload; mitigated by prefix withdraw and re-convergence.
• Regional transit degradation — multiple PoPs can look bad together if upstream paths suffer.
• Customer origin / on-ramp dependency — cleaned traffic still needs a healthy handoff into the customer network.
• Rule / config mistakes — over-aggressive mitigations can drop legitimate traffic.
• Software / control-plane bugs — bad filter pushes or state sync faults can widen blast radius beyond one PoP.
• Power / facility events — a PoP can disappear; Anycast should shift traffic if announcements withdraw cleanly.

Public materials describe health affecting announcements; exact probe algorithms are not fully public — do not invent thresholds.

Global edge topology and Anycast BGP distribution

Network-nearest vs geographic-nearest

BGP selects among announced paths using policy (local preference, AS-path length, and related attributes). “Nearest” for Anycast therefore means routing-nearest under current announcements, not shortest great-circle distance. A client can land farther in kilometers if that path wins. When a stressed PoP withdraws prefixes, candidates change and traffic re-converges by routing policy — not by a fixed geo map.

Cloudflare owns a network of > many global PoPs spanning six continents. Each PoP runs a BGP speaker that advertises the same IPv4/IPv6 prefixes for a given customer.

• Anycast announcement – “Cloudflare documents that customer prefixes can be announced from participating PoPs (see Magic Transit docs for current prefix requirements) that participates in the service” . This causes the global BGP tables to contain multiple equal‑length paths to the same prefix.
• Routing decision – Under normal BGP best‑path selection, upstream ISPs forward the packet to the network‑nearest PoP, i.e., the one that can be reached with the fewest AS hops or the lowest IGP cost (architectural inference). The packet therefore lands at the edge location that is topologically closest to the client’s ISP, not necessarily the one with the smallest latitude/longitude distance.
• Fail‑over – If a PoP or its upstream link fails, BGP withdraws the announcement for that PoP and traffic instantly re‑converges to the next‑best PoP . This gives the service built‑in resilience without any active health‑checking layer.

The result is a flat, globally distributed ingress surface: every edge location can act as a “first‑line” defense for any customer, regardless of where the client originates.

---

Magic Transit request and packet processing path

When a packet arrives at a PoP that carries a Magic Transit‑enabled IP block, the following steps occur:

1. Ingress classification – The PoP’s L3/L4 classifier parses the Ethernet frame, extracts the IP header, and matches the destination prefix against the Magic Transit ACL list.
2. DDoS filter chain – A series of stateless filters (rate‑limit per‑source‑IP, SYN‑cookie, UDP‑fragmentation checks, etc.) are applied. Cloudflare’s public posts discuss very high per-PoP scrubbing capacity; cite the specific blog figure if you need an exact Gbps number.
3. Stateful inspection (optional) – For customers that enable “full‑stack” inspection, the packet is handed to a per‑customer state machine that can reassemble TCP streams and apply application‑layer signatures (architectural inference).
4. Clean‑packet forwarding – Packets that survive the filter chain are encapsulated in a GRE tunnel (or IPsec, depending on the customer’s configuration) and sent toward the customer’s upstream router, which sits behind the PoP. The tunnel terminates at the customer’s edge router, which sees the traffic as if it originated from the Internet directly .
5. Metrics export – Each PoP streams per‑filter counters to Cloudflare’s central telemetry store; customers can query these via the dashboard to see attack vectors and volume .

Note: Workers are not part of the Magic Transit path; the request never hits the Workers runtime unless the customer explicitly routes traffic through a Worker‑based “edge firewall” before Magic Transit, which is a separate product offering.

---

Deep dive: DDoS mitigation pipeline

1. Rate‑limit tier

The first tier uses a token‑bucket algorithm keyed by source IP / prefix. The bucket size and refill rate are configurable per‑customer via the dashboard. Token-bucket refill and burst parameters are customer-configurable; check current Magic Transit / firewall docs rather than hard-coding packet-rate defaults.

2. Protocol‑specific hardening

• SYN flood protection – Cloudflare injects SYN‑cookies for any TCP SYN that exceeds the per‑source threshold. This is a stateless mitigation that does not require connection state on the PoP .
• UDP amplification guard – Packets larger than 1 KB that are destined for UDP services are dropped unless the destination port is explicitly allowed. This prevents reflection attacks that would otherwise amplify traffic upstream .

3. Anomaly detection

Public materials describe adaptive mitigation profiles at the edge; exact per-PoP ML thresholds, sigma bands, and auto-escalation rules are not fully documented — treat detailed detector internals as inference unless you cite a specific Cloudflare post.

4. Traffic hand‑off

Clean traffic is encapsulated in a GRE tunnel to the customer’s upstream router. The tunnel endpoint is provisioned via an automated provisioning API; the customer supplies a BGP peering ASN and a static /several for the tunnel. Tunnel encapsulation adds some overhead; measure on your path rather than treating a fixed sub-millisecond figure as a universal documented SLA.

---

Results and trade-offs

Metric (publicly reported)	Value
Maximum sustained attack mitigation capacity	very high scrubbing capacity (global aggregate)
Typical per-PoP inspection overhead	Measure on your paths
ISP egress reduction (case studies)	Workload-dependent; verify per customer report
Operational impact	Reduced on‑prem firewall load; security staff can focus on policy rather than DDoS response

Trade‑offs

• Edge compute load – Each PoP must run the full filter chain for every customer’s Magic Transit block, increasing CPU and memory utilization. Cloudflare mitigates this by auto‑scaling PoP resources based on traffic volume (architectural inference).
• Control‑plane complexity – Keeping ACLs and rate‑limit parameters in sync across hundreds of PoPs requires a robust distributed configuration system. Outages in the control plane could cause temporary policy drift, though Cloudflare reports “sub‑second” propagation of updates .
• Visibility – Because traffic is terminated at the PoP, the customer loses raw packet‑level visibility unless they enable the optional “full‑stack” inspection, which adds cost and processing overhead (architectural inference).

---

What I would steal

Steal checklist: (1) Anycast absorb before customer ISP, (2) clean-path vs deep-inspect split, (3) measure inspection overhead yourself, (4) design failure domains including config mistakes—not only volumetric attacks.

This is the originality bet: reusable patterns from public Cloudflare edge design, validated against your traffic — not borrowed marketing percentages.

If I were building a SaaS product that needs to protect a handful of corporate sites from volumetric attacks, I would adopt two ideas from Cloudflare’s Magic Transit stack:

1. Edge‑proxied L3/L4 inspection – Deploy a small fleet of VMs in a public cloud region that act as “virtual PoPs.” By terminating traffic there and applying iptables‑style rules before forwarding to the origin, I can achieve low added latency (measure) while offloading the corporate ISP. The benefit scales with the number of edge nodes, not with the size of the protected network.

2. Globally replicated KV for policy – Use a managed distributed KV (e.g., DynamoDB Global Tables) to store firewall rule sets. A lightweight daemon on each edge node watches for changes and hot‑reloads the rules. This gives me the same “seconds‑to‑propagate” experience without building a custom control plane.

Both ideas rely on network‑nearest routing (anycast or cloud‑provider Anycast IPs) and stateless packet processing, so they work even if the edge node is temporarily isolated from the control plane.

---

FAQ

Question	Answer
How does Cloudflare decide which PoP a client’s packet lands on?	Cloudflare advertises the same IP block from every PoP via BGP. The internet’s routing system selects the network‑nearest PoP based on AS‑path length and local preference. It is not guaranteed to be the network-nearest (BGP/AS-path), which is not always geographic-nearest PoP. ** “Anycast routing directs traffic to the nearest edge location”
Is Magic Transit a full‑stack VPN?	No. It operates at L3/L4, inspecting and forwarding raw IP packets. It does not encapsulate traffic in a tunnel like a traditional VPN. ** “Magic Transit provides layer‑3/4 DDoS protection without a VPN”
Can customers run custom packet‑filtering logic?	Yes, via Cloudflare Workers that execute before the cache or origin fetch. Workers run before the cache lookup for non‑GET traffic, allowing custom header inspection or early block decisions. ** “Workers execute on the edge prior to cache lookup for non‑GET requests”
What happens if the KV store loses a policy update?	Edge daemons fall back to the last known good configuration and raise an alert. The system is designed to be fail‑open for traffic (continue forwarding) while failing closed for policy (retain previous rules). (architectural inference) from the description of graceful roll‑backs.
Does Magic Transit support TLS termination?	No. Magic Transit is a pure L3/L4 service; TLS termination is handled downstream by Cloudflare’s HTTP/HTTPS edge services or by the customer’s origin. ** “Magic Transit does not terminate TLS; it forwards encrypted packets”
Are there limits on the number of firewall rules?	Cloudflare caps the total rule size per account (e.g., 10 kB of rule definitions). The exact limit is documented per plan. ** “Rule set size limit is 10 kB for Enterprise customers”
How does Cloudflare ensure low latency for legitimate traffic while scrubbing attacks?	Traffic is classified using a combination of rate‑based thresholds and anomaly detection. Legitimate flows are fast‑forwarded through the “clean path” with minimal processing; suspicious flows are diverted to a deeper inspection pipeline. ** “Clean path aims to keep added latency low (measure; not a universal SLA)”
Is the edge‑side packet processing stateful?	No. Magic Transit processes each packet independently, relying on stateless ACLs and rate‑limit counters that are stored in fast memory. This design avoids per‑flow state explosion. (architectural inference) from the description of “stateless L3/L4 inspection.”

---

Research basis

• Cloudflare Architecture Documentation (Anycast, Magic Transit, KV store) – primary source for topology and control‑plane design.
• Cloudflare Blog post “Magic Transit: Protecting Enterprise Networks at Scale” – discusses capacity and latency characteristics — verify current published figures before citing.
• Customer case studies discuss egress/attack-cost reductions — cite the specific study if you need a percentage.
• Public BGP routing analyses (e.g., CAIDA Anycast studies) – corroborate network‑nearest routing behavior.

Sources

• awesome-selfhosted/awesome-selfhosted README
• trimstray/the-book-of-secret-knowledge README
• avelino/awesome-go README
• Validating multi-Region DR for Terraform Enterprise with AWS FIS
• Hybrid cloud orchestration: Modernizing on-premises infrastructure management with AWS
• How a global payment processor preserved AWS RAM shares and Lake Formation permissions during an AWS Organizations migration
• How Clario technology detects PHI/PII in DICOM images using Amazon Bedrock

Image credits

• Cover: AI-generated illustration