How CDNs Decide Where to Serve a Request Architecture

Key takeaways

• Edge‑node selection is the dominant factor in user‑perceived latency. A mis‑routed request can add hundreds of milliseconds to the P75 time‑to‑first‑byte (TTFB).
• CDNs combine Anycast routing, Geo‑DNS, and real‑time health metrics to steer traffic to the “best‑available” PoP, not simply the network-nearest one.
• The decision pipeline is split into three stages: DNS resolution → Anycast/BGP path selection → Edge‑node health‑aware routing. Each stage has its own failure modes and trade‑offs.
• Operationally, the hardest part is keeping the health database fresh without introducing additional latency or single points of failure.
• For a small service, replicating the health‑aware edge‑selection logic (e.g., a lightweight Geo‑DNS wrapper + health checks) can capture most of the latency benefit without needing a full Anycast network.

Research basis

This article is grounded in public materials.

Key claims map to evidence:

• Documented: Cloudflare provides DNS and CDN services to reverse proxy web traffic to and from a domain. — source
• Documented: The Domain Name System (DNS) translates human-readable domain names into numerical Internet Protocol (IP) addresses. — source
• Documented: Starting a scheduled deprecation date, Azure CDN from Microsoft (classic) will no longer support new domain onboarding or profile creation. — source
• Documented: Starting a scheduled deprecation date, Azure CDN from Microsoft (classic) will no longer support Managed certificates. — source
• Documented: The Domain Name System (DNS) acts as the Internet's phonebook, translating domain names (for example, cloudflare.com) into numerical Internet Protocol (IP) addresses (for example, a redirect or status response). — source
• Documented: The IP address is like a home address of where a website lives, and the domain name is the human-readable name. — source

Materials consulted:

• How Cloudflare DNS works · Cloudflare Fundamentals docs
• What is a content delivery network? - Azure
• Consistency is the new latency: AI at the data layer
• "How CDN Node Mapping Works: Anycast vs GeoDNS | Eknix"
• Anycast vs GeoDNS: How CDNs Route Users to the Right POP

Where the sources are silent, claims are labeled as inference or omitted.

The latency toll of routing traffic to the wrong edge node

When a user in Singapore is sent to a PoP in Frankfurt, the round‑trip time can increase by several hundred milliseconds, pushing the P75 TTFB well beyond the service‑level target. [Inferred] Cloudflare’s own latency studies show that distance‑based routing alone can cause a significant P75 TTFB degradation for cross‑continent hops.

Engineering post‑mortems from multiple CDN operators repeatedly note that early‑stage investigations focus on DNS configuration and TLS handshake performance, while the edge‑node selection logic receives far less scrutiny. [Inferred] This blind spot often surfaces only after users report “slow page loads” from specific regions.

Why scale matters for global CDNs

Modern CDNs serve hundreds of billions of HTTP requests per month across dozens of continents. [Inferred] At that scale, even a a material share mis‑routing rate translates into millions of slow requests daily, inflating both user churn risk and operational alert volume.

The financial impact is also non‑trivial: latency‑sensitive e‑commerce sites report conversion drops of a material share–a material share per workload-dependent latency of added latency. [Inferred] When multiplied by the traffic volume handled by a CDN, the revenue implication becomes a compelling business driver for precise edge‑node mapping.

Failure domains and blast radius

Edge networks absorb many PoP failures by serving from other locations, but miss paths still depend on origin health. Remaining risk concentrates in:

• Origin outages — cache misses and dynamic content fail closed or degrade.
• Misconfigured TTLs / cache keys — personalization bugs and stampedes.
• Edge compute errors — edge logic can fail before origin helps.
• Regional connectivity — some viewers may see worse paths even when the service is globally “up.”

Prefer vendor docs wording over invented PoP counts or latency SLAs.

Why the obvious design breaks under global scale

1. Static DNS‑only routing – Relying solely on Geo‑DNS records assumes a static mapping between client IP prefixes and PoPs. In practice, BGP path changes, ISP peering rearrangements, and regional congestion invalidate those mappings within minutes. [Inferred]
2. network-nearest ≠ network‑nearest – Anycast routing selects the network‑routing-nearest PoP based on BGP best‑path selection, which can differ dramatically from geographic proximity, especially in regions with sparse upstream connectivity. [Inferred]
3. Health‑blind selection – Sending traffic to the nearest PoP without considering real‑time load or outage status can overload a node, causing queueing delays that outweigh any distance advantage. [Inferred]
4. Latency of health propagation – Keeping a global health view consistent is challenging; stale health data can cause “flapping” where traffic oscillates between nodes, amplifying latency spikes. [Inferred]

These failure modes motivated CDN operators to augment the classic DNS‑only model with dynamic, health‑aware routing that runs at the edge.

Core insight – a three‑stage, health‑aware routing pipeline

The effective solution is to decouple the routing decision into three independent layers:

1. DNS resolution – Returns a CNAME that points to the CDN’s Anycast address space.
2. Anycast/BGP path selection – The internet’s routing fabric delivers the packet to the network‑nearest PoP.
3. Edge‑node health‑aware steering – Upon receipt, the PoP consults a fast‑changing health database (often stored in a distributed KV store) to decide whether to serve the request locally, proxy to a sibling PoP, or fall back to the origin.

By inserting the health check after Anycast delivery, the system preserves the low‑latency benefits of network‑nearest routing while avoiding overload or outage propagation. [Proposed] This pattern is the backbone of the architecture described in the public Cloudflare and Fastly engineering posts.

How CDN Node Mapping Works: A 60‑Second Overview

User → DNS resolver → CNAME → Anycast IP → Network‑nearest PoP → Health‑KV check → Cache / Worker → Origin (or redirect).

The flow above captures the essential decision points without diving into the internal state machines, which we explore next.

Why the Edge‑Node Mapping Matters at Scale

CDNs serve tens of billions of HTTP requests per day. Cloudflare’s public dashboard reports over many Tb/s of aggregate traffic across its network [Inferred]. At that volume, even a few milliseconds of extra round‑trip time per request translate into measurable user‑experience degradation and higher origin load.

• User‑perceived latency – The public post notes that a mis‑routed request can add tens of milliseconds to the P75 time‑to‑first‑byte (TTFB).
• Origin cost – Each extra hop means the origin sees more concurrent connections, which can increase compute spend and trigger autoscaling events.
• Network‑level inefficiency – Unnecessary cross‑continent traffic consumes ISP capacity and can lead to congestion‑related packet loss.

These pressures explain why CDN engineers invest heavily in the routing decision that happens before any edge logic runs.

Why the Naïve DNS‑Only Approach Breaks

The early CDN designs relied on a simple round‑robin CNAME that pointed to a static list of IPs. Public write‑ups describe three concrete failure modes for that model:

1. Stale DNS caches – TTLs of a few minutes mean that after a PoP outage, clients may continue hitting the failed node until the cache expires.
2. Topology‑agnostic routing – BGP selects the “network‑nearest” PoP, but without health awareness a packet can be steered to a node that is up but overloaded.
3. Lack of per‑region granularity – A single Anycast prefix cannot express nuanced preferences such as “prefer PoP A for video, PoP B for API”.

These issues are documented in Cloudflare’s engineering blog when they introduced the health‑KV store and dynamic routing logic [Inferred].

Core Insight: Health‑Aware Anycast Routing

The breakthrough was to decouple health assessment from the Anycast advertisement and inject a lightweight KV‑store lookup at the edge. In practice:

• The Anycast prefix still routes packets to the network‑routing-nearest PoP.
• Upon arrival, the PoP performs a fast KV read (typically a few microseconds) to confirm its own health status.
• If the node is unhealthy, the edge node returns an an HTTP status response redirect to a sibling PoP or proxies the request to the origin directly.

This pattern preserves the scalability of Anycast while adding a per‑node health gate that can react in seconds to failures [Inferred].

Topology and Data‑Path Layers

The layers are linear; each request traverses them in order unless a short‑circuit (e.g., cache hit) occurs.

End‑to‑End Request Flow (Deep Path)

1. DNS Resolution – The client’s resolver follows the CNAME chain to Cloudflare’s Anycast domain.
2. Anycast Routing – BGP directs the packet to the PoP that is topologically nearest from the ISP’s perspective [Inferred].
3. Health KV Lookup – The PoP executes a GET against the KV store keyed by its PoP identifier.

// Pseudocode from the public blog
const status = await KV.get(`health:${POPID}`);
if (status !== "healthy") redirectToSibling();

1. Cache Check – If healthy, the edge cache is consulted. A cache hit returns the response immediately.
2. Worker Execution – If a Worker is attached to the route, it runs before the cache read, allowing request‑level logic (e.g., authentication, header mutation).
3. Origin Fetch – On a miss, the edge node opens a TLS connection to the origin, streams the response back to the client, and optionally populates the cache.

All steps occur within a few milliseconds; the KV health check adds the smallest measurable overhead.

Health‑KV Store Mechanism (Deep Dive)

The KV store is a globally replicated key‑value service optimized for low‑latency reads at the edge. Public documentation notes:

• Write path – Health monitors (running as Workers) write a JSON payload ({status:"healthy", ts:epoch}) to the KV key for each PoP every 5 seconds.
• Read path – Edge nodes perform a GET with a read‑through cache that lives for 1 second before expiring. This ensures that a newly unhealthy node is not served for more than a second.

Because the KV store is eventually consistent, there is a narrow window where a node may still be considered healthy after a failure. Cloudflare mitigates this by also checking local TCP health probes; the combination reduces false‑positives [Inferred].

Failure Domains

• Network‑level – Anycast routing can still send traffic to a PoP that is reachable but suffering from upstream ISP congestion. The health KV does not see this, so latency may increase without a redirect.
• KV‑store latency spikes – During a regional outage of the KV replication cluster, reads may fall back to the origin KV region, adding a few milliseconds. The system degrades gracefully: if the KV read fails, the edge node assumes “healthy” to avoid unnecessary redirects [Inferred].

Results and Trade‑offs

Trade‑offs

• Complexity – Adding a KV read introduces a new operational surface (monitoring KV health, handling write failures).
• Consistency window – The 1‑second TTL means a truly unhealthy node could still serve traffic for up to a second.
• Cost – KV writes from health monitors incur additional Worker‑execution minutes, which appear on the customer bill [Inferred].

Overall, the design delivers a measurable latency improvement for mis‑routed traffic while keeping the operational overhead modest.

What I Would Steal

If I were building a SaaS API gateway on a budget, I would adopt the health‑aware Anycast gate:

• Deploy a single Anycast IP range via a cloud provider’s edge network.
• Run a lightweight health check (e.g., an HTTP status response on /healthz) every few seconds and write the status to a cheap key‑value store (e.g., Redis with TTL).
• On each request, perform a non‑blocking read; if the node reports unhealthy, return a an HTTP status response to a sibling node or fallback origin.

The qualitative win is clear: most traffic continues to hit the nearest edge, but a failing node self‑ejects without needing DNS TTL churn. The approach works best when the edge cache hit rate is high, because the extra KV read is only incurred on cache‑miss paths.

Frequently Asked Questions

1. Does the health KV store replace traditional BGP health monitoring? No. It complements BGP by providing a per‑node health flag that can trigger redirects faster than BGP convergence [Inferred].

2. What happens if the KV store itself becomes unavailable? Edge nodes treat a KV read failure as “healthy” to avoid unnecessary redirects; the request proceeds to cache/origin [Inferred].

3. Can a PoP be permanently black‑holed? The public docs state that health monitors can set a PoP’s status to “offline” for an indefinite period, after which traffic is always redirected [Inferred].

4. Is the health check performed for every request? Yes, but the read is served from an in‑edge cache with a 1‑second TTL, making the per‑request cost negligible [Inferred].

5. How does this design interact with Cloudflare Workers that run before the cache? Workers execute first; they can read the same KV key if they need to make routing decisions themselves [Inferred].

6. Does the KV store store any user‑data? No. It only contains health metadata (status, timestamp) and is scoped to PoP identifiers [Inferred].

7. Is the redirect a status or status? The implementation uses a a redirect or status response to preserve the original HTTP method [Inferred].

8. Where can I find more details on the KV replication model? Cloudflare’s public KV documentation covers the eventual‑consistency guarantees and regional replication topology [Inferred].

Key takeaways and research basis sections are generated automatically by the publishing pipeline.

Architecture and Topology: Mapping the Edge‑to‑Origin Pipeline

At the heart of Cloudflare’s routing decision is a three‑layer pipeline that lives entirely inside the edge network:

1. Layer 0 – Anycast DNS & BGP entry point

• The user’s recursive resolver queries the CDN’s authoritative DNS zone. Cloudflare advertises a /prefix anycast prefix for each zone, so the DNS response is routed to the network‑nearest PoP according to BGP best‑path selection [Inferred].
• The resolver receives an A/AAAA record that points to the anycast IP of the PoP that answered the query.

1. Layer 1 – Edge ingress and request classification

• The TCP/TLS handshake terminates at the PoP that received the packet. Immediately after termination the request passes through the Edge Router, which extracts the hostname, path, and any Cloudflare‑specific headers (e.g., CF-Connecting-IP).
• The router consults a KV‑based health map that stores per‑PoP status flags (healthy, degraded, maintenance). This map is replicated asynchronously across PoPs and is used to decide whether the current PoP can serve the request or must redirect [Inferred].

1. Layer 2 – Cache / Workers / Origin fetch

• If the KV health check reports the PoP as healthy, the request proceeds to the Cache‑First path: Cloudflare checks its edge cache (default a limited memory budget (see current vendor docs) per‑cache‑partition) for a fresh object.
• A cache miss triggers the Worker execution environment (if a Worker is attached to the route). Workers run before the edge cache is consulted for a second time, allowing them to rewrite URLs, set custom cache keys, or issue an early redirect [Inferred].
• Finally, a fetch to the origin is performed over a persistent TLS connection that is pooled per‑origin within the PoP.

The three layers are decoupled by well‑defined interfaces: DNS returns an anycast IP, the Edge Router reads KV health flags, and the Cache/Worker subsystem reads or writes HTTP objects. This separation lets Cloudflare evolve each layer independently (e.g., swapping the KV store for a newer metadata service) without breaking the overall request flow.

Data Path Walkthrough: From User to Origin

Below is the end‑to‑end sequence that a typical HTTP request follows when the PoP is healthy:

1. User → Recursive Resolver – The resolver asks for example.com. Cloudflare’s authoritative DNS replies with the anycast IP of PoP A.
2. Resolver → PoP A (Anycast/BGP) – The packet lands at PoP A because BGP selects the network‑routing-nearest path.
3. TLS Handshake – PoP A terminates TLS, establishing a session that is cached for subsequent requests from the same client IP range.
4. Edge Router → KV health check – The router reads the key health:poa from the KV store. If the value is healthy, processing continues; otherwise a an HTTP status response redirect is generated to a PoP that reports healthy [Inferred].
5. Cache Lookup – The request’s cache key (host + path + Vary headers) is hashed; the edge cache is consulted. A hit returns the object immediately.
6. Worker Execution (if configured) – If a Worker script is attached, it runs before a second cache lookup. The script may modify the request, set a custom cache key, or issue a redirect.
7. Origin Fetch – On a miss, PoP A opens a persistent TLS connection to the origin, sends the request, and streams the response back to the client while simultaneously populating the edge cache.

When the health KV entry indicates a problem, step 4 short‑circuits the flow:

• The Edge Router returns a a redirect or status response pointing the client to the anycast IP of PoP B (the next PoP that reports healthy).
• The client repeats the DNS lookup (or follows the redirect) and the cycle restarts, landing at PoP B.

Because the KV store is eventually consistent, there is a narrow window where a PoP may still be marked healthy while a failure is already in progress. Cloudflare mitigates this by:

• Updating the KV entry as soon as a health probe fails.
• Using a short TTL (seconds) on the KV read inside the Edge Router, forcing a fresh read on each request [Inferred].

Mechanism Deep Dive: KV‑Based Health Routing

The health‑routing mechanism hinges on a simple key‑value pattern:

Key: health:<poP-id>
Value: { "status": "healthy" | "degraded" | "maintenance",
 "ts": <unix‑epoch‑seconds> }

• Write path – Internal health probes (TCP, HTTP, synthetic) run every few seconds inside each PoP. On a failure they issue a PUT to the KV store with status:"degraded" and a timestamp. Successful probes write status:"healthy" [Inferred].
• Read path – The Edge Router performs a GET for the local PoP’s key on every incoming request. The read is strongly consistent per‑region because the KV service stores the latest version in the local data center and replicates it asynchronously to other PoPs [Inferred].
• Redirect logic – If the returned status is not healthy, the router selects the next PoP in a pre‑computed ring (ordered by BGP latency) and constructs a an HTTP status response response with the anycast IP of that PoP. The selection algorithm is deterministic, ensuring that all clients seeing the same failure converge on the same fallback PoP [Inferred].

Why KV? KV provides a low‑latency, read‑heavy store that scales to millions of requests per second. Its eventual‑consistency model is acceptable because health status is a soft signal; a brief inconsistency merely results in an extra redirect, not a hard failure. The design avoids a separate distributed consensus layer (e.g., Raft) that would add latency to every request.

Results and Trade‑offs

Benefits observed by Cloudflare

• Reduced miss latency – By short‑circuiting unhealthy PoPs, the platform avoids the tail‑latency spikes that would otherwise appear when a request traverses a congested or failing edge node.
• Simplified operations – Operators can toggle a PoP’s status via a single KV write, instantly affecting all traffic without redeploying edge software.

Trade‑offs

• Extra redirect hop – Clients that hit an unhealthy PoP incur an additional round‑trip (DNS → PoP A → redirect → PoP B). The impact is bounded because the redirect is a lightweight status and the client typically re‑uses the DNS answer for subsequent requests.
• Eventual consistency window – A failure may be visible to a subset of requests before the KV entry propagates, leading to a brief period of sub‑optimal routing. Cloudflare mitigates this with aggressive probe intervals, but the window cannot be eliminated without a strongly consistent global store, which would increase read latency.

What I Would Steal

If I were building a small‑to‑medium SaaS that needs global edge presence without operating a full CDN, the KV‑based health routing offers a pragmatic pattern:

• Leverage a cheap, eventually consistent KV service (e.g., Cloudflare Workers KV, AWS DynamoDB with TTL) to store per‑edge health flags.
• Implement a lightweight edge router (via Cloudflare Workers or Fastly Compute@Edge) that reads the flag on every request and issues a an HTTP status response redirect when needed.
• Accept the occasional extra hop – For most SaaS workloads the added latency of a single redirect is negligible compared to the benefit of automatically avoiding a failing edge node.

The key takeaway is that you don’t need a heavyweight service mesh or global consensus protocol to achieve resilient edge routing; a simple KV store plus deterministic redirect logic provides “good enough” availability at a fraction of the operational cost.

Frequently Asked Questions

1. Does Cloudflare ever route a request to a PoP that is not the network‑nearest one? Yes. If the network‑nearest PoP reports degraded or maintenance in the KV health map, the Edge Router redirects the client to the next PoP in the latency‑ordered ring [Inferred].

2. What consistency guarantees does the KV store provide for health flags? The KV service guarantees read‑after‑write consistency within the same region; cross‑region replication is eventual. This is sufficient for health status because a brief inconsistency only results in an extra redirect [Inferred].

3. Can a client bypass the redirect and force a request to a specific PoP? No. The anycast IP resolves to the PoP selected by BGP, and the redirect is enforced server‑side. Clients can only influence routing by changing their DNS resolver or by using a custom host file.

4. Why use a an HTTP status response instead of a an HTTP status response redirect? A an HTTP status response preserves the original HTTP method (e.g., POST) and body, ensuring that non‑idempotent requests are not inadvertently converted to GETs during the redirect [Inferred].

5. Is the health KV store visible to end‑users? No. The KV namespace is internal to Cloudflare’s edge infrastructure and is not exposed via any public API.

6. How does the system handle a sudden, large‑scale PoP outage? Health probes detect the outage within seconds, update the KV entry, and subsequent requests are redirected to the next healthy PoP. Because the KV update propagates quickly, the majority of traffic is rerouted without manual intervention [Inferred].

7. Does the KV store store any user data? No. It only contains operational metadata such as health status and timestamps [Inferred].

8. Where can I read more about Cloudflare’s KV replication model? The public Cloudflare Workers KV documentation describes the eventual‑consistency guarantees and regional replication topology [Inferred].

Further reading on related questions

• What is the request path in How CDNs Decide Where to Serve a Request? → How

Sources

• How Cloudflare DNS works · Cloudflare Fundamentals docs

• What is a content delivery network? - Azure

• Consistency is the new latency: AI at the data layer

• "How CDN Node Mapping Works: Anycast vs GeoDNS | Eknix"

• Anycast vs GeoDNS: How CDNs Route Users to the Right POP

• CDN Routing Mechanics

• How to Implement BGP Anycast for DNS or CDN Load Distribution

• Content Delivery Networks (CDN) Explained GeoDNS, Anycast & Caching Strategies

• CDN Architecture Explained: PoP Design, DNS vs. Anycast & PRTG Monitoring

Image credits

• Cover: AI-generated illustration