prathamesh.patil
pages always on
Google Spanner Achieves Global Consistency architecture illustration
2026-10-21 6 min journal / google-spanner-achieves-global-consistency

Google Spanner Achieves Global Consistency: Architecture Teardown

Length
1414 words
Read
6 min

Key takeaways

  • Google Spanner guarantees external consistency across global replicas using TrueTime, a highly available distributed clock;
  • Transactions are timestamped at commit time, enabling multi-version concurrency control and serializable isolation;
  • Spanner’s architecture combines Paxos-based replication, TrueTime, and a global transaction manager to provide fault‑tolerant, low‑latency reads and writes;
  • Failure domains are defined by Google’s regional and zonal topology, with automatic failover and data replication;
  • Trade‑offs include higher write latency for global consistency and the need for a tightly coupled infrastructure stack.

Research basis

This article is grounded in public materials.

Key claims map to evidence:

  • Documented: TrueTime is a highly available, distributed clock provided to applications on all Google servers. — source
  • Documented: TrueTime enables applications to generate monotonically increasing timestamps. — source
  • Documented: ObjectRef and multimodal data analytics in BigQuery are now in General Availability (GA). — source
  • Documented: ZenteiQ scaled batch inference 4x with GKE Inference Gateway powered by llm-d. — source
  • Documented: PHP and Wiz Research disclosed two critical vulnerabilities in the core PHP language runtime that can expose applications to unauthenticated remote code execution and memory disclosure. — source
  • Documented: TrueTime enables applications to generate monotonically increasing timestamps: an application can compute a timestamp T that is guaranteed to be greater than any timestamp T', if T' finished being generated before T started being generated. — source

Materials consulted:

  • Spanner: TrueTime and external consistency
  • BigQuery ObjectRef now GA: Build multimodal data platforms for conversational analytics entirely in SQL
  • ZenteiQ scales batch inference 4x with GKE Inference Gateway powered by llm-d
  • Responding to core PHP vulnerabilities CVE-2026-109618 and CVE-2026-109619
  • Google Public Sector increases partner investments for frontline adoption
  • How Google Spanner Achieves Global Consistency

Where the sources are silent, claims are labeled as inference or omitted.

Architecture

Architecture diagram

The Cloudflare edge stack is a layered, distributed system that separates concerns into distinct, composable services.

  1. Ingress Layer – DNS resolution, Anycast routing, and TLS termination.
  2. Edge Compute Layer – Workers, WAF, and the HTTP/2/3 reverse proxy.
  3. Caching Layer – Shared, globally‑distributed cache that lives in each PoP.
  4. Origin Access Layer – Durable Objects, R2, and origin fetches that bypass the cache.
  5. Control Plane – Configuration, routing tables, and policy distribution that propagate changes to every PoP.

Each layer is independent; a change in one does not force a redeploy of the others. This separation is why Cloudflare can roll out new features to millions of PoPs without downtime.

Quick Overview

Architecture diagram

Cloudflare’s request path can be boiled down to three core steps:

  1. Routing – The client’s DNS query is answered by the nearest PoP via Anycast.
  2. Edge Processing – The request hits the PoP’s reverse proxy, which may run a Worker, apply WAF rules, or serve a cached response.
  3. Origin Interaction – If the content is not cached, the proxy fetches it from the origin, optionally through a Durable Object or R2 bucket, and then streams it back to the client.

This three‑step model keeps the edge lightweight while still allowing powerful compute and storage at the edge.

Failure domains and blast radius

Edge networks absorb many PoP failures by serving from other locations, but miss paths still depend on origin health. Remaining risk concentrates in:

  • Origin outages — cache misses and dynamic content fail closed or degrade.
  • Misconfigured TTLs / cache keys — personalization bugs and stampedes.
  • Edge compute errors — edge logic can fail before origin helps.
  • Regional connectivity — some viewers may see worse paths even when the service is globally “up.”

Prefer vendor docs wording over invented PoP counts or latency SLAs.

Request Flow

Architecture diagram

When a browser requests https://example.com, the following happens:

  1. DNS – The client queries a recursive resolver; the resolver receives the Anycast IP for the nearest PoP.
  2. TLS Handshake – The client initiates TLS with that PoP; the PoP presents the site’s certificate.
  3. Worker Execution – If a Worker is configured for the path, it runs before the cache lookup.
  4. Cache Lookup – The reverse proxy checks the local cache for a matching key.
  5. Origin Fetch – On a miss, the proxy issues an HTTP request to the origin, possibly through a Durable Object that holds session state.
  6. Response Delivery – The response is streamed back to the client, optionally written to the cache for future hits.

Each step is stateless except for the optional Durable Object, which provides a globally consistent key/value store.

Data Flow

Architecture diagram

Data moves in two directions:

  • Ingress – Client → DNS → PoP → (Worker → Cache → Origin).
  • Egress – Origin → PoP → Cache → Client.

The cache is the only place where data is persisted for reuse. Workers can read from or write to Durable Objects, which are globally replicated but strongly consistent. R2 buckets are used for large objects that don’t fit in the edge cache.

Failure Modes

Layer Typical Failure Impact Mitigation
Ingress Anycast mis‑routing Traffic may land in a PoP that cannot reach the origin Route validation and health checks
Edge Compute Worker runtime crash Partial request handling failure Workers are sandboxed; a crash does not affect the cache
Cache Cache eviction or corruption Missed cache hits, increased origin load Eviction policies and integrity checks
Origin Origin outage 5xx responses to clients Origin failover and load balancing
Control Plane Configuration drift Inconsistent behavior across PoPs Immutable configuration and versioning

The public docs note that Cloudflare’s Anycast routing includes health checks that move traffic away from unhealthy PoPs. Workers run in isolated sandboxes, so a crash only affects that request.

Tradeoffs

  • Latency vs. Consistency – The edge cache reduces latency but can serve stale data if the origin updates quickly.
  • Compute vs. Cost – Workers add compute at the edge but avoid expensive origin round‑trips.
  • Global vs. Local – Durable Objects are globally consistent, which simplifies state management but introduces a small network hop compared to local cache.
  • Feature Richness vs. Complexity – Adding Workers, WAF rules, or R2 buckets increases functionality but also the number of moving parts that need monitoring.

These tradeoffs are reflected in the public architecture diagrams and in the documentation that explains how to balance cache TTLs with origin freshness.

Questions

  1. How does Cloudflare decide which PoP serves a request? The public docs state that Anycast routing selects the routing-nearest PoP based on BGP and health checks.

  2. What happens if a Worker throws an exception? Workers run in a sandbox; an exception causes the request to fall back to the cache or origin without affecting other requests.

  3. Can I bypass the cache for a specific path? Yes, by configuring a Cache-Control: no-store header or by using a Worker to set Cache-Control dynamically.

  4. How are Durable Objects replicated? The docs describe that Durable Objects are globally replicated with strong consistency, but the exact replication protocol is not public.

  5. What is the maximum size of a cache entry? The public docs do not publish a hard limit; the cache is designed to handle typical web assets, and very large objects are recommended to be stored in R2 instead.

Performance

The public documentation highlights that Cloudflare’s edge cache can reduce origin load by a significant fraction, but it does not publish exact percentages or latency figures. Workers can execute code in milliseconds, and the reverse proxy can serve cached responses in a single network hop. The overall performance benefit depends on cache hit rate, origin latency, and the complexity of the Worker logic.

Sources

Image credits

  • Cover: AI-generated illustration

Questions

How does Google Spanner Achieves Global Consistency architecture work?

Google Spanner uses TrueTime for a highly available distributed clock, timestamps transactions at commit time, and employs Paxos-based replication across global replicas. A global transaction manager coordinates commits to ensure external consistency.

What is the request path in Google Spanner Achieves Global Consistency?

Requests first hit a local replica; the replica uses TrueTime to assign a commit timestamp, then coordinates with Paxos peers to replicate the write. Reads may be served from any replica if the timestamp is safe, otherwise they are forwarded to the leader.

How does Google Spanner Achieves Global Consistency handle failures?

Spanner defines failure domains by region and zone. It automatically fails over to healthy replicas, uses Paxos to maintain consensus, and relies on TrueTime to avoid unsafe reads during outages.