Gateway Routing
An API Gateway is a single entry point that sits in front of all your backend services. It routes requests to the right service, terminates TLS, enforces authentication and rate limiting, transforms requests and responses, and shields clients from the internal topology. Think of it as the front desk of your microservices hotel: every visitor checks in here, gets vetted, and is escorted to the right room — without ever learning the building's floor plan.
How it works
An API gateway receives every external request, applies a stack of cross-cutting concerns, and forwards it to the right backend service. It is a reverse proxy with opinions. Where a generic reverse proxy just forwards bytes, an API gateway actively participates in the request: authenticating, transforming, rate-limiting, composing responses.
The core responsibilities are:
- Routing —
/api/users/*goes to the user service,/api/orders/*to the order service,/api/billing/*to the billing service. Clients see one host; the gateway fans out to many. - TLS termination — the gateway holds the TLS certificate and decrypts HTTPS, so backend services can speak plain HTTP on a private network.
- Authentication & authorization — validate JWTs, API keys, or OAuth tokens once at the gateway, then pass identity downstream. Backends don't each re-implement auth.
- Rate limiting & quotas — enforce per-client limits (100 req/min for free tier, 10000 for paid) at the edge, before backends ever see the traffic.
- Request/response transformation — reshape payloads for clients (e.g., strip internal fields, rename keys, aggregate multiple backend calls into one response).
- Observability — centralized logging, metrics, and distributed tracing for every request.
- Protocol translation — accept REST from clients, call gRPC internally; accept HTTP/1, speak HTTP/2 to backends.
API Gateway vs Reverse Proxy:
These terms overlap and the line is fuzzy, but the intent is different:
| Reverse Proxy | API Gateway | |
|---|---|---|
| Primary job | Forward HTTP requests to backends | Full request lifecycle: route + transform + auth + rate limit |
| Awareness of business | Low — usually path/host routing | High — knows about clients, scopes, quotas, products |
| Cross-cutting concerns | TLS, basic LB, caching | TLS, auth, rate limit, request shaping, response aggregation, quotas |
| Example tools | NGINX, HAProxy | Kong, AWS API Gateway, Apigee, Envoy (configured as gateway) |
| Aggregation | No | Often — one client request may fan out to N backend services |
A reverse proxy is a building block. An API gateway is a reverse proxy plus a stack of business-aware middlewares. NGINX can be configured as either; the difference is in what middlewares you enable and what rules you encode. Some teams call their NGINX front-end an 'API gateway' because it terminates TLS and routes by path — that's a fair use of the term. The pattern matters more than the label.
API Gateway vs Load Balancer: A load balancer distributes traffic across instances of one service. An API gateway routes traffic across many services, and applies business logic (auth, rate limit). Production systems often stack both: a load balancer in front of a fleet of API gateway instances, which in turn route to backend services.
The BFF variant — one gateway per client:
A single API gateway serving all clients (web, mobile, partner) often becomes a compromise: mobile wants smaller payloads, web wants richer aggregation, partners want SOAP. The Backend-for-Frontend (BFF) pattern instead runs one gateway per client type — a mobile BFF, a web BFF, a partner BFF — each tailored to its client. Each BFF is owned by the team that owns the client. This avoids the 'one gateway to rule them all' anti-pattern where every change needs agreement across all client teams.
The trade-off: more gateways means more operational surface. For most products, a single API gateway with route-based separation is enough. For organizations with sharply different client needs (mobile + web + partner APIs in different formats), BFFs are worth the cost.
Every external request flows through the gateway. If it goes down, your entire API is down — no matter how healthy your backends are. Mitigations: run multiple gateway instances behind a load balancer, deploy across availability zones, have automated failover, and keep the gateway stateless (store rate-limit counters and auth state in Redis, not in the gateway process). Treat the gateway's SLO as the strictest in your system: it must be at least as available as the most available backend.
What not to put in the gateway:
The gateway is a routing and policy layer, not a backend. Avoid the temptation to put business logic here:
- Business rules (order validation, pricing calculations, inventory checks) belong in backend services. The gateway should not know what a valid order looks like.
- Long-running work (calling slow third parties, generating reports) belongs in background jobs. A gateway should respond in milliseconds, not minutes.
- Stateful operations (session storage, shopping cart contents) belong in databases or Redis. The gateway should be stateless.
- Heavy request bodies — gateways that buffer full request/response bodies to transform them pay a memory and latency tax. Where possible, stream.
The risk of putting too much in the gateway is that it becomes a distributed monolith: every business change requires a gateway redeploy, and the gateway team becomes a bottleneck. Keep the gateway thin; let the backends own the product.
Your team is debating whether to put order-validation rules in the API gateway or in the orders service. Which is the better choice, and why?
Pick one answer.
What is the most important operational property of an API gateway, and how is it usually achieved?
Pick one answer.
A mobile client needs a single response that combines user profile, recent orders, and recommendations. Where should this aggregation happen?
Pick one answer.
Engineering mental model
Mental model. Think of Gateway Routing as a deliberate boundary in a system. The boundary exists because something becomes harder to manage when everything is done in one place: latency, scale, failure isolation, consistency, cost, or team ownership. The useful question is not “what does Gateway Routing mean?” but “what pressure makes this boundary worth introducing, and what new failure mode does it create?”
Before choosing Gateway Routing, name the workload, the critical user path, the dominant bottleneck, the failure you are trying to absorb, and the trade-off you are willing to accept. If you cannot name those five things, the design is probably premature.
curl -i https://api.example.com/v1/api-gateway
# Look for:
# - status code
# - latency
# - retryability
# - response sizeBack-of-the-envelope reasoning
Numerical lens: write down traffic, payload size, read/write ratio, peak multiplier and durability target before choosing a component. The numbers should justify the architecture.
Interactive thought experiment: Gateway Routing
Change the variables below and predict what breaks first in Gateway Routing. The production lab can later reuse these same inputs.
Change one variable at a time. Predict the failure mode first, then move the slider and see whether your mental model matches the simplified system response.
If you are stuck on Gateway Routing, start by drawing the request path and marking every network hop, stateful component, queue, cache and failure boundary. Then estimate where the system will saturate.
You increase traffic by 10× in a system using Gateway Routing. What should you inspect first?
Pick one answer.
Which statement is the safest engineering habit when using Gateway Routing?
Pick one answer.
You have dashboards for traffic, latency, errors and saturation. You can change the architecture, but every change has operational cost.
Production scenario: your system uses Gateway Routing, traffic suddenly spikes, and p99 latency doubles. What is your first move?
Interview drill
Answer this without notes: When would you choose Gateway Routing, and when would you intentionally avoid it? Mention at least one bottleneck it addresses, one failure mode it introduces, and one alternative. Then quantify the workload you are designing for.
A useful engineering lens for Gateway Routing: define the problem it solves, the simpler design that fails first, the constraint that forces you to introduce this concept, and the new failure modes the concept creates.
Numerical sanity check
Back-of-the-envelope reasoning beats fake precision. State your traffic, payload, concurrency and growth assumptions explicitly, then calculate enough to know whether the current architecture is orders of magnitude away from the target.
Do not optimize for a memorized definition. Reason from the workload and failure mode.
Imagine the simplest version of a system using Gateway Routing. What breaks first as traffic grows by 10×, and what would you change before reaching 100×?
Pick one answer.
What you gain, what you pay
- +Single entry point for clients — topology of backends is hidden.
- +Centralized cross-cutting concerns: TLS, auth, rate limit, logging, tracing.
- +Protocol translation: clients speak REST, backends speak gRPC.
- +Client-tailored responses (BFF pattern) without burdening backends.
- +Independent backend evolution — clients are shielded from internal restructuring.
- −The gateway is a single point of failure — must be engineered for high availability.
- −Adds a network hop and processing latency (~1–5ms typical, more with heavy transformation).
- −Risk of becoming a distributed monolith if business logic leaks into the gateway.
- −Operational complexity: configuration, deploy pipeline, observability for the gateway itself.
- −A new layer of failure modes (auth bugs, rate-limit bugs, routing misconfigurations) that affect every request.
How this breaks in production
- Gateway outage takes down the entire API, even when all backends are healthy.
- Business logic accumulating in the gateway — every business change requires a gateway redeploy.
- Auth misconfiguration leaking unauthenticated traffic to backends.
- Rate-limit state stored in-process (not Redis) breaking under horizontal scaling — each instance counts independently.
- Heavy request/response buffering causing memory pressure and head-of-line blocking.
Don't fall into these traps
- •Putting business rules in the gateway — turns it into a distributed monolith.
- •Running a single gateway instance — single point of failure.
- •Storing rate-limit counters in-process — breaks under horizontal scaling.
- •Letting the gateway make slow third-party calls inline — every request pays the cost.
- •Forgetting that the gateway's SLO must be stricter than any backend's — it bounds total availability.
- •Skipping distributed tracing through the gateway — request flows become opaque.
Real systems using this
How real systems implement this
- Kong — Open-source API gateway built on NGINX/OpenResty with a plugin architecture for auth, rate limiting, logging, and transformation. Deployed as multiple stateless instances behind a load balancer, with configuration stored in PostgreSQL or declarative YAML.
- AWS API Gateway — Managed service that handles TLS, API-key auth, JWT authorizers via Lambda, per-client usage plans and quotas, request/response transformations, and direct integration with Lambda, HTTP backends, and other AWS services. Scales horizontally and bills per-request.
- Netflix Zuul — Netflix's in-house API gateway, designed to handle runtime routing, auth, and insights for Netflix's streaming service at global scale. Pioneered many BFF and gateway patterns now common in the industry.
Practice saying it out loud
- Q1Design an API gateway for a public API with 100,000 requests/sec. How do you keep it available and fast?
- Q2How does an API gateway differ from a reverse proxy and a load balancer? When do you need each?
- Q3Your gateway team is becoming a bottleneck — every business change requires a gateway redeploy. What went wrong, and how do you fix it?
- Q4When would you use the BFF pattern (one gateway per client) instead of a single API gateway?
- Q5How do you enforce rate limits across multiple gateway instances without inconsistency?
Further reading & references
Core explanations are original NO CAP material. External references are provided for deeper study and standards.
What next?
Mark as understood once the mental model clicks.
Next recommended
Reverse Proxy