When you have one service, clients call it directly. When you have fifty, you don’t want each client to know fifty addresses, and you don’t want fifty services each re-implementing authentication, rate limiting, and TLS. An API gateway is the single front door that handles those cross-cutting concerns once, at the edge. This deep dive covers what a gateway does, how to build one with Spring Cloud Gateway, and where a gateway ends and a service mesh begins.
flowchart LR
Cl[Client] --> GW[API Gateway]
GW -->|authenticate| A[Validate JWT]
GW -->|rate limit| RL[Redis limiter]
GW --> S1[Service A]
GW --> S2[Service B]
The gateway is where edge concerns live so individual services don’t each reinvent them:
The payoff is one stable address for clients and one place to enforce edge policy — instead of duplicated, drifting implementations across every service.
Spring Cloud Gateway is the Java-native gateway: reactive (built on Spring WebFlux/Project Reactor and Netty, so it handles high concurrency on few threads) and configured with routes made of predicates (match conditions) and filters (actions). Most of it is YAML.
spring:
cloud:
gateway:
routes:
- id: orders
uri: lb://orders-service # load-balanced via service discovery
predicates:
- Path=/api/orders/**
filters:
- StripPrefix=1
- name: RequestRateLimiter
args:
redis-rate-limiter.replenishRate: 100 # tokens/sec
redis-rate-limiter.burstCapacity: 200
- name: CircuitBreaker
args:
name: ordersCb
fallbackUri: forward:/fallback/ordersPredicates can match path, host, method, headers, or time; filters can rewrite paths, add headers, rate-limit, circuit-break, or retry. You can also write custom filters in Java for bespoke logic (e.g. injecting a tenant header from a claim).
The gateway is the natural place to validate access tokens, so unauthenticated traffic never reaches your services. Configure it as an OAuth2 resource server (or client, for login flows) to validate the JWT’s signature, issuer, audience, and expiry, then forward the request — often propagating the token or claims downstream.
spring:
security:
oauth2:
resourceserver:
jwt:
issuer-uri: https://login.example.com/An important caveat: edge authentication is necessary but not sufficient under zero trust. Services should still validate tokens themselves rather than blindly trusting “it came through the gateway,” because an attacker inside the network could otherwise bypass the front door. The gateway reduces load and centralizes policy; it doesn’t excuse services from verifying.
Rate limiting protects backends and enforces fair use, but it only works if the limit is shared across all gateway instances — an in-memory counter per instance lets a client multiply its quota by the number of replicas. Spring Cloud Gateway’s RequestRateLimiter uses Redis and a token-bucket algorithm so the limit is global. You define who a limit applies to with a KeyResolver — per user, per API key, or per IP.
@Bean
KeyResolver userKeyResolver() {
// limit per authenticated user; fall back to remote address
return exchange -> exchange.getPrincipal()
.map(Principal::getName)
.defaultIfEmpty(exchange.getRequest().getRemoteAddress().getAddress().getHostAddress());
}Return a clear 429 Too Many Requests with a Retry-After header so well-behaved clients can back off. Consider tiered limits (a paid tier gets a higher rate) keyed off the token’s claims.
The gateway is a strategic place to apply resilience patterns because it sees every inbound call. Add timeouts so a slow backend can’t tie up gateway resources, circuit breakers (Resilience4j) so a failing service fails fast into a fallback instead of cascading, and bounded retries for idempotent routes. This complements — rather than replaces — the in-service resilience covered in our circuit breakers deep dive; defense in depth at both layers is normal.
Centralize browser concerns and protection at the edge: configure CORS once at the gateway rather than per service; enforce maximum request body size to blunt abusive payloads; and use filters to normalize headers, strip internal ones, or inject correlation/trace IDs so everything downstream is consistent. Doing this at the gateway keeps services focused on business logic.
These are frequently confused but solve different problems, defined by traffic direction:
| API Gateway | Service Mesh | |
|---|---|---|
| Traffic | North-south (clients → system) | East-west (service ↔ service) |
| Lives | At the edge | Between services (sidecars) |
| Concerns | Auth, rate limiting, routing, TLS termination | mTLS, retries, traffic shifting, observability |
| Example | Spring Cloud Gateway | Istio, Linkerd |
They are complementary. A large platform commonly runs a gateway at the front door for client-facing policy and a mesh inside for transparent, language-agnostic service-to-service security and reliability. Use the gateway for what clients see; use the mesh for what services do among themselves.
An API gateway gives your microservice platform one secure, observable front door — centralizing routing, authentication, rate limiting (distributed via Redis), TLS, and edge resilience so individual services don’t each reinvent them. Spring Cloud Gateway is the reactive, Java-native way to build one. Remember its boundaries: pair it with per-service token validation for zero trust, complement it with a service mesh for east-west traffic, and keep business logic out of it. Done right, the gateway is the calm, consistent entry point that makes everything behind it simpler.
What is an API gateway and why do microservices need one?
An API gateway is a single entry point in front of your services that handles cross-cutting concerns — routing, authentication, rate limiting, TLS termination, CORS, and request shaping — so each service does not reimplement them. It gives clients one stable address and centralizes edge policy.
What is the difference between an API gateway and a service mesh?
A gateway handles north-south traffic (clients into the system) at the edge. A service mesh handles east-west traffic (service-to-service) inside the system, providing mTLS, retries, and observability transparently. They are complementary: a gateway at the front door, a mesh between services.