Skip to content

Support multiple configurable backend upstreams #2

Description

@quwin

Overview

The gateway currently supports a single upstream backend through UPSTREAM_URL. This works for a simple reverse proxy, but it limits the gateway to one backend service at a time.

To behave more like a real API gateway, the service should support multiple backend upstreams and route requests to the correct target based on path, host, or route configuration.

Current behavior

The gateway reads one UPSTREAM_URL, parses it once at startup, and configures a single reverse proxy target:

upstreamURL := getenvString("UPSTREAM_URL", "http://localhost:9000")
parsedUpstreamURL, err := url.Parse(upstreamURL)
...
r.SetURL(parsedUpstreamURL)

This means all proxied requests are forwarded to the same backend service.

Expected behavior

The gateway should be able to route requests to multiple backend services.

Example:

/v1/ask -> RAG assistant backend
/v1/ingest -> RAG ingestion backend
/api/hello -> mock/demo backend
/metrics, /healthz, and /readyz remain gateway-owned routes

Proposed implementation

Introduce a route table or upstream registry loaded from config:

upstreams:
  rag-api:
    url: https://rag.quwin.dev
    audience: https://rag.quwin.dev
  mock-api:
    url: http://backend:9000

routes:
  path_prefix: /v1/
    upstream: rag-api
  path_prefix: /api/
    upstream: mock-api

At request time, the gateway should:

  • Match the incoming request to a configured route.
  • Resolve the route's upstream.
  • Rewrite/proxy the request to that upstream.
  • Apply the correct auth, rate-limit, and observability behavior for that route.
  • Design considerations
  • Preserve gateway-owned routes like /healthz, /readyz, and /metrics.
  • Support route-specific upstream audiences for Cloud Run authenticated service-to-service calls.
  • Return a clear 404 or 502 when no route or upstream is configured.
  • Avoid hardcoding backend URLs in code.

Acceptance criteria

Gateway supports more than one configured upstream.
Requests are routed by path prefix or route matcher.
Each upstream can define its own URL.
Each upstream can optionally define its own authentication audience.
Tests cover routing to at least two different mock backends.
README/docs include an example multi-backend configuration.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions