This is the most detailed rung of the “how it works” ladder — start with Why FoundryGate? for the plain-English version or the system overview for the middle level.
FoundryGate’s premise: a drop-in Azure solution that provisions Microsoft Foundry models for agentic coding harnesses (Claude Code, Codex CLI) and gives platform teams cost management, per-developer quotas, rate limiting, and metrics in one deployment. This page assesses that premise against the Azure platform as of September 2026 and states the architecture decisions that follow. All claims trace to Microsoft Learn or Anthropic documentation current at that date.
Feasible — and better-timed than when the project was specced. Every pillar now has a first-party Azure mechanism:
| Requirement | Mechanism | Assessment |
|---|---|---|
| Drop-in provisioning (Bicep) | AIServices account + Anthropic/OpenAI deployments + APIM v2, Marketplace attestation inline | ✅ Fully supported |
| Monthly token quota per developer | APIM llm-token-limit with token-quota-period="Monthly" per subscription key | ✅ Native, GA, real-time |
| Transient TPM/RPM smoothing | Per-developer TPM policy caps + backend pools + circuit breakers honoring Retry-After | ✅ Gateway-native |
| Per-developer metrics | llm-emit-token-metric (dashboards) + ApiManagementGatewayLlmLog (billing-grade) | ✅ Native |
| Per-developer cost | Token telemetry × rate card | ⚠️ Buildable — not available from Azure Cost Management for Claude |
| Claude at large scale | Multi-region pools; no PTU exists for Claude | ⚠️ Structural ceiling remains |
| ”One click” drop-in | One Bicep deploy + ~30-minute tenant prep checklist | ⚠️ Near, not absolute |
Three developments since the original spec reshape the architecture:
The llm-token-limit policy (GA, all tiers except Consumption) supports, in one policy:
tokens-per-minute — rolling TPM cap per counter-key, returning 429 with
Retry-After when exceeded.token-quota + token-quota-period="Hourly|Daily|Weekly|Monthly|Yearly" — a
fixed calendar-window token budget per counter-key, returning 403 when exhausted.With counter-key="@(context.Subscription.Id)", each developer’s APIM subscription
carries its own TPM cap and monthly budget, enforced on the request path with no
sync lag. The original design — a Function polling Log Analytics every 15 minutes and
suspending subscriptions — is no longer the enforcement mechanism. It survives as
reconciliation: authoritative usage numbers for dashboards, drift detection, and
audit.
On APIM v2 tiers (Basic v2 / Standard v2 / Premium v2), the LLM policies
(llm-token-limit, llm-emit-token-metric) support the Anthropic Messages API
schema alongside OpenAI Chat Completions/Responses — token metering works on Claude
Code’s native /v1/messages traffic without format translation. This is the single
most consequential fact for FoundryGate: it makes one gateway with two front doors
(Anthropic-format for Claude Code, OpenAI-format for Codex CLI) fully meterable.
FoundryGate splits cleanly into a control plane and a data plane:
Control plane (FoundryGate app) — identity and lifecycle: Entra sync, APIM subscription key issuance/rotation/revocation, quota tier assignment, increase approvals, deployment provisioning/resizing, dashboards, audit.
Data plane (APIM v2 gateway) — everything on the request path: subscription-key auth, per-developer TPM caps and monthly budgets, token metric emission, backend pool routing with circuit breakers, retry-across-deployments on 429.
FoundryGate system diagram — data plane and control plane
FoundryGate system diagram. Real-time path: developer CLIs (Claude Code sends x-api-key, Codex CLI sends api-key) call the Azure API Management v2 gateway over HTTPS with a personal subscription key. The gateway has 2 front doors, /anthropic and /openai/v1, requires the subscription to belong to a tier product (Standard 5M tokens per month and 20K tokens per minute, Power 20M tokens per month and 40K tokens per minute, Unlimited with no monthly cap and 100K tokens per minute), applies the tier model allowlist and the llm-token-limit policy which returns 403 when the monthly quota is exhausted and 429 with Retry-After when the per-minute cap is hit, then routes with managed identity to backend pools with circuit breakers (Anthropic pool: priority 1 eastus2, priority 2 swedencentral; single OpenAI backend) and on to Azure AI Foundry accounts in eastus2 (primary) hosting claude-sonnet-4-5, claude-haiku-4-5, claude-opus-4-5, gpt-4-1-mini; swedencentral (pool-member) hosting claude-sonnet-4-5, claude-haiku-4-5, claude-opus-4-5. Gateway diagnostics stream LLM logs and token metrics to Log Analytics and Application Insights. Control plane, out of the request path: Entra ID signs users in and is the source of users and groups; the Blazor UI on Azure Static Web Apps calls FoundryGate.Api on Azure Container Apps, which stores state in Azure SQL through EF Core; FoundryGate.Functions run the monthly reset, usage reconciliation (reading ApiManagementGatewayLlmLog from Log Analytics, every 15 minutes) and an Entra directory sync daily at 02:00 UTC that reconciles users then groups from Microsoft Graph; Key Vault holds the APIM key-encryption key (no client secrets; Graph is called as each host's own managed identity). The API manages APIM subscriptions through the Management API: create under a tier product, regenerate both keys, move tier, delete. FoundryGate.Functions reaches the same Management API for one operation only — moving a subscription to another tier product when a reset resolves a new tier. Legend: solid azure arrows are the real-time enforcement path on the request; dashed grey arrows are telemetry and reconciliation, out of band; dashed amber arrows are control-plane calls for lifecycle and tiers.
x-api-key · Codex CLI → api-key
· one personal APIM subscription key per developer
/anthropic/openai/v1fg-require-product — a key with no tier product is refused (403) llm-token-limit policy each, counter keyed on the subscription
eastus2 → priority 2swedencentralclaude-sonnet-4-5 · claude-haiku-4-5 · claude-opus-4-5 · gpt-4-1-mini claude-sonnet-4-5 · claude-haiku-4-5 · claude-opus-4-5 ApiManagementGatewayLlmLog — per-request prompt / completion tokens, joined to the subscription (billing-grade)
· llm-emit-token-metric — near-real-time dashboards
ApiManagementGatewayLlmLog → usage reconciliation, every 15 min
fg-apim-key-encryption key · no app secrets (Graph uses the API identity)@KeyVault(...) references resolved at startup
infra/ and src/
at build time.
The solid path is the only thing a developer’s request ever touches. The telemetry path (diagnostics → Log Analytics → reconciliation Function) can lag or fail without changing what the gateway enforces — it reports, it does not decide. The control plane’s management-plane calls are different: they select what the gateway enforces (which tier product a subscription lives under), so a failed call means the intended change never lands — but the previously configured limits keep being enforced meanwhile.
Quota tiers as products
Products (Standard / Power / Unlimited) carry the token-limit policies. Changing a developer’s budget = moving their subscription between products. Per-user arbitrary values are a PoC question (policy-expression support); tiers are the safe default.
Two front doors, one gateway
/anthropic/* passes Anthropic Messages traffic to Foundry for Claude
Code; /openai/v1 serves Codex CLI’s Responses API. Each front door
accepts the key in the header its CLI actually sends (wire-verified):
x-api-key for Claude Code, api-key for Codex — which
additionally needs env_http_headers in its config.
Pools instead of PTU
Claude has no provisioned throughput and no spillover. Backend pools with priority
groups and circuit breakers that honor Retry-After are the equivalent:
priority-1 deployment saturates → traffic drains to priority-2 (other region,
DataZone, or hosting variant) automatically.
Cost from telemetry
Claude bills as one aggregate Marketplace CCU meter — Cost Management can’t split it by user or deployment. FoundryGate computes per-developer cost as tokens × rate card, reconciled from gateway logs.
The rate limits reference documents the deployment-level ceilings. The gateway addresses the transient symptoms in three layers:
tokens-per-minute caps mean one
developer’s agentic burst gets a targeted 429 + Retry-After (which Claude Code and
Codex both back off on) instead of saturating the shared deployment for everyone.Retry-After, which for AI backends
can be long) and the pool retries the request against the next deployment. The
client sees a slower response, not an error.What APIM does not provide is queueing — there is no request-queue primitive. For interactive CLI workloads this is acceptable: the CLIs’ own backoff is the queue.
One Bicep deployment provisions: the Foundry account, Claude + GPT deployments (with Marketplace attestation handled inline — no portal click-through), the APIM v2 gateway with products/policies/pools, SQL, the API container app, the static web app, Key Vault, and monitoring. Tenant prep that cannot be Bicep’d — roughly 30 minutes:
FoundryGate.Admin app role, Graph
permissions + admin consent).Microsoft.SaaS/register/action et al.).That checklist is the honest gap between “drop-in” and “one click” — documented in the fork guide rather than papered over.