Azure OpenAI in Production: Deployments, PTUs, and Identity-Only Access

Problem Context

Calling OpenAI's public API works for prototypes. It stops working the moment your enterprise security team asks where prompts go, whether traffic crosses the public internet, what region data is processed in, who can mint a key, and what your SLA is when usage spikes during a product launch. Azure OpenAI Service answers all of those by hosting the same models inside your Azure subscription with VNet integration, Entra ID auth, content filters, regional pinning, and committed throughput via PTUs.

The trade-offs are real: per-region model availability is uneven, you have to deploy a model before you can call it, capacity is governed by quota, and the SDK story has been churning (the legacy OpenAIClient from Azure.AI.OpenAI is being superseded by the official openai package configured against your Azure endpoint). This guide is the production setup that holds up in 2026.

Concept Explanation

Azure OpenAI exposes OpenAI's models (GPT-4o, GPT-4.1, o-series reasoning models, embeddings, DALL·E, Whisper) through Azure-managed endpoints. Three concepts you must internalize:

• Resource — your Azure OpenAI account in a region. Has its own endpoint URL and quota allocation.
• Deployment — a named instance of a specific model + version. Your code calls the deployment name, not the model name.
• Capacity — Standard (pay-per-token, shared pool) or Provisioned (PTUs, dedicated throughput, committed cost).

flowchart LR
    A["App<br/>(Managed Identity)"] -->|Entra ID token| B["Azure OpenAI Resource<br/>(eastus2)"]
    B --> C{"Routing"}
    C -->|deployment: gpt-4o-prod| D["GPT-4o<br/>Standard or PTU"]
    C -->|deployment: o3-reasoning| E["o3 reasoning"]
    C -->|deployment: ada-002| F["text-embedding-3-large"]
    B --> G["Content Filter<br/>(prompt + completion)"]
    B --> H["Diagnostic Logs →<br/>Log Analytics"]

    style B fill:#0078D4,color:#fff,stroke:#005a9e
    style G fill:#dc2626,color:#fff,stroke:#b91c1c

Implementation

Step 1: Provision with Bicep — keys disabled, Entra only

// infra/openai.bicep
param location string = resourceGroup().location
param resourceName string

resource openai 'Microsoft.CognitiveServices/accounts@2024-10-01' = {
  name: resourceName
  location: location
  kind: 'OpenAI'
  sku: { name: 'S0' }
  properties: {
    customSubDomainName: resourceName
    publicNetworkAccess: 'Disabled'           // require private endpoint
    disableLocalAuth: true                    // KEYS OFF — Entra only
    networkAcls: { defaultAction: 'Deny', ipRules: [], virtualNetworkRules: [] }
  }
  identity: { type: 'SystemAssigned' }
}

resource gpt4o 'Microsoft.CognitiveServices/accounts/deployments@2024-10-01' = {
  parent: openai
  name: 'gpt-4o-prod'                         // ← your code uses THIS name
  sku: { name: 'GlobalStandard', capacity: 50 } // 50 K TPM
  properties: {
    model: { format: 'OpenAI', name: 'gpt-4o', version: '2024-11-20' }
    raiPolicyName: 'Microsoft.DefaultV2'
  }
}

output endpoint string = openai.properties.endpoint

Step 2: Grant the app role assignment, not a key

// Cognitive Services OpenAI User — call deployments
var openAiUserRoleId = '5e0bd9bd-7b93-4f28-af87-19fc36ad61bd'

resource appRoleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  scope: openai
  name: guid(openai.id, appPrincipalId, openAiUserRoleId)
  properties: {
    roleDefinitionId: subscriptionResourceId(
      'Microsoft.Authorization/roleDefinitions', openAiUserRoleId)
    principalId: appPrincipalId               // your app's managed identity
    principalType: 'ServicePrincipal'
  }
}

Step 3: Call from .NET 9 with DefaultAzureCredential

// Program.cs
using Azure.AI.OpenAI;
using Azure.Identity;
using OpenAI.Chat;

var endpoint = new Uri(Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")!);
var azureClient = new AzureOpenAIClient(endpoint, new DefaultAzureCredential());
var chat = azureClient.GetChatClient(deploymentName: "gpt-4o-prod"); // ← deployment name

var completion = await chat.CompleteChatAsync(
    [
        new SystemChatMessage("You are a concise assistant."),
        new UserChatMessage("Give me one tip on Azure OpenAI quotas.")
    ],
    new ChatCompletionOptions { MaxOutputTokenCount = 200, Temperature = 0.2f });

Console.WriteLine(completion.Value.Content[0].Text);

Step 4: Stream tokens with the official Python SDK against Azure

# pip install openai azure-identity
import os
from openai import AzureOpenAI
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

token_provider = get_bearer_token_provider(
    DefaultAzureCredential(),
    "https://cognitiveservices.azure.com/.default",
)

client = AzureOpenAI(
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    azure_ad_token_provider=token_provider,
    api_version="2025-01-01-preview",
)

stream = client.chat.completions.create(
    model="gpt-4o-prod",                       # ← deployment name
    messages=[{"role": "user", "content": "Stream me a haiku about Bicep."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Step 5: Standard vs PTU — pick by latency variance, not just cost

Standard / GlobalStandard
  • Pay per 1K input/output tokens
  • Shared capacity → P99 latency varies under noisy-neighbor load
  • Quota measured in TPM (tokens/min) per deployment
  • Best: low-volume APIs, batch jobs, dev/test, spiky traffic

Provisioned (PTU / Provisioned-Managed)
  • Buy units of throughput (PTU). Each PTU = guaranteed TPM for that model
  • Predictable P99 latency — no noisy neighbors
  • Hourly billing, monthly commit available (cheaper)
  • Best: chat with strict SLA, real-time agents, high-volume RAG

Rule of thumb: if a single user-facing call must complete in < 2s p99
on GPT-4o, you want PTUs. If batch tolerates 5-10s, Standard is cheaper.

Step 6: Content filter — what it actually does

Every call is screened by Azure AI Content Safety on both the prompt and the completion (Hate, Sexual, Violence, Self-harm, Jailbreak, Protected-Material). The default policy Microsoft.DefaultV2 blocks high-severity content. Custom policies let you tune severity thresholds per category or add allow-lists. Filtered responses come back with finish_reason: "content_filter" and acontent_filter_resultsobject — log it, don't silently retry.

var resp = await chat.CompleteChatAsync(messages);
if (resp.Value.FinishReason == ChatFinishReason.ContentFilter)
{
    logger.LogWarning("Filtered: {Categories}",
        resp.Value.ContentFilterResults?.PromptFilterResults);
    return Results.Problem("Request blocked by content policy.");
}

Pitfalls

1. Calling the model name instead of the deployment name.The OpenAI SaaS uses the model name. Azure OpenAI uses the deployment name you assigned in step 1. Mixing them yields 404 with a confusing "model not found" error.

2. Leaving keys enabled. If disableLocalAuth: true is not set, anyone with the key bypasses Entra and your Conditional Access policies. Always disable local auth in production resources.

3. Treating quota as capacity. Quota (TPM) is a soft cap your subscription is allowed to consume. Capacity is what the region can actually serve right now. You can have quota and still get 429s during a regional spike. PTUs are the only deterministic answer.

4. One resource per workload in one region. If eastus2 capacity dries up, your app dies. Provision the same deployment name in 2-3 paired regions and put Azure API Management or your own retry logic in front to fail over.

5. Pinning the model version to latest. Azure OpenAI models have explicit dated versions (gpt-4o:2024-11-20). Using auto-update means a model upgrade can change behavior under your tests. Pin the version, schedule the upgrade.

6. Ignoring the deprecated OpenAIClient. The old Azure.AI.OpenAI v1 client is replaced byAzureOpenAIClient (which delegates to the official OpenAI types). New code should use AzureOpenAIClient + theOpenAI.Chat namespace.

Practical Takeaways

• Provision with Bicep, keys disabled, system-assigned identity, private endpoints. No exceptions for "dev".
• Deploy each model with a name you control (e.g., gpt-4o-prod, gpt-4o-canary) — your code calls the name.
• Auth via DefaultAzureCredential (.NET) or get_bearer_token_provider (Python). Never store a key.
• Pick PTUs the moment you have an SLA. Standard for batch, dev/test, and burst.
• Always inspect finish_reason; log content-filter trips with the categories that fired.
• Pin model versions explicitly. Upgrade through deployments, not through silent updates.
• Provision in 2-3 regions and put a gateway in front for cross-region failover when quota tightens.