Connections

Overview

Connections define how Prompty authenticates with LLM providers. Every .prompty file can specify a connection inside its model.connection frontmatter block. The connection tells the runtime where to send requests and how to authenticate.

model:
  id: gpt-4o
  provider: foundry
  connection:          # ← this section
    kind: key
    endpoint: https://my-resource.services.ai.azure.com/api/projects/my-project
    apiKey: ${env:AZURE_AI_PROJECT_KEY}

Prompty ships six connection types — each backed by a corresponding schema type — so you can match the auth strategy to your environment.

---

Connection Types at a Glance

classDiagram
    class Connection {
        <>
        kind
    }
    class ApiKeyConnection {
        kind: key
        endpoint
        apiKey
        apiVersion
    }
    class ReferenceConnection {
        kind: reference
        name
    }
    class RemoteConnection {
        kind: remote
        name
        endpoint
    }
    class AnonymousConnection {
        kind: anonymous
        endpoint
    }
    class FoundryConnection {
        kind: foundry
        endpoint
        name
        connectionType
        authenticationMode
    }
    class OAuthConnection {
        kind: oauth
        endpoint
        clientId
        clientSecret
        tokenUrl
        scopes
    }
    Connection <|-- ApiKeyConnection
    Connection <|-- ReferenceConnection
    Connection <|-- RemoteConnection
    Connection <|-- AnonymousConnection
    Connection <|-- FoundryConnection
    Connection <|-- OAuthConnection

---

ApiKeyConnection — kind: key

The simplest connection: provide an endpoint and an API key. Ideal for local development and quick experiments.

model:
  id: gpt-4o
  provider: openai
  connection:
    kind: key
    endpoint: https://api.openai.com/v1
    apiKey: ${env:OPENAI_API_KEY}

For Microsoft Foundry, set the endpoint to your Foundry project or classic Azure resource:

model:
  id: gpt-4o
  provider: foundry
  connection:
    kind: key
    endpoint: ${env:AZURE_AI_PROJECT_ENDPOINT}
    apiKey: ${env:AZURE_AI_PROJECT_KEY}

You can also use a classic Azure OpenAI endpoint with the Foundry provider:

model:
  id: gpt-4o
  provider: foundry
  connection:
    kind: key
    endpoint: ${env:AZURE_OPENAI_ENDPOINT}
    apiKey: ${env:AZURE_OPENAI_API_KEY}

For Anthropic, use provider: anthropic with your Anthropic API key:

model:
  id: claude-sonnet-4-20250514
  provider: anthropic
  connection:
    kind: key
    endpoint: https://api.anthropic.com
    apiKey: ${env:ANTHROPIC_API_KEY}

Tip

Always use ${env:VAR} references instead of hardcoding secrets. The runtime resolves them from your environment (or a .env file) at load time.

---

ReferenceConnection — kind: reference

References a pre-registered SDK client by name. This is the recommended pattern for production because it supports Microsoft Entra ID, managed identity, custom retry policies, and any authentication method the SDK supports.

model:
  id: gpt-4o
  provider: foundry
  connection:
    kind: reference
    name: my-foundry-client

Register the client at application startup:

Python

from openai import AzureOpenAI
import prompty

prompty.register_connection("my-foundry-client", client=AzureOpenAI(
    azure_endpoint="https://my-resource.services.ai.azure.com/api/projects/my-project",
    api_key="...",
))

TypeScript

import { AzureOpenAI } from "openai";
import { registerConnection } from "@prompty/core";
import "@prompty/foundry";

registerConnection("my-azure-client", new AzureOpenAI({
  endpoint: "https://my-resource.openai.azure.com/",
  apiKey: "...",
}));

C#

using Prompty.Core;

ConnectionRegistry.Register("my-foundry-client", new AzureOpenAI(
    new Uri("https://my-resource.services.ai.azure.com/api/projects/my-project"),
    new ApiKeyCredential("...")
));

Rust

use serde_json::json;

prompty::register_connection("my-foundry-client", json!({
    "kind": "key",
    "endpoint": "https://my-resource.services.ai.azure.com/api/projects/my-project",
    "apiKey": "...",
}));

The executor looks up "my-azure-client" in the connection registry and uses the pre-configured SDK client directly — no additional auth logic is needed in the .prompty file.

Note

Because the client is constructed in application code, you have full control over authentication, retries, timeouts, and custom headers — things that can’t be expressed in YAML alone.

---

RemoteConnection — kind: remote

A named endpoint reference for remote model-serving services. Useful when you have a separate inference service behind a gateway.

model:
  id: my-fine-tuned-model
  provider: openai
  connection:
    kind: remote
    name: my-service
    endpoint: https://my-model.azurewebsites.net

---

AnonymousConnection — kind: anonymous

Endpoint only — no authentication. Perfect for self-hosted models like Ollama or vLLM running locally.

model:
  id: llama3
  provider: openai
  connection:
    kind: anonymous
    endpoint: http://localhost:11434/v1

Caution

Only use anonymous connections with trusted local endpoints. Never expose unauthenticated model endpoints to the public internet.

---

FoundryConnection — kind: foundry

Integrates with Microsoft Foundry for managed model deployments, including serverless and managed-compute endpoints.

model:
  id: gpt-4o
  provider: foundry
  connection:
    kind: foundry
    endpoint: ${env:FOUNDRY_ENDPOINT}
    name: my-deployment
    connectionType: serverless

Field	Description
endpoint	Foundry project endpoint
name	Deployment name within the Foundry project
connectionType	serverless or managedCompute
authenticationMode	Optional — override default Foundry authentication

---

OAuthConnection — kind: oauth

OAuth 2.0 client credentials flow for services that require token-based authentication.

model:
  id: my-model
  provider: openai
  connection:
    kind: oauth
    endpoint: https://api.example.com
    clientId: ${env:CLIENT_ID}
    clientSecret: ${env:CLIENT_SECRET}
    tokenUrl: https://auth.example.com/token
    scopes:
      - api.read

Field	Description
clientId	OAuth 2.0 client identifier
clientSecret	OAuth 2.0 client secret
tokenUrl	Token endpoint URL
scopes	List of scopes to request
endpoint	The model-serving endpoint to call

---

Connection Resolution Flow

The following diagram shows how the runtime resolves a connection when a .prompty file is loaded and executed.

flowchart TD
    A["Read model.connection"] --> B{"kind = ?"}
    B -->|key| C["Use directly\nendpoint + apiKey → SDK client"]
    B -->|reference| D["Registry lookup\nname → pre-configured client"]
    B -->|foundry| E["Foundry resolve\nendpoint + name → Foundry SDK"]
    B -->|"anonymous · remote · oauth"| F["Kind-specific resolution"]
    C -.-> G["Expand ${env:VAR} references\nLoadContext.pre_process resolves all string values"]
    D -.-> G
    E -.-> G
    F -.-> G
    G --> H["Configured SDK Client\nReady for executor to call LLM"]

---

Connection Registry

For production workloads you’ll typically use kind: reference combined with a connection registry configured at application startup. This keeps secrets and complex auth logic in application code — not in YAML.

Python

import os
import prompty
from openai import AzureOpenAI
from azure.identity import DefaultAzureCredential, get_bearer_token_provider

# Build a client with Entra ID token-based auth
client = AzureOpenAI(
    azure_endpoint=os.environ["AZURE_ENDPOINT"],
    azure_ad_token_provider=get_bearer_token_provider(
        DefaultAzureCredential(),
        "https://cognitiveservices.azure.com/.default",
    ),
)

# Register it by name — .prompty files reference this name
prompty.register_connection("foundry-prod", client=client)

TypeScript

import { AzureOpenAI } from "openai";
import { DefaultAzureCredential, getBearerTokenProvider } from "@azure/identity";
import { registerConnection } from "@prompty/core";
import "@prompty/foundry";

// Build a client with Entra ID token-based auth
const credential = new DefaultAzureCredential();
const scope = "https://cognitiveservices.azure.com/.default";
const azureADTokenProvider = getBearerTokenProvider(credential, scope);

const client = new AzureOpenAI({
  endpoint: process.env.AZURE_ENDPOINT!,
  azureADTokenProvider,
});

// Register it by name — .prompty files reference this name
registerConnection("foundry-prod", client);

C#

using Azure.Identity;
using Prompty.Core;

// Build a client with Entra ID token-based auth
var credential = new DefaultAzureCredential();
var client = new AzureOpenAI(
    new Uri(Environment.GetEnvironmentVariable("AZURE_ENDPOINT")!),
    credential
);

// Register it by name — .prompty files reference this name
ConnectionRegistry.Register("foundry-prod", client);

Rust

use serde_json::json;

// Register a named connection — .prompty files reference this name
prompty::register_connection("foundry-prod", json!({
    "kind": "key",
    "endpoint": std::env::var("AZURE_ENDPOINT").unwrap(),
    "apiKey": std::env::var("AZURE_API_KEY").unwrap(),
}));

// Check if a connection exists
if prompty::has_connection("foundry-prod") {
    // Connection is registered and ready
}

Then in your .prompty file:

model:
  id: gpt-4o
  provider: foundry
  connection:
    kind: reference
    name: foundry-prod

Tip

The connection registry is an in-process dictionary. Register connections once at startup (e.g., in your FastAPI lifespan, Django AppConfig.ready(), or __main__ block) before calling prompty.run().

---

Environment Variable Resolution

All string values in the frontmatter support ${env:VAR} references. The runtime resolves them at load time via LoadContext.pre_process.

Syntax	Behavior
${env:VAR}	Reads VAR from the environment. Raises ValueError if unset.
${env:VAR:default}	Falls back to default when VAR is unset.

model:
  connection:
    kind: key
    endpoint: ${env:OPENAI_ENDPOINT:https://api.openai.com/v1}
    apiKey: ${env:OPENAI_API_KEY}

Environment variables are loaded from a .env file (via python-dotenv) if one exists alongside the .prompty file or in a parent directory.

---

Which Connection Should I Use?

Local Development

Use kind: key with environment variables. It’s the fastest way to get started and keeps secrets out of source control.

model:
  connection:
    kind: key
    endpoint: ${env:OPENAI_ENDPOINT:https://api.openai.com/v1}
    apiKey: ${env:OPENAI_API_KEY}

Production (Entra ID)

Use kind: reference with the connection registry. Register an AzureOpenAI client backed by DefaultAzureCredential at startup.

model:
  connection:
    kind: reference
    name: foundry-prod

Microsoft Foundry

Use kind: foundry to connect directly to a Foundry-managed deployment — serverless or managed compute.

model:
  connection:
    kind: foundry
    endpoint: ${env:FOUNDRY_ENDPOINT}
    name: my-deployment

Self-Hosted Models

Use kind: anonymous for local inference servers like Ollama or vLLM that don’t require authentication.

model:
  connection:
    kind: anonymous
    endpoint: http://localhost:11434/v1

OAuth Services

Use kind: oauth for services that require an OAuth 2.0 client credentials flow.

model:
  connection:
    kind: oauth
    endpoint: https://api.example.com
    clientId: ${env:CLIENT_ID}
    clientSecret: ${env:CLIENT_SECRET}
    tokenUrl: https://auth.example.com/token
    scopes:
      - api.read

Quick Decision Guide

Scenario	Connection Kind	Why
Local dev with API key	key	Simplest setup, env-var-based secrets
Anthropic (Claude)	key	API key auth with provider: anthropic
Production with Microsoft Entra ID	reference	Full SDK control, managed identity
Microsoft Foundry deployments	foundry	Native Foundry service integration
OAuth 2.0 services	oauth	Client credentials token flow
Remote gateway / proxy	remote	Named endpoint reference