LLM API Cheat Sheet
Quick reference for the major LLM provider APIs. Covers authentication, text generation, function calling, structured output, streaming, prompt caching, and model selection across OpenAI, Anthropic, Google Gemini, Mistral, and Cohere.
Model IDs and SDK calls below were checked against each provider's official docs in November 2026. Model lineups change every few months — confirm IDs on the provider's models page before you ship.
Model Comparison
Flagship Models (as of November 2026)
Prices are list prices per 1M tokens from the provider's own models/pricing page. Where a price could not be verified, the cell links to the pricing page instead of guessing.
| Provider | Model ID | Context | Strengths | Input $/1M | Output $/1M |
|---|---|---|---|---|---|
| OpenAI | gpt-6-astra | 1.05M | Flagship: complex reasoning, coding | $10.00 | $50.00 |
| OpenAI | gpt-6.1-sol | 1.05M | Balance of intelligence and cost | $2.00 | $10.00 |
| OpenAI | gpt-6.1-terra | 1.05M | Cheaper, supports none reasoning | $1.00 | $5.00 |
| OpenAI | gpt-6-luna | 1.05M | Cheapest, high volume | $0.10 | $0.50 |
| Anthropic | claude-fable-5-1 | 1M | Most capable: hardest reasoning, long-horizon agents | $10.00 | $50.00 |
| Anthropic | claude-opus-5-5 | 1M | Default pick: agentic coding, knowledge work | $4.00 | $20.00 |
| Anthropic | claude-sonnet-5-5 | 1M | Speed + intelligence balance | $2.00 | $10.00 |
| Anthropic | claude-haiku-5-5 | 1M | Fastest: classification, extraction, routing | from $0.10 | from $0.50 |
gemini-4-argon | 1M | Flagship reasoning | pricing | pricing | |
gemini-3.8-flash | see docs | Fast default used across Google's docs | pricing | pricing | |
| Mistral | Mistral Large 4 (mistral-large-latest) | see docs | Open-weight, multimodal, EU-hosted | pricing | pricing |
| Mistral | Mistral Small 4 (mistral-small-latest) | see docs | Apache 2.0; instruct + reasoning + code | pricing | pricing |
| Cohere | command-a-plus-05-2026 | 128K | RAG, tools, image input (MoE) | pricing | pricing |
| Cohere | command-a-reasoning-08-2025 | 256K | Reasoning, agentic tool use | pricing | pricing |
Official models / pricing pages: OpenAI models · Anthropic models / pricing · Gemini models · Mistral models · Cohere models.
Pricing Changes
LLM pricing changes frequently. The prices above are list prices as of November 2026. Always verify current pricing on each provider's website before making architectural decisions. Batch APIs are typically 50% off and cached input is much cheaper (see Prompt Caching).
Retired / renamed since early 2026
- OpenAI:
gpt-4o,gpt-4o-mini,o3are no longer the recommended models; start withgpt-6-astra(orgpt-6.1-sol/gpt-6-lunafor cost). - Anthropic:
claude-sonnet-4-20250514and the other Claude 4 / 3.x dated IDs are superseded. Since the 4.6 generation, IDs are dateless pinned snapshots (claude-opus-5-5) — don't append a date. - Google:
gemini-2.0-flashis shut down;gemini-2.5-pro/gemini-2.5-flashare deprecated and reroute on Dec 1, 2026. The oldgoogle-generativeaiSDK is replaced bygoogle-genai. - Mistral: Magistral models are deprecated (reasoning is built into Small 4 / Large 4);
mistral-large-2512(Large 3) is deprecated. - Cohere: the
command-r/command-r-plusaliases are deprecated — use dated IDs such ascommand-r-08-2024, or move to Command A.
Capabilities Matrix
| Capability | OpenAI | Anthropic | Gemini | Mistral | Cohere |
|---|---|---|---|---|---|
| Chat / text generation | Yes (Responses + Chat Completions) | Yes (Messages) | Yes (generate_content, Interactions) | Yes | Yes (Chat v2) |
| Function calling | Yes | Yes (tool use) | Yes | Yes | Yes |
| Structured output | Yes (JSON schema, strict) | Yes (JSON schema via output_config.format) | Yes (JSON schema) | Yes (JSON mode + JSON schema) | Yes (JSON mode + JSON schema) |
| Vision (images) | Yes | Yes | Yes | Yes (Large 4, Ministral 3) | Yes (Command A Vision, A+) |
| Audio input | Yes | No | Yes (native) | Yes (Voxtral) | No |
| Streaming | Yes | Yes | Yes | Yes | Yes |
| Embeddings | Yes | No (use Voyage) | Yes | Yes | Yes |
| Batch API | Yes | Yes | Yes | Yes | No |
| Prompt caching | Automatic (explicit breakpoints on newer models) | Explicit cache_control or top-level automatic | Implicit (default) + explicit on generate_content | — | — |
| Reasoning / thinking | Yes (reasoning effort) | Yes (adaptive thinking + effort) | Yes (thinking_level) | Yes (Small 4 / Large 4) | Yes (Command A Reasoning) |
Authentication
Every SDK reads its key from an environment variable if you don't pass one — prefer that over hardcoding keys.
# OpenAI — pip install openai
from openai import OpenAI
client = OpenAI() # reads OPENAI_API_KEY
# Anthropic — pip install anthropic
import anthropic
client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY
# Google Gemini — pip install google-genai (NOT the old google-generativeai)
from google import genai
from google.genai import types
client = genai.Client() # reads GEMINI_API_KEY (or GOOGLE_API_KEY)
# Mistral — pip install mistralai (current SDK imports from mistralai.client)
import os
from mistralai.client import Mistral
client = Mistral(api_key=os.environ["MISTRAL_API_KEY"])
# Cohere — pip install cohere
import cohere
client = cohere.ClientV2() # reads CO_API_KEY
// OpenAI — npm i openai
import OpenAI from "openai";
const openai = new OpenAI(); // reads OPENAI_API_KEY
// Anthropic — npm i @anthropic-ai/sdk
import Anthropic from "@anthropic-ai/sdk";
const anthropic = new Anthropic(); // reads ANTHROPIC_API_KEY
// Google Gemini — npm i @google/genai (NOT @google/generative-ai)
import { GoogleGenAI } from "@google/genai";
const ai = new GoogleGenAI({}); // reads GEMINI_API_KEY
// Mistral — npm i @mistralai/mistralai
import { Mistral } from "@mistralai/mistralai";
const mistral = new Mistral({ apiKey: process.env.MISTRAL_API_KEY });
Text Generation
OpenAI — Responses API (recommended for new code)
response = client.responses.create(
model="gpt-6-astra",
instructions="You are a helpful assistant.", # system prompt
input="Explain REST in one sentence.",
max_output_tokens=256,
)
print(response.output_text)
Multi-turn: pass a list of {"role": ..., "content": ...} items as input, or chain with previous_response_id=response.id.
OpenAI — Chat Completions (still supported; same shape Mistral/Cohere copy)
response = client.chat.completions.create(
model="gpt-6-astra",
messages=[
{"role": "system", "content": "You are a helpful assistant."}, # "developer" also accepted
{"role": "user", "content": "Explain REST in one sentence."},
],
max_completion_tokens=256, # max_tokens is deprecated for reasoning models
)
print(response.choices[0].message.content)
Anthropic
response = client.messages.create(
model="claude-opus-5-5",
max_tokens=1024, # required
system="You are a helpful assistant.", # system prompt is a separate parameter
messages=[
{"role": "user", "content": "Explain REST in one sentence."},
],
output_config={"effort": "low"}, # low | medium | high | xhigh | max
)
print(next(b.text for b in response.content if b.type == "text"))
Current Claude models think adaptively by default; tune depth/cost with output_config.effort instead of temperature (removed — see gotchas).
Google Gemini (google-genai)
response = client.models.generate_content(
model="gemini-3.8-flash",
contents="Explain REST in one sentence.",
config=types.GenerateContentConfig(
system_instruction="You are a helpful assistant.",
max_output_tokens=256,
),
)
print(response.text)
# Multi-turn (SDK keeps history, resends it each turn)
chat = client.chats.create(model="gemini-3.8-flash")
print(chat.send_message("I have 2 dogs.").text)
print(chat.send_message("How many paws?").text)
Google's newer Interactions API (client.interactions.create(model=..., input=..., system_instruction=...) → interaction.output_text, multi-turn via previous_interaction_id) keeps state server-side; generate_content remains the central, supported endpoint.
Mistral
response = client.chat.complete(
model="mistral-large-latest",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain REST in one sentence."},
],
temperature=0.7,
max_tokens=256,
)
print(response.choices[0].message.content)
Cohere
response = client.chat(
model="command-a-plus-05-2026",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain REST in one sentence."},
],
temperature=0.7,
max_tokens=256,
)
print(response.message.content[0].text)
Side-by-Side Differences
| Aspect | OpenAI (Responses) | Anthropic | Gemini | Mistral | Cohere |
|---|---|---|---|---|---|
| Call | responses.create | messages.create | models.generate_content | chat.complete | chat |
| System prompt | instructions= (or system/developer item) | Separate system param | system_instruction in config | In messages array | In messages array |
| Response path | .output_text | .content[] text block | .text | .choices[0].message.content | .message.content[0].text |
| Max tokens | max_output_tokens (optional) | max_tokens required | max_output_tokens (optional) | max_tokens (optional) | max_tokens (optional) |
| Sampling params | Model-dependent | temperature/top_p/top_k removed (400) | Ignored on Gemini 4 / 3.6+ | Supported | Supported |
| Conversation state | Stateless or previous_response_id | Stateless (resend history) | Stateless (chats helper) or Interactions | Stateless | Stateless |
Streaming
OpenAI
stream = client.responses.create(
model="gpt-6-astra",
input="Write a haiku about APIs.",
stream=True,
)
for event in stream:
if event.type == "response.output_text.delta":
print(event.delta, end="")
elif event.type == "response.completed":
print()
Chat Completions equivalent: stream=True, then chunk.choices[0].delta.content.
Anthropic
with client.messages.stream(
model="claude-opus-5-5",
max_tokens=1024,
messages=[{"role": "user", "content": "Write a haiku about APIs."}],
) as stream:
for text in stream.text_stream:
print(text, end="")
final = stream.get_final_message() # full Message incl. usage
Use streaming for anything with large max_tokens — the SDKs refuse very large non-streaming requests to avoid HTTP timeouts.
Google Gemini
for chunk in client.models.generate_content_stream(
model="gemini-3.8-flash",
contents="Write a haiku about APIs.",
):
print(chunk.text, end="")
Mistral
stream = client.chat.stream(
model="mistral-large-latest",
messages=[{"role": "user", "content": "Write a haiku about APIs."}],
)
for event in stream:
if event.data.choices[0].delta.content:
print(event.data.choices[0].delta.content, end="")
Cohere
for event in client.chat_stream(
model="command-a-plus-05-2026",
messages=[{"role": "user", "content": "Write a haiku about APIs."}],
):
if event.type == "content-delta":
print(event.delta.message.content.text, end="")
Function Calling / Tool Use
OpenAI (Responses API)
import json
tools = [{
"type": "function",
"name": "get_weather", # flat: no nested "function" key in Responses
"description": "Get current weather for a location",
"strict": True,
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "City name"},
},
"required": ["location"],
"additionalProperties": False, # required with strict
},
}]
input_list = [{"role": "user", "content": "Weather in Tokyo?"}]
response = client.responses.create(model="gpt-6-astra", input=input_list, tools=tools)
input_list += response.output # keep the model's function_call items
for item in response.output:
if item.type == "function_call":
args = json.loads(item.arguments) # arguments is a JSON string
input_list.append({
"type": "function_call_output",
"call_id": item.call_id,
"output": "22°C, cloudy",
})
response = client.responses.create(model="gpt-6-astra", input=input_list, tools=tools)
print(response.output_text)
tool_choice: "auto" (default) · "required" · "none" · {"type": "function", "name": "get_weather"} · {"type": "allowed_tools", "mode": "auto", "tools": [...]} (restrict without changing the tool list, which keeps the cache warm).
OpenAI (Chat Completions)
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "City name"},
},
"required": ["location"],
},
},
}]
response = client.chat.completions.create(
model="gpt-6-astra",
messages=[{"role": "user", "content": "Weather in Tokyo?"}],
tools=tools,
tool_choice="auto",
)
# Check for tool calls
if response.choices[0].message.tool_calls:
tc = response.choices[0].message.tool_calls[0]
print(f"Call: {tc.function.name}({tc.function.arguments})")
Anthropic
tools = [{
"name": "get_weather",
"description": "Get current weather for a location",
"strict": True, # optional: guarantees input matches the schema
"input_schema": { # Note: input_schema, not parameters
"type": "object",
"properties": {
"location": {"type": "string", "description": "City name"},
},
"required": ["location"],
"additionalProperties": False,
},
}]
messages = [{"role": "user", "content": "Weather in Tokyo?"}]
response = client.messages.create(
model="claude-opus-5-5",
max_tokens=1024,
messages=messages,
tools=tools,
)
if response.stop_reason == "tool_use":
messages.append({"role": "assistant", "content": response.content}) # keep ALL blocks
results = []
for block in response.content:
if block.type == "tool_use":
print(f"Call: {block.name}({block.input}) id={block.id}")
results.append({"type": "tool_result", "tool_use_id": block.id, "content": "22°C, cloudy"})
messages.append({"role": "user", "content": results}) # all results in ONE user message
response = client.messages.create(model="claude-opus-5-5", max_tokens=1024, messages=messages, tools=tools)
The Python SDK can run this loop for you: @beta_tool functions + client.beta.messages.tool_runner(...).
Google Gemini
get_weather = types.FunctionDeclaration(
name="get_weather",
description="Get current weather for a location",
parameters={
"type": "object",
"properties": {
"location": {"type": "string", "description": "City name"},
},
"required": ["location"],
},
)
config = types.GenerateContentConfig(tools=[types.Tool(function_declarations=[get_weather])])
contents = [types.Content(role="user", parts=[types.Part(text="Weather in Tokyo?")])]
response = client.models.generate_content(model="gemini-3.8-flash", contents=contents, config=config)
for fc in response.function_calls or []:
print(f"Call: {fc.name}({fc.args})")
contents.append(response.candidates[0].content) # model turn
contents.append(types.Content(role="user", parts=[
types.Part.from_function_response(name=fc.name, response={"result": "22°C, cloudy"}, id=fc.id),
]))
response = client.models.generate_content(model="gemini-3.8-flash", contents=contents, config=config)
# Python-only shortcut: automatic function calling — pass the callable itself
def get_current_temperature(location: str) -> str:
"""Get current temperature for a city."""
return "22°C"
client.models.generate_content(
model="gemini-3.8-flash", contents="Temperature in Tokyo?",
config=types.GenerateContentConfig(tools=[get_current_temperature]),
)
Mistral
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "City name"},
},
"required": ["location"],
},
},
}]
response = client.chat.complete(
model="mistral-large-latest",
messages=[{"role": "user", "content": "Weather in Tokyo?"}],
tools=tools,
tool_choice="auto",
)
# Same structure as OpenAI Chat Completions
if response.choices[0].message.tool_calls:
tc = response.choices[0].message.tool_calls[0]
print(f"Call: {tc.function.name}({tc.function.arguments})")
Cohere uses the same {"type": "function", "function": {...}} wrapper (cohere.ToolV2); calls come back in response.message.tool_calls with finish_reason == "TOOL_CALL".
Tool Calling Syntax Comparison
| Aspect | OpenAI Responses | OpenAI Chat / Mistral / Cohere | Anthropic | Gemini |
|---|---|---|---|---|
| Schema key | parameters | parameters | input_schema | parameters |
| Tool wrapper | {"type": "function", "name": ...} (flat) | {"type": "function", "function": {...}} | {...} (flat) | types.Tool(function_declarations=[...]) |
| Response location | output[] items with type == "function_call" | .message.tool_calls[].function | .content[] (type == "tool_use") | response.function_calls |
| Arguments | JSON string (json.loads) | JSON string | Parsed dict (block.input) | Dict (fc.args) |
| Tool result | {"type": "function_call_output", "call_id": ...} | {"role": "tool", "tool_call_id": ...} | user message with tool_result blocks | Part.from_function_response(...) in a user turn |
| Force a tool | {"type": "function", "name": ...} | {"type": "function", "function": {"name": ...}} | Not on Opus 5.5 / Sonnet 5.5 / Fable 5.1 — use auto + prompt or structured output | FunctionCallingConfig(mode="ANY") |
| Parallel calls | Yes | Yes | Yes | Yes |
Tool Result Format: Anthropic Is Different
Anthropic requires tool results to be sent as a user message with a tool_result content block, not as a separate tool role:
# OpenAI Chat Completions / Mistral
{"role": "tool", "tool_call_id": "call_123", "content": "22°C, cloudy"}
# OpenAI Responses
{"type": "function_call_output", "call_id": "call_123", "output": "22°C, cloudy"}
# Anthropic
{"role": "user", "content": [
{"type": "tool_result", "tool_use_id": "toolu_123", "content": "22°C, cloudy"}
]}
Structured Output
from pydantic import BaseModel
from typing import Literal
class Language(BaseModel):
name: str
type: Literal["compiled", "interpreted", "jit"]
class Languages(BaseModel):
languages: list[Language]
PROMPT = "List 3 programming languages with their types."
# OpenAI — Pydantic helper (Responses API)
response = client.responses.parse(model="gpt-6-astra", input=PROMPT, text_format=Languages)
langs = response.output_parsed
# OpenAI — raw JSON schema (strict: every property required, additionalProperties false)
response = client.responses.create(
model="gpt-6-astra",
input=PROMPT,
text={"format": {
"type": "json_schema",
"name": "languages",
"strict": True,
"schema": {
"type": "object",
"properties": {
"languages": {
"type": "array",
"items": {
"type": "object",
"properties": {
"name": {"type": "string"},
"type": {"type": "string", "enum": ["compiled", "interpreted", "jit"]},
},
"required": ["name", "type"],
"additionalProperties": False,
},
},
},
"required": ["languages"],
"additionalProperties": False,
},
}},
)
# Chat Completions equivalent: response_format={"type": "json_schema", "json_schema": {"name": ..., "schema": ..., "strict": True}}
# Anthropic — Pydantic helper
response = client.messages.parse(
model="claude-opus-5-5", max_tokens=1024,
messages=[{"role": "user", "content": PROMPT}],
output_format=Languages,
)
langs = response.parsed_output
# Anthropic — raw JSON schema (native; no forced tool needed)
response = client.messages.create(
model="claude-opus-5-5", max_tokens=1024,
messages=[{"role": "user", "content": PROMPT}],
output_config={"format": {"type": "json_schema", "schema": Languages.model_json_schema()}},
)
# Gemini — JSON schema
response = client.models.generate_content(
model="gemini-3.8-flash",
contents=PROMPT,
config={
"response_mime_type": "application/json",
"response_json_schema": Languages.model_json_schema(),
},
)
langs = Languages.model_validate_json(response.text)
# Mistral / Cohere — JSON schema via response_format
response = client.chat.complete(
model="mistral-large-latest",
messages=[{"role": "user", "content": PROMPT}],
response_format={"type": "json_schema",
"json_schema": {"name": "languages", "schema": Languages.model_json_schema(), "strict": True}},
)
Old Anthropic pattern
Forcing a tool (tool_choice={"type": "tool", "name": ...}) to get JSON returns a 400 on Claude Opus 5.5, Sonnet 5.5 and Fable 5.1. Use output_config.format (or client.messages.parse). The old top-level output_format request parameter is deprecated — output_format= is only the parse() helper's argument.
Prompt Caching
| Provider | How | Read discount | Check hits in |
|---|---|---|---|
| OpenAI | Automatic on prompts ≥ 1,024 tokens; optional prompt_cache_key; explicit breakpoints on newer models | Up to 95% off input | usage.input_tokens_details.cached_tokens |
| Anthropic | cache_control={"type": "ephemeral"} top-level (automatic) or on blocks (max 4 breakpoints); TTL 5m default or "ttl": "1h" | Reads 0.1× base (0.05× on Opus 5.5 / Sonnet 5.5, 0.025× on Fable 5.1); writes 1.25× (5m) / 2× (1h) | usage.cache_read_input_tokens, usage.cache_creation_input_tokens |
| Gemini | Implicit caching on by default (min 4,096 tokens on Gemini 4 / 3.8 Flash); explicit client.caches.create(...) on generate_content for Gemini 3 / 2.5 only | See pricing page | usage_metadata.cached_content_token_count (Interactions: usage.total_cached_tokens) |
# Anthropic — cache a large, stable system prompt
response = client.messages.create(
model="claude-opus-5-5",
max_tokens=1024,
system=[{"type": "text", "text": BIG_STABLE_DOC, "cache_control": {"type": "ephemeral"}}],
messages=[{"role": "user", "content": "Summarize section 3."}],
)
print(response.usage.cache_read_input_tokens)
Caching is a prefix match: put stable content (tools, system prompt, documents) first and volatile content (timestamps, the user's question) last. Minimum cacheable prefix on current Claude models is 512 tokens; shorter prompts silently don't cache.
Embeddings
| Provider | Model | Default dims | Max input tokens |
|---|---|---|---|
| OpenAI | text-embedding-3-large | 3072 (reducible via dimensions) | 8192 |
| OpenAI | text-embedding-3-small | 1536 (reducible) | 8192 |
gemini-embedding-2 (multimodal) | 3072 (128–3072) | 8192 | |
gemini-embedding-001 (text) | 3072 (128–3072) | 2048 | |
| Mistral | mistral-embed | 1024 | see docs |
| Cohere | embed-v4.0 | 1536 (256–1536) | 128K |
| Cohere | embed-v5.0-pro / embed-v5.0-fast | 2048 (256–2048) | 128K |
| Voyage | voyage-4-large / voyage-4 / voyage-4-lite | 1024 (256–2048) | 32000 |
Prices: see each provider's pricing page — embeddings are usually a few cents per 1M tokens.
# OpenAI
resp = client.embeddings.create(model="text-embedding-3-small", input=["Hello world"])
vector = resp.data[0].embedding # list[float], length 1536
# Cohere
resp = client.embed(
model="embed-v4.0",
texts=["Hello world"],
input_type="search_document", # or "search_query"
embedding_types=["float"],
)
vector = resp.embeddings.float_[0]
# Mistral
resp = client.embeddings.create(model="mistral-embed", inputs=["Hello world"])
vector = resp.data[0].embedding
# Google
result = client.models.embed_content(
model="gemini-embedding-2",
contents="Hello world",
config=types.EmbedContentConfig(output_dimensionality=768),
)
vector = result.embeddings[0].values
WARNING
Embedding spaces are model-specific: switching models (even gemini-embedding-001 → gemini-embedding-2) means re-embedding your corpus. Smaller-than-default dimensions from gemini-embedding-001 must be normalized yourself.
Error Handling Pattern
The official SDKs already retry 408/409/429/5xx and connection errors with backoff (max_retries, default 2 for OpenAI and Anthropic). Tune that first; catch the typed exceptions for anything else:
import anthropic, openai
client = anthropic.Anthropic(max_retries=5, timeout=60.0)
try:
response = client.messages.create(...)
except anthropic.RateLimitError as e: # 429 after retries — back off / queue
...
except anthropic.APIStatusError as e: # other non-2xx: e.status_code, e.response
...
except anthropic.APIConnectionError: # network
...
# OpenAI has the same names: openai.RateLimitError, openai.APIStatusError, openai.APIConnectionError
Provider-agnostic fallback (for SDKs without built-in retries):
import time
def call_with_retry(fn, max_retries=3, base_delay=1.0):
"""Universal retry pattern for any LLM API."""
for attempt in range(max_retries + 1):
try:
return fn()
except Exception as e:
error_type = type(e).__name__
# Rate limit — always retry with backoff
if "rate" in error_type.lower() or "429" in str(e):
delay = base_delay * (2 ** attempt)
print(f"Rate limited. Retrying in {delay}s...")
time.sleep(delay)
continue
# Overloaded / server error — retry
if "overloaded" in str(e).lower() or "500" in str(e) or "529" in str(e):
delay = base_delay * (2 ** attempt)
time.sleep(delay)
continue
# Authentication, invalid request — do not retry
if "401" in str(e) or "400" in str(e) or "authentication" in str(e).lower():
raise
# Unknown error on last attempt
if attempt == max_retries:
raise
raise RuntimeError("Max retries exceeded")
Quick Decision Guide
| I need... | Use |
|---|---|
| Best all-around model | claude-opus-5-5 or gpt-6-astra |
| Most capable, cost no object | claude-fable-5-1 or gpt-6-astra |
| Cheapest for simple tasks | gpt-6-luna or claude-haiku-5-5 (both from $0.10 / $0.50 per 1M) |
| Long context | All current flagships: ~1M tokens (OpenAI 1.05M; Claude, Gemini 4 Argon 1M) |
| Best reasoning | claude-fable-5-1, gpt-6-astra, or gemini-4-argon |
| EU data residency / open weights | Mistral Large 4 / Small 4 |
| Best RAG support | Cohere Command A+ with Rerank v5 |
| Image understanding | Any current flagship (GPT-6, Claude 5.5, Gemini, Mistral Large 4) |
| Audio processing | Gemini (native audio), OpenAI audio models, Mistral Voxtral |
| Embeddings | OpenAI text-embedding-3-*, gemini-embedding-2, Cohere embed-v5.0, Voyage 4 — compare on your own retrieval eval |
See Also
- OpenAI API Patterns — Deep dive into OpenAI-specific patterns
- Anthropic Claude API Patterns — Deep dive into Claude-specific patterns
- LLM Integration Patterns — Provider-agnostic architecture
- Embeddings Deep Dive — Embedding strategies beyond the API
- AI Agents Architecture — Building agents with tool calling
Test Yourself
-
What is the key difference in how Anthropic handles the system prompt vs OpenAI? Anthropic uses a separate
systemparameter; OpenAI Responses usesinstructions=(Chat Completions puts it in the messages array withrole: "system"). -
How does Anthropic return tool call results differently from OpenAI? Anthropic requires tool results as a
usermessage withtool_resultcontent blocks; OpenAI Chat Completions uses atoolrole and Responses usesfunction_call_outputitems. -
What parameter is required for Anthropic but optional for OpenAI?
max_tokensis required for Anthropic. -
What context window do current flagships offer (as of November 2026)? About 1M tokens: GPT-6 models 1.05M, Claude 5.5 / Fable 5.1 1M, Gemini 4 Argon 1M.
-
How do you get schema-valid JSON from current Claude models?
output_config={"format": {"type": "json_schema", "schema": ...}}orclient.messages.parse(..., output_format=Model)— forcing a tool viatool_choicereturns a 400 on Opus 5.5 / Sonnet 5.5 / Fable 5.1. -
Which Python package replaces
google-generativeai?google-genai(from google import genai; client = genai.Client()). -
What key name does Anthropic use for the tool schema (vs OpenAI's
parameters)?input_schema -
How do you access the response text from an OpenAI Responses API call?
response.output_text -
What OpenAI feature guarantees the response matches a JSON schema? Structured Outputs with
strict: True—text={"format": {"type": "json_schema", ...}}in Responses (orresponses.parse(text_format=Model)). -
What retry strategy should you use for rate limit (429) errors? Exponential backoff — the official SDKs do it for you (
max_retries); otherwisedelay = base_delay * (2 ** attempt), honouringretry-after.
Common Gotchas
- Anthropic requires
max_tokens-- forgetting it raises an error. OpenAI and others default to a reasonable value, but Anthropic forces you to be explicit. - Anthropic removed
temperature,top_p,top_k. On API version2026-10-21(sent by current SDKs) any of them returns a 400 on every model. Useoutput_config.effortand prompting instead. Gemini 4 / 3.6+ ignore them and will 400 in a future release. - No assistant prefill on current Claude models — a trailing
assistantmessage returns a 400. Use structured output or instructions. - Tool result format differs between providers. OpenAI Chat/Mistral use
role: "tool", OpenAI Responses usesfunction_call_output, Anthropic usesrole: "user"with atool_resultblock. Mixing formats causes errors. - OpenAI/Mistral/Cohere tool arguments are JSON strings — always
json.loads(); never string-match. - Return every parallel tool result together. Anthropic expects all
tool_resultblocks in oneusermessage; OpenAI expects one output percall_id. - Don't add date suffixes to new Claude IDs (
claude-opus-5-5, notclaude-opus-5-5-2026...). - Old Gemini SDK code silently drifts.
google.generativeai/@google/generative-aiare legacy; new features land only ingoogle-genai/@google/genai. - Check the stop reason before reading content. Anthropic can return
stop_reason: "refusal"or"max_tokens"; truncated JSON won't parse. - LLM pricing changes frequently. Architectural decisions based on pricing can become outdated in weeks. Always check current pricing on each provider's website.
temperature=0does not guarantee determinism. Where temperature is still supported, providers say "mostly deterministic" at temperature 0 but do not guarantee identical outputs across requests or model versions.
One-Liner Summary
LLM APIs follow similar patterns across providers (text generation, tool calling, structured output, streaming, caching) but differ in subtle ways -- know each provider's system prompt placement, response paths, tool calling syntax and which parameters are gone (Anthropic sampling params, Gemini's old SDK) to switch between them effortlessly.