[Preview] v1.79.1-stable - FAL AI Support
Deploy this version
- Docker
- Pip
docker run litellm
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:v1.80.0-stable
pip install litellm
pip install litellm==1.80.0
Key Highlights
- Container API Support - End-to-end OpenAI Container API support with proxy integration, logging, and cost tracking
- FAL AI Image Generation - Native support for FAL AI image generation models with cost tracking
- UI Enhancements - Guardrail Playground, Cache Settings, Tag Routing, SSO Settings
- Batch API Rate Limiting - Input-based rate limits support for Batch API requests
- Vector Store Expansion - Milvus vector store support and Azure AI virtual indexes
- Memory Leak Fixes - Resolved issues accounting for 90% of memory leaks on Python SDK & AI Gateway
Dependency Upgrades
- Dependencies
New Models / Updated Models
New Model Support
| Provider | Model | Context Window | Input ($/1M tokens) | Output ($/1M tokens) | Features |
|---|---|---|---|---|---|
| Mistral | mistral/codestral-embed | 8K | $0.15 | - | Embeddings |
| Mistral | mistral/codestral-embed-2505 | 8K | $0.15 | - | Embeddings |
| Gemini | gemini/gemini-embedding-001 | 2K | $0.15 | - | Embeddings |
| FAL AI | fal_ai/fal-ai/flux-pro/v1.1-ultra | - | - | - | Image generation - $0.0398/image |
| FAL AI | fal_ai/fal-ai/imagen4/preview | - | - | - | Image generation - $0.0398/image |
| FAL AI | fal_ai/fal-ai/recraft/v3/text-to-image | - | - | - | Image generation - $0.0398/image |
| FAL AI | fal_ai/fal-ai/stable-diffusion-v35-medium | - | - | - | Image generation - $0.0398/image |
| FAL AI | fal_ai/bria/text-to-image/3.2 | - | - | - | Image generation - $0.0398/image |
| OpenAI | openai/sora-2-pro | - | - | - | Video generation - $0.30/video/second |
Features
-
- Extended Claude 3-7 Sonnet deprecation date from 2026-02-01 to 2026-02-19 - PR #15976
- Extended Claude Opus 4-0 deprecation date from 2025-03-01 to 2026-05-01 - PR #15976
- Removed Claude Haiku 3-5 deprecation date (previously 2025-03-01) - PR #15976
- Added Claude Opus 4-1, Claude Opus 4-0 20250513, Claude Sonnet 4 20250514 deprecation dates - PR #15976
- Added web search support for Claude Opus 4-1 - PR #15976
-
- Fix empty assistant message handling in AWS Bedrock Converse API to prevent 400 Bad Request errors - PR #15850
- Allow using ARNs when generating images via Bedrock - PR #15789
- Add per model group header forwarding for Bedrock Invoke API - PR #16042
- Preserve Bedrock inference profile IDs in health checks - PR #15947
- Added fallback logic for detecting file content-type when S3 returns generic type - When using Bedrock with S3-hosted files, if the S3 object's Content-Type is not correctly set (e.g., binary/octet-stream instead of image/png), Bedrock can now handle it correctly - PR #15635
-
- Add deprecation dates for Azure OpenAI models (gpt-4o-2024-08-06, gpt-4o-2024-11-20, gpt-4.1 series, o3-2025-04-16, text-embedding-3-small) - PR #15976
- Fix Azure OpenAI ContextWindowExceededError mapping from Azure errors - PR #15981
- Add handling for
v1under Azure API versions - PR #15984 - Fix azure doesn't accept extra body param - PR #16116
-
- Add codestral-embed-2505 embedding model - PR #16071
-
- Add vision support for Qwen/Qwen3-chat-32b model - PR #15976
-
- Fix vercel_ai_gateway entry for glm-4.6 (moved from vercel_ai_gateway/glm-4.6 to vercel_ai_gateway/zai/glm-4.6) - PR #16084
-
- Don't add "accounts/fireworks/models" prefix for Fireworks Provider - PR #15938
-
- Add OpenAI-compatible annotations support for Cohere v2 citations - PR #16038
-
- Handle Deepgram detected language when available - PR #16093
Bug Fixes
New Provider Support
-
- Add FAL AI Image Generation support - PR #16067
-
OCI (Oracle Cloud Infrastructure)
- Add OCI Signer Authentication support - PR #16064
LLM API Endpoints
Features
-
- Respect
LiteLLM-Disable-Message-Redactionheader for Responses API - PR #15966 - Add /openai routes for responses API (Azure OpenAI SDK Compatibility) - PR #15988
- Redact reasoning summaries in ResponsesAPI output when message logging is disabled - PR #15965
- Support text.format parameter in Responses API for providers without native ResponsesAPIConfig - PR #16023
- Add LLM provider response headers to Responses API - PR #16091
- Respect
-
- Moderations endpoint now respects
api_baseconfiguration parameter - PR #16087
- Moderations endpoint now respects
-
- Support multi-part form data on passthrough - PR #16035
Management Endpoints / UI
Features
-
Virtual Keys
-
Models + Endpoints
-
Guardrails
-
Cache Settings
- Allow setting cache settings on UI - PR #16143
-
Routing
- Allow setting all routing strategies, tag filtering on UI - PR #16139
-
Admin Settings
Logging / Guardrail / Prompt Management Integrations
Features
-
- Ensure key's metadata + guardrail is logged on DD - PR #15980
-
- Add Base64 handling for SQS Logger - PR #16028
-
General
- Fix: User API key and team id and user id missing from custom callback is not misfiring - PR #15982
Guardrails
-
- Support during_call for model armor guardrails - PR #15970
-
- Upgrade to Lasso API v3 and fix ULID generation - PR #15941
-
- Add per-request profile overrides to PANW Prisma AIRS - PR #16069
-
- Improve Grayswan guardrail documentation - PR #15875
-
- Graceful degradation for pillar service when using litellm - PR #15857
-
General
- Ensure Key Guardrails are applied - PR #16025
Prompt Management
Spend Tracking, Budgets and Rate Limiting
-
Cost Tracking
- Fix spend tracking for OCR/aOCR requests (log
pages_processed+ recognizeOCRResponse) - PR #16070
- Fix spend tracking for OCR/aOCR requests (log
-
Rate Limiting
MCP Gateway
- OAuth
Performance / Loadbalancing / Reliability improvements
-
Memory Leak Fixes
-
Configuration
Documentation Updates
-
Provider Documentation
-
Model References
- Add supports vision field to qwen-vl models in model_prices_and_context_window.json - PR #16106
-
General Documentation
New Contributors
- @RobGeada made their first contribution in PR #15975
- @shanto12 made their first contribution in PR #15946
- @dima-hx430 made their first contribution in PR #15976
- @m-misiura made their first contribution in PR #15971
- @ylgibby made their first contribution in PR #15947
- @Somtom made their first contribution in PR #15909
- @rodolfo-nobrega made their first contribution in PR #16023
- @bernata made their first contribution in PR #15997
- @AlbertDeFusco made their first contribution in PR #15881
- @komarovd95 made their first contribution in PR #15789
- @langpingxue made their first contribution in PR #15635
- @OrionCodeDev made their first contribution in PR #16070
- @sbinnee made their first contribution in PR #16078
- @JetoPistola made their first contribution in PR #16106
- @gvioss made their first contribution in PR #16093
- @pale-aura made their first contribution in PR #16084
- @tanvithakur94 made their first contribution in PR #16041
- @li-boxuan made their first contribution in PR #16044
- @1stprinciple made their first contribution in PR #15938
- @raghav-stripe made their first contribution in PR #16137
- @steve-gore-snapdocs made their first contribution in PR #16149