Technology

batch inference

1 synchronized story carry this tag.

LiteLLM 1.99 stabilises OAuth login and batch-cost accounting

LiteLLM 1.99.0 adds an OAuth authorization-code flow with PKCE for its CLI, stores credentials and refresh tokens in the operating-system keychain, can configure Claude Code at login, and accounts for batch spend through enqueue limits, refunds and atomic cost claims designed to avoid double billing across polling workers. It also exposes configurable routing classifiers and provisioned-throughput declarations.

Why it matters: A model gateway now sits directly on identity, credentials, routing, rate limits and cost attribution. Mistakes in those paths can cause secret leakage, duplicate spend or traffic shifts even when the underlying model works correctly.

Read storyOriginal source · docs.litellm.ai