What changed

Amazon SageMaker AI now monitors eligible vLLM GPU inference endpoints by default for week-over-week latency drift, unused capacity, stale serving containers and missed optimizations, then sends an AWS Health finding that can open a separate benchmark job in SageMaker Studio.

Why it matters

Operational drift can waste GPU spend or degrade time to first token without tripping a failure alarm. The new signal separates observation from a benchmark and keeps production changes behind an explicit decision.

What to do

Inventory eligible endpoints, decide whether telemetry collection is acceptable, verify one AWS Health finding end to end, benchmark on separate compute, and roll any winning configuration to one variant with rollback preserved.

Caveat

AWS defines the detection logic and controls but does not publish independent false-positive or savings results. Monitoring is opt-out, observes endpoint telemetry, benchmarks cost money and no production change is automatic.