SageMaker starts monitoring inference endpoints for drift and idle GPUs
Amazon SageMaker AI now monitors eligible vLLM GPU inference endpoints by default for week-over-week latency drift, unused capacity, stale serving containers and missed optimizations, then sends an AWS Health finding that can open a separate benchmark job in SageMaker Studio.
Why it matters: Operational drift can waste GPU spend or degrade time to first token without tripping a failure alarm. The new signal separates observation from a benchmark and keeps production changes behind an explicit decision.