Products

Amazon SageMaker Studio

2 synchronized stories carry this tag.

SageMaker starts monitoring inference endpoints for drift and idle GPUs

Amazon SageMaker AI now monitors eligible vLLM GPU inference endpoints by default for week-over-week latency drift, unused capacity, stale serving containers and missed optimizations, then sends an AWS Health finding that can open a separate benchmark job in SageMaker Studio.

Why it matters: Operational drift can waste GPU spend or degrade time to first token without tripping a failure alarm. The new signal separates observation from a benchmark and keeps production changes behind an explicit decision.

Read storyOriginal source · aws.amazon.com

SageMaker JumpStart adds a model leaderboard for quality, speed and cost

Amazon SageMaker Studio now compares more than 70 JumpStart models using AWS-run BBH, GPQA Diamond and MMLU-Pro evaluations plus measured latency, throughput and estimated instance cost for standardized deployment profiles.

Why it matters: A common harness can narrow a model shortlist faster than comparing unrelated model cards, but it remains a provider-run view of selected benchmarks and SageMaker configurations rather than an adoption verdict.

Read storyOriginal source · aws.amazon.com