AWS adds NVIDIA Nemotron 3.5 Super VL to SageMaker JumpStart

AWS has made NVIDIA Nemotron 3.5 Super VL available through Amazon SageMaker JumpStart, giving customers a managed way to deploy the multimodal model from the JumpStart catalog in SageMaker Studio or with SageMaker Python SDK v3. The SDK route requires version 3.25 or later. The catalog also hosts an OpenFold3 preview in SageMaker JumpStart.

For NVFP4, JumpStart defaults to ml.g7.24xlarge and also supports ml.g7.48xlarge; both use NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs. The BF16 checkpoint runs on ml.p5en.48xlarge with NVIDIA H200 GPUs. AWS lists the optimized NVFP4 checkpoint at about 75 GB and the full-precision BF16 reference weights at about 249 GB, positioning BF16 for customization and evaluation.

Nemotron 3.5 Super VL is a 120-billion-parameter hybrid mixture-of-experts vision-language model with 12 billion active parameters. It accepts text, images and video and returns text. The JumpStart addition is a deployment and distribution update, not the model’s debut: NVIDIA introduced the model and its agent-focused multimodal capabilities on October 15, before AWS added the managed endpoint path on November 9.

JumpStart’s serving presets are narrower than the model’s underlying capability. NVIDIA’s BF16 model card and NVFP4 model card describe context support up to 1 million tokens. AWS, however, caps the combined prompt and completion length at 520 tokens for interact, 2,275 for generate, 13,650 for summarize and 15,600 for modify. Images and video also consume that budget.

A request can contain no more than one image and one video. Raw PDF files are not accepted, so users must render a PDF page as an image and send one page per request, or stitch multiple pages into a single image. Video must fit the endpoint payload limit. Frame sampling is fixed when an endpoint starts, and the service rejects per-request media_io_kwargs. AWS did not specify the payload limit in bytes.

The endpoint serves base weights only and does not load LoRA adapters for either precision variant. Although the model has a native Multi-Token Prediction head, speculative decoding is disabled in the JumpStart configuration. Reasoning is enabled by default and can be changed per request with enable_thinking; tool calling and JSON-schema output are supported.

In a test using 250 input tokens, 1,500 output tokens and five concurrent streaming requests, AWS reported mean per-request output throughput of 63.07 tokens per second on ml.g7.24xlarge NVFP4, 64.01 on ml.g7.48xlarge NVFP4 and 140.85 on ml.p5en.48xlarge BF16. Mean time to first token ranged from 0.45 to 0.53 seconds. AWS cautioned that the tests used different GPU systems and configurations, so the results do not isolate the effect of BF16 versus NVFP4 precision.

New AWS accounts may need to request a quota increase before deploying the G7 configurations because SageMaker endpoint quotas for ml.g7 instances start at zero. AWS did not give regional availability or endpoint pricing in the announcement.