Running AV Reasoning Models Within Onboard Compute and Latency Limits

Autonomous vehicle (AV) developers need reasoning models that can operate within the compute and latency constraints of an onboard system. Larger models provide a foundation for developing these capabilities, but moving them into a vehicle requires additional work to reduce model size, preserve useful reasoning, and validate performance on the target hardware.

NVIDIA Alpamayo 2 Edge addresses this need with an adaptive reasoning vision-language-action (VLA) model designed for real-time operation on NVIDIA DRIVE AGX Thor. Built on the 2B Nemotron-based reasoning backbone of NVIDIA Cosmos 3 Edge and paired with a diffusion action expert, it leverages surround-camera observations, temporal context, and navigation inputs to generate Chain-of-Causation (CoC) reasoning and future driving trajectories. It reasons adaptively, generating reasoning only when needed, reducing unnecessary token generation and conserving inference compute during routine driving. 

Alpamayo 2 Edge is post-trained with closed-loop reinforcement learning (RL) in NVIDIA AlpaGym, improving closed-loop driving performance by 48.9% on top of the best-performing checkpoint without closed-loop RL. Together, these capabilities give developers a strong, edge-ready foundation for bringing reasoning-based driving into an onboard stack.

Making a reasoning model compact enough for onboard deployment introduces a key challenge: preserving the quality of its reasoning and driving behavior as model size is reduced. This requires a way to measure reasoning quality and use that feedback to improve the model. NVIDIA AlpaJudge is the human-calibrated evaluation and reward model used in Alpamayo 2 Edge development to evaluate reasoning traces, select higher-value training data, and provide RL rewards.

Source: NVIDIA Technical Blog — Read the full article on the original site ↗