AI Weekly Malaysia

NVIDIA published Cosmos3-H-Dreams, a world-foundation-model simulator that learns surgical scene dynamics directly from video plus robot kinematics rather than modeling tissue physics explicitly. It was trained on clinical cholecystectomy recordings from the CMR Surgical Versius platform — the team claims this is the first real-time generative surgical simulator trained on clinical cholecystectomy footage with live kinematic control — and runs interactively on a single NVIDIA RTX PRO 6000 GPU using the FlashDreams streaming-inference library. It follows the earlier Cosmos-H-Dreams (built on Cosmos-Predict2.5) which only handled dVRK tabletop suturing.

Why: The concrete shift is the hardware and data recipe: real-time interactive generation on one RTX PRO 6000, driven by video + kinematics instead of a physics engine. If you are prototyping an agent or control loop against a learned environment, this is a template — but note the article's own caveat that visual realism is not proof of biomechanical accuracy, and that small errors accumulate across generated video blocks, so control-following must be validated separately. The post gives no benchmarks, license, or weights/availability details in the text shown, so don't plan around it yet.