Source documents checked: October 3, 2026. Previous update label: July 24, 2026.
The featured image illustrates the motion-forecasting concept; it is not a MolmoMotion test result.
TL;DR: MolmoMotion predicts future 3D point trajectories from a short visual history, query points on an object, and a language instruction. Ai2 released autoregressive checkpoints, code, trajectory annotations, and a benchmark for research. The approach is relevant to robot planning and controlled video generation, but the published results come from the project team and the model still uses a sparse set of points with limited closed-loop evaluation.
What MolmoMotion predicts
A motion tracker explains where an object moved. MolmoMotion addresses the next step: where selected points on an object are likely to move after an instruction. The paper defines a metric 3D world frame anchored to the camera at the reference time, t₀; future coordinates stay in that frame rather than following later camera motion. The released implementation takes 2D query points with their 3D history and predicts camera-frame coordinates in meters. Preparing those inputs requires attention to depth and camera calibration.
The project describes two variants. MolmoMotion-AR emits quantized coordinates in sequence, while MolmoMotion-FM uses flow matching to generate continuous trajectories. Both use Molmo 2 as the vision-language backbone that connects the instruction to the target object and its query points. The current repository says all released checkpoints are autoregressive; the collection lists H3-F30 and H1-F32 checkpoints. The paper’s flow-matching results should not be read as confirmation of a downloadable FM checkpoint.
Released research assets
Ai2’s June 17 release article links the model collection, MolmoMotion-1M data, PointMotionBench, the technical report, and public code. That June article describes MolmoMotion-1M as drawn from 1.16 million videos, pairing action descriptions with object-grounded 3D trajectories, and PointMotionBench as about 2,700 human-validated clips across 111 object categories and 61 motion types.
The current benchmark card lists 90 DAVIS, 2,475 HOT3D, and 155 WorldTrack clips, totaling 2,720. The June 17 arXiv v1 report documents a 742-clip evaluation set: 497 HOT3D, 155 WorldTrack, and 90 DAVIS clips. Appendix C explains that its HOT3D subset comes from subsampling 2,534 sub-clips to 507, then excluding 10. The reviewed sources do not establish how the current card’s 2,475 HOT3D clips correspond to that 497-clip evaluation subset. Record the exact benchmark version and subsets when reproducing results; do not assume the downloadable total is the paper’s evaluation set.
The project team says it built the data pipeline because existing collections did not combine large-scale video, object-grounded 3D paths, and action descriptions. The pipeline tracks points in 2D, lifts them into a shared 3D frame, filters incoherent motion, smooths trajectories, and clips each sample around the meaningful action.
Robotics and video use cases
For robotics, a predicted object path can initialize or guide a manipulation policy. Ai2 reports that a policy initialized from MolmoMotion reached a higher pick-and-place success rate than its Molmo 2 baseline in the team’s simulated MolmoSpaces experiment. Its DROID results evaluate predicted trajectories on held-out robot videos; the paper leaves closed-loop real-robot evaluation to future work. These are source-reported research results, not a guarantee for a different robot, camera, object set, or controller.
For video generation, predicted trajectories can act as an explicit motion-control signal. The paper evaluates this by conditioning an image-to-video system on the paths and comparing several motion metrics. Teams should reproduce the result on their own scene types before treating it as evidence of general video quality.
Readers comparing research releases through AI News should keep the distinction between an open research artifact and a supported production service. The reviewed official release materials do not specify a hosted commercial SLA or a managed API price, and the benchmark is not a promise of performance in a buyer’s environment.
Limits that matter
The project page says MolmoMotion uses eight query points per object during training. That sparse representation cannot describe dense surface geometry, and complex deformable motion remains difficult. The authors also identify broader real-world and closed-loop robot evaluation as future work.
The repository labels code and trained weights Apache 2.0, but the training-data card lists separate subset licenses, including CC BY-NC 4.0 and CC BY 4.0. Source videos and benchmark inputs remain subject to their upstream terms and may need local reconstruction. The repository also warns that vendored data-generation dependencies are not all Apache 2.0. Inspect the applicable artifact and upstream terms, input preprocessing, compute requirements, failure cases, and any safety boundary around a physical controller. A forecast should be treated as a model output that a control system can reject, not as a command that automatically reaches hardware.
Research artifact versus product
The public release is valuable because its paper, code, weights, data, and benchmark can be inspected together. It still requires engineering work to reproduce the environment, prepare inputs, choose query points, measure latency, and connect a forecast to a downstream system. The reviewed official release materials do not provide a managed-service commitment covering uptime, support, privacy, or production incidents.
That distinction should shape the evaluation. A research team may accept manual setup to study a new representation. A product team needs an owner for model updates, data provenance, monitoring, rollback, and the operational cost of keeping the pipeline working.
Alternatives to compare
The published study compares MolmoMotion with constant-velocity extrapolation, pixel-space video generators, and parametric 3D methods. For an evaluation, choose baselines that match the output you need: selected-point paths, full video frames, or a category-specific 3D representation. Those are different tasks and representations; the authors’ benchmark results do not replace a comparison on your own inputs.
Practical evaluation
- Reproduce one published example with the released checkpoint and code.
- Inspect how query points and camera calibration affect the predicted path.
- Test rigid, articulated, and deformable objects separately.
- Compare the released checkpoints on accuracy, latency, and compute.
- Keep any robot trial behind collision checks, speed limits, and human supervision.
Official sources
Related Kingy AI links
The Kingy Brief
Get the free AI price sheet.
26 AI plans and models on one page, from official pricing pages, updated every month. Free when you join The Kingy Brief: AI changes, original tests and one practical thing to try, Fridays at 09:00 Vancouver time.
Free · Double opt-in · Unsubscribe anytime
Sign up for The Kingy Brief on Beehiiv.
Signup help and newsletter schedule
Signup form provided by Beehiiv. After submitting, check your inbox for "Confirm your subscription to The Kingy Brief" and open its confirmation link. Your price sheet download is on the page that opens. Check Spam or Promotions if you cannot find it.
Fridays at 09:00 Vancouver time: source-checked AI changes, original tests and one practical thing to try. We skip a week when there is not enough verified material. Free. Unsubscribe anytime.