Can a 15M-Parameter World Model Solve Real Planning Problems?

The Model Was Not Trained on These Tasks — That's the Point

LeWM was trained on four specific simulation environments: Two-Room (2D navigation), Reacher (arm control), Push-T (block manipulation), and OGBench Cube (3D pick-and-place). None of these are warehouses, delivery trucks, or packing lines.

The question is whether the underlying capabilities — planning from pixels, understanding physical constraints, predicting object states — transfer to problems that share structural similarities. We mapped three real-world problems to the closest available environments.

100% Planning Success Rate in Constrained Navigation — Latent Space Preserves Room Topology

A warehouse worker needs to travel between rooms through doorways to pick items efficiently. We tested LeWM's ability to plan navigation paths through the Two-Room environment — a 2D space with a dividing wall and a single doorway connecting two rooms.

The model had to encode start and goal images, then compute an action sequence to navigate from one room to the other through the doorway — purely from pixel observations.

Left: path efficiency distribution · Center: latent vs. physical distance · Right: agent positions colored by latent PC1 show room structure Routing

90% Success Pushing Objects to Target Positions — Block Location Recoverable at R² = 0.97

Bin packing requires placing items into constrained spaces at precise positions and orientations. We tested LeWM's ability to plan object manipulation using the Push-T environment — where a circular agent must push a T-shaped block to match a target position and rotation angle.

This is the core primitive of packing: move an object from where it is to where it needs to be, accounting for shape, orientation, and contact physics.

Left: block displacement per episode · Center: object state recovery from latent space · Right: block rotation distribution Packing

100% Planning Success Navigating Around Walls — Surprise Scores Confirm Obstacle Awareness

A self-driving system must detect obstacles, plan collision-free paths, and navigate to destinations from visual input. We tested LeWM on the Two-Room environment as an obstacle navigation proxy — the dividing wall forces the agent to plan indirect paths through a doorway rather than straight-line trajectories.

We measured both planning success and prediction surprise — if the model understands obstacles, its surprise should remain low when the agent correctly navigates around walls, indicating the dynamics model has internalized wall constraints.

Left: navigation paths through doorway · Center: prediction surprise along trajectory · Right: final distance to goal distribution Driving

The Architecture Works — The Training Data Is the Bottleneck, Not the Model

Three strategic implications emerge: First, world models are not competing with LLMs — they solve a fundamentally different class of problems (physical planning vs. text generation). Second, the ~15M parameter footprint and single-GPU training make this accessible for domain-specific deployment, not just research labs. Third, the path from these proofs-of-concept to production is a data problem (build better simulators), not an architecture problem (build bigger models).

Every Result Is Reproducible with a Single Shell Script

All three experiments, including environment setup, data download, checkpoint building, and evaluation, are packaged in run_all.sh. Running the script on a fresh Ubuntu 24.04 machine with an NVIDIA GPU reproduces every plot and metric in this report.