Task-diverse
Robot Data
Robot Data
Robotic Task Generalization.
Robotic policies deployed in the physical world must handle a broad range of manipulation tasks, including tasks outside their training distribution. This task-level distribution shift remains a central challenge in general-purpose manipulation. A straightforward way is to keep scaling manipulation data so that it covers as many tasks as possible. Yet real-world manipulation tasks are extraordinarily diverse, and language instructions alone often leave crucial procedural details ambiguous. This motivates a complementary visual interface that can ground an unseen task in a concrete demonstration.
0:00
0:00
In-context Human Video Prompt.
Human demonstration videos provide such a visual interface by showing the intended interaction and its visual progression. Compared with language instructions, the examples below illustrate three complementary forms of visual task specification: grounding the intended object instance, demonstrating the required interaction with an unfamiliar object, and specifying the temporal order of a long-horizon task. Training a single policy to follow both language instructions and human video prompts strengthens its ability to generalize across manipulation tasks. Achieving this capability requires scaling semantically aligned human-robot ICL pairs. When language is insufficient, a relevant human demonstration found on the internet or provided on site can serve as an in-context prompt, allowing the policy to execute the specified task without task-specific robot demonstrations or parameter updates.
Which object?
Identify the intended instance among visually similar objects.
> 10Dataset Sources
> 8kRobotic Tasks
> 70kHuman-robot Pairs
Zero-WAM: In-context World-Action Pre-training.
To make this human-video ICL interface trainable at scale, we introduce Zero-WAM, an in-context world-action pre-training framework for zero-shot robotic task generalization.
- Unified task interface. We formulate zero-shot robotic task generalization as in-context world action modeling, where a single causal video-action policy supports both language and human videos as task instructions.
- Scalable in-context data. We introduce a generation pipeline that converts task-sampled robot trajectories into semantically aligned human video instructions, yielding 74.2K human-robot ICL pairs across 8.6K tasks.
- Learning to follow the visual prompt. We introduce in-context future chunk prediction (IFP) to discourage shortcut learning from robot history and text, strengthening the policy's use of the human video when predicting longer-term task evolution.
In-context Human Video Generation Pipeline
Scaling the human-video ICL interface introduced above requires task-rich, semantically aligned human-robot pairs, which are expensive to collect manually. Robotic pre-training corpora already contain diverse trajectories with executable actions. We therefore sample tasks from these corpora, convert the corresponding robot videos into matched human demonstrations, and retain only physically plausible, task-consistent ICL pairs.
Sampled Robot Video
Task Info:
Retrieve the children's tableware from the organizer ...
Prompt for Editing:
Convert to a first-person human view. Preserve the organizer, blue tray, and tableware ...
Human Observation
Nano Banana
Qwen-Image
Nano Banana
Qwen-Image
Prompt:
First-person view over a white counter. Both hands remain visible as the right hand retrieves each piece of children's tableware from the transparent organizer and places it into the blue tray, while the organizer and countertop remain fixed ...
Video Gen. Prompt
Human Video Generation
Semantics Score
✓ 5: Perfect
× 3: Acceptable
× 1: Bad
× 3: Acceptable
× 1: Bad
Physics Score
✓ 5: Perfect
✓ 3: Acceptable
× 1: Bad
✓ 3: Acceptable
× 1: Bad
In-context
Human Data
Human Data
Task-diverse Data
Zero-WAM pre-training uses two complementary data components. Task-diverse VA is constructed from five public datasets by repartitioning robot trajectories into more than 6,000 manipulation tasks and sampling them at the task level. The pipeline above constructs HumanGen, a collection of aligned human-robot ICL pairs spanning public datasets, in-house platforms, simulation, and real-world collection.
Zero-shot Testing.
We evaluate Zero-WAM on long-horizon sequential manipulation and precision-demanding insertion. Both task families are difficult to specify fully in language: the former requires an exact execution order, while the latter depends on precise target configurations.
Long-horizon Sequential Manipulation
Silver coffee cup, sponge, and gourd
Place the silver coffee cup in the wooden basket. Place the sponge in the white tray. Place the gourd on the pink oval plate.
0:00
0:00
0:00
0:00
Precision-demand Inserting
Table leg inserting
Move the white tabletop base to the left. Insert the green table leg into the bottom-right corner of the white tabletop, then insert the blue table leg into the top-left corner.
0:00
0:00
Speed ×2
0:00
0:00
RoboTwin 2.0 Evaluation
Seven Unseen Tasks.
We evaluate Zero-WAM's cross-task generalization in the RoboTwin 2.0 simulation benchmark. Seven tasks are held out at the task level; Zero-WAM is trained on the remaining 43 tasks and evaluated zero-shot on the held-out set.
Unseen Object or
Unseen Container
Bimanual Manipulation
Articulated Object Manipulation
Long-horizon Manipulation
Task-level Zero-shot Results.
Across this held-out suite, Zero-WAM reaches 46.95% average success, surpassing the video-action baseline LingBot-VA by 29.50 percentage points.
WAN
LingBot-VA
Zero-WAM
Place object on scale
Stamp seal
Open microwave
Move stapler
to pad
to pad
Place bread into basket
Place empty cup
Stack three blocks
Average
@misc{zhou2026zerowam,
title = {Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization},
author = {Zhou, Jiaming and Zhang, Qihang and Xu, Gangwei and Fan, Cunxin and Zhao, Yujie and Wang, Ruilin and Luo, Yiming and Yang, Shuai and Zhu, Xing and Shen, Yujun and Liang, Junwei and Xu, Yinghao},
year = {2026},
eprint = {2608.26103},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2608.26103}
}