Abstract
FieldGen is a semi-automatic data generation framework that enables scalable collection of diverse, high-quality real-world manipulation data with minimal human involvement.
Large-scale diverse data underpin robust manipulation, yet existing pipelines trade scale, diversity, and quality: simulation scales but leaves sim-to-real gaps; teleoperation is precise but costly and behaviorally narrow. We present FieldGen, a field-guided framework with a two-phase split: a pre-manipulation phase, where trajectory variation is acceptable, and a fine manipulation phase, which requires precise expert demonstration. This decoupling concentrates human effort on contact-critical states while scaling varied real-robot reach data. Policies trained on FieldGen data surpass teleoperation baselines in success, require markedly less human labor, and enable stable long-duration collection.
Method Overview
FieldGen Pipeline: Our framework splits manipulation into two phases. In the fine manipulation phase, human operators provide sparse teleoperated demonstrations focusing on contact-critical manipulation poses. These key poses are then used to construct the Pre-Manipulation Field (PMF), which consists of a cone field for positional guidance and a spherical field for orientation alignment. In the pre-manipulation phase, we automatically generate diverse reach trajectories by sampling random initial configurations and computing attraction-based actions toward the PMF, enabling scalable data collection with minimal human effort.
Pre-Manipulation Field Construction
Cone Field for Position
The cone field guides the end-effector's positional approach toward the goal. Points inside the cone follow smooth half-cycloid trajectories, while points outside are first projected onto the cone surface.
Spherical Field for Orientation
The spherical field aligns the gripper orientation during the approach phase using rotation matrices and logarithmic mapping in SO(3) for smooth corrective angular updates.
Equal-Time Data Effectiveness
We evaluate FieldGen on four diverse manipulation tasks under equal wall-clock collection time.
Pick
Basic grasping with position variation
Rotate Pick
Requires orientation change
Transparent Pick
Challenging transparent perception
Affordance Pick
Specific grasp region targeting
Key Findings: Across all 4–20 minute checkpoints, FieldGen's success rate exceeded teleoperation by an average of +41.9%. With just 20 minutes of collection time, FieldGen achieves over 90% success rate on average across all tasks, with DP reaching 100% on three tasks. This demonstrates FieldGen's superior time efficiency in data collection.
Generalization Performance
FieldGen demonstrates superior generalization across start poses, object positions, and object instances.
Generalization Test Setup
Three generalization regimes: varied initial poses, object positions, and unseen objects
Generalization Evaluation: We test three generalization regimes: (1) varied initial end-effector poses, (2) varied object positions, and (3) unseen intra-category objects. FieldGen consistently shows advantages across all scenarios—e.g., DP hits 100% success on Start EE Pose and Object Generalization with only 4000 frames, demonstrating the robustness of policies trained on FieldGen-generated data.
Trajectory Diversity and Spatial Coverage
Increased spatial coverage from FieldGen directly translates into stronger manipulation policies.
Spatial Coverage Analysis: By voxelizing the workspace, we find FieldGen (High diversity) occupies 18.14% of voxels, significantly outperforming teleoperation with fixed (Low: 9.04%) or varied (Middle: 15.44%) starts. Success rates increase from 0% → 54.2% → 83.3%, providing strong evidence that increased spatial coverage yields more robust manipulation policies.
Ablation Studies
Curve Type Analysis
We compare different trajectory curve types for the cone field. The half-cycloid curve provides the smoothest and most effective approach trajectories.
Step Size (β) Analysis
The step size parameter β controls trajectory granularity. We find optimal performance with β values that balance smoothness and computational efficiency.
Data Collection Efficiency
Collection Rate Over Time
FieldGen maintains stable data collection rates over extended periods, while pure teleoperation suffers from operator fatigue.
Capture Ratio Analysis
The capture ratio measures the percentage of sampled configurations successfully attracted to the goal. Higher ratios indicate more effective field design.
Key Insight: The automated reach generation enables continuous high-quality data collection with minimal human intervention, making FieldGen practical for large-scale dataset creation.
BibTeX
@misc{wang2025fieldgenteleoperatedpremanipulationtrajectories,
title={FieldGen: From Teleoperated Pre-Manipulation Trajectories to Field-Guided Data Generation},
author={Wenhao Wang and Kehe Ye and Xinyu Zhou and Tianxing Chen and Cao Min and Qiaoming Zhu and Xiaokang Yang and Yongjian Shen and Yang Yang and Maoqing Yao and Yao Mu},
year={2025},
eprint={2510.20774},
archivePrefix={arXiv},
primaryClass={cs.RO},
url={https://arxiv.org/abs/2510.20774},
}