End-to-End VLA Model Development

I am expanding an existing embodied-AI stack and now want to double-down on Vision-Language-Action modelling. The core goal is to design, train and scale a full pipeline that takes raw multimodal data, learns a joint representation and closes the loop all the way to real-time action on a physical robot.

You will start from large, messy datasets (images, video clips, proprioception, language annotations) that already sit on our cluster. The job is to craft a new architecture in PyTorch, schedule distributed training, and iterate until the model achieves reliable closed-loop visuomotor reasoning in simulation and on hardware. Robust multimodal representation learning, a world-model/JEPA component, temporal memory, and predictive control need to come together in a single, maintainable codebase.

I handle the robot side, so you can stay laser-focused on the Vision-Language-Action models themselves, while still having access to logs and live telemetry from our arms and mobile bases. If your approach can integrate ideas from the broader Robotic Intelligence literature or streamline the research-to-deployment pathway, that flexibility is welcome, but not mandatory.

Deliverables
• Clean, well-documented PyTorch implementation of the model architecture
• Reproducible training scripts with dataset loaders and evaluation hooks
• Checkpoints that perform end-to-end in both sim and real, with latency