> ML_LITERATURE // BROHAN-2022-RT-1-ROBOTICS-TRANSFORMER-REAL-WORLD-CONTROL_v1.0
RT-1: Robotics Transformer for Real-World Control at Scale
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Mariya Jesmonth, Nikhil J. Joshi, Ryan Julian, Dmitry Kalashnikov, Yuheng Kuang, Isabel Leal, Kuang-Huei Lee, Sergey Levine, Yao Lu, Utsav Malla, Stefano Montabilo, Suraj Nair, Annie Nguyen, Trevor Pham, Jodilyn Peralta, Sharreth R. Kumar, Kanishka Rao, Dorsa Sadigh, Pannag Sanketi, Michael Schwarz, Menglong Zhu, Vincent Vanhoucke · Robotics: Science and Systems (RSS) (2022)
Principal Contribution
Trained a 35M parameter transformer tokenizing camera images and natural language instructions into discrete arm and base actions at 3 Hz, evaluated across 130,000 real-world robotic tasks.
Operational Relevance
Serves as qualified theoretical and systems foundation for task-robotics, task-continuous-control.
Assumptions
- Markovian state dynamics and stationary reward functions hold in target evaluation environments
Limitations
- Sample efficiency, exploration stability, and real-world sim-to-real transfer gaps require specialized tuning