Google's new Gemini AI model can run robots locally without an internet connection
Google has recently introduced an enhanced version of its Gemini AI, known as Gemini Robotics On-Device, which is capable of operating on robotic hardware independently without the need for an internet connection. This innovative model offers task generalization and fine-tuning with minimal data requirements, providing bi-arm robots with local, low-latency control.
The Gemini Robotics On Device processes language, action, and vision inputs directly on the device itself, distinguishing it from models that rely on cloud computing. This feature can be particularly beneficial in environments such as manufacturing floors or remote locations where low latency or limited connectivity is crucial.
Key Features of Gemini Robotics On-Device:
- Handles language, action, and vision inputs locally
- Requires minimal data for task generalization
- Provides bi-arm robots with low-latency control
According to Google, this model can understand and execute natural language instructions with just 50 to 100 demonstrations, enabling tasks like zipping bags, folding clothes, and pouring liquids to be accomplished efficiently.
The on-device version of Gemini Robotics is designed as a lightweight extension of the Gemini 2.0 architecture, maintaining multi-step reasoning and precise control capabilities while optimizing for smaller computing footprints. It has undergone successful testing on various robots beyond its initial training setup, such as the Apptronik Apollo humanoid and the bi-arm Franka FR3, allowing for swift adaptation to new tasks or robot configurations.
Development and Testing:
Google has initiated a comprehensive testing program and is providing a Gemini Robotics SDK to a select group of developers for evaluation. This SDK enables users to test the model in MuJoCo physics simulations and customize it for specific tasks. While access to the SDK is currently limited, Google is evaluating its efficacy and security in real-world scenarios.
The development of this model aligns with Google's internal AI Principles, incorporating additional safety measures to ensure semantic integrity and physical safety. Under the oversight of the company's Responsibility & Safety Council, the system is undergoing evaluation using a new semantic safety benchmark.
Implications and Future Outlook:
Google's introduction of the on-device Gemini Robotics model signifies a shift towards more autonomous and adaptable robotics systems that can operate locally. This advancement could have significant implications for industries such as logistics, industrial automation, and beyond, offering enhanced efficiency and flexibility in diverse applications.
For further reading, check out: OpenAI and Jony Ive’s first AI hardware might not be wearable, court documents reveal
