Google launches Guided Vision in Gemini Live on Android

Google launched Guided Vision in Gemini Live on compatible Android devices on Oct. 1, bringing conversational, camera-based audio descriptions to blind and low-vision users. The feature is eligible on devices running Android 9 or later in regions and languages where Gemini Live is supported. However, Google says access is rolling out slowly, so some otherwise eligible users may not see it yet.

During a Gemini Live session, users can share their phone’s camera to hear descriptions, ask follow-up questions and receive spoken prompts to reposition the camera. Supported uses listed by Google include reading or translating text, identifying or locating objects, describing colors, shapes and patterns, and describing immediate surroundings.

Guided Vision can be enabled in the Gemini app’s settings, configured as an Android accessibility shortcut or launched from the TalkBack menu. Google previewed the feature in its September Android update before announcing its availability in October.

Guided Vision is distinct from Google’s recent Gemini text-to-speech model rollout: the accessibility feature uses a phone camera in Gemini Live to describe what it sees and answer follow-up questions.

Google says it developed Guided Vision with the accessibility community and Aira. The company says more than 1,000 members of Aira’s Trusted Tester program stress-tested and refined the model using tens of thousands of hours of interpreted visual data.

Google also cautions that Guided Vision uses generative AI and can make mistakes. It is not a medical device, mobility aid, white-cane replacement, navigation aid, safe-travel guide or substitute for obstacle detection, the company says.