Made by Google 2024 was the company's opportunity to infuse Gemini into everything

Today at its Made by Google 2024 event, Google finally put some rumors to rest and made announcements across its range of products. Notably, the company officially made Gemini the default assistant in the Pixel 9 devices, released Gemini Live, its alternative to OpenAI's Advanced Voice Mode, and a handful of new image editing and generation, and accessibility capabilities. This is not to say that Google Assistant is being permanently banished, as Google has already announced that Assistant (powered by Gemini models) will remain in the Home/Nest ecosystem.

Gemini Live lets users have voice conversation experiences with the Gemini Assistant that emulate some aspects of human conversation like interrupting Gemini mid-response to delve into a particular detail, or pausing the conversation and following up on it later. Moreover, Gemini Live is always available, even hands-free when the app is in the background or the phone is locked, and users can pick between 10 natural-sounding voices. The feature starts rolling out today for English-language Gemini Advanced subscribers using Android devices. Google plans to gradually expand availability to iOS users and other languages over the next few weeks.

Google has already been hinting at and working towards integrations that make Gemini more useful without switching apps. On mobile devices, this translates to being able to ask Gemini to do things like pull information from Gmail, add new items to lists on Keep, and soon, even ask Gemini whether users are free on a given day and tell the assistant to set reminders on Calendar. Other useful related capabilities include "Ask about my screen" or "Ask about this video" (on YouTube), and a drag-and-drop feature to add Gemini-generated images to emails or messages in Gmail and Messages. These features will be powered by smaller models, like Gemini 1.5 Flash, to ensure speed without losing response quality.

Even if the flashiest features announced today were tied into the Pixel 9 devices' launch, Gemini's integration into Android is coming to various devices from Google's partners, supporting several languages and geographic locations. Google has emphasized how this is partly possible because of Google's approach to safety and privacy, which largely stems from the company's decision to power some of the most sensitive AI applications with an on-device model: Gemini Nano. Thus, users who activate the Call Notes feature to summarize phone calls or the Pixel Screenshots functionality to organize the important information in their screenshots can be sure that their data will not be shared with third parties because it won't even leave their device.

Finally, Google also announced a selection of features to improve accessibility on Pixel and Android devices. Guided Frame, the feature that helps low vision and blind users take selfies by providing spoken guidance on the camera angle, positioning, and lighting levels, is now available directly on the camera settings rather than through TalkBack (Android's screen reader) only. The Magnifier feature also got a revamp, letting users search for specific words, leverage a picture-in-picture mode, choose the best lens according to context, and turn on the front-facing camera to use as a mirror.

Live Transcribe is getting a dual-screen mode exclusive to foldable phones, including the Pixel 9 Pro Fold. Live Caption is expanding its language availability, now covering Korean, Polish, Portuguese, Russian, Chinese, Turkish, and Vietnamese. These languages will also be available on-device for Live Transcribe, making it possible to use the feature in 15 languages offline, and 120 online.

These announcements lend credibility to the idea that Google is now in its Gemini era. It is also remarkable that many of these features have been launched with a particularly subdued factor and several even sound practical. We doubt that the Gemini assistant will be exempt from the criticism and skepticism faced by other AI-powered assistants, and it is also very likely that many of these features lend themselves to future improvements. For now, Google has shown its hand, and it will be interesting to see how it shapes up against its competitors and into the future.