At its I/O 2024 event on Tuesday, Google shared exciting news about its upcoming AI projects, including improvements to Gemini, Project Astra, and AI-powered search. In addition to these search and Gemini app updates, the company also unveiled new features for Android, its flagship operating system.
Circle to Search. This isn't a new Android feature, given that it debuted on the Galaxy S24 and Pixel 8 Pro a few months back. However, Google has announced that the feature will gradually become available on more devices, although it hasn’t provided a timeline on when to expect it.
Gemini updates and more app integration. Gemini will have a special role in Android and offer integration with apps like Google Messages. In fact, Gemini’s interface will be directly integrated into the app rather than just being an accessory that's displayed on top.
Similarly, in apps like YouTube, Gemini will be able to analyze the content of the screen, providing information about what users are watching and combining it with what it “knows,” that is, what it finds on Google’s search engine.
Google Photos, Messages, YouTube… These apps are just the beginning of everything Gemini will be added into. It won’t be a regular assistant app, but rather, it'll be integrated with both the system and the apps.
TalkBack enhancements. Thanks to Gemini Nano's multimodal capabilities, many expect improvements to the next-generation Pixels. For one, Google TalkBack, which is Android’s accessibility service, will provide better image descriptions.
In the upcoming months, blind or visually impaired users will be able to ask TalkBack to analyze the content on their screen. Gemini Nano will perform this analysis, providing users with accurate descriptions.
Better protection against spam and scams. Android has always offered great protection against scam calls. In fact, its Phone app warns users when a suspicious number is calling. However, the system is not infallible, and some calls may slip through.
With user permission, Gemini will be able to analyze calls in real-time and provide instant warnings about whether it suspects they're spam or not. For instance, if users are asked for unusual information during a call, Gemini will send us an alert in real-time.
Google will be gradually rolling out these new features to Android phones, although it released details on which models will be compatible yet or which OS version will be required. From the looks of it, these features will depend on Google’s Gemini Nano, which will allow them to be executed locally.
Image | Google