Google is infusing AI into Android with Gemini app and on-device Nano

Gemini Nano is a new, 3.8B parameter that is designed to run on-device and can process not just text input but also audio and images. It’s coming later this year “starting with Pixel” and will be used for:

• Clearer descriptions with TalkBack: TalkBack will soon be able to automatically generate more useful image descriptions. This will help people with visual impairments who can’t see images, especially when those images don’t have alt text already.

• Scam detection:
A new feature that processes voice calls to detect and warn you when the person on the other end is trying to scam you. The feature will look for conversation patterns commonly associated with scams, like when a “bank representative” asks you to urgently transfer funds. If so, your phone will buzz and show a warning. This feature is opt-in and will be processed on-device by Gemini Nano.

ℹ️ Credits: @MishaalAndroidNews
🔗 9to5Google

🧑‍💻 @agamtechtricks

Gemini for Android is getting upgraded with new contextual capabilities!

• A new “ask this video” chip will appear if you invoke Gemini while a YouTube video is open. Gemini will then use the YouTube video as context.

• A new “ask this PDF” chip will appear when you’re viewing a PDF file in supported apps. Tapping this will let you ask questions about/summarize the document. This requires Gemini Advanced, though, because of its longer context window.

• You’ll be able to drag and drop generated images directly into Gmail, Messages, and other supported apps.

• Finally, devices with the next-gen multimodal Gemini Nano will surface dynamic suggestions. The way this works is that a screenshot of the screen is fed to Gemini Nano, which can then generate chips of potential things you might want to ask about.

ℹ️ Credits: @MishaalAndroidNews
🧑‍💻 @agamtechtricks

Gemini Live

In Gemini Live, you’ll get a fullscreen experience with a cool audio waveform effect. This will let you have a 2-way dialogue, with Gemini returning concise responses. 

You can speak at your own pace, with Google adapting, and interrupt Gemini as it’s replying to add new information or ask for clarification.

Available for Gemini Advanced subscribers, it’s launching in the coming months.

🔗 9to5Google
🧑‍💻 @agamtechtricks

More Gemini Updates:

Gemini 1.5 Pro:
Google announced Gemini 1.5 Pro in February and is now launching it in the paid Gemini Advanced subscription.

New Extension:
The new YouTube Music extension lets you search for songs by "mentioning a favorite verse or a featured artist."

Gems:
In the coming months, Gemini Advanced users will be able to create customized versions of Gemini. Examples include a "gym buddy, sous chef, coding partner, etc." All Gemini users will have access to a number of pre-made Gems, like Learning Coach.

Immersive Planner:
Gemini Advanced on the web is getting an “immersive planner” that can create a custom, timeline-based itinerary. Google says this “new planning experience will go beyond showing a list of suggested activities.”  

Gemini 1.5 Flash:
Google is introducing 1.5 Flash as its “fastest and most versatile multimodal AI model.” It has the same 1 million context window and is aimed at use cases where low latency and cost matters the most.

🔗 9to5Google
🧑‍💻 @agamtechtricks

Gmail on Android, iOS getting more Gemini: Q&A, better Smart Reply, Summarize

Gmail on Android and iOS is getting a “Summarize this email” feature for longer threads. Workspace Labs users will get Summarize this week, with a launch for Google One AI Premium subscribers and paying Gemini for Workspace customers in June.

Meanwhile, Gmail Q&A will let you go beyond summarizing which will let you enter full prompts. You can ask a question from the e-mail.

Google is building on 2017’s Smart Reply and Smart Compose in 2018 with Contextual Smart Replies. Appearing as a carousel of chips, each suggestion is quickly summarized, like “Proceed & confirm time” or “suggest new time.” 

🔗 9to5Google
🧑‍💻 @agamtechtricks