Gemini for Android is getting two new features that will allow users to ask questions about their smartphone’s current screen. Dubbed ‘Ask about this screen’ and ‘Ask about this video’, these tools capture a screenshot and process the content of the screen. Then, when prompted by the user, it can answer with relevant information that the user is seeking. The video feature, which only appears to be compatible with YouTube, can also provide video summaries provided there are captions added to the video.
Gemini Now Lets Users Ask Queries About Their Screen
Gadgets 360 staff members spotted the features in the Google app version 15.33.38.28.arm64 and later in several smartphones. When a user summons the AI assistant, these appear on the top of the Gemini floating window. Those with access to the feature will see a separate rectangular strip with the text “Ask about this screen” and a screenshot icon. If Gemini was activated while playing a video on YouTube, the text is replaced with “Ask about this video” and the YouTube icon.
Gemini Ask about this screen feature
Tapping the icon for ‘Ask about this screen’ captures a temporary screenshot within Gemini, and then the user can ask any query about the screen. The responses will only be based on what the AI can see in the screenshot, however, users can also prompt Gemini to access the Internet to find more information. One of its use cases could be asking for a summary of a news article.
To ask a follow-up question, users can just tap the microphone icon or the keyboard icon at the bottom of the floating window. Notably, the captured screenshot is not saved on the device. Additionally, long screenshots are not supported with this feature.
