Gemini’s new beta feature provides proactive responses based on your photos,emails, and more

Google announced a new beta feature for its Gemini app on Wednesday. This feature allows the AI assistant to tailor its responses by connecting across your Google ecosystem, starting with Gmail, Photos, Search, and YouTube history.

While Gemini could already retrieve information from these apps, it can now reason across your data to provide proactive results. For example, it can connect a thread in your emails to a video you watched. Google states this means Gemini understands context without being told where to look.

This beta experience, called Personal Intelligence, is off by default. Users have the option to choose if and when they want to connect their Google apps to Gemini. If you do connect your apps, Gemini will only use Personal Intelligence when it determines doing so will be helpful.

Google explains that Personal Intelligence has two core strengths: reasoning across complex sources and retrieving specific details from an email or photo to answer your question. It often combines these, working across text, photos, and video to provide uniquely tailored answers.

A Google executive shared an example of being at a tire shop and not remembering his car’s tire size. While most AI chatbots can determine a car’s tire size, Gemini offered a personalized response by suggesting all-weather tires after identifying family road trip photos in Google Photos. In the same scenario, Gemini was able to pull a forgotten license plate number from a picture in Photos.

The executive also noted getting excellent tips for books, shows, clothes, and travel. For an upcoming spring break, Gemini analyzed the family’s interests and past trips in Gmail and Photos to skip tourist traps. It instead suggested an overnight train journey and specific board games to play along the way.

Google says it has guardrails for sensitive topics, as Gemini will avoid making proactive assumptions about sensitive data like health. However, Gemini will discuss this data if you ask it to directly.

Additionally, Gemini does not train directly on your Gmail inbox or Google Photos library. It trains only on specific prompts in Gemini and the model’s responses. In the examples given, the photos and emails are only referenced to generate a response and are not used to train the model.

Personal Intelligence is rolling out to Google AI Pro and AI Ultra subscribers in the U.S. Google plans to expand the feature to more countries and to Gemini’s free tier. Google provided example prompts to try, such as “Help me plan my weekend in New York based on things I like to do,” “Recommend some documentaries based on what I’ve been curious about,” or “Based on my delivery and grocery receipts in Gmail, Search history, and YouTube watch history, recommend 5 YouTube channels that match my cooking style.”