The biggest AI announcements during Google I/O 2024

Surprising no one, Google announced a ton of AI goodies during the first day of Google I/O. The show is all wrapped up now, but we are all still reeling from the incredible announcements that Google made. Undoubtedly, 2024 has been a big year for Gemini, and this announcement only emphasizes that fact. So, what were the biggest AI announcements during Google I/O? Here’s a rundown.

Before we start, if you want an overview of more announcements, both big and small, then you should check out our Google I/O recap. It dives into the nitty gritty of what the company announced.

Biggest AI Announcements during Google I/O 2024

If we list every single AI announcement that the company made, this article will be so long that you’d be reading it until Google I/O 2025! So, we will list down the most groundbreaking announcements that Google gave us. These are features that will be widely available to people and will have a significant impact on how you use Google products, boost productivity, or any combination.

Gemini 1.5 Pro/Flash

Starting off the announcement, Google introduced us to its latest model called Gemini 1.5 Pro. This is the company’s most advanced model, and it’s rolling out to several of the company’s AI-powered products.

Gemini 1.5 Pro improves upon the older version of Gemini in all key areas. It’s faster at processing, smarter, and much more powerful. What’s most notable about it is the expanded context window. Gemini 1.5 Pro can accept up to a million tokens. That’s unheard of, and it means that it can absorb much more information per prompt. We’re talking about an entire book series worth of text, hours of audio, a full hour of video, Etc. Imagine uploading an entire podcast and getting a succinct summary of what was discussed. Along with that, Google is also testing out a 2 million token version of Gemini 1.5 Pro.

Along with that, Google also announced Gemini 1.5 Flash. This is basically a stripped-down version of Gemini 1.5 Pro. It’s faster, more compact, and more efficient. Be that as it may, it still has that 1 million token limit.

AI additions to the Gmail app

Gmail is one of the most popular email clients in the world, and Google just made it a lot better with some amazing Gemini integration. Firstly, Gmail is getting a Summarize feature. This is straightforward. If you have a lengthy threat of emails, and you don’t feel like reading through all of them, you can use the summarize feature to get a concise and easy-to-read summary of all of the emails. Obviously, if you want to see all of the minute details of the email thread, you can read through the emails. However, in most cases, getting a quick summary of everything is all you need.

Free users will be disappointed, as you will only be able to use this feature if you are a Workspace customer or use the Google One AI Premium subscription.

Another great feature is the Gmail Q&A feature. This will allow you to ask Gemini questions about the contents of email threads. Then, Gemini will be able to answer these questions for you. If you’re having a back-and-forth conversation about comparing housing prices, you can ask Gemini something like “Which potential buyer gave me the best offer?” Gemini should be able to answer that question for you.

Ask Google photos

Right now, Google Photos has some pretty insane AI, but that doesn’t mean that the company can’t improve it. With Gemini, the artificial intelligence powering Google Photos is now much more advanced.

The company unveiled the Ask Photos feature, and it’s pretty impressive. Basically, you can ask Gemini certain broad questions that could pertain to images and videos that you captured. Google gave the example of asking how a child improved with swimming over the years. Gemini will be able to look through your photos, identify the child, identify all of the pictures and videos of them swimming, and give you a timeline of how they progressed. That’s only one example of this feature’s utility.

This option will be pinned on the bottom bar of the Google Photos app.

Project Astra

This is an extremely impressive showcase of Gemini’s power. During the presentation, the company showed off Project Astra, which is a tool that gives Gemini a set of eyes. Google will implement this into the Gemini app at some point in time.

When you activate the feature, you will see a viewfinder in the app. Gemini will be able to identify what is in the viewfinder. In one example that Google gave, a Google engineer asked Gemini which item on their desk makes noise. The engineer then pointed to a speaker, and Gemini was able to identify that the speaker makes noise. Also, they drew an arrow on the screen pointing to the speaker’s tweeter. After the engineer asked what that was, Gemini was able to identify that it was the tweeter of the speaker and described what it was.

That’s only one example. The engineer was also able to have Gemini identify their location by pointing the camera outside. Gemini was able to identify which part of the city they were in.

Since we’re talking about Gemini, you can pretty much ask it anything you want. You can ask it questions about items that are in the viewfinder, ask it to create content based on them, Etc.

The company stated that some of Project Astra’s features would be coming to the Gemini app later this year. However, we have no idea what those features are, so we’ll have to wait.

Imagen 3

In case you don’t know, all of the images generated using Gemini use Google’s proprietary image model named Imagen. Well, the company unveiled Imagen 3, its most advanced model. With it, people can produce much more realistic images with fine details added in. So, if you are into AI image generation, then this is something to be excited about. It strives for photorealism, and it achieves it.

Veo

Taking things further, Google also introduced its video generation platform. This is going to remind people of OpenAI’s video generation platform called Sora. Sora is able to create some very realistic-looking video clips.

During Google I/O, the company unveiled its own take on video generation. This is a platform that can create stunningly realistic video clips. It’s able to create high-quality 1080p clips that go beyond Sora’s 60-second limit. At this point, we are not sure when it will make it to the public. However, when it does, we are certain that we’re going to be seeing a lot of AI-generated videos on the internet.

Video questions for Google Search

Think of this feature as Google Lens for videos. You can send a recording to Google, and ask Gemini questions regarding that video. The question can be about anything in it. Gemini will be able to identify what’s going on in the video and answer the question based on it.

Google gave an example during Google I/O where the presenter asked Google a question based on a video of a broken turntable. Gemini was able to identify what part of the turntable was broken and give her information on what to do to fix it. This feature is true to Google’s original vision of letting Gemini reason across text, audio, video, and images.

The possibilities are endless with this feature, and it will make Google much more accessible to people who can’t quite type out a question.

Gems

This new feature is a great way to customize your Gemini experience. A Gem is a personalized version of Gemini that you can use for very specific reasons. For example, if you want to create a customized version of Gemini that is specially tuned to create ghost stories, you can create a gem for that.

In the Gem creator, you can describe what sort of chatbot you want to create. You can type something like “A ghost story creator that makes stories with mysterious characters and plot twists.” When you do that, Gemini will get to work creating the instruction set for that chatbot.

It goes further than that, as you are also able to give Gemini access to your files across other Google products. For example, if you give Gemini access to your Google Docs, and you have story drafts of ghost stories, Gemini will be able to read through those drafts in order to get a feel for what kind of stories you want it to write.

You’re able to create multiple gems, so you’ll be able to use them for different reasons. This feature will be rolled out later this year.

Gemini-powered scam call detector

This is a neat feature to have if you care about protecting yourself from getting scammed. We’re all used to getting scam notifications once a call comes in. However, some calls do get through, and Google will be able to warn you during a call if it’s a scam.

Gemini will be able to listen in on your call and ascertain whether the person calling you is trying to scam you in some way. For example, if you get called by a bank, and the bank asks you to transfer your money to a secure account (which is something that banks NEVER do), Gemini will be able to flag that call and tell you that it is likely a scam.

What’s neat about this is the fact that all of the processing is happening on-device. So, you will not have to worry about Google listening in on your conversation. All of the data remains stored locally on your device.

Notebook LM upgrade

Notebook LM is an educational tool that helps kids learn and research. It was announced during Google I/O back in 2023. Now, Notebook LM is getting a pretty big update.

This feature is called audio overviews. It will read through the material that you load on it, and come up with an educational discussion based around it. The educational discussion will be in a podcast Style with two people talking. It will emulate a real conversation so that kids can better associate with it.

Not only that, but you can suggest changes to the discussion, and Gemini will make the changes in real-time. For example, if you create a discussion about Jupiter’s moons, but you want to change it so that it uses basketball analogies, you can type that. After that, Gemini will reconstruct the educational discussion to fit that.

Tighter integration between Workspace apps

Right now, Google’s Workspace applications like Docs, Sheets, Slides, Gmail, etc. can integrate with one another. However, Google is tightening that integration, and Gemini is the vice being used to tighten it. You will be able to ask Gemini to perform certain actions with your files Within Workspace.

For example, if you want Gemini to look into your emails, look through an invoice for a service you have done, and paste the information from that invoice into a Google doc, you’ll be able to do that. That’s only one example of this tight integration.