Android's new Expressive Captions feature conveys emotion

Google is improving its Live Captions feature by announcing a new feature within it called Expressive Captions. As the name suggests, Expressive Captions will make the captions that pop up more expressive to convey emotion.

Captions in general are nothing new and you’ve been able to add captions to videos in most situations on Android for a while. YouTube has built-in captions that you can toggle as do other video player options. It wasn’t until the addition of Live Captions though that captions in general became a lot more useful. Specifically, because they would play regardless of whether the video had captions embedded or not. And it works across most apps on Android now.

There is one thing that Live Captions is missing though. Emotion. Google says that with this new feature, Live Captions “will not only tell you what someone says, but how they say it.” Now you won’t just read what someone is saying and know that they’re angry by the words being used. You might also see emotional actions visibly typed out along with the words being said. Like a gasp or scream. Furthermore, when someone is screaming, like during a sports broadcast, those parts of the captions can show up in all caps.

The Expressive Captions feature uses AI to convey various elements like tone

Naturally, the power behind this new feature is Google’s AI technology. The company says that Expressive Captions uses AI on your phone to convey several different types of feelings. This includes tone, volume, environmental cues, and human noises.

So you could also see in the captions if someone sneezes for example. Other vocal bursts will be visible in the captions as well, such as grunting and sighing. As for the environmental cues, this references background or ambient noises, like applause, cheers, or perhaps the sounds of things clanging together.

The feature rolls out to US users today in English

As with many of these features, Google is starting small with the rollout. Beginning today, Expressive Captions will start to roll out to users in the US in English, Google says. You also need to have an Android device that’s running on Android 14 or above to gain access. As the feature is part of Live Captions, and Live Captions requires at least Android 14.