What is Apple Live Text and how does it work in photos?

This page was translated using AI and machine learning.

(Pocket-lint) – Apple introduced a new way to take advantage of text in photos with iOS 15, whether it’s photos of places you’ve visited or your handwritten notes.

The feature is called Live Text. It applies to the whole system and works on all of your Apple devices. Here’s what you need to know.

What is live text?

Live Text uses device intelligence and deep neural networks to unlock information or text captured in your photographs. The idea is that you can copy notes from an image of a whiteboard in your meeting, or call a number from a restaurant menu that you snapped last week.

There are several use cases where Live Text can help you quickly find what you’re looking for and discover more in your photos.

What can you do with Live Text?

There are a number of different use cases for Live Text. When the feature was first introduced, Apple offered a few examples of how it could be used in your everyday life. Some of them include:

Copy and paste text

For whiteboard notes for example, you can just pull out your iPhone, take a picture of the whiteboard and an indicator will appear in the lower right corner.

When you tap it, the text is highlighted and you can use normal text selection gestures to copy it. From there, you can switch to your Notes app or Mail app and paste the text to save for later or share with others. Convenient, isn’t it?

Find more information

Live Text works on the photos you take in the moment and those in your library. For example, maybe you took a photo of a friend in a restaurant with words visible in the background on a building. With Live Text, you can select that text, then select Search from the available options that appear to find out more, including which restaurant is pictured, what it serves, and its location.

Make a phone call

Now suppose you receive a photo from a friend in the Messages app. Live Text will also work on these images. For example, say you have a photo where a restaurant number can be seen in the background, you can zoom in on the number, tap the live text indicator, then the text will instantly be recognized with a link. Tap it and you can make a call there.

Visual search

In addition to being able to manipulate and use text in photos, you can also use Live Text to search for information about recognized objects and scenes. Just tap on a photo and you can search for information like a dog’s breed or flower type.

Live Text also works for art books, nature, pets, and monuments. We expect this to expand over time.

Does Live Text work on screenshots?

Live Text supports any kind of photos you take, sent to you, or stored in your photo library, and it also works across the entire Apple system, meaning you can use it with screenshots, Quick Looks, and even photos. photos on the web. .

What Apple devices offer Live Text?

Live Text is available on iPhone as well as iPad and Mac. It is said that Live Text will only work on iPhone and iPad with A12 Bionic chip or higher, which means you will need Apple devices from 2018 to use it.

What languages ​​does Live Text support?

At launch, Live Text supports seven languages: English, Chinese (Simple and Traditional), French, Italian, German, Spanish, and Portuguese.

When will Live Text be available?

Live Text is available with iOS 15, iPadOS 15, and macOS Monterey software versions. iOS 15 and iPadOS 15 are both available for download now, although macOS Monterey is only available in public or developer beta at this time.

Written by Maggie Tillman. Edited by Britta O’Boyle. Originally published on June 11, 2021.