iOS 14: Screen Recognition

Featured Image: 
Image of an iPhone with "iOS 14" annotated on the white screen.

What if . . . everything on the screen was labeled correctly for VoiceOver? What if every button, icon, image and element is accessible?

iOS 14 incorporates on-device intelligence to enhance VoiceOver recognition for image descriptions, text recognitions and the new screen recognition.

Image Descriptions

Using on-device machine learning to analyze and process images, VoiceOver provides a sentence describing the image. Automatic image descriptions is compatible with photos within apps like Facebook, Instagram and the web.

Text Recognition

For text included in a photo, VoiceOver uses Optical Character Recognition (OCR) to extract and read aloud the text.

Screen Recognition

This new feature enables VoiceOver to detect and announce elements such as buttons, sliders, icons and other elements which were not optimized for VoiceOver. Trained on thousands of images in apps, Apple created a computer vision system which enables iOS devices to automatically recognize and label buttons, icons and other elements that were not previously compatible with VoiceOver.

Settings > Accessibility >  VoiceOver > VoiceOver Recognition turn on image, screen and text recognition

Note: Screen Recognition is only supported on iPhone XS/XR models and later; Screen Recognition is also available for some iPads.

Additional Resources

Interested in learning about all the new features in iOS 14? Check out Apple's New Features iOS 14 page, which is organized by apps and/or sections!

In the Designing for Everyone: Accessibility Innovation at Apple video below, Matthew Panzarino (TechCrunch Editor in Chief) interviews Apple's Chris Fleizach (Accessibility Engineering Lead for iOS) and Sarah Herrlinger (Senior Director of Global Accessibility Policy & Initiatives).