Seeing AI App: Explore Pictures Feature

By Diane Brauner on March 14, 2019

The recent update for Seeing AI, a free Microsoft app, has added a fun new feature: Explore Photos by Touch. This feature enables users to drag a finger around the photo to explore people and objects and to learn where they are located. Photos can be accessed several different ways. One easy way is to access photos that are already in your photo app. The Seeing AI app is fully accessible with VoiceOver.

Directions

Open the Seeing AI app
Select the Menu button (top right corner)
Select Browse Photos button
Select desired photo (music indicates that the app is processing the picture)
- Basic description of photo is announced
Select Explore Photo button (bottom right; music indicates that the app is processing the picture)
- Will sometimes announce how many items were found in the screen
Drag your finger around the screen to find the discovered items
- More detailed information is announced

Hint: Take a picture just prior to opening the app; this picture will now be the first picture in the Photo app.

Examples

4 year old smiling girl sitting in front of an iPad with a Braille Display beside her in a preschool classroom.

Short Description: "Probably a person standing in front of a laptop; contains text."

Detailed Description (items announced as finger is dragged over the items): "8 year old girl with brown hair wearing glasses looking happy." Announces the letters in the alphabet on the bulletin board behind her, along with the sign "social studies center". Does not announce the bookshelves, iPad or braille display.

Smiling toddler wearing a hat sitting at the top of bright orange slid on a sunny day.

Short Description: "One face; probably a young boy sitting on a slide."

Detailed Description (items announced as finger is dragged over the items): Person. 1 year old boy with blond hair looking happy. Footwear. Shoe.

Photo of a training taken from the back of the room. Multiple participants looking at PowerPoint Slide and presenter.

Short Description: "Group of People looking at a screen."

Detailed Description (items announced as finger is dragged over the items): Announces "person" as finger is dragged over multiple people in audience. Announces all the text that is visible on the PowerPoint slide (some text is blocked by presenter).

Note: Image is of darkened room and text on slide is blurry and not readable in photo.

Editor's Note: Personally, I was excited that Seeing AI was able to accurately determine the text on the slide! This may be useful for students in class who occasionally want to take a photo of text on the board.

Note: Additional images were tested and the results were mixed. Artificial Intelligence continues to improve, but has not been perfected yet.

Accuracy

Seeing AI does an amazing job with finding and correctly announcing text - even text that appears blurry or small in the photo!
Seeing AI will provide more information about the person in the photo - gender, age approximation, hair color, etc. - if the photo is of good quality.
Seeing AI may announce one person in the photo but may or may not announce additional people.
Seeing AI is not always accurate about identifying objects, especially if there are multiple objects in the photo.

Bonus

Once you have explored a photo using the Seeing AI app, the general description will automatically be announced by VoiceOver when the photo is in the Photo App.