As a college student with low vision, I constantly find myself surrounded by things I can't see. I don't always have someone with me that can help me figure out what's around me, but I do always have my iPad and internet access on my college campus. Because of this, I have the next best thing to a seeing eye friend, and that's Microsoft's new app, Seeing AI.
What is Seeing AI?
Seeing AI is a free app developed by Microsoft that helps people with vision impairments get information in real time about the world around them using artificial intelligence- which is the "AI" in Seeing AI. It was initially released a few months ago in late 2017, though new functions are being added often. It is currently only available for iOS, and can be downloaded here. I use Seeing AI for iPad on my 5th generation iPad running iOS 11.
Seeing AI opens in portrait mode and immediately activates a live camera with voice narration built into the app. The app uses dynamic text which causes minor display issues in the settings menu, but I still found it pretty easy to use. Users can choose what function of Seeing AI they want to use by swiping across options on the bottom of the screen (more on those options in a minute). I find the app interface similar to using Snapchat- you can read my review of Snapchat for low vision here.
Since it's an app developed for people with vision impairments, this app is fantastic with accessibility. I appreciate that users can change the voice type and speed in the settings menu to something that is faster or slower, and that there is large, high contrast, and bold text. Read more about what makes apps accessible for users with vision impairments here.
Seeing AI has two different options for reading text- short text or long text. I use the short text function for things that take less than twenty seconds to read, including signs, labels, logos, and buttons. I use the long text function when reading instruction sheets, ingredient lists, longer memos, and menus. There's also a handwriting option for reading handwritten notes that works well, though it didn't recognize my professor's handwriting. If I need a device that can input text into my iPad, I use my Scanmarker Air- read my review of the Scanmarker Air here.
Identifying products and currency
A barcode scanner is built into Seeing AI that can help users identify everyday products and learn about their sizes and other related details. I tested the barcode scanner on almost everything at my dorm desk and it worked flawlessly with detecting products and telling me more details- read about what my college desk looks like here. Seeing AI also has a currency reader that works with US and Canadian dollars, as well as British pounds and euros. The currency reader worked great for dollars, but had mixed results with detecting coins.
Colors and light
Color readers can be very expensive, so having a color detection feature built into a free app like Seeing AI is exciting. One cool thing is that it can detect more than one color at a time. To give an example, I'm wearing a sweater with shades of purple, teal, page,ink, and white, and when I had the different colors within the color selection area, Seeing AI was able to identify them successfully- read more about getting dressed with low vision here. There's also a feature where users can figure out how much light is around them as the app creates an audible tone- the louder the tone, the more light there is. While I don't have much use for the light feature given that I can perceive light, it is still really cool.
Reading faces and scenes
My friend and I were playing with this app and decided to test out the reading faces and scenes functions. When I pointed the app at their face and took a picture with it, the app correctly recognized their gender, age (within two years), hair color, and also recognized that they were smiling. I was able to figure out how far away they were from the camera by simply using the live view, no picture taking necessary.
For the scene recognition part, we walked around a room and the app told me about potential obstacles, where objects were located, and also identified when my friend's dog walked in the room- read more about dogs and low vision here. There is also an option to take a picture of a scene and have it described, but it is harder to judge distance.
How I use it
I use this app both inside and outside the classroom, probably more often outside the classroom than inside. This is because I use Microsoft Office Lens more often in the classroom- read more about Microsoft Office Lens here. Here are some examples of how I use Seeing AI:
In the classroom
- Using scene description to look at pictures
- Magnifying a sign
- Looking at color contrast
- Reading very short passages of text
- Reviewing sticky notes
Outside the classroom
- Checking a label on a bottle
- Identifying the size of a product
- Assessing how much money is in my wallet
- Looking for obstacles
- Using buttons on a microwave
How my professors and friends have reacted
All of my friends and professors have been fascinated with this app and what it is capable of. I am the only person with low vision in many of my friend groups and classes, so it's always cool to show people cool innovations in assistive technology, accessibility, and similar. I do enjoy having friends with similar conditions that I can share apps like this with though- read my post on why having friends with similar conditions is awesome here.
Seeing AI is a really awesome app, and I would probably use it constantly if it was available on my Android phone, since it isn't always convenient to walk around with my iPad in portrait mode. I'm looking forward to seeing the features added to Seeing AI in the future, and recommend this app for anyone who is blind, has low vision, or any other type of vision impairment.