Seeing AI App: Making It Work for You
by Cody Froeter, CVRT Tags: accessible apps, iPhone, OCR, Seeing AI
Cody works with Lee on reading a bar code with Seeing AI
With the rise of smartphone technology, many apps have been developed to assist people with vision loss. Rather than having to purchase multiple assistive devices, at sometimes significant costs, a person can now download an app, often for free and accomplish the same tasks.
Increasing Power and Efficiency
As technology has advanced, apps have increased in power and efficiency. The recent release of Microsoft’s Seeing AI app for iOS, demonstrates how this technology has reached new and exciting levels. Seeing AI is one of the most powerful apps designed for individuals with vision loss. Before Seeing AI, a smartphone user would need to download a multitude of apps ranging in price from free to one hundred dollars. Seeing AI is able to perform all of these tasks bundled into one app that is free!
How Seeing AI Works
Microsoft’s Seeing AI app utilizes artificial intelligence and the iPhone’s camera to accomplish a variety of tasks. At the launch of the app, the app offered five features: Short Text, Document, Product, Person, and Scene.
- Short Text uses optical character recognition (OCR) to read a short amount of printed text.
- Document utilizes OCR technology to read full pages of printed text.
- Product scans barcodes on products to give information about the products.
- Person uses facial recognition to give detailed information about a person’s face in the camera.
- Scene offers a description of a photo taken with the camera.
New Features
With the recent update to the app, the app now features a color identifier, a currency reader, a light detector and one feature that uses OCR technology to read handwritten notes — a first for OCR readers!
My Experiences Working with Seeing AI
While the app is powerful, easy to use and free, it does take a lot of processing power which can drain the iPhone’s battery quickly. It is important to close the app by double tapping the home button to access App Switcher and then sliding one finger up (three finger swipe up for VoiceOver users) to close the app when done using it. Here is a quick review of all the features:
- The Color Identifier is more accurate than any other app or device on the market. Lighting in the environment, however, can give incorrect identifications. Also, incorrect colors or the app describing the color as white or gray, can be due to the camera flash being on and reflecting off the item. An option to turn off the camera flash is available in Settings, but there is no option to turn it back on.
- The Currency Reader sometimes repeats the currency value too many times. Because the currency reader is located within the Seeing AI app, it is not as quick to use as an independent, stand-alone app like NantMobile.
- Short Text may cut off text if the text is not within the camera view. It can also be confusing as it has a tendency to repeat. Covering the camera lens or setting down the iPhone on a surface seems to clear it, enabling you to read something new.
- The Scene channel is good for reading street signs. However, if there are other signs in the vicinity such as a billboard, it will include this in the response read out.
- The Doc reader is solid and VoiceOver does not have to be enabled to have Seeing AI read the text.
- The Product Bar Code Scanner beeps very quickly when near the actual bar code. You must continue to move the item in the same direction so the camera can pick up the entire code. The scanner will announce “processing” when it has acquired the entire bar code.
- The Person identifier is only effective for people 15 feet away. I don’t see much value here as at this distance using your own voice will be just as efficient.
- The Handwriting is not yet 100% and is better with print than with cursive writing. This channel is still in beta version so hopefully this will improve with future versions.
The currency reader, scene, color identifier and handwriting are all in beta versions, so hopefully they will ally improve with future versions.
All in all, this is a terrific app and will only improve in the future.
Cody Froeter is a Certified Vision Rehabilitation Therapist at Second Sense. He works with our clients who are 55 years and older, teaching a wide variety of independent living skills and assistive technology.
