Wearable device boosts "sight" for the blind and visually impaired

Jake Schmude was born without optical nerves — a rare condition that left him blind. But the talkative computer whiz has coped exceedingly well.

Schmude, 32, a network administrator at a nonprofit organization for the visually impaired, lives independently in Gwynn Oak. He commutes to work by Uber and Lyft. And he has long used smartphone apps that can scan text and “read” it aloud.

But it wasn't until 2016 that he felt fully comfortable carrying out many of his necessary daily tasks.

That spring, Schmude bought a cutting-edge assistive device that fits atop the frames of an ordinary pair of glasses.

Known as the OrCam MyEye, the device consists of two parts: A one-ounce camera that mounts on his spectacles and a six-ounce computer clipped to his belt. It’s the first optical character recognition technology that can be worn on the head.

By directing its gaze toward a block of text or object, or by simply pointing a finger at the target, he activates a process by which MyEye turns scanned visual information into spoken language. He hears the voice through an earpiece.

The process happens almost instantly, and with an accuracy that independent studies have shown to exceed that of other technologies on the market.

Schmude says he can now comfortably carry out tasks that had always been challenging or impossible, from reading the text on a curved can or a distant billboard to distinguishing faces or the color of an object.

“It has made a lot of what I do so much easier,” he says.

Schmude’s employer, Blind Industries and Services of Maryland in Halethorpe, was so impressed with the device that it set up a payment plan to help its 250 legally blind employees meet the price of $3,500. More than 70 staff members have purchased the device.

Frederick J. Puente, president of Blind Industries, persuaded the board of trustees to make the investment.

“When an associate comes to you with tears in their eyes and tells you that for the first time they are able to read the newspaper themselves, it makes the decision to offer this program a very easy one,” Puente says.

Independent studies suggest that the device brings a new level of ease to daily activities.

Researchers at the University of California, Davis, for example, showed that 12 legally blind adults could perform routine daily tasks with significantly greater ease while using MyEye than while using other devices with which they were familiar.

An article detailing the results appeared in the journal JAMA Ophthalmology last May.

“Patients with low vision often are dependent on hand-held or electronic magnifiers, which may be somewhat cumbersome to use,” Dr. Elad Moisseiev, a co-author and a UC Davis vitreoretinal surgery fellow, said when the study was complete. “This is the first independent clinical study to evaluate this new low-vision-aid device based on novel optical character recognition technology.

“Our results show that it can be a very useful aid for patients with low vision in performing activities of daily living, and increase their independence.”

Optical character recognition — a branch of computer science that involves translating optically scanned printed or written text characters into a coded form that can be manipulated by a computer — is not new. An Israeli inventor, Emmanuel Goldberg, created a machine a century ago that could read characters and convert them into telegraph code. The Irish physicist Edmund d'Albe invented one that could scan text and translate it into audible tones.

By the 1970s, the American computer scientist Ray Kurzweil was working to perfect systems that could read text in any font. It was his company, Kurzweil Computer Products Inc., that combined the use of two new technologies — the CCD flatbed scanner and the text-to-speech synthesizer — into the first optical character recognition computer program in 1976.

In more recent years, scores of devices have hit the market that adapt the technology for use by the blind and visually impaired. Smartphone apps help users count cash (LookTel Money Reader), detect colors (Color Identifier) and hear the prompts on their phones read aloud (VoiceOver).

OrCam co-founder Amnon Shashua earlier founded Mobileye, a firm that developed technology that could use a camera and software algorithms to detect the presence of vehicles on the road.

Computer giant Intel bought MobilEye for $15.3 billion, and Shashua and his team of 100 research and development associates turned the technology toward helping the blind. Orcam spokeswoman Anat Nulman says the company's algorithms allowed designers to make the MyEye camera a mere 3 inches long and the computer-and-battery pack 5 inches long, and thus mountable on the rims of glasses.

In order to take the needed “picture,” the user employs much the same motion a sighted person would use to “look,” directing his or her “gaze” toward the target, then triggers the action in one of two ways: by flipping a switch on the pack or by holding that gaze for a preselected number of seconds.

To “view” a tighter area, the user points at the target, and the device, which is loaded with thousands of pictures of the human finger, recognizes the gesture as an activation command.

The algorithms also enable the device to read print from curved surfaces — a boon for the blind as they peruse cans in grocery stores — as well as to read text as large as the lettering on a billboard or as small as the writing on a passport.

As with all OCR devices, Nulman says, its degree of accuracy depends on external factors such as lighting and distance. Studies have shown areas of needed improvement.

But Schmude, the IT maven at Blind Industries who turned his bosses and co-workers on to the technology, is impressed.

“Just a few years ago you’d have needed a backpack to carry enough equipment to do what this device does, and it would have had to offload the information to a server,” he says. “What they’ve managed to pack into this device is honestly incredible.”

His colleague, Chris Jones, a shipping and receiving clerk, is less tech savvy, but he agrees with the larger point: the device helps him in ways others have not.

Legally blind with very limited vision, Jones, 27, can see larger shapes such as people, streets and moving cars, but not the kind of detail that would identify them further.

A fitness nut, he enjoys biking to work, and MyEye — which he wears atop his sunglasses — allows him to vary his route based on the workout he wants.

Before he got the device, he couldn’t read street signs. Now he points and hears their names read into his earphones.

“The voice will say, ‘Lansdowne Road,’ or ‘Hammonds Ferry Road,’ and I know exactly where I am,” he says.

Jones says he can now read his own bank statements and work orders without bothering his sighted friends, and MyEye even helps him keep tabs on his two young daughters.

At times they’ll playfully try to hide from him in the aisles at Wal-Mart, he says, but to their shock, the facial recognition function helps him “nail” them by name.

He laughs.

“This technology just helps you be more independent and self-sufficient,” he says. “I'm all about that."

jonpitts@baltsun.com

twitter.com/jonpitts77

Copyright © 2017, The Baltimore Sun, a Baltimore Sun Media Group publication | Place an Ad
28°