By now, you may have seen the viral Youtube video (above) where a black man and a white woman point out the shortcomings of an HP Media Smart web cam when it came to tracking the man's face. It's a video -- titled "HP computers are racist" -- that's done with some fun, but it's darn effective in pointing out an embarassing shortcoming of the HP product in a way that quickly became a PR headache for the company.
The topic of facial-recognition tech intrigued me, so I called up Bill Anderson, head of Oculis Labs in Owings Mills, which makes Private Eye, a sophisticated face and gaze-tracking software for security uses. Bill's been working on face-recognition tech for awhile. Below are key points and quotes from my interview with him today:
* There can be settings or environments where a facial recognition system finds it very hard to understand what it's looking at, Bill said. "A very white face like mine ... will not get picked up on some backgrounds if they are a pinkish or yellowish kind of color."
* "It's a relatively hard problem to make a facial recognition packgage smart enough to find faces in arbitrary environments," Bill said.
* "There's actually less difference between human skin tones [black vs. white], as far as the camera is concerned, than meets the eye."
* Computer software can't compete with the brain in terms of facial recognition. "The human brain has a disproportionate amount of its processing power dedicated to face recognition. A huge part of the brain is hard-coded to recognize faces. It starts since birth. It instinctively locks onto faces.... You can think of it as the human brian is several orders of magnititude more capable than the best supercomputers we have on earth, in terms of recignizing faces. That's the competition."
* Software typially is only capable of a "few bag of tricks" in its capacity to find and recognize a face, Bill said.
* Typical steps in software to find faces include: The software first is designed to locate an oblong shape (the head), then it goes looking for the eyes, which are typically two darker spots in the upper part of the oblong shape. Then the software goes looking for a nose, or a mouth, or lips. Once through those steps, the software determines whether it has a candidate for a face. That's just the "first pass."
The second pass involves gauging skin tone, considering intensity and color. "There a narrow band of intensity [regardless of actual skin color] that's unique" to all human skin, he said. "It's looking at something that looks like skin," he said.
The third pass will look for motion, to determine if the image is "acting like a face," and not a static object, like a poster of a human face, Bill said. "Our software looks for motion, because a face moves."
* What Bill thinks might have gone wrong in HP's case: "My guess is that of the various parameters the software uses to determine a face, that face was probably triggering only some of those parameters. It probably found the head and eyes.... but there was something else it didn't find. Something didn't pass that threshold. I don't think it was skin tone because his skin was clearly not the same as the background behind him. It wasn't a contrast problem... Now, it could've been the color band of the skin set by the software was a little bit too low, and the software was looking for skin tone that fit within a certain color band. It could've been the way that his skin tone looked at that particular lighting
* Bill has 250 people in his beta program who help test Oculis software. "We've had issues too, where software would not recognize a face."
* There are good (software) packages out there, but there are no perfect packages out there... and there probably never will be. They're just working at an incredible computational disadvantage compared to the human brain."
* Said Bill: "Having looked at that ["HP computers are racist"] video, I think the software we're using would've found it's face. I'd love to try. I'd suspect that HP has a little bit of work to do to fix their algorithms. I'm sure they'll fix them, but it's a little bit disappointitng they didn't have it working right the first time."