Novel Headset Cameras Track Your Facial Expressions and Read Your Lips

Wearable device consisting of two mini-cameras mounted on earphones can continuously track full facial expressions by following the cheeks' contour and then translating expressions into emojis or silent speech commands, reports a new study. The study's findings will be presented at the Association for Computing Machinery Symposium on User Interface Software and Technology. The tool called C-Face was developed by Cheng Zhang at Cornell University and his colleagues. It looks at the wearer's head's sides and uses machine learning to accurately visualize facial expressions by analyzing small changes in cheek contour lines.

With C-Face, avatars in virtual reality scenes could show how their users feel, and instructors could get important information about student engagement during online lessons. It could also direct a computer system, such as a music player, using only facial signals.

"This device is simpler, less intrusive, and more capable than any existing ear-mounted wearable technologies for tracking facial expressions," stated Cheng Zhang, senior author of C-Face."

Zhang and his colleagues examined C-Face in nine volunteers. When compared with a library of images taken with a front-positioned camera, the device precisely predicted facial arrangements for different expressions (measured by changes at 42 points on the face) to within 1 millimeter.

One limitation to C-Face is the earphones' limited battery capacity, says Zhang. As its next step, the team intends to work on a sensing technology that uses less power.

Source-Medindia