Wearable Technology Detects When Your Voice Needs Rest

Northwestern University researchers have created the world's first smart wearable device that continuously monitors how much people use their voices, alerting them to misuse before vocal fatigue and potential harm set in (1^✔ ✔Trusted Source
Closed-loop network of skin-interfaced wireless devices for quantifying vocal fatigue and providing user feedback

Go to source).

The first-of-its-kind, battery-powered, wireless device, and accompanying algorithms might be game changers for professional singers, teachers, politicians, call-center workers, trainers, and anyone else who relies on their voices to effectively communicate and earn a living. It could also aid clinicians in remotely and continuously monitoring patients with vocal abnormalities during their treatment. The research behind the new technology, developed by an interdisciplinary team of materials scientists, biomedical engineers, opera singers, and a speech-language pathologist, will be published in the Proceedings of the National Academy of Sciences during the week of Feb. 20.

The soft, flexible, postage-stamp-sized device attaches to the upper chest and detects the small vibrations associated with talking and singing. The collected data is then instantly sent over Bluetooth to the user's smartphone or tablet, allowing them to monitor their vocal activity in real-time throughout the day and calculate their total vocal consumption. Proprietary machine-learning algorithms discriminate between speech and singing, allowing vocalists to track each activity individually.

Users can customize their voice thresholds using the app. When consumers approach that threshold, their smartphone, smartwatch, or wrist-mounted gadget offers real-time haptic input as an alert. They can then rest their voices before going too far.

"The device precisely monitors the amplitude and frequency of speaking and singing," said John A. Rogers, a bioelectronics pioneer who oversaw the device's creation at Northwestern University. "These two criteria are the most essential in defining the overall stress on the vocal folds." "Being aware of those factors, both in the present moment and over time, is critical for controlling healthy vocalization patterns."

"It's common for individuals to forget how much they use their voice," said Theresa Brancaccio, a voice expert at Northwestern who co-led the study. "Because they have lived and learned, experienced classical singers are more mindful of their vocal usage." Yet, some people, particularly less trained singers or those who must speak frequently for their occupations, such as teachers, legislators, and sports coaches, are often unaware of how far they are pushing it. "We aim to raise their awareness to help prevent damage."

"What leads people into trouble is when events stack up," Brancaccio said. "They might have rehearsals, teach lessons, talk during class discussions, and then go to a loud party where they have to yell over the background noise." Then, throw a cold or illness into the mix. People have no idea how much they are coughing or clearing their throats. When these events stack up for days, that can put major stress on the voice.

Advertisement

Relationship Between Disciplines

Brancaccio, a vocal health advocate, has spent decades researching strategies to keep her students aware of how frequently they use their voices. In 2009, she challenged her kids to keep a paper budget, which required them to physically record every time they spoke, sang, or drank water, among other things. Almost ten years later, she transformed the concept into Singer Savvy, an app that provides each user with a tailored vocal budget and assists them in staying within that budget.

Separately, Rogers developed a wireless wearable gadget to track swallowing and speaking in stroke patients in partnership with researchers at the Shirley Ryan AbilityLab. The bandage-like sensor monitors stroke patients' recovery processes by measuring swallowing ability and speech patterns. Rogers' team updated the system in the early stages of the COVID-19 pandemic to monitor coughing, a major indication of the illness.

"I wanted to gather more data and make our tracking system more precise and accurate," Brancaccio said. "So, I reached out to John to see if his sensors could help us gather more information."

"I thought it was a great opportunity for us to extend our technologies beyond our very important, but narrowly targeted, uses in health care to something that might capture a broader population of users," Rogers said. "Anyone who uses their voice extensively could benefit."

They also collaborated with Aaron M. Johnson, a speech pathologist, and voice expert, to investigate how the devices could be used to evaluate and track treatment for patients with vocal abnormalities. Johnson, who co-directs the NYU Langone Voice Center, believes the compact, wireless gadget might be used to follow patients' voices outside of a clinical setting.

"A key part of voice therapy is helping people change how—and how much—they use their voice," said Johnson, study co-author and associate professor in the department of otolaryngology at NYU Grossman School of Medicine.

"This device will enable patients and their clinicians to understand voice use patterns and make adjustments in vocal demand to reduce vocal fatigue and speed recovery from voice disorders." "Generalizing vocal techniques and exercises from therapy sessions into daily life is one of the most challenging aspects of voice therapy, and this device could greatly enhance that process."

Advertisement

Algorithms Trained by Singers

Rogers' existing instruments were modified to precisely track vocal load over time. This covers frequency, volume, amplitude, duration, and day of the week. The new device, like Rogers' prior devices for COVID-19 and stroke victims, detects vibrations rather than recording audio. This allows the device to distinguish between the user's vocal activity and the ambient noise around them.

The most difficult challenge was creating algorithms capable of discriminating between speaking and singing. To solve this obstacle, Brancaccio enlisted the help of voice and opera students to perform a range of singing exercises to train the machine-learning algorithms.

A group of classical vocalists ranging from bass to soprano wore the devices while humming, singing staccato scales and melodies, reading, and doing other activities. Each singer produced 2,500 one-second-long singing windows and 2,500 one-second-long speaking windows.

The developed algorithm can distinguish between singing and speaking with greater than 95% accuracy. When used in a chorus, the device solely records data from the wearer and not noise from other vocalists.

"Prolonged talking is one of the most fatiguing activities for people who are training to become professional singers," Brancaccio said. "By separating singing and speaking, it can help people develop more awareness around how much they are speaking." "There is evidence that even brief 15- to 20-minute periods of total silence interspersed throughout the day can help vocal fold tissues recover and repair."

Advertisement

How to Use It

To use the device, simply attach it to the sternum, just below the neck, and sync it with the accompanying software. Rogers' team is now developing a mechanism for customizing vocal budgets for each user. If users suffer vocal discomfort at any point during the day, they will push a button on the app, essentially collecting the instantaneous and cumulative vocal load at the time. These results can be used to create a customized threshold for vocal fatigue. A haptic gadget will vibrate as an alarm when the user approaches or surpasses their chosen threshold.

This haptic device, which is similar in size and shape to a wristwatch, contains numerous motors that can activate in different patterns and with variable levels of intensity to send diverse messages. Within the app, users can also view a graphical display that categorizes information into categories for speaking and singing.

"It communicates with any gadget that has an inbuilt haptic motor because it uses Bluetooth," Rogers explained."Therefore you don't need to wear our bracelet." "For haptic stimulation, you could just use a basic smartwatch."

Other vocal-monitoring systems exist, but they require large collars, tethering lines, and unwieldy apparatus. Some employ implanted microphones to record audible verbal data, which raises privacy concerns.

"They don't operate in a real-world situation for continuous monitoring," Brancaccio added. "Instead of heavy, connected apparatus, I can stick on this soft, wearable device." I don't even notice it after it's turned on. "It's very light and simple."

What Comes Next?

Because Rogers' prior devices recorded body temperature, heart rates, and respiratory activity, the researchers incorporated these features into the vocal-monitoring gadget. They hope that the additional data will aid in the investigation of fundamental scientific concerns surrounding vocal fatigue.

"This is more speculative, but it might be interesting to see how physical activity affects vocal fatigue," Rogers said. "If someone is dancing while singing, is that more stressful on the vocal folds compared to someone who is not physically exerting themselves?" Those are the kinds of questions we can ask and quantitatively answer.

Meanwhile, Brancaccio is thrilled that her students will have a tool to help prevent damage. She hopes that others, particularly non-singers, will recognize the importance of maintaining healthy voice chords.

Reference:

1. Closed-loop network of skin-interfaced wireless devices for quantifying vocal fatigue and providing user feedback - (https://www.pnas.org/doi/10.1073/pnas.2219394120)

Source-Medindia