Disruptive Solutions: The Next Generation of Personal Sound Amplifier Products

Hearing Without Barriers: Jacoti Hearing Suite Brings Amplification, Self-Screening Apps and Professional Services Directly to Consumers
There is an enormous worldwide need for effective, simple, and affordable strategies for hearing assistance. The World Health Organization (WHO) estimates that more than 328 million adults and 32 million children suffer from moderate-to-profound hearing loss that is considered “disabling.” In addition to the most critical cases, around 15 percent of the world’s adult population experiences some degree of hearing loss.

However, the large global hearing health consumer market has barely been reached through traditional products and services, despite the numerous physical and social problems that have a proven association with hearing loss: increased risk of heart disease, dementia, depression, social isolation, and low income. Research shows that hearing aids - the most common product-based solution to date - meet the needs of less than 3 percent of people in developing countries.

Audiology Practices (AP) staff recently had the opportunity to connect with Jacques Kinsbergen, CEO of Jacoti bvba, a Belgium-based hearing technology company that has taken a keen interest and practical steps to change existing paradigms and to ensure hearing without barriers.

The company’s mission - to bring advanced hearing technology in reach of large populations with a special focus on children during their education - is achieved by integrating Jacoti’s software with Internet-ready consumer hardware such as smartphones.

AP: Jacques, thank you for taking time to share information about Jacoti with audiologists who are interested in learning about new technologies and delivery channels for hearing services. Can you provide readers with background information about your passion for removing barriers to hearing and the formation of Jacoti?

Kinsbergen: Since I began my career more than three decades ago in the cochlear implant field, I have been continuously aware of the pressing need for affordable and accessible hearing technology. In 1984, I joined a group of researchers to develop the first European multichannel cochlear implant for people with profound hearing loss at the University of Antwerp, Belgium. This device, called “Laura,” was further developed through a private firm that I co-founded, and eventually became part of Cochlear Ltd.—a company that specialized in the development, manufacturing, and commercialization of the Laura Cochlear Implant.

Jacoti, founded in 1990, started in 2010 together with a group of partners to investigate the needs of those with hearing difficulties and chart the way forward for hearing technologies worldwide, particularly in fast-developing markets such as India. Together with corporate partners like Med-El (a leader in cochlear implants), our goals are to deliver effective, affordable hearing healthcare to people across the globe, to create a sustainable services-based hearing healthcare model and - as important to us as profitability - to have the greatest impact possible.

AP: Jacoti operates on the assumption that smartphones using established technology can deliver effective and accessible hearing assistance. Why do you think that this approach will result in increased adoption of amplification, improved satisfaction, and better outcomes for those who are hearing impaired?

Kinsbergen: The cost of traditional hearing aids are very high, and therefore access to them is restricted, especially in many developing countries. Jacoti believes that consumer technologies like smartphones have highly advanced audio processing capabilities that make them a perfect vehicle for screening, data collection and treatment, especially in under-served communities. By leveraging affordable mobile consumer technology, Jacoti can help consumers and providers minimize existing barriers to hearing health care, improving access and reducing the stigma often associated with hearing loss.

We believe that personalizing audio in consumer devices should become normal for all users. According to the Hearing Loss Association of America, it typically takes seven years for people to seek treatment from the time they first suspect that they might have a hearing loss. Therefore, by normalizing the idea of hearing assistance through the use of smartphones and other widely used and familiar consumer devices, we can more successfully encourage those who could benefit from professional audiological services to seek help much sooner.

AP: Can you describe the products that are available in Jacoti Hearing Suite?

Kinsbergen: There are four interconnectable applications in Jacoti Hearing Suite, all of which are designed to improve hearing and communications for adults as well as children. The apps are currently available as free downloads at the Apple Store, so that both audiologists and users can become familiar with their capability and Jacoti’s approach:
  • Jacoti Hearing Center, a patented, Conformite European (CE) approved and FDA-registered self-test hearing application.
  • Jacoti ListenApp, the world’s first medically certified software hearing aid application that is both CE approved and registered by the FDA as a medical device.
  • Jacoti Lola Classroom, an assistive listening solution for classrooms, meeting rooms, and lecture halls that provides low-latency multi-peer wireless audio streaming over consumer-grade Wi-Fi.
In addition, these apps connect to myJacoti, a web service that allows users to store their audio profiles in the cloud, share them across devices and connect to a remote hearing professional for remote fitting assistance. The cloud-based myJacoti is a perfect fit with many of the key recommendations from the recently released report by the National Academy of Sciences (NAS) to collect detailed, longitudinal data on a person’s hearing loss over a user’s lifetime and also enable that user to have broad access to that information to do with as he or she sees fit.

Our approach is influenced by the following design philosophy:
  • User-centric and user-driven design - All Jacoti apps place the user at the center of his or her listening experience. For example, the self-test in Hearing Center is so easy and enjoyable that my 6-year-old granddaughter treats it like a game - but a game that generates clinically valid audiological data! Lola Classroom is designed so that a teacher can set up a sophisticated multi-directional, multi-user assistive listening system in her classroom with only a few taps. In addition, users has complete access to their hearing health data, tracking how their hearing changes over time, and easily sharing their data with the healthcare professionals of their choice. In keeping with the NAS report’s recommendations, users of Jacoti products have full ownership and control all their data, including audiograms and the history of their hearing fittings.
  • Interoperability - All Jacoti apps work together in a complementary fashion. For example, the data collected by Hearing Center can be used to create an audiological fitting in ListenApp. And ListenApp fittings can then be used to create personalized hearing for students using Lola Classroom. Both test data and hearing fittings are stored in myJacoti so they can be downloaded to any iOS device the user has.
  • Simplified servicing - Jacoti provides hearing assistance via apps on common off-the-shelf consumer hardware, which improves reliability rates. Consumers can simply exchange faulty or obsolete devices, essential in remote areas where the turnaround on repairs can leave a user without the ability to hear for weeks.
AP: The traditional product-focused model poses significant challenges that, for many people, inhibits access to advanced hearing healthcare. Can you describe Jacoti’s service-based model of products that allows consumers to acquire audiological care via a subscription service?

Kinsbergen: Jacoti recognizes how important the audiologist is in providing comprehensive diagnostic and treatment services. There is a worldwide shortage of audiologists. Most communities in the world lack face-to-face access to the experts needed to set up, fit and readjust current hearing hardware. Jacoti seeks to extend access to audiological services, providing audiologists with a global reach for their services, enabling them to care for many more patients than they currently do.

For example, in order to provide under-served communities with access to high-quality hearing support, Jacoti Hearing Center’s self-test uses a standard mobile phone and standard earbuds. Users can then fine-tune and update their fitting via cloud-based consultations with Jacoti’s hearing experts.

Jacoti’s hearing health services will be available on a subscription basis for comprehensive, life-long hearing healthcare. Data collected by Jacoti’s apps - far more extensive and detailed data than can be collected by traditional hearing aids - will enable a user’s hearing health to be monitored and their device’s audio personalization to be precisely adjusted over the course of their lifetime and this as part of their overall healthcare data (eHealth). Audiologists will work with this data to provide advanced hearing healthcare services.

AP: What is next for Jacoti?

Kinsbergen: Together with our partners, Jacoti is developing the next generation of hearing technologies. We are extremely excited by the recommendations proposed by the NAS report on adult hearing loss, all of which are so compatible with our mission, it’s as if they had written our strategic plan!

We have a platform that can fully support NAS research recommendations to gather extensive, consistent, and detailed longitudinal data on hearing loss. We strongly believe in evidence-based hearing personalization for everyone, regardless of whether they have a profound hearing loss or have normal hearing and simply want to hear a bit better in a noisy environment. We have always believed that everyone should have easy access to their hearing data and be able to share it with whomever they authorize. And we have a deep commitment to bringing quality-hearing healthcare to under-served areas via Telehealth, the Internet, and other modern communications technologies

We believe that hearing loss compensation and personalization of audio should become a fundamental part of all audio-enabled devices. We are seeking to license Jacoti’s technologies in order to deeply integrate them into consumer products such as mobile phones, hearables and other hardware. Our goal is complete inter-operability of audio connectivity and the use of extensive data mining to provide advanced hearing support in the most difficult situations. Other active projects include research into applying machine learning to hearing health which enable us to provide cloud-based adaptive sound personalization for difficult listening situations, where it is needed the most. Jacoti is also working on improving music perception for people with moderate to moderately severe hearing losses.

In all our efforts, we will always ensure that Jacoti’s hearing services and products put consumers in the driver’s seat. After all, hearing loss is both a medical condition and a consumer challenge. Therefore, it needs consumer-driven and consumer-oriented solutions.      
Using My iPhone and Jacoti ListenApp
Author: By Richard Einhorn

In 2010, I experienced Sudden Sensorineural Hearing Loss and, overnight, lost all usable hearing in my right ear (the sound I do hear now in that ear is completely garbled and extremely loud). My left ear has a combination of moderate-severe otosclerosis and a mild sensorineural hearing loss. In even relatively mild noise, this unusual asymmetrical hearing loss can create enormous problems for me in terms of speech comprehension. Often, the distorted sound in my right ear masks the (relatively) clear sound in my left. In environments with significant noise or high levels of ambience - like the New York City restaurants where I love to meet friends, or in movie theaters - I often cannot understand a word, even with the finest hearing aids.

The reason is simple: hearing aids - as incredible as they are technologically - often do not provide a high-enough signal-to-noise ratio. For one thing, an in-ear mic placement is often too far away to pick up a clean audio signal despite the use of advanced beam-forming and highly sophisticated on-board digital signal processing. Furthermore, depending upon assistive listening technology that venues provide is dicey. While many theaters have decent and reliable assistive listening systems, many do not. And of course, very few restaurants, especially in urban areas, provide truly quiet dining alcoves, let alone hearing devices for diners with hearing loss.

Fortunately, due to my professional background in music composition, record production, and audio engineering, I’ve been able to come up with situation-specific solutions that enable me to continue to write my music as well as go out and stay fully connected with my life.

It turns out that the iPhone has superb specifications and excellent sound performance rivaling professional audio equipment (Rockwell, 2013, 2014). Combine the iPhone with terrific, affordable in-ear earphones (for example, from Etymotic Research or Futuresonics, between $100 and $220), attach a good directional microphone - like the amazing Shure Motiv MV 88 ($150) - to the lightning port, and the iPhone is transformed into a fully professional and portable live sound amplifier - literally, the world’s best-sounding “Pocket Talker.” All that’s needed is a hearing app.

And that’s where Jacoti’s apps come in. I test my hearing with Hearing Center which stores my audiogram in the cloud. When I sign into my account from ListenApp, the apps uses that audio program to provide me with audio personalized to my hearing loss via an audiological fitting complete with appropriate equalization, compression, volume control, several presets, and bass/treble adjustments.

So when I go somewhere where it is too noisy for me to hear clearly, I simply remove my hearing aids, put them someplace safe and fire up ListenApp on my iPhone. I’ve used it at concerts, at plays, at movie theaters, at business meetings, and in the noisiest restaurants. It works incredibly well.

Occasionally when meeting people, I’ve forgotten to pack my iPhone or had the battery run down. So I borrow a friend’s, download ListenApp, sign into my myJacoti account and then I can hear with ListenApp customized to the same audio profile as on my own phone - I just need to remember to return the phone when we’ve finished eating!

Please don’t misunderstand. I love my hearing aids; given the severity of my hearing loss, they are essential to my life. What my iPhone and ListenApp do is add to the palette of hearing technologies available to me in a way that’s non-stigmatizing. And it’s convenient: like everyone I know, I always have my phone and earphones with me - and carrying the tiny (and rather beautiful) Shure mic is no big deal. It’s a very comfortable assistive listening system to use - I never feel self-conscious.

It’s obvious to me that for people with serious hearing losses like myself, consumer audio technology - like that available in smartphones - can and should be far more tightly integrated with hearing aids. This will require, among other things, a common, non-proprietary wireless standard and a shift away from a strict medical model of hearing healthcare to one that incorporates features of the burgeoning e-health field - including self-testing, remote assistance, and longitudinal data storage in the cloud.

It also seems obvious that people with mild to moderate hearing losses should have easy access to audiologically effective hearing personalization via consumer devices. This will serve to make hearing assistance nothing to hide - it’s simply something anyone would do whenever it gets too noisy. In fact, there is data out there that demonstrates that the proactive use of hearing tech to improve the signal to noise ratio helps with comprehension even for people with normal hearing (Magann Faivre, et al, 2013). And the implications of making affordable, easy-to-use, and attractive hearing assistance “normal” will have a profound effect on people with learning disabilities and/or hearing loss. Millions of people, especially students, could benefit.

This is the importance of Jacoti’s approach and why I believe the future of hearing loss technology will prominently feature deep integration with common consumer devices.

Disclosure: Richard Einhorn is a consultant for Jacoti, bvba.    
Magann Faivre, M., Ismail, F., Sterkens, J., Thunder, T., and Chung, K., The Effects of Hearing Loop Systems on Speech Understanding and Sound Quality in Real World Listening Environments, Northern Illinois University, 2013.

Rockwell, K., iPhone 5 Audio Quality, Web site accessed June 2, 2016 at, http://www.kenrockwell.com/apple/iphone-5/audio-quality.htm, 2013.

Rockwell, K., Apple iPhone 6 Plus, Web site accessed June 2, 2016 at http://www.kenrockwell.com/apple/iphone-6-plus.htm#measurements, 2014.

Richard Einhorn has become one of only a small handful of living composers who not only reaches a large worldwide audience but whose music receives widespread critical praise for its integrity, emotional depth, and craft. Einhorn has written opera, orchestral and chamber music, song cycles, film music, and dance scores. Before turning his attention exclusively to composition, Einhorn worked as a record producer for such artists as Meredith Monk and The New York Philharmonic. His production of the Bach Cello Suites with Yo-Yo Ma won a Grammy for Best Instrumental Performance. Einhorn graduated summa cum laude in music from Columbia University. Einhorn currently serves as a member of the board of directors of the Hearing Loss Association of America (HLAA).
Alango Goes Beyond PSAP to Make a More Holistic Hearable
In its October 2015 report, the President’s Council of Advisors on Science and Technology (PCAST) identified several reasons for the lack of adoption of hearing aids by consumers, including:
  1. High cost - Over 75 percent of potential users in United States identified financial factors as a barrier.
  2. Social stigma - the association of hearing aids with old age or infirmity.
  3. Poor performance and/or poor fit
As Albert Einstein once said, “No problem can be solved by the same kind of thinking that created it”. Hearing aids are not going to be sufficiently cheaper, look much fancier or start performing significantly better unless they transform into something very different.

Today, a new type of electronic device has emerged – the wearable. Smart watches, fitness bracelets, smart glasses, smart clothes and smart “others” are often considered “the next being thing” by experts and electronics industry professionals. Some of them use sound for communication with the user and this class of wearables is, sometimes, called “hearables”. The PCAST report specifically mentions this type of product, noting, “Hearables can combine multiple functions (from listening to music to accessing calendar appointments), coordinate with other technologies (such as smartphones), and record health information and vital signs”. Certain Personal Sound Amplification Products (PSAP) definitely belong to this category.

Alango Technologies has been developing and licensing digital voice and audio enhancement solutions for 15 years. During these years, they have accumulated a substantial theoretical knowledge as well as extensive practical experience in microphone array, noise reduction, echo and feedback cancellation and other digital acoustic technologies. In fact, their algorithms are integrated into a variety of cars, mobile phones, Bluetooth audio accessories, intercom and teleconferencing systems. Roughly two years ago, Alango decided that it was the right time to start leveraging their experience in the consumer electronics industry to try to help hearing impaired people that cannot or don’t want to use traditional hearing aids.

AP had a chance to interview Alango CEO, Dr. Alexander Goldin to get more information about the next generation of hearing technologies under development.

AP: Why did you decide to enter the hearable market?

Goldin: Wearables are the fastest growing segment in the consumer electronics market. And hearables are the next big thing in wearables. We, at Alango, believe that a low cost, self-tuned, personal sound amplifier with additional assistive and communication capabilities is the most needed hearable device. 

Hearing enhancement will enter into the realm of consumer electronics and no longer be associated solely with a medical device.  Increased competition, unfettered distribution, and a more open development environment will be a tremendous benefit to the consumer.

AP: Please tell us about the products that Alango has under development right now to help enhance hearing?

Goldin: Alango is working on three important products and technologies to help enhance communication capabilities for consumers:
  • PersonaSound- sound customization that adjusts the sound you want according to your device, your hearing and your current listening environment. 
  • HearPhones- a licensable software reference design transforming a Bluetooth headset or other “hearable” into a truly personal, self-tunable sound amplifier with assistive listening and other advanced capabilities. HearPhones incorporates PersonaSound plus hearing enhancement and assistive listening functionality.
  • Smart Assistive Listening Transceiver (SALT), a powerful hardware and software platform integrating the full HearPhones solution with extended assistive functionality, utilizing four additional SALT microphones, loudspeaker, vibration motor, motion sensor and telecoil.

PersonaSound allows individual customization and playback sound enhancement of both voice and music, and is designed to improve the listening experience for those with normal hearing and those with a hearing disorder. PersonaSound technologies are not intended for enhancing live music listening or face-to-face communication. They improve listening experience of recorded or streamed content, as well as, of remote voice communication. The improvement is done according to 3 factors:
  1. listening preferences of the user (hearing loss can be taken into account),
  2. ambient noise at the user’s location (sound level and frequency equalization is set based on ambient noise so that intelligibility is preserved without significantly increasing volume), and
  3. the user’s device properties (characteristics of the device are taken into account to minimize signal distortions and enhance the listening experience).
AP: How does HearPhones use the PersonaSound technology?

Goldin: A modern Bluetooth headset includes all basic hardware components present in digital hearing aids: microphone, speaker, digital signal processor, battery, non-volatile memory and connection to a mobile phone. All that is necessary to transform it into an advanced hearing aid is the corresponding software. HearPhones is ready, licensable solution allowing Bluetooth headset manufacturers to integrate hearing enhancement functionality easily without investing significant resources.

After integrating the HearPhones solution, a Bluetooth headset can additionally operate as a fully customizable personal sound amplifier product as well as a wireless receiver for a remote Bluetooth microphone or TV transmitter. A HearPhones-enabled headset can be controlled from a connected mobile phone using a special smartphone (Android or iOS) application. Besides controlling the standard Bluetooth headset functionality, the HearPhones smartphone application allows the user to:
  • Switch the hearing enhancement on and off.
  • Perform a simple and intuitive listening preference test. The test is performed with the HearPhones enabled headset and the test results are used to define the most suitable sound processing parameters for the specific measured hearing preferences.
  • Fine tune processing parameters for different use cases using signals prerecorded in specific environments (voice conversation in quiet environment, restaurant, lecture, live music concert etc).
  • Fine tune processing parameters in real live situations.
  • Enable communication with a remote microphone or TV system and fine-tune hearing specific sound enhancement for such case.
AP: You mentioned that SALT incorporates the HearPhones solutions and provides additional assistive technology features. Can you elaborate?

Goldin: SALT can be used as a small, self-tuned body worn hearing aid, remote microphone receiver and TV listening system. SALT also allows direct, wireless, two-way person-to-person voice communication via a Bluetooth voice link between two SALT devices. This function is very helpful for two hearing impaired people in challenging environments, but can be useful for people with normal hearing as well.

In addition, SALT can switch between “user mode” (receiver) and “auxiliary mode” (transmitter). SALT integrates additional hardware components thus creating a powerful hardware platform for a variety of assistive applications.

Together it enables many additional SALT applications. Six of them are briefly described below, but more can easily be “invented”.

Wireless microphone: An auxiliary SALT given to another person or left on the table works as a remote

Hand-held “interview” microphone: User can hold SALT in hand directing it toward the sound of interest in difficult acoustic environments. The resulting stereo sound is delivered to the user via earphones or to user’s hearing aids via induction neckloop (mono).

TV listening system: Auxiliary SALT is plugged into the TV headphone (line) output. The noise free, reverberation free signal is transmitted to user’s SALT directly via Bluetooth with low latency audio link. It can be delivered to user’s ears via earphones or induction neck loop.

Vibration alert: SALT’s powerful vibration motor alerts users of incoming calls when a hearing impaired person does not use SALT sound amplification. It can also work as a powerful vibration alarm.

Intelligent fall detector: Hearing impaired people have three times higher probability to fall than people with normal hearing. SALT built-in three dimensional motion sensor allows automatic detection of such fall following by lack of movement. When this sequence of events is detected, SALT will vibrate, play an alarm signal (via earphones and the built-in speaker) and ask the user if any help is needed. If the alarm is not deactivated, SALT will automatically call a preprogrammed number(s). User will be able to talk “hands-free” when the call is answered. The emergency numbers are programmed from the user’s smartphone application.

Telecoil receiver: Built-in telecoil allows SALT users to receive direct sound via earphones in places equipped with induction loop (theaters, churches, supermarkets, etc.). The sound will be processed and enhanced according to user’s hearing preferences based on the hearing preference test conducted.

What are the advantages of using devices that integrate all three layers of solutions being developed by Alango?

Goldin: Bluetooth headsets with integrated PersonaSound™ package (such as HearPhones™ or SALT™) allow the user to determine his/her hearing preferences and optimize sound amplification as well as other device sounds according to the results.

In difficult cases, or when help is requested by the user, the results of the hearing preference test can be sent by the smartphone application to an on-line audiologist office that will provide recommendations for the manual device parameters tuning. In cases when profound, suspicious or otherwise problematic hearing loss is detected, the application can recommend the user to seek audiological or medical care.

AP: We understand that Alango will also license the HearPhones software. Why did you decide to do that?

Goldin: People have different tastes, lifestyles, budgetary limitations and other preferences or prejudices. One device cannot fit all cases and one company, definitely cannot provide the optimal solution for everyone. That is why we see Bluetooth headsets in different forms and with unique features. Alango wants to provide device manufacturers with the necessary freedom to invent, to customize and to market their devices to different users. It will also allow for the introduction of features beyond hearing enhancement such as heart rate monitoring, fitness and activity tracking and other new technologies that improve our lives.

AP: Do you believe that PSAPs and other hearables will disrupt the traditional hearing aid industry?

Goldin: Yes, of course! Hearables will eventually integrate all the functionally found in modern hearing aids including the full customization to the user’s specific hearing preferences (or hearing loss). However, they will be sold for a fraction of the hearing aids cost while providing additional assistive listening functionality.

Sound personalization is the key to success. For example, with Alango technologies, we expect that 70% of the users shall be able to do such personalization without requiring any extra help, just by interacting with an intuitive software application guiding them through a hearing preference test performed with the same device. Real time adjustment of processing parameters in real life situations must also be a simple straightforward procedure. We are still in the process of developing methodology for hearing self-tests and transformation of the results into personal signal processing parameters, but the first versions are not far off.

Virtually millions of potential users around the world will finally have access to hearing enhancement devices they badly need, but cannot afford or use for one reason or another. Due to their connectivity to mobile phones and control by mobile phone applications, these devices naturally become members of the IoT (Internet of Things) family thus allowing users to create social networks exchanging experience and helping each other with fine-tuning various device parameters.

The good news is that new hearables and other technologies will actually increase the number of people that realize that they have a hearing impairment and will eliminate the stigma of using hearing aids thus significantly increasing the number of hearing health conscious consumers.

AP: What will the impact be for the audiology profession?

Goldin: I think it opens new exciting opportunities, as the number of people potentially seeking help will only grow. There will be more people who are aware of their hearing difficulties discovered during hearing self-tests. There will be many people who cannot fit the devices themselves, but can be helped remotely by on-line audiologists. In the case of a problematic hearing loss detected, these users will be advised to receive a comprehensive audiologic exam and treatment.

Let’s work together to make it happen for our mutual benefit and for the benefit of our patients and users.

AP: Thank you for taking time to share information about the products and technologies under development at Alango Technologies. Please keep us posted on the timeline for their release to the public.    
Dr. Alexander Goldin is the founder and the current CEO of Alango Technologies, a company he founded in Israel in 2001 to develop a new generation of front-end voice enhancement technologies. He can be contacted at alexander.goldin@alango.com.