Synergy in the evolution of hearing instruments

Author: Don Hayes, Ph.D.

The word synergy is derived from the Greek word synergos, meaning “to work together.” It describes the effect of a collection of different elements working together to produce results not obtainable by any of the elements alone. Synergy can be the ability of a group to outperform even its best individual member. This is why the term is often associated with the slogan, “the whole is greater than the sum of its parts”. But it would be more accurate to say that synergy refers to effects that the parts cannot achieve alone, effects that are interdependent without necessarily being better. Thus synergies demonstrated by “the whole” arise more from the relationships and interdependencies between “the parts” than from the individual characteristics of those parts. The unique characteristics of “the whole” that distinguish it from the individual components are called emergent behaviors. Here is an example of an emergent behavior, which results from the unique relationship of two common elements. When the light metal sodium and chlorine gas, two elements that are normally poisonous to humans, are combined they form sodium chloride (table salt) with new properties, which in moderate amounts, are beneficial to humans. But the properties of table salt (emergent behavior) cannot be understood in terms of the properties of its constituent elements; the parts lose their identities and combine to create a new substance. The synergistic effect of the relationship of two otherwise poisonous substances results in a desirable emergent behavior: the creation of the very beneficial compound, table salt.

Why should I care?
Over the last 20 years our industry has seen the progression of technology move from fully analog hearing aids to instruments with completely digital signal processing. New fitting tools and new ways of thinking about hearing aid fitting have advanced in lockstep with the technology along the way. We started with trimmers and screwdrivers on analog products. Then we moved to fitting software on programmable and digital instruments. Now digital products have become ubiquitous throughout the world and clinicians no longer worry about our customers walking off with our screwdrivers to fix their own glasses. In fact, if you have been fitting hearing aids for 10 years or less, you probably didn’t even grasp the humor in that last sentence. The point here is that the confluence of personal computers, digital hearing aids and more recently wireless programming has led to desirable emergent fitting behaviors which yield better results than we could have achieved in the past and nobody is lamenting the lost glories of the “good old days”.

The synergistic impact of faster computers, advances in miniature chip technology and wireless programming have been two-fold. The first emergent behavior, on the part of manufacturers, is to develop instruments for each new generation which are consistently more complex and flexible than the last, but which also require more time to fit and to troubleshoot. The second emergent behavior, on the part of clinicians, is to rapidly embrace the latest technological gadgetry, while being simultaneously repulsed by the increased complexity, requiring forever more time to learn, to fit and to troubleshoot complaints. Emergent behaviors are always different. But they are not always improvements. A perfect case in point is our current fascination with the a la carte menu approach to fitting software. It is this approach which has probably spawned more “fitter fatigue” through unnecessary complexity than anything else we have done with digital hearing instruments in the last 10 years.

A la carte menus and fitter fatigue
The a la carte menu approach works as follows. Manufacturers are rewarded for creating novel technologies or even for spawning variations on a common theme. Take microphones for example. Most hearing aid users reside in relatively quiet listening environments as much as 80% of the time. They prefer an omnidirectional microphone for such situations. They are in noise or in speech and noise type environments, for which a directional microphone may be beneficial, less than 20% of the time. Yet their single biggest complaint is that they cannot function in noise. Manufacturers naturally respond by developing; fixed directional systems, adaptive directional systems, multiband adaptive systems, split omni/directional systems and so on. It is quite likely that one of those directional systems will prove more efficacious than the others for any given individual in their unique listening environments. But nobody can really say which system will be best for each individual in any one or more of their listening environments. Since we cannot provide specific individual direction we put the onus of discovery on the shoulders of the clinician by making all of our directional microphone selections available all of the time in a handy drop down menu box. The manufacturer is then let conveniently off the hook while reinforcing the notion that, “you the clinician are closest to your client and best suited to decide what they should wear. Here is a lovely a la carte menu from which you may select whatever you desire.” If only it were that simple.

The problem with the a la carte approach is that it is not confined to microphones. Fitting software screens are currently filled with a plethora of drop down boxes, pop-up windows and drag and drop menu selections for everything from manual program choices to noise cancellers, speech enhancement, phase cancellers, transient noise limiters, wind noise managers, etc. Furthermore, every one of those parameters comes with a range of strength settings; 1 - 10, off-mild-moderate-maximum and so on. The clinician must wade through a myriad of program choices, features settings and parameter adjustments during every fitting. And every time some new feature gets developed it adds to the complexity and time required to learn the software and undertake the fitting in an already time-constrained clinical setting. Thus the emergent behavior that results from the interaction of these factors is a form of fatigue and paralysis in clinicians. They haven’t the time to do thorough evaluation and adjustment of a range of parameters on all clients and settle instead with leaving most settings at the manufacturer’s defaults in the hopes that they will be the best first choice. It is exactly this a la carte approach of leaving everything up to the discretion of the time-constrained clinician rather than optimizing parameters together during development that leads to fitter fatigue. What is less obvious to the clinician is the effect it has on the engineers who are designing and implementing these new features.

A la carte’s impact on product development
Consider the evolution of the current generation of digital hearing instruments. The devices we fit today did not spontaneously spring fully formed with all sorts of directional microphones and adaptive features out of a test box one afternoon at an Audiology Now! convention. Over a period of years layer upon layer of more sophisticated and complicated technology has been painstakingly added to the body of hearing aid features. One of the pitfalls of this genesis is that we treat each new innovation as if it were a thing unto itself that exists somehow adjacent but distinct from every other hearing aid component. Unfortunately, this viewpoint is fundamentally flawed. Every component in the instrument must work and play well with every other component with which it interacts. Not only must each feature perform a task, it must do so without interfering with the tasks performed by all of the other features. Furthermore, when the performance characteristics of each feature and the interactions between different features are understood the synergies created will encourage the most desirable emergent behaviors from the hearing aid. But that is not what happens in the a la carte approach. This is what happens when every possible parameter is available at every conceivable strength setting all of the time.

When several features are running as if they were adjacent but distinct, as in the a la carte approach, a clinician may choose to put each parameter into any one of (n) states. For example, the fitting software may make; n = 3 microphone states (omni., fixed dir. & adaptive dir.), n = 4 noise canceller states (off, mild, moderate, maximum) or n = 4 speech enhancement states (off, mild, moderate, maximum) available and freely adjustable in up to say 4 listening programs. This means that the clinician could choose from up to 3 x 4 x 4 x 4 = 192 unique sets of feature combinations on one ear during one fitting. Not only does this level of granularity frustrate the clinician, it severely hampers feature development. This is the process by which this occurs.

Ideally new features are designed to provide the greatest potential benefit. Assuming that making a desirable feature such as a noise canceller, directional microphone or speech enhancement stronger will make it better, such features are initially designed to operate at their maximum capacity. Significant benefits are often measurable for each individual feature in this state. However, it does not end there. The new feature must also be integrated with all of the other features in the instrument. It is at this point where synergistic interactions between the new feature and the existing features can cause undesirable emergent behaviors to occur. For example, an adaptive directional microphone may work optimally when designed to provide a very narrow target area and aggressive reduction of sounds in the nulls. Similarly, most hearing impaired individuals may desire aggressive noise cancelling of up to 10 dB or 12 dB to obtain their preferred listening comfort in noise. Taken individually each feature yields the best performance at its most aggressive setting. But once they begin to work together in a single hearing instrument their combined gain reduction impact becomes additive and very undesirable at the most aggressive settings. The synergistic effect of combining these two features at individually desirable settings is an overly aggressive gain reduction for both speech and environmental sounds. The emergent behavior on the part of the wearer is to complain that the hearing instrument shuts down in noise and that they struggle even more to understand speech in noise than they did before. Even a less aggressive fixed directional setting may be too strong when added to aggressive noise cancelling.

The emergent behavior on the part of the hearing aid developers is two-fold. The first is the aforementioned a la carte approach that shows up in the fitting software. If enough levels of strength settings are provided for each feature the developers hope the clinician will somehow be able to choose the correct distribution of settings across the range of parameters so as to provide the optimal performance in the listener’s specific environments. This approach has the previously described baggage attached to it. Also, taken alone such a solution may not work for another reason. It is possible that the synergies between any two or more adaptive features can create an undesirable emergent behavior regardless of how they are set. For example, a noise cancelling algorithm that reduces gain in bands that are dominated by noise may be desired by a listener to promote comfort. Whereas a speech enhancement algorithm which boosts gain in channels dominated by speech may be preferred to improve the clarity and ease of speech understanding in the same environment. Individually they provide benefit. But running simultaneously, acoustic artifacts and distortions may occur as they pull the gain in adjacent bands in opposite directions. Given a noise canceller that can provide 12 dB of gain reduction in one band and a speech enhancement algorithm that can provide 10 dB gain increase in an adjacent band it is not hard to see how both algorithms running at once could yield uncomfortable and unnatural sound quality. The developer’s reaction to these artifacts will be to reduce the strength of one or both algorithms until there is no possible combination of settings available to the clinician which could yield an undesirable emergent behavior. Herein lies the product developer’s dilemma.

Development that minimizes synergies
The strongest speech enhancement setting may provide significant speech clarity benefit with mild or moderate noise cancelling. Or the strongest noise canceller setting may provide optimal comfort in noise without artifacts in the presence of mild or moderate speech enhancement. However, when the strongest speech enhancement is combined with the strongest noise canceller setting the emergent behavior may be distortion and artifacts. Since the a la carte approach does not limit clinician’s options to avoid the strongest settings for both features at once, the developer has no choice but than to reduce the strength of each at the maximum setting. Thus a negative synergy that only creates problems at one pair of parameter settings (maximum and maximum) forces a reduction of the potential benefit at all other possible settings. Unfortunately, it gets worse. Not only can an undesirable emergent behavior occur when the noise canceller and speech enhancement interact at the wrong settings. But such an effect is possible when any combination of two or more features interact with each other. That includes, all of them, noise cancellers, speech enhancement, adaptive directionality, wind noise managers, transient limiters and on and on. In every case, the developer must compromise performance to minimize undesirable emergent behaviors which can occur because of the wide open nature of the a la carte approach. When every feature is allowed to interact freely with every other feature at all possible settings, product developers have no choice but to reduce the effectiveness of all features to avoid undesirable artifacts that may occur at only a few possible extreme choices.

Using synergy in hearing aid development
In the examples above the emergent behaviors caused by the convergence of the synergies between hearing aid features and a less than optimal “a la carte” fitting approach were far more of an enemy than an ally. But it is also possible to take advantage of synergies by carefully controlling the relationships between select adaptive features to achieve a desirable and predictable emergent behavior. The first step is simple, eliminate the a la carte fitting approach. It is not the range of possible settings available for each feature that causes the trouble. It is the fact that each feature (noise canceller for example) can exist in any one of many possible states relative to all the other features (microphones, speech enhancement, etc) at any point in time. However, the problem of undesirable emergent behaviors, such as distortion and artifacts is often a component of a small subset of extreme settings of one or more features. The artifacts can be eliminated without reducing the effectiveness of any individual features by keeping all of the interacting features in a known state relative to all of the others. In this way it is possible to avoid the clusters of parameter settings that create problems.

If the noise canceller and the speech enhancement provide benefit in all combinations except where both features are at their maximum setting then the occurrence of undesirable artifacts can be eliminated by adjusting both features simultaneously such that they can never both achieve that state simultaneously. In other words, if an adjustment to one of the features automatically leads to simultaneous complementary adjustment of the other. Then both features will always exist in a known state relative to one another. The same can be true for greater numbers of features. For example, complementary adjustment of gain, microphones, noise cancellers and speech enhancement are already possible using the SmartFocus™ control in most Unitron hearing instruments. SmartFocus was designed to avoid the pitfall of the a la carte fitting approach. As mentioned above, hearing aid features are not toned down by developers because they fail to work well at aggressive settings, but rather because of negative interactions between features at specific settings leading to undesirable emergent behaviors. The idea with systems like SmartFocus is to eliminate these negative interactions and design in positive emergent behaviors by taking advantage of their synergies; optimizing all of the features simultaneously to achieve specific performance goals. In this case, the emergent behavior creates a new performance goal that was not achievable with any of the individual components that make up SmartFocus.

Figure 1. The synergistic effect of the four features of SmartFocus (microphone strategy, digital noise reduction, spectral enhancement and gain) at three different user settings: neutral, maximum clarity and maximum comfort.

The emergent behavior that SmartFocus was designed to provide is that of a single control that offers a range of adjustment on a continuum between two perceptual constructs, comfort and clarity. If you can agree that for any situation encountered by a hearing aid wearer there is a desirable balance somewhere between hearing with complete clarity (possibly sacrificing some comfort) and listening in total comfort (when clarity is not essential), then it should be desirable to have a single control with which to adjust your hearing aids to achieve it. A noise canceller or a gain reduction may improve comfort, particularly in noise. Speech enhancement or appropriate microphone configurations may also provide better clarity without making many listening situations any more comfortable. But no one of those features truly does both. However, all of their adjustments may be linked together under a single control such that all four features are in known states relative to one another at every position on that control. Thus, as the control is modified in one direction or another, each feature is carefully adjusted relative to the other three so as to achieve greater comfort or greater clarity as desired. Furthermore, since all of the features always exist in a known state relative to the other three, the maximum strength of each feature does not have to be diminished to avoid artifacts. Instead the relative settings across each feature are optimized at every position of the control such that the artifacts are always avoided. Thus, the control is optimized to utilize the synergistic relationships of “the parts” so as to provide the beneficial emergent behavior of “the whole”, a simple comfort or clarity adjustment. The outcome is superior to that achievable using the a la carte approach because the component features under adjustment can provide maximum benefit without being toned down to control for artifacts that may occur at only a few settings. Figure 1 is a schematic depicting the comfort – clarity settings of the four features combined on the SmartFocus algorithm. The end user may adjust the SmartFocus algorithm from the neutral position, which is established at the initial fitting by entering the audiogram thresholds and other information in the fitting software, with a manual control (remote or on-board wheel/lever) or the algorithm may be allowed to automatically switch to either the comfort or clarity setting, depending on how the input signal is classified by the hearing aid’s signal classification system.

In Summary
The real purpose of this article is to describe of the current state of hearing instrument development and fitting while providing a glimpse of the future. The narrative was written employing the concept of synergy. It has been the trend of the industry to provide all new hearing aid features to the clinician with a range of adjustment. But negative synergistic relationships between those features required that the range of adjustment be minimized to avoid artifacts and distortions at some subset of their settings. However, some manufacturers have begun taking advantage of synergistic effects by optimizing whole sets of features together under a single control. Since all features in the set are always in a known state, it is possible to avoid artifacts while actually extending the range of adjustment rather than by reducing it. This leads to a novel (emergent) behavior in the form of higher level control of perceptual constructs such as direct adjustment of sound quality on a continuum from total clarity to total comfort. Thus the developers improve performance by utilizing synergies rather than optimizing them out of their fittings.    
Don Hayes, Ph.D is the Director of Audiology for Unitron. Unitron’s headquarters are located in Kitchener, Ontario, Canada. Dr. Hayes can be contacted at

DISLOSURE: AP Editor Brian Taylor is employed by Unitron.