Contact Centres at the Heart of the Information Age
Simply put, contact centres are the factories of customer communication. In the same way that the proliferation of traditional factories has been central to the industrial revolution of the nineteenth and early twentieth centuries, contact centres are firmly at the heart of the information age.
Insofar as artisanal production of goods and services still harks back to a kinder and gentler world when time flowed at a more placid pace, the cost of such nostalgic indulgence is high. Labour is an increasingly expensive commodity as consumers have grown to expect high quality at rock bottom prices. As cheap exploitative labour in poor countries is abhorrent to most of us, the only way to satisfy this demand is through automation.
Automation and Contact Centres
There’s nothing new about automation in factories. From the cotton mills of Victorian England to the ultramodern robotic car production lines of today, industry has always been at the forefront of automation. Factories have always relied on increasingly sophisticated machines to manufacture goods ever more efficiently and reduce the price of their products while boosting profits at the same time. In today’s Customer Experience (CX) factories, or contact centres as they are commonly known, automation plays the same role as it ever did. The product of these factories is customer interactions. The aim, as with any factory, is to turn out as many of these products as possible, of the highest quality in the shortest time using the lowest number of workers.
Over the years, many technologies have been created and put to good use in the customer interaction factories of yesteryear and are still in use today. Workforce Optimisation (WFO) has brought the rigour of statistics into the contact centre. Interactive Voice Response (IVR), Call Scripting, and Robotic Process Automation (RPA) have introduced consistent business logic to each customer interaction. Voice Analytics has helped increase the quality of each interaction. Omnichannel has allowed the contact centre agent to deal with several interactions simultaneously.
Conversational Artificial Intelligence (AI) in Contact Centres
More recently, however, all this has been eclipsed by the arrival of Conversational Artificial Intelligence (AI). Although still in its infancy, Conversational AI holds the promise of automating a vast proportion of the work that contact centres carry out today using humans. The recent storm around Microsoft’s ChatGPT (and its Google rival, Bard) has raised further expectations in this area. Whether AI will ever have the capacity to replace most humans in contact centres in the same way that giant robots have replaced them in the car industry, remains to be seen. Whatever the case may be, one thing remains certain. The contact centre industry’s appetite for automation will only grow as technology keeps moving forward.
New Era in the Contact Centre World
Against this background of unstoppable innovation, one technology has evolved on a parallel track, almost entirely unperturbed by the frenzy of automation revolutionising the contact centre world decade after decade. As the contact centre industry was focused on streamlining the ever-increasing numbers of calls coming into organisations, this was a different kind of automation, aimed singularly at the teams of agents who were going in the opposite direction and trying to actively contact customers by placing outbound calls. Although originally these agents may have been engaging in sales, over the years telemarketing has given way to other less intrusive and more targeted outbound activities such as proactive customer service. As call centres transformed into contact centres and the world of customer experience emerged, it became abundantly apparent that answering customers calls efficiently and effectively was not enough. In this new world, customers expect businesses to be able to anticipate their needs and contact them well before they are forced to pick up the phone.
Our story starts sometime in the early to mid-1980s when some new technologies were hitting the mainstream in the emerging call centre world. Computer Telephony Integration (CTI) enabled the Private Branch eXchanges (or PBXs) that delivered telephone calls to agents to communicate directly with the computers that ran the applications that the same agents were using on their desktops. This delivered many benefits. For instance, call centre applications could now trigger “screen pops” in real time, delivering relevant information to the agent’s screen at the same time as the call was dropping into their headset. Another benefit was enabling software to control the flow of calls through the PBX, providing advanced Automated Call Distribution (ACD) features. Furthermore, a computer program could now not only initiate a call, but also determine through CTI whether that call was successfully connected or not. This became known as Call Progress Detection (CPD).
The arrival of CTI and CPD signalled a revolution in call centres’ outbound calling capabilities. Instead of having agents dial one number after another and hitting busy tones or no answers or fax machines or modems (remember those?) the software could do that automatically and only deliver to agents calls that actually managed to connect? And so, power diallers (also known as auto-diallers) were born. Now, the problem with power diallers is that although they shield call centre agents from the laborious task of dialling and dispositioning unproductive calls, they don’t actually save much time, as the agent still needs to wait for the dialler to make one call after another until it eventually stumbles upon one that actually happens to connect.
The Pioneering Idea Behind the Predictive Dialling
Sometime in the late 1980s, a chap by the name of Douglas A. Samuelson, or Doug to his friends, a statistician working for a company called International Telesystems Corporation somewhere in Virginia, USA, was struck by a great idea. Like most great ideas, it was simple. What if an auto-dialler could dial more than one call at a time for each available agent, in the knowledge that a large proportion of calls will not be successful. Being a statistician, Doug understood that any probabilistic errors would cancel each other out when distributed over a larger team of agents and if the algorithm had a way of learning from its mistakes it would eventually reach an almost perfect equilibrium guaranteeing that all agents were talking to customers most of the time.
If this sounds vaguely familiar it is because what Doug came up with was an early example of Artificial Intelligence boosted by a Machine Learning engine. Of course, in the late 80s, the concepts of AI and ML were confined to the more exotic corners of science fiction and Doug was not pretentious enough to consider his invention worthy of such sobriquets. He chose to modestly call his creation a Predictive Dialler (or Dialer, as he was American). He proceeded to run several simulations of his statistical models in the lab and the results were impressive. In his own words, for some reason only published a decade later, he describes his achievement:
“I used queuing and simulation to invent predictive dialing, a method to determine when computer-directed outbound telephone dialing systems should dial. I included a real-time estimation updating feature that was highly robust against sudden changes in the system’s operating environment; thorough validation to ensure that the models tracked all important features of the real systems; and a modular software design that allowed “plug-in” replacement of the control software, eliminating debugging of field upgrades. The improved systems kept operators busier and drastically reduced the number of calls the systems abandoned because no operator was available to talk to the answering party.” Samuelson, Douglas. (1999). Predictive Dialing for Outbound Telephone Call Centers. Interfaces. 29. 66-81. 10.1287/inte.29.5.66
“The improved systems kept operators busier and drastically reduced the number of calls the systems abandoned because no operator was available to talk to the answering party.”
Samuelson, Douglas. (1999).
Predictive Dialing for Outbound Telephone Call Centers. Interfaces. 29. 66-81. 10.1287/inte.29.5.66
The Beginning of The ‘Golden Age’ of the Predictive Dialler
The results of Doug’s experiments were so spectacular that by the mid to late 90s, several companies developed commercial products that embodied Doug’s pioneering ideas. It was the beginning of the ‘golden age’ of the predictive dialler. The return on investment of predictive dialling technology was so mind boggling that early vendors such as Davox and Melita were able to charge anywhere up to £10,000 per seat for their licences. The stats were compelling. The use of predictive dialler technology was categorically proven to double or triple the proportion of the time agents spent talking to customers when compared to manual or simple auto-dialling methods. The implications were and still are mind-blowing. A call centre could carry out the same work that normally would require 100 agents with only 35-50, saving the cost of half to two thirds of the workforce. Now, if this is not automation at its most impressive, then nothing is. Although these statistics are still valid today, fierce competition had in following years brought the price of predictive dialler products down to much more affordable levels.
Predictive Diallers in the Era of Tightening Regulations
Around the turn of the millennium, it became apparent that predictive diallers were becoming a victim of their own success. Like any other powerful technology, when put in the wrong hands it could be misused in ways that were not envisaged by its creators. This has driven governments around the globe towards imposing ever tightening regulations against its unrestricted use. Ofcom in the UK, FCC in the US, CRTC in Canada and many others found themselves at the receiving end of scores of complaints from consumers who were being harassed by dubious call centres bombarding them with calls at all hours sometimes hanging up as soon as they picked up the phone.
The reasons for these undesirable side-effects were buried deep in Doug’s original invention.
As the methods for determining the numbers of calls to be dialled at any given time was probabilistic in nature, it behaved roughly the same as tossing a coin. If you were to toss a coin a million times you were effectively guaranteed that half would come out heads and the other half tails.
However, if you only tossed the same coin twice, the outcome is entirely unpredictable. What this meant for predictive diallers was that generally the number of calls connected at any point in time would be roughly equal to the number of agents available to handle these calls when taken on average over a team of agents and a whole day of dialling.
However, there would always be some times during the day when the number of connected calls would occasionally exceed the number of available agents just as there would be stretches of the coin toss experiment when heads would outnumber tails. In such rare cases, the excess calls would need to be unceremoniously abandoned. Unsurprisingly, these came to be known as ‘abandoned calls’ or ‘dropped calls’.
‘Self-pacing’ as the Next Step in the Evolution of Predictive Diallers
As predictive diallers were being pushed to the limits of their capacity, it became clear that the more risk that the predictive formula was allowing, the higher the number of dropped calls it was generating. So, the next innovation in the evolution of predictive diallers was the concept of ‘self-pacing’. In today’s parlance, this was another example of Machine Learning, whereby the dialler would constantly monitor the number of abandoned calls it generated as a percentage of all connected calls. Once that percentage approached a pre-set limit, known as the Abandoned Call Rate (ACR) limit, the dialler would automatically cut back on the over-dialling formula coefficients, thus scaling down the level of risk it was taking in order to maintain the ACR just below its desired limit. As soon as this innovation became widely available, most regulators around the world (including Ofcom in the UK in 2006) settled around a figure of 3% for an ‘acceptable’ ACR limit. Not only that, but each dropped call was required to play a short identifying recording alongside a raft of other measures intended to protect the public.
Answer Machine Detection (AMD)
An attempt to solve one of the biggest challenges presented by predictive diallers
Just as the annoying issue of abandoned calls was being, at least temporarily, put to bed, another innovation, destined to trigger an even worse controversy over the coming years, was becoming widespread on the booming predictive dialling scene. As early a 1993, a chap by the name of Chris A. Hamilton, filed a patent application in the USA for what he called “Machine Answer Detection”. The patent was eventually granted in 1994 and the rest, as they say, is history. This new invention was an attempt to solve one of the biggest challenges presented by predictive diallers. Even though the dialler was delivering call after call to agents, a large proportion of these calls, sometimes well in excess of 50%, were to answering machines. This meant that agents had to listen to answerphone messages or voicemails and manually disposition these unproductive calls, wasting precious time and lowering morale. So, Answer Machine Detection (AMD) came to the rescue.
The idea, again, was a simple one. What Chris A. Hamilton and the others who followed in his footsteps observed was that the noise pattern typical of an answering machine and that of a call answered by a real live person were broadly different. A human would answer the phone with a short noise (usually “Hello” or similar) followed by a period of silence, waiting for a reply. In contrast, an answering machine would just drone on all the way to the inevitable beep. Brilliant. AMD did indeed deliver the goods and call centre productivity soared even further as a result.
So, where’s the rub? Well, one of the problems is that people are infuriatingly reluctant to behave uniformly.
Some insisted on answering the phone reciting their family name, telephone number and other pointless information (“Hello, the Bucket residence, lady of the house speaking…” springs to mind).
Other clever clogs thought it was hilarious to record quirky answering machine messages just for the fun of it.
Another problem was that AMD needed at least a few seconds to determine whether a connected call was live or a machine. During those few seconds, the recipient of the call would hear nothing but silence. Annoying, to say the least, but not as bad as the first problem which caused the AMD algorithm to misidentify real live people (including, one presumes, the likes of Hyacinth Bucket), taking them to be answering machines and simply hanging up on them. This anomaly became known as a “silent call”.
Silent Calls and Answer Machine Detection (AMD)
Silent calls are fundamentally different from abandoned or dropped calls since the dialler has no way of determining when it made a mistake and hung up on an actual real live person (if it knew, it wouldn’t have hung up in the first place), therefore cannot play an explanatory recording. This meant that AMD was fatally holed under the waterline. Now, opinions differ as to the accuracy of AMD technologies. Various vendors have historically made outlandish claims as to the results of their tests. These range from 95% down to 65% accuracy but none can claim 100%. It is therefore accepted that all AMD technologies would generate ‘some’ silent calls. What followed was an attempt to limit the number of these calls to what would be an acceptable level, in the same way that abandoned calls were successfully controlled by limiting the ACR.
However, this proved easier said than done. Insofar as abandoned call were easily quantifiable, silent calls were not. So, in 2008, the UK regulator, Ofcom, came up with a compromise in order to limit the damage that AMD was causing without banning it outright. The new scheme relied on the concept of a ‘reasoned estimate’ of the number of silent calls that AMD would generate as a proportion of all calls connected to real live people. Despite Ofcom’s high level recommendations as to the way that this ‘reasoned estimate’ was to be worked out, it remained largely open to some interpretation. Some users decided to listen to a sample of the very short call recordings (two seconds or less) relating to calls which AMD classified as answering machines. Others took the view that the number of silent calls would be roughly equal to the number of answering machines that were delivered to agents. The reasoning behind that was that the errors that AMD made would be equally distributed between ‘false positives’ (live people identified as machines) and ‘false negatives’ (machines identified as humans). Others, who shall remain nameless, took a more ‘finger in the air’ approach and simply made this estimate up as they went along.
Once the ‘reasoned estimate’ was worked out, whatever the method, Ofcom’s requirement was for it to be plugged into a rather complex mathematical formula which took into account several factors, but in essence stipulated that the combined total of abandoned and silent calls should not form more than 3% of all calls connected to real live people. Perhaps unsurprisingly, in the years that followed the number of complaints from the public relating to nuisance calls, both abandoned and silent went through the roof. AMD was being used with impunity as the ‘reasoned estimate’ proved notoriously difficult to police. These were the ‘Wild West’ years of the outbound industry. The regulators found themselves between a rock and a hard place. On one hand it was imperative to clamp down on silent and abandoned calls, but on the other hand the industry was reluctant to discontinue its use of a technology that was delivering huge efficiencies and profits.
Technological Alternatives to AMD
Attempts were made to find technological alternatives to AMD. One option was to tackle it at source. For instance, in the UK, a task force including representatives from dialler vendor companies, under Ofcom’s auspices was set up in 2014 to lobby telecom carriers and mobile operators to provide specific signals that would categorically indicate to diallers that they have reached an answering service or voicemail. Funnily enough, the commercial operators could not see any return on the investment required to develop of such features, and as nobody was prepared to pay for it, they politely declined. As a result of this failed attempt, Ofcom decided to get tough with the industry and launched a lengthy consultation which resulted in new regulations being formally introduced in March 2017. Under the new rules, the 3% limit was scrapped altogether and the use of AMD was practically banned as Ofcom’s tolerance to silent calls was reduced to zero. At the same time, the removal of the ACR safe limit threatened to spell the end for predictive dialling in general as no dialler could work effectively without generating some level of abandoned calls.
Noetica’s Technology Innovations – a New Lease of Life for Predictive Diallers
You may think that this the end of the story. A sad ending to a technology that delivered both huge benefits and bitter controversy over the course of decades. But wait. Like with any good story, there is a final twist in the tale. In 2013, a small British company called Noetica lodged a patent application for an invention called Live Person Detection (LPD). It was a new way of looking at AMD, which reversed the concept. Instead of looking for answering machines, the new technology leveraged a form of AI aimed at identifying actual live people. More importantly, apart from being demonstrably more accurate than AMD, and developed in consultation with Ofcom, it guaranteed that it could be used safely in the knowledge that it will never generate a single silent call. As if this wasn’t enough, Noetica followed that with an even better innovation. A technology called SNoDrop™ was launched in 2014 enabling Noetica’s predictive dialler to demonstrably deliver high performance predictive dialling with a 0% ACR.
Both innovations are gaining momentum today, promising to breathe a new lease of life into a discipline that hasn’t stopped evolving since the day when Doug had his brainwave all those years ago. A very early example of the benefits of AI and ML, the evolving journey of predictive dialling technologies has provided an insight into both the incredible benefits as well as the perilous pitfalls that such innovations can bring. What has become clear however over the last few years is that despite the justified decline and imminent demise of cold calling, the need for outbound telephone contact is not diminishing and may be actually increasing. Outbound calls are simply changing in nature. The decline in telemarketing is more than compensates by outbound activity which is much more focused on proactive customer service, and personalised customer contact. Being able to predict and fulfil customers’ needs accurately and contact them before they do is becoming a crucial part of retaining their loyalty over time.
Customer Experience is the new business battleground and outbound is central to winning those battles. A new chapter in the long story of predictive dialling may be about to start.
