Measuring Performance in Call Centers

In my first post, I introduced myself: the work that I do is call center consulting, trying to help companies provide better customer service.  This post will elaborate on the way that inbound call centers, which represent the majority of my experience, measure the performance of their centers and of their on-the-phone personnel, who I shall call “agents” as a generic term, since the titles companies give them varies.

I have been involved in the measurement of call center performance for twenty or more years, and have been deeply ambivalent about it the whole time.  The ambivalence stems from an uneasiness about some of the ways that measurements are used.  My uneasiness is reflected in the contrasting statements in the post called “Misapplication of Measurement” attributed to Condorcet and Constant.  Condorcet felt that standardization would lead to freedom and quality, and Constant felt that it would lead to suppression of difference of thought.

Having owned a business myself, I know the importance of measuring the results of the business as a whole, and of the employees individually.  I also know that unless carefully managed, employees may not take “ownership” of their work and will act as slaves doing their master’s bidding, to insure their continued income.  One way is healthy for both owner of the business and the employees, the other is unhealthy for all concerned.

In business, performance is always measured, The principal measure is always aimed at bottom-line considerations with the main questions being is the business generating enough income and enough profit?

Since call centers are not necessarily the business that a company is in (I am excluding consideration of those companies that provide outsourced call center services for other companies, for the time being), the call center will be judged by how it contributes to the bottom-line measurement.  This usually requires figuring out what the “product” is, and how that “product” either contributes revenue or helps to avoid costs.

The principal product of all inbound call centers is the mass production of customer interactions.  Call centers come in two flavors: inbound and outbound, and in some instances, a single center may have groups performing each of these.  The kinds of business function performed by inbound call centers ranges from catalog sales to queries about information to managing accounts.  Outbound functions are generally sales oriented, although collections make up a substantial amount of outbound call center calling.  In inbound call centers, the object of the interaction is to satisfy the customer’s reason for calling.

Mass production of services is relatively new: mass production is usually associated with manufacturing tangible goods.  Mass production of goods relies on parts that have standardized sizes so that the parts are interchangeable.  It also changes the organization of manufacturing goods from one producer – one product, to many part-producers participating in making multiple copies of the same product.  While it is obvious that there are advantages to this method of producing goods, since the beginning of the industrial revolution in Europe, there has been the feeling that mass production de-humanizes the work.  Even before the advent of production lines, the resisters were labeled with names like “luddites” and “saboteurs”.  But the products that were turned out proved to be so successful at providing answers to many of the problems of living that returning to only hand-crafted manufacture of goods will not happen.

Mass production of customer interactions with the goal of satisfying the customers is not as mature as the mass production of tangible goods, and since the satisfaction of a customer is an emotional state that accompanies accomplishing the customer’s desired transaction, measuring the results of inbound call centers is a more subtle and trickier proposition.  For a long time, inbound call centers measured their results based on data captured by their phone systems: how many calls were offered; how many calls were actually answered; how quickly were they answered; how long did it take, on average, to handle each of the answered calls; how many calls abandoned – generally looked at as group numbers.  Based on these group level statistics, judgments can be made about whether there were sufficient numbers of people to handle the amount of work being processed, and if this was cost-effective for the company.

However, once the managers and directors of call centers figured out that they had data about each of the individual agents performance, agents were judged by their share of the data.  They could tell how many calls an agent handled per hour, and what their average handling time was for the calls.  Targets could be set for agent productivity, within certain limits: no one will ever handle more than 30 calls that are 2 minutes long in an hour.

Group level statistics led to some strange equivalencies.  For example, if customers had to wait more than 30 seconds in queue, customer satisfaction must be low.  Or, if too many customers abandoned because they were in queue too long, customer satisfaction must be low.  If calls, on average, were more than 4 minutes long, we are wasting the customer’s time, so satisfaction must be low.

When it was pointed out to companies that none of these measures had ever been checked against their customers’ perception of the service and that these various timers did not measure emotional states, rather than asking their customers, companies expanded the metrics for measuring the performance of their agents.  At first, supervisors were asked to monitor a portion of each of their agents calls per month, so they could hear what was going on, and then technology was brought to bear: calls were recorded, compressed, and quality assurance experts were trained to listen to and evaluate/grade the interactions.

With the recordings, both the agent’s and the caller’s portions of the interaction were not only captured for the precise words used, but usually, emotional states could be inferred.  A caller who becomes abusive, for instance, was fairly easy to judge as not being satisfied.  But what would drive a caller to be abusive?  Was it the company’s policies, or the agent’s presentation of them?  Only in rare instances was the fault deemed to be the company’s policies: usually it was the agent’s “inadequate presentation” of them.  But, since not all calls were recorded, nor were all recordings eventually listened to, an agent might stand a chance of “getting away” with occasional bad performances for a while before being “caught”.

At the same time, because recordings can be listened to multiple times, those calls for the agent that were recorded and evaluated could be used for multiple purposes.  In the cases of relatively new agents, both the agents and their supervisors or coaches could listen to calls to help the agent figure out how better to handle their callers’ requests.  The recorded calls could be scored, and if an agent disagreed with a score, could be judged by a third party.  Additionally, the recordings of agents who were consistently caught with bad performance could be used to build a case for terminating them.  And in the case of an agent doing an especially good job, in addition to making them a candidate for advancement, the recording can be used for teaching other agents how to handle calls.

Within call centers, I’ve seen monitoring systems used in all of these ways.  In those centers that used this measurement in a way that the agents perceived to be fair, the agents know what they will be graded on, which means that their company has developed standards for interactions, with lists of the attributes that need to be part of each call.  The agents can disagree with scores and have the evaluation arbitrated.  Usually, these centers as a group perform better, and the agents individually perform better than those using recordings of calls strictly in a punitive way.

However, this gets right to the heart of the dilemma I raised: if the individual freedom of the agents is so heavily constrained, can they ever take ownership of their work?  The major difference between these agents and stage actors is that the scripts, attitude and acting the agents perform is supposed to be nearly the same for each call every hour of every day worked.  For the stage actor, the play only has a single run-through of the script for each performance, and the actor uses nuances to elucidate the role or the play’s meaning.

Does this standardization of scripting and attitude provide “freedom” to the callers?  I’m not sure freedom applies, though after the first call, the company will have established the quality of service that a customer will expect in the future.  The only time there is a problem for the customer occurs when there is a mismatch between what the customer wanted to accomplish and what the agent has been empowered to perform.  And it is here that call centers can be successful or fail: if they have defined the service that they will deliver in a manner broad enough to accommodate the desires of a large enough portion of their customer base, the center will be successful, the product delivered will satisfy their customers.  Believe it or not, there are a fair number of centers that are successful delivering this quality and breadth of service.

Since the numbers and evaluations described above are essentially oriented to internal performance, without checking with customers what their perception is, and since many companies I visit don’t check with their customers, I have to look for indicators among the data that the centers may have gathered internally.  There are a few numbers that I try to obtain when I am consulting with a call center, measurements that help me to understand how well the center as a whole is doing.

I do look at their telephone switch data, the group numbers above.  When there are anomalies, I try to figure out why they don’t add up correctly.  For instance, in one center I worked with, I found that the number of calls offered (NCO), the number of calls handled (NCH) and the number of calls abandoned (NCA) did not add up, when they should have.  The formula would be NCO = NCH + NCA.  Pretty straight forward, but in one particular instance, I found that NCH + NCA always was less than NCO.  During the early hours of the day it was only one or two calls different, but during peak hours, it was off by twenty to thirty calls.  The number representing the difference showed up in a report as being transferred, so I asked, to whom do they get transferred to?  The answer was that they didn’t think anything was being transferred.  So we looked further, and found that if an agent had not put him/herself in an unavailable state when leaving their desk, a call would ring their phone extension three times and then be transferred to a voicemail box.  This customer’s staff did not know there was a voicemail box that calls could be transferred to.  When we checked the voicemail box, there were thousands of calls, the earliest being over a year old.

However, there are some numbers that I find even more diagnostic.  One of them, which is much more difficult to capture, is the number or percentage of calls that are resolved during the first call.  As a customer, as a caller to a company I’ve done business with that uses a call center to handle customer interactions, very few things frustrate me more than having to call back multiple times to resolve a simple problem.  Many centers do not track First Call Resolution (FCR) because it is difficult to capture, but there are ways.  I have recommended to centers that they capture and understand FCR since I have found it to be an important indicator of how well the center is delivering their service to their customers.

But it is an indicator only, as are all numbers associated with call centers.  If you are trying to deliver a product called “satisfaction,” an emotional state, generally you will have to understand more than just the numbers.  But when the captured numbers for certain types of data are outside of reasonable parameters, they can help determine where to start inquiries.

A story that I’ve used to illustrate why one has to understand the numbers that are outside the normal is based on a very clever television advertisement from the 90’s.  (Clever, yes, but effective?  I remember the ad: I don’t remember which company ran the ad.)

The scene starts with a caller making travel arrangements with a call center agent, with the caller on-screen when he is speaking, and the agent on-screen when she is responding.  When they finish, she asks if there is anything more.  The caller says “yes, hang on a second” as the camera pulls back from the caller, revealing a telephone box (remember them?) on a military installation with a line of soldiers behind the first caller, who hands the phone to the next in line.  The ad ends up with the agent looking exhausted but happy, and her supervisor saying something like: “Well, Sally, I see you only took one call today.”  Clearly, that supervisor needs to understand that number.

Another diagnostic number that I look for when I work with call centers is their rate of turnover of agents.  If the rate is high, I usually assume that they are asking their agents to do work that the agents would be glad to leave.  Besides being very expensive, since hiring and training of new agents is not cheap, and during training, agents are not productive at all, it either indicates that the agents are being underpaid for what they are being asked to do, or they have insufficient empowerment to do anything but frustrate customers and themselves, or both.  I’ve seen turnover as high as 100% per year, which tells me immediately that management has decided that it is okay to burn people out.  But it also tells me that this is a company that I probably don’t want to do business with, because if I need to call them, I have a high probability of being frustrated.

Again, this is a diagnostic number that makes me look carefully at the business process behind the service that the center is supposed to provide.  There are other reasons that could drive high turnover, though, so I, or the center, need to proceed methodically to find out what is driving that measure.

Ultimately, though, the satisfaction of the majority of the customer/callers is desirable, which is why many centers now ask if a caller would be willing to answer some questions about the service that they receive.  There are services that a company can hire to design and deliver a customer satisfaction survey as a third party, and my experience with those is that they will deliver a higher level of comprehensiveness and believability than a survey with a few questions at the end of a call.  I have responded to some of the on-call surveys, only to find them disappointing.  They do not ask the questions that I would like to answer: they have usually been designed within the center to take the place of or supplement their recording system, and only ask questions about the performance of the agent during the interaction that just occurred.  This does not address whether the reason that I called was satisfied, whether I found the process of getting through their automated response system to be difficult or tedious, and whether I would ever be a repeat customer.

I bought a laptop computer several years ago, and used it successfully as my extra brain for nearly 5 years without problems.  True, the CD drive had to be replaced at one point, but when I called, the computer manufacturer’s technical help was good and even though the warrantee was already passed and I had to pay for the new drive, the replacement went in easily and worked fine.

About a year ago, the hard drive became erratic and then crashed, so I called to get some help.  I wanted to make sure that I wouldn’t lose any of the recent data that had been put on it since the last full backup.

Imagine my surprise when I was told by the technician that there was nothing that he could do to help me since the model that I had had passed its end of life.  There was not even an offer to assist doing an upgrade to a new computer purchased from them, or a discount on a replacement.  The technician was quite sympathetic, but stressed that there was no procedure in place to give me help.  My reaction was quite negative, needless to say, but this was a company problem, not a technician problem.  If I had been surveyed about my satisfaction with his tone of voice, his empathy, etc.  I would have given him good marks.  But if I had been surveyed about my satisfaction with policies or the company overall, well, frankly I will never buy another of their products.  And I did not replaced the laptop with one of theirs.

Although the general rule of thumb in marketing is that a satisfied customer will tell 10 people how good the company is and an unsatisfied one will tell 25 their tale of woe, I will refrain from publicizing this company.  I have already told my 25 in person.

In the next post, we shall look more at customer satisfaction surveys, and the tools that are used to understand their results: statistics.

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

One Response to Measuring Performance in Call Centers

  1. Interesting post! I looked at something very similar in a technology website.
    Truly worth checking out

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s