Over the years, survey researchers have implemented many technological means to enhance interviewer productivity. These methods have varied widely in both cost and effectiveness: from simple add-on automated dialers to complex telemarketing-derived predictive dialing systems. All promised productivity gains, but none were designed for the unique survey research environment. What makes survey research different than telemarketing? How do these differences impact transplanting the hardware and expectations from one industry to the other?
On the surface, telemarketing and research data collection have much in common. Both utilize outbound telephone facilities and employ numerous “agents” or “interviewers” whose job it is to contact the public and either make sales or conduct surveys. This functional similarity has even caused many research firms to open telemarketing operations. However, as many have come to realize, there are fundamental differences that make such a transition difficult and the technology transfer just as chancy.
What could be so different? How about the basic economic models? Or, put another way, the incentives that govern agent/interviewer motivation and productivity. In telemarketing operations, both management and telephone agent employees have a common and mutually reinforcing incentive system -- compensation is based on the quantity of “sales” (i.e., subscriptions sold, contributions, etc.). More importantly, this means that both manager and employee benefit by increasing the number of contacts that can be made in any given time period.
Unfortunately, this incentive system is not typical of survey data collection operations. Interviewers are usually paid a flat rate per hour regardless of their productivity. Of course there are often incentives paid for extra hours, longevity, and even for the best production records and lowest refusal rates. But, in the short term, interviewers are paid a flat rate and, unless production is truly abysmal, there is no monetary incentive to “over produce” or “beat the rate.” Need one ask why the phenomenal productivity gains promised by those selling predictive dialers seem elusive and, all to often, unattainable?
Bear in mind that predictive or other dialing-assist mechanisms are simply tools which help to decrease an interviewer’s unproductive time. But you need to be careful about that “unproductive time.” It’s a lot like government estimates of unemployment rates. Few realize that to be classified as “unemployed” you must satisfy two conditions: you must not be working and, just as importantly, you must be actively seeking work.
This distinction has a direct analogy in survey data collection. The best productivity tools will only benefit an interviewer who is actively seeking a respondent. In other words, when an interviewer isn’t talking, it does not necessarily follow that a predictive dialing system will automatically force that interviewer to start talking again. Interviewers must signal the dialer, in some way, that they are “ready”; if they don’t, the dialer won’t work the way you expect. In fact, with many predictive systems, interviewers ability to “hide” can increase abandonment rates because the dialer is expecting them to be ready to take a new call after some predetermined period of time near the end of the current contact or interview.
Bottom line, there is no substitute for active, consistent and on-going supervision. Without this your investment may never pay for itself, nor will you increase competitiveness through increased “talk time” or lengthening the “interviewing hour.” Research interviewers do not have the same motivating incentives as telemarketers.
Besides the differences in basic incentive structures, there are other reasons why telemarketing technology often fails to meet researchers’ expectations. This is usually due to a failure to account for the underlying philosophical differences and operational constraints imposed on predictive dialing systems by researchers. These factors impact directly on expected productivity gains.
Types of Lists/Sample. Telemarketers typically use customer/prospect lists or listed household samples. This contrasts with random-digit samples (RDD) and their higher proportions of non-working/disconnected numbers and no answer dispositions. Predictive dialing systems will always produce longer expected “wait-times” when household hit rates decrease. This loss of efficiency is most evident on first attempts, when most non-working/FAX/modem lines are identified, but larger proportions of no answers continue to drag down efficiency throughout the data collection process.
Script/Questionnaire Length. Introductions are more descriptive, screeners are typically longer and greater efforts are made to secure respondent cooperation in survey work. All these increase the average length of even unsuccessful contacts. Moreover, many surveys require a predestinated or randomly selected respondent. The process of identification, getting that person to the phone, or scheduling a callback again increases the average length of contacts.
Completion & Response Rates. The difference between simply calling a virtually unlimited list and attempting to complete a screen/interview with a high percentage of a limited sample impacts significantly on relative dialer productivity gains. Researchers place multiple calls on no answers and callbacks. The number of rings allowed for no answers is typically longer -- seven rings translates to nearly one minute. Minimizing refusals through more thorough explanation as to purpose, sponsor, etc.; refusal conversion and calling back initial refusals all lower potential productivity gains.
Survey Operations. One fact often overlooked is that a predictive dialing system loses effectiveness as the number of interviewers assigned to a study decreases. In general, predictive algorithms are only effective when the system is attempting to find a contact for 15 or more interviewers at one time, on one study (unless, of course, you are willing to accept abandonment rates of 50% or more). Most survey data collection operations are running multiple studies, with comparatively long interviews. That contrasts with successful telemarketing calls which typically run five minutes or less. And, as the end of a study nears, or during “wind-down”, predictive algorithms are worthless due to interviewer and sample limitations.
Current Predictive Dialing Assumptions. Telemarketing predictive dialing models are designed to maximize the likelihood that as soon as an agent has hung up from one call, another is already waiting to be answered. In other words, the dialer is meant to minimize down-time, translated as the amount of time an agent needs to wait for a new contact after his or her last call was terminated. In queuing theory parlance, the idea is to maximize use of the agents’ or servers’ time, by always insuring a customer is in the queue waiting to be served.
This is accomplished by “predicting” the server/agent availability and recruiting/calling ahead to maximize the probability that a live respondent will be immediately available for each agent. When a contact is made and an agent is not available, the predictable outcome is that the person who answered the call simply hangs up -- in other words, the call is “abandoned.” In fact, this is the most commonly used control variable in current predictive algorithms -- the abandonment rate. The abandonment rate can typically be set anywhere from 0 to 50 percent -- in other words, the dialer estimates the number of agents who will be ready for a new contact within the next time period; it then commences dialing additional numbers so that the expected live connects exceeds the number of agents expected to be available -- the abandonment rate
determines this overage. However, it should be stressed that if the abandonment rate is lowered, there is always a corresponding increase in the expected, or average wait-time.
The success of a predictive algorithm is essentially a function of how reliably the distribution of future agent availability can be estimated. These estimates are made in real-time and will vary in complexity based on the number of variables that are taken into account. However, all essentially deal with expected length of contact, how far into the script the agent is, and how long before they will terminate. Another critical factor here is that telemarketing calls are typically short, with many refusals and hang-ups. But keep in mind that abandoned calls have minimal, if any, cost from a telemarketers’ standpoint. The critical cost to be avoided is idle agent time -- with an infinite number of households to call, the question is how many sales were made, not how many calls were abandoned.
The PRO- T-S® Dialer Concept. The survey research data collection model is quite different than the telemarketing mode of operation. A primary goal is high response rates -- the objective is to complete an interview with as high a percentage of the sample as possible. Typically, there are a finite number of sample pieces/telephone numbers; multiple attempts to each sample telephone number, across different days, are made to increase the likelihood of making a contact. In many cases, there are even multiple follow-up attempts to “convert” respondents who initially refused to be interviewed. As one can imagine, the idea of abandoning live respondents and potentially reducing their future willingness to participate is anathema to most professional researchers.
Another major difference between survey research and telemarketing is in the nature of the lists of telephone numbers utilized. Telemarketers typically employ lists of known household numbers, with many of those households selected based on specific characteristics. In contrast, a large proportion of survey research is conducted using random digit dialing (RDD) a statistical method of selecting telephone numbers at random, so that all numbers have an equal probability of being selected -- the objective being to sample all households equally, regardless of whether or not they are listed in the telephone directory or included in the customer/prospect call list.
The difference is fundamental -- survey researchers are employing a probability sample in order to insure responses are representative of the population at large. Not only does this speak to the inherent problems with call abandonment, but also increases the variance of expected call lengths -- these samples include a large number of non-working numbers, non-household numbers, etc. Coupled with short qualification interviews of 15 seconds to 2 minutes and long follow-up interviews of 5 to 30 minutes or more, the ability to predict when interviewers will be available to take another call is, at best, little more than a guess. It is also important to remember that interviewers are often required to complete some paperwork following completion of each interview - editing of responses and open-ended questions.
In short, the goals, objectives and list sources of survey researchers are quite different than those of telemarketers. The PRO- T-S® System offers two unique and heretofore unknown methodologies for increasing research dialing efficiencies. Both of these dialing modes are aimed at controlling the interviewers’ timing. The first, called Pacing, allows the site manager to specify a “wait-time” after termination of a call before dialing recommences for the next contact. In this scenario, there is basically a one-to-one correspondence between an interviewer and the outgoing line on which automated dialing takes place. Wait-times are variable by type of contact completed. So, for example, if a business were contacted, dialing may commence immediately; but if an interview was completed, there may be a set delay of one to two minutes allowing the interviewer to rest and complete editing of verbatim. Obviously, different surveys have different requirements, schedules and complexity, so the wait-times can be varied to meet individual project needs. The primary objective of the Pacing mode is to provide a greater degree of control over the interviewing process. Although each interviewer will still encounter an expected wait-time once dialing commences, the Pacing mode eliminates uncontrolled/unmonitored interviewer downtime. It imposes a degree of control previously unavailable, yielding gains in interviewer productivity with no abandonment rate downside.
For those surveys, or parts of surveys where predictive dialing has a potential for large productivity gains, our Research Predictive mode represents a totally new approach to the problem. Rather than attempting to predict future availability of interviewers, the PRO- T-S® System predicts the dialing outcome distribution. Conceptually, if there are interviewers waiting for a contact, the system computes a probabilistic distribution of calls in progress and the number that can be expected to result in a “live” contact. The control variable in the process is a simple stopping rule, which sets a limit as to the likelihood that the numbers being dialed will result in more live contacts than there are interviewers available. For example, one might set this limit at .001, .0001, or .0000001 (i.e., 1 in 1000, 1 in 10,000, or even 1 in a million odds of making more live contacts than interviewers available).
This algorithm operates in real-time, taking into account the number of dialing in progress, the number of interviewers “waiting” and the probability of each sample telephone number resulting in a live contact. The last item is critical, and is based on what we know about every telephone number in the US -- whether it is a known household number; whether it is a known business, fax, or modem number; and if not any of these, the conditional probability that it is a residence given the density of assigned numbers in that NAP/NXX or specific portions of the exchange. In addition, the system takes into account the call history for that specific telephone number (e.g., it previously resulted in a contact, or the previous five attempts all resulted in “no answers”).
Telemarketing predictive dialing systems and call handling protocols are not designed for the complexities, unique operational needs and differing objectives of survey research data collection. Too often and too late, researchers have realized that fitting their projects to a telemarketing operational model requires unacceptable compromises and becomes very expensive and inefficient. PRO- T-S® is the first computer-assisted dialing and interviewer management system created specifically for survey research environments.