External Performance Metrics

By Penny Reynolds
May 2004

The long-term success of any organization, particularly a service organization such as a call center, depends upon continuous improvement.  The key to continuous improvement involves listening to customers to learn how effectively the organization is meeting their needs.  In defining a relationship with an organization, customers consider a number of factors where the service organization or call center "touches" them.  Understanding these customer "touch points" and the internal processes that contribute to the success or failure of each contact can only be accomplished through a systematic customer surveying process.

It is important for the call center to perform its own surveys in order to understand the perceptions of customers specifically related to call center transactions.  Many organizations perform regular customer satisfaction surveys, but these surveys usually focus on products, pricing, and a variety of other concerns.  The call center experience is sometimes buried in the overall questions and scope of the survey.  To truly evaluate how effectively the call center is serving customers and representing the organization, it is crucial to do customer surveys solely focused on the call center experience.

Types of Surveys: There are several different types of surveys that an organization might do.  These three types of surveys are:

 Specific purpose surveys – These surveys may be conducted to ask a specific question about call center operations.  For example, the organization may wish to query customers about the adequacy of hours of operation or to test out a new pricing structure.  These surveys are limited to a very few questions around a single topic of interest.

Periodic surveys – These surveys are used to gauge perceptions around issues in the overall relationship between the customer and the organization or department.  These surveys are related to no specific transaction.  Instead, they ascertain how well the organization is doing in the customer's eyes with respect to ease of doing business, value delivered, areas needing change, and importance attached to certain service attributes.

Transaction surveys - These surveys are performed in conjunction with some specific event or transaction.  They are used to gauge the customer's perception of that particular transaction experience.  These surveys are event-driven and typically happen very soon after the event being evaluated.

Customer Survey Steps: Regardless of the type of survey to be performed, there are five basic steps to be followed in performing customer surveys.  These steps are:

• Project planning

• Instrument development

• Survey administration

• Data analysis

• Reporting and action

Project Planning: Every call center survey should start with a statement of purpose.  This statement should outline the motivation for the survey, the target audience, the needed results, and what actions will be taken with the results.  The statement of purpose should be simple, but detailed enough to serve as a "beacon" to keep the project focused and moving forward.

Another step in planning a customer survey will be to identify the required resources.  Outline any facilities that will be needed along with any special equipment or tools to perform the study.  Personnel will also be defined at this point by assembling a cross-functional project team.  Members of this team should include a project manager who will oversee the project and perhaps do most of the work, along with those who will be involved in questionnaire design, implementation, and analysis.  It is desirable to enlist a project sponsor or champion who is a member of senior management to ensure resources will be allocated to the project as needed.  This project team may also include select customers as well as supporting departments and outside partners or vendors.

Every survey project should begin by planning a budget for the project.  The budget should include such items as staff salaries, computer hardware, and software, postal or telephone fees for survey implementation, training, incentives for participants, and perhaps survey software tools.

A project schedule should be defined by outlining how long each phase of the project will take and what milestones are dependent upon other tasks being completed first.  The phases to be outlined in the schedule include project planning, questionnaire design and development, questionnaire review, survey administration, data analysis, and reporting.

The final stage in the planning process is content development.  This step is sometimes referred to as "the survey before the survey."  It is used to ensure all critical issues and concerns are indeed being covered in the actual questionnaire.  This initial survey can take the form of one-on-one interviews with individuals, or can be accomplished in a focus group setting.  It is important to include individuals or companies in this pre-survey process who will willingly bring out issues and concerns.  These initial surveys should be done until no new issues surface, so that all potential customer concerns are addressed in the survey instrument.

Instrument Development: In developing the actual survey instrument or questionnaire, it is important to start with an idea of what information is needed in the final report and work backwards.  Designing the questionnaire will be an iterative process and ample time should be allowed to get it right.  Three categories of questions are typically included in a survey, including ones that address service delivery attributes, demographic variables, and customer attitudes toward the call center and the organization as a whole. 

When writing the actual questions, it is important to consider phrasing to avoid loaded or leading questions, jargon, and ambiguous wording.  Questionnaire design is critical so that customers will not just take the time to complete the survey, but will be able to complete it correctly.

Each survey instrument should include an introduction that explains who should complete the survey, how much time it should take, how to respond to questions, what to do when finished, and what the deadline is for participation.  Critical terminology should be defined at the beginning and sequencing instructions should be given, whether given verbally in a telephone interview, or displayed legibly in a written survey.

Another crucial design element is the type of data to be used.  Some questions may be unstructured and require a textual response.  Other questions may be structured in a multiple-choice format, or in a rating scale of some sort.

Survey Administration: There are various ways to administer a customer satisfaction survey.  Traditional paper-based mail surveys have declined in popularity in recent years due to their low response rates and availability of faster means of communication.  A high percentage of call centers utilize telephone surveys conducted either by in-house staff or by a third-party specialist.  Electronic surveys via email and the Web are also growing in number.  The variety of administration methods that can be used, along with their relative advantages and disadvantages, is provided in matrix form below:

Another key decision in the survey administration process will be the sampling procedure.  The organization should consider its customer population and determine whether a census is needed or whether a representative sample will be used.  If a sample is to be used, correct sampling procedures should be employed in order to arrive at a precise and accurate conclusion.  The sample size will depend upon the number of responses needed as well as the expected response rate from the participants.  The number of needed responses will depend upon the desired accuracy and precision of the survey, with the amount of accuracy needed being dependent upon the amount of variation in the sample.

Once the sample size and sampling procedure has been determined, a pre-survey notification should go out, either via traditional letter or by email.  This notification should explain why the survey is being done, how the survey will benefit the participant, how results will be communicated, and what, if any, incentives are provided for participation.  Upon conclusion of this notification, the actual survey is distributed.

Data analysis: Once the survey has been distributed and responses gathered, the next step in the survey process is to analyze the results.  Analysis will vary depending upon whether the questions are purely statistical in nature, or where unstructured textual responses will be received.  The data will typically be analyzed to determine averages or other means of central tendency.  The data will also be analyzed to determine how the responses are distributed.  It is important to not only look at the average responses or scores, but also to look at the dispersion of results from low to high.  Statistical analysis (beyond the scope of this article) should be employed to arrive at meaningful conclusions from all the data gathered in the survey process.

Reporting and Action: The purpose of doing a customer survey is to determine how effectively customer needs are being met.  Once the data has been analyzed, the next step is to report the survey results to relevant parties, and also to act upon results of the study to improve the service process and strengthen relationships with customers.  Actions might include follow-up research or root cause analysis, immediate service actions, or continuous process improvement initiatives.

Penny Reynolds is a Founding Partner of The Call Center School, a Nashville, Tennessee based consulting and education company.  The company provides educational offerings for call center professionals, including traditional classroom courses, Web-based seminars, and self-paced e-learning programs at the manager, supervisor, and front-line staff level.  For more information, see www.thecallcenterschool.com or call 615-812-8400.

Return to List of Articles || Read more articles at MyArticleArchive.com