By Claes Fornell, ACSI Founder and Chair

INTRODUCTION

The steep decline in U.S. overall customer satisfaction is not just caused by COVID, shortages, and inflation. While these factors have exacerbated the problem, the weakness and subsequent deterioration in customer satisfaction began about a decade ago. It is not because of a lack of effort by business either. Companies collect much more data on their customers today than they did 10 years ago. Paradoxically, the more data companies collect about their customers, the less they seem to know about how to satisfy them. That’s because data and information are not the same thing. Customer survey data in particular contain a great deal of noise. For such data to become useful beyond descriptive statistics, filtration and calibration are paramount—something that is lacking in just about any analytics tool used by business today.

In my comments below, I list the challenges and briefly address how raw customer survey data can be transformed into relevant, precise, and actionable information by making the technology of the American Customer Satisfaction Index (ACSI®) available on a broad scale and why scientific analytics tools are preferable to what is used today. Let’s start with relevance. Without it, everything else is for naught.

RELEVANCE IS THE NAME OF THE GAME

A critical gauge for assessing the value of data and analytics tools is relevance. A multitude of independent peer-reviewed research has verified the economic and financial relevance of ACSI both as a macroeconomic indicator and as a measure of company performance. At  the national level, ACSI is predictive of changes in gross domestic product (GDP), consumer spending, and quality of economic output. Among company-level metrics, ACSI has been shown to have positive effects on profitability, cash flows, stock returns, earnings surprises,  revenue, gross margins, market share, return on investment (ROI), customer retention,  stability of cash flows, credit ratings, and operating margins. Companies with strong ACSI scores have lower cost of capital, selling cost, volatility of cash flows, and lower systematic risk.

SCIENTIFIC TOOLS ARE SUPERIOR

The American Customer Satisfaction Index was developed by faculty at the University of Michigan with expertise in statistics, mathematics, econometrics, psychometrics, survey methodology, and business management. The Index was designed to affect financial outcomes via tools that can handle the many challenges posed by customer experience (CX) data, including non-normal frequency distributions, extreme multicollinearity, and high levels of noise.

Like measurement instruments in the physical sciences, ACSI’s measurement instruments are calibrated to maximize the effect on profitability via customer retention. Remarkable as it may seem, the laws of diminishing returns and high risk/high return are reversed in this context. That is, returns on customer retention are exponentially increasing (until exceedingly high) and characterized by low rather than high risk, but the opposite is also true. With declining customer satisfaction, customer churn increases exponentially as does revenue decline. However, as long as demand exceeds supply, which has been the case lately in the U.S. economy, profit margins may well remain high, but that would be a temporary phenomenon.

PRACTICALITY AND EASE-OF-USE

The ACSI model involves an intricate causal system of equations with more unknowns than equations where solutions rely on a type of machine learning (ML). However, the results are not difficult to understand or complicated to use. They begin with a description of the competitive landscape, including information about where the company in question is strong and where it is weak. The system pinpoints what to improve for optimal effect (measured as elasticity) on customer satisfaction and retention, and then predicts the outcomes—financial and otherwise—from these improvements.

SAMPLE SIZE AND SAMPLING ERROR

The issue of sample size is often misunderstood. If all assumptions of statistical significance testing are met, a larger sample would generate better estimates. But when they are not, large samples may lead to more inaccurate results. ACSI has a track record of strong predictive power, especially regarding business and financial results. Sampling error—which refers to the disparity between population and sample—is more difficult to control today than in the past. Respondents used to be sampled from phone books or address listings, so it was possible to know the probability with which they were chosen.

This is not the case today. Nevertheless, ACSI comes exceedingly close to probability sampling, not from increasing sample size, but by using stringent rules for respondent selection via its large panel exchanges.

DATA VERSUS INFORMATION

Data is not the same as information. It is not true that “numbers don’t lie.” They actually “lie” quite often. Raw data from questionnaires must be filtered from what is referred to as “measurement error.” The most frequent such errors come from mistakes by respondents, misinterpretations, and fatigue, but there are many other sources as well. The longer the questionnaire, the greater the error. While measurement errors combined with predictive irrelevance make up about 40% to 50% of the data variance in most consumer surveys, the ACSI system identifies and eliminates all data variance (information) that is not relevant for prediction.

SKEWED FREQUENCY DISTRIBUTIONS

Traditional inferential statistical methods are ill-suited for customer experience and satisfaction data because the frequency distributions of the data are almost always heavily negatively skewed. This is a characteristic of competitive markets: Companies tend to have more satisfied than dissatisfied customers. Otherwise, it would be an indication of monopoly power, where dissatisfied customers don’t have options to purchase elsewhere. But skewed and thus non-normal distributions are big problems for most statistical methods. Even regression analysis—which is often used in CX analytics to determine the impact of price, quality aspects, and service personnel—is dubious because the regression residuals are often non-normal in customer satisfaction data.

MULTICOLLINEARITY

Most variables in customer satisfaction/experience data are very highly correlated. This means that it is difficult to isolate the true effect on customer satisfaction or retention from an improvement in a specific aspect of the customer experience, relative to some other improvement action. The ACSI model addresses this problem via multiple indicators and latent variables. As a result, the high collinearity between variables occurs primarily in the measurement portion of the system of equations and is vastly reduced in the rest of the system.

SIGNIFICANCE TESTS

Statistical significance tests gauge the difference between sample and population. Such tests make assumptions about sampling that are difficult to meet. Significance tests are also widely misunderstood in that results are typically dismissed for failing to meet the standard 95% confidence level. For business decisions, it would usually be better to pick a confidence level closer to 51% (more probable than not) rather than 95%. At any rate, it is ill-advised to fixate on some customary level of significance without considering alternatives and the cost of being wrong versus the benefit of being correct. In addition to probability sampling, statistical significance testing requires a known frequency distribution and no measurement error. While consumer surveys almost always fail to meet these requirements, ACSI comes very close.

ACSI concurs with the guidance in The American Statistician article “Abandon Statistical Significance,” which suggests that statistical significance should depend on the context within which decisions about alternative courses of action should be taken. This is especially so with respect to business decisions.

LINEARITY

Many relationships among variables in a wide variety of contexts, including customer data, are nonlinear. That may be a problem because it is much easier to deal with linear relationships than with nonlinear ones. For major changes or changes over a long period of time, almost nothing is linear. For incremental changes in the short run, however, almost everything is linear. Therefore, if customer experience/satisfaction is measured frequently, nonlinear relationships are not a concern because most changes will be small and approximately linear.

CAUSE AND EFFECT

Cause-and-effect inference is critical for resource allocation, which, in turn, may well be the most important part of business management. Effective allocation of resources depends on what is known about causes and effects. In statistical modeling, this is a challenging task. ACSI relies on a causal model developed at the University of Michigan that meets these challenges as evidenced by the many financial effects that have been

found to be related to its outcomes. This, too, is different from the analytics tools used in CX management today, which are typically based on patterns, correlations, and artificial intelligence (AI) and have little to do with cause and effect. AI, especially, is often assumed to provide causal interpretations, but as of today, it has a long way to go before it can render any type of causal inference.

PRECISION

Good measurement requires precision. While this may seem obvious, most currently used CX measurement tools don’t provide it. Without precision, there is no relevance and it wouldn’t be possible to establish the effects found for ACSI regarding profits, cash flows, stock returns, earnings surprises, revenue, gross margins, return on investment, operating margins, and so forth, unless the ACSI measures were precise.