The Idea in Brief
Managers have tough jobs: Under intense pressure to make decisions with incomplete information, even the best among us make mistakes. The good news? Evidence abounds to help us make the right choices. The bad? Many of us ignore it—relying instead on outdated information or our own experiences to arrive at decisions. Some of us fall victim to hype about “miracle” management cures, or we adopt other companies’ “best practices” without asking whether they’ll work just as well for our organizations.
Result? Poor-quality decisions that waste time and money (at best) and risk your company’s future (at worst).
To avoid this scenario, start an evidenced-based management movement in your company: Every time someone proposes a change, ask for evidence of its efficacy. Clarify the logic behind that evidence—looking for faulty reasoning. Encourage managers to experiment with new ideas—rewarding those who learn from these efforts, even if an experiment itself fails. And insist that managers stay current in their field—and provide continuing professional education opportunities to help them do.
Your reward? You and your colleagues face the hard truths about what works and what doesn’t. You expose the dangerous half-truths that mar much conventional business wisdom. And you make smart decisions on the most pressing issues facing your company.
The Idea in Practice
To start an evidenced-based management movement in your firm:
Demand evidence. Whenever someone makes a seemingly compelling claim, ask for supporting data. Example:
At DaVita, an operator of kidney dialysis centers, facility administrators use disciplined measures to evaluate patient care quality and operational efficiency—and to make confident claims about DaVita’s performance. Reports and meetings begin with data on patient health as well as operational efficiency—as measured by metrics such as treatments per day and employee retention. Formerly teetering on the edge of bankruptcy, DaVita now lays claim to the best patient care quality in the industry.
Examine logic. Parse the logic behind evidence presented to you, looking for faulty cause-and-effect reasoning. Example:
A manager who has benchmarked top-performing companies’ best practices recommends adopting a particular practice. You ask him: 1) Does the benchmarked company’s success clearly stem from the practice you want us to emulate? 2) Are our strategy, business model, and workforce similar enough to the benchmarked firm to enable us to learn from that company? 3) Precisely how did this practice make a difference? 4) What are the downsides to implementing this practice, and how might we mitigate them?
Encourage experimentation. Invite managers to conduct small experiments to test the viability of proposed strategies. Example:
Gaming giant Harrah’s offered one control group of customers the company’s typical promotional package worth $125 (a free room, two steak dinners, and $30 worth of free gambling chips). It offered customers in an experimental group just $60 worth of free chips. The $60 offer generated more gambling revenue than the $125 offer did—demonstrating that Harrah’s didn’t have to spend nearly as much as it believed was needed to boost revenues.
Reinforce continuous learning. When managers constantly expand their knowledge, they acquire increasingly more reliable evidence with which to make decisions. Encourage use of inquiry and observation to gather evidence about causes and potential cures for business problems. And provide resources for the continuing professional education of managers. Example:
At one computer manufacturer beleaguered by poor sales, top managers initially blamed the firm’s corporate sales staff—initially dismissing their claims that weak revenues were a result of poor product quality. Then senior managers were encouraged to further investigate the problem. When managers posed as customers at retailers who carried their computers, store salespeople dissuaded them from purchasing their company’s product—citing the computer’s excessive price, weak features, and clunky appearance. By practicing inquiry and observation, company managers learned that they needed to reexamine product quality.
Leer en español
Ler em português
A bold new way of thinking has taken the medical establishment by storm in the past decade: the idea that decisions in medical care should be based on the latest and best knowledge of what actually works. Dr. David Sackett, the individual most associated with evidence-based medicine, defines it as “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients.” Sackett, his colleagues at McMaster University in Ontario, Canada, and the growing number of physicians joining the movement are committed to identifying, disseminating, and, most importantly, applying research that is soundly conducted and clinically relevant.
If all this sounds laughable to you—after all, what else besides evidence would guide medical decisions?—then you are woefully naive about how doctors have traditionally plied their trade. Yes, the research is out there—thousands of studies are conducted on medical practices and products every year. Unfortunately, physicians don’t use much of it. Recent studies show that only about 15% of their decisions are evidence based. For the most part, here’s what doctors rely on instead: obsolete knowledge gained in school, long-standing but never proven traditions, patterns gleaned from experience, the methods they believe in and are most skilled in applying, and information from hordes of vendors with products and services to sell.
The same behavior holds true for managers looking to cure their organizational ills. Indeed, we would argue, managers are actually much more ignorant than doctors about which prescriptions are reliable—and they’re less eager to find out. If doctors practiced medicine like many companies practice management, there would be more unnecessarily sick or dead patients and many more doctors in jail or suffering other penalties for malpractice.
It’s time to start an evidence-based movement in the ranks of managers. Admittedly, in some ways, the challenge is greater here than in medicine. (See the sidebar “What Makes It Hard to Be Evidence Based?”) The evidence is weaker; almost anyone can (and often does) claim to be a management expert; and a bewildering array of sources—Shakespeare, Billy Graham, Jack Welch, Tony Soprano, fighter pilots, Santa Claus, Attila the Hun—are used to generate management advice. Managers seeking the best evidence also face a more vexing problem than physicians do: Because companies vary so wildly in size, form, and age, compared with human beings, it is far more risky in business to presume that a proven “cure” developed in one place will be effective elsewhere.
Still, it makes sense that when managers act on better logic and evidence, their companies will trump the competition. That is why we’ve spent our entire research careers, especially the last five years, working to develop and surface the best evidence on how companies ought to be managed and teaching managers the right mind-set and methods for practicing evidence-based management. As with medicine, management is and will likely always be a craft that can be learned only through practice and experience. Yet we believe that managers (like doctors) can practice their craft more effectively if they are routinely guided by the best logic and evidence—and if they relentlessly seek new knowledge and insight, from both inside and outside their companies, to keep updating their assumptions, knowledge, and skills. We aren’t there yet, but we are getting closer. The managers and companies that come closest already enjoy a pronounced competitive advantage.
What Passes for Wisdom
If a doctor or a manager makes a decision that is not based on the current best evidence of what may work, then what is to blame? It may be tempting to think the worst. Stupidity. Laziness. Downright deceit. But the real answer is more benign. Seasoned practitioners sometimes neglect to seek out new evidence because they trust their own clinical experience more than they trust research. Most of them would admit problems with the small sample size that characterizes personal observation, but nonetheless, information acquired firsthand often feels richer and closer to real knowledge than do words and data in a journal article. Lots of managers, likewise, get their companies into trouble by importing, without sufficient thought, performance management and measurement practices from their past experience. We saw this at a small software company, where the chair of the compensation committee, a successful and smart executive, recommended the compensation policies he had employed at his last firm. The fact that the two companies were dramatically different in size, sold different kinds of software, used different distribution methods, and targeted different markets and customers didn’t seem to faze him or many of his fellow committee members.
Another alternative to using evidence is making decisions that capitalize on the practitioner’s own strengths. This is particularly a problem with specialists, who default to the treatments with which they have the most experience and skill. Surgeons are notorious for it. (One doctor and author, Melvin Konner, cites a common joke amongst his peers: “If you want to have an operation, ask a surgeon if you need one.”) Similarly, if your business needs to drum up leads, your event planner is likely to recommend an event, and your direct marketers will probably suggest a mailing. The old saying “To a hammer, everything looks like a nail” often explains what gets done.
Hype and marketing, of course, also play a role in what information reaches the busy practitioner. Doctors face an endless supply of vendors, who muddy the waters by exaggerating the benefits and downplaying the risks of using their drugs and other products. Meanwhile, some truly efficacious solutions have no particularly interested advocates behind them. For years, general physicians have referred patients with plantar warts on their feet to specialists for expensive and painful surgical procedures. Only recently has word got out that duct tape does the trick just as well.
Numerous other decisions are driven by dogma and belief. When people are overly influenced by ideology, they often fail to question whether a practice will work—it fits so well with what they “know” about what makes people and organizations tick. In business, the use and defense of stock options as a compensation strategy seems to be just such a case of cherished belief trumping evidence, to the detriment of organizations. Many executives maintain that options produce an ownership culture that encourages 80-hour workweeks, frugality with the company’s money, and a host of personal sacrifices in the interest of value creation. T.J. Rodgers, chief executive of Cypress Semiconductor, typifies this mind-set. He told the San Francisco Chronicle that without options, “I would no longer have employee shareholders, I would just have employees.” There is, in fact, little evidence that equity incentives of any kind, including stock options, enhance organizational performance. A recent review of more than 220 studies compiled by Indiana University’s Dan R. Dalton and colleagues concluded that equity ownership had no consistent effects on financial performance.
The use and defense of stock options as a compensation strategy seems to be a case of cherished belief trumping evidence, to the detriment of organizations.
Ideology is also to blame for the persistence of the first-mover-advantage myth. Research by Wharton’s Lisa Bolton demonstrates that most people—whether experienced in business or naive about it—believe that the first company to enter an industry or market will have a big advantage over competitors. Yet empirical evidence is actually quite mixed as to whether such an advantage exists, and many “success stories” purported to support the first-mover advantage turn out to be false. (Amazon.com, for instance, was not the first company to start selling books online.) In Western culture, people believe that the early bird gets the worm, yet this is a half-truth. As futurist Paul Saffo puts it, the whole truth is that the second (or third or fourth) mouse often gets the cheese. Unfortunately, beliefs in the power of being first and fastest in everything we do are so ingrained that giving people contradictory evidence does not cause them to abandon their faith in the first-mover advantage. Beliefs rooted in ideology or in cultural values are quite “sticky,” resist disconfirmation, and persist in affecting judgments and choice, regardless of whether they are true.
Finally, there is the problem of uncritical emulation and its business equivalent: casual benchmarking. Both doctors and managers look to perceived high performers in their field and try to mimic those top dogs’ moves. We aren’t damning benchmarking in general—it can be a powerful and cost-efficient tool. (See the sidebar “Can Benchmarking Produce Evidence?”) Yet it is important to remember that if you only copy what other people or companies do, the best you can be is a perfect imitation. So the most you can hope to have are practices as good as, but no better than, those of top performers—and by the time you mimic them, they’ve moved on. This isn’t necessarily a bad thing, as you can save time and money by learning from the experience of others inside and outside your industry. And if you consistently implement best practices better than your rivals, you will beat the competition.
Benchmarking is most hazardous to organizational health, however, when used in its “casual” form, in which the logic behind what works for top performers, why it works, and what will work elsewhere is barely unraveled. Consider a quick example. When United Airlines decided in 1994 to try to compete with Southwest in the California market, it tried to imitate Southwest. United created a new service, Shuttle by United, with separate crews and planes (all of them Boeing 737s). The gate staff and flight attendants wore casual clothes. Passengers weren’t served food. Seeking to emulate Southwest’s legendary quick turnarounds and enhanced productivity, Shuttle by United increased the frequency of its flights and reduced the scheduled time planes would be on the ground. None of this, however, reproduced the essence of Southwest’s advantage—the company’s culture and management philosophy, and the priority placed on employees. Southwest wound up with an even higher market share in California after United had launched its new service. The Shuttle is now shuttered.
We’ve just suggested no less than six substitutes that managers, like doctors, often use for the best evidence—obsolete knowledge, personal experience, specialist skills, hype, dogma, and mindless mimicry of top performers—so perhaps it’s apparent why evidence-based decision making is so rare. At the same time, it should be clear that relying on any of these six is not the best way to think about or decide among alternative practices. We’ll soon describe how evidence-based management takes shape in the companies we’ve seen practice it. First, though, it is useful to get an example on the table of the type of issue that companies can address with better evidence.
An Example: Should We Adopt Forced Ranking?
The decision-making process used at Oxford’s Centre for Evidence-Based Medicine starts with a crucial first step—the situation confronting the practitioner must be framed as an answerable question. That makes it clear how to compile relevant evidence. And so we do that here, raising a question that many companies have faced in recent years: Should we adopt forced ranking of our employees? The question refers to what General Electric more formally calls a forced-curve performance-ranking system. It’s a talent management approach in which the performance levels of individuals are plotted along a bell curve. Depending on their position on the curve, employees fall into groups, with perhaps the top 20%, the so-called A players, being given outsize rewards; the middle 70% or so, the B players, being targeted for development; and the lowly bottom 10%, the C players, being counseled or thrown out of their jobs.
Without a doubt, this question arose for many companies as they engaged in benchmarking. General Electric has enjoyed great financial success and seems well stocked with star employees. GE alums have gone on to serve as CEOs at many other companies, including 3M, Boeing, Intuit, Honeywell, and the Home Depot. Systems that give the bulk of rewards to star employees have also been thoroughly hyped in business publications—for instance, in the McKinsey-authored book The War for Talent. But it’s far from clear that the practice is worth emulating. It isn’t just the infamous Enron—much praised in The War for Talent—that makes us say this. A couple of years ago, one of us gave a speech at a renowned but declining high-technology firm that used forced ranking (there, it was called a “stacking system”). A senior executive told us about an anonymous poll conducted among the firm’s top 100 or so executives to discover which company practices made it difficult to turn knowledge into action. The stacking system was voted the worst culprit.
Would evidence-based management have kept that company from adopting this deeply unpopular program? We think so. First, managers would have immediately questioned whether their company was similar enough to GE in various respects that a practice cribbed from it could be expected to play out in the same way. Then, they would have been compelled to take a harder look at the data presumably supporting forced ranking—the claim that this style of talent management actually has caused adherents to be more successful. So, for example, they might have noticed a key flaw in The War for Talent’s research method: The authors report in the appendix that companies were first rated as high or average performers, based on return to shareholders during the prior three to ten years; then interviews and surveys were conducted to measure how these firms were fighting the talent wars. So, for the 77 companies (of 141 studied), management practices assessed in 1997 were treated as the “cause” of firm performance between 1987 and 1997. The study therefore violates a fundamental condition of causality: The proposed cause needs to occur before the proposed effect.
Next, management would have assembled more evidence and weighed the negative against the positive. In doing so, it would have found plenty of evidence that performance improves with team continuity and time in position—two reasons to avoid the churn of what’s been called the “rank and yank” approach. Think of the U.S. Women’s National Soccer Team, which has won numerous championships, including two of the four Women’s World Cups and two of the three Olympic women’s tournaments held to date. The team certainly has had enormously talented players, such as Mia Hamm, Brandi Chastain, Julie Foudy, Kristine Lilly, and Joy Fawcett. Yet all these players will tell you that the most important factor in their success was the communication, mutual understanding and respect, and ability to work together that developed during the 13 or so years that the stable core group played together. The power of such joint experience has been established in every setting examined, from string quartets to surgical teams, to top management teams, to airplane cockpit crews.
If managers at the technology firm had reviewed the best evidence, they would have also found that in work that requires cooperation (as nearly all the work in their company did), performance suffers when there is a big spread between the worst- and best-paid people—even though giving the lion’s share of rewards to top performers is a hallmark of forced-ranking systems. In a Haas School of Business study of 102 business units, Douglas Cowherd and David Levine found that the greater the gap between top management’s pay and that of other employees, the lower the product quality. Similar negative effects of dispersed pay have been found in longitudinal studies of top management teams, universities, and a sample of nearly 500 public companies. And in a recent Novations Group survey of more than 200 human resource professionals from companies with more than 2,500 employees, even though over half of the companies used forced ranking, the respondents reported that this approach resulted in lower productivity, inequity, skepticism, decreased employee engagement, reduced collaboration, damage to morale, and mistrust in leadership. We can find plenty of consultants and gurus who praise the power of dispersed pay, but we can’t find a careful study that supports its value in settings where cooperation, coordination, and information sharing are crucial to performance.
In a recent survey of more than 200 HR professionals, respondents reported that forced ranking had consequences such as lower productivity, inequity, damage to morale, and mistrust in leadership.
Negative effects of highly dispersed pay are even seen in professional sports. Studies of baseball teams are especially interesting because, of all major professional sports, baseball calls for the least coordination among team members. But baseball still requires some cooperation—for example, between pitchers and catchers, and among infielders. And although individuals hit the ball, teammates can help one another improve their skills and break out of slumps. Notre Dame’s Matt Bloom did a careful study of over 1,500 professional baseball players from 29 teams, spanning an eight-year period, which showed that players on teams with greater dispersion in pay had lower winning percentages, gate receipts, and media income.
Finally, an evidence-based approach would have surfaced data suggesting that average players can be extremely productive and that A players can founder, depending on the system they work in. Over 15 years of research in the auto industry provides compelling evidence for the power of systems over individual talent. Wharton’s John Paul MacDuffie has combined quantitative studies of every automobile plant in the world with in-depth case studies to understand why some plants are more effective than others. MacDuffie has found that lean or flexible production systems—with their emphasis on teams, training, and job rotation, and their de-emphasis on status differences among employees—build higher-quality cars at a lower cost.
Becoming a Company of Evidence-Based Managers
It is one thing to believe that organizations would perform better if leaders knew and applied the best evidence. It is another thing to put that belief into practice. We appreciate how hard it is for working managers and executives to do their jobs. The demands for decisions are relentless, information is incomplete, and even the very best executives make many mistakes and undergo constant criticism and second-guessing from people inside and outside their companies. In that respect, managers are like physicians who face one decision after another: They can’t possibly make the right choice every time. Hippocrates, the famous Greek who wrote the physicians’ oath, described this plight well: “Life is short, the art long, opportunity fleeting, experiment treacherous, judgment difficult.”
Teaching hospitals that embrace evidence-based medicine try to overcome impediments to using it by providing training, technologies, and work practices so staff can take the critical results of the best studies to the bedside. The equivalent should be done in management settings. But it’s also crucial to appreciate that evidence-based management, like evidence-based medicine, entails a distinct mind-set that clashes with the way many managers and companies operate. It features a willingness to put aside belief and conventional wisdom—the dangerous half-truths that many embrace—and replace these with an unrelenting commitment to gather the necessary facts to make more informed and intelligent decisions.
As a leader in your organization, you can begin to nurture an evidence-based approach immediately by doing a few simple things that reflect the proper mind-set. If you ask for evidence of efficacy every time a change is proposed, people will sit up and take notice. If you take the time to parse the logic behind that evidence, people will become more disciplined in their own thinking. If you treat the organization like an unfinished prototype and encourage trial programs, pilot studies, and experimentation—and reward learning from these activities, even when something new fails—your organization will begin to develop its own evidence base. And if you keep learning while acting on the best knowledge you have and expect your people to do the same—if you have what has been called “the attitude of wisdom”—then your company can profit from evidence-based management as you benefit from “enlightened trial and error” and the learning that occurs as a consequence.
When it comes to setting the tone for evidence-based management, we have met few chief executives on a par with Kent Thiry, the CEO of DaVita, a $2 billion operator of kidney dialysis centers headquartered in El Segundo, California. Thiry joined DaVita in October 1999, when the company was in default on its bank loans, could barely meet payroll, and was close to bankruptcy. A big part of his turnaround effort has been to educate the many facility administrators, a large proportion of them nurses, in the use of data to guide their decisions.
To ensure that the company has the information necessary to assess its operations, the senior management team and DaVita’s chief technical officer, Harlan Cleaver, have been relentless in building and installing systems that help leaders at all levels understand how well they are doing. One of Thiry’s mottoes is “No brag, just facts.” When he stands up at DaVita Academy, a meeting of about 400 frontline employees from throughout the organization, and states that the company has the best quality of treatment in the industry, that assertion is demonstrated with specific, quantitative comparisons.
A large part of the company’s culture is a commitment to the quality of patient care. To reinforce this value, managers always begin reports and meetings with data on the effectiveness of the dialysis treatments and on patient health and well-being. And each facility administrator gets an eight-page report every month that shows a number of measures of the quality of care, which are summarized in a DaVita Quality Index. This emphasis on evidence also extends to management issues—administrators get information on operations, including treatments per day, teammate (employee) retention, the retention of higher-paying private pay patients, and a number of resource utilization measures such as labor hours per treatment and controllable expenses.
The most interesting thing about these monthly reports is what isn’t yet included. DaVita COO Joe Mello explained that if a particular metric is deemed important, but the company currently lacks the ability to collect the relevant measurements, that metric is included on reports anyway, with the notation “not available.” He said that the persistent mention of important measures that are missing helps motivate the company to figure out ways of gathering that information.
Many impressive aspects of DaVita’s operations have contributed to the company’s success, as evidenced by the 50% decrease in voluntary turnover, best-in-industry quality of patient care, and exceptional financial results. But the emphasis on evidence-based decision making in a culture that reinforces speaking the truth about how things are going is certainly another crucial component.
Simply asking for backup research on proposals is insufficient to foster a true organizational commitment to evidence-based management, especially given the problems that bedevil much so-called business research. As managers or consultants make their case, pay close attention to gaps in exposition, logic, and inference. (See the sidebar “Are You Part of the Problem?”) This is particularly important because, in management research, studies that use surveys or data from company records to correlate practices with various performance outcomes are far more common than experiments. Such “nonexperimental” research is useful, but care must be taken to examine the logic of the research design and to control statistically for alternative explanations, which arise in even the best studies. Managers who consume such knowledge need to understand the limitations and think critically about the results.
When people in the organization see senior executives spending the time and mental energy to unpack the underlying assumptions that form the foundation for some proposed policy, practice, or intervention, they absorb a new cultural norm. The best leaders avoid the problem of seeming captious about the work of subordinates; they tap the collective wisdom and experience of their teams to explore whether assumptions seem sensible. They ask, “What would have to be true about people and organizations if this idea or practice were going to be effective? Does that feel true to us?”
Consultant claims may require an extra grain of salt. It is surprising how often purveyors of business knowledge are fooled or try to fool customers. We admire Bain & Company, for example, and believe it is quite capable of good research. We do wonder, however, why the company has a table on its Web site’s home page that brags, “Our clients outperform the market 4 to 1” (the claim was “3 to 1” a few years back). The smart people at Bain know this correlation doesn’t prove that their advice transformed clients into top performers. It could simply be that top performers have more money for hiring consultants. Indeed, any claim that Bain deserves credit for such performance is conspicuously absent from the Web site, at least as of fall 2005. Perhaps the hope is that visitors will momentarily forget what they learned in their statistics classes!
Treat the organization as an unfinished prototype.
For some questions in some businesses, the best evidence is to be found at home—in the company’s own data and experience rather than in the broader-based research of scholars. Companies that want to promote more evidence-based management should get in the habit of running trial programs, pilot studies, and small experiments, and thinking about the inferences that can be drawn from them, as CEO Gary Loveman has done at Harrah’s. Loveman joked to us that there are three ways to get fired at Harrah’s these days: steal, harass women, or institute a program without first running an experiment. As you might expect, Harrah’s experimentation is richest and most renowned in the area of marketing, where the company makes use of the data stream about customers’ behaviors and responses to promotions. In one experiment reported by Harvard’s Rajiv Lal in a teaching case, Harrah’s offered a control group a promotional package worth $125 (a free room, two steak dinners, and $30 in casino chips); it offered customers in an experimental group just $60 in chips. The $60 offer generated more gambling revenue than the $125 offer did, and at a reduced cost. Loveman wanted to see experimentation like this throughout the business, not just in marketing. And so the company proved that spending money on employee selection and retention efforts (including giving people realistic job previews, enhancing training, and bolstering the quality of frontline supervision) would reduce turnover and produce more engaged and committed employees. Harrah’s succeeded in reducing staff turnover by almost 50%.
Similarly, CEO Meg Whitman attributes much of eBay’s success to the fact that management spends less time on strategic analysis and more time trying and tweaking things that seem like they might work. As she said in March 2005, “This is a completely new business, so there’s only so much analysis you can do.” Whitman suggests instead, “It’s better to put something out there and see the reaction and fix it on the fly. You could spend six months getting it perfect in the lab…[but] we’re better off spending six days putting it out there, getting feedback, and then evolving it.”
Yahoo is especially systematic about treating its home page as an unfinished prototype. Usama Fayyad, the company’s chief data officer, points out that the home page gets millions of hits an hour, so Yahoo can conduct rigorous experiments that yield results in an hour or less—randomly assigning, say, a couple hundred thousand visitors to the experimental group and several million to the control group. Yahoo typically has 20 or so experiments running at any time, manipulating site features like colors, placement of advertisements, and location of text and buttons. These little experiments can have big effects. For instance, an experiment by data-mining researcher Nitin Sharma revealed that simply moving the search box from the side to the center of the home page would produce enough additional “click throughs” to bring in millions more dollars in advertising revenue a year.
A big barrier to using experiments to build management knowledge is that companies tend to adopt practices in an all-or-nothing way—either the CEO is behind the practice, so everyone does it or at least claims to, or it isn’t tried at all. This tendency to do things everywhere or nowhere severely limits a company’s ability to learn. In particular, multisite organizations like restaurants, hotels, and manufacturers with multiple locations can learn by experimenting in selected sites and making comparisons with “control” locations. Field experiments at places such as McDonald’s restaurants, 7-Eleven convenience stores, Hewlett-Packard, and Intel have introduced changes in some units and not others to test the effects of different incentives, technologies, more interesting job content, open versus closed offices, and even detailed and warm (versus cursory and cold) explanations about why pay cuts were being implemented.
Embrace the attitude of wisdom.
Something else, something broader, is more important than any single guideline for reaping the benefits of evidence-based management: the attitude people have toward business knowledge. At least since Plato’s time, people have appreciated that true wisdom does not come from the sheer accumulation of knowledge, but from a healthy respect for and curiosity about the vast realms of knowledge still unconquered. Evidence-based management is conducted best not by know-it-alls but by managers who profoundly appreciate how much they do not know. These managers aren’t frozen into inaction by ignorance; rather, they act on the best of their knowledge while questioning what they know.
Evidence-based management is conducted best not by know-it-alls but by managers who profoundly appreciate how much they do not know.
Cultivating the right balance of humility and decisiveness is a huge, amorphous goal, but one tactic that serves it is to support the continuing professional education of managers with a commitment equal to that in other professions. The Centre for Evidence-Based Medicine says that identifying and applying effective strategies for lifelong learning are the keys to making this happen for physicians. The same things are surely critical to evidence-based management.
Another tactic is to encourage inquiry and observation even when rigorous evidence is lacking and you feel compelled to act quickly. If there is little or no information and you can’t conduct a rigorous study, there are still things you can do to act more on the basis of logic and less on guesswork, fear, belief, or hope. We once worked with a large computer company that was having trouble selling its computers at retail stores. Senior executives kept blaming their marketing and sales staff for doing a bad job and dismissed complaints that it was hard to get customers to buy a lousy product—until one weekend, when members of the senior team went out to stores and tried to buy their computers. All of the executives encountered sales clerks who tried to dissuade them from buying the firm’s computers, citing the excessive price, weak feature set, clunky appearance, and poor customer service. By organizing such field trips and finding other ways to gather qualitative data, managers can convey that decisions should not ignore real-world observations.
Will It Make a Difference?
The evidence-based-medicine movement has its critics, especially physicians who worry that clinical judgment will be replaced by search engines or who fear that bean counters from HMOs will veto experimental or expensive techniques. But initial studies suggest that physicians trained in evidence-based techniques are better informed than their peers, even 15 years after graduating from medical school. Studies also show conclusively that patients receiving the care that is indicated by evidence-based medicine experience better outcomes.
At this time, that level of assurance isn’t available to those who undertake evidence-based management in business settings. We have the experience of relatively few companies to go on, and while it is positive, evidence from broad and representative samples is needed before that experience can be called a consistent pattern. Yet the theoretical argument strikes us as ironclad. It seems perfectly logical that decisions made on the basis of a preponderance of evidence about what works elsewhere, as well as within your own company, will be better decisions and will help the organization thrive. We also have a huge body of peer-reviewed studies—literally thousands of careful studies by well-trained researchers—that, although routinely ignored, provide simple and powerful advice about how to run organizations. If found and used, this advice would have an immediate positive effect on organizations.
Does all this sound too obvious? Perhaps. But one of the most important lessons we’ve learned over the years is that practicing evidence-based management often entails being a master of the mundane. Consider how the findings from this one little study could help a huge organization: An experiment at the University of Missouri compared decision-making groups that stood up during ten- to 20-minute meetings with groups that sat down. Those that stood up took 34% less time to make decisions, and the quality was just as good. Whether people should sit down or stand up during meetings may seem a downright silly question at first blush. But do the math. Take energy giant Chevron, which has over 50,000 employees. If each employee replaced just one 20-minute sit-down meeting per year with a stand-up meeting, each of those meetings would be about seven minutes shorter. That would save Chevron over 350,000 minutes—nearly 6,000 hours—per year.
Leaders who are committed to practicing evidence-based management also need to brace themselves for a nasty side effect: When it is done right, it will undermine their power and prestige, which may prove unsettling to those who enjoy wielding influence. A former student of ours who worked at Netscape recalled a sentiment he’d once heard from James Barksdale back when he was CEO: “If the decision is going to be made by the facts, then everyone’s facts, as long as they are relevant, are equal. If the decision is going to be made on the basis of people’s opinions, then mine count for a lot more.” This anecdote illustrates that facts and evidence are great levelers of hierarchy. Evidence-based practice changes power dynamics, replacing formal authority, reputation, and intuition with data. This means that senior leaders—often venerated for their wisdom and decisiveness—may lose some stature as their intuitions are replaced, at least at times, by judgments based on data available to virtually any educated person. The implication is that leaders need to make a fundamental decision: Do they want to be told they are always right, or do they want to lead organizations that actually perform well?
If taken seriously, evidence-based management can change how every manager thinks and acts. It is, first and foremost, a way of seeing the world and thinking about the craft of management; it proceeds from the premise that using better, deeper logic and employing facts, to the extent possible, permits leaders to do their jobs more effectively. We believe that facing the hard facts and truth about what works and what doesn’t, understanding the dangerous half-truths that constitute so much conventional wisdom about management, and rejecting the total nonsense that too often passes for sound advice will help organizations perform better.
A version of this article appeared in the January 2006 issue of Harvard Business Review.