Teaching Students to Follow the Evidence

If we are to persuade our students to question their strongly held opinions, we can’t just present the best available evidence. We must help students overcome the most common sources of their resistance to research.
Teaching Students to Follow the Evidence

MANAGERS OFTEN MAKE decisions without consulting the best available management research. Why? Researchers offer two likely explanations: Either managers are unaware of relevant research findings or they do not implement them because of challenges involved in accessing and applying those findings.

Recently, we have seen a troubling third reason emerge. A growing body of evidence suggests that even when people are aware of research findings, supported by a vast majority of studies, many choose not to believe them. The best-known example of this phenomenon involves climate change. Despite strong scientific consensus that human activity is driving rising global temperatures, surveys by the Pew Research Center show that twofifths of Americans do not believe the earth is warming.

We see many similar examples in management. Hundreds of studies show that intelligence is a better predictor of a job applicant’s future performance than either values or conscientiousness, that algorithmic hiring models out-predict those based on managerial judgment, and that minorities score as high on tests of integrity as whites do. And, yet, a majority of human resource managers do not believe those findings are true.

When we and our colleagues at other business schools survey students on the first day of class, we see the same phenomenon—many students’ beliefs contradict the best available research. They express highly divergent opinions about topics such as the benefits of diversity, the outcome of pay-for-performance plans, and the extent of discrimination in labor markets, even though the best available evidence clearly weighs in on one side over the other.

It’s inevitable that some students will resist evidence-based arguments. To persuade these students to believe—or even to consider—the evidence, we must acknowledge the validity of multiple viewpoints, even as we nudge students toward evidence-based beliefs. Here, we share four strategies that faculty can use to educate managers who can think critically, who can distinguish strong evidence from weak evidence, and who understand the importance of applying the best research to business problems.

REDUCE RELIANCE ON INTUITION

Sometimes people resist management research because they are overconfident in their own intuition. They believe that good management is a function of common sense and “going with your gut”—that although other people might not make good decisions, they do.

One way to overcome this resistance is to catch them in the act of making incorrect or biased decisions. For example, on the first day of his MBA courses, Amir Erez of the University of Florida in Gainesville asks students to take a 15-statement true-false quiz, where each statement is either consistent or inconsistent with the study findings on topics such as goal setting, hiring, or compensation practices. He then posts the quiz results on the board. Many students are surprised to discover that they scored no better than chance, showing them that intuition will not always help them recognize research-supported answers.

Gary Latham of the University of Toronto in Ontario, Canada, has used another experiential technique to illustrate bias in decision-making. He gave a detailed job description to middle managers with several years of experience in conducting performance appraisals. He then divided them into two groups and showed each group a video of a job interview. In one video, the applicant said that he had two brothers, a father with a PhD in physics, and a mother with a master’s degree in social work; in the other, the same applicant said he had 12 siblings, a father who was a bus driver, and a mother who was a maid. The first group rated the applicant’s qualifications as close to 9 on a 9-point scale, while the second group rated this same person’s qualifications between 5 and 6.

This exercise illustrated common biases, such as the “similar-to-me” error, and made these managers aware of stereotypes based on social class. Similar experiments using male/female or white/minority protagonists produce similar findings despite decades of anti-discrimination legislation, affirmative action programs, and diversity and inclusion training. Catching people “in the act” makes it more difficult for them to claim they aren’t subject to such biases.

EXPLORE THE ‘WHY’ OF SURPRISING FINDINGS

Another area where many people hold beliefs inconsistent with the research involves the topic of pay-for-performance (PFP). Many practicing managers have heard in their business classes that money is low on people’s hierarchy of needs (Abraham Maslow); that money is a dissatisfier rather than a motivator (Frederick Herzberg); and that PFP destroys intrinsic motivation, thus reducing performance (Edward Deci). Some current organizational behavior textbooks still emphasize the positive influence of intrinsic motivators and the detrimental effect of extrinsic motivators on performance. However, meta-analytic reviews of PFP programs in real work organizations show this latter claim to be overwhelmingly false.

Sara Rynes, the first author of this article, tackles this discrepancy in her class at the University of Iowa’s Tippie College of Business in Iowa City. Before a discussion of compensation and motivation, she shows students a widely viewed TED talk by author Daniel Pink. In his talk, Pink draws from his book Drive: The Surprising Truth About What Motivates Us, in which he claims that PFP schemes are ineffective ways to motivate performance, especially in professional and creative fields. At first, most students are favorably impressed. (Granted, most come from the management and organizations department. Those in economics might be more likely to believe in the power of monetary incentives.)

Rynes then shares with them meta-analyses of studies that dispute Pink’s claims and asks them to consider why Pink’s argument contradicts these studies’ findings. They point to reasons why the studies Pink cites might have limited applicability to the real world. They were conducted in labs, not in real workplaces; they were conducted in compressed time periods; they used miniscule amounts of compensation; and they described stimuli as “games” or “puzzles” rather than work.

Once students engage in this type of critical thinking, they are more prepared to accept the overwhelming evidence that PFP is associated with higher performance in real organizations. They also can explain why the best professional workers—who sometimes produce ten or 20 times more value than the average worker—are disproportionately likely to leave their organizations unless they are paid differentially for their higher value to the firm. Contrary to Pink’s claims, research shows this to be especially true of professional and creative workers.

As Laszlo Bock, former head of Google’s People Department says, “If your best person is worth ten of the average people, you must pay ‘unfairly’... otherwise you’re just giving them a reason to quit.” After this discussion, students better understand why companies such as Microsoft, Google, and Amazon strongly differentiate the pay of their most creative and innovative employees.

ACKNOWLEDGE RESISTANCE, ENCOURAGE CRITICAL THINKING

At least a few students in every class are likely to be skeptical of the evidence, even if they don’t express their doubts aloud. For this reason, instructors may want to acknowledge and give voice to these doubts, opening up an opportunity to explore the reasons behind them.

One strategy is to teach critical thinking skills at the beginning of every management course. In the first meeting of her courses, Amy Colbert—the second author of this article, also from Tippie College—encourages students to become critical consumers of research who work to understand study findings deeply before attempting to apply them.

Colbert and her students also explore the boundaries of evidence-based practices and look for guidance on when a practice is likely to be most effective. This discussion can prevent students from automatically concluding that a practice “will never work in my organization” and offer insights into how the evidence might be applied successfully.

Students come to realize that evidence should neither be accepted at face value nor doubted just because it does not align with their pre-existing beliefs or individual experiences. Once students improve their critical thinking skills in this way, their discussion about an ineffective diverse team, for example, is more likely to include a conversation about factors that might have contributed to poor team performance or strategies that might have improved the outcome. They won’t be so quick to simply dismiss evidence on the effectiveness of diversity.

USE A CREDIBLE MESSENGER

While some research findings are disbelieved by a majority of people (like those on the importance of intelligence to job performance), other findings are strongly believed by some and strongly disbelieved by others (like those addressing topics such as corporate social responsibility, immigration, and diversity). On many issues, beliefs tend to split along political, religious, regional, socioeconomic, and demographic lines. In such cases, it helps to supplement the best available evidence with a business story told by a credible messenger.

Take the case of diversity. Research does not depict a universally positive relationship between diversity and performance, but a preponderance of the evidence suggests that diverse organizations outperform less diverse ones.

Rynes and a co-instructor in the Tippie Executive MBA program bring in Mark Buthman, the former vice president of finance from the Fortune 200 firm Kimberly-Clark, to speak to students. Buthman tells students how, after the firm’s stock price had stalled at US$60 per share from 2000 to 2009, an analysis revealed that a lack of diversity was inhibiting the company’s progress.

A consultant helped the company craft an aggressive diversity strategy. In five years, the company increased its percentage of women directors by more than 90 percent and the number of minorities at the director level or above by 59 percent. It was named one of the Five Best Places to Work, won the 2014 Catalyst Award for its diversity and inclusion efforts, and saw its stock price increase to $115 by the end of 2014. Students—both male and female, white and minority—tend to ask him questions long after class is officially over.

LOOKING TO THE FUTURE

Ironically, only a few studies have been done on why strategies such as those we describe above work to help students rely more on research to make managerial decisions. That’s why we encourage further development and dissemination of such tactics, accompanied by research to ascertain their validity. Now that the current polarization of our culture has found its way into our classrooms, it is of even greater importance for us to overcome our students’ resistance to management research.




For Additional Reading

“When the ‘Best Available Evidence’ Doesn’t Win: How Doubts About Science and Scientists Threaten the Future of Evidence-Based Management” by Sara L. Rynes, Amy E. Colbert, and Ernest C. O’Boyle, published online August 30, 2018, in the Journal of Management.

“Separating Data from Intuition:Bringing Evidence into theManagement Classroom” by Amir Erez and Adam M. Grant, published online December 13, 2013, in Academy of Management Learning and Education.




Sara L. Rynes is the Tippie-Rollins Chair in Excellence and Amy E. Colbert is department executive officer and the Leonard A. Hadley Chair in Leadership at the University of Iowa’s Tippie College of Business in Iowa City.

This article originally appeared in BizEd's July/August 2019 issue. Please send questions, comments, or letters to the editor to bized.editors@aacsb.edu.

Advertise With BizEd