Measures that Matter

For one business school, a successful approach to assessment is driven by two fundamental questions: What can we discover about student learning? What should we do about it?
Measures that Matter

The navigational menus on most business school websites feature links related to areas such as “Admissions,” “Programs,” or “Research.” But the College of Business Administration (CBA) at the University of Wisconsin–La Crosse adds a less expected heading to its navigational lineup: “Assurance of Learning.” The link leads visitors to the college’s learning goals for its undergraduate and MBA programs, its annual assessment reports, descriptions of its rubrics, and resources on best practices in assessment and assurance of learning (AoL). The school presents this information on its website at www.uwlax.edu/ba/AOL/aol.htm.

“We make this information available for our stakeholders to see,” says Betsy Knowles, a senior lecturer in the economics department and coordinator of the school’s assessment task force. “This is a way we can let students, faculty, and employers know: These are the learning outcomes that matter to us.”

Such transparency is just one component of the CBA’s comprehensive approach to AoL, which has become an integral part of the school’s educational culture. “Everything we do is guided by two questions,” says Knowles. “What can we discover about student learning? And what should we do about it?” In fact, the efforts of CBA’s faculty to impact student learning recently garnered national attention. The college was one of four schools to receive the 2013 Council for Higher Education Accreditation’s (CHEA) Award for Outstanding Institutional Practice in Student Learning Outcomes—and only the second business school ever to receive the recognition. (In 2008, Seton Hall University’s Stillman School of Business in South Orange, New Jersey, received the honor.)

The story of how the CBA helped its faculty get excited about measuring and enhancing their students’ learning could serve as a blueprint for other schools that seek to ramp up their own AoL efforts. It’s one that the college is eager to tell.

Energy and Expertise

CHEA’s recognition is a culmination of a process that began in 2009, just after the CBA had spent two semesters revising its under-graduate student learning goals and rethinking its general approach to AoL. “We recognized that we needed to develop a structure and process that emphasized faculty participation—what you might call a culture,” says Knowles. “We wanted a process that was more sustainable over time.”

Forming an assurance of learning task force was a critical first step, says Bruce May, dean and professor of management at the CBA. “We handpicked a team of junior and senior faculty and instructors who were well-respected, could work together, and had the energy and expertise to play key roles in assessment activities,” he says. The college chose individuals who had served on the undergraduate and graduate curriculum committees to create synergy between curriculum development and assessment activities.

The second step was to create a system where conversations about assessment occurred frequently and naturally throughout the college, says Knowles. To that end, the task force asked professors from each department to volunteer as core course coordinators to work directly with the task force to link learning goals and effect curricular change. They also act as liaisons between the task force and departments, and they arrange meetings with their department’s faculty to discuss how they should respond to assessment results.

A third step was to invite stake-holders from across the college to participate in the assessment process. Today, students and representatives from companies that employ the school’s graduates meet with task force members each semester and attend the annual faculty retreat. “The structure of our task force provides channels of instant communication throughout the college,” says Knowles.

Strong Support Systems

When the college first started to implement its assessment plan, its task force let faculty know about the school’s renewed emphasis on assessment at every committee or faculty meeting. “We talked about the language of assessment, we talked about outcomes—at one stage we even did an entire college reading of a single course-embedded assessment, so that when we would say, ‘We’re doing a reading,’ everyone would know what we meant,” says Knowles.

Now, faculty are well-versed in assessment, and May makes sure they have everything they need to produce high-quality assessment data and respond in meaningful ways. (See “Committing to Assessment” at right.) For example, the school has integrated a number of events, workshops, and procedures into its regular schedule:

An annual faculty retreat. Held each spring semester, the faculty retreat is one of the most important events of the CBA’s academic calendar. During each retreat, the faculty review assessment data from the previous year and hash out what worked and what didn’t. They can propose adding a new subject to the curriculum or augmenting a current one. At this year’s retreat, faculty discussed how to integrate social responsibility to their courses, as well as what learning objectives and rubrics would best suit that topic.

The retreats give faculty an entire day to engage in cross-disciplinary discussions centered on student learning, says Knowles. “They can sit at a table together and ask the big questions: ‘Why do our students struggle with the big picture during presentations?’ or ‘How can we help them become more comfort-able with public speaking?’”

At the last faculty retreat, members of the board of advisors talked about what they, as employers, want students to know upon graduation. “Faculty said that it was good to look at assessment from a business perspective,” says May.

An annual assessment orientation. For the past two years, the school has held an annual orientation for new faculty, which covers why assessment is important and how the school approaches the process. The content covers areas such as the difference between direct and indirect assessment, the nature of course-embedded assessments, and the  ways the CBA defines terms such as “outcomes,” “goals,” and “objectives.”

Workshops on teaching. Faculty have asked for more professional development opportunities to help them improve their teaching, particularly in the areas of critical thinking and written communication. In response, the college worked with UW-L’s Center for Advancing Teaching and Learning to develop a series of afternoon faculty workshops that target those subjects. During the 2011–2012 academic year, says Knowles, 18 of the school’s 60 faculty members reported making changes to the way they teach critical thinking and 16 reported making changes to their teaching of written communication as a direct result of what they learned.

Faculty-led assessment. At the end of each semester, the college asks faculty to volunteer to read course-embedded assessments and record the resulting data. Faculty read assessments at the same table, where they can discuss the results on the spot. The information gathered during these meetings, as well as during the rest of the year, is reported to the college before the next retreat.

Faculty anonymity. Perhaps most important, the college makes it clear that assessment is about student learning, not faculty performance. For that reason, reports never identify instructors, which increases faculty’s willingness to participate and see the value of the process. “Faculty must understand that we are not assessing individual courses, but how the courses work together across the entire pro-gram,” says May. “Once they realize that this isn’t about them, they are able to relax.”

‘Let’s Go for It’

 Although the college made a plan before it moved forward, its task force did not set the plan’s boundaries too rigidly. “We debated what approaches we should use. Course-embedded? Standardized? Something else? Finally, we just said, ‘Let’s try them all. Let’s go for it.’ From the beginning, we’ve kept open minds,” Knowles says.

The fact that the school’s assessment activities are transparent and faculty-driven means that CBA faculty feel free to raise concerns and suggest courses of action. For instance, one of the school’s assessment rubrics measures how well students identify and consider information relevant to the task at hand. Recently, a faculty member suggested that it might be just as important to know whether students successfully disregard irrelevant information, by creating a rubric in which faculty have deliberately integrated meaningless data. In response to that proposal, faculty have created course-embedded assessment tasks that integrate irrelevant content. The undergraduate curriculum commit-tee also is considering an addition to the CBA’s critical thinking rubric to address this issue.

Knowles acknowledges that such a “let’s try anything” approach may sound too broad or diffuse for some, but she emphasizes that it’s actually quite consistent with the college’s central objective: to improve educational outcomes.

“As faculty, we see this process as a constant state of learning—we’re learners, just as our students are learners. It’s gratifying to hear our faculty say, ‘I never thought of it that way’ or ‘I have a new idea to try in class,’” says Knowles. “We’ve realized that with an ounce of flexibility, we can provide a richer learning environment for our students.”

Engaged and Excited

Knowles notes that the CBA’s approach melds well with the intentions of Standard 8 of AACSB International’s new set of accreditation standards, which AACSB’s members approved at its annual meeting in April. Standard 8 highlights the need to document the level of faculty engagement. In 2011–2012, for instance, 45 of the CBA’s 60 faculty attended the faculty retreat, 28 went to teaching workshops, and nine went to AACSB assessment seminars. “We have high participation rates, and some of what faculty have learned has translated into papers and presentations about student learning. That’s something we’re really excited about,” she says.

By keeping the school’s focus on student learning—and not on faculty performance—the CBA has been able to keep its faculty’s enthusiasm and participation rates high. In the end, Knowles offers these words of advice: Take it slow. Do what matters. Let faculty drive the process.

After all, most faculty want to improve students’ learning. When that’s the goal, says Knowles, everything else just makes sense.