Measures of Success

With assurance of learning standards in full force, business educators evaluate how well their assessment plans are working—and where those plans should lead next.
Measures of Success

For many business schools, assessment isn’t just an accreditation requirement; it’s a crucial component of higher education. It’s also widely misunderstood, say those in assessment circles. They note that some administrators view assessment as a task to check off for accreditation purposes, rather than an ongoing process of continuous improvement. Meanwhile, many faculty view assessment with apprehension—they fear the gathering of data will disrupt their schedules, add to their workloads, or even be used to critique their teaching.

Kathryn Martell is the associate dean of Montclair State University’s School of Business in New Jersey and a frequent presenter at AACSB International’s assessment seminars. Since AACSB put its assurance of learning (AoL) standards in place in 2003, the business school community has made great strides where assessment is concerned, Martell believes. “It’s on everybody’s radar screen,” she says. “When I first began working with AACSB in 2003, the language was new to everybody. Faculty said, ‘We do student surveys. We’re doing great.’ Now they know that those surveys are not appropriate evidence for the purpose of assessment.”

A long-time advocate for assessment, Douglas Eder now serves as the associate provost of institutional effectiveness at the University of North Florida in Jacksonville. He emphasizes that while assessment activities are now firmly in place, attitudes still need to change. “We must assure faculty that we’re not interested in assessment itself,” says Eder. “We’re interested in what it tells us about what our students are learning.”

Like many proponents of learning assessment, Martell and Eder believe that schools are doing many things right when it comes to assessment. For example, Martell has seen a dramatic shift in resources, citing that business schools have gone from spending an average of only $5,000 a year on assessment to spending $50,000. Now, schools must put that money to best use by making assessment an integral part of their programs and mindset, she says. More important, administrators and faculty must realize that assessment itself is not the ultimate goal, but a means to a more important end.

Closing the Loop

Although most business schools have begun assessment in earnest, say Martell and Eder, many have yet to take an all-important next-step: “closing the loop.” That is, they fail to use the data they collect to make tangible improvements. In fact, some have yet to finalize their assessment plans. According to AACSB’s new standards, schools were to have their assessment plans fully in place by 2007, but many have fallen behind that original timetable.

“Few industries are slower moving than academia. Many faculty believe they can debate learning goals for three years,” says Martell. “But they have to realize that they don’t have to approach assessment as they would a research study.

“Many faculty believe they can debate learning goals for three years. But they don’t have to approach assessment as they would a research study. They just have to jump in, get started, and do their best.” —Kathryn Martell, Montclair State University

They don’t need a measure that’s been validated from here to Timbuktu. They just have to jump in, get started, and do their best.”

Last November, AACSB released its white paper, “AACSB Assurance of Learning Standards: An Interpretation,” to help schools through this process. It aims to provide guidance, answer frequently asked questions, and spark further dialogue about assessment.

Course-embedded assignments, capstone courses, and computer simulations provide a wide array of tools for business schools; software providers offer programs to help schools more easily gather and utilize data. Martell also points to standalone testing instruments such as the Major Field Tests in Business issued by ETS, headquartered in Princeton, New Jersey. She emphasizes that while the MFT alone isn’t a sufficient assessment measure, it offers business schools yet another tool for their larger assessment toolbox. “The results of standardized tests are difficult to use to improve the curriculum,” she says. “But if you need data quickly, the Major Field Test can be easily implemented.”

Some schools have already taken action to close the loop, Martell notes. For example, when business professors at Montclair State University discovered that their students weren’t retaining their skills in statistics, they decided to record mini-lectures on quantitative techniques. These files were then stored in an “online toolbox” that all students could access. What started as a plan to help students refresh their statistics skills will soon be expanded to include mini-lectures on other subject matter, such as technology and business communication.

At Temple University’s Fox School of Business in Philadelphia, faculty found that students needed to improve their ability to integrate information across disciplines. Rather than address this issue with a typical capstone course at the end of the program, faculty created a sophomore-level course built around the Capstone computer simulation from Capsim Management Simulations (formerly Management Simulations Inc.) of Northfield, Illinois. Through the course and simulation, students now learn to integrate what they’re learning early in the curriculum.

“While faculty are learning new pedagogies, they won’t be submitting grant proposals or writing research papers. Business schools need to adopt new systems that reward faculty for more than their research.” —Douglas Eder of the University of North Florida

When faculty at the Mays Business School at Texas A&M University in College Station examined their assessment data, they found that their students had strong technical skills but showed weaker performance in soft skills such as ethical reasoning, communication, and teamwork. Mays faculty created a course specifically designed to improve students’ soft skills. This fall, the new course will be required for all undergraduates.

That kind of swift action toward genuine improvement lies at the heart of effective AoL, Martell and Eder emphasize. And with so many tools and approaches available, the process invites experimentation and refinement over time. Before long, faculty can have a process that works smoothly and is in line with the school’s mission-based objectives.

“After a year or two, many schools find that assessment is an incredibly gratifying experience,” says Martell. “They love that it brings them back to talking about teaching and learning.”

Engaging Faculty

Assessment advocates emphasize that unless both faculty and administrators share an enthusiasm for assessment initiatives, there is a risk that their efforts will fall flat. In that case, programs won’t benefit from the curricular improvements that can arise from an effective AoL plan.

Faculty buy-in presents one of the biggest obstacles to assessment, says Matt Bartel, president of Digital Measures, an assessment software company headquartered in Milwaukee, Wisconsin. The most successful schools present AoL not as an externally driven requirement for accreditation, but as a plan that will benefit the faculty and the school, Bartel notes.

“Schools that offer an artificial purpose for pursuing assurance of learning get artificial buy-in,” he says. “One of the most important parts of the equation is communicating the true value of the information to everyone involved.”

Many assessment experts suggest that achieving genuine enthusiasm about assessment starts with faculty development. Most faculty are trained to explore their disciplines, not new modes of teaching, argues Eder of the University of North Florida. Therefore, an even bigger challenge, he says, is to ensure that faculty learn new pedagogies that will translate to better learning outcomes.

It will be up to business school administrators to support that kind of development, he says. He adds that if faculty aren’t encouraged to use assessment as a springboard to inspire new approaches in the classroom, the data they gather will have no true purpose. “Schools have to protect faculty as they learn new ways to teach,” says Eder. “While faculty are learning new pedagogies, they won’t be submitting grant proposals or writing research papers. Business schools need to adopt new systems that reward faculty for more than their research.”

“We don’t need to assess a huge number of students; assessing a representative sample is enough. The point is to generate data we can use.” —Kathryn Martell, Montclair State University

Too Little, Too Late?

Even with all their efforts so far, business schools in the U.S. may still face a bigger challenge than convincing faculty of the benefits of assessment, says Eder. “Higher education has had 20 years to get this right, and most institutions haven’t,” he says. “Now, the federal government is threatening to intervene.” If that happens, business schools may have to work hard to convince the federal government that their assessment efforts are working.

Already two major groups are advocating measures that could substantially influence assessment in public undergraduate education. In November, the National Association of State Universities and Land-Grant Colleges and the American Association of State Colleges and Universities together approved the Voluntary System of Accountability (VSA) initiative. Under the VSA, leaders from more than 70 four-year public institutions of higher education set standards for assurance of learning, creating a data-reporting template that could become a starting point for federal oversight.

AACSB adopted its new standards in 2003; the release of the Spellings Report in 2006 only increased the pressure on colleges and universities to set standards when it called for an overhaul of U.S. higher education. Some fear that the next step may be an act of Congress to make assessment mandatory for public higher education institutions.

Martell, however, believes that business schools still control their own destiny—if they act now. “The Spellings Report frightened a lot of people, because it called for some type of standardized assessment. That, of course, wasn’t welcomed because different schools have such different missions,” she says. “It’s important that organizations like AACSB educate Congress on the professional requirements that are already in place. As long as Congress gets the message that we are pursuing assurance of learning voluntarily, we’ll be able to continue to police ourselves.”

Reaching Beyond Assessment

Since 2003, many business schools have been fine-tuning their assessment plans. Now that they’re seeing those plans translate into tangible, positive improvements to their programs, a growing number of faculty are joining Eder and Martell in their enthusiasm for assessment and its place in improving learning outcomes.

One of the key points for school administrators to remember, says Martell, is not to overdo it. “Some schools try to make this way too complicated. I’ve seen some schools that have set 100 learning objectives,” says Martell. “We don’t have to assess everything—AACSB only asks us to identify five or six goals that we think are important. Moreover, we don’t need to assess a huge number of students; assessing a representative sample is enough. The point is to generate data we can use.”

Coles College of Business at Kennesaw State University in Marietta, Georgia; the College of Business at California State University, Chico; and Open University’s OU Business School in the United Kingdom are three schools where assessment plans have come to fruition. Their stories are shared in “Lessons of Assessment” on page 24, “The Devil’s in the Data” on page 26, and “Assessment at a Distance” on page 28. Educators at these schools note that a majority of their faculty have embraced the process as an opportunity, rather than an added burden. Their experiences represent how far business schools have come where assessment is concerned. Yet, as many assessment experts argue, challenges still lie ahead.

So far, business schools have been trying and testing their approaches to find a system of accountability that works for their missions. By emphasizing streamlined systems and team involvement, many schools have found that effective assessment can be a strong catalyst for positive change. Once school faculty and administration realize the power of assessment data, say many experts, they become more excited about the prospect of improving their teaching, their programs, and their graduates.

For more information about AACSB’s assurance of learning standards, visit www.aacsb.edu/resource_centers/assessment. AACSB’s white paper, “Assurance of Learning Standards: An Interpretation,” can be found at www.aacsb.edu/accreditation/Papers/index.asp. For information about the Major Field Test in Business, visit www.ets.org. To read more about the VSA initiative, visit www.voluntarysystem.org.

For more information about AACSB’s assurance of learning standards, visit www.aacsb.edu/resource_centers/assessment. AACSB’s white paper, “Assurance of Learning Standards: An Interpretation,” can be found at www.aacsb.edu/accreditation/Papers/index.asp. For information about the Major Field Test in Business, visit www.ets.org. To read more about the VSA initiative, visit www.voluntarysystem.org.