Assessment Investment

A five-point strategy helps schools overcome faculty resistance to assurance of learning programs and encourages professors to invest in the assessment process.
Assessment Investment

Mention assessment to most faculty, and they suddenly become too busy to have even a three-minute chat. Their file cabinets need cleaning out, their class notes from 1999 need updating, they’re on their way to an off-campus appointment. Assessment is an anathema to many professors, who consider it just another way for administrators to interfere with teaching. And yet, business faculty must get on board with assessment practices, because AACSB International requires accredited schools to maintain faculty-driven assurance of learning programs.

I’ve been involved in a number of assessment programs, and I’ve concluded that faculty usually have three reasons to resist. First, they’re already so busy with research, teaching, and service requirements that they have little time for additional activities, particularly ones they perceive as busywork. Second, they question the value of assurance of learning activities because the benefits are abstract, while the costs are concrete. Furthermore, the benefits—better learning—accrue to the students, while the costs—additional work—accrue to the faculty.

Third, and this is paramount, they think these programs impinge on their academic freedom. They are adamant about maintaining complete authority to design and deliver their courses as they see fit.

But it is possible for schools to design assurance of learning programs that overcome these obstacles to faculty participation. At Bryant University’s College of Business, we have adopted strategies that have resulted in more than half of our faculty participating in the assurance of learning process. Other areas of the university have also benefited, as many of our assessment practices have spilled over into the College of Arts & Sciences.

We believe there are five key factors in making faculty enthusiastic about assessment: a supportive administration, a faculty champion, an evolving development process, a well-defined structure, and an emphasis on excellent communication. In our case, two other factors helped us enlist faculty support when we needed it most: an impending AACSB Maintenance of Accreditation visit, and a growing program that appealed to new faculty, who might have been more receptive to assurance of learning than more entrenched professors. But while these last two components provided an extra push, we believe that the real keys to our success are the first five factors—and that any other school can employ them as well.

One: Top-Level Support

For any program to succeed, the most senior-level members of the administration must be behind it. At Bryant, our most ardent supporter of faculty-led assurance of learning proved to be Jack Trifts, who became dean of the College of Business in 2005. Trifts immediately convened an Assessment Committee composed of one representative from each college department. He also served as an active participant on the committee.

In three ways, he signaled to faculty across the college that assurance of learning was critical. First, by serving on the assessment committee himself, he showed how important he thought it was, and he encouraged other faculty members to move quickly on assessment efforts. He also modeled behavior and provided insights about assessment practices, since he had served on AACSB reaccreditation teams for a number of schools.

Second, he made sure assessment was an agenda topic at all collegewide faculty meetings, which were held three or four times a year. At those meetings, even faculty who weren’t yet actively involved in assurance of learning were kept informed of the progress made by members of the Assessment Committee.

Finally, Trifts gave other members of the committee financial and strategic support. He made sure they were sent to AACSB assessment conferences, and he gave them copies of Assessment of Student Learning in Business Schools: Best Practices Each Step of the Way, edited by Kathryn Martell and Thomas Calderon.

Two: A Faculty Champion

A school that wants a faculty-led assurance of learning process must have an outspoken proponent who is knowledgeable about the subject and will speak about it enthusiastically to anyone, anywhere, anytime. This assessment cheerleader should be a full-time faculty member with teaching, research, and service responsibilities who has credibility with the rest of the faculty and is respected by colleagues. He or she also should be an outgoing person who proactively connects with other faculty and actively shares information. Ideally, the champion can share positive outcomes from other assessment experiences to show that, indeed, assessment is a good thing!

At Bryant, I serve as our faculty champion. Although I was new to the college in 2005, I had assessment experience at my previous school, Northern Illinois University, and I staunchly believe the assessment process has benefits for both students and faculty. I first served as our department’s representative to the Assessment Committee, and I became chair of the committee in 2006.

As faculty champion, I always listen to alternate opinions and suggestions, but I will not let problems or criticisms get in the way of progress. In fact, that attitude—that determination to move forward constantly despite setbacks—is essential for anyone involved in the assessment process.

Three: Continuous Development

School administrators shouldn’t wait to craft the perfect assessment plan before they start implementing it, or they will surely fail. They need to design an initial plan and then jump right in, making incremental improvements as they go along.

At Bryant, our goal was to develop and implement our assurance of learning process quickly, recognizing that it wouldn’t be perfect but that we could improve upon it over time. We started out by identifying learning goals and objectives for the primary undergraduate program, the BSBA. We presented a draft of the learning goals to the faculty in spring 2006, and we were already planning how to assess them while the College of Business faculty reviewed and discussed them. It took about three months before the faculty approved the goals. Then the hard work began.

For an assessment program to succeed, administrators must devise a system that encourages faculty to participate—and lets them know exactly what’s expected of them when they do.

The Assessment Committee developed a matrix that illustrated which required courses covered or evaluated the objectives of the six learning goals. The matrix also indicated where we might find student artifacts, such as class assignments or completed projects, that could be used for assessment.

We chose to assess each of the six goals every academic year. However, we developed a multiyear schedule for assessment because we decided to assess only one objective for each goal in any one year. We believed this was sufficient to give us information about student achievement while keeping the workload at a manageable level. Furthermore, most of the learning goals had between two and four objectives, so we thought that a rolling schedule of assessment would give us complete information about the goal approximately every two years.

We then determined—and this was crucial—that the committee members would conduct the initial assessments. That way, we could work out any problems with the process, develop and revise rubrics, and present the faculty with a system that worked. We were very deliberate about letting the rest of the faculty know that we were “protecting” them from assessment activities, but we shared and discussed the results of our findings with them at collegewide meetings.

This strategy proved to be highly successful. By the time we were ready to roll out the assurance of learning process to the rest of the faculty, they were already on board because they had seen positive results. For instance, we were able to show measurable improvements in the ETS Major Field Test for Business, which we use to assess our general business knowledge learning goal. The results of curriculum revisions and performance incentives for students had a significant impact on subject areas where previous performance had been below expectations.

We were also able to show faculty how student performance had improved in the learning objective for written communication. One of our early assessments showed that our students needed improvement in this area—no surprise there. We invited faculty to offer potential solutions, and they suggested instituting a business communications course, increasing written assignments, and providing more detailed feedback on papers.

We encouraged faculty, when considering solutions, to use the “fatal flaw policy” developed by Kathryn Martell and shared with us at an AACSB Assessment Conference. (See “No More Fatal Flaws,” page 30.) This policy states that students’ written work must be professionally acceptable—i.e., free from errors—or it will be returned without grading for revisions and subsequent grade penalties.

The Assessment Committee spent three semesters conducting assurance of learning studies and reporting results. When we had made our final changes to the process, we rolled it out to the entire college.

Four: A Well-Defined Structure

For an assessment program to succeed, administrators must devise a system that encourages faculty to participate—and lets them know exactly what’s expected of them when they do.

At Bryant, our Assessment Committee initially consisted of seven people: the dean; five faculty members, one from each department; and an administrative assistant who had been involved in assurance of learning processes during accreditation efforts with both AACSB and the New England Association of Schools and Colleges.

When we were ready to roll out the process to the rest of the college, we added three members, recruiting them from the graduate school administration, the Graduate Faculty Advisory Committee, and the University Curriculum Committee. This helped us establish links between assessment and all aspects of curriculum management; it also ensured that the entire college would be aware of assurance of learning activities and results. At the same time, we changed the committee’s focus to strategic management of the assurance of learning process, and we renamed it the Assessment Steering Committee.

Next, we created Goal Assessment Teams (GATs), each accountable for the assessment of one learning goal. All members of the steering committee became GAT liaisons, responsible for guiding their teams’ assessment activities and reporting the results to the committee. Every GAT was composed of the liaison plus three to five faculty members.

To select potential GAT members, Dean Trifts and I met with the department chairs, who identified professors who might have an interest in a particular learning goal and professors who were too busy to participate at this time. This method resulted in a high degree of participation. Of about 35 faculty who were selected as potential GAT members, only one deferred. But five or six who were not originally chosen volunteered to participate on a team. Even some liberal arts faculty wanted to serve on GATs where they had some natural interest. For example, a math/statistics professor joined the GAT that assesses problem solving and critical thinking.

We announced the GAT structure at a collegewide faculty meeting dedicated to assurance of learning. At that time, the teams met and planned out their activities for the semester. Their directive was ambitious: to collect and report assessment data for their learning objectives within three months.

Five: Continuous Communication

The last key to a successful assurance of learning program is communication. It should be widespread; it should happen at the college, department, and individual levels; and it should follow both formal and informal paths.

Since we have begun the assessment program, every one of Bryant’s collegewide faculty meetings has included a session on assurance of learning. At first, these updates essentially consisted of status reports. After a few semesters, though, the assessment portions of the two-hour meetings had expanded to fill at least one hour. We had lively discus¬sions about assessment results and how to “close the loop” between measuring results and improving outcomes. The spring faculty meeting in May—which is solely dedicated to assurance of learning results—has become the College of Business’s most highly attended meeting of the year.

Communication also happens at the department level. From the very beginning, members of the Assessment Committee, and now the Assessment Steering Committee, have instituted formal discussions to keep their own departments apprised of developments. There are also formal communication paths between the steering committee and other relevant groups, such as the Curriculum Committee, the Graduate Faculty Advisory Committee, and the school’s departments.

Finally, on an ongoing basis, we promote communication at the individual level. When the GAT system was implemented, every faculty member in the College of Business received an Assessment Handbook, and every new hire also receives a copy. Professors have frequent discussions about assurance of learning, partly because so many of them are actively involved in performing assessment. Originally, discussions centered on efficient ways to accomplish assessment, but once results started coming in, talk turned to improving student performance. While many discussions may start within a GAT, they quickly spill over into other forums. For example, the faculty dining room is a prime location for exchanging information about pedagogical methods for closing the loop.

Overcoming Obstacles

I want to revisit the three primary objections faculty have to assessment and explain how they can be overcome if schools follow Bryant’s five keys to success.

I don’t have enough time. This objection is less convincing when the members of the assessment committees and teams are all faculty members who are also teaching, conducting research, and serving on other committees. If a professor’s colleagues manage to incorporate assessment activities into their busy schedules, he or she probably can as well. Furthermore, a structure like the GAT reduces the amount of assessment work that any individual has to do. Most GAT members spend 12 hours or less a semester on assessment activities.

I see the costs, but not the benefits. At Bryant, we made the benefits clear, while reducing the costs of time and energy. Early on, we protected faculty from the work of conducting assessment, while we shared with them the improvements we’d made when we were able to close the loop. For instance, we showed how instituting the “fatal flaw policy” can improve student performance and shorten the amount of time faculty spend grading written assignments.

We also stressed that the GAT structure reduced how much time and effort an individual had to spend performing assessment tasks. Ultimately, however, a few faculty were only motivated to get involved when we made it clear that substandard assurance of learning programs could cause us to lose AACSB accreditation.

I won’t give up my academic freedom. Early in the process at Bryant, we spent time educating faculty about what assessment is and isn’t. We specifically stated that assessment would not result in the school dictating what faculty should or shouldn’t do in the classroom.

To date, the only mandate arising from the assessment process is that all faculty must have course objectives on their syllabi and those objectives should align with some of the program learning goals. We do not specify what the objectives should be nor which learning goals they should support. We also reiterate, at virtually every faculty meeting, that assurance of learning is about evaluating degree programs, not faculty or students.

Finally, we make sure that faculty drive the activities that close the loop. Suggestions about how to improve student performance, or maintain good performance, come from the faculty either in collegewide meetings or in smaller, informal groups. Individual professors are free to adopt suggestions or not, but the involvement and enthusiasm of their colleagues is contagious. Because of the way we have implemented assessment strategies, some of the faculty who were most concerned about academic freedom have become our strongest proponents of assurance of learning.

All Aboard

There are many ways business schools can structure their assurance of learning programs. While we believe our system can work for other schools, every situation is unique and every school will need to find its own strategies. But we’re convinced that, no matter what system is implemented, faculty are more likely to get on board when they perceive that assurance of learning is beneficial to them and won’t cost them a lot of time or effort.

We believe that the five approaches we adopted can bolster any assurance of learning program that’s flagging for lack of participation. Top-level support, faculty champions, incremental improvements, precisely structured systems, and constant communication will spark enthusiasm for assurance of learning among faculty across the business school.

Carol W. DeMoranville is professor of marketing and Director of College of Business Assessment and Accreditation at Bryant University in Smithfield, Rhode Island.