All Systems Go

ANY GOOD LEARNING management system (LMS) must achieve multiple goals. It must add value to the performance of instructors and administrators, integrate with other widely used applications, and introduce students to the types of technology they might encounter in the workplace.

While Cornell University had been using the Blackboard LMS since 2001, we wanted to see whether a different system might better suit our needs. In 2017, we decided to evaluate how Brightspace, Canvas, and Blackboard Ultra might work for us going forward.

I was a member of the Academic LMS Evaluation Committee that was tasked with choosing the system that would be installed for the entire university. Committee members focused on these key features as we discussed the evaluation process:

Training systems. Training reduces time to competence and the frustration factor for new users. If products did not include this feature, we conducted cost-benefit analyses of implementing such systems.

Transferable skills. We examined whether the technological skills and integrations of a given LMS would be useful for the careers of the students once they graduated. We also considered whether the LMS setup indicated a commitment to keeping up with the changing technological patterns of global business.

Ease of use. We wanted an LMS that would be easy, intuitive, and convenient for students and faculty at various levels of computer literacy.

Customization. We wanted a truly customizable LMS that allowed us to modify the appearance of the interface where end users access information, training materials, and tests.

Integration. We looked for LMS software that would easily integrate with other systems that were part of the learning infrastructure, as well as enterprise resource planning, human resources, payroll, and registration systems.

Support. We considered how responsive the LMS vendors were at the initial inquiry stage, as well as which companies provided onboarding support in addition to ongoing on-site and online training options.

Organizational structure. Finally, we considered which platforms would enable our systems manager to see all learner and instructor activity.

To assess the options of different platforms, the evaluation committee gathered feedback from stakeholders and asked faculty members to test various LMS offerings. As one of the testers, I shifted in one of my courses from using Blackboard Learn, an established platform in use since the late 1990s, to Canvas, which launched in 2011.

After reviewing basic information, I shared with the evaluation committee my own observations, as well as feedback from my students and teaching assistant. All of us noted how each of the systems we were evaluating performed on key measures such as tracking functions, calendar functions, and ease of use. We also described what we particularly liked about each system, including specific applications that they offered and how they interacted with outside providers such as Dropbox or TED-Ed. In addition, we noted the features we found frustrating or difficult to master.

Our evaluation of the LMS systems took a little over a year and involved testing in 32 courses. We believe that any university could benefit from undergoing a similar evaluation, because it gives faculty the chance to step back and answer core questions about why operations are conducted the way they are. For us, an additional benefit is that the process we used to select our LMS has taught students to appreciate the strategic decisions they must make when they choose the systems that they will use throughout their careers.