Deploying Data to Drive Decisions

How two schools are relying on data to improve the student experience.
Deploying Data to Drive Decisions

LIKE MANY UNDERGRADUATE programs, Pitt Business has long collected data on student outcomes, educational programs, student perceptions, employer engagement, and recruitment efforts. But until recently we did not fully understand how we could leverage that data to advance our mission and contribute to knowledge within business education.

We began asking some probing questions: What is the difference between big data and “good” data? How can we make smarter decisions once we gain good data? How can data be used not only to evaluate outcomes, but also to drive program improvements? How should we communicate data outcomes to inform and engage key stakeholders?

As a result of trying to answer these and many other questions, we have learned four lessons that help us better understand educational outcomes through strategic data analytics.

Big data—or lots of data—does not mean good data.We first developed clarity on what knowledge we wanted to gain and what data would ultimately help us answer key questions. Our initial goal was to make sure that knowledge was driving data analytics, not the reverse. Thus, we held discussions with staff teams to isolate critical questions; we also emphasized the idea that any data we collected should produce knowledge that would be useful given our strategic goals. This simple first step caused us to be very intentional about data collection.

Collecting data can take less effort than understanding the data that has been collected. Once we agreed on key questions, we focused on mapping an ongoing analytics strategy and execution plan. We hired a director of operations and analytics who not only is responsible for overseeing data collection and reporting, but also serves as the chief project manager who implements knowledge gained from the data. We also made data capture a shared responsibility to complement our director’s work.

For example, one of our first efforts in our data analytics strategy was to take a deep dive into our recruitment of incoming freshmen. We had historical data on admission patterns and outcomes, but we added new data from an admitted student survey that compared students who selected Pitt Business to those who did not. The survey covered a number of factors, including demographics and personal preferences.

Based on this data, we crafted targeted messages for our desired student segments. Now we can geo-target applicants based on the locations that have historically provided the most academically qualified students. Each year, we refine what information is the most useful based on the outcomes of our recruitment. In just two years, we have raised the overall quality of our incoming class while growing our overall class size and meeting key recruitment goals, such as significantly increasing the number of business honors students.

By customizing data analytics, we can move from strategy to impact. We knew that student engagement was an important driver of student success, so we designed our own student engagement scorecard to track impact on key student outcomes.

We learned, for example, that participation in one of our business student organizations is the key predictor of a student landing a job at graduation and that students who participate in such organizations earn an extra US$2,500 in their average starting salaries compared with students who do not participate. Similar data showed us that students who study abroad or who actively engage with academic and career advising services also enjoy advantages in their starting salaries.

Additional data indicated that students who live in our freshman living-learning residence hall are more likely to show early engagement, which produces stronger academic and professional outcomes over their academic careers.

We also have collected student employment data such as the average starting salaries our students earn at specific companies or the engagement factors certain companies look for. This enables our students who are entering the job market to negotiate from positions of confidence and clarity—and it also drives their interactions with the career development office.

On the other side of the table, we can share data with our corporate recruiting partners. We are able to show them the ideal student profile based on their specific hiring histories.

Effective data analytics should balance descriptive and prescriptive objectives. Through data analytics, we not only can make yearly evaluations of success, we also can achieve long-term objectives. The effective application of analytics allows us to address student outcomes proactively, not just describe them once they occur.

To conduct these analyses, we take advantage of a variety of tools, including Tableau for interactive data visualization and SPSS for statistical analysis and validation. The business intelligence we gain from these efforts helps us shape our strategy around key issues such as student retention and on-time graduation.

For instance, schools often fail to identify students who are struggling until it is too late to intervene. At Pitt Business, we have used data analytics to develop an early-warning system. As an example, we know that if students perform poorly in certain business courses, they are more likely to experience serious academic trouble down the road or perhaps find that business is the wrong academic major for them.

Because our student engagement scorecard measures extracurricular participation, it allows us to identify students who might feel disconnected or detached from the program. Our academic and career advisors can use the data to identify potential trouble spots and suggest to students strategies for reversing negative trends. Our in-person advising and early warning systems have helped us achieve a 97 percent freshman retention rate and a 100 percent six-year graduation rate this past academic year for those admitted as business freshmen.

“The initiative allowed us to be self-reflective and taught us to ask more meaningful and relevant questions.”

WHAT’S NEXT?

While we have experienced initial success on this journey, we see some areas that need improvement. We are looking for ways to better communicate data outcomes to key stakeholders, and we have recently begun sharing information with both prospective and current students to help them choose better program options. For example, we have created data-rich infographics that we use throughout our digital and print communications. This makes data part of the culture at Pitt Business, and we hope that approach will make our students more evidence-based in their academic and career decision making.

In addition, we have begun focusing our data analytics strategy on our alumni. We recently launched an online alumni-student mentoring platform where students can search for alumni who can answer questions and give career advice. The platform gives us a new data source to indicate how our students’ engagement with alumni can impact key outcomes. It also opens up the potential for us to assess longer-term impact of our student-alumni engagement.

As we have learned these key lessons, Pitt Business has moved from merely collecting data to using data analytics to implement programs, support students, and evaluate the impact on key outcomes. We have enhanced the quality of our freshman class, improved freshman retention, and increased both on-time graduation and career outcomes. We are also seeing a positive impact on student engagement and satisfaction.

Of course, our efforts with data analytics are not without challenges. We must devote additional effort and resources to manage these initiatives in a way that is both focused and sustainable. We also must take the time to build data literacy among staff, create buy-in across academic units, and link our strategy to important university-wide initiatives. However, based on our experiences to date, the return on these efforts is well worth the investment.

Audrey J. Murrell is the associate dean of the University of Pittsburgh College of Business Administration in Pennsylvania and the director of the David Berg Center for Ethics and Leadership.