Analytics in the Era of Black Swans

(A Video Series)

Black swan events are rare and of even greater magnitude than they first appear. 2020 is a banner year for this theory, with a global pandemic, global recession, and Presidential election. This combination of events has well exceeded our understanding and ability to anticipate their influences and outcomes.

So…how are your predictive enrollment models working for you?

On July 21, 2020, we invited four of our higher ed client analytic leaders to discuss how they are applying analytics to steer their institutions through these tumultuous times.

Greg Flanik, Chief Information Officer; Baldwin Wallace University (OH)
Dr. David Kowalski, Associate VP for Institutional Effectiveness and Strategic Innovation; Montgomery County Community College (PA)
Dr. Nicola Richmond, Assistant Vice Chancellor of Strategy, Analytics & Research; Pima Community College (AZ)
Dr. Nick Wagner, Executive Director of Institutional Effectiveness; Saginaw Valley State University (MI)

When asked to discuss the top three challenges that COVID-19 has presented to their institutions, they all relayed common themes:

Transitioning to virtual operations (administrative and academic)
Identifying and meeting changing student needs
Helping students find clarity in mixed messages – internally and externally to campus
Predicting the impact on enrollment and revenue

In our “Black Swan” series of videos, we will share their experiences and insights in the hopes that it will help other institutions consider how they are applying data to inform their strategies for this academic year and beyond. The series will cover the following:

Defining the “new normal”
Value of leading indicators
Data privacy concerns
Data governance
The new business model for higher education

Part I: The Year of the Asterisk

Back in March 2020, we all wondered when things would get back to “normal.” Now, we wonder what “normal” even means. The only thing we know for certain is that the landscape is ever-changing, and the impacts will be felt for years to come. Major economic players including the music and airline industries don’t anticipate being back on track until 2025. What does that mean for higher ed?

During our conversation with four clients who are analytic leaders at their institutions, everyone observed that there will likely be several iterations of the “new normal” that will have a tremendous impact on the use of data to inform decision making. This year’s data may be worth more in the short term than the long term, when it will always be accompanied by a giant asterisk. Enrollment managers are struggling to predict incoming classes by comparing this fall to previous years when registration periods opened later, students are hedging their bets longer, and colleges that planned to open are reversing course even at this late date. Additionally, it will be difficult to use this fall as a new baseline moving forward, as the pandemic will have long-lasting effects on student behavior even once the current healthcare concerns have passed.

What can analytic leaders do in the face of these unknowns? While the comparability of data may be compromised, the fundamentals of good data management and dissemination practices remain the same. Consider how the data is delivered, and how it’s interpreted and by whom. It is more critical than ever to have the right people interpreting the right data, turning it into information that leaders can use to set guideposts…and change them as needed!

The video below is the first segment in a series from a conversation ASR’s Higher Ed practice had in July 2020 with several of our client analytic leaders, about the impacts of the year’s events on their use of analytics. In this segment, you will hear the voice of moderator John Van Weeren, Principal at ASR; followed by Dr. Nicola Richmond, Chief Strategist and Vice Chancellor of Strategy, Analytics and Research at Pima Community College; and Dr. David Kowalski, Associate Vice President for Institutional Effectiveness and Strategic Innovation. Greg Flanik, Chief Information Officer at Baldwin Wallace University; and Dr. Nick Wagner, Executive Director of Institutional Effectiveness at Saginaw Valley State University, also participated in the conversation.

Part II: The Value of Leading Indicators

In our part one of the the Black Swan series, we discussed with several of our client analytic leaders the difficulty of using institutional data to make predictions in this volatile year. But don’t throw out all of your asterisked data just yet! The consensus among our clients was to use this data in the short term to develop leading indicators: data points that look forward at future events and identify what is influencing the outcomes.

Leading indicators are not necessarily predictive but can be correlated with future outcomes and reveal patterns, trends, and changes in student behavior. Though enrollment and retention data are typically the biggest focus, leading indicators are also critical in finance, human resources, and academic operations. Declining non-resident enrollment may indicate potential budget shortfalls for institutions with differential tuition rates. Changes in a three-year rolling average of students enrolled in a particular program may indicate declining popularity and an increased delivery cost per student. All of these factors impact the institution’s overall health.

Leading indicators depend on clean, trusted data that managers and leaders understand and agree on, so that the focus of the conversation is taking action, rather than discussing whose numbers are correct. To develop leading indicators, consider your top priorities for institutional stability and success. By setting goals and benchmarks based on these indicators, you can turn your data into actionable information, even in an unpredictable year.

The video below is the second segment in a series from a conversation ASR’s Higher Ed practice had in July 2020 with several of our client analytic leaders, about the impacts of the year’s events on their use of analytics. In this segment, you will hear the voice of Moderator John Van Weeren, Principal at ASR; and Greg Flanik, Chief Information Officer at Baldwin Wallace University. Dr. Nicola Richmond, Chief Strategist and Vice Chancellor of Strategy, Analytics and Research at Pima Community College; Dr. David Kowalski, Associate Vice President for Institutional Effectiveness and Strategic Innovation; and Dr. Nick Wagner, Executive Director of Institutional Effectiveness at Saginaw Valley State University, also participated in the conversation.

Part III: The Limitation of Predictive Analytics – Even in Stable Times

In Part II of our Black Swan series, we discussed how valuable leading indicators can be during periods of volatility. One of the biggest challenges for analytics during the era of the black swan involves decisions regarding the validity and usefulness of predictive models.

Over the last five to ten years, many institutions have made investments in home-grown predictive analytics capabilities and commercial software products. Institutional leaders are very interested in advanced analytics because of the potential they offer to help plan effectively – especially for impacts on revenue. However, even during more stable times, questions that involve student behavior are not easy to forecast: What applicants are most likely to enroll? What students are most likely to retain or complete? What alumni are most likely to contribute to a campaign? So many internal and yet seemingly prosaic changes can affect the accuracy of predictive models in determining outcomes including altering of financial aid award structures, varying of the timing of when applications are released, or transforming the campus visit experience. The impact of these internal pressures is magnified by external forces—a pandemic, economic instability, civil unrest—that dramatically influence one of the biggest and most expensive decisions people can make for themselves and their children: choosing a college.

As analytic leaders and practitioners it is up to us to have honest conversations with leadership about the effectiveness and reliability of these tools and techniques during such volatile times. In this segment of our conversation you will hear from Dr. Nick Wagner and Dr. David Kowalski as they discuss how they have been navigating the use of predictive analytics and providing guidance to their leadership at their respective institutions.

The video below is the third segment in a series from a conversation ASR’s Higher Ed practice had in July 2020 with several of our client analytic leaders, about the impacts of the year’s events on their use of analytics. In this segment, you will hear the voice of Moderator John Van Weeren, Principal at ASR; Dr. Nick Wagner, Executive Director of Institutional Effectiveness at Saginaw Valley State University; and Dr. David Kowalski, Associate Vice President for Institutional Effectiveness and Strategic Innovation. Dr. Nicola Richmond, Chief Strategist and Vice Chancellor of Strategy, Analytics and Research at Pima Community and Greg Flanik, Chief Information Officer at Baldwin Wallace University also participated in the conversation.