By Jonathan Meilaender
Saint Vincent went through the re-accreditation process last year. The college did not pass with flying colours--quite the contrary. It failed to receive re-accreditation. Saint Vincent was flagged in one particular area: something called “assessment.”
The purpose of accreditation is to ensure that suitable standards are maintained at institutions of higher education. Losing accreditation is a serious business: it would mean a loss of federal funding, as well as the inability for individual students to receive federal student aid. In practice, it would likely mean the closure of the college.
However, the situation probably won’t reach that extreme. According to Middle States’ website, Saint Vincent has two years to comply with accreditation standards, but even then likely wouldn’t immediately lose accreditation--the period can be extended. The main purpose of “non-compliance actions” like that taken against Saint Vincent is to get colleges back in line, not to strip accreditation, explained Julia Cavallo, Director of Assessment and Institutional Research.
“Generally, what happens is they’ll sort of keep asking you to show more and more evidence--it’s almost like they string you along [until they think it’s fixed],” she said.
But what did Saint Vincent do wrong? The college, as noted, was flagged for not adhering to Middle States’ “assessment” standards.
“Assessment,” in this context, refers to a process that attempts to quantify how effectively a college is educating its students. Accrediting institutions like Middle States want colleges to use empirical data to assess how much students are learning by measuring results against a pre-determined set of learning outcomes, and then using that data to improve institutional performance. For example, explained Dr. John Smetanka, Vice President for Academic Affairs and Academic Dean, every course has “student learning outcomes” and “course learning objectives” to measure results.
“We are requiring student learning outcomes on all syllabi. So if you’re taking a course for a philosophy major, that course has the student learning outcomes that that particular course is addressing. That’s what the student needs to know on graduation. Then the course has course learning objectives; those are what you’re going to learn by the end of the course,” Smetanka said.
Progress towards these objectives is measured by various means. According to one of the reports sent to Middle States by Saint Vincent, the business department uses a standardized exam.
“The Business department. . . has been administering the ETS Major Field Test for more than a decade to assess business competencies (SLOs) in the 45-credit business core courses. . . The department monitors percentile rankings from ETS longitudinally to prioritize areas for improvement, particularly in subscales of the ETS exam,” the report says.
An important element of all this is actually to do something with the results. That is, assessment is supposed to drive continual improvement in how courses are taught.
“I’m an alum. If a course that I took twenty years ago is being taught the same exact way for students who have changed, I think that’s a problem. An assessment can help you tweak and refine and continuously improve courses and pedagogy and curriculum,” Cavallo said.
This, in a way, was the main problem Middle States had with Saint Vincent: not that assessment was nonexistent, but that it wasn’t being used in a way that enabled the college as a whole to make positive changes to the curriculum, Smetanka explained. Control of assessment procedures was delegated to individual departments. That meant information wasn’t adequately shared between departments, making substantive change more difficult. Essentially, Middle States wanted to know, “Do we have an assessment process for our assessment process?” as Smetanka put it.
Saint Vincent is well on the way back to Middle States’ good graces. A “monitoring report,” detailing all the changes SVC has already made, will be submitted to the accreditor on March 1 and is available on the portal. (The quote above was taken from that report.)
The whole issue, though, is more complex than merely meeting Middle States requirements. Assessment itself is a somewhat controversial topic in higher education. It’s easy to find articles online complaining about the process, and a professor from another college, who preferred to remain anonymous, actually told The Review that he considers assessment to be “deeply problematic” and “entirely useless.”
Why would someone object so strongly to a process that’s supposed to improve student learning? The Review spoke to Dr. George Leiner, associate professor of philosophy, to get a better understanding of the potential pitfalls of the process.
Leiner does not object to assessment per se. Indeed, he said, “I think assessment is fundamental. It’s essential. It’s important that you step back and see how you’ve done. And I think that Saint Vincent has had a good assessment strategy in the thirty years I’ve been here.”
What concerns him is the increased emphasis on
assessment on the part of accrediting agencies (and especially Middle States), an emphasis Smetanka and Cavallo affirmed. (“They’ve ratcheted up expectations,” Smetanka said.)
“What’s happened is, the broader culture has lost faith in the ability of professional educators to be competent in what they’re doing. I see it as a lack of confidence in legitimate expertise. The primary accrediting bodies have felt that skepticism on the part of the public and government,” he said.
In response to this skepticism, Leiner suggested, the accrediting agencies have turned to a flawed model--one that is “very narrowly empirical and very capable of being quantified,” and hence does not really measure all aspects of what students learn. Leiner thinks that a liberal arts question is about more than merely fulfilling “learning outcomes.” Instead, he wants to enrich students’ lives--to teach them to think.
“How do you get a definitive demonstration of a person’s action to show that they are leading an examined life? The ultimate test, as far as I’m concerned, of our success at Saint Vincent, is what are our students’ lives a decade from now, twenty years from now, forty years from now? It’s good to have the empirical, quantifiable things too, but I think that the accrediting body is using a different epistemology than Saint Vincent traditionally has.”
There’s another question worth asking: is assessment worth all the work? Clearly, Saint Vincent has had to work hard to swiftly bring its assessment procedures up to par, and clearly significant resources have been devoted to the task. Leiner said, “I’m not convinced, from my own perspective, that spending as much time and energy garnering that information is as useful as other things we could be doing,” but he was not willing to suggest that assessment is simply a waste of time.
“Certainly change is hard, and we’re making a fundamental shift by centralizing assessment,” Cavallo said. Right now, she explained, the work seems harder than usual because Saint Vincent has such a short timeline to catch up. “But my hope is that this really gives us the opportunity to create these mechanisms for continuous improvement,” she added.
No one the Review spoke to would quite suggest that Saint Vincent’s warning is a good thing. But both Leiner and Cavallo, despite having different opinions of assessment’s value, think that the college’s efforts to meet Middle State’s requirements can be an opportunity.
Leiner, though disappointed in the cultural pressure to measure education empirically, thinks that the best response higher education can give is a good-faith assessment effort aimed at regaining public confidence.
“It’s important for responding to the demands of the broader culture in good faith. While we may not be convinced that it’s the most efficacious way to proceed, it’s important to respond to sincere concerns,” he said.
Cavallo approaches assessment from a different standpoint. But in a way, she agrees with Leiner. Yes, she says, there is a societal distrust of higher education. That might not be ideal--but her job is to make the best of it.
“I will tell you that I think there is a lack of trust in higher education right now,” she said. “I think that accreditors are getting a lot of pressure from the government and also from the public. . . All of those pressures make this enterprise very complex.”
There are two ways to approach assessment, she suggested: either as a matter of compliance--simply getting the paperwork out of the way--or “as a mechanism for continuous improvement.” That’s what she wants to to do.
“This nine-month period [since SVC was found in non-compliance] has been stressful and a big change for a lot of people, I really think it provides us an opportunity to get better as an institution.”
Is failure a good thing? Who knows. But Saint Vincent is at least trying to make sure that it is.