While improvement should be the intent behind assessment practices in higher education, the subsequent assessment of the implemented improvements resulting from assessment data is often lacking or missing altogether. This leads to a gap in the assessment of whether the improvements are functioning as intended. In response, the Assessment Institute introduced the learning improvement and innovation track in 2018 aimed at supporting higher education to administer and assess learning improvement efforts. The track offers three strategies for implementing "improvement-focused" assessment.
Strategy 1: Adopt a Learning Improvement Heuristic
One of the most significant obstacles to improving student learning is that we often emphasize the "what"- assessment- over the "why"- learning improvement. This backward prioritization can be seen in assessment reports that highlight assessment practices but rarely discuss the interventions taken from one year to the next. The resulting reports make it impossible to analyze the previous or subsequent improvements' success.
In traditional assessment reports, learning improvement information is often stifled and fragmented. A learning improvement heuristic foregrounds improvement, ensuring that the assessment process is designed to clearly outline whether improvement resulted from program changes.
Strategy 2: Develop and Disseminate Technical Practices That Enhance Improvement Efforts
Analyzing learning improvement consistently throughout multiple programs with dozens of faculty and hundreds of students can be a challenging undertaking at any institution. The Assessment Institute provides three techniques for enhancing learning improvement efforts throughout the institution.
Program Theory
Improving a program requires an understanding of how said program works. Program theories illustrate the anticipated correlation between program components and their intended outcomes and provide factual support for instructional and curricular decisions. Good program theory consistent across an institution is necessary for good assessment practices that lead to learning improvement.
Implementation Fidelity
Implementation fidelity is a tool that can be used to determine if, and how, the program delivered to students (the implemented program) deviates from the designed program (the intended program). This process is aided by the program theories mentioned above.
Implementation fidelity requires program observation. Subsequently, observational data is used to determine the level of adherence to the program design. Adherence data includes whether critical program components actually occurred, how well the components were implemented (and for how long), and whether students were responsive.
Implementation fidelity is a crucial technique for enhancing learning improvement, as it provides important data about whether or not the planned program intervention was delivered as intended. This information is critical when assessing whether an intervention was successful in improving learning, and it affects the subsequent decisions made about the program. Should an investment be made to rebuild the program, or should the focus be shifted to developing instructors?
Measurement
Measurement allows us to see whether improvement has occurred. But, it's not enough to measure; we must measure what matters to get meaningful information. A strong alignment between the measure and the student learning outcome is the most important aspect of sound measurement. There are two ways that misalignment between measures and student learning outcomes can occur. The first is when scores are regularly influenced by factors other than the student learning outcome. The Assessment Institute refers to this as "noise" that obscures actual changes in student learning. The second is when a measure lacks key components of the student learning outcomes. In this case, real increases in students' learning may not be recorded because the test prevents students from fully demonstrating their learning. To ensure that we accurately measure the impact of the improvement intervention on student learning, we must create measures that are strongly aligned with student learning outcomes.
Strategy 3: Attend to Social Dynamics
Simply put, improvement of student learning is impossible without faculty. Institutions must find ways to facilitate this faculty work and provide professional development opportunities that give faculty the skills and time they need to strategize collaboratively.
The Assessment Institute is doing great work advancing the theory and practice of assessment in higher education. This post highlights but one initiative from the Assessment Institute. If you are interested in learning more, I highly suggest you visit their website here, or purchase Trends in Assessment here.