Taking Learning Outcomes to the Gym: An Assignment-Based Approach to Developing and Assessing Learning Outcomes
EXECUTIVE SUMMARY: While there has been great interest and progress in terms of defining core learning outcomes related to the completion of various postsecondary programs, there has been far less progress in terms of elucidating powerful ways to assess these outcomes. Without clear assessment methods it is difficult to see how one could perform course or program redesign with these learning objectives in mind.
To date, attempts to address this “assessment of learning outcomes” gap have focused mostly on the use of qualitative tools that are given to students as they leave some institution or program, tools like the National Survey of Student Engagement (NSSE) questionnaire or the Collegiate Learning Assessment (CLA) index. But these tools rely on subjective report, are often difficult to administer repeatedly at scale, and typically only provide an “after education” snapshot of learning. This report highlights and validates a different assignment-based approach that has much greater potential to provide quantitative data with much higher resolution.
This assignment-based approach is illustrated via a muscle-building analogy to represent the preferred goal: the ability to simultaneously develop and assess skills on an assignment level. That is, when you pump as much weight as you can in the gym, you are simultaneously building the muscles needed to pump that weight and you are providing a clear measure of how strong those muscles currently are. Similarly, our most desired learning outcomes all have basic cognitive skills underlying them, skills that develop through repeated effective practice. If we could thus develop technologies that could simultaneously exercise and assess these skills quantitatively, then we could track their development on an assignment by assignment basis, providing a much more accurate index of the attainment of the learning outcomes associated with those skills.
The primary purpose of this report is to provide a concrete example of how an assignment-based approach can be instituted and to provide some initial data supporting the validity of exercising and assessing learning outcomes in this manner. We first describe briefly some of the core learning objectives virtually all educators see as critical, including critical thought, creative thought, self-reflective thought, and effective communication in both its receptive and expressive forms. We then draw on the example of a specific learning technology, peerScholar, to demonstrate that this assignment-based approach to assessing learning outcomes is indeed possible even within the current constraints of the higher education system. This potential is then demonstrated via two experiments, one that illustrates the development of self-reflective thought and another that illustrates the development of critical thought.
We conclude that the example we highlight with peerScholar could be extended to other learning technologies and, especially if learning technologies were built with the assessment of learning objectives in mind, we could ultimately create a toolbox that would allow us to simultaneously build and assess our progress as educators in an information-rich and powerful manner. To some extent, the entire value of the specification of learning objectives hinges on our ability to assess their development well, and it is our contention that educators should be looking towards assignment-based approaches to fill this need.