Jeff Flesher, an independent consultant, was asked to reflect on the recent webinar he and Debra Bragg conducted.
On Tuesday, April 8, Debra Bragg and I presented a webcast on Process Improvement and Evaluation, which is now available on the Pathways to Results website. The session was based on the PTR modules Process Improvement & Evaluation[1] and Evaluation and Benchmarking Module[2] along with some of reflections on previous practice and suggestions to ensure a more successful accomplishment of the process.
In each part of the PTR process, there is a need to maintain a focus on the targeted improvements in student outcomes and equity. As teams design potential solutions, opportunities exist to engage and involve key stakeholders and consider the multiple possible improvement ideas these perspectives can contribute to the team. Teams may also use internal benchmarking to identify possibilities to integrate institutionally meaningful tools (i.e., existing College planning models and processes) to leverage resources and align with institutionally meaningful practices.
The benchmarking process is actually helpful in both design and implementation of solutions and evaluation as components like accreditation standards can be helpful in both design and the measurement of outcomes. Another great opportunity for teams is to leverage the experience of previous PTR teams through OCCRL materials and direct communication. These connections can build on past success and continue to grow our collective knowledge of what leads to successful outcomes and equity improvements. Benchmarking for evaluation efforts could include survey design, using technology to capture the number of website visits, or innovative practices in embedding measures during the implementation of improvements.
Evaluation measures in PTR include both the number and duration of interventions (Activities), and a focus on the impact of those solutions on the outcome and equity improvements (Outcomes). Evaluation planning is accomplished before implementation efforts to ensure that activity measures are accurately collected and that the teams are able to get feedback for adjustments to solutions if needed. In many cases the final measurement of outcomes will take time as solutions are fully implemented and improvements realized. In these cases the PTR teams must also plan to create a sustained process of measurement that may outlive the formal team, being integrated into institutional measures and staff responsibilities.
[1] Harmon, T., Liss, L., Umbricht, M., & Flesher, J. (2012). Process improvement & evaluation (Rev. ed.). Champaign, IL: Office of Community College Research and Leadership, University of Illinois at Urbana-Champaign.
[2] Flesher, J., & Bragg, D. (2013). Evaluation and benchmarking module. Champaign, IL: Office of Community College Research and Leadership, University of Illinois at Urbana-Champaign.
Jeff Flesher is president of Prairie Fire Development a management, training and organization development consultancy providing global services to corporate, education and non-profit organizations. Dr. Flesher also holds academic appointments at the University of Illinois at Urbana-Champaign and Southern Illinois University Carbondale. His teaching has included classroom and online courses in strategic planning, technology transfer, evaluation, quality and process improvement, and organization development. Dr. Flesher has consulted with OCCRL on PTR projects, TAACCCT evaluation, and the Transformative Change Initiative.