After recently attending the Transformative Change Initiative’s Learning Lab, Sarah Stewart, a Trade Adjustment Assistance Community College and Career Training (TAACCCT) grant program evaluator, reflects on evaluation in the process of scaling up.
This may be a statement of the obvious, but TAACCCT-funded programs are complex, with a multitude of components in flux, program staff with varying jobs, and plenty of U.S. Department of Labor (DOL) requirements to adhere to. It’s not always easy to define the role of the evaluator within these programs, particularly when it comes to sustainability and scaling up. How do we help grantees take their innovative programs to the next level?
First, as an evaluator, we should occasionally take a step back and look at data through the lens of sustainability and scaling. The key is to not step too far back—we need to be looking at program components as opposed to the program as a whole. Scaling or sustaining the entire program intact is probably unrealistic, but if you focus on the individual parts, there can be ample opportunities for maintaining and spreading the hard work of the program staff. Find those things that work well and think outside the box—show your client what is working and offer ideas on how to share those things beyond the expected scope. Don’t just focus on spreading to other schools; some components may be better suited to stay within the school, but in a different department or in another program. Be creative.
To put this into practice, pose questions during the evaluation like: What pieces are working well—would they work well in other settings? Examine components like sources of funding, credentialing, curriculum, workshops, professional development, mission, etc. Do program components have the potential for institutionalization?
Last, don’t forget your supporting role as a “reminder.” Pepper questions about sustainability and scaling into program staff interviews and surveys. It may not magically scale up the program, but it encourages grantees to ponder it.
While an evaluator’s main job is to observe progress and impact from the outside, the findings can do more than simply populate a compliance report to DOL. They can serve to help these TAACCCT programs forge ahead and bring the best parts of their programs to the forefront, setting them up for successful sustainability and scale up.
Sarah Stewart is a senior research analyst at Hezel Associates, LLC, a research and evaluation firm in Syracuse, NY. The company is currently evaluating five TAACCCT projects, including the National STEM Consortium and the SUNY T.E.A.M. project.