This post is part of a new blog series called Transformative Change Initiative (TCI) Featured Evaluator, that includes interviews with members of TCI’s Evaluation Collaborative. This community of evaluators has a wealth of knowledge, experience and insights into evaluation of the TAACCCT grants that are being implementation throughout the United States. Want to be profiled or know someone who would make a great feature? Email us at occrl@illinois.edu.
Name: Marisa Castellano
Current position: Consultant, OCCRL, Transformative Change Initiative
Short bio: Dr. Marisa Castellano joined the OCCRL evaluation team in 2014 and assisted in the evaluation of a multisite TAACCCT consortium grant. Dr. Castellano received her PhD from the University of California, Berkeley. She has 17 years’ experience leading research and policy studies of secondary and postsecondary career pathway efforts, first for the Center for Social Organization of Schools at Johns Hopkins University and then for the National Research Center for Career and Technical Education at the University of Louisville. This work has included longitudinal studies of student achievement in secondary career pathway programs, a systematic literature review on educational transitions, and examinations of retention interventions for nontraditional community college students and promising practices in career pathway programs for low-skilled adults.
Questions.
Q. What is the design and predominant methods for your TAACCCT evaluation?
A. The TAACCCT evaluation I have been involved with is mixed method: the team has been collecting qualitative and quantitative data since the inception of this Round One grant. Prior to site visits, grant recipients responded to a self-assessment tool prompting them to provide information on the implementation progress of various TAACCCT activities along with challenges they have faced. Site visits have included interviews with relevant stakeholders at colleges, workplaces, and workforce agencies. All of these data have helped the team understand the student outcome data.
Q. What advice do you have for new TAACCCT evaluators?
A. My evaluation mentor once gave me these words to live by: get the facts and then explain them accurately and succinctly. The rest of these comments are relevant to evaluators who join an existing team that has already collected data: Do your homework prior to the first site visit so that you know who the stakeholders are and their positions. Read all materials produced by the team and acquaint yourself with the college and program webpages. You have some catching up to do!
Q. What questions do you have for others about TAACCCT evaluation?
A. I think the big question is, as the funding Rounds come to an end, what activities will be sustained, scaled, and why?
Q. Do you have tools, reports or other products you are willing to share?
A. We have a great opportunity to drill down and learn more about the student experience in this TAACCCT evaluation of a health professions consortium. Approximately 60 students across 9 community colleges were interviewed; these audiotapes were transcribed and we are currently analyzing them to capture the student voice: What led them to the program? Was it useful? Did they interact with their success/retention coach, and to what effect? Finally, what are their next steps? We plan to adopt a framework that puts identity development and adult transitions at the forefront of understanding these participants’ experiences. We hope to complete this work in early 2016.
Marisa can be reached at mcastell@illinois.edu