This post is part of a new blog series called Transformative Change Initiative (TCI) Featured Evaluator, that includes interviews with members of TCI’s Evaluation Collaborative. This community of evaluators has a wealth of knowledge, experience and insights into evaluation of the TAACCCT grants that are being implementation throughout the United States. Want to be profiled or know someone who would make a great feature? Email us at occrl@illinois.edu.
Name: Tara Donahue
Current position: Managing Evaluator
Bio: Tara Donahue, Ph.D., managing evaluator, performs a variety of functions at McREL International. She conceptualizes, manages, and conducts research and evaluation projects on several different education-related programs. Examples of her work at McREL include managing the external evaluation of an Investing in Innovation (i3) grant focused on improving college and career readiness in six high schools in Jefferson County Public Schools (Louisville, KY), overseeing two Teacher Quality Partnership evaluation projects in Kentucky and Virginia, and leading the evaluation of two Trade Adjustment Assistance Community College and Career Training (TAACCCT) grants in Missouri and Tennessee. Additionally, Tara leads proposal writing teams on content development, evaluation design, and budgets for submission to a wide variety of clients and funding sources, such as the U.S. Department of Education, the U.S. Department of Labor, and the National Science Foundation. Tara has had papers accepted and has presented at the American Evaluation Association and American Education Research Association, where her team won a Publication Award from Division H in 2014. Tara holds a B.A. in English and psychology from Vanderbilt University and a Ph.D. in educational policy from Michigan State University.
Q. What is the design and predominant methods for your TAACCCT evaluation?
A. The TAACCCT evaluations use a quasi-experimental design, with comparison groups coming from both other institutions and historical cohorts. Predominant methods include conducting interviews and administering surveys with a variety of stakeholders, including project staff, partners, and students. Outcome data, supplied by the colleges, are used to help the colleges meet Department of Labor reporting requirements and to address other evaluation questions the colleges may want to address through their project. TAACCCT evaluations also have an intensive fidelity of implementation project to help them gauge the extent to which they have implemented the project according to their plan. When changes are made, the evaluation team documents how and why.
Q. What stands out as major accomplishments so far?
A. Although the evaluations are in the early stages, there have been several accomplishment thus far. First, great conversations have begun to determine what critical evaluation questions need to be asked and how those questions will be answered. Second, partnerships with the data coordinators at the colleges have been established to facilitate transmission of the school’s data to the evaluation team. Third, strong relationships between the evaluation team and the project team have encouraged brainstorming and flexibility in approaching survey administration from a variety of ways to increase response rates.
Q. What advice do you have for new TAACCCT evaluators?
A. Major accomplishments thus far are based on key communication strategies. Start as early in the process as possible to make sure clear lines of communication are established. Be clear and concise in what is being communicated. For many project staff, this is the first time that they have worked with an external evaluator and expectations need to be made clear at the beginning.
When meeting with project staff and stakeholders, make sure to clearly explain the purpose of the evaluation and why it is necessary. Focus on continuous improvement—a primary objective of the evaluation is to examine project strengths and weaknesses and to help project staff implement strategies to shore up any areas that may not be as strong. Again, because this may be a new process for some staff, they may need to go slowly to fully understand the process and the intention behind it.
Q. What questions do you have for others about TAACCCT evaluation?
A. What strategies have been implemented to increase survey response rates for students? What challenges have you faced in developing relationships with workforce partners and having access to state employment databases? How have those challenges been resolved?
You can reach Tara at tdonahue@mcrel.org and learn more about McRel International at www.mcrel.org.