Tracking & Evaluation
The purpose of the Tracking & Evaluation program is to provide tracking and evaluation for the ACTSI and to document the success of its activities. Tracking & Evaluation uses evaluation data to provide evidence that the ACTSI reaches its specific aims and mission to facilitate, enhance, and transform the conduct of clinical and translational research in the metropolitan Atlanta area and to impact the health of the community in Georgia. Each core program of the ACTSI has stated aims and plans for accomplishing those aims. Tracking & Evaluation uses this information to assist in the development of metrics and benchmarks related to each of the aims, continuously monitor the metrics, and evaluate the progress of the overall aims of each ACTSI program. This information is then documented in the form of progress summaries and annual reports and provided to the ACTSI Executive Council, the ACTSI Leadership Council, and governance to share findings on ACTSI¿s progress and provide continuous quality improvement. Regular progress and outcomes meetings are used to discuss tracking of the ACTSI activities and review evaluation data to assist with decision making, recommendations, and modifications for activities going forward.
Background History of Tracking and Evaluation
The ACTSI has had existing program evaluation of the training programs (former K30 and K12, current KL2 and TL1) at Emory, Georgia Tech, and Morehouse School of Medicine (MSM) and former Clinical Research Centers at Emory and MSM. For the training programs, evaluation methods included: course evaluations, feedback from mentors for mentored scientists or students, feedback from students on their mentors, focused curricular discussions with students, and tracking of their career development in the field of clinical and translational research through academic appointments, publications, presentations, and extramural funding awards. Tracking & Evaluation continues to build on past evaluation methods and metrics as well as build new evaluation tools to assess the effects of the specific aims of the ACTSI.
The Tracking & Evaluation program has been innovative due to the unique structure of the ACTSI. First, the evaluation plan of Tracking & Evaluation requires comprehensive and synergistic data collection from all of the ACTSI core programs. A major challenge in the current funding period was the coordination of program data across ACTSI member institutions. As a CTSA organization, knowledge gained from challenges in these efforts will benefit other CTSA organizations as well as the National CTSA Consortium. The process for the proposed funding period will involve enhancing existing data collection systems, building new data information systems and surveys, and the continued coordination of reporting of data across core programs and institutions.
Second, the evaluation employs mixed methodologies of quantitative (metrics/indicators) and qualitative data collection procedures. Tracking & Evaluation has used social network analysis, web data extraction, and web applications in conjunction with traditional techniques of surveys, interviews, and secondary data collection to perform evaluation tasks. Building on this, the use of a variety of methods will produce a comprehensive, data-rich assessment of the transformative home for clinical and translational science in the metropolitan Atlanta area.
Third, the evaluation applies a stakeholder-evaluation approach to identify and elicit evaluation feedback from the ACTSI stakeholders, including trainees, core directors, university health sciences administration, collaborating universities and partners, and the community at large. The use of stakeholder evaluation will allow for involvement of key people at all levels and provide multiple perspectives of the clinical and translational research enterprise in the Atlanta region.
Lastly, the evaluation is integrative in that it will document ACTSI activities across the programs among participating institutions and partnerships. A unique challenge to the ACTSI is the involvement of multiple institutions and partnerships, which requires creativity in creating a collaborative environment that is able to benefit all parties involved. Prior to the ACTSI, many of the institutions and organizations that became part of the grant functioned independently. The formation of the ACTSI made it necessary for the cultures of each of the organizations to adapt to one another to function symbiotically. To properly take this into account, the evaluation plan must allow the various institutions to represent themselves in a manner where all of the institutions are on equal ground.Tracking & Evaluation is integral to monitoring program implementation and assessing the progress of the ACTSI in pursuing goals of focused education and training, innovative support of discovery, and ethical community engagement. In doing so, the collaborative partners of the Atlanta Clinical & Translational Science Institute rapidly and efficiently translate scientific discoveries to impact all populations of the Atlanta community.