Development of a Telephone-Based Clinical Skills Assessment Tool

DESCRIPTION (provided by applicant): There is a clear and accelerating trend toward making trainings available to alcohol and drug abuse treatment clinicians and clinical staff via the internet and distance learning methods (e.g., Weingardt, Cucciare, Bellotti, & Lai, 2009). This movement is cost effective and offers tremendous possibilities for dissemination of evidence-based practices to audiences for which access had been previously limited or unavailable. However, these benefits come with a cost, especially with regard to clinical skill evaluation. These dissemination efforts are moving faster than the development of tools to assess the clinical skills of trainees who participate in these technology-based training efforts. Role play is a critical method by which clinicians can practice a clinical intervention, receive feedback, and monitor application of that feedback to their clinical practice (e.g., Bosse et al., 2010). Remote and asynchronous training pose significant challenges when conducting role plays or similar unrehearsed assessments of clinical skill. While some methods are available, such as video conferencing, these can often rely on expensive equipment and technology, and may be impractical for most field clinicians to use. This project proposes to build on existing data collected as a part of a previously funded R01 award that examined different kinds of post-training support for alcohol and drug abuse clinicians (Barrick & Homish, 2011). Using a Generalizability Theory-based approach (e.g., McGillicuddy, Rychtarik, & Morsheimer, 2004; Cronbach, Gleser, Nanda, & Rajaratnam, 1972) we will develop an alternate form of the telephone-based clinical skill assessment tool (T-CAT) and conduct a psychometric evaluation to examine the generalizability, alternate form reliability, and construct validity of the T-CAT. This method employs a telephone-based approach using interactive voice recording (IVR). Clinicians call a toll-free telephone number, listen to a series of clinical scenarios, and speak their responses to the situations. The responses are recorded using IVR, then coded for clinical fidelity and skill, and rated by the research team. This approach has promise as a method to allow for clinical skill assessment for distance learning and web-based trainings in the alcohol and drug abuse treatment fields.