Assessing to Learn and Learning to Assess: Examining Assessment Fidelity for the Parent-Child Interaction Coding System

Abstract

Fidelity to treatment protocol has been found to predict treatment outcome (Schoenwald, Sheidowm & Chapman, 2009; Henggeler, Schoenwald, Liao, Letourneau, & Edwards, 2002) and is especially important to evaluate within the context of PCIT dissemination; however few studies have addressed assessment fidelity and its role in empirically based practice. Thirty-four DPICS observations were submitted to the Auburn University Parent-Child Lab by PCIT trainees and evaluated using a DPICS fidelity checklist based on the DPICS manual (Eyberg et al., 2005). A fidelity check was completed for 2 categories: room set-up and DPICS procedure. In addition, twenty-nine videos were coded to assess coding accuracy. A multi-level regression was conducted to explore whether fidelity to assessment protocol is predictive of coding accuracy. Other variables examined included the predictive relationship of latency between training and case completion and therapist characteristics predictive of fidelity. Analyses reveal differential implementation of room set-up and procedure to the DPICS assessment. The multi-level regression failed to support the hypothesis that room set-up and procedural fidelity would predict coding accuracy as well as the hypothesis that latency between training and case would predict fidelity. No relationship was found between specific therapist characteristics of degree type and therapist self-rating, suggesting that these therapist variables may be less important for the dissemination of an EBP. A significant relationship was found, however, between the number of client hours per week and room set-up fidelity, suggesting that therapists are focusing on maintaining the active ingredients of the DPICS assessment. Specific changes to the manual and supervision process are suggested. Simple changes to the manual could facilitate therapist learning, and enhanced attention to the DPICS assessment during supervision could improve fidelity and the level at which the DPICS is implemented in the community. The importance of accurate assessment cannot be over-emphasized, given that PCIT is an assessment-driven intervention.