THE EFFECTIVENESS OF GROUP PARENT-CHILD INTERACTION THERAPY (PCIT) WITH COMMUNITY FAMILIES Except where reference is made to the work of others, the work described in this dissertation is my own or was done in collaboration with my advisory committee. This dissertation does not include proprietary or classified information. _____________________________ Lorraine Elizabeth Ridgeway Certificate of Approval: _____________________________ ______________________________ Steven K. Shapiro Elizabeth Brestan Knight, Chair Associate Professor Associate Professor Clinical Psychology Clinical Psychology ______________________________ ______________________________ Jennifer Gillis Adrian Thomas Assistant Professor Assistant Professor Clinical Psychology Industrial/Organizational Psychology ______________________________ George T. Flowers Interim Dean Graduate School THE EFFECTIVENESS OF GROUP PARENT-CHILD INTERACTION THERAPY (PCIT) WITH COMMUNITY FAMILIES Lorraine Elizabeth Ridgeway A Dissertation Submitted to the Graduate Faculty of Auburn University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Auburn, Alabama December 17, 2007 iii THE EFFECTIVENESS OF GROUP PARENT-CHILD INTERACTION THERAPY (PCIT) WITH COMMUNITY FAMILIES Lorraine Elizabeth Ridgeway Permission is granted to Auburn University to make copies of this dissertation at its discretion, upon request of individuals or institutions and at their expense. The author reserves all publication rights. ________________________ Signature of Author ________________________ Date of Graduation iv VITA Lorraine E. Ridgeway is the daughter of Michael D. Ridgeway of Greenwood, South Carolina, and Victoria R. Gillis, of Anderson, South Carolina. Born on October 13, 1975 in Orlando, Florida, Lorraine completed her B.S. at the College of Charleston in 1998, graduating Magna cum Laude. She entered the clinical psychology doctoral program at Auburn University in 2001, where she received her M.S. in Clinical Psychology in 2005. v DISSERTATION ABSTRACT THE EFFECTIVENESS OF GROUP PARENT-CHILD INTERACTION THERAPY (PCIT) WITH COMMUNITY FAMILIES Lorraine Elizabeth Ridgeway Doctor of Philosophy, December 17, 2007 (M.S., Auburn Univsersity, 2005) (B.S., College of Charleston, 1998) 132 Typed Pages Directed by Elizabeth Brestan Knight Parent-Child Interaction Therapy (PCIT) is a behavioral intervention that incorporates both operant learning and play therapy techniques to treat child disruptive behavior problems. Efficacy research indicates that PCIT reduces parent reports of children?s disruptive behavior, parenting stress, and maternal psychosocial distress. In addition, PCIT has been shown to increase observed prosocial behaviors and to decrease observed inappropriate behaviors for both parents and children. Recently, researchers have begun to investigate the use of PCIT with alternative populations and in alternative formats. However, to date, little systematic research exists examining the effectiveness vi and transportability of PCIT, especially as it relates to the group format. The proposed study sought to examine the potential effectiveness and transportability of group PCIT with a sample of community families from Lee and Macon Counties in Alabama. The hypothesis that caregivers would report significantly fewer child behavior problems and significantly less parenting stress after completing treatment was supported. The hypothesis that caregivers and children would engage in more prosocial and fewer negative behaviors after completing treatment was not supported. The hypothesis that parenting stress would predict poor treatment response and retention was not supported. Implications of the present study as well as directions for future research are discussed. vii ACKNOWLEDGMENTS There are a number of people I would like to thank for their help and support in making this project possible. First, I would like to thank Dr. Elizabeth Brestan Knight and committee members, Dr. Steven Shapiro, Dr. Jennifer Gillis, Dr. Adrian Thomas, and Dr. Francesca Adler-Baeder, for their assistance during the completion of this study. I would also like to extend my appreciation to my coders, Pamela Dolbear, Alicia Luscomb, Daron Flory, Jaci Cremisi, Rachel Harris, Shalonda Jones, and Christie Salamone, for their time and expertise. I am grateful to Lori Klinger, Ember Lee, and Meena Lambha for their endless (and usually unpaid) hours conducting groups and taking care of children, and to Emilyn Gipson and the staff of the East Alabama Child Advocacy Center for grant funding, facilities, and programmatic support (not to mention dinner for our families!). Thanks are also due to Mei Jang for her development of Kappa IV and her assistance with trouble-shooting and to the many undergraduate research assistants who helped with data collection, scoring, and entry. I would like to thank my family and friends for all of their support and encouragement, and for keeping me sane. In particular, I am eternally grateful to my mother, Vicki Gillis, and sister, Gentry Ridgeway, for inspiring me to strive to be a better person, for showing me infinite and unconditional love, and for always believing in me even when I didn?t believe in myself. Words cannot express how blessed I am to have them as both my family and my friends. viii Style manual or journal used APA Publication Manual (5 th edition) Computer software used Microsoft Word; Kappa IV; SPSS ix TABLE OF CONTENTS LIST OF TABLES............................................................................................................ xii LIST OF FIGURES ......................................................................................................... xiii I. INTRODUCTION ...................................................................................................1 Parent-Child Interaction Therapy: Development & Theory ........................1 What Does PCIT Look Like? ......................................................................3 PCIT: Efficacy Research..............................................................................5 PCIT: Adaptations to the Original Model....................................................6 Parent Training & PCIT in a Group Format ................................................9 Transportability and Effectiveness Research.............................................12 Rationale for Present Study .......................................................................14 Hypotheses.................................................................................................15 II. METHOD ..............................................................................................................19 Participants.................................................................................................19 Study Measures..........................................................................................20 DPICS-II Coding & Transcribing..............................................................29 Procedures: Project Development & General Design................................35 Procedures: Intake Assessment..................................................................36 Procedures: Group Sessions.......................................................................36 x Procedures: Follow-up Assessments..........................................................41 Data Analyses ............................................................................................42 III. RESULTS ..............................................................................................................44 Preliminary Analyses.................................................................................44 Regression Analysis for ECBI Scores over Time......................................49 Regression Analyses ? Parent Stress as Predictor .....................................51 Repeated Measures MANOVA ? Caregiver Reports ................................54 Repeated Measures MANOVA ? DPICS-II Observations........................56 Treatment Satisfaction Analysis ................................................................59 Reliable Change Index Analysis................................................................60 IV. DISCUSSION........................................................................................................63 Differences between Treatment Completers and Non-Completers ...........63 Demographic Characteristics and Treatment Outcomes............................64 Changes in Child Behavior Problems across Sessions ..............................66 Parenting Stress as a Predictor of Treatment Response and Retention .....68 Treatment Outcomes for Caregiver-Reported Behavior............................72 Treatment Outcomes for Observed Caregiver and Child Behavior...........73 Treatment Satisfaction ...............................................................................79 Reliability of Treatment Change................................................................80 Limitations of Study Design and Directions for Future Research.............81 Unique Contributions and Implications of Present Study..........................86 REFERENCES ..................................................................................................................89 xi APPENDIX......................................................................................................................102 xii LIST OF TABLES 1. Demographic Characteristics for Participants ...........................................................102 2. List of the DPICS-II Parent and Child Behavior Categories .....................................105 3. Descriptions of the DPICS-II Categories...................................................................107 4. List of the DPICS-II Composite Categories ..............................................................110 5. List of Session Content and Assessment Measures ...................................................112 6. Correlations of Demographics with Pre-Treatment DPICS-II Composites.................48 7. Correlations of Demographics with Post-Treatment DPICS-II Composites ...............48 8. Regression Analysis for Session Number as Predictor of ECBI Scores......................51 9. Regression Analysis for PSI Total Stress as Predictor of ECBI Scores ......................53 10. Repeated Measures MANOVA for Caregiver Self-Report Measures.........................56 11. Reliable Change Indices for Caregiver Self-Report Measures....................................62 xiii LIST OF FIGURES 1. ECBI Intensity Scores across Time for Treatment Non-Completers.........................114 2. ECBI Intensity Scores across Time for Treatment Completers.................................115 3. ECBI Intensity Scores across Time for All Participants............................................116 4. ECBI Problem Scores across Time for Treatment Non-Completers.........................117 5. ECBI Problem Scores across Time for Treatment Non-Completers.........................118 6. ECBI Problem Scores across Time for All Participants ............................................119 1 INTRODUCTION Parent-Child Interaction Therapy: Development and Theory Parent-Child Interaction Therapy (PCIT), developed by Sheila Eyberg, was modeled after Constance Hanf?s two-stage approach to therapy for child disruptive behavior (1969). Early parent training programs for child behavior problems focused either on relationship building between parents and children or principles of behavior management (Eyberg, 1988; Querido, Bearss, & Eyberg, 2003). Hanf?s two-stage model was based on operant learning and involved teaching parents to shape their children?s behavior through positive attention and ignoring in the first stage, and then teaching discipline skills in the second stage. Modeling after Hanf?s work, other researchers began to develop such two-stage models of parent-training. Forehand and McMahon (1981) and Eyberg (1988) developed two-stage programs such as ?Helping the Non- Compliant Child? and PCIT designed to change maladaptive patterns of child behavior by changing the ways in which parents responded to their children?s positive and negative behaviors. Like Hanf?s two-stage model, the first stage of both ?Helping the Non-Compliant Child? and PCIT involves teaching positive parenting skills, while the second stage teaches discipline skills. However, in addition, to operant learning techniques, PCIT incorporates play therapy approaches into treatment in order to enhance the parent-child relationship (Eyberg, 1988; Querido et al., 2003). PCIT, then, combines both behavioral and play therapy approaches to treat child behavior problems. 2 PCIT has its theoretical underpinnings in social learning, attachment, and behavioral theories (Brinkmeyer & Eyberg, 2003; Querido et al., 2003). The parent-child relationship is seen as the most salient and critical source of learning for a child. PCIT is based on the idea that children?s behavior is shaped by the environment in which they live, and in particular by their parents. Parents are seen not only as models for their children?s behavior, but also as providing the context for the development and maintenance of those problems. Parents often inadvertently reinforce and maintain problem behaviors through the ways in which they respond to their children (Patterson, 1982). PCIT seeks to alter child behavior by changing parent behavior. In addition, the nature of the parent-child relationship is a critical component of PCIT. Baumrind?s approach to parenting styles and Ainsworth?s attachment theory emphasize the importance of warmth, structure, and stability to the healthy development of children?s social, emotional, and behavioral functioning (Brinkmeyer & Eyberg, 2003; Querido et al., 2003). A nurturing, supportive, and warm (secure) attachment paired with a parenting style that provides both nurturance and firm, appropriate limits for behavior (authoritative style) have been linked to more positive outcomes for children across domains of functioning (Querido et al., 2003). PCIT seeks to foster these relationship components by enhancing the warmth between parents and children while teaching parents how to set firm, consistent, clear limits for their children (Eyberg, 1988; Brinkmeyer & Eyberg, 2003; Querido et al., 2003). Consistent with Hanf?s model, this is done through a combination of child-focused and parent-focused treatment stages. 3 What Does PCIT Look Like? PCIT begins with a child-directed interaction (CDI) phase in which the focus is on teaching positive parenting skills and enhancing the quality of the parent-child relationship (Brinkmeyer & Eyberg, 2003; Eyberg, 1988; Querido et al., 2003). Parents are taught specific skills, much like play therapy techniques, to use during play with their children. They are taught to provide positive attention, through praise, describing what their children are doing, and reflecting what their children say (Brinkmeyer & Eyberg, 2003; Hembree-Kigin & McNeil, 1995; Querido et al., 2003). These skills serve to both reinforce appropriate child behavior and to convey to children that their parents are completely focused on them, view what they are doing as exciting and valuable, and want to spend time with them. In addition, parents are taught to selectively ignore those child behaviors that are inappropriate but not dangerous or destructive (Brinkmeyer & Eyberg, 2003; Hembree-Kigin & McNeil, 1995; Querido et al., 2003). For example, a parent would ignore a child who was whining for attention until he or she used an appropriate voice, at which time the parent would provide specific praise to the child for using a nice voice. Most importantly, PCIT does not just rely on teaching these skills to parents and then sending them home to practice. Parents are coached in vivo with their children through a one-way mirror so that the therapist can provide feedback and shape appropriate parenting behavior (Brinkmeyer & Eyberg, 2003; Hembree-Kigin & McNeil, 1995; Querido et al., 2003). The approach of teaching and coaching parents in more adaptive parenting behaviors continues in the second phase of treatment, the parent-directed interaction 4 (PDI) phase, which focuses on discipline and behavior management (Brinkmeyer & Eyberg, 2003; Hembree-Kigin & McNeil, 1995; Querido et al., 2003). The skills learned in CDI lay the foundation for implementing the skills of PDI, in that parents use their positive attending skills, particularly praise, to respond to child compliance. In addition, the positive relationship build during CDI provides a nurturing, calm, more positive context within which to establish consistent boundaries for child behavior. Parents are taught how to give effective commands and then follow through with either praise (for compliance) or punishment (time out for noncompliance) consistently (Brinkmeyer & Eyberg, 2003; Hembree-Kigin & McNeil, 1995; Querido et al., 2003). They are progressively instructed and coached to use these techniques in more representative situations (i.e., begin with specific practice time, then move to using the techniques in daily life at home, then in public places). This phase of treatment also includes other behavior management techniques such as behavior charts and house rules (rule that, when broken, result in immediate time-out without warning). Throughout treatment, parents practice the skills learned in CDI during the week, and add practice of PDI skills to their daily regimen once they move to that phase of treatment. Regular assessment of parent skill level and child behavior is conducted through both parent-report measures and direct observation and coding of dyadic interactions. Therapy is considered ?successful? when parent and child behavior reaches specific criterion levels, required both to move from CDI to PDI and to finish treatment (or end PDI) (Brinkmeyer & Eyberg, 2003; Hembree- Kigin & McNeil, 1995; Querido et al., 2003). 5 PCIT: Efficacy Research There is a large body of research to support the efficacy of PCIT for externalizing disorders in children (for a review, see Brinkmeyer & Eyberg, 2003). It is one of three parent training programs for behavior disorders identified as either well-established or probably efficacious by Brestan and Eyberg (1998) in their comprehensive review of the treatment literature for children. PCIT has demonstrated both statistically and clinically significant improvement in child behavior on both parent-reported symptom inventories and direct observations of child behavior (Brinkmeyer & Eyberg, 2003). Children?s oppositional, noncompliant, hyperactive, and inattentive behaviors have all decreased from clinically significant levels to within-normal-limits following treatment with PCIT (Brinkmeyer & Eyberg, 2003). In addition, improvements in family and parent functioning have been demonstrated. Self-reports of personal dysfunction (e.g., depression, stress) have shown significant improvements. Parents report significantly fewer depressive symptoms and significantly decreased levels of personal and parent-related stress following PCIT (Schuhmann, Foote, Eyberg, Boggs, & Algina, 1998). Positive behavior change has also been demonstrated in settings outside the home. Funderburk et al. (1998) found that children?s teacher-reported and observed classroom behavior improved to within-normal- limits following PCIT, and the gains were maintained up to 12 months after treatment completion. In addition, non-treated siblings have shown improvements in disruptive behavior. Brestan, Eyberg, Boggs, and Algina (1997) found that fathers reported non- treated siblings? problem behaviors as occurring less frequently, while mothers reported 6 non-treated siblings? behaviors as less problematic, after treatment completion. Parents also report a high level of satisfaction with PCIT, both in terms of improvements in their children?s behavior and the process and format of the treatment itself (Schuhmann et al., 1998). Father involvement has been shown to enhance treatment effects, in that when both parents were involved in PCIT, children?s behavior gains were maintained at greater levels and for longer periods of time than when only one parent (the mother) was involved in treatment (Bagner & Eyberg, 2003). Finally, research indicates that treatment gains persist at a statistically and clinically significant level up to six years after treatment completion. Hood and Eyberg (2003) found that 75% of their treatment sample maintained their clinically significant treatment gains three to six years after completing treatment. Most of this research has been conducted using the traditional PCIT model with children who have externalizing disorders (i.e., ODD and/or ADHD). However, an increasing body of literature exists examining the efficacy of PCIT with other populations and in alternative formats. PCIT: Adaptations to the Original Model Recently, researchers have started to investigate the use of PCIT to treat DSM disorders other than ODD and ADHD (Bagner & Eyberg, 2007; Chaffin et al., 2004; Choate, Pincus, Eyberg, & Barlow, 2005; Gurwitch, Mulvihill, & Chaffin, 2006; McDairmid & Bagner, 2005; Pincus, Eyberg, & Choate, 2005; Vess & Campbell, 2006), and to treat alternative populations of children and families (Querido, 2004; Timmer, Urquiza, & Zebell, 2005; Timmer, Urquiza, & Zebell, 2006; Ware, Timmer, & Urquiza, 7 2006). In some cases, these applications have involved implementing PCIT in its original format. For example, researchers have found that PCIT produces significant improvement in child and parent functioning for children in foster care, children in Head Start, and families with a history of maltreatment, including sexual abuse and neglect (Querido, 2004; Timmer et al., 2005; Timmer et al., 2006). In other cases, researchers have added to the original model, but retained all the components of PCIT in their original format. For example, Chaffin and colleagues (2004) investigated PCIT as a treatment for families with a history of child physical abuse in an RCT comparing PCIT, PCIT plus wrap-around services (e.g., individual counseling for depressed parents), treatment as usual (community parenting group), and non-treated controls (those who decided not to start treatment but were followed for research purposes). Both groups that received PCIT had an initial motivation- enhancement component designed to increase parents? motivation to change their parenting behavior. The motivation enhancement model was added in part due to the nature of the sample (abusive parents ordered to treatment may be resistant to therapy and may benefit from such techniques) and in part to ensure that all treatments would be of the same length. Chaffin et al. found that PCIT alone resulted not only in significant improvement in child behavior and parent functioning but also in significant decreases in re-reports of abuse. PCIT alone was superior to both enhanced PCIT (E-PCIT) and treatment as usual (TAU), with only 19% of the PCIT sample experiencing a re-report of abuse compared to 36% of the E-PCIT and 49% of the TAU group (Chaffin et al., 2004). Researchers have also used PCIT with children diagnosed with Separation Anxiety 8 Disorder by adding a Bravery Directed Interaction (BDI) immediately following CDI designed to decrease children?s anxious behaviors during separation from parents. Preliminary results indicated that the treatment was effective in decreasing anxious behaviors, increasing brave behaviors, decreasing parenting stress and parents? controlling behaviors, and improving children?s negative and inappropriate behaviors (Choate et al., 2005; Pincus et al., 2005). Other diagnostic populations and presenting problems currently being investigated in efficacy studies with PCIT include fetal alcohol syndrome (Gurwitch et al., 2006), co-morbid disruptive behavior and mental retardation (Bagner & Eyberg, 2007; McDairmid & Bagner, 2005), autistic spectrum disorders (Vess & Campbell, 2006), and interparental violence (Ware et al., 2006). These investigations are still ongoing, but preliminary data suggest that PCIT may be efficacious across a wide variety of presenting problems. In addition, researchers have adapted PCIT for use with specific cultural groups. Currently investigations are underway examining cultural adaptations to PCIT for Mexican American (McCabe, Yeh, Garland, Lau, & Chavez, 2005), Puerto Rican (Matos, Torres, Santiago, Jurado, & Rodriguez, 2006), and Native American (Bigfoot, Funderburk, & Gurwitch, 2006) families. Preliminary data indicate that PCIT translated for use in Puerto Rico produces positive outcomes consistent with other efficacy trials (Matos et al., 2006). Though data are not yet available for other culture- specific adaptations, researchers hope to find that PCIT can be used effectively with a variety of cultural groups. 9 Parent Training and PCIT in a Group Format Parent training programs are often offered in both individual (as with traditional PCIT) and group (as with Webster-Stratton?s program) formats. Both modalities have proven effective for treating child conduct problems (Burns, Hoagwood, & Mrazek, 1999; Hoag & Burlingame, 1997; Shechtman & Ben-David, 1999). In fact, meta- analyses comparing group and individual treatments for children and studies comparing these modalities within the same investigation have found little overall difference between the efficacy of individual and group formats of treatment (Burns et al., 1999; Shechtman & Ben-David, 1999; Weisz, Weiss, Han, Granger, & Morton, 1995). Given the relative equality of these two modalities, does it matter which approach a therapist takes? Researchers who favor a group format of treatment assert that it provides social support for patients, exposure to shared experience (universality), group problem solving resulting in a greater sense of mastery, and a more cost-effective method of treatment delivery (Manassis et al., 2002). Researchers who favor an individual format assert that some of these benefits can occur in individual treatment and that individual therapy allows therapists to tailor treatment to individual client needs (Kivlighan & Kivlighan, 2004; Taylor & Biglan, 1998). Unfortunately, there is a dearth of empirical research regarding the truth of such assertions. There are a number of patient variables that do affect treatment outcomes, regardless of the modality used. Families with greater parent psychopathology, marital discord, family stressors, and child behavior problems have poorer outcomes than do families without such problems (Assemany & McIntosh, 2002; Taylor & Biglan, 1998). 10 Individual therapy might seem to be ideal in such situations, given that the therapist would have more flexibility to focus on issues other than child behavior problems. Adding these types of interventions to parent training has been shown to improve therapy gains and the maintenance of those gains over time for some treatments (Taylor & Biglan, 1998). However, Chaffin et al. (2004) found that adding additional wrap-around services to PCIT actually decreased treatment outcomes for high-risk, abusive families, in that families participating in traditional PCIT saw greater gains in child behavior and had fewer re-reports for abuse than did families who received additional therapeutic services to address other family problems. In addition, the families served by community agencies often have limited resources, social support, and high levels of family and parenting stressors (Taylor & Biglan, 1998). Group treatment might be able to address such issues by providing a potentially less costly treatment option with a ?built-in? social support network (Niec, Hemme, Yopp, & Brestan, 2005; Taylor & Biglan, 1998). Such families, who are at high risk for further child and family problem development, may also be more likely to avail themselves of therapy services if they are packaged as a parenting support group. Evidence suggests that some minority groups, such as immigrant families, find group programs more acceptable and are more likely to take advantage of such services than they are of traditional therapy services (Cunningham, Bremner, & Boyle, 1995; Kazdin, 1997; Taylor & Biglan, 1998). In addition, families presenting to community agencies often have a negative history with ?helping? services and may be distrustful of traditional mental health services (Kazdin, 1997). In such cases, though group treatment may not produce greater treatment gains, it may be that more families 11 will engage in, and remain in, such services over individual therapy (Cunningham et al., 1995; Kazdin, 1997). With such factors in mind, Brestan and colleagues (Brestan, Valle, Funderburk, Niec, & Yopp, 2003) developed a group adaptation of PCIT for use with high-risk, drug abusing mothers and their behavior-disordered children. Researchers hoped to be able to provide an efficacious treatment program in a cost-effective manner with added benefits of social support. Social support, in particular, has been linked to increased treatment gains for parents of behavior-disordered children as well as to increased maintenance of those gains over time (Harwood & Eyberg, 2006). In their preliminary study of group PCIT, researchers (Niec, 2006) found that treatment gains on measures of child behavior problems and parenting stress were commensurate with results from other PCIT efficacy trials. In addition, parents in the group PCIT program developed a formal social support network (calling list) to keep in touch with and provide emotional and problem-solving support for each other both during and after treatment (Niec et al., 2005). Currently, Niec is conducting a RCT investigating the relative efficacy of group and individual PCIT. However, no studies exist examining the group format as administered within the community rather than in a university setting or with a more representative sample of families that might seek help in community agencies as opposed to traditional clinical- trials samples. Such research is a necessary step in the development and dissemination of ESTs such as PCIT (Schoenwald & Hoagwood, 2001; Weisz, Chu, & Polo, 2002). 12 Transportability and Effectiveness Research As stated earlier, there is a solid base of knowledge regarding the efficacy of PCIT, both with behavior disordered children and a variety of other presenting problems. However, treatment efficacy is only the beginning. Kazdin (1997) argues that our current knowledge base for treatment efficacy is flawed in that RCTs have not included clinic- referred youth, do not involve treatments that mirror current clinical practice, evaluate statistical but not clinical significance, do not include adequate follow-up assessment, and fail to examine the myriad of contextual factors that may moderate outcome. Other authors have outlined these deficits as well (Hoagwood, 2002; Hoagwood, Burns, Kiser, Ringeisen, & Schoenwald, 2001; Hogan 2002; Shirk, 2004), many of which speak directly to the effectiveness and transportability of treatments for children. Whereas ?efficacy? refers to the extent to which a given treatment produces significant improvement in a given population under circumscribed conditions, ?effectiveness? refers to the extent to which a treatment that has been shown to be efficacious can also produce significant improvement when administered in a clinical setting with clients who are more representative of those who typically seek mental health services in the general community (Chorpita, 2003). Investigations of effectiveness still involve oversight by the researcher, adherence to protocol, and systematic research gathering practices (Shirk, 2004). Transportability refers to the ease (or lack thereof) with which a given EST can be moved from the research setting in which it was developed and tested to the ?real world? (community agency and clinic settings) (Schoenwald & Hoagwood, 2001). A treatment?s transportability is integral to 13 its effectiveness, in that a treatment must be transportable in order for its effectiveness to be investigated (Schoenwald & Hoagwood, 2001). Evidence of effectiveness necessitates that we examine whether a given parent training EST can effect meaningful change in clinical settings with clinic-referred populations (who bring complicated contextual variables to bear). In addition, authors argue that knowledge regarding the transportability and effectiveness of any given parent training EST will need to precede successful dissemination of that EST (Schoenwald & Hoagwood, 2001; Weisz et al., 2002). Ultimately, the goal of clinical researchers is to effect meaningful change for clients. In order to do so on a broader scale for the greatest number of clients, ESTs must eventually be disseminated. If an EST stays only in the research clinic, then the number of clients who will have access to its benefits is limited. Examining a given EST?s effectiveness and transportability is a crucial step in the process of dissemination. Some researchers argue that dissemination does not require establishing effectiveness and transportability first (Hayes, 2002). However, if a program such as PCIT is disseminated without research to determine if it is effective and transportable, clinicians may implement a program that will be of no help to ?real-world? clients. In addition, if the program failed to effect change, it might not be possible to determine whether it was due to shortcomings in the program itself (i.e., it does not lend itself to transportability or does not adequately address complex client needs found in community settings), or due to a lack of treatment fidelity (i.e., clinicians are simply not following the treatment model) or some other influence of which the researchers and clinicians are 14 unaware. One would then have to go ?backwards?, so to speak, and investigate these issues. Researchers who argue for early dissemination usually cite the amount of time that it takes to get from the research clinic to the ?real world? as a reason for moving more quickly (Addis & Waltz, 2002; Hayes, 2002; Hoagwood, 2002; Hoagwood et al., 2001; van de Wiel, Matthys, Cohen-Kettenis, & van Engeland, 2002). Though the concern over time is a reasonable one, skipping ahead does not solve the problem if researchers end up having to backtrack to determine ?where it went wrong?. Therefore, it is important to determine if successful dissemination of PCIT is feasible by investigating its effectiveness and transportability first. Though researchers in California have disseminated PCIT to centers serving maltreated child populations for several years (Porter, Timmer, Urquiza, Zebell, & McGrath, 2006), systematic research into the processes and outcomes for their initiative is lacking. In addition, no such efforts have been conducted with alternative formats of PCIT, such as group PCIT. Given the potential benefits of group parent training (e.g., cost-effectiveness, social support, and greater acceptability with some populations), it is important to determine if group PCIT can be both effective and transportable in a community based setting. Rationale for Present Study Though PCIT is well-studied as an efficacious treatment for a variety of child problems, very little research exists examining issues beyond efficacy. There is little to no systematic research investigating the effectiveness of PCIT. PCIT may be very effective in clinical trials populations, but such samples have been criticized for failing to 15 adequately represent the families and children that clinicians see in community practice (Kazdin, 1997). In addition, the research examining PCIT has traditionally been conducted in academic clinics, so little is known about whether PCIT can be transported into more typical community settings. This is not to say that community mental health providers are not receiving training in and implementing PCIT with their populations, or that PCIT does not work when disseminated in this manner. Rather it is simply that there is a lack of systematic research to support such implementation. The present study provides a preliminary step into effectiveness and transportability research for PCIT, particularly as it relates to the group format, which can begin to set the stage for effective, well-controlled dissemination of an efficacious, promising intervention for troubled families. Hypotheses The present study addressed the question of whether PCIT administered in a group format can demonstrate effectiveness with a community sample of caregiver-child dyads. Based on previous research evaluating the efficacy of PCIT for behavior- disordered children, PCIT in alternative formats, and PCIT with alternative populations, the following hypotheses were made: 1. There would be no difference across demographic variables between dyads that completed treatment and those that discontinued treatment prior to completion. 2. Caregivers would report a significant decrease in child behavior problems, as measured by the ECBI Intensity and Problem Scales, across sessions. 16 a. There would be a significant negative relationship between scores on the ECBI Intensity Scale and time (as measured by treatment sessions). b. There would be a significant negative relationship between scores on the ECBI Problem Scale and time (as measured by treatment sessions). 3. Caregivers? initial levels of stress would predict treatment response and retention. a. There would be a significant negative relationship between pre-treatment Total Stress Scores on the PSI-SF and magnitude of change in ECBI Intensity and Problem Scale scores over time (as measured by treatment sessions). b. There would be a significant negative relationship between Total Stress Scores on the PSI-SF at pre-treatment and treatment completion status. 4. Caregivers would report significantly fewer child behavior problems following treatment completion. a. Scores on the BASC Externalizing Composite and BASC Behavioral Symptoms Index would be significantly lower at post-treatment than at pre- treatment. b. ECBI Intensity Scores would be significantly lower at post-treatment than at pre-treatment. c. ECBI Problem Scores would be significantly lower at post-treatment than at pre-treatment. 5. Caregivers would report less stress, as measured by the PSI-SF Total Stress Scale, at post-treatment than at pre-treatment. 17 6. Caregivers would engage in more prosocial and fewer negative behaviors during dyadic interactions with their children following treatment. a. Frequencies of caregiver prosocial behaviors, as measured by the Prosocial Composite of the DPICS-II, would be significantly higher at post-treatment than at pre-treatment. b. Frequencies of caregiver negative behaviors, as measured by the Inappropriate Behavior Composite of the DPICS-II, would be significantly lower at post- treatment than at pre-treatment. 7. Caregivers? discipline skills would improve following treatment, as demonstrated by significantly higher observed Direct Command Ratios at post-treatment than at pre- treatment. 8. Children would engage in more prosocial and fewer negative behaviors during dyadic interactions with their caregivers following treatment. a. Frequencies of child prosocial behaviors, as measured by the Prosocial Composite of the DPICS-II, would be significantly higher at post-treatment than at pre-treatment. b. Frequencies of child negative behaviors, as measured by the Inappropriate Behavior Composite of the DPICS-II, would be significantly lower at post- treatment than at pre-treatment. 9. Children would demonstrate greater levels of compliance with adult requests following treatment, as measured by significantly higher observed Compliance Ratios on the DPICS-II at post-treatment than at pre-treatment. 18 10. Treatment completers would report a high level of satisfaction with treatment, commensurate with previous research on PCIT, as measured by the TAI total score. 19 METHOD The current research project used archival data from a clinical research project that had already been conducted. The present chapter describes the methodology used for the Group PCIT project as well as for the current research study. Participants Participants for the present study were taken from the subject pool of a research project conducted over two and a half years to provide group parent training to community families in Lee and surrounding counties in Alabama. Participants were 27 dyads consisting of one primary caregiver and one minor child. The sample included 3 male caregivers (all Caucasian) and 24 female caregivers (14 Caucasian; 10 African American), ranging in age from 24 to 57 years, with an average age of 31.95 years (SD = 7.90). Children in the dyads included 17 males (9 Caucasian; 8 African American) and 10 females (9 Caucasian; 1 African American) ranging in age from 2 to 7.67 years, with an average age of 5.02 years (SD = 1.69). Of the participating dyads, 16 successfully completed treatment. Caregivers in the dyads who completed treatment consisted of 1 male (Caucasian) and 15 females (11 Caucasian; 5 African American), ranging in age from 24 to 57 years, with an average age of 32.38 years (SD = 8.72). Children in the dyads who completed treatment included 12 males (7 Caucasian; 5 African American) and 4 females (all Caucasian) ranging in age from 2 to 7.67 years, with an average age of 5.12 years (SD = 1.56). 20 To be included in the study, caregivers had to have physical custody of the target child (who had to be between the ages of 2 and 8 years) and had to self-identify as being in need of assistance to deal with child behavior problems. Of the dyads in the sample, 13 were referred through the Department of Human Resources, and 14 were self-referred. Children were not required to meet diagnostic criteria for a DSM-IV disorder, nor were they excluded based on co-morbidity or DSM-IV diagnosis. The only exceptions to this policy were DSM-IV diagnoses that might have precluded a caregiver or child from understanding or being able to comply with treatment (i.e., severe mental retardation). However, no referred dyads were excluded from the parenting group for this reason. See Table 1 in the Appendix for a more detailed description of demographic information for the participating dyads. Study Measures A variety of measures used in the Group PCIT project were used in the present study including self-report measures from caregivers, caregiver reports about child behavior, intelligence screeners for caregivers and children, and behavioral observations for caregiver-child dyads. To ensure confidentiality for participants, all questionnaires and videotapes were labeled with a study code number and stored in a locked file cabinet in a locked room. A coded list, including each dyad?s study code, caregiver?s name, and child?s name, was stored in a separate cabinet. The list was necessary to allow researchers to track dyads over time and link pre- and post-treatment information. 21 Kaufman Brief Intelligence Test With regard to intelligence screeners, general estimates of caregiver IQ were obtained at the pre-treatment assessment using the Kaufman Brief Intelligence Test (K- BIT; Kaufman & Kaufman, 1990). The K-BIT is a brief, individually administered measure of verbal and nonverbal intelligence for individuals aged 4 to 90 years. The K- BIT consists of two subtests: Vocabulary (including Expressive Vocabulary and Definitions) and Matrices. Age-based standard scores have a mean of 100 and a standard deviation of 15 and are provided for Vocabulary, Matrices, and the K-BIT IQ Composite. Standardization data for the K-BIT are based on a sample of 2,022 subjects who were recruited from a variety of organizations including schools, universities, day cares, community colleges, churches, and learning centers. Split-half reliability coefficients for the Vocabulary and Matrices subtests were good to excellent, with values ranging from .74 to .98. For the overall IQ Composite, reliability coefficients were excellent, with values ranging from .88 to .98. Test-retest reliability coefficients for both subtests and the overall IQ Composite were also excellent, with values ranging from .86 to .97. The K-BIT has also demonstrated adequate internal consistency and concurrent validity (Kaufman & Kaufman, 1990; Naugle, Chelune & Tucker, 1993; Prewett, 1995). The K- BIT was chosen because it allows for an estimate of both verbal and performance IQ in a brief (approximately 20 minutes) period of time. Peabody Picture Vocabulary Test, 3 rd Edition General estimates of child receptive language were obtained at the pre-treatment assessment using the Peabody Picture Vocabulary Test ? 3 rd Edition (PPVT-III; Dunn & 22 Dunn, 1997). The PPVT-III is a brief, individually administered measure of receptive vocabulary and listening comprehension for individuals aged 2 years, 6 months to 90+ years. Age-based standard scores have a mean of 100 and a standard deviation of 15. Standardization data for the PPVT-III are based on a sample of 2,725 subjects who were recruited from a variety of organizational settings and stratified across variables such as parents? education, ethnicity, geographic location, and enrollment in special education services. Split-half internal consistency reliability coefficients for the PPVT-III score were excellent, with values ranging from .89 to .97 for the 25 age groups examined. Test-retest reliability coefficients were also excellent, with values ranging from .85-.90. The PPVT-III has also demonstrated adequate concurrent and construct validity (Dunn & Dunn, 1997; Stetson, Stetson, & Sattler, 2002). The PPVT-III was chosen because it allows for an accurate estimate of a child?s receptive language and correlates well (r values ranging from .81 to .91) with measures of verbal intelligence, such as the Wechsler Intelligence Scale for Children, 3 rd Edition, Verbal IQ composite (WISC-III) and K-BIT Vocabulary composite (Dunn & Dunn, 1997). Given that IQ scores may be lower than average in an at-risk population, the PPVT-III provided an important estimate of the child?s ability to understand commands and directives, an integral part of PCIT. Child Abuse Potential Inventory Caregivers were asked to complete the Child Abuse Potential Inventory (CAP; Milner, 1986) at pre- and post-treatment visits as well as at 3 and 6-month follow-up visits. The CAP is a 160-item self-report measure administered to caregivers that is designed to assess potential for child physical abuse. Parents read statements, which are 23 written on a 3 rd grade reading level, and are then asked to indicate whether they agree or disagree with the statement. Scoring for the CAP results in a primary abuse scale and six factor scales: distress, rigidity, unhappiness, problems with child and self, problems with family, and problems from others. The CAP also has three validity scales that provide three indices of distorted responding: faking-good, faking-bad, and random response (Milner, 1986). With respect to reliability and validity, the CAP has demonstrated a high level of internal consistency and test-retest reliability (r values ranged from .75 to .90) (Feindler, Rathus, & Silver, 2003). The CAP possesses strong evidence for construct validity, particularly as it relates to the rigidity and abuse scales. Scores on these scales are significantly correlated with a parental history of abuse, measures of negative family interactions, such as the Conflict Tactics Scale, and measures of parental adjustment and stress (Feindler et al., 2003). The CAP has also demonstrated concurrent and predictive validity, correctly classifying abusive and non-abusive caregivers with accuracy rates in the low 80% to low 90% range (Feindler et al., 2003; Milner, 1986). In addition, longitudinal studies have demonstrated predictive validity in that researchers have found significant relationships between elevated CAP scores and future confirmed physical abuse incidents (Milner, 1986). The CAP also has normative data from a sample of over 2,000 parents (Feindler et al., 2003). Overall, the CAP is a well-researched, reliable, and valid measure of physical abuse risk (Feindler et al., 2003). The present study used the CAP to describe potential differences between caregivers who completed treatment and those who did not. 24 The Parenting Stress Index ? Short Form Caregivers were asked to complete the Parenting Stress Index ? Short Form (PSI- SF; Abidin, 1995) at pre- and post-treatment visits as well as at 3 and 6-month follow-up visits. The PSI-SF, an abbreviated version of the Parenting Stress Index full-length measure, is a 36-item self-report measure administered to caregivers that is designed to assess the stress they are experiencing related to their duties as a caregiver (Abidin, 1995). Parents read statements and then rate the degree to which they agree or disagree with each statement on a 5-point Likert scale. Scoring for the PSI-SF results in a Total Stress (TS) score and three factor scales: Difficult Child (DC), Parent Distress (PD), and Parent-Child Dysfunction (P-CDI). The PSI-SF also has a validity scale (Defensive Responding) that provides an index of parental defensiveness, or tendency to ?fake-good? (Abidin, 1995). Internal consistency reliability coefficients were excellent for the total stress score and for the three factor scales, with values of .91 (TS), .85 (DC), .87 (PD), and .80 (P-CDI), respectively. Test-retest reliability coefficients were good to excellent for the total stress score and three factor scales, with values of .84 (TS), .78 (DC), .85 (PD), and .68 (P-CDI), respectively. With regard to validity, initial analyses of the PSI- SF indicated that total stress scores on the PSI-SF correlated well with total scores on the full-length PSI (r = .94) (Abidin, 1995). Confirmatory factor analyses have provided mixed support for the three-factor model of the PSI-SF (Abidin, 1995; Deater-Deckard & Scarr, 1996; Reitman, Currier, & Stickle, 2002). In an analysis using a sample of parent-child dyads with and without a history of child maltreatment, Haskett, Ahern, Ward, and Allaire (2006) found that the 25 most parsimonious model was a 2-factor model. They argue for two subscales rather than three, with one factor representing Personal Distress (consisting of items from the Parent Distress scale) and one factor representing Childrearing Stress (consisting of items from the Difficult Child and Parent-Child Dysfunctional Interaction scales). Haskett et al. found that both factors were highly correlated with the Total Stress score. In addition, the two-factor model demonstrated construct validity, in that scores on the PD scale were significantly related to reports of global dysfunction in parents, and scores on the CD scale were significantly related to reports of negative parent behavior, observed child behavior, and parent reports of child behavior problems. The PSI-SF was chosen because previous research indicated parental stress is related to treatment response, attrition from treatment, and risk for child maltreatment (Gurwitch et al., 2006; Haskett et al., 2006), and reducing parenting stress is considered important in treating child disruptive behavior disorders (Kazdin, Siegel, & Bass, 1992). Researchers chose the short form as opposed to the full PSI in order to provide a psychometrically sound measure of caregiver stress while decreasing the burden on respondents. For the purposes of this study, the Total Stress scores was used as a measure of parenting stress, since previous research calls into question the original three- factor structure of the PSI-SF. Based on the standardization information for the PSI-SF, cut-off score of 90 on the Total Stress scale has been established, and scores above this cut-off are considered clinically significant (Abidin, 1995). 26 Eyberg Child Behavior Inventory Caregivers were asked to report on child behavior problems by completing the Eyberg Child Behavior Inventory (ECBI; Eyberg & Pincus, 1999) at pre- and post- treatment visits, each week during treatment, and at 3 and 6-month follow-up visits. The ECBI is a 36-item self-report measure designed to assess children?s problem behaviors, specifically those associated with externalizing disorders such as Oppositional Defiant Disorder and Attention-Deficit/Hyperactivity Disorder. Caregivers are asked to read descriptors of various maladaptive child behaviors and then rate how often they occur on a 7-point Likert scale and whether or not the behavior is a problem for them. The ECBI consists of two scales: Intensity and Problem. The Intensity Scale gives a total score that reflects the frequency of maladaptive behaviors, and the Problem Scale gives a score that reflects the number of behaviors that the caregiver currently considers problematic. The ECBI has demonstrated adequate reliability and validity with children ages 2 to 16 and with Caucasian, African-American, and Hispanic samples (Colvin, Eyberg, & Adams, 1999; Eyberg & Pincus, 1999). Internal consistency coefficients for 2-6 year- olds for the Intensity and Problem scales were .94 and .93, respectively. Test-retest reliability coefficients for the Intensity and Problem scales were high (r = .86 and .88, respectively). Interparent agreement between mothers and fathers was good, with r values of .69 and .61 for the Intensity and Problem scales, respectively. The ECBI has demonstrated evidence for convergent validity, in that scores on the ECBI were significantly related to other measures of child behavior problems and to observed child behavior. The discriminant validity of the ECBI was supported by significant score 27 discrepancies between clinic-referred and non-referred children as well as between children referred for behavior problems and children referred for other problems (e.g., learning disabilities). In addition, the ECBI is sensitive to treatment effects of parent- training interventions for disruptive behavior disorders (Eyberg & Pincus, 1999). The ECBI has normative data for children ages 2 to 16 both with and without disruptive behavior disorders (Colvin et al., 1999). Based on the most recent standardization information for the ECBI, cut-off scores of 131 on the Intensity scale and 15 on the Problem scale have been established, and scores above these cut-offs are considered clinically significant (Colvin et al., 1999; Eyberg & Pincus, 1999). Behavior Assessment System for Children Caregivers were also asked to complete the Behavior Assessment System for Children (BASC; Reynolds & Kamphaus, 1992) at pre- and post-treatment visits as well as at 3 and 6-month follow-up visits. They completed either the Preschool or Child Version, depending on the age of the target child. The BASC is a norm-referenced self- report measure designed to assess children?s general functioning across a number of domains. The Preschool Form (for ages 2.5 to 5) has 131 items, while the Child Form (for ages 6 to 11) has 138 items. Caregivers are asked to read various statements regarding child behavior and then indicate how often that behavior occurs on a 4-point Likert scale. The BASC-P and BASC-C yield 10 common domain scales: Adaptability, Aggression, Anxiety, Attention Problems, Atypicality, Depression, Hyperactivity, Social Skills, Somatization, and Withdrawal. The BASC-C yields 2 additional domain scales: Conduct Problems and Leadership. Both versions also generate Externalizing Problems, 28 Internalizing Problems, Adaptive Skills, and Behavioral Symptoms Index Composites. Each domain and composite scale yields a T-score with a mean of 50 and standard deviation of 10 based both on clinical and non-clinical norms. T-scores between 60 and 70 are considered to be in an at-risk range, while T-scores of 70 or greater are considered to be clinically significant. Standardization data for the BASC parent-report forms were based on a sample of 309 children ages 4-5 years and 2,084 children ages 6-11 years. The sample was representative of census data for variables of age, gender, and ethnicity and was drawn from sites representing a variety of geographical regions. Internal consistency reliability coefficients were good to excellent for domain scales, ranging from .70 to .86, and excellent for composite indexes, ranging from .86 to .92. Test-retest reliability coefficients were good to excellent for domain scales, ranging from .72 to .91, and excellent for composite indexes, ranging from .85 to .90. Interrater reliability coefficients (as measured by interparent correlations) were moderate to good, ranging from .35 to .72 for domain scales and from .38 to .70 for composite indexes (Reynolds & Kamphaus, 1992). The BASC has demonstrated adequate construct and criterion validity for both internalizing and externalizing dimensions (Reynolds & Sattler, 2002). Demographics Questionnaire Caregivers completed a demographics questionnaire at the pre-treatment assessment to gather information related to caregiver and child age, gender, and ethnicity. In addition, the demographics questionnaire asked questions regarding family income, caregiver education, caregiver marital status, and the relationship of the caregiver and child. 29 Therapy Attitude Inventory After treatment completion (and at 3 and 6-month follow-up assessments), caregivers were asked to complete the Therapy Attitude Inventory (TAI; Eyberg, 1993; Eyberg & Johnson, 1974), a 10-item self-report measure of satisfaction with treatment designed to assess the impact of treatment on caregivers? skills and child behavior. The TAI was developed specifically for use with behavioral parent training programs such as PCIT. Caregivers rate items on a 5-point Likert scale ranging from 1 (dissatisfaction with treatment or worsening of problems) to 5 (maximum satisfaction with treatment or improvement of problems). The TAI measures satisfaction with both the process and outcome of treatment and has good evidence for reliability and validity. The TAI has demonstrated high internal consistency (Cronbach?s alpha = .88) and has demonstrated sensitivity to treatment (Brestan, Jacobs, Rayfield, & Eyberg, 1999). Previous research on PCIT indicates that average scores on the TAI for caregivers who complete treatment range from 43 to 46 (Brestan et al., 1999; Schuhmann et al., 1998) Dyadic Parent-Child Interaction Coding System, 2 nd Edition Caregivers and children completed a videotaped observation using the Dyadic Parent-Child Interaction Coding System, Second Edition (DPICS-II; Eyberg, Bessmer, Newcomb, & Edwards, 1994) at pre- and post-treatment assessments. The DPICS-II is a behavioral coding system designed for use in both research and clinical settings. DPICS observations involve a standardized 25-minute play situation divided into three segments: Child-Directed Interaction (CDI), Parent-Directed Interaction (PDI), and Clean Up (CU). A caregiver and child dyad is observed through a one-way mirror as they play together. 30 For 10 minutes, the caregiver is instructed to follow the child?s lead and allow the child to direct the play. For 10 minutes, the caregiver is instructed to lead the play him or herself and get the child to follow along according to the caregiver?s rules. The last five minutes is a clean up period in which the caregiver is instructed to have the child clean up all the toys alone. The first five minutes of CDI and PDI are considered ?warm-up? periods to allow the dyad to get used to the situation, and are therefore not coded. There is no warm-up period for the clean-up interaction, primarily because it is unlikely that the child would require 10 minutes to clean up the toys. For all observations, three toys were selected from a standard set of age appropriate toys (e.g., Lincoln Logs, Legos, farm animals, and Potato Heads). These toys were chosen for their unstructured, interactive quality. All toys were removed from their containers, the containers were placed beside the toys, and the location of the toys in the playroom was kept consistent for all observations. Each dyad was videotaped from behind a one-way mirror in the three DPICS-II standard situations. During observations, the caregivers wore a bug-in-the-ear device, an audio receiver used to signal to the caregivers as unobtrusively as possible when each situation began (i.e., CDI, PDI, or CU). At five-minute intervals, the therapist/assessor read standard instructions over the transmitter to the caregiver in the play room. For CDI the following directions were given: ?In this situation, tell______________ that he/she may play with whatever he/she chooses. Let him/her choose any activity he/she wishes. You just follow his/her lead and play along with him/her.? 31 After the five minute warm-up period, the caregiver was told: ?You?re doing a nice job of allowing _________________ to lead the play. Please continue to let him/her lead.? For PDI the following instructions were given: ?That was fine. Do not clean up the play things at this time. Now we?ll switch to another situation. Tell _____________that it is your turn to choose the game. You may choose any activity. Keep him/her playing with you according to your rules.? After the five minute warm-up period, the caregiver was told: ?You?re doing a nice job of leading the play. Please continue to get ________ to play along with you according to your rules.? For CU the following instructions were given: ?That was fine. Now I?d like you to tell______________ that it is time to leave the playroom and the toys must be put away. Make sure you have him/her put the toys away by him/herself. Have him/her put all the toys in their containers and all the containers on the table.? Verbal, vocal, and physical child and caregiver behaviors are coded into one of 27 categories (see Table 2 and Table 3 in Appendix for a list and a complete description of these categories, respectively). In keeping with previous research using the DPICS-II, codes were compiled into composite scores, whereby two or more individual codes were combined into a single category designed to reflect a more general construct (e.g., inappropriate verbal behavior, commands, compliance, questions, and appropriate 32 behaviors) (Aragona & Eyberg, 1981; Bessmer, Brestan, & Eyberg, 2007; Brestan, Foote, & Eyberg, 2007; Deskins, 2004; Eyberg, Nelson, Duke, & Boggs, 2004; Webster- Stratton, 1985). The Inappropriate Behavior Composite category (Criticism, Whine, Yell, Smart Talk, Physical Negative, Destructive) was created to describe both caregiver and child inappropriate verbal, physical, and vocal behavior. The Prosocial Behavior Composite category (Acknowledgment, Behavioral Description, Information Description, Unlabeled Praise, Labeled Praise, Contingent Labeled Praise, Reflective Statement, Laugh, and Physical Positive) was created to describe caregiver and child appropriate verbal, physical, and vocal behavior. The child Prosocial Composite also included the category of Compliance. In addition, a Direct Command Ratio (Direct Command / Direct Command + Indirect Command) was calculated for caregivers, and a Compliance Ratio (Compliance / Total commands with opportunity for compliance) was calculated for children (see Table 4 for a list of composite categories and corresponding DPICS-II behaviors). Investigations of the DPICS-II have demonstrated evidence for the reliability and validity of the coding system with 3 to 7 year-old children, both clinic referred for ODD and non-referred children, as well as standardization data for the same sample (Bessmer et al., 2007). Bessmer et al. assessed reliability using percent agreement, intraclass correlations, and Cohen?s kappa. Overall, the DPICS-II categories were shown to have acceptable reliability estimates comparable to those found for the original DPICS categories (average r = .91 for parent categories and .92 for child categories). In addition, the DPICS-II demonstrated adequate reliability with father-child dyads (Brestan 33 et al., 2007). The DPICS-II also demonstrated convergent validity with measures of child behavior problems, parental locus of control, and parenting stress for mothers and fathers. Significant differences in code frequencies between dyads that included a behavior- disordered child and those that did not provide evidence for the discriminative validity of the DPICS-II (Bessmer et al., 2007; Brestan et al., 2007). DPICS-II Coding & Transcribing Videotaped interactions for the sample were coded from fall of 2004 to spring of 2006. Before coding videotapes for the dyads, all observers successfully completed training procedures for the DPICS II in accordance with the recommendations provided by The Workbook: A coder training manual for the Dyadic Parent-Child Interaction Coding System II (Eyberg, Edwards, Bessmer, & Litwins, 1994). Standard training consists of a minimum of 30 hours of didactic training in the DPICS-II, which includes reading the coding manual, studying and successfully completing paper and pencil training exercises and quizzes, and coding transcripts of actual parent-child interactions. After completing the workbook exercises, observers code training videotapes with a transcript, are given feedback from a trained coder, and finally code criterion tapes to evaluate their level of mastery. The coders are considered successfully trained when they achieve a minimum of 80% agreement with correct coding of a criterion tape using kappa coefficients (assessed through code by code agreement). All coders for the present study were required to complete training as described above. In addition, training sessions were held on a weekly basis by a faculty member with expertise in the DPICS-II, during which observers discussed coding issues and practiced coding categories that they 34 considered difficult. Weekly training sessions occurred throughout the duration of the study to prevent observer drift. Transcripts of the videotaped parent-child interactions were prepared by undergraduate research assistants who watched the videotapes and created a verbatim transcription. Transcribers were trained by a graduate student who had achieved successful training (at least 80% agreement) on the DPICS-II. Transcripts included the time each segment began and ended as well as the end time for each verbal and motoric DPICS-II behavior during the interaction. For quality control, the first author watched approximately 33% of the transcript segments and compared the written transcript with the videotaped interaction. Any necessary changes to the transcript were noted, and the coders were asked to edit the transcript as they coded the tape. In addition, primary coders were asked to check for and correct any inconsistencies between transcripts and videotapes. These transcripts allowed for more accurate coding of the parent-child interactions, as the transcription aided coders when the videotape was difficult to hear and/or understand. Videotape coding was completed by a team of undergraduate and graduate students trained in the DPICS-II. Primary observers coded three videotaped situations (CDI, PDI, CU) from the pre-treatment and post-treatment assessments for each dyad. All primary observers were unaware of the specific hypotheses of the study, and no primary coder coded both the pre- and post-treatment observation for any given family. In order to assess reliability, one randomly selected segment for each DPICS observation was re-coded by a team member (typically the first author) who did not serve as the 35 primary coder for that particular family; therefore, reliability was assessed on 33% of the video recorded observations. To ensure that coders remain blind to assessment point (e.g., coders do not know if a specific observation is pre- or post-treatment), a random number list was generated and each tape was given a tape number. The first author then created a master list, including the family subject number and tape number corresponding with each assessment point for each family. This procedure ensured that the first author could identify each tape as a pre- or post-treatment assessment, while keeping coders from being able to do so. Procedures: Project Development & General Design After obtaining IRB approval for the project, participants were recruited from caregiver/child dyads that sought treatment in the group PCIT program offered through the Child Advocacy Center of East Alabama (CACEA). The Group PCIT program began as a joint effort between Dr. Elizabeth Brestan Knight, her graduate students, and the CACEA director, Emilyn Gipson. After obtaining grant funding, a treatment protocol was developed. The protocol for the group was modeled after one developed by Brestan and colleagues (2003). The group included an intake assessment and 12 therapy sessions. The research protocol also included two follow-up assessments (see Table 5 in the Appendix for a list of session content and assessment measures). Dyads who presented for treatment were asked if they would consider participating in a research project as part of the program. Participation in the research portion of the project was not required for treatment participation. Caregivers who chose to enroll in the research study were asked to complete intake procedures prior to beginning treatment, which took approximately 3 36 hours to complete. They were then enrolled in the next available group. Nine groups were conducted between January 2003 and December 2005. Procedures: Intake Intake assessments took approximately 3 hours and included a brief clinical interview, intelligence screening, questionnaires, and a behavioral observation. At intake, caregivers were given the K-BIT, while children were given the PPVT. Both measures were administered by trained graduate students. Caregivers then completed the demographics questionnaire, ECBI, BASC, CAP, and PSI-SF. Dyads also completed a DPICS-II observation. Each caregiver and child pair was videotaped in the observational interview room at the CACEA. The room was cleared of all objects except three toys chosen from a set of toys designated for the observations. These procedures follow standard DPICS observation requirements, as described above. Procedures: Group Sessions After completing the intake assessment, participants began the treatment group. Groups were conducted over a 2-year period at the rate of one to two groups per academic semester. Each group consisted of three to four families. Sessions were held weekly for 12 weeks and lasted approximately 2 hours each. At each weekly session (2 through 11), caregivers were asked to complete the ECBI to track weekly changes in child behavior. The first session was an introductory session consisting of both didactic and interactive elements. The session included ice-breaking activities to allow group members to get to know each other, an introduction to behavior management principles and PCIT, a description of the way each session would be conducted and what caregivers 37 would be expected to do in treatment and at home, and exercises taken from Brestan et al. (2003) modeled after principles of motivational interviewing designed to increase caregivers? motivation to engage in treatment and change their parenting practices. The second session was the Child-Directed Interaction (CDI) Didactic session. The CDI Didactic followed standard PCIT procedures (Hembree-Kigin & McNeil, 1995). The session was designed to teach caregivers about ?special time? and the rationale behind this play interaction. Therapists taught caregivers the play therapy skills they would be expected to use during ?special time?, known as PRIDE skills (praise, reflection, imitation, description, and enthusiasm). In addition, caregivers were instructed in the ?don?t? behaviors for special time and asked to refrain from questions, commands, and criticisms. Therapists demonstrated the skills for caregivers, and when time permitted, caregivers practiced the skills themselves. Caregivers were also given instruction in selective attention and strategic ignoring. They were taught to use the PRIDE skills to reinforce appropriate behaviors and to ignore inappropriate (non- destructive) behaviors. Caregivers were given handouts about the PRIDE skills, the ?don?t? behaviors of special time, the types of toys to use, and the best way to plan for special time. Caregivers were also given homework sheets and instructed to practice the skills daily in 5-minute ?special time? interactions (they were given standard instructions to introduce special time to their children) and keep a record of their practice sessions and any issues or problems they had. Sessions two through five were designated as CDI practice sessions. During these sessions, the first 15 to 30 minutes were designated as a time of discussion and problem 38 solving in a group format. After this discussion time, each caregiver was coached individually as he or she played with his or her child during child-directed play. During this time, one dyad was in the CACEA playroom while all other caregivers and therapists were in the observation room. Thus, caregivers not only received direct coaching of their behaviors, but they were also able to experience vicarious learning by watching other dyads interact and hearing the coaching for those caregivers. In this way, caregivers served as models for one another. Prior to beginning coaching, therapists observed each dyad for 5 minutes, coding caregiver behavior using a clinical version of the DPICS-II (Hembree-Kigin & McNeil, 1995) in order to assess skill progress. Therapists coached caregivers following standard PCIT procedures (Hembree-Kigin & McNeil, 1995), using the bug-in-the-ear transmitter system to talk to the caregivers, and provided direct feedback related to PRIDE skills and any caregiver commands, questions, or criticisms. Once all caregivers received approximately 15 minutes of observation and coaching time, everyone gathered as a group again. During the last 15 minutes of each session, caregivers were provided with feedback from the therapists and from each other regarding their skills and progress. Typically, caregivers are required to reach a certain level of skill mastery prior to moving from CDI to PDI (Hembree-Kigin & McNeil, 1995). Given the nature of the group format, this requirement could not be enforced. However, caregivers continued to receive coaching in CDI skills throughout treatment, and their skills were assessed at the beginning of each coaching session throughout treatment. 39 The sixth session was the Parent-Directed Interaction (PDI) Didactic session, which followed standard PCIT procedures (Hembree-Kigin & McNeil, 1995). This session was designed to teach caregivers appropriate discipline procedures. The session included instruction in effective ways to give commands (e.g., be concise, simple, direct) and consistent discipline strategies for child non-compliance. Specifically, caregivers were taught to use a time-out procedure when children failed to comply with commands. They were also taught a back-up for the time-out chair to use when children refused to stay in the chair. Therapists demonstrated time-out and had each caregiver practice a holding chair procedure, in which the child was held in a separate chair for a few seconds and then placed back in the time-out chair. Caregivers were taught specific, standard scripts to say to children following non-compliance (i.e., ?you have two choices, you can either _________ or you can go to the time-out chair? and ?you chose not to mind, so now you have to go to time out?). Each caregiver was provided with handouts on effective commands and the words to use following non-compliance. Caregivers were instructed to study the PDI procedures but not to practice at home. Caregivers were required to refrain from implementing the new discipline procedures until they had practiced in session. In this way, the therapists could ensure that caregivers understood the procedures and were implementing them correctly. Finally, caregivers were instructed to continue with the daily 5-minute CDI practice. The seventh session, the first PDI practice session, was conducted individually for each family. This allowed therapists to provide adequate time in case of lengthy time- outs. In addition, caregivers could practice the skills without worrying about other group 40 members observing what could be potentially embarrassing child behaviors. It is imperative that during the first coaching session, therapists have adequate time to devote to a dyad and that the caregiver stay calm and focused. For these reasons, families were seen individually. At the beginning of the session, the therapists reviewed PDI skills with the caregiver. Prior to beginning coaching, the therapists and caregiver explained the new discipline procedures to the child and demonstrated each contingency using either an adult pretending to be the child or ?Mr. Bear? (depending on the child?s age). Following the demonstration, the make-up of the session was the same as for the other PDI coaching sessions, described below. The 8 th through 11 th sessions were designated as PDI practice sessions. During these sessions, the first 15 to 30 minutes were used for discussion, problem solving, and for teaching new skills. For example, during the 8 th session, caregivers were taught how to implement a behavior chart, during the 10 th session, caregivers were taught how to implement house rules (behaviors that result in immediate time-out, such as physically hurting another person), and in the 11 th session, caregivers were taught how to use time- out procedures in public places. The following hour to 90 minutes of the session was used for coaching, as with CDI sessions. Each dyad received approximately 15 minutes of therapist attention. For the first five minutes, therapists observed and coded each dyad using the DPICS-II, and for the remaining 10 minutes, the therapist coached PRIDE skills and discipline skills. Once all caregivers had received approximately 15 minutes of observation and coaching time, everyone gathered as a group again. During the last 15 minutes of each session, caregivers were provided with feedback from therapists and 41 from each other regarding their skills and progress. As they progressed through each session, caregivers were instructed to increase the difficulty of commands given, to move from specific PDI practice times to implementing PDI throughout the day, and to practice PDI in multiple settings. The 12 th and final session was a wrap-up session. During this session, the therapists reviewed progress with caregivers and discussed troubleshooting for potential problems that might develop with regard to child behavior. In addition, caregivers completed post-treatment assessment measures, including the ECBI, BASC, CAP, PSI- SF, and TAI. Dyads also completed a post-treatment DPICS-II observation. Finally, a ?graduation ceremony? was conducted, in which caregivers and children were given completion certificates and gift bags that included a variety of toys and materials. For example, all caregivers were given foam stadium seats to use as time-out cushions when away from home and spiral bound index cards to use as behavior charts for public places. Procedures: Follow-Up Assessments Three months after the group ended, all caregivers (both those who completed treatment and those who dropped out of treatment) were invited by phone and mail to participate in a follow-up research assessment. Each caregiver was sent the ECBI, BASC, PSI-SF, CAP, PRFCS, and TAI by mail and asked to return them. Caregivers were provided with a self-addressed, stamped envelope with which to return the forms. Six months after the group ended, all caregivers were invited again to return to the CACEA for a follow-up research visit consisting of the same self-report forms as the 3- month assessment as well as a DPICS-II observation. Caregivers who completed the 6- 42 month assessment were compensated monetarily for the visit. No follow-up data were included in the present study due to a very low response rate. Data Analyses Independent samples t-test analyses were conducted to determine what differences, if any, existed between caregivers who completed treatment and those who did not with respect to demographic variables such as age, gender, ethnicity, referral source, marital status, and family income. Interrater reliability analyses using Cohen?s kappa were conducted on 33% of all DPICS-II videotaped interaction segments. In addition, a kappa program was used to generate frequency counts for all parent and child DPICS-II behaviors, and frequency counts were calculated for all DPICS-II composite categories. A repeated measures multiple analysis of variance (MANOVA) was conducted to examine differences between pre- and post-treatment functioning. Prior to conducting the repeated measures MANOVA, bivariate correlations were run between demographic variables and outcome variables to determine if any relationships existed between them. Where significant correlations existed, demographic variables were entered into the MANOVA as covariates to remove their potential confounding effects from the analysis. After conducting correlational analyses, a repeated measures analysis was used to investigate treatment effects on reports of child behavior problems (ECBI Intensity, ECBI Problems, BASC Externalizing Composite) and caregiver levels of stress (PSI-SF Total Stress). In addition, a similar repeated measures analysis was conducted to investigate 43 treatment effects on caregiver and child behavior, as measured by the DPICS-II composite scores (Prosocial, Inappropriate, Direct Command Ratio, Compliance Ratio). Regression analyses were conducted to examine improvement in child behavior problems over time and to examine the relationship between parenting stress and treatment response and retention. Standard regressions were conducted to determine if there was a relationship between caregiver reports of child behavior problems (as measured by scores on the ECBI Intensity and Problem scales) and length of time in treatment (as measured by session number). In addition, scores on the ECBI Intensity and Problem scales were graphed to examine the relationship between treatment progress (as measured by sessions completed, or time) and reported levels of child behavior problems (as measured by the ECBI Intensity and Problem scales). Hierarchical regressions were also conducted to determine if pre-treatment scores on the PSI-SF Total Stress scale predicted change in reports of child behavior problems (as measured by the ECBI Intensity and Problem scales) and treatment retention (as measured by treatment completion status, number of sessions completed, and last session attended). Finally, a Reliable Change Index (RCI) analysis was conducted to examine further any observed changes in caregiver reports of child behavior problems over time. An RCI analysis is designed to determine how many participants in a given treatment sample achieved a level of change on treatment outcome measures that is unlikely to be due to the unreliability of the measures themselves. This analysis was conducted for all treatment outcome measures that demonstrated statistically significant change in the repeated measures MANOVA analyses. 44 RESULTS Preliminary Analyses Prior to conducting the primary study analyses, preliminary analyses were conducted to ensure a usable dataset and to gain a better understanding of the data. For all analyses, missing data were managed by casewise deletion, such that for a given analysis, if a participant was missing data included in that analysis, he or she was eliminated prior to conducting the analysis. Cohen?s Kappa Reliability Analysis Observations of caregiver-child dyads were coded by a trained primary coder using the DPICS-II, and 33% of all observation segments (one segment from each tape) were randomly selected and coded by a trained secondary coder to ensure reliability. Reliability was assessed using Cohen?s kappa, and a cutoff kappa value of .80 was established for both parent and child DPICS-II codes. Thus, for the data from an observation to be included in analyses, the overall kappa coefficient for the reliability segment of that observation for both parent codes and child codes had to be greater than or equal to .80. For any observation in which either the child or parent code kappa value was below .80, the observation was recoded and reliability was reassessed. No observation had to be coded more than two times, and all observations included in the present study had kappa coefficients of at least .80. Across all segments coded for 45 reliability, the average Cohen?s kappa value for parent and child codes were r = .85 and r = .87, respectively. Hypothesis 1 A preliminary analysis was conducted to determine if there were any significant differences between participants who completed treatment and treatment drop-outs across demographic variables and pre-treatment caregiver-report measures. It was hypothesized that there would be no significant differences between treatment completers and non- completers across any of the demographic variables. To test this hypothesis, two-tailed independent-samples t-tests were conducted. The following variables were included in the analysis: caregiver age, caregiver gender, caregiver ethnicity, caregiver?s marital status, caregiver?s relationship to the child, caregiver?s education level, yearly family income, child?s age, child?s gender, child?s ethnicity, caregiver?s estimated IQ (K-BIT), child?s estimated receptive language ability (PPVT), caregiver?s pre-treatment abuse potential (CAP), child?s pretreatment negative behavior (ECBI Intensity, ECBI Problem, BASC Externalizing, BASC Behavior Symptoms Index), and caregiver?s pre-treatment level of stress (PSI-Total). Caregivers and children who completed treatment did not differ significantly from each other across any demographic variables or pre-treatment measures. Correlational Data A preliminary analysis was also conducted to examine what relationships existed, if any, between demographic variables and treatment outcomes. Bivariate Pearson product moment correlations and biserial correlations were calculated in order to detect 46 such relationships, to allow for control of potential confounds, and to understand how all the variables were related to one another. The following demographic variables were included in the correlational statistics: caregiver age, caregiver gender, caregiver ethnicity, caregiver?s marital status, caregiver?s relationship to the child, caregiver?s education level, yearly family income, child?s age, child?s gender, and child?s ethnicity. Correlations were run both for pre and post-treatment scores and for difference scores (calculated by subtracting post-treatment scores from pre-treatment scores) across outcome measures (ECBI Intensity, ECBI Problem, BASC Externalizing, BASC Behavior Symptoms Index, PSI-Total Stress, DPICS Composite scores). In addition, where significant relationships existed, variables were graphed using scatterplots to examine the nature of the relationship between them. Of the demographic variables examined, none demonstrated significant relationships with caregiver-report outcome measures. However, six demonstrated significant relationships with pre- and/or post-treatment observational measures (DPICS- II Composite scores): caregiver?s relationship to the child, caregiver?s gender, ethnicity, caregiver marital status, caregiver education, and child gender. Caregiver?s relationship to the child was positively correlated with the child?s pre-treatment inappropriate behavior, such that higher levels of child inappropriate behavior prior to treatment were related to grandparent status while lower levels were related to biological parent status, r b = .63, n = 14, p = .016. Caregiver gender was negatively correlated with the child?s post- treatment compliance ratio, such that higher levels of child compliance after treatment were related to male caregiver status while lower levels of compliance were related to 47 female caregiver status, r b = -.73, n = 13, p = .004. Ethnicity was negatively correlated with the caregivers? pre-treatment level of prosocial behavior, both pre- and post- treatment levels of inappropriate behavior, and with the child?s post-treatment level of prosocial behavior. Higher levels of caregiver pre-treatment prosocial behavior were related to Caucasian ethnic status, while lower levels were related to African American ethnic status, r b = -.75, n = 14, p = .002. In addition, higher levels of caregiver inappropriate behavior at both pre-treatment (r b = -.56, n = 14, p = .034) and post- treatment (r b = -.56, n = 14, p = .034) were related to Caucasian ethnic status, while lower levels at both pre- and post-treatment were related to African American ethnic status. Caregiver?s marital status was positively correlated with the child?s pre-treatment level of inappropriate behavior and the caregivers? post-treatment level of inappropriate behavior. Lower levels of child pre-treatment inappropriate behavior were related to single or married caregiver status, while higher levels of inappropriate behavior were related to divorced or separated status, r = .59, n = 14, p = .027. In addition, lower levels of caregiver post-treatment inappropriate behavior were related to single or married status, while higher levels of inappropriate behavior were related to divorced or separated status, r = .54, n = 14, p = .048. Caregiver education was positively correlated with caregiver?s post-treatment prosocial behavior, such that higher levels of prosocial behavior after treatment were related to higher levels of educational attainment, r = .57, n = 14, p = .032. Finally, child gender was positively correlated with the child?s pre-treatment level of inappropriate behavior, such that higher levels of inappropriate behavior before treatment were related to female child status, while lower levels of inappropriate behavior 48 were related to male child status, r b = -.56, n = 14, p = .04. Correlational results are presented in Tables 6 and 7 below. Table 6. Correlations of Demographics with Pre-Treatment DPICS-II Composites Pre-Tx Parent Prosocial Composite Pre-Tx Parent Inapp. Composite Pre- Tx Child Inapp. Composite Corr. .268 -.136 .627(*)Parent-Child Relationship Sig. (2-tailed) .354 .642 .016 Pearson Corr. -.119 -.511 -.029Parent Gender Sig. (2-tailed) .687 .062 .923 Corr. -.750(**) -.569(*) -.459Parent Ethnicity Sig. (2-tailed) .002 .034 .099 Corr. .349 .341 .589(*)Marital Status Sig. (2-tailed) .221 .233 .027 Corr. .373 .035 .061Education Sig. (2-tailed) .189 .905 .835 Corr. .423 -.108 .555(*)Child Gender Sig. (2-tailed) .132 .713 .040 Note. For all correlations, n = 14; *p < .05; **p < .01 Table 7. Correlations of Demographics with Post-Treatment DPICS-II Composites Post-Tx Parent Prosocial Composite Post-Tx Parent Inapp. Composite Post-Tx Child Prosocial Composite Post-Tx Child Comp. Ratio Corr. .086 -.142 -.033 .186Parent-Child Relationship Sig. (2-tailed) .770 .629 .910 .543 Corr. .200 .213 -.307 -.734(**)Parent Gender Sig. (2-tailed) .493 .464 .285 .004 Corr. -.460 -.568(*) -.598(*) -.426Parent Ethnicity Sig. (2-tailed) .098 .034 .024 .147 Corr. .427 .537(*) .161 .034Marital Status Sig. (2-tailed) .128 .048 .582 .913 Corr. .574(*) .404 -.193 -.456Education Sig. (2-tailed) .032 .152 .509 .118 Corr. .325 .031 .404 .152Child Gender Sig. (2-tailed) .257 .916 .152 .621 Note. For all correlations, n = 14; *p < .05; **p < .01 49 When examining correlational results, it should be noted that for demographic variables such as caregiver gender and the caregiver?s relationship to the child, there were very few participants in some groups. With regard to gender, there was very limited enrollment of male caregivers in the sample, and only one male caregiver completed treatment. With regard to caregiver-child relationships, only one non-biological caregiver enrolled in and completed treatment. Therefore, the results for these variables are suspect given that the group sizes are so discrepant. Consequently, examining group differences for caregiver gender or caregiver-child relationships may not be a useful means for analyzing the data, and female and male caregivers, as well as biological and non-biological caregivers, were examined as one group for primary analyses. Based on these preliminary analyses, no demographic variables were included in the repeated measures MANOVA examining treatment outcomes for caregiver-report measures. However, ethnicity, caregiver marital status, caregiver education, and child gender were included as covariates in the repeated measures MANOVA examining treatment outcomes for DPICS Composite scores. Regression Analysis for ECBI Scores over Time A regression analysis was conducted to examine the relationship between caregiver reports of child behavior problems and treatment progression. For this analysis, all 27 participants were included. Two hierarchical regressions were conducted, one for ECBI Intensity scores and one for ECBI Problem scores, with session number entered as a predictor variable in each. For these analyses, the database had to be adjusted such that each individual ECBI rating served as its own observed case in the database rather than 50 having session ECBI scores grouped by participant. In order to control for the fact that observations were not independent, family subject number was entered first into the regression to remove any potentially confounding effects. In addition, ECBI Intensity and Problem Scale scores were graphed by session number, both for the entire sample and for treatment completers and non-completers separately, in order to provide a visual representation of changes in ECBI scores over time (see Figures 1 through 6 in Appendix). Hypothesis 2 It was hypothesized that caregivers would report a significant decrease in child behavior problems, as measured by the ECBI Intensity and Problem Scales, across sessions. Specifically, it was expected that there would be a significant negative relationship between scores on both the ECBI Intensity Scale and the ECBI Problem Scale and time (as measured by treatment sessions). In step 2 of first hierarchical regression analysis, session number was a significant predictor and explained unique variance (?R 2 = .044) in ECBI Intensity scores, F (1, 213) = 10.21, p = .002. Adding session number helped explain an additional 4.4% of the variance in caregivers? reports of frequency of child behavior problems, making it a significant contribution to the equation. Intensity scores decreased over time, and higher session numbers were related to lower scores on the Intensity Scale. In the second hierarchical regression analysis, session number did not significantly predict ECBI Problem Scale scores. There was no observed relationship between scores on the Problem Scale and session number. Family 51 subject number significantly predicted ECBI Intensity scores, but it did not predict ECBI Problem scores. Results of each standard regression are presented in Table 8 below. Table 8. Regression Analyses for Session Number as a Predictor of ECBI Scores Unstandardized Coefficients 95% CI Predictor ECBI Scale B SE T Sig. Lower Upper Intensity -1.07 .45 -2.39 .018 -1.94 -.19 Family Subject Number Problem -.08 .10 -.73 .47 -.28 .13 Intensity -2.90 .91 -3.20 .002 -4.68 -1.11 Session Number Problem -.34 .21 -1.60 .11 -.75 .08 Regression Analyses ? Parent Stress as Predictor of Response/Retention Regression analyses were conducted to examine the relationship between caregiver reports of parenting stress and treatment response and retention/completion. For these analyses, all 27 participants were included. Two standard regression analyses were conducted for treatment retention, as measured by the last session attended and total number of sessions attended, with pre-treatment Total Stress scores on the PSI-SF entered as a predictor variable for each. In addition, a logistical regression analysis was conducted for treatment completion, as measured by a dichotomous variable whereby participants were considered either completers or non-completers, with pre-treatment Total Stress scores on the PSI-SF entered as a predictor variable. A logistic regression was chosen due to the dichotomous nature of the dependent variable (Hinton, Brownlow, McMurray, & Cozens, 2004). Finally, two hierarchical regressions were conducted for change in ECBI Intensity scores and ECBI Problem scores, with pre-treatment Total Stress scores on the PSI-SF entered as a predictor variable in each regression. For these 52 analyses, change in ECBI Intensity and ECBI Problem scores was defined as the difference between the first Intensity or Problem score and the last observed Intensity or Problem score (regardless of corresponding session). Because not all participants completed the same number of sessions, and not all last observed ECBI scores came from the same session, last session attended was entered first into each regression to control for confounding effects. Hypothesis 3 It was hypothesized that pre-treatment parenting stress levels would predict treatment retention. Specifically it was expected that there would be a significant negative relationship between Total Stress Scores on the PSI-SF at pre-treatment and treatment retention measures, such that higher levels of pre-treatment parenting stress would predict poor treatment retention, as measured by completion status, number of sessions attended, and last session attended. There were no relationships between pre- treatment levels of caregiver parenting stress and treatment retention measures, and pre- treatment stress did not significantly predict treatment retention or completion. Pre- treatment parenting stress scores accurately classified participants into treatment completion groups 55.6 percent of the time overall. It was also hypothesized that pre-treatment parenting stress levels would predict level of change in child behavior problems over time. Specifically, it was expected that there would be a significant negative relationship between pre-treatment Total Stress Scores on the PSI-SF and magnitude of change on ECBI Intensity and Problem Scale scores, such that higher levels of pre-treatment stress would predict less improvement in 53 child behavior problems. In step 2 of the hierarchical regression analysis for ECBI Intensity scores, parenting stress was a significant predictor and explained unique variance (?R 2 = .188), F (1, 24) = 9.09, p = .006. Adding pre-treatment level of parenting stress helped explain an additional 18.8% of the variance in caregivers? reports of frequency of child behavior problems, making it a significant contribution to the equation. However, the relationship between pre-treatment parenting stress and changes in child behavior problems was in the opposite direction expected, indicating that caregivers who reported higher levels of parenting stress before beginning treatment actually reported greater improvement over time in the frequency of their children?s problem behaviors. Last session attended was also a significant predictor of change in ECBI Intensity scores. However, neither pre-treatment parenting stress nor last session attended significantly predicted change in ECBI Problem scores. There was no observed relationship between scores on the Problem Scale and either parenting stress or sessions attended. Results of both regressions are presented in Table 9 below. Table 9. Regression Analysis for PSI Total Stress as a Predictor of ECBI Scores Unstandardized Coefficients 95% CI Predictor ECBI Scale B SE T Sig. Lower Upper Last Session Intensity 3.79 1.01 3.74 .001 1.70 5.88 Problem .54 .39 1.39 .179 -.27 1.35 PSI Total Stress Intensity .66 .22 3.02 .006 .21 1.11 Problem .03 .08 .40 .696 -.14 .20 54 Repeated Measures MANOVA ? Caregiver-Reports A repeated measures MANOVA was conducted on caregiver-report measures to determine if caregivers reported improvement in child behavior problems and levels of parenting stress after completing treatment. For this analysis, 15 participants who completed treatment, and who also completed all pre- and post-treatment measures, were included. One participant who completed treatment was excluded from the analysis due to lack of valid and/or scoreable measures on the majority of the caregiver-report measures included in the MANOVA. Pre- and post-treatment scores on the BASC Externalizing Composite, BASC Behavioral Symptoms Index, ECBI Intensity Scale, ECBI Problem Scale, and PSI-SF Total Stress Scale were utilized in the MANOVA. Hypothesis 4 It was hypothesized that caregivers would report significantly fewer child behavior problems following treatment completion. Specifically, it was expected that scores on the BASC Externalizing Composite, BASC Behavioral Symptoms Index, ECBI Intensity Scale, and ECBI Problem Scale would be significantly lower at post-treatment than at pre-treatment. Caregivers did report significantly fewer child behavior problems on the BASC Externalizing Composite after completing treatment (M = 59.13, SD = 18.73) than when they entered treatment (M = 71.60, SD = 23.68), F (1, 14) = 19.00, p = .001; Wilks? Lambda = .242; partial eta squared = .576. Caregivers reported significantly fewer child behavior problems on the BASC Behavioral Symptoms Index after completing treatment (M = 51.33, SD = 18.98) than when they entered treatment (M = 62.33, SD = 21.33), F (1, 14) = 15.76, p = .001; Wilks? Lambda = .242; partial eta 55 squared = .530. Caregivers reported significantly fewer child behavior problems on the ECBI Intensity Scale after completing treatment (M = 91.40, SD = 45.97) than when they entered treatment (M = 135.20, SD = 55.88), F (1, 14) = 30.72, p < .001; Wilks? Lambda = .242; partial eta squared = .687. Finally, caregivers reported significantly fewer child behavior problems and on the ECBI Problem Scale after completing treatment (M = 11.40, SD = 11.43) than when they entered treatment (M = 19.93, SD = 10.07), F (1, 14) = 26.75, p < .001; Wilks? Lambda = .242; partial eta squared = .656. Prior to beginning treatment, on average, caregivers were reporting child behavior problems in a clinically significant range on both the ECBI Intensity Scale, the ECBI Problem Scale, and the BASC Externalizing Composite, and in an at-risk range on the BASC Behavior Symptoms Index. After completing treatment, caregivers were reporting child behavior problems in a normal range for all four scales (see Table 10 below). Hypothesis 5 It was hypothesized that caregivers would report significantly less stress following treatment completion. Specifically, it was expected that caregivers would have significantly lower scores on the PSI-SF Total Stress Scale at post-treatment than at pre- treatment. Caregivers did report significantly less parenting stress on the PSI-SF Total Stress Scale after completing treatment (M = 78.33, SD = 23.25) than when they entered treatment (M = 99.13, SD = 22.44), F (1, 14) = 28.59, p < .001; Wilks? Lambda = .242; partial eta squared = .671. Prior to beginning treatment, on average, caregivers reported that they were experiencing a level of overall stress that was in a clinically significant 56 range. After completing treatment, caregivers reported experiencing a level of stress below the cutoff for clinical significance (see Table 10 below). Table 10. Repeated Measures MANOVA for Caregiver Self-Report Measures Source Measure Sum of Squares Df Mean Square F Sig. Partial Eta Squared Time ECBI Intensity 14388.30 1 14388.30 30.72 .000 .687 ECBI Problem 546.13 1 546.13 26.75 .000 .656 BASC Externalizing 1165.63 1 1165.63 19.00 .001 .576 BASC BSI 907.50 1 907.50 15.76 .001 .530 PSI Total Stress 3244.80 1 3244.80 28.59 .000 .671 Error ECBI Intensity 6558.20 14 468.44 (time) ECBI Problem 285.87 14 20.42 BASC Externalizing 858.87 14 61.35 BASC BSI 806.00 14 57.57 PSI Total Stress 1589.20 14 113.51 Repeated Measures MANOVA ? DPCIS-II Observations A repeated measures MANOVA was conducted on DPICS Composite scores to determine if caregivers? and children?s behaviors changed after completing treatment. For this analysis, 14 participants who completed treatment, and who also completed usable pre- and post-treatment DPICS observations, were included. Two participants had to be excluded from the analysis due to compromised or missing videotapes (e.g., could not hear any of dialogue between caregiver and child or tape was corrupted). It should be noted, however, that caregivers in some dyads did not have valid Direct Command Ratios for either pre- or post-treatment, as they did not give any commands during the observation. In addition, the children in some dyads did not have valid Compliance Ratios, since no commands that allowed for compliance were given. Thus, repeated measures MANOVAs were run in two steps to ensure inclusion of the largest number of 57 dyads. First, a repeated measures MANOVA was run for all DPICS-II Composite scores, which included 13 participants. Second, separate repeated measures MANOVAs were conducted for Prosocial and Inappropriate Behavior Composites, which included 14 participants, and for Direct Command and Compliance Ratio Composites, which included 13 participants. Pre- and post-treatment scores on the parent Prosocial Behavior, parent Inappropriate Behavior, child Prosocial Behavior, child Inappropriate Behavior, parent Direct Command Ratio, and child Compliance Ratio Composites for the DPICS-II were utilized in the MANOVAs. Based upon results from preliminary analyses the following demographic variables were included as covariates in the MANOVAs: ethnicity, caregiver marital status, caregiver education, and child gender. The MANOVAs were also run without these covariates. Results from all MANOVAs were comparable. Therefore, results from the MANOVAs conducted for Prosocial/Inappropriate Composites and for Direct Commands/Compliance Composites were utilized in order to maximize the number of participants in each analysis. Hypothesis 6 It was hypothesized that caregivers would engage in more prosocial and fewer negative behaviors during dyadic interactions with their children following treatment completion. Specifically, it was expected that frequencies of caregiver prosocial behaviors, as measured by the Prosocial Composite of the DPICS-II, would be significantly higher at post-treatment than at pre-treatment. It was also expected that frequencies of caregiver negative behaviors, as measured by the Inappropriate Behavior Composite of the DPICS-II, would be significantly lower at post-treatment than at pre- 58 treatment. The multivariate test of significance did not yield significant results. Therefore, individual between- and within-subjects effects could not be examined. This was the case for the MANOVA both with and without demographic covariates. There were no significant differences between the frequencies of caregivers? prosocial or negative behaviors prior to beginning treatment and at treatment completion. Hypothesis 7 It was hypothesized that caregivers? discipline skills would improve following treatment. Specifically it was expected that caregivers would have significantly higher observed Direct Command Ratios at post-treatment than at pre-treatment. The multivariate test of significance did not yield significant results. Therefore, individual between- and within-subjects effects could not be examined. This was the case for the MANOVA both with and without demographic covariates. There were no significant differences between the proportion of caregiver commands given as direct commands prior to beginning treatment and at treatment completion. Hypothesis 8 It was hypothesized that children would engage in more prosocial and fewer negative behaviors during dyadic interactions with their caregivers following treatment. Specifically, it was expected that frequencies of child prosocial behaviors, as measured by the Prosocial Composite of the DPICS-II, would be significantly higher at post- treatment than at pre-treatment. It was also expected that frequencies of child negative behaviors, as measured by the Inappropriate Behavior Composite of the DPICS-II, would be significantly lower at post-treatment than at pre-treatment. The multivariate test of 59 significance did not yield significant results. Therefore, individual between- and within- subjects effects could not be examined. This was the case for the MANOVA both with and without demographic covariates. There were no significant differences between the frequencies of children?s prosocial or negative behaviors prior to beginning treatment and at treatment completion. Hypothesis 9 It was hypothesized that children would demonstrate greater levels of compliance with adult requests, as measured by the Compliance Ratio of the DPICS-II, at post- treatment than at pre-treatment. The multivariate test of significance did not yield significant results. Therefore, individual between- and within-subjects effects could not be examined. This was the case for the MANOVA both with and without demographic covariates. There were no significant differences between children?s level of compliance with caregiver commands prior to beginning treatment and at treatment completion. Treatment Satisfaction Analysis Hypothesis 10 A treatment satisfaction analysis was conducted by calculating descriptive statistics for the Therapy Attitude Inventory. For this analysis, 15 participants who completed treatment and completed the TAI were included. One participant was excluded from the analysis due to lack of a valid or scoreable TAI. It was hypothesized that participants who completed treatment would report a high level of satisfaction with treatment that was commensurate with satisfaction levels reported in previous research on individual PCIT, as measured by the TAI total score. Participants who completed 60 treatment did report a high level of satisfaction (M = 46.93, SD = 3.47). Caregivers? scores on the TAI ranged from a minimum of 40 to a maximum of 50. Of the 15 participants included in this analysis, 12 had scores of 45 or higher, and 6 had scores of 50 (highest possible score). In addition, caregivers? reported satisfaction with treatment was commensurate with the results of previous studies, which have found average TAI scores after treatment completion that range from around 43 to 46 (Brestan et al., 1999; Schuhmann et al., 1998). Reliable Change Index Analysis A post-hoc analysis was conducted to examine the reliability of the changes caregivers reported in child behavior problems. Reliable Change Index (RCI) scores were calculated for those outcome variables that demonstrated significant change in the earlier MANOVA analyses. Calculation of an RCI is based on the standard deviation of the sample and the reliability coefficient of the given outcome measure and is calculated by dividing the difference score for a given participant by the standard error of the difference score (Jacobson, Roberts, Berns, & McGlinchey, 1999). RCIs that are greater than 1.96 are considered to be of sufficient magnitude to constitute reliable change. An RCI analysis enhances one?s understanding of treatment outcome results, because examining statistical significance alone does not account for the inherent variability in self-report measures over time (Jacobson et al., 1999). That is, participants who complete the same measure at two different time points are likely to have scores that vary somewhat based on the reliability of the measure itself. Examining RCIs allows researchers to determine the proportion of participants who report a level of change that 61 cannot be accounted for by the tendency of scores to change over time as a function of the measure used rather than of the treatment. For the present study, an RCI score was calculated for the ECBI Intensity and Problems scales, the PSI Total Stress scale, and the BASC BSI and Externalizing Composite scales. Pre-post difference scores were then calculated for each treatment completer across self-report scales, and those difference scores were compared to the RCIs for each scale to determine how many participants achieved reliable change for each measure. Of the total sample, 16 participants who completed treatment were included for the ECBI Intensity and Problem Scales, 15 were included for the BASC Externalizing Composite and Behavioral Symptoms Index, and 15 were included for the PSI-SF Total Stress Scale. One participant who completed treatment was excluded from the analysis for BASC scores and for PSI scores due to lack of valid and/or scoreable measures. Of those participants who completed treatment and were included in the analysis, 62.5% (n = 10) achieved a reliable level of change on the ECBI Intensity Scale, 50.0% (n = 8) achieved a reliable level of change on the ECBI Problem Scale, 73.3% (n = 11) achieved a reliable level of change on the BASC Externalizing Composite, 73.3% (n = 11) achieved a reliable level of change on the BASC Behavioral Symptoms Index, and 80.0% (n = 12) achieved a reliable level of change on the PSI-SF Total Stress Scale. Results of the RCI analysis are presented in Table 11 below. 62 Table 11. Reliable Change Indices for Caregiver Self-Report Measures Freq. (n) Percent (%) ECBI Intensity Scale Non-Reliable Change 6 37.5 Reliable Change 10 62.5 ECBI Problem Scale Non-Reliable Change 8 50.0 Reliable Change 8 50.0 BASC Externalizing Composite Non-Reliable Change 4 26.7 Reliable Change 11 73.3 Non-Reliable Change 4 26.7BASC Behavioral Symptoms Index Reliable Change 11 73.3 PSI-SF Total Stress Scale Non-Reliable Change 3 20.0 Reliable Change 12 80.0 63 DISCUSSION The present study attempted to address the question of whether or not PCIT administered in a group format could demonstrate effectiveness with a community sample of caregiver-child dyads. Caregiver-child dyads from the general community, both self- and DHR-referred, were enrolled in a 12-week Group PCIT program at the local Child Advocacy Center designed to enhance caregiver-child relationships and to help caregivers learn to manage child behavior problems in a more effective and appropriate way. Caregivers were observed interacting with their children and were asked to report on their children?s behaviors and their own level of stress in order to assess treatment effectiveness. Results from the present study provide preliminary support for the effectiveness and transportability of PCIT, particularly as it relates to the group format implemented in the community. Differences between Treatment Completers and Non-Completers The hypothesis that there would not be significant differences across demographic variables and pre-treatment measures between dyads who completed treatment and those who did not was supported. Treatment completers and non-completers did not differ with regard to caregiver gender, ethnicity, caregiver age, caregiver educational status, yearly family income, caregiver estimated IQ, child gender, child age, child estimated receptive language abilities, or referral source. In addition, the groups did not differ with regard to pre-treatment child abuse potential scores, pre-treatment ratings of child behavior 64 problems, or pre-treatment parenting stress. In the Group PCIT project, every effort was made to decrease barriers to treatment that might be related to family characteristics, particularly those related to socioeconomic status. Researchers provided meals for participating families, provided transportation when needed, and provided childcare for all children within the families rather than just for the target children. These steps were taken to make it easier for families with limited resources to participate in group, attend session regularly, and complete treatment. The fact that the group of participants who completed treatment and the group that dropped out of treatment did not differ significantly across such family characteristics as ethnicity, income, and educational level, or across pre-treatment measures of child and caregiver functioning, could indicate that efforts to eliminate treatment barriers related to socioeconomic status were successful. Demographic Characteristics and Treatment Outcomes Though no caregiver or child characteristics were related to caregiver-reported outcome measures, there were significant relationships between caregiver and child characteristics and observational outcome measures. Caucasian caregivers tended to have higher levels of prosocial behavior pre-treatment, and higher levels of inappropriate behavior both pre- and post-treatment, while African American caregivers tended to have lower levels of both prosocial and inappropriate behavior. In addition, caregivers who were either single or married tended to exhibit lower levels of inappropriate behavior after treatment, and their children tended to exhibit lower levels of inappropriate behavior prior to beginning treatment, whereas caregivers who were divorced or separated tended 65 to exhibit higher levels of inappropriate behavior after treatment, and their children tended to exhibit higher levels of inappropriate behavior before beginning treatment. It should be noted that ethnicity and marital status were significantly related, in that the majority of African American caregivers were single (9 of 10 participants), whereas marital status of Caucasian caregivers tended to be more evenly distributed across categories. Therefore, it is difficult to determine the implications of these findings. With regard to ethnicity, it is possible that Caucasian caregivers were simply more likely to engage in verbal behavior in general, which could increase their total number of both prosocial and inappropriate verbal behaviors. The majority of researchers for the present study were Caucasian. It is possible that African American caregivers were more reserved due to the fact that they were being observed and ?evaluated? by people from the majority culture. There is evidence to suggest that African Americans are mistrustful of behavioral and medical research in general (Murry et al., 2004). In addition, research suggests that African Americans often develop a ?healthy paranoia? with regard to mental health providers and members of the majority culture due to a history of discrimination and prejudice (Bario et al., 2003). For example, the present study took place 30 minutes away from the Tuskeegee Institute. African American caregivers in the Lee and Macon county areas may be affected by a local history of discrimination at the hands of ?helping? professionals, even if they are unaware of it. Caregivers with higher levels of education tended to engage in more prosocial behaviors after completing treatment than did caregivers with lower levels of education. 66 It is possible that caregivers who were more educated had an easier time learning what was expected of them in treatment and were therefore more likely to exhibit target behaviors (included in the prosocial category) after completing treatment. Finally, male children tended to exhibit fewer inappropriate behaviors prior to beginning treatment, while female children tended to exhibit more inappropriate behaviors. These results are inconsistent with past research, which indicates that male children are more likely to exhibit externalized behavior problems than are female children (Lumley, McNeil, Herschell, & Bahl, 2002; Maughan, Rowe, Messer, Goodman, & Meltzer, 2004). Given the small sample size in the present study, it would be premature to presume that the present findings are representative of the larger population. It is interesting to note, however, and a larger sample size in future studies might help clarify the present findings. It is possible that, at least for the present sample, caregivers may have been more likely to seek treatment for male children regardless of their level of behavior problems, but that a higher level of disruptive behavior was required before caregivers would seek treatment for female children. Changes in Child Behavior Problems across Sessions The hypothesis that caregivers would report a significant decrease in child behavior problems, as measured by the ECBI Intensity and Problem Scales, across sessions was partially supported. As sessions progressed, caregivers did report decreases in ECBI Intensity Scores. Time (as measured by session number) was a significant predictor of ECBI Intensity Scores. This is in keeping with previous research on PCIT and with general expectations for a parent-training program. One would expect that, as 67 sessions progress and parents learn and practice new skills to address child behavior problems, the frequency of those problem behaviors would decrease. In the present study, this seems to have been the case. As dyads progressed through treatment, the frequency of child behavior problems, as reported by caregivers, decreased. These results might indicate that over time caregivers were gaining skills that improve their ability to manage their children?s behavior. However, there was no relationship between time and ECBI Problem scores, indicating that though caregivers reported a decrease over time in the frequency of disruptive child behaviors, they did not consider those behaviors to be any less of a problem for them. On average, caregiver-child dyads that enrolled in the study completed 7.6 sessions, and they stayed in the group program for an average of 8.7 weeks (out of 12 weeks). However, this average is skewed by the fact that 16 of the 27 dyads completed treatment, attending at least 10 sessions and staying in the program for 12 weeks. Dyads that dropped out prior to completing treatment attended an average of 3 sessions and stayed in the program for an average of 4 weeks. It is possible that during the time they spent in treatment, caregivers felt that their children?s behavior was improving but they still felt their children?s behavior was problematic for them. In other words, even a small dose of treatment might have resulted in a perceived decrease in disruptive behavior, but caregivers may have felt that those behaviors, while slightly less frequent, were still problematic. 68 Parenting Stress as a Predictor of Treatment Response and Retention The hypothesis that high pre-treatment levels of caregivers? parenting stress would predict poor treatment response and retention was not supported. Parenting stress was not a significant predictor of treatment retention measures. Thus, there was no relationship between caregivers? stress prior to beginning treatment and the length of time they remained in treatment. These findings seem inconsistent with past research, which has shown that parents who are experiencing higher levels of stress, and more life stressors in general, when they enter treatment are more likely to drop out before completing treatment (Gurwitch et al., 2006; Kazdin, Holland, & Crowley, 1997). However, studies indicate that there are a number of parental factors related to treatment dropout, including social isolation, lower socioeconomic status (SES), and barriers to treatment accessibility, which may be related to both SES and social isolation (Reyno & McGrath, 2006; Taylor & Biglan, 1998). In addition, Kazdin et al. (1997) found that when parents perceived fewer barriers to treatment, that perception attenuated dropout. It is possible that for parents in the current sample factors other than parenting stress played a significant role in treatment retention. For example, it could be that contributors to treatment dropout, such as parenting stress, were counterbalanced by efforts to decrease barriers to treatment and by possible increases in social support that might have been a natural outcome of the group format. From results of the present study, it is not possible to determine with any certainty if this is the case or to determine what role, if any, these factors might have played in treatment retention. Future research on group PCIT should examine factors such as perceived social support and perceived 69 barriers to treatment in addition to parenting stress in order to gain a clearer picture of the factors related to treatment dropout and retention. Though pre-treatment levels of caregiver stress did not predict treatment retention, parenting stress did partially predict response to treatment. Pre-treatment parenting stress was a significant predictor of change over time in caregiver reports of the frequency of child behavior problems. Stress levels accounted for 18.8% of the variability in the difference between caregiver reports of child disruptive behavior prior to beginning treatment and at the last session attended. However, the relationship between caregiver stress and treatment response was in the opposite direction expected. In keeping with previous findings on the relationship between parenting stress and treatment response (Gurwitch et al., 2006), it was expected that high levels of stress before beginning treatment would predict fewer treatment gains in terms of child behavior problems. Contrary to expectations, in the present study, caregivers who reported higher levels of stress when starting treatment actually reported greater decreases in the frequency of their children?s behavior problems. In other words, the more stress caregivers reported when beginning treatment, the more improvement they reported in their children?s disruptive behaviors. These findings seem contradictory. One might expect that high levels of stress in caregivers would interfere with their engagement in treatment, their ability to follow through with therapeutic assignments, and therefore, their response to treatment. However, this did not appear to be the case for the present sample. Why did caregivers in the present sample who were experiencing more stress report more improvement in their 70 children?s behavior? It is possible that they reported greater improvement simply because they could. Previous research has found that caregiver stress and caregiver reports or child behavior problems are significantly related, in that the more disruptive behaviors children exhibit the more parenting stress caregivers report experiencing (Eyberg, Boggs, & Rodriguez, 1992). In fact, this does appear to be the case for the present sample. Pre-treatment parenting stress and pre-treatment child behavior problems on the ECBI Intensity Scale were highly positively correlated (r = .754, n = 27, p < .001), indicating that caregivers who reported higher levels of stress also reported higher levels of child behavior problems. By extension, the more disruptive child behavior caregivers are reporting, the more room for improvement that exists. In other words, the potential for change is greater for children with more behavior problems, and the greater their scores on the ECBI Intensity Scale, the more those scores might be able to drop. If caregivers are reporting relatively low levels of behavior problems, there is less room for improvement. Thus, the relationship between pre-treatment stress and reported improvement in behavior problems could be accounted for by pre-treatment levels of child behavior problems and the related potential for change rather then to the effects of parenting stress directly. However, it is also possible that caregivers who were experiencing higher levels of stress related to their children?s disruptive behaviors were more invested and engaged in treatment. Previous researchers have speculated that the opposite would be the case. In the present study, significant efforts were made to decrease barriers to treatment. Caregivers were provided with comprehensive childcare, dinner, and transportation when 71 needed in order to make it easier for them to attend group. As previously discussed, it is possible that these efforts mitigated the negative effects of parenting stress on caregivers? tendency to drop out of treatment. It is also possible that efforts to decrease barriers to treatment mitigated the effects of parenting stress on caregivers? ability to participate effectively in treatment. For the present study, caregivers who were extremely stressed in relation to their children?s behavior may have been more motivated to engage in treatment and more able to do so due to a decrease in treatment barriers. In order to examine whether efforts to decrease barriers mitigated the effects of stress, future research would need to compare outcomes for caregiver samples that did and did not receive assistance, such as childcare, dinner, and transportation. Pre-treatment levels of parenting stress were not related to changes in ECBI Problem scores. Caregivers? stress levels prior to beginning treatment did not predict change over time in how problematic caregivers perceived their children?s disruptive behaviors to be. Just as time was not a significant predictor of ECBI Problem scores, it is possible that though caregivers reported a decrease in the frequency of disruptive behaviors, they still perceived those behaviors as equally problematic regardless of their level of parenting stress. It is also possible that these results are related to the nature of the ECBI Problem Scale itself. Unlike the ECBI Intensity Scale, which has a range of potential scores from 36 to 252, the ECBI Problem Scale has a smaller range of potential scores, with a minimum of 0 and a maximum of 36. Thus, the lack of findings could be due to a restricted range of scores and restricted potential variability. 72 Treatment Outcomes for Caregiver-Reported Behavior The hypothesis that caregivers would report significantly fewer behavior problems and significantly less parenting stress after completing treatment than they did prior to beginning treatment was supported. Caregivers who completed treatment reported significant decreases in the number of disruptive behaviors their children exhibited. In addition, they reported significant decreases in the frequency of those disruptive behaviors and their perceptions of how problematic their children?s behaviors were. Thus, caregivers who completed the group PCIT program perceived their children?s behavior as significantly improved by the end of the group. In addition, caregivers who completed treatment reported a significant decrease in the amount of parenting stress they were experiencing. Thus, caregivers who completed the group PCIT program felt that their own stress related to their children?s behavior, their parental distress, and their relationships with their children was significantly improved by the end of the group. When evaluating clinical treatment outcomes, it is important to look at not only statistically significant change but also clinically significant change (Kazdin, 1992). That is, are the statistical differences found in a treatment sample meaningful on a clinical level. At the beginning of treatment, the present sample of caregivers reported levels of child disruptive behavior and parenting stress that were, on average, above cutoffs for clinical significance and were at levels that would be expected from a clinical population (i.e., a population of children exhibiting disruptive behavior disorders). However, when they finished treatment, caregivers not only reported significant improvement in child 73 behavior problems and parenting stress, but they reported improvement to levels below clinical cutoffs and commensurate with normative populations. These findings indicate that, on average, the changes reported by caregivers in the present study were not only statistically significant, but that they were also clinical significant. There are a multitude of research studies demonstrating the efficacy of individual PCIT with regard to decreasing reports of both child disruptive behavior and parenting stress when compared to other treatment modalities or to no treatment. The present findings are consistent with past research on PCIT, indicating that the group PCIT program was able to effect change with caregivers and children related to perceived behavior problems and stress. Research on the efficacy of PCIT has examined its use with a variety of clinical populations, but such research almost exclusively has examined PCIT as administered in the traditional individual format. The results of the present study add to the literature in this area by demonstrating that PCIT administered in a group format with a sample of families that may be more representative of those seen in community mental health settings can produce similar outcomes on caregiver reports of child behavior problems and their own distress as PCIT administered in a more traditional format. The present study lends support to the effectiveness and transportability of group PCIT with community families. Treatment Outcomes for Observed Caregiver and Child Behavior It was expected that caregivers? and children?s behavior when interacting with each other would be significantly different after completing treatment. With regard to caregivers? behavior, the hypothesis that caregivers would engage in more prosocial and 74 fewer negative behaviors following treatment was not supported. There was no significant change in either the frequency of caregiver prosocial behaviors or the frequency of caregiver negative behaviors from pre- to post-treatment, indicating that caregivers were exhibiting a similar number of prosocial and negative behaviors when they completed the group program as they had when they began it. In addition, the hypothesis that caregiver?s discipline skills would improve following treatment was not supported. It was expected that after completing treatment, caregivers would be more likely to give effective, direct commands with their children than they had been when they started treatment. However, there was no significant difference between the ratio of commands caregivers gave as direct rather than indirect when they completed treatment than when they began it. With regard to children?s behavior, the hypothesis that children would engage in more prosocial and fewer negative behaviors following treatment was not supported. There was no significant change in either the frequency of child prosocial behaviors or the frequency of child negative behaviors from pre- to post-treatment, indicating that children were exhibiting a similar number of prosocial and negative behaviors when they completed the group program as they had when they began it. In addition, the hypothesis that children would demonstrate greater levels of compliance with adult requests following treatment was not supported. There was no significant difference between the ratio of commands given by caregivers with which children complied rather than refused to comply when they completed treatment than when they began it. 75 It appears that for the present sample, though caregivers reported significant improvement in their children?s behavior, such improvement was not seen during clinic observations. In addition, it appears that for the present sample, no improvement was seen with regard to parenting behavior during clinic observations. These findings are inconsistent with both caregiver reports from this study and with past research examining the efficacy of PCIT. Researchers examining outcomes for individual PCIT have found significant improvement in parent and child behaviors during DPICS observations. An examination of the data set revealed that mean child and parent frequencies for a select set of the DPICS-II categories were comparable to those for the normative sample of clinic-referred children in both CDI and PDI segments (Eyberg et al., 1994). Thus, the lack of findings cannot be accounted for by a predisposition for the current sample of dyads to be excessively negative, to exhibit a lack of positive behaviors, or to exhibit a lack of verbal behavior in general. There are a number of potential reasons that previous research findings were not replicated in this study. It is, of course, possible that caregivers perceived change where there was not any. It is also possible that caregivers? perceptions of their children?s behavior changed more than did the behavior itself. In other words, caregivers may have viewed their children?s behavior differently after completing treatment. Generally, PCIT is thought of as not only increasing effective discipline skills in caregivers but also as a treatment that improves the warmth and attachment between caregivers and children. By improving attachment and warmth, it is thought that caregivers and children will begin to see each other in a more positive light. In addition, the nature of the group PCIT format 76 means that caregivers observe each other and each other?s children interacting. Having the opportunity to see other people?s children, and thus observe other children who are likely to engage in disruptive behaviors, might have had an effect on the way that caregivers viewed their own children?s behaviors and on their tolerance for disruptive behavior. It would seem that if caregivers? tolerance and acceptance of negative behaviors in their children increased, then their reports of child behavior problems would decrease. However, in order to examine this possibility, ratings of parental tolerance and collateral ratings of child behavior (e.g., teacher or other non-participating caregiver ratings) would be necessary. It should also be noted that the current sample of treatment completing families was relatively small. Thus, it is also possible that the size of the present sample was just too small to detect significant change. A larger sample of dyads might result in different findings. In addition, due to the small size of the present sample, behavioral composite categories for parents and children had to be collapsed across situations. In other words, frequency counts were totaled across child-directed, parent-directed, and clean-up segments of the DPICS observations. There were simply not enough participants to examine composite categories in each situation. Therefore, it is possible that by collapsing categories across these situations, information regarding caregiver and child behavior was lost. For example, one would expect parent prosocial behavior to be different in the child-directed portion of the observation than in the clean-up portion. By extension, one might expect differences in parent commands and child compliance during 77 clean-up than during other segments. Being able to examine each section of the DPICS observations separately might result in different results. It is also possible that the DPICS observations were simply not representative of the caregivers? and children?s ?real? behaviors at home, and that there was change in caregiver and child behaviors that was simply not reflected in the clinic observations. Kazdin (1992) suggests that one limitation of behavioral observations is that the observed behaviors are not necessarily representative of the behaviors that occur during times when no observations are taken. In addition, Kazdin (1992) argues that behavioral observations conducted in contrived situations may pull for demand characteristics in subjects or may include or exclude certain stimuli that affect the way subjects behave. However, it must be acknowledged that past PCIT research has demonstrated changes in behavior using DPICS observations. Even so, the nature of group PCIT is very different in some ways than the nature of individual PCIT. In individual PCIT, caregivers are required to meet certain behavioral criteria in the child-directed part of treatment before they can move on to the discipline phase. They are also required to meet criteria before they can ?graduate? from treatment. This process typically takes 16 to 20 weeks to complete. Thus, the dose of treatment provided by the group (12 weeks) is shorter than what is typically provided in individual PCIT. In addition, the group nature of treatment in the present study did not allow for criteria requirements. There was really no way to require that all caregivers meet criteria before moving on to the next phase, since it is unlikely they would all meet criteria at the same time, and some caregivers might do so quite early while others might never do so. Thus, caregivers were allowed to continue 78 through the planned progression of sessions regardless of their skill level. It is possible, then, that caregivers simply did not receive a high enough dosage of treatment to effect enough change in their behavior to be detected as significant in the MANOVA. It should also be noted that in individual PCIT parents are expected to engage in child- or parent-directed play with their children for approximately 30 minutes at a time. In group PCIT, however, there are more parents, and thus less time to coach each parent, during weekly sessions. Typically during a group session, individual caregivers received 10 to 15 minutes of individual coaching with their children. It is possible that parents in individual PCIT become more used to engaging in the behaviors taught during treatment (and measured by composite categories) for longer periods of time than do parents in group PCIT. Given that the DPICS observations take approximately 30 minutes to conduct, it is possible that caregiver-child dyads in the present sample simply got fatigued more easily, which might account for the lack of observed differences in behavior. It is only the second, fourth, and fifth 5-minute segments of each observation that are coded using the DPICS. Thus the first five minutes of the interaction are not typically examined. Dolbear et al. (2006) examined the ?warm-up? segments of DPICS observations for a sample of the participants from the current study and found preliminary evidence that caregivers and children did seem to ?fatigue?. For example, caregivers exhibited fewer prosocial behaviors during the coded segments than during the warm-up segments. In addition, though not statistically significant, children tended to engage in more negative behaviors during coded segments than in warm-up segments. 79 Thus, it is possible that the very nature of the DPICS observation itself contributed to the lack of observed change in caregiver and child behaviors. Treatment Satisfaction The hypothesis that caregivers who completed treatment would report a high level of satisfaction with treatment, commensurate with previous research on PCIT was supported. The Treatment Attitude Inventory (TAI) is a measure of treatment satisfaction that has a possible range of scores from 10 to 50. In previous PCIT research studies using the TAI, caregivers have reported average scores ranging from 43 to 46. The current sample of caregivers who completed treatment reported an average score of around 47 (range = 40 to 50), which is slightly higher than the average score reported in previous research. These results indicate that caregivers who completed treatment reported a high degree of satisfaction with the group PCIT program, which would be expected given the significant decreases caregivers reported in both child behavior problems and their own stress levels. The results also indicate that caregivers who participate in group PCIT are at least as satisfied with their treatment experience as are caregivers who participate in individual PCIT. Evidence suggests that for some caregivers, particularly minorities, group programs are more acceptable than individual therapy, and they are more likely to engage in group than in individual treatment modalities (Cunningham et al., 1995; Kazdin, 1997; Taylor & Biglan, 1998). The current findings provide support for group PCIT as a palatable alternative to the individual format. 80 Reliability of Treatment Change A reliable change index analysis was conducted in order to determine how many caregivers who completed treatment achieved a reliable level of change across treatment measures. The results of the RCI indicate that at least 50% of treatment completers achieved reliable change regarding reports of child behavior across all measures, and at least 70% achieved reliable change on the BASC measures of child behavior problems. With regard to parenting stress, 80% of completers reported reliable change with regard to parenting stress, indicating that the vast majority of caregivers who completed treatment reported that they were experiencing significantly less stress related to parenting after completing the group. It is interesting to note that a higher percentage of caregivers reported reliable change in their own levels of parenting stress than reported such change in their children?s behavior. It is possible that even small perceived changes in child behavior problems resulted in a large decrease in parenting stress. For example, caregivers who enrolled in the groups would often comment that they had ?tried everything? and nothing had helped them, or that they felt as if nothing would be able to help them manage their children?s behavior. For such parents, even small perceived changes in child behavior might provide hope to caregivers that things can improve, which might significantly decrease their stress even if the resultant child behavior changes are relatively small. In addition, the nature of the group format may have contributed more to decreases in parenting stress than to changes in child behavior. The phenomena of seeing other caregiver-child dyads with similar problems and receiving social support from other 81 caregivers are unique to the group format. Caregivers in individual PCIT do not typically get exposed to other caregivers in treatment. It is possible that just knowing they are not alone and receiving encouragement and support from other caregivers going through similar difficulties with their children provide a significant source of relief for caregivers. The group format may supply a valuable element that caregivers cannot get from individual PCIT. Limitations of Study Design and Directions for Future Research The present results and their interpretations should be considered in the context of several important limitations. A primary and significant limitation of this study is related to characteristics of the sample. The sample of caregiver-child dyads in the present study is relatively small. In particular, the sample of treatment completers only included 16 dyads. Though the sample size was enough to satisfy requirements for the MANOVA analyses, such a small sample size could result in a significant decrease in power and an increase in the likelihood of a Type II error. For example, with regard to observational data, it is possible that significant changes in child and caregiver behaviors occurred but that the current sample size was not large enough to detect them. It should also be noted, however, that due to the large number of independent analyses conducted for the current study, there is a high probability of a Type I error. Though the repeated measures MANOVA analyses were employed to correct for the increased likelihood of a Type I error, no such correction was employed for the regression analyses run for the study. The estimated likelihood of a Type I error for the present study was approximately 99% with an alpha level of .05. Employing a 82 Bonferroni correction results in an alpha level required for significance of p = .005. If this standard is applied to the regression analysis in the present study examining the relationship between ECBI Intensity scores and time (session progression), then the results would still be significant, as the observed p-value for the regression was .002. However, the same would not be true for the regression examining the relationship between pre-treatment parenting stress and magnitude of change on the ECBI Intensity Scale, since the observed p-value for that analysis was .006. At this level of significance, the probability of a Type I error is 6%. However, the results of the regression should be still be interpreted with caution given the high number of analyses run for the current study. In addition to concerns regarding Type I and II errors, there are other limitations related to sample size. It is possible that because the current sample was so small, it was not representative of the population at large. However, it could be argued that traditional research using highly screened samples is not representative of the population at large, regardless of sample size, and that the present sample may be more representative than larger samples from more restrictive research studies. Researchers in the current study endeavored to employ a more representative sample by recruiting participants from the community without employing the restrictive screening strategies commonly found in efficacy research. Due to the small nature of the current sample, little variability existed across demographic variables. For example, the sample consisted almost exclusively of female caregivers. Previous PCIT research has demonstrated the importance of including fathers 83 for long-term treatment outcomes (Bagner & Eyberg, 2003). Male caregivers are consistently underrepresented in family research, and future research would benefit from using alternative recruitment strategies that address the needs and obstacles of paternal caregivers (Phares, 1996). The current group PCIT project did include both the maternal and paternal caregivers for three families. However, only one caregiver from each dyad was included in the present research sample in order to eliminate the potential confound resulting from a lack of independence in caregiver reports and observations. Though there were not enough father-mother pairs to allow for a comparison of results between caregivers within families, it would be interesting to recruit multiple caregivers for a given family in future research in order to allow for such comparisons. Future research should endeavor to recruit a larger, more diverse sample of caregivers and children. Though it was beyond the scope of this study, given the fact that demographic variables, such as gender, ethnicity, marital status, and caregiver-child relationship, were correlated with some of the observational outcome measures, future research would benefit from employing recruitment strategies designed to increase the variability of participants across these categories. A more diverse sample could allow for a more thorough examination of potential differences and might shed light on whether the correlations found in the present study were an artifact of the study sample or whether such differences might also be found in other sample populations. A secondary limitation of this study is related to measurement strategies. The initial study design provided for a wait-list control group to serve as a comparison for the group of treatment completers. In addition, it was intended that dyads who dropped out 84 of treatment would continue to complete research measures so as to compare outcomes between dyads who successfully completed the group and those who did not. However, there was not enough response to allow for either a wait-list group or for a comparison of post-treatment functioning for dropouts versus completers. Thus, there is no way to ensure that treatment gains reported by caregivers are a result solely of the treatment itself. It is possible that parents would have reported improvement regardless of their treatment status. Future research should endeavor to address this limitation by employing a control group to provide a way to compare outcomes over time for people who receive treatment and those who do not. Niec and colleagues are addressing such limitations at present in their randomized control trial comparing group and individual PCIT. In addition, the present study is limited by the fact that no follow-up data were available to determine if the treatment gains reported by parents were robust and persisted over time. In the present study, only three participating caregivers responded to requests for 3 and 6-month follow-up data. In the future, more aggressive recruitment and retention measures might be needed to increase the follow-up response for both treatment completers and treatment dropouts. Another measurement concern is the fact that caregiver-report data on child behavior were only collected from caregivers participating in the group. As previously discussed, one possible explanation for the lack of findings with regard to observed parent and child behavior may be that caregivers? perceptions of their children?s behavior changed while the actually behavior did not change. However, in order to determine whether this explanation is likely, researchers would need to collect reports of child 85 behavior from collaterals not involved in treatment. Though gathering collateral reports was beyond the scope of this study, future research could include reports from other adults who play a significant role in each child?s life (i.e., teachers, other caregivers), in order to gain a more complete picture of the child?s behavior. There are also measurement concerns related to the nature of the DPICS observations. As previously discussed, each DPICS observation includes two 5-minute warm-up segments: one at the beginning of the CDI interaction and one at the beginning of the PDI interaction. These segments were included in an effort to allow caregivers to acclimate to the observational situation before coding began. However, to date no empirical research exists examining the utility of the warm-up segments. In the group format, when caregivers are used to shorter periods of play, it is possible that they ?burnout? more quickly, exhibiting fewer behaviors as time progresses, which might account for a lack of findings with regard to observed behavior. In fact, preliminary research results support the idea that caregivers? and children?s behaviors are different in the warm-up than in the coded segments of the DPICS observation (Dolbear et al., 2006). Having a shorter warm-up period that only occurs at the beginning of the interaction, then going straight into the CDI and then PDI situations, might provide the same benefits without increasing the likelihood of ?burnout?. A final measurement-related limitation of the present study relates to the data collection process. Due to fact that the current study employed a community sample with resultant complicating stressors and barriers to retention, caregivers occasionally failed to complete all measures across all sessions. For example, some caregivers did not attend 86 all sessions or reported that they were unable to stay to complete measures but would return another day to do so, which did not always occur. In addition, unforeseen complications corrupted some of the observational data. Thus, the present data set is incomplete, which could have an effect on the resultant analyses. Unique Contributions and Implications of Present Study The present study provides an important first step toward effectiveness research for group PCIT. Though some clinicians have begun using PCIT in a group format, very little research exists examining the group model of PCIT. It is not known if the group format, as currently designed, has the same efficacy as individual PCIT, or if the group format can be effectively implemented with more representative community families. The present study begins to answer these questions by providing preliminary evidence that the currently designed group format can effect change for caregivers and children who are more representative of those that might be referred for treatment in the general community. Caregivers reported significant decreases in both their own parenting stress and their children?s disruptive behavior after completing the group. In addition, caregivers reported being highly satisfied with the group and its outcomes for them and their children. These findings indicate that group PCIT may provide a more cost-effective alternative to individual PCIT that is acceptable to families that might not otherwise seek treatment. Caregivers in group PCIT may also gain benefits that are not available from the individual format. For example, in the group caregivers are exposed to other families with children who have behavior problems and are provided the opportunity to problem 87 solve with and support each other with the help of therapist facilitators. Anecdotally, some families in the current group project reported continuing to stay in touch with and provide parenting support to each other more than six months after completing the group. In addition, the most robust findings of the present study were related to caregivers? reported reduction in parenting stress. Given the reported importance of decreasing caregiver stress in parent training programs (Gurwitch et al., 2006; Haskett et al., 2006; Kazdin et al., 1992), the fact that the group is able to do so, and may contribute something that other PCIT formats cannot, is hopeful. However, despite reported improvement in both stress and child behavior, caregivers? and children?s actual behavior did not appear to change in the present study. This contradiction has implications for the format of group PCIT. These findings do not necessarily mean that group PCIT cannot effect observable behavioral change, but that the group format as currently designed may not provide a large enough treatment dose. Perhaps adjusting the group format would allow for the retention of the positive stress- reducing elements found for the current group project while also increasing its effectiveness with regard to changing child and caregiver behavior. For example, extending the length of the group so that it is more reflective of the length of individual PCIT might provide the increased dose needed to effect similar behavioral changes as have been seen in individual PCIT. In addition, having separate CDI and PDI groups, such that caregivers would be required to meet criteria in the first to ?graduate? to the second, might be another alternative to ensure parents are reaching the same criteria in group PCIT as is required in the individual format. 88 Despite its limitations, the present study begins to set the stage for effective, well- controlled dissemination of an efficacious, promising intervention for troubled families. Given the growth in research for individual PCIT with regard to various populations, ethnic groups, and presenting disorders, it seems possible that the same growth will manifest itself for group PCIT. It will be interesting to follow the evolution and progression of the group format and the research that examines both its efficacy and effectiveness. 89 REFERENCES Abidin, R. R. (1995). Parenting Stress Index (PSI) manual (3 rd ed.). Charlottesville, VA: Pediatric Psychology Press. Addis, M., & Waltz, J. (2002). Implicit and untested assumptions about the role of psychotherapy treatment manuals in evidence-based practice. Clinical Psychology: Science and Practice, 9, 421-424. Aragona, J. A., & Eyberg, S. M. (1981). Neglected children: Mother?s report of child behavior problems and observed verbal behavior. Child Development, 52, 596- 602. Assemany, A. E., & McIntosh, D. E. (2002). Negative treatment outcomes of behavioral parent training programs. Psychology in the Schools, 39, 209-219. Bagner, D., & Eyberg, S. M. (2003). Father involvement in parent training: When does it matter? Journal of Clinical Child and Adolescent Psychology, 32, 599-605. Bagner, D., & Eyberg, S. M. (2007). Parent-Child Interaction Therapy for disruptive behavior in children with mental retardation: A randomized controlled trial. Journal of Clinical Child and Adolescent Psychology, 36, 418-429. Barrio, C., Yamada, A., Atuel, H., Hough, R.L., Yee, S., Berthot, B., & Russo, P.A. (2003). A tri-ethnic examination of symptom expression on the positive and negative syndrome scale in schizophrenia spectrum disorders. Schizophrenia Research, 60, 259-269. 90 Bessmer, J. L., Brestan, E. V., & Eyberg, S. M. (2007). The Dyadic Parent-Child Interaction Coding System II (DPICS II): Reliability and validity with Mother- Child Dyads. Manuscript submitted for publication. Bigfoot, D., Funderburk, B., & Gurwitch, R. (2006, January). Dissemination of PCIT for American Indians and Native Alaskans. In K. McCabe (Chair), PCIT with diverse cultural groups. Symposium conducted at the 6 th Annual Parent-Child Interaction Therapy Conference, Gainesville, Florida. Brestan, E. V., & Eyberg, S. M. (1998). Effective psychosocial treatments of conduct- disordered children and adolescents: 29 years, 82 studies, and 5,272 kids. Journal of Clinical Child Psychology, 27, 180-189. Brestan, E. V., Eyberg, S. M., Boggs, S. R., & Algina, J. (1997). Parent-Child Interaction Therapy: Parents? perceptions of untreated siblings. Child and Family Behavior Therapy, 19, 13-28. Brestan, E. V., Foote, R. C., & Eyberg, S. M. (2007). The Dyadic Parent-Child Interaction Coding System II (DPICS II): Reliability and validity with father-child dyads. Manuscript submitted for publication. Brestan, E. V., Jacobs, J. R., Rayfield, A. D., & Eyberg, S. M. (1999). A consumer satisfaction measure for parent-child treatments and its relation to measures of child behavior change. Behavior Therapy, 30, 17-30. Brestan, E. V., Valle, L. A., Funderburk, B. W., Niec, L. N., & Yopp, J. M. (2003). Group Parent-Child Interaction Therapy: A clinician?s guide. Unpublished manuscript. 91 Brinkmeyer, M. Y., & Eyberg, S. M. (2003). Parent-Child Interaction Therapy for oppositional children. In A.E. Kazdin & J.R. Weisz (Eds.), Evidence-based psychotherapies for children and adolescents (pp. 204-223). New York: Guilford Press. Burns, B. J., Hoagwood, K., & Mrazek, P. J. (1999). Effective treatment for mental disorders in children and adolescents. Clinical Child and Family Psychology Review, 2, 199-254. Chaffin, M., Silovsky, J. F., Funderburk, B., Valle, L. A., Brestan, E. V., Balachova, T., et al. (2004). Parent-Child Interaction Therapy with physically abusive parents: Efficacy for reducing future abuse reports. Journal of Consulting and Clinical Psychology, 72, 500-510. Choate, M. L., Pincus, D. B., Eyberg, S. M., & Barlow, D. H. (2005). Parent-Child Interaction Therapy for treatment of Separation Anxiety Disorder in young children: A pilot study. Cognitive and Behavioral Practice, 12, 126-135. Chorpita, B. F. (2003). The frontier of evidence-based practice. In A. E. Kazdin & J. R. Weisz (Eds.), Evidence-based psychotherapies for children and adolescents (pp. 42-59). New York: Guilford Press. Colvin, A., Eyberg, S. M., & Adams, C. D. (1999). Restandardization of the Eyberg Child Behavior Inventory. Unpublished manuscript. Cunningham, C. E., Bremner, R., & Boyle, M. (1995). Large group community-based parenting programs for families of preschoolers at risk for disruptive behaviour 92 disorders: Utilization, cost effectiveness, and outcome. Journal of Child Psychology & Psychiatry, 36, 1141-1159. Deater-Deckard, K., & Scarr, S. (1996). Parenting stress among dual-earner mothers and fathers: Are there gender differences? Journal of Family Psychology, 10, 45-59. Deskins, M. M. (2005). The dyadic parent-child interaction coding system II: Reliability and validity with school-aged dyads (Doctoral dissertation, Auburn University, 2005) Dissertation Abstracts International: Section B: The Science and Engineering, 66, 2302. Dolbear, P., Flory, D., Luscomb, A., Painter, D., Stahl, K., White, E., Ridgeway, L., & Brestan, E. (2006, January). Analysis of the utility of the warm-up segments of DPICS-II observations. Poster presented at the Sixth Annual Parent-Child Interaction Therapy Conference, Gainesville, Florida. Dunn, L. M., & Dunn, L. M. (1997). Manual for the Peabody Picture Vocabulary Test (PPVT-III; 3rd ed.). Circle Pines, MN: American Guidance Services. Eyberg, S. M. (1988). Parent-Child Interaction Therapy: Integration of traditional and behavioral concerns. Child and Family Behavior Therapy, 10, 33-46. Eyberg, S. M. (1993). Consumer satisfaction measures for assessing parent training programs. In L. Van de Creek, S. Knapp, & T. L. Jackson (Eds.), Innovations in clinical practice: A source book (Vol. 12) (pp. 377-382). Sarasota, FL: Professional Resource Press. 93 Eyberg, S., Bessmer, J., Newcomb, K., Edwards, D., & Robinson, E. (1994). Dyadic Parent-Child Interaction Coding System ? II: A manual (Social and Behavioral Sciences Documents Ms. No. 2897). Eyberg, S. M., Boggs, S. R., & Rodriguez, C. M. (1992). Relationships between maternal parenting stress and child disruptive behavior. Child & Family Behavior Therapy, 14, 1-9. Eyberg, S. M., Edwards, D., Bessmer, J., & Litwins, N. (1994). The workbook: A coder training manual for the Dyadic Parent-Child Interaction Coding System (DPICS- II). Unpublished manuscript. Eyberg, S. M., & Johnson, S. M. (1974). Multiple assessment of behavior modification with families: Effects of contingency contracting and order of treated problems. Journal of Consulting and Clinical Psychology, 42, 594-606. Eyberg, S. M., Nelson, M. M., Duke, M., & Boggs, S. R. (2005). Manual for the Dyadic Parent-Child Interaction Coding System (3rd ed.). Retrieved July 28, 2006 from The University of Florida, Parent-Child Interaction Therapy Web site: www.pcit.org. Eyberg, S. M., & Pincus, D. (1999). Eyberg Child Behavior Inventory and Sutter-Eyberg Student Behavior Inventory: Professional manual. Odessa, FL: Psychological Assessment Resources. Feindler, E. L., Rathus, J. H., & Silver, L. B. (2003). Assessment of family violence. Washington, D. C.: American Psychological Association. 94 Forehand, R., & McMahon, R. J. (1981). Helping the noncompliant child: A clinician?s guide to training. New York: Guilford Press. Funderburk, B. W., Eyberg, S. M., Newcomb, K., McNeil, C. B., Hembree-Kigin, T., & Capage, L. (1998). Parent-Child Interaction Therapy with behavior problem children: Maintenance of treatment effects in the school setting. Child and Family Behavior Therapy, 20, 17-38. Gurwitch, R., Mulvihill, J. J., & Chaffin, M. (2006, January). Group PCIT for children with Fetal Alcohol Syndrome. In D. Pincus (Chair), PCIT with special diagnostic groups. Symposium conducted at the 6 th Annual Parent-Child Interaction Therapy Conference, Gainesville, Florida. Hanf, C. (1969). A two stage program for modifying maternal controlling during the mother-child interaction. Paper presented at the meeting of the Western Psychological Association, Vancouver, British Columbia. Harwood, M. D., & Eyberg, S. M. (2006). Child-Directed Interaction: Prediction of change in impaired mother-child functioning. Journal of Abnormal Child Psychology, 34, 335-347. Haskett, M. E., Ahern, L. S., Ward, C. S., & Allaire, J. C. (2006). Factor structure and validity of the Parenting Stress Index-Short Form. Journal of Clinical Child and Adolescent Psychology, 35, 302-312. Hayes, S. C. (2002). Getting to dissemination. Clinical Psychology: Science and Practice, 9, 410-415. 95 Hembree-Kigin, T., & McNeil, C. B. (1995). Parent-Child Interaction Therapy. New York: Kluwer Academic/Plenum Publishing. Hinton, P. R., Brownlow, C., McMurray, I., & Cozens, B. (2004). SPSS explained. London: Routledge. Hoag, M. J., & Burlingame, G. M. (1997). Evaluating the effectiveness of child and adolescent group treatment: A meta-analytic review. Journal of Clinical Child Psychology, 26, 234-246. Hoagwood, K. (2002). Making the translation from research to its application: The je ne sais pas of evidence-based practices. Clinical Psychology: Science and Practice, 9, 210-213. Hoagwood, K., Burns, B. J., Kiser, L., Ringeisen, H., & Schoenwald, S. K. (2001). Evidence-based practice in child and adolescent mental health services. Psychiatric Services, 52, 1179-1189. Hogan, M. F. (2002). A partly built bridge between science and services: Commentary on the Hawaii Empirical Basis to Services Task Force. Clinical Psychology: Science and Practice, 9, 200-203. Hood, K. K., & Eyberg, S. M. (2003). Outcomes of Parent-Child Interaction Therapy: Mothers? reports of maintenance three to six years after treatment. Journal of Clinical Child and Adolescent Psychology, 32, 419-429. Jacobson, N. S., Roberts, L. J., Berns, S. B., & McGlinchey, J. B. (1999). Methods of defining and determining the clinical significance of treatment effects: 96 Description, application, alternatives. Journal of Consulting and Clinical Psychology, 67, 300-307. Kaufman, A.S., & Kaufman, N.L. (1990). Kaufman Brief Intelligence Test manual. Circle Pines, MN: American Guidance Service. Kazdin, A. E. (1997). Parent management training: Evidence, outcomes, and issues. Journal of the American Academy of Child and Adolescent Psychiatry, 36, 1349- 1356. Kazdin, A. E. (1992). Research design in clinical psychology. Boston: Allyn & Bacon. Kazdin, A. E., Holland, L., & Crowley, M. (1997). Family experience of barriers to treatment and premature termination from child therapy. Journal of Consulting and Clinical Psychology, 65, 453-463. Kazdin, A. E., Siegel, T. C., & Bass, D. (1992). Cognitive problem-solving skills training and parent management training in the treatment of antisocial behavior in children. Journal of Consulting and Clinical Psychology, 60, 733-747. Kivlighan, D. M., & Kivlighan, M. C. (2004). Counselor intentions in individual and group treatment. Journal of Counseling Psychology, 51, 347-353. Lumley, V. A., McNeil, C. B., Herschell, A. D., & Bahl, A. B. (2002). An examination of gender differences among young children with disruptive behavior disorders. Child Study Journal, 32, 89-100. Manassis, K., Mendlowitz, S. L., Scapillato, D., Avery, D., Fiksenbaum, L., Freire, M., et al. (2002). Group and individual cognitive-behavioral therapy for childhood 97 anxiety disorders: A randomized trial. Journal of the American Academy of Child and Adolescent Psychiatry, 41, 1423-1430. Matos, M., Torres, R., Santiago, R., Jurado, M., & Rodriguez, I. (2006). Adaptation of Parent-Child Interaction Therapy for Puerto Rican families: A preliminary study. Family Process, 45, 205-222 Maughan, B., Rowe, R., Messer, J., Goodman, R., Meltzer, H. (2004). Conduct disorder and oppositional defiant disorder in a national sample: Developmental epidemiology. Journal of Child Psychology and Psychiatry, 45, 609-621. McCabe, K. M., Yeh, M., Garland, A. F., Lau, A. S., & Chavez, G. (2005). The GANA program: A tailoring approach to adapting Parent-Child Interaction Therapy for Mexican Americans. Education and Treatment of Children, 28, 111-129. McDiarmid, M. D., & Bagner, D. M. (2005). Parent-Child Interaction Therapy for children with disruptive behavior and developmental disabilities. Education and Treatment of Children, 28, 130-141. Middlemiss, W. (2003). Brief report: Poverty, stress, and support: Patterns of parenting behaviour among lower income black and lower income white mothers. Infant and Child Development, 12, 293-300. Milner, J.S. (1986). The Child Abuse Potential Inventory: Manual (2 nd ed.). Webster, NC: Psytec. Murry, V. M., Kotchick, B. A., Wallace, S., Ketchen, B., Eddings, K., Heller, L., & et al. (2004). Journal of Child and Family Studies, 13, 81-99. 98 Naugle, R.I., Chelune, G.J., & Tucker, G.D. (1993). Validity of the Kaufman Brief Intelligence Test. Psychological Assessment, 5, 182-186. Niec, L. N. (2006, January). A good tree bearing new fruit: Developing alternate formats of PCIT. Symposium conducted at the 6 th Annual Parent-Child Interaction Therapy Conference, Gainesville, Florida. Niec, L. N., Hemme, J. M., Yopp, J., & Brestan, E. V. (2005). Parent-Child Interaction Therapy: The rewards and challenges of a group format. Cognitive and Behavioral Practice, 12, 113-125. Patterson, G. R. (1982). A social learning approach to family intervention: III. Coercive family process. Eugene, OR: Castalia. Phares, V. (1996). Conducting nonsexist research, prevention, and treatment with fathers and mothers: A call for a change. Psychology of Women Quarterly, 20, 55-77. Pincus, D. B., Eyberg, S. M., & Choate, M. L. (2005). Adapting Parent-Child Interaction Therapy for young children with Separation Anxiety Disorder. Education and Treatment of Children, 28, 163-181. Porter, A., Timmer, S., Urquiza, A., Zebell, N., & McGrath, J. (2006, January). Dissemination of PCIT to child maltreatment agencies: A snapshot in time. Poster session presented at the 6 th Annual Parent-Child Interaction Therapy Conference, Gainesville, Florida. Prewett, P.N. (1995). A comparison of two screening tests (the Matrix Analogies Test? Short Form and the Kaufman Brief Intelligence Test) with the WISC-III. Psychological Assessment, 7, 69-72. 99 Querido, J. G. (2004). Early intervention for child conduct problems in Head Start families (Doctoral dissertation, University of Florida, 2004). Dissertation Abstracts International: Section B: The Science and Engineering, 64, 4630. Querido, J. G., Bearss, K., & Eyberg, S. M. (2003). Parent-Child Interaction Therapy. In F.W, Kaslow & T. Patterson (Eds.), Comprehensive Handbook of Psychotherapy, Volume Two: Cognitive/Behavioral/Functional Approaches (pp. 91-113). New York: Wiley. Reitman, D., Currier, R. O., & Stickle, T. R. (2002). A critical evaluation of the Parenting Stress Index-Short Form (PSI-SF) in a Head Start population. Journal of Clinical Child and Adolescent Psychology, 31, 384-392. Reyno, S. M., & McGrath, P. J. (2006). Predictors of parent training efficacy for child externalizing behavior problems: A meta-analytic review. Journal of Child Psychology and Psychiatry, 47, 99-111. Reynolds, C. R., & Kamphaus, R. W. (1992). Behavior Assessment System for Children: Manual. Circle Pines, MN: American Guidance Service. Reynolds, W. M., & Sattler, J. M. (2002). Assessment of behavioral, social, and emotional competencies. In J. M. Sattler (Ed.), Assessment of children: Behavioral and clinical applications (2 nd ed., pp. 163-188). San Diego: Jerome M. Sattler, Publisher, Inc. Schoenwald, S. K., & Hoagwood, K. (2001). Effectiveness, transportability, and dissemination of interventions: What matters when? Psychiatric Services, 52, 1190-1197. 100 Schuhmann, E. M., Foote, R. C., Eyberg, S. M., Boggs, S. R., & Algina, J. (1998). Parent-Child Interaction Therapy: Interim report of a randomized trial with short- term maintenance. Journal of Clinical Child Psychology, 27, 34-45. Shechtman, Z., & Ben-David, M. (1999). Individual and group psychotherapy of childhood aggression: A comparison of outcomes and processes. Group Dynamics: Theory, Research, and Practice, 3, 263-274. Shirk, S. R. (2004). Dissemination of youth ESTs: Ready for prime time? Clinical Psychology: Science and Practice, 11, 308-312. Steinberg, L., Dornbusch, S. M., & Brown, B. B. (1992). Ethnic differences in adolescent achievement: An ecological perspective. American Psychologist, 47, 723-729. Stetson, R., Stetson, E. G., & Sattler, J. M. (2002). Assessment of receptive and expressive language. In J. M. Sattler (Ed.), Assessment of children: Cognitive applications (2 nd ed., pp. 610-634). San Diego: Jerome M. Sattler, Publisher, Inc. Taylor, T. K. & Biglan, A. (1998). Behavioral family interventions for improving child- rearing: A review of the literature for clinicians and policy makers. Clinical Child and Family Psychology Review, 1, 41-60. Timmer, S. G., Urquiza, A. J., & Zebell, N. M. (2005). Parent-Child Interaction Therapy: Application to maltreating parent-child dyads. Child Abuse & Neglect, 29, 825- 842. Timmer, S. G., Urquiza, A. J., & Zebell, N. (2006). Challenging foster caregiver- maltreated child relationships: The effectiveness of Parent-Child Interaction Therapy. Children and Youth Services Review, 28, 1-19. 101 Van de Wiel, N., Matthys, W., Cohen-Kettenis, P. C., & van Engeland, H. (2002). Effective treatments of school-aged conduct disordered children: Recommendations for changing clinical and research practices. European Child & Adolescent Psychiatry, 11, 79-84. Vess, S., & Campbell, J. (2006, January). Utility of PCIT in treating autism spectrum disorders. Poster session presented at the 6 th Annual Parent-Child Interaction Therapy Conference, Gainesville, Florida. Ware, L., Timmer, S., & Urquiza, A. (2006, January). Parent-Child Interaction Therapy with victims of interparental violence. Poster session presented at the 6 th Annual Parent-Child Interaction Therapy Conference, Gainesville, Florida. Webster-Stratton, C. (1985). The effects of father involvement in parent training for conduct problem children. Journal of Child Psychology and Psychiatry, 26, 801- 810. Weisz, J. R., Chu, B. C., & Polo, A. J. (2004). Treatment dissemination and evidence- based practice: Strengthening intervention through clinician-researcher collaboration. Clinical Psychology: Science and Practice, 11, 300-307. Weisz, J. R., Weiss, B., Han, S. S., Granger, D. A., & Morton, T. (1995). Effects of psychotherapy with children and adolescents revisited: A meta-analysis of treatment outcome studies. Psychological Bulletin, 117, 450-468. 102 APPENDIX TABLES & FIGURES Table 1 Sample Characteristics ________________________________________________________________________ Variable Completers (n = 16) Non-Completers (n = 11) n / Mean (SD) n / Mean (SD) ________________________________________________________________________ Gender of Child 4 female 6 female 12 male 5 male Child Age (in years) 5.12 (1.56) 4.88 (1.96) Gender of Parent 15 female 9 female 1 male 2 male Parent Age (in years) 32.38 (8.72) 30.60 (4.88) Ethnicity African American 5 5 Caucasian 11 6 Marital Status Single 7 6 Maried 5 2 ________________________________________________________________________ 103 Table 1 ? continued ________________________________________________________________________ Variable Completers (n = 16) Non-Completers (n = 11) n / Mean (SD) n / Mean (SD) ________________________________________________________________________ Separated 0 1 Divorced 3 2 Unknown 1 0 Education Level <12 0 1 High School 6 2 Some College 5 5 Associates 2 1 Bachelors 2 1 Professional 1 1 Family Income <$10K 6 2 $10K ? $20K 3 1 $20K ? $30K 1 2 $30K ? $40K 0 1 $40K ? $60K 2 0 $60K ? $100K 1 0 >$100K 2 1 ________________________________________________________________________ 104 Table 1 ? continued ________________________________________________________________________ Variable Completers (n = 16) Non-Completers (n = 11) n / Mean (SD) n / Mean (SD) ________________________________________________________________________ Unknown 1 4 Referral Source Self-Referred 8 6 DHR-Referred 8 7 IQ Estimates Child PPVT Score 88.08 (16.50) 97.33 (14.12) Parent K-BIT Score 97.43 (11.24) 88.50 (20.07) ________________________________________________________________________ Note. PPVT = Peabody Picture Vocabulary Test; K-BIT = Kauffman Brief Intelligence Test 105 Table 2 Categories of the Dyadic Parent-Child Interaction Coding System II (DPICS II) Parent Behavior Child Behavior Acknowledgement Acknowledgement Answer Answer Behavioral Description Behavioral Description Compliance Compliance Contingent Labeled Praise Criticism Criticism Descriptive/Reflective Question Descriptive/Reflective Question Destructive Destructive Direct Command Direct Command Indirect Command Indirect Command Information Description Information Description Information Question Information Question Labeled Praise Labeled Praise Laugh Laugh No Answer No Answer No Opportunity for Answer No Opportunity for Answer No Opportunity for Compliance No Opportunity for Compliance 106 Table 2 ? Continued Parent Behavior Child Behavior Noncompliance Noncompliance Physical Negative Physical Negative Physical Positive Physical Positive Play Talk Play Talk Reflective Statements Reflective Statements Smart Talk Smart Talk Unlabeled Praise Unlabeled Praise Yel Yel Whine Whine Warning 107 Table 3 Summary of the DPICS II Categories ________________________________________________________________________ 1. Acknowledgement (AK)?a brief verbal response that indicates attention to verbal or nonverbal behavior of either person in the dyad, but does not describe or evaluate the behavior. 2. Information Description (ID)?a declarative sentence that gives an account of people, the play, or events that does not clearly describe the other member of the dyad?s current or immediately completed behavior. 3. Behavioral Description (BD)?a declarative sentence where the subject is the other member of the dyad and the verb describes the other person?s verbal or nonverbal observable behavior. 4. Reflective Statement (RF)?a declarative statement that immediately repeats the other person?s verbalization. 5. Descriptive/Reflective Questions (DQ)?a descriptive or reflective comment or acknowledgement expressed in a question form. Requires only a simple acknowledgement in response (i.e., ?yes? or ?no? response). 6. Information Questions (IQ)?questions that require specific information from the other person other than a simple acknowledgement. 7. Unlabeled Praise (UP)?a verbalization that expressed a nonspecific favorable judgment of the other person/self, an attribute of the other/self, or a nonspecific activity or product of the other/self. 8. Labeled Praise (LP)?a verbalization that expresses a favorable judgment upon a specific activity or product of the other member of the dyad or the speaker. 9. Contingent Labeled Praise (CP)*--when the parent issues a labeled praise in response to the child?s compliance to a command. 10. Indirect Command (IC)?an order, demand, or direction for a behavioral response that is implied, nonspecific, or stated in a question form. ________________________________________________________________________ 108 Table 3 ? Continued ________________________________________________________________________ 11. Direct Command (DC)?a clearly stated order, demand, or direction in a declarative form that is sufficiently specific as to indicate the behavior that is expected from the other person. 12. Criticism (CR)?a verbal expression of disapproval of the other person, and/or the other?s attributes, activities, products, or choices. 13. Smart Talk (ST)?sassy, sarcastic speech, rude, or impudent speech. 14. Play Talk (PT)?any verbalization given as part of ?pretend talk,? where the speaker is talking as a toy or character other than him/herself. 15. Laugh (LA)?any chuckling or laughing that is not associated with any teasing or taunting behavior. 16. Whine (WH)?words uttered in a slurring, nasal, high-pitched, falsetto tone, clearly distinct from the normal tone of the speaker?s verbalizations. 17. Yell (YE)?a loud screech, scream, shout, or loud crying that is clearly above the intensity of the speaker?s normal speech volume. 18. Physical Positive (PP)?any touching of the other person that is neutral or positive. 19. Destructive (DS)?any action that destroys, damages, or attempts to damage any object. 20. Physical Negative (PN)?any touching of the other person that attempts to restrain or inflict pain. 21. Compliance (CO)?when the person obeys, begins to obey, or attempts to obey a direct or indirect command given by the other person. 22. Noncompliance (NC)?when the person does not obey a direct or indirect command given by the other person within 5 seconds. 23. No Opportunity for Compliance (NOC)?when the person is not given adequate chance to comply with a command issued by the other member of the dyad. ________________________________________________________________________ 109 Table 3 ? Continued ________________________________________________________________________ 24. Answer (AN)?when the person answers, starts to answer, or tries to answer an information question posed by the other member of the dyad. 25. No Answer (NA)?when the person does not answer the other person?s information question within 5 seconds either by giving no response or by giving a rude, sassy, or deliberately false response. 26. No Opportunity for Answer (NOA)?when the person is not given an adequate chance to respond to an information question issued by the other member of the dyad. 27. Warning (W)*?when the parent issues a statement following a command indicating that the child will be placed in time out following further noncompliance to the command. 28. No Code (NCD) ** ?used to designate verbalizations that are incomplete, noises that are not coded part as play talk, or other verbalizations that do not fit into other categories. ________________________________________________________________________ Note: Because the DPICS II categories are reflexive, all categories apply to both parent and child behavior, unless otherwise specified. *These categories are coded for parents only. **Not a true DPICS or DPICS II category. Created only to aid in coding. 110 Table 4 DPICS-II Composite Categories ________________________________________________________________________ Parent Composite Child Composite ________________________________________________________________________ Prosocial Behavior Prosocial Behavior Acknowledgement Acknowledgement Behavioral Description Behavior Description Information Description Information Description Unlabeled Praise Unlabeled Praise Labeled Praise Labeled Praise Contingent Labeled Praise Reflective Statement Reflective Statement Laugh Laugh Physical Positive Physical Positive Compliance Inappropriate Behavior Inappropriate Behavior Criticism Criticism Whine Whine Yel Yel Smart Talk Smart Talk Physical Negative Physical Negative Destructive Destructive _______________________________________________________________________ 111 Table 4 ? Continued ________________________________________________________________________ Parent Composite Child Composite ________________________________________________________________________ Direct Command Ratio (DC / IC + DC) Indirect Command Direct Command Compliance Ratio (CO / CO + NC) Compliance Noncompliance ________________________________________________________________________ 112 Table 5 Group Parent-Child Interaction Therapy Session Content ________________________________________________________________________ Session Number Session Content Assessment Measures ________________________________________________________________________ 0 Intake Assessment DPICS-II ECBI BASC PSI-SF K-BIT / PPVT CAP Demographics 1 Introduction/Motivation Enhancement ECBI 2 Child-Directed Interaction Didactic ECBI 3 1 st CDI Coaching ECBI 4 2 nd CDI Coaching ECBI 5 3 rd CDI Coaching ECBI 6 Parent-Directed Interaction Didactic ECBI 7 1 st PDI Coaching (Individual) ECBI 8 2 nd PDI Coaching (Behavior Charts) ECBI 9 3 rd PDI Coaching ECBI 10 4 th PDI Coaching (House Rules) ECBI 11 5 th PDI Coaching (Time-out in Public) ECBI 12 Wrap-Up / Graduation (Trouble-Shooting) DPICS-II ECBI BASC PSI-SF TAI ________________________________________________________________________ 113 Table 5 ? Continued ________________________________________________________________________ Session Number Session Content Assessment Measures ________________________________________________________________________ 3-X 3-Month Follow-Up (Mail) ECBI BASC PSI-SF TAI 6-X 6-Month Follow-Up (In Person) DPICS-II ECBI BASC PSI-SF TAI ________________________________________________________________________ Note. DPICS-II = Dyadic Parent-Child Interaction Coding System ? 2 nd Edition; PPVT = Peabody Picture Vocabulary Test; K-BIT = Kauffman Brief Intelligence Test; ECBI = Eyberg Child Behavior Inventory; BASC = Behavior Assessment Scale for Children; PSI-SF = Parenting Stress Index ? Short Form; CAP = Child Abuse Potential Inventory; TAI = Treatment Attitude Inventory 114 Figure 1. ECBI Intensity Scores across Time for Treatment Non-Completers EC BI In t e n s i t y S c a l e f o r N o n - C o m p l e t e r s 0 50 100150200250 WK 0 WK 1 WK 2 WK 3 WK 4 WK 5 WK 6 WK 7 WK 8 WK 9 WK 10 WK 1 1 WK 12 Se s s i o n s Sc o r e 115 Figure 2. ECBI Intensity Scores across Time for Treatment Completers EC BI I n tens i t y Sca l e Sco r es fo r C o m p l e ters 0 50 100150200250 W K 0 W K 1 W K 2 W K 3 WK 4 WK 5 WK 6 W K 7 W K 8 WK 9 WK 1 0 WK 1 1 WK 1 2 S e ssi o n s Sc o r e 116 Figure 3. ECBI Intensity Scores across Time for All Participants EC B I I n t e n s i t y Sc a l e Sc o r e s 0 50 100150200250 W K 0 W K 1 WK 2 WK 3 WK 4 WK 5 WK 6 W K 7 W K 8 WK 9 WK 1 0 WK 1 1 WK 1 2 S e ssi o n s Sc o r e 117 Figure 4. ECBI Problem Scores across Time for Treatment Non-Completers EC BI P r ob l e m S c or e s f o r N o n - C o m p l e te r s 05 101520253035 WK 0 WK 1 WK 2 WK 3 W K 4 WK 5 W K 6 WK 7 WK 8 W K 9 W K 1 0 WK 1 1 W K 1 2 S e ssi o n s Sc or e 118 Figure 5. ECBI Problem Scores across Time for Treatment Completers EC BI P r o b l e m Sco res fo r C o m p l e ters 05 10152025303540 W K 0 WK 1 W K 2 W K 3 WK 4 W K 5 W K 6 WK 7 W K 8 W K 9 W K 10 W K 1 1 W K 12 S e ssi o n s Sc o r e 119 Figure 6. ECBI Problem Scores across Time for All Participants E C B I P r ob l e m S c al e S c or e s 05 10152025303540 WK 0 W K 1 W K 2 W K 3 W K 4 WK 5 W K 6 W K 7 W K 8 W K 9 WK 10 WK 1 1 WK 1 2 S e ssi o n s Sc or e