Examination of an Attachment Training for Mentoring Effectiveness by Shalonda Jones Brooks A dissertation submitted to the Graduate Faculty of Auburn University in partial fulfillment of the requirements for the Degree of Doctor of Philosophy Auburn, Alabama August 6, 2011 Keywords: Mentoring, Training, PCIT Copyright 2011 by Shalonda Jones Brooks Approved by Elizabeth Brestan Knight, Chair, Associate Professor of Psychology Christopher Correia, Associate Professor of Psychology Alejandro Lazarte, Associate Professor of Psychology Jennifer M. Gillis Mattson, Assistant Professor of Psychology ii Abstract Youth mentoring can be defined as a unique relationship between an adult and child or adolescent that is characterized by mutual respect, reciprocity, transmission of knowledge from mentor to mentee, and social-emotional support (DuBois & Karcher, 2005; Eby, Rhodes, & Allen, 2000). The goals of mentoring are to ?increase desirable behavior, decrease undesirable behavior? (Eby, Allen, Evans, Ng, & DuBois, 2008, p.256), and foster growth and development for the mentee (Keller, 2007). These goals are accomplished by the mentors building trust, providing understanding, and creating relationship reciprocity (Zeldin, Larson, Camino, & O?Connor, 2005). In this manner, the mentoring relationship appears to be the process by which change occurs. Currently, very little research focuses on how to improve mentor-mentee relationships. This project adapted the first phase of Parent-Child Interaction Therapy (PCIT), a program typically used with parents and children to increase warmth and decrease child behavior problems, into a training module designed to help mentors build a supportive relationship with their mentee. PCIT (Eyberg, 1988) is a behaviorally-based intervention for children with disruptive behavior disorders. The first phase of PCIT helps parents and children to build a warm, stable relationship by using principles from attachment theory (Bell & Eyberg, 2002). Research has suggested that a secure parent-child attachment is linked to prosocial development whereas maladaptive attachment is linked to increased aggressive child behavior (Querido, Bearss, & Eyberg, 2002). iii The purpose of the project was to apply the principles from the PCIT Child Directed Interaction (CDI) teach session to help mentors build a warm relationship with their mentees and thereby increase the effectiveness of a local mentoring program, Project Uplift. The objectives of the project were to develop a mentor training module based on the PCIT CDI Teach session and evaluate the effectiveness of this attachment training based on child and mentor reports of relationship quality and parent report of child behavioral and social outcomes. Thirty-seven mentee-mentor dyads and 18 teachers participated in this study. Mentors were randomly assigned to receive either the attachment training based on the CDI Teach session or a general communication training. Mentors completed measures of relationship quality at pre- intervention and post-intervention. Mentees also completed measures of relationship quality as well as a measure that assessed mentors? use of PRIDE skills at pre-intervention and post- intervention. Parents and teachers completed the BASC-2 to report child behavior at pre- intervention and post-intervention. Post-intervention measures were given 9 months after the pre- intervention assessment. The results indicated that although there were no differences in use of the PRIDE skills based on group assignment, there was a stronger association between mentee report of PRIDE skills and relationship quality (as reported by the mentees) for mentors in the attachment group suggesting that the attachment training helped to facilitate a good relationship. It was also expected that there would be differences in relationship quality based on the type of training and over time. Based on the mentee?s report of relationship quality, this hypothesis was not supported. However, an examination of the mean scores for relationship quality showed that mentees reported a positive relationship at pre-intervention and the positive quality remained throughout the dyad relationship to post-intervention. Mentors did report a significant change in iv relationship quality over time, but there were no differences based on group assignment. Although mentoring relationships were positive, the expected changes in child behavior were not observed. There were no differences based on group assignment or across time for most behavioral areas. The only significant change in child outcomes was for Social skills. However, post hoc analysis revealed that there were positive changes from pre-intervention to post- intervention for Externalizing Behaviors, Adaptive Behaviors, Resiliency, and Social Skills scales on the BASC although they did not reach statistical significance. Overall, the results suggest that there were little differences on outcome measures (relationship quality, child behavior) from pre-intervention to post-intervention based on group assignment. However, child behavior remained stable at subclinical levels over time (or slightly improved), social skills increased, and mentees and mentors were satisfied with their relationships. Limitations of this study included a significantly abbreviated training session, lack of an objective measure of mastery of the skills, and low compliance for continued practice activities. v Acknowledgments This has been a long but rewarding project and it would not have been possible without the continued help and support from Chris Nunn and Project Uplift. I would like to thank Project Uplift for allowing me to be a part of their amazing program and for being open minded as I implemented a new program while keeping their goals of uplifting the lives of children in mind. Next I would like to thank Jamie Travis, my project manager, who was my right hand during this project. She made sure that everything ran smoothly and efficiently. In addition, I also appreciate the help from the many undergraduate students that worked hard to make this project happen in a variety of capacities from entering data, to conducting trainings, and going on home visits. I am grateful to my committee members for being behind this project 100% and for their suggestions and thoughtfulness, which helped make this project the best that it could be. My family and friends have continued to provide me with support and a shoulder to lean on through this endeavor and all my endeavors. Lastly, I thank the Auburn University Outreach Office for recognizing the importance of this project and for providing the monetary resources to make it possible. vi Table of Contents Abstract ......................................................................................................................................... ii Acknowledgments......................................................................................................................... v List of Tables ............................................................................................................................. viii List of Figures .............................................................................................................................. ix List of Abbreviations .................................................................................................................... x Mentoring Overview ..................................................................................................................... 1 A Definition of Youth Mentoring .................................................................................... 3 Goals and Purposes of Youth Mentoring ........................................................................ 4 The Purported Benefits of Mentoring ............................................................................. 5 The Evidence of Youth Mentoring Effectiveness ........................................................... 6 Under What Conditions is Mentoring Effective? ......................................................... 13 What is the Current Status of Mentoring Research? ...................................................... 14 The Current Study ....................................................................................................................... 17 The Need to Build Warmth/attachment in Mentoring Relationships ............................ 17 Parent Child Interaction Therapy ................................................................................... 19 Project Uplift ................................................................................................................... 22 Hypotheses and Analyses ........................................................................................................... 24 Preliminary Analyses ..................................................................................................... 24 Primary Goal .................................................................................................................. 24 vii Secondary Goal .............................................................................................................. 25 Tertiary Goal .................................................................................................................. 25 Method ....................................................................................................................................... 27 Participants ...................................................................................................................... 27 Procedures ....................................................................................................................... 28 Training Sessions ............................................................................................................ 29 Measures ......................................................................................................................... 33 Results ........................................................................................................................................ 39 Preliminary Hypotheses ................................................................................................. 39 Hypothesis 1.................................................................................................................... 40 Hypothesis 2.................................................................................................................... 41 Hypothesis 3.................................................................................................................... 42 Hypothesis 4.................................................................................................................... 43 Additional Analyses ........................................................................................................ 44 Discussion ................................................................................................................................... 45 Implications..................................................................................................................... 60 Limitations ..................................................................................................................... 65 Future Directions ............................................................................................................ 66 References .................................................................................................................................. 83 Appendices .................................................................................................................................. 92 viii List of Tables Table 1 Participant Demographics and Chi-square Analyses ................................................ 69 Table 2 Inter-Item Correlation Matrix for the Youth Mentor Relationship Questionnaire ... 71 Table 3 Inter-Item Covariance Matrix for the YMRQ........................................................... 72 Table 4 Correlations for Mentee Relationship Questionnaires .............................................. 73 Table 5 Factor Loadings for Exploratory Factor Analysis with Varimax Rotation for the YMRQ ............................................................................................ 74 Table 6 Factor Loadings for Exploratory Factor Analysis with Varimax Rotation for the YMRQ-MV .................................................................................... 75 Table 7 Pre-intervention and Post-intervention Data for the Experimental Group ............... 76 Table 8 Pre-intervention and Post-intervention Data for the Control Group.......................... 77 Table 9 2 x2 Split Plot ANOVAs for Parent Reported BASC-2 Scores .................................. 78 ix List of Figures Figure 1 Flow Chart of Original Study Design ....................................................................... 79 Figure 3 Flow Chart of Modified Study Design ..................................................................... 80 Figure 3 Pre-intervention and Post-intervention Relationship Scores for Mentor Report...... 81 Figure 4 Percentage of Mentees Showing Improvement or No Change on ........................... 82 BASC-2 Scales x List of Abbreviations PCIT Parent Child Interaction Therapy CDI Child Directed Interaction PDI Parent Directed Interaction PRIDE Praise, Reflections, Imitation, Descriptions, Enthusiasm BBBS Big Brothers/Big Sisters NIMH National Institute of Mental Health DHR Department of Human Resources BASC-2 Behavior Assessment System for Children ? Second Edition MYAS Mentor Youth Alliance Scale YMRQ Youth Mentor Relationship Questionnaire MEI Mentoring Evaluation Inventory TAI Training Attitude Inventory ANOVA Analysis of Variance SPANOVA Split Plot Analysis of Variance CBPR Community Based Participatory Research 1 Mentoring Overview ?Mentors not only touch someone?s life?they have the potential to touch and change the life of the nation? (Newsweek, 1999 as cited by Mentoring Children and Adolescents). Quotes like the above can have a powerful impact on the perception of mentoring intervention effectiveness. In addition to anecdotal reports, common sense also suggests that mentoring is an effective intervention for various societal youth ailments. This is supported by a long history of the concept of mentoring that dates back to the epic poem, Odyssey (Buckley & Zimmerman, 2003; Allen & Eby, 2007). In the Odyssey, Odysseus leaves for war. In his absence, he puts his old friend Mentor in charge of his household. It is also Mentor?s responsibility to counsel and guide Odysseus? young son while he is away. Similarly, current day mentoring pairs a child with an older adult who can help guide them. However, despite this age- old tradition of a mentoring relationship and its perceived value, little empirical work has been conducted to evaluate the effectiveness of youth mentoring programs. The contemporary mentoring movement began in the early 1900s with increased interest in the social movement. During this time, industrialization and technology drew more people into cities, thus, creating urban centers. However, the boom and growth did not last and by the mid 1900s there was an increasing discrepancy between the rich and the poor (Buckley & Zimmerman, 2003). With the rise of poverty came stress, overcrowding, and delinquency. Social reformists took an interest in how to help poor youth. Formal institutions such as the Juvenile court system and Big Brothers and Big Sisters were established but informal mentoring also took place (Baker & MacGuire, 2005). Middle class women volunteered their time and resources to try to build positive relationships with youth to help keep them out of trouble 2 (Rhodes, 2002). Since that time, over 4,000 organizations have been established to address this social issue and the mentoring movement continues to grow. Mentoring is a recent construct in the scientific literature. The first scientific study began with Kram in 1980 with her dissertation entitled, Mentoring processes at work: Developmental relationships in managerial careers. In subsequent research, she identified the four stages of mentoring relationships in the workplace: initiation, cultivation, separation, and redefinition. The initiation phase is the beginning of the relationship. In this phase, support and positive expectations are built. Successfully navigating this phase allows for transition into the next phase: cultivation. During this phase, the positive expectations set forth in the first phase are tested. In addition, both parties begin to better understand the value of their relationship and, as a result, psychosocial benefits such as friendship and acceptance emerge. In the separation phase, separation may occur physically and/or emotionally. Separation may create anxiety in both parties as the function of their relationship changes. In the final phase, redefinition, the relationship becomes more of a friendship vs. a role model relationship; however, counseling and support may continue. Although early mentoring research focused on organizations and mentoring in the workplace, the field of mentoring has since broadened to three areas: work mentoring, youth mentoring, and academic mentoring. Based on current research, the four stages of mentoring as described by Kram appear to apply to youth and academic mentoring (Lentz & Allen, 2007). Since the early research on mentoring in the work place there has been a flurry of activity and research in the area of mentoring. For example, a search of mentoring literature from 1985 ? 2006 conducted by Eby and colleagues (2005) yielded over 15,000 articles and reports. Although this is reflective of the immense growth in the field of mentoring, the subject has been plagued 3 by many difficulties, particularly defining a clear construct with an underlying theory, providing an operational definition of mentoring, and separating the vernacular use of the term with other concepts such as socialization, training, and supervision. This study evaluated the effectiveness of attachment training on youth mentoring outcomes. The current state of research on the effectiveness of mentoring will be reviewed; with a special emphasis on youth mentoring (although other forms of mentoring have informed youth mentoring). Before delving into a discussion of mentoring program effectiveness, the document will outline the concept and definition of youth mentoring, the goals of mentoring interventions, and the purported outcomes of mentoring interventions. A Definition of Youth Mentoring ?It is difficult to gauge the effectiveness of a social technology absent some clarity about its meaning? (Bozeman & Feeney, 2007, p. 730). As such, a statement suggests; the first issue at hand is to establish a definition of mentoring. In the literature, there are many definitions of mentoring. These definitions of mentoring vary from study to study and some studies do not provide a definition of mentoring at all. However, some commonality exists in the elements that characterize mentoring. Most definitions are comprised of three key areas associated with mentoring: instrumental support (guidance and instruction for career, academic, or civic development), psychosocial support (social and emotional outcomes), and knowledge transmission (by someone of greater experience/wisdom) (Bozeman & Feeney, 2007; DuBois & Karcher, 2005). The problem with the definitions of mentoring (and operationally defining mentoring) is that the definitions have no clear boundaries due to lack of a clear underlying theory. The definitions do not allow for the explanation of when and under what conditions mentoring is occurring or not occurring (Bozeman & Feeney, 2007) and the lack of a clear 4 definition leaves room for questions such as: Does mentoring have to be one-on-one or can it occur in groups? Are mentors volunteers? and How often does knowledge need to be transmitted? Some of the answers to these questions depend on what type of mentoring is being evaluated (work, academic, youth or formal vs. informal). For the purpose put forth in this paper, youth mentoring will be defined as a unique relationship between an adult and youth that is characterized by mutual respect and reciprocity, transmission of knowledge from mentor to mentee, and social-emotional support (DuBois & Karcher, 2005; Eby, Rhodes, & Allen, 2007). The Goals and Purposes of Youth Mentoring The goals/purposes of youth mentoring are to ?increase desirable behavior and decrease undesirable behavior? (Eby, Allen, Evans, Ng, & DuBois, 2008, p.256) and to foster growth and development (Keller, 2007). Although these are general goals, some youth mentoring programs have more specific goals such as positive youth development, and education or employment attainment (DuBois, Holloway, Valentine, & Cooper, 2002). Very few studies state the goal of their youth mentoring program and even fewer state the process, mechanism, or theory by which the goal can be achieved. Most studies have approached the evaluation of mentoring programs from one of the two following perspectives: protective/resiliency (fostering health promoting behavior and/or competence) or prevention (providing information about risky behaviors) (Bogenschneider, 1996). The protective/resiliency approach recognizes that children are often exposed to multiple risk factors however, it asserts that there may be individual and/or environmental factors that help insulate children from the negative effects of stressful events. This research project focused on the protective/resiliency approach in that the relationship with a mentor can be seen as a protective factor. Although building a strong relationship is not often stated as a primary goal of mentoring, findings from evaluations of youth mentoring programs 5 suggest that the development of a strong relationship is an indirect goal (or a mechanism by which the goal is accomplished) (Eby, Allen, Evans, Ng, & DuBois, 2008; Liang & Rhodes, 2007). The Purported Benefits of Mentoring The concept of youth mentoring is steeped in the idea that caring adults can provide children and adolescents with benefits above and beyond the relationship these youths have with their parents. Several broad terms encapsulate the concept of extraparental relationships with adults such as intergenerational relationships and youth-adult partnerships (Zeldin, Larson, Camino, & O?Connor, 2005). Each concept identifies the pairing of youth with an older adult as a key component in child development. Zeldin and colleagues (2005) stated that there are three purposes for youth adult relationships: to enhance relationships (build autonomy, child feels respected and acknowledged), to encourage positive development (acquire knowledge, empower youth), and to build community. These purposes are accomplished by building trust, providing understanding, and creating relationship reciprocity within the context of youth-adult partnerships (Zeldin, Larson, Camino, & O?Connor, 2005). More specifically, Rhodes (2005) posits that development occurs through our social experiences and relationships with adults help form a child?s internal working model of the world. Further, children who do not have adequate, positive, or nurturing relationships with an adult may rely on others, such as peers, for an example of how to behave. For this reason, Liang and Rhodes (2007) calls the adult-youth relationship the ?vital element? in mentoring programs. Grounded in Jungian theory (warmth and attachment are needed for human development), the vital element in mentoring is a close caring relationship between an adult and the mentee (Liang & Rhodes, 2007). Although the reported outcomes of youth mentoring studies vary, they generally fall under the following broad areas: 6 improvement in the academic/educational setting, development of healthy and safe behaviors, and the promotion of social, emotional, and behavioral outcomes (Jekielek, Moore, & Hair, 2002). The Evidence of Youth Mentoring Effectiveness Academic achievement. Rose and Jones (2007) conducted a qualitative study of the effects of a volunteer mentoring program in a school setting. Specific to youth outcomes, the goal was to evaluate the impact of mentoring on social and academic performance. Participants (ages 11-14) were identified as ?at-risk? by their school officials and assigned a mentor (Rose & Jones, 2007). After six months, an independent research agency evaluated the effectiveness of the goals. A sample of parents (n = 5), teachers (n = 5), and students (n = 9) involved in the project were interviewed. Teachers and students reported improvement in attitudes and behavior at school (Rose & Jones, 2007). These reports were confirmed by data from more formal school reports, such as records of school attendance and reported disciplinary action. In addition, students reported relational benefits of having a mentor, including increased confidence and perceived support (Rose & Jones, 2007). Although this study provides some evidence for the positive effects of mentoring on academic achievement, it lacks a scientifically sound methodology (such as a comparison group and objective outcome measures), which prohibits drawing inferences from the authors? conclusions. Thompson and Kelly-Vance (2003) found similar results in their study. Using a quasi- experimental design, participants, boys ages 7-15, were assigned a mentor through Big Brothers/Big Sisters (BBBS) (n = 12) or were accepted into the program but were waiting for a mentor (n = 13). The purpose of the program was ?to provide the child with weekly, ongoing contact from a positive role model? (Thompson & Kelly-Vance, 2003, p. 235). Participants 7 (experimental group and waitlist control group) were administered the Kaufman Test of Educational Achievement and Kaufman Brief Intelligence Test at pre-intervention and post- intervention (Thompson & Kelly-Vance, 2003). Results demonstrated that children with a mentor showed greater improvement in academic achievement, specifically in the areas of reading and math relative to the children on the waitlist (Thompson & Kelly-Vance, 2003). Social-emotional functioning. Keating et al. evaluated the effects of mentoring on at? risk youth (Keating, Tomishima, Foster, & Alessandri, 2002). They focused on internalizing and externalizing behavior, hopelessness, self-concept, and delinquent acts. Participants were young males (ages 10 ? 17) referred for intervention by their parents, the school system, or the court system. For the experimental group, young males (n = 34) were matched with a mentor while controls were boys on the waitlist (n = 34) (Keating, et al., 2002). Pre-intervention and post- intervention interviews included reports from multiple sources (parents, teachers, and self). Preliminary analyses revealed that at pre-intervention parent and teacher reports differed from child self-report on multiple areas. For instance, children did not identify any problems on measures of externalizing and internalizing problems, while parents and teachers rated participants in the clinically significant range. After 6 months of mentoring, post-intervention measures were given. Parents and teachers rated mentored youth as showing a decrease in externalizing and internalizing problems. No changes were found for self-report measures of hopelessness, self-esteem, or delinquency (Keating, et al., 2002). Overall, this study suggests that mentoring may help decrease problem behavior. This conclusion may be bolstered by the strengths of methodological design of the study: quasi-experimental design, integrity checks for the mentoring program, psychometrically sound measures (such as the Child Behavior Checklist), and utilization of multiple informants. Although the study suggests that mentoring 8 was effective it is unclear as to how the mentors affected change. This lack of process information is a continued gap in the effectiveness of mentoring literature (Cavell & Smith, 2008; Keller, 2007). Schmidt, McVaugh, and Jacobi (2007) also evaluated the effects of mentoring on social- emotional outcomes (specifically anxiety, depression, and relationships with parents and peers). This study took place in a school setting and compared at-risk children participating in a mentoring program to other at-risk children in the same school that were not being mentored. Participants included 31 mentored children and 22 non-mentored children from the 4th (mean age was 9.67 years) and 5th grades (mean age was 11.25 years). Teachers completed measures of student behavior, including items related to academic performance, social skills, and problem behavior. Children completed measures of self-concept, anxiety, depression, and parent and peer relationships. The results suggested that mentored children reported greater self-concept in some areas (such as behavior, anxiety, and happiness) but not all areas (social popularity). Children in the mentored group also reported less anxiety (specifically in the areas of general worry and social concerns; Schmidt, et al., 2007). These findings are notable in that very few studies show a significant result for self-concept or self-esteem. These findings are also of particular interest because a 2002 meta-analysis (DuBois, et al., 2002) suggested that mentoring is generally less effective in the school setting. In addition, contrary to other studies (such as Keating et al., 2002), teachers reported that children who were mentored worsened in the areas of problem behavior and social skills. This is in direct contradiction to the purported effects of mentoring. The authors posit that perhaps teachers felt hostility toward mentors coming into their classroom once a week (Schmidt, et al., 2007). One major limitation of this study was that children were not randomly assigned to groups. 9 Multiple outcome areas. Many of the studies mentioned above fall short of being able to draw firm conclusions about the effectiveness of mentoring because they do not use random assignment. Big Brothers/Big Sisters (BBBS) is one of the few mentoring programs that have begun to systematically evaluate its? outcomes using randomized controlled trials. For their study, De Wit and colleagues, randomly assigned participants to an experimental group (n = 32) or control waitlist group (n = 29) (DeWit, Lipman, Manzano-Munguia, Bisanz, Graham, Offord, et al., 2006). Both groups were administered pre-intervention and post-intervention measures to evaluate program impact. Measures included an assessment of problem behaviors (child and parent report), school behaviors (child and parent report), academic achievement (child and parent report), attachment to school, social-emotional development (depression, self-esteem, social support, quality of relationships, coping, social skills, social anxiety), and involvement in the community. Repeated measures ANCOVAs were used to examine group differences. The independent variable was group (mentored vs. waitlist). The dependent variables were post- intervention measures and the covariates were the pre-intervention measures. The authors found significant differences in five areas of social-emotional development (DeWit et al., 2006). Mentored youth reported decreased emotional problems, decreased social anxiety for negative peer evaluations, decreased social anxiety and distress, increased perception of teacher social support, and increased social skills involving self-control. No differences were found for academic achievement, problem behaviors, school attachment, or community involvement. The major limitation of this study was a lack of follow-up assessments to assess if changes were maintained over time. Frecknall and Luk (1992) evaluated parental perception of the effectiveness of BBBS. Parents reported how much their child had improved in the areas the program was designed to 10 address. The results indicate that parents found that their children were greatly improved in all areas. Specifically, more than 50% of parents reported great improvement in getting along with family members, getting along with friends, staying out of trouble, being more responsible, and self-esteem. Although this study is also flawed (due to the lack of pre-intervention/post- intervention comparisons, lack of a comparison group, and the reliance on the self-reports of parents), it provides some preliminary convergent evidence that the BBBS program is effectively meeting its goals of addressing problems of youth delinquency, poor academic performance, and low self-esteem in its at-risk participants. Using the ecological perspective as an underlying theory, Chan and Ho (2008) evaluated the effectiveness of mentoring while primarily focusing on how perceptions of relationship closeness affect outcomes. Participants included 124 students who were considered at-risk. The program was designed to ?enrich social and cultural lives, improve interpersonal skills, strengthen social support, positive attitude toward school, improve parent and peer relationships, and reduce violent behavior? (Chan & Ho, 2008, p. 842). In addition to the goals of the program, the researchers also assessed the quality of mentor/mentee relationship, self-esteem, and problem behaviors. The results suggested that over half of children showed improvement in at least three of the core areas (the highest rates of improvement were for social/cultural enrichment, social communication skills, and social support) and 90% showed improvement in at least one area. The authors also evaluated how relationship quality mediated outcomes. This will be further discussed in relation to mediating and moderating factors. From the above reviewed studies, mixed reviews are obtained. Some studies support the claims of effectiveness while others do not. Overall, it seems that the greater amount of support is for social-emotional benefits with less evidence for academic achievement, problem behaviors, 11 and self-sufficiency. However, because these studies are so different in methodology, it is difficult to draw conclusions. Meta-analyses. After many years of reports of ?success,? mentoring programs are now at the point of systematic evaluation. As stated above, difficulties have arisen in the evaluation of mentoring for many reasons. Researchers are now able to feasibly examine different studies and synthesize the information into a more comprehensive overview of the research area. Jekielek and colleagues (2002) provided a mentoring synthesis that summarized research from random- controlled trials (5 programs), quasi-experimental design (2 programs), and non-experimental designs (3 programs) (Jekielek, Moore, & Hair, 2002). The synthesis examined the broad areas mentoring typically addresses: academic achievement (grades, GPA), health and safety (substance use, aggressive behaviors, and delinquent behaviors), social and emotional development (self-worth, interpersonal relationships, and emotional support), and self- sufficiency (employment and productivity). A review of mentoring programs found the following: for academic achievement, mentoring was helpful for decreasing school absences, increasing perceived scholastic competence and attitudes about school, increasing the likelihood of college attendance, and improving school behavior. For health and safety, participants in a mentoring program reported decreased drug and alcohol use. Contrary to predictions, the results for delinquency received mixed reviews, with some studies reporting improvements while others reporting no differences. For social and emotional development, mentoring positively affected attitudes toward school, attitudes toward the future, increased helping behaviors, and improvement in family and peer relationships. Despite positive findings, this report should be interpreted with caution for many reasons, such as the inclusion of only formal mentoring 12 programs, the inclusion of programs that included interventions other than mentoring, small sample sizes, and limited inclusion of studies with an experimental or quasi-experimental design. DuBois and colleagues evaluated mentoring research by conducting a large meta-analysis (DuBois, Holloway, Valentine & Cooper, 2002). The goals were to evaluate mentoring research based on the ?Best Practices? guide put forth by the National Mentoring Partnership in 2002 and to evaluate potential moderators. Overall, the study found that youth that were mentored scored 1/8 of a standard deviation higher than comparisons on outcome measures. Although this evidence may be interpreted as supporting mentoring, a closer look at the results reveal the average effect size was only d = .14 (with a range of d = .11 to d = .25), which is considered small, according to Cohen (1988). We can conclude from the above that mentoring has a small but positive effect on youth outcomes. Dubois and colleagues (2002) also looked at potential moderations after evaluating the overall effects; however, we will examine these findings in a later section. Critique of the evidence. The results from individual reports provided inconclusive results. In addition, most have serious methodological flaws. For instance, they lack a control or comparison group, lack random assignment, use a single informant, or use measures that are not psychometrically sound. In addition, for some studies (particularly those that evaluated natural or informal mentoring) the concept of mentoring was ill-defined. These limitations lead to concerns about construct validity, internal validity, and external validity. Because of these problems, we turn to meta-analysis to help synthesize the material. Meta-analyses suggest that, overall, mentoring is effective; however, the effects are small. In addition, meta-analyses also have their own methodological issues. For example, the quality of studies included in the meta-analyses was not consistent. Most included experimental studies, but also allowed the inclusion of less 13 formal designs (less scientifically rigorous reports). In addition, some of the studies that were included did not solely use mentoring as an intervention. In some cases, mentoring was only part of a larger intervention effort. Also, some meta-analyses used statistical procedures that were correlational in nature, which limits the conclusions that can be drawn about the effectiveness of mentoring. Under What Conditions is Mentoring Effective? Moderators and Mediators Although there is only modest evidence of effectiveness, there are some conditions in which mentoring seem to be more successful. In this manner, we evaluate potential mediators and moderators. Mediators are variables that ?specify how a given effect occurs? while moderators affect the impact of the predictor variable on the outcome variable (Holmbeck, 2003). Most research has pointed to the influence of the length and quality of the mentor/mentee relationship. This is not surprising given the premise that it is the relationship between youth and caring adults that provides the means for healthy youth development. In the mentoring literature, some evidence has suggested that the relationship is part of the process by which change occurs. Below, we examine the mediators and moderators as outlined in several studies. Mediation. Many studies point to the mentor-mentee relationship as affecting the results of mentoring. More specifically, three areas have been implicated: the amount of contact between mentor and mentee, the length of the relationship, and the quality of the relationship. Chan and Ho (2008) reported that mentors with at least weekly contact were more likely to report benefits from the mentoring relationship. Similarly, DuBois and Neville (1997) found that the amount of contact (on average mentors and mentees met three times per week) with the mentor was related to perceived benefits to youth. These studies suggest that increased contact hours are related to greater mentee benefits. Similar, in addition to the findings for the amount of 14 contact, DuBois and Neville (1997) found that the quality of the mentor?mentee relationship was related to greater mentee benefits. Specifically, mentors that reported a close relationship with their mentee also reported greater benefits to the mentee (DuBois & Neville, 1997). In regards to the length of the relationship, longer relationships were related to greater psychosocial benefits (Chao, Walz, & Gardner, 1992; Rhodes, Reddy, Hoffman, & Grossman, 2005). Grossman and Rhodes (2002) specifically looked at the effects of mentoring over time. This study revealed that matches that lasted longer than 12 months showed the greatest benefits, while matches that lasted less than three months resulted in fewer positive benefits and even detrimental results in some cases. Moderation. Similar to the mediation data, moderator effects were found for variables related to the mentor-mentee relationship. Chan and Ho (2008) found that if the mentoring relationship was asymmetrical (poor quality), mentees reported minimal beneficial outcomes. In fact, relationships that were asymmetric were more similar to the control group. Relationships of short length led to poor outcomes (Grossman & Rhodes, 2002; Rhodes, Spencer, Keller, Liang, & Noam, 2006). In addition to these relationship variables, at-risk status also emerged as a significant moderator variable. For example, youth that were considered at-risk (because of individual or environmental factors) benefited greatly from mentoring while youth that were not at-risk reported no benefits from mentoring (DuBois et al., 2002). In addition, DuBois and Silverthorn (2005) found that mentoring had no affect on the specific outcome of hurting another or the self-esteem/life satisfaction if the youth was not at-risk. What is the current status of mentoring research? The NIMH has proposed three phases of research: pre-intervention, preventative intervention, and preventive service systems. At the pre-intervention stage, basic research is 15 conducted and new intervention strategies are designed. Because mentoring is new in the scientific realm, it is in the pre-intervention stage. DuBois et al. (2006) identified several areas in which the research on mentoring is having difficulties in this stage. First, there are several areas in which basic research is not adequately being conducted. For instance, there are many concerns about the samples that have been used. Specifically, instruments used have not been validated for the sample. The sample usually consists of only one informant assessed on only one occasion (usually post-intervention). In addition, mentoring research has methodological issues concerning study design, assessment, and data analysis. With regard to study design, most of the present research is cross-sectional while longitudinal data?that would provide information on the long-term effects of mentoring?are typically not available. For assessment, some instruments used in research on mentoring have poor psychometric properties or their psychometric properties have not been evaluated. Also, many studies solely use an interview or questionnaire format to assess outcomes. In the area of data analysis, most of the current research is correlational, or makes comparisons using a simple t-test. More complicated statistical procedures, such as factor analysis and structural equation modeling, are needed to gain a better understanding of the construct of mentoring. This review of the mentoring literature encountered many, if not all of these methodological issues. According to the National Mentoring Partnerships, following the best practices is a good launching point for creating an effective mentoring intervention. The guidelines included, but are not limited to: ?a well-defined mission statement and established operating principles?r egular, consistent contact between the mentor and the mentee?s upport by the family or guardian of the mentee/prot?g??a dditional community support services?p rogram evaluation and ongoing 16 assessment? [and] a long-range plan that has community input? (Buckley & Zimmerman, 2003, p. 299-300). Liang and Rhodes (2007) echo many of the above principles as they recommend the inclusion of adequate preparation for mentors and mentees, matching pairs based on interests, and ongoing mentor communication, training, and evaluation. Bogenschneider (1996) also stressed the need for a mentoring program to have well-defined goals and involve community stakeholders. Similarly, Rhodes, Reddy, Roffman, and Grossman (2005) identified three common elements of good mentoring programs: screening of mentors and mentees, orientation and training, and supervision for mentee-mentor dyads. With these guidelines and suggestions for improvement via research, mentoring has the potential to be an effective intervention. The current study addressed just one aspect of mentoring programs: Mentor training. 17 The Current Study The evidence reviewed suggests that youth mentoring can be effective (DuBois, Holloway, Valentine & Cooper, 2002; Frecknall & Luk, 1992) even if the benefits are small. In addition, there are conditions in which youth mentoring is likely to be more or less effective. This study aimed to report on key factors thought to facilitate the benefits of mentoring such as relationship quality and mentor training. Specifically, the current study evaluated socio- emotional functioning in at-risk youth and investigated outcomes related to mentor training. The current study also intended to include aspects of research design that have been absent in previous research such as random assignment, a comparison group, multi-informants, 9-month follow-up data, psychometrically sound measures, and more sophisticated statistical procedures. This study was different from other studies in that it evaluated two types of mentor training as opposed to solely evaluating if the mentoring program was effective. The study focused on teaching one group of mentors how to facilitate change in a young child by building a strong attachment relationship while another group of mentors received a general communication training. As echoed in the suggested best practices, this study focused on preparing and training mentors by implementing a training program with continued support of the skills through booster quizzes. The Need to Build Warmth/Attachment in Mentoring Relationships From the previous reviewed literature, it is clear that the relationship between the mentor and child is a key factor for youth mentoring effectiveness (Dubois & Neville, 1997; Grossman & Rhodes, 2002; Liang & Rhodes, 2007, Rhodes, Reddy, Roffman, & Grossman, 2005). Some have stated that it is ?the vital element? (Liang and Rhodes, 2007) and other research has likened mentor relationships to attachment relationships (Rhodes, 2005). Attachment theory is most 18 often associated with Ainsworth and the strange situation for infants (Belsky, 2002). However, Ainsworth?s work was heavily influenced by Bowlby and Robertson (predecessors to current attachment theory (Belsky, 2002). The theory of attachment proposes that an individual can be attached to more than one individual (Belsky, 2002). In their model of how and why mentoring relationship work, Rhodes et al., (2006) purport that mentoring relationships are similar to other attachment relationships in that the mentor is a person who can provide warmth and comfort to the child. Similar to parental attachment figures, children learn from multiple experiences that the mentor can be trusted and that their needs (albeit social and emotional) will be met. In this regard, children are likely to form internal working models of positive relationships with adults and peers. As a sense of security is gained, personal growth in the areas of social and cognitive development should develop (Rhodes et al., 2006). Attachment to people other than parents is key in that the parent?child attachment relationship changes over time (Ammaniti, Ijzendoorn, Speranza, & Tambelli, 2000). More specifically, children tend to report more rejection from parents as they get older. Relationship changes such as the above are somewhat consistent with developmental status which suggests that older children are expected to be more autonomous and independent as they get older. Attachment theory also suggests that children during early and middle childhood period (ages 10-14) are less attached to parents and therefore less influenced by parents. In this manner, it may be beneficial to have other influential people in their lives (e.g., peers, teachers, counselors, and mentors) who can serve as positive role models (Ammaniti et al., 2000). Researchers suggest that the mentoring relationship is important. Some studies are now beginning to focus on what makes a good mentoring relationship. Key areas that have been suggested are trust, reciprocity, and respect. Chan and Ho (2008) found a two-factor model using 19 a lexical approach: Relationship Intimacy and Relationship Asymmetry. Relationship Intimacy is related to greater mentoring outcomes and descriptor items include words like trust, harmony, friendly, and close. Strategies for helping to build adult-youth relationships include ?listening, trusting, and relating, shared focus or jointness, and reciprocity? (Zeldin, Larson, Camino, & O?Connor, 2005). Allen and Poteet (1999) evaluated mentoring relationships in the workplace. Mentees were interviewed about their mentor/mentee relationships. The interviews were transcribed and categorized and twenty ?themes? emerged. The most-often cited characteristics of an ideal mentor included the following skills: listening and communication, understanding, honest/trustworthy, caring for each other, and setting goals (Allen & Poteet, 1999). Spencer and Rhodes (2005) identify similar mentor characteristics that are important in youth mentoring, such as showing warmth and empathy, being open and responsive to the mentee?s needs, and being engaging and attentive. Parent-Child Interaction Therapy Parent-Child Interaction Therapy (PCIT) is a behaviorally based intervention for children with disruptive behavior disorders (McNeil & Hembree-Kigin, 2011). PCIT has been described as a probably efficacious treatment for children ages 2-7 with disruptive behavior disorders (Eyberg, Nelson, & Boggs, 2008). Recently, PCIT has been applied to new populations such as older children (ages 8-12) children with a history of child abuse (Chaffin, Silovsky, Funderburk, Valle, Brestan, Balachova, et al., 2004), children diagnosed with developmental disabilities (Bagner & Eyberg, 2007), Separation Anxiety Disorders(Choate, Pincus, Eyberg, & Barlow, 2005), and Autism Spectrum Disorders (Solomon, Ono, Timmer, & Goodlin-Jones, 2008). In addition to using PCIT with new child populations, PCIT is being used with other important adults in a child?s life including grandparents, foster parents, and teachers (Storch & Floyd, 20 2005). Mentoring research suggests that mentors play a key role for their mentees and that the relationship built between mentor and mentee is what makes mentoring work. As such, the main objective of this study was to incorporate aspects of PCIT into a mentoring program. It was expected that a training program informed by a parent child therapy program would help mentors build a better relationship with their mentees based on principles of attachment theory, which are proposed to be a part of the child-directed portion of PCIT (Bell & Eyberg, 2002). PCIT is built on two theoretical models: Hanf?s model and Baumrind?s parenting styles (Querido, Bearss, & Eyberg, 2002). Hanf?s model utilizes a two-stage treatment to teach parents how to increase child compliance via play therapy skills. In the Hanf model, parents are coached in vivo (Eyberg, 2004). Similarly, PCIT involves into two phases, Child-Directed Interaction (CDI) and Parent-Directed Interaction (PDI), and uses live therapist coaching of parent behavior during weekly, one-hour sessions. PCIT is also rooted in Baumrind?s theory of effective parenting, which asserts that optimal parenting takes place within a balance of demandingness and responsiveness. In CDI, the goal is to build warmth with the child by allowing the child to lead and by providing an environment that reduces negativity (responsiveness). In PDI, the parent is coached on how to provide effective discipline (demandingness) within the context of responsiveness. This combination of demandingness and responsiveness is designed to mirror Authoritative parenting which is associated with positive child outcomes (Storch & Floyd, 2005; Querido, Bearss, & Eyberg, 2002). In addition to these two underlying theories, PCIT also builds upon attachment theory and social learning theory. CDI utilizes attachment theory by helping parent and child build a stable attachment (Bell & Eyberg, 2002). Research has suggested that a secure attachment is linked to positive prosocial development whereas maladaptive attachment has been linked to 21 increased aggressive behavior (Querido, Bearss, & Eyberg, 2002). Social learning theory asserts that individuals can learn the consequences of behavior directly or indirectly. PCIT asserts that functional or dysfunctional interaction patterns develop between parent and child based on past contingencies for behavior. Dysfunctional patterns can be interrupted and new patterns can be learned but there must first be a change in parenting behavior (Querido, Bearss, & Eyberg, 2002). PCIT begins with a comprehensive assessment that includes parent and child interviews, psychometrically sound behavior rating scales, and behavior observations of the child in the clinic. The intervention begins with CDI. During CDI, parents are taught nondirective play skills that allow the child to lead the play. These skills are called PRIDE skills. They include Praise, Reflecting the child?s statements, Imitating the child?s play, Describing play activities, and the use of Enthusiasm to keep the child engaged. For misbehavior, parents are taught to ignore while providing praise for appropriate behavior. Parents are also taught to minimize questions and commands, which take away the lead from the child, and to avoid criticism, which creates a negative play environment. Parents are given homework to practice their skills for at least five minutes daily. The CDI phase continues until parents show mastery of the PRIDE skills (Querido, Bearss, & Eyberg, 2002). When CDI is mastered, parents progress to the PDI phase. In PDI, parents are taught to give effective commands to their children and how to follow through with appropriate consequences for child compliance or noncompliance. Similar to CDI, PDI continues until parents show mastery in giving effective commands and following through appropriately (Querido, Bearss, & Eyberg, 2002). In this study, mentors were taught the PRIDE skills typically used in CDI. Although mentors did not have to meet mastery criteria for the skills, continued support and 22 booster quizzes were used to help them acquire the level of skill needed to build a successful attachment with their chosen mentees. Project Uplift Project Uplift is a nonprofit organization that pairs willing adults with children in need. Project Uplift began as part of Big Brothers and Big Sisters in 1973 but was formed into an independent organization in 1979. The purpose of the organization is to uplift the lives of children by providing them with positive role models. Additional goals of Project Uplift are to prevent contact with the Juvenile court system and provide a positive experience for the mentor and mentee. In past years, families have been most often referred by the child?s school (65%). For the 2009 fiscal year, which is the most recent report available, Project Uplift reported that only 23% of their mentees were referred by the child?s school. The majority of children were referred by their parents (71%) in 2009. Referrals may also come from DHR, Juvenile Court of Lee County, East Alabama Medical Center, or Lee County Youth Development Center. For safety reasons, male volunteers are only paired with male children. Female volunteers can be paired with girls or boys under the age of 10. The goals of Project Uplift are evaluated yearly based on survey data completed by parents. According to the 2008 survey data collected by the Project Uplift director, 94% of parents reported child improvement and 93% reported that their child had not had contact with the juvenile court system. Currently, Project Uplift does not use standardized or normed measures to evaluate mentee change. This project added the missing piece of needed objective data. In addition, it provided meaningful training to mentors that could be used to facilitate accomplishing Project Uplift?s goals. As many Project Uplift clients reportedly have behavioral or emotional difficulties the PCIT protocol of skills training and the client needs of Project Uplift fit well. Because many of 23 the home environments in which the children have been raised are less than optimal, Project Uplift children may experience a lack of structure, discipline, and/or affection in the home. It was expected that the PCIT based attachment training would help build warmth by teaching mentors to use the PRIDE skills typically used in CDI. It was also hoped that the attachment training would provide mentors with some strategies for dealing with misbehavior (i.e., ignoring, redirecting). 24 Hypotheses and Analyses Given the importance of the mentoring relationship for mentoring effectiveness and the benefits of PCIT in facilitating a warm relationship, it was hypothesized that children assigned a mentor with attachment training would show improved outcomes relative to mentees assigned to the general communication training condition. In addition, it was expected that mentors and mentees in the experimental group (attachment training) would report greater relationship quality. Below, specific hypothesis are further broken down in accordance with the goals of the study. Preliminary Analyses 1. The psychometric properties of the Mentor-Youth Alliance Scale and the Youth- Mentor Relationship Questionnaire were evaluated. a. It was expected that both measures would show adequate internal consistency. b. Statistical Analysis: Factor Analyses were conducted to evaluate if the factor structure of the measures for this sample was similar to the factor structure of other reported samples. In addition, correlations were used to evaluate the relationship between the two measures. It was expected that they would be positively correlated. Primary Goals 1. Assess if use of the PRIDE skills affected relationship quality. a. It was hypothesized that mentors who received the attachment training would demonstrate greater use of the PRIDE skills as assessed by mentee report on the MEI. 25 b. Statistical analysis: A one-way ANOVA was used to evaluate group differences in mentee responses at post-intervention (Time 2) on the Mentoring Evaluation Inventory. In addition, bivariate correlations were used to evaluate the relationship between the mentee?s report of relationship quality and their perception of the mentor?s use of positive relationship skills (as assessed by the Mentor Evaluation Inventory). 2. Evaluate if children who were assigned a mentor that received the attachment training reported better relationship quality in comparison to children who were assigned a mentor in the control group and if relationship quality changed over time (from Time 1 to Time 2). a. It was hypothesized that participants in the experimental group would report greater relationship quality as measured by mentee and mentor reports on the MYAS and YMRQ. b. Statistical analysis: A 2-way mixed split plot ANOVA (SPANOVA) was conducted to asses if there were changes in relationship quality based on group assignment and from Time 1 to Time 2 (main effects) and if there was an interaction effect between group assignment and time. Secondary Goals 3. Assess if relationship quality affected change in behavioral outcomes. a. It was hypothesized that children who were assigned to the attachment training group would show greater improvements in behavior, as measured by parent and teacher report on the BASC-2, and compared to children in the control condition. 26 b. Statistical Analyses: A 2-way mixed split plot ANOVA (SPANOVA) was conducted to evaluate change in BASC-2 scores based on group assignment and from Time 1 to Time 2 (main effect) and if there was an interaction effect for group assignment and time. The Independent variable was training (standard training vs. additional attachment training). The Dependent variables were the parent BASC-2 composite scores, resiliency subscale, and social skills subscale. T-Tests were also conducted as posthoc analyses to determine any change in BASC-2 scores from pre-intervention (Time 1) to post-intervention (Time 2). Tertiary goals 4. Evaluate the association between relationship quality and behavioral outcomes a. It was hypothesized that relationship quality and BASC-2 scores would be positively correlated for adaptive scales and negatively correlated for maladaptive scales and composite scores. b. Statistical Analysis: Bivariate Correlations were conducted to evaluate if child and mentor reports of mentoring relationship were related to parent BASC-2 scores. 5. Evaluate whether the TAI (a training satisfaction measure) is reliable and if mentors were satisfied with the training provided. 27 Method Participants Participants included clients of Project Uplift (parents and children), volunteer mentors, and teachers (see Table 1 for participant demographics). To be eligible to participate in this study, children had to be a new client with no experience with a previous Project Uplift mentor. Thirty-seven paired dyads (mentors and mentees), 35 parents, and 18 teachers participated in this study. Twenty four dyads were in the experimental group and 13 dyads were in the control group. Clients of Project Uplift are considered disadvantaged families. To be included in the program families had to meet at least one of the following criteria: low income, unmet basic needs, school problems, family dysfunction, personality dysfunction, or trauma. For the 2009 fiscal year, the program reported that 73% of their children were from single parent homes and had a mean income of $22,198. In addition, more than 30% of families endorsed at least 4 or more of the above problems and 68% endorsed 1 to 3 of the above problems. In this sample, 69% of parents indicated they were single or divorced and the mean family income was between $10,000 - $20,000 dollars. In addition, parents reported an average education of a high school diploma. Project Uplift children range in age from 5 to 12 years of age. However, to be eligible to participate in this study, children had to be between the ages of 6 ? 11. In this sample, children were mostly 9-years-old and were in the third grade. Volunteer mentor participants were recruited by Project Uplift. Flyers were distributed in various locations on the Auburn University campus inviting students to attend a nonobligatory training session if they were interested in becoming a mentor. Mentors from the university or 28 from the community-at large are also recruited by viewing the Project Uplift website. To participate in Project Uplift, volunteers must be at least 18 years of age. If they are under 21 years of age, volunteers must have written parental permission. For this project, volunteers had to be at least 19 years of age. Project Uplift reports that volunteers are typically female (86%) and Auburn University students (97%) (C. Nunn, personal communication, March 25, 2009). This was consistent with the current sample of mentors in this study. Procedures In the original study design (see figure 1), potential mentors attended a general training session hosted by Project Uplift. During the training session, volunteers heard a lecture on the purpose of the Project Uplift program, the types of families typically involved in the program and their potential responsibilities as a Project Uplift volunteer. Volunteers also complete several forms to ensure the safety of the child including a background check, fingerprinting, and driving record. Volunteers who were approved were typically placed with a child within 4- 6 weeks after the training session. Due to unforeseen and unavoidable circumstances, some placements took as along as 2-3 months. Volunteers who are accepted into Project Uplift are asked to commit to one year of service (two academic semesters) and to be engaged with their mentees at least 3- 4 hours per week. Upon acceptance, volunteers received an acceptance letter from the program director. At that time, they were informed of an additional training that they needed to complete as soon as possible by signing up with the researcher. In the initial study design, volunteers were randomly assigned to participate in the attachment training or the general communication training session. The researcher sent all approved mentors an email with days and times that the training would take place. This method of recruitment into the study was ineffective as most approved volunteers did not sign up for the 29 additional training session even after several attempts to contact them. For example during the months of September and October, 49 volunteers were approved as mentors. Of those 49, 6 withdrew from Project Uplift. Of the remaining 43 mentors, 22 received the additional training and 18 were matched with mentees. Of the 18 matched dyads, 11 families were unable to be contacted to request their participation in the study. This resulted in 7 participants being added to the study (mentors and mentees) out of a potential 44. Because of these recruitment issues, the method of recruitment was changed at the end of October (see figure 2). Project Uplift staff allowed the researcher to conduct the additional training in tandem with the initial general training session attended by all potential volunteers. As such, potential volunteers received either the attachment training or the communications training (sessions were randomly selected to be in the intervention or control condition) before they were approved and matched with a mentee. Therefore, some potential mentors that were trained did not continue with Project Uplift (due to a decision not to become a mentor or they were not approved). Teachers were also involved in the project as participants. If parents granted the researcher permission to talk with the child?s teacher, the teacher was contacted. At pre- intervention (Time 1) 37 teachers were contacted and asked to participate in the study. Eighteen teachers returned the consent form and completed BASC-2-TRF form for the mentee that was in their class. At post-intervention (Time 2) 11 teachers returned the consent from and completed BASC-2-TRF form. Of the 11 forms completed at Time 2, only 3 were for children who also had teacher report information (BASC-2-TRF) at Time 1. Training Sessions The attachment training consisted of a teaching session about the PRIDE skills, role plays practicing the skills, a quiz, and review of materials. In addition, the mentors completed a 30 demographic questionnaire, a relationship quality measure, and a Training Attitude Inventory. The training sessions took approximately 1 to 1.5 hours. At the completion of the training, the potential mentors were asked to give their consent for information that they provided during the training session to be used for research. (Participation in the training was required by Project Uplift; however, participation in research was voluntary). All mentors who received the training gave consent and were given a copy of the informed consent form. Potential mentors who attended the control group meetings did not receive the attachment training. Upon arriving at the training location, they were given a handout about effective communication, read the informed consent form, and were asked to complete a demographic questionnaire and a relationship quality measure (See appendix for handout). The general communication training took approximately 30 minutes to complete. After both types of training sessions (attachment training and general communication training), individuals who decided to pursue involvement with Project Uplift completed the selection process. Those volunteers who were selected to become Project Uplift mentors met with the Project Uplift director to select their mentee. In the revised research design, the Director of Project Uplift provided the researcher with a list of all the new mentor/mentee relationships and client contact information as mentors were matched with mentees. The researchers called the client?s parent and he/she was given an overview of the study. If the parent was interested in participating in the project, a meeting was set-up with the parent and child. If parents authorized a meeting, the visit took place within 4 weeks. Four weeks was set as a deadline to try to assure that the relationship with the mentee was still in its early stages. At times it was difficult to contact parents and therefore visits may have taken place outside of this initial 4-week period. 31 At the in-home visit, parents were given an overview of the study and the consent form was reviewed. If parents chose to participate, they completed the BASC-2- PRS, a demographic questionnaire, and were asked to give the researcher permission to contact the child's teacher to complete the BASC-2-TRS. Children were also provided with an overview of the study and were asked to participate by signing an assent form. Children completed the Me and My Mentor Questionnaire and the Mentor Evaluation Inventory. Upon completion of the questionnaires, the parent received $10.00 and the child received a small toy or candy (with parent permission). After being given permission by the parent, the researcher contacted the child's teacher either directly or via the school principal. Teachers were sent a letter that included information about the study, a consent form, and the BASC-2-TRS. Initially the researcher planned to collect follow-up measures at 3, 6, and 12 months. Due to low participation at 3 months (three out of three families declined), parent dissatisfaction at the frequency of data collection, and the restricted time frame due to recruitment difficulties, post intervention measures were obtained at 9 months. Although 12 months has been noted as the suggested time frame to see maximum mentoring benefits, some studies demonstrate that mentoring can be effective for shorter relationships. For example, Goldner and Mayseless (2009) reported positive relationship quality between mentee-mentor dyads at an 8-month assessment. Their study found that the dyad relationship improved over time and that the positive relationship was associated with positive academic, social, and emotional functioning. These findings provide some support for the shortened (9-month) follow-up period. To assess mentoring integrity, Project Uplift staff contacts mentors and clients monthly. For the monthly check-in, mentors were asked to report the number of visits, number of hours, and number of contacts (includes telephone calls and emails) made with their child within the 32 last month. In addition, they are asked to list the dates they met with their child, the activity completed, and how the relationship is progressing. In addition to the monthly check-in about their volunteer activities, mentors also complete a monthly survey about their volunteer experience. This includes how the mentor/mentee relationship is progressing and the most recent activity with the child. For this project, three additional questions were added to the monthly survey that mentors provided about their experience with volunteering: Have you used the skills you learned in training? How often have you used each of the following: praise, reflections, behavior descriptions, and imitation? and Are there any problems to report? (See appendix). The inclusion of the additional questions was to evaluate if mentors were using the skills and to serve as a reminder of the skills. These monthly check-ins were part of Project Uplift normal procedures and are required of all mentors. The researchers were granted access to the monthly reports for mentors in the control and experimental groups. To increase exposure to the training materials, mentors in the experimental group also received monthly assignments from the experimenter. The assignments were distributed via the volunteer?s preferred email address. The assignments included a brief scenario and 2-5 follow-up questions. The scenarios included examples of mentors using the attachment skills (see appendix). As an incentive for completing the assignment, mentors were entered into a raffle to win $200 for every month they completed the assignment. Because completion of the assignments was not mandated by Project Uplift, participation was low. Mentors in the general communication training and the attachment training groups were contacted at 9 months to complete the mentoring relationship measures. 33 Measures Demographic Questionnaire. The demographic questionnaire requested information about the parent and child. Parents completed the survey and answered questions with regard to their gender, age, ethnicity, level of education, marital status, and household income. In the current sample, parent age ranged from 24 to 56 (M = 36.27, SD = 8.9), 91.9 % were African American, and 8.1 % were Caucasian. As reported previously, 69% were single or divorced and most parents reported obtaining a High School diploma. In regards to the child, the parent provided information on the child?s date of birth, age, gender, ethnicity, length of time on Project Uplift?s waiting list, school, and grade. Children were between that ages of 9 and 11 (M = 9, SD = 1.67) and 64.7% were female. Time on the waiting list ranged from 1 month to 24 months, with an average wait time of list 7.75 months (SD = 7.08). Additional information on the members of the household (e.g., number and ages of children living in the home and other children participating in Project Uplift) was acquired. (See Table 1). Demographic information for the mentors included gender, age, ethnicity, level of education, marital status, and household income. Mentors ranged in age from 19-25 (M = 19.70, SD = 1.36), were mostly Caucasian (90.9%) and female (80.8%). Mentors also provided information about their reasons for volunteering. They were given a list of possible reasons for volunteering and were asked to rank the importance from 1 (most important) to 7 (least important). All mentors reported that they were volunteering to give back to the community (M = 1.9) and to enhance their own life or knowledge (M = 1.8). Thirty-eight percent of the mentors reported that they were volunteering to receive course credit and on average, they ranked the importance of course credit as 4.5. The demographic questionnaire also obtained information about the mentor?s amount of experience working with children (often for babysitting, work in a 34 daycare/preschool setting, and volunteering with children in other settings), and courses taken related to psychology or child development (M = .67). The Behavior Assessment System for Children - Second edition (BASC ? 2, Reynolds & Kamphaus, 2004). The BASC-2 is a standardized and normed measure that assesses adaptive and maladaptive behavior. The BASC-2 is a multimethod system that allows for multiple informants to report across multiple contexts (school, home) and spans ages 2- 25. The versions used in this project were the Parent Rating Scale and the Teacher Rating Scale for children ages 6-11. The forms have 160 questions rated on a 4-point scale of how of often the child behavior occurs. The BASC-2 was normed in 2004 and included a general sample population greater than 13,000. The sample characteristics (sex, ethnicity, SES) were similar to the characteristics of the US population according to the Current Population Survey conducted in 2001 (Reynolds & Kamphaus). The BASC-2 includes several composites (Adaptive Skills, Behavioral Symptoms Index, Externalizing Problems, Internalizing Problems, and School Problems), clinical scales, adaptive scales, and content scales. The parent BASC-2 and teacher BASC-2 have good reliability and validity. Reynolds and Kamphaus (2004) reported that the reliability for TRS ranged from .77 to .97 with an average median score of .88. Scales with the greatest reliability are the scales that make up the Externalizing composites score (Hyperactivity, Aggression, Conduct Problems, and Attention Problems) (Reynolds & Kamphaus, 2004). Validity was assessed via scale intercorrelations, factor analyses, and comparison to similar measures. The large sample used and the diversity of the population used (sex, ethnicity, diagnosis, SES), coupled with the good reliability and validity, make the BASC-2 a good instrument for assessing difficulties and improvements in children. Test-retest scores also provide a measure of reliability. The median 35 time span for retest administration was six weeks. Test-retest correlations were high (ranging from .64 to .90 on teacher rated scales, and .65 to .88 on parent rated scales) indicating that the test is measuring a consistent concept. Composite scores yielded higher reliability scores (ranging from .78 to .93). In addition, scores were most reliable for the child age group (ages 6- 11). This study utilized the Externalizing, Internalizing, Behavior Symptoms Index, and Adaptive Behavior Composites. In addition, the Social Skills and Resiliency scales were used in the analyses. For behavioral and emotional composites and scales, T-scores of 70 and above are considered clinically significant. T-scores of 60-69 are considered at-risk. For adaptive composites and scales, T-scores of 30 and below are considered clinically significant and T- scores of 31-40 are considered at-risk. Mentor-Youth Alliance Scale (MYAS; Zand, Thomson, Cervantes, Espiritu, Klagholz, LeBlanc, & Taylor, 2009). The MYAS was developed to assess the positive aspects of mentoring relationships. Its psychometric properties have been tested in two separate studies. In the first study, 276 participants? ages 9-19 years old completed 31 items (Zand et. al., 2009). The items were developed by expert judges in the areas of high-risk youth and mentoring. Questions were rated by participants on a 4-point scale from (1) very true to (4) very false. Twenty-one items that showed poor inter-item correlations or were confusing were dropped, leaving 10 items. A principle components factor analysis was performed on the 10 items. The results suggested a two factors model: acceptance and caring. This model showed good internal consistency ? = .85. In the second study, 219 participants ages 9 to 16 participating in mentoring project completed the MYAS (Zand et. al., 2009). Participants were reassessed at 8 months for follow-up. The scales showed good reliability ? = .84 (for caring) and ? = .91 (for acceptance). 36 Confirmatory factor analysis was performed to confirm the factors found in the first study. Results showed that the original two factors were highly correlated (r = .95) and showed poor discriminate validity, therefore a one-factor model in which items were allowed to cross load was used. This model was a better fit for the data. The one factor model (Mentor-Youth bond) was retained. This model of the MYAS showed good internal consistency ? = .92. Scores can range from 10 ? 40. In the original studies, items were reverse scored so that lower scores indicated greater mentoring alliances. For this study, the one factor model was used. All items for the measure were retained, however, the scale was modified such that ?1? corresponded to very false whereas ?4? corresponded to very true; therefore higher scores reflect a greater mentoring alliances. In the current study, the MYAS also showed good internal consistency at Time 1 (? = .76) and Time 2 (? = .91). Youth-Mentor Relationship Questionnaire (YMRQ, Rhodes, Reddy, Roffman, & Grossman, 2005). The Youth Mentor Relationship Questionnaire was adapted from two previous measures developed by Morrow and Styles (1995) and Lynch and Wellborn (1987). It assesses the youth?s positive and negative experiences and impressions about his or her mentor. This measure was evaluated using 347 youth who were assigned to mentors through the Big Brothers Big Sisters mentoring program. Participants ranged in age from 9 -16. The measure was given to participants at post-test only and consisted of 74 items. Items were rated on a 4-point likert scale. Items were dropped if they showed high inter-item correlation or were ambiguous. This resulted in 61 items being used for exploratory factor analysis. Fifteen factors were found, however, the first 4 accounted for the most variance. The factors were identified as: trust not broken, not dissatisfied, helped to cope, and not unhappy. The first four factors included 15 items and these items were retained and used in confirmatory factor analysis. Four factors were entered 37 into the model based on the exploratory factor items. In this model, items were not allowed to cross load and the initial model fit the data poorly. Based on mentoring research, adjustments were made allowing three items to cross load. This modified model fit the data well. Internal consistency was adequate to good (? = .74 to ? = .85). Scores can range from 15 - 60 with higher scores reflecting a greater mentoring bond. In this study, the four factor model was used and the internal consistency was variable. At Time 1, the measure did not show good reliability (? = .22); however, reliability was improved at Time 2 (? = .76). To minimize the amount of paper work given to children, the MYAS, and YMRQ were combined into one measure: Me and my mentor. All items were retained from the MYAS and YMRQ. Me and my mentor uses a 4-point likert scale with 1 = Not true at all and 4 = Very true. In addition to the likert scale, a smiley face 4-point likert scale was used to assist children that had difficulty with the numeric scale. The first 15 items were from the YMRQ and the last 10 items were from the MYAS. These combined measures were also modified to access the mentor?s perspective of the mentoring relationship. For instance on the mentee version an item states, ?My mentor sometimes promises me that we well do something and then we did not do it.? On the mentor version, the same item read, ?I sometimes promise my mentee that we will do something and then we do not do it.? This mentor version of the relationship questionnaires, the Mentoring Relationship Questionnaire, is also rated on a 4-point likert scale and scores can range from 25 to 100. As with the mentee relationship questionnaires, the mentor version of the MYAS (MYAS-MV) and the mentor version of the YMRQ (YMRQ-MV) will be analyzed separately. Mentor Evaluation Inventory. This questionnaire was based on a previous measure, the Volunteer Evaluation Inventory, developed by Meenakshi Lambha (2010, unpublished manuscript) to obtain child report of their interactions and relationship with their volunteer in a 38 community setting. The Mentor Evaluation Inventory is comprised of exact items on the Volunteer Evaluation Inventory; however, the word volunteer has been changed to mentor for each item. The measure has 12-items and is rated on a 7-point scale ranging from 1 (Is not true at all) to 7 (Is exactly true). In addition to the likert scale, a smiley face 7-point scale was used to assist children who had difficulty with the numeric scale. Items on this measure correspond to specific skills taught in the PCIT CDI teach session and to material learned in the attachment training session (e.g., attention, praise, descriptions, reflections, and enthusiasm). Scores can range from 15 ? 84, with higher scores reflecting greater use of the PRIDE skills. Training Attitude Inventory. This inventory was developed by Meenakshi Lambha (2010, unpublished manuscript) to assess the impact of training on volunteers? skills and child client behavior. The measure has 8 items and is rated on a 5-point Likert-type scale with scores ranging from 1 (dissatisfaction with training) to 5 (maximum satisfaction with training). This inventory is based on the Therapy Attitude Inventory (TAI; Eyberg, 1993; Eyberg & Johnson, 1974), which was developed to access the impact of behavioral parent training programs such as PCIT. The TAI has good internal consistency (Cronbach?s alpha = .88) and has demonstrated sensitivity to changes due to treatment. For this scale, higher scores reflected higher training satisfaction. Scores can range from 5 ? 50. 39 Results Preliminary Analyses It was expected that the mentee and mentor relationship measures would show good internal consistency at pre-intervention (Time 1) and post-intervention (Time 2). Based on mentee report, The Mentor Youth Alliance Scale (MYAS) demonstrated good internal consistency at Time 1 (? = .762) and at Time 2 (? = .905). However, the Youth Mentor Relationship Questionnaire (YMRQ) did not show good reliability at Time 1 (? = .219). Examination of the covariance and correlation matrices was helpful in explaining these results. The covariance matrix had several items that negatively covaried (see Tables 2 and 3). This may be due to the small sample size, small number of items comprising the scale (n = 11 after four items were dropped from the scale because the variance was zero), or because of the negative wording for some items. At Time 2 the reliability for the YMRQ improved (? = .755) and only one item was dropped from the analysis due to zero variance. The MYAS-MV and the YMRQ- MV (mentor versions) demonstrated adequate to good internal consistency at Time 1 (YMRQ- MV: ? = .704, MYAS-MV: ? = .916) and Time 2 (YMRQ-MV: ? = .638, MYAS-MV: ? = .867). It was also expected that the MYAS and YMRQ would be positively correlated as they both purportedly measure relationship quality. At Time 1 there was only a small insignificant correlation between the MYAS and YMRQ. This may be a manifestation of the low reliability of the YMRQ at time 1. At Time 2, the MYAS and YMRQ were positively correlated as expected (r = .426, p = .009). In addition, the YMRQ-MV and MYAS-MV were positively correlated at Time 1 (r = .624, p < .001) and Time 2 (r = .676, p= .016) as expected (see Table 4). Rhodes and colleagues reported a four-factor structure for the YMRQ in their 2005 study. An exploratory factor analysis was used to assess the structure of the YMRQ in this study. The 40 exploratory factor analysis resulted in a five factor structure that accounted for 62.95% of the variance and all items adequately loaded on a scale (.32 or greater) (see Tables 5).An exploratory factor analysis was repeated for the YMRQ Mentor Version (YMRQ-MV) and revealed a four factor structure that accounted for 83.235% of the variance. This model resulted in two questions that did not load well on any scale: When my mentee is with me s/he feels bored (all negative loadings) and I wish I knew my mentee better (highest loading of .156 on factor 4). Exploratory factor analyses were also conducted for the MYAS and MYAS Mentor version (MYAS-MV). The EFA for the MYAS demonstrated a three factor structure and accounted for 68.969% of the variance. For the MYAS-MV, an exploratory factor analysis resulted in a one factor structure that accounted for 71.861% of the variance. The one factor model fit the data well. Hypothesis 1 It was expected that mentors who received the attachment training would demonstrate greater use of the PRIDE skills as assessed by mentee report on the MEI. This hypothesis was not supported. Mentors in the experimental group did not use more reflections, praises, or descriptions when compared to mentors in the control group, F (1, 37) = 2.087, p= .57. As the PRIDE skills are utilized in building attachment, it was expected that mentors who used the PRIDE skills frequently would have a better relationship with their mentees. Bivariate correlations were used to examine the relationship between use of the PRIDE skills (as measured by the MEI) and relationship quality (as measured by the MYAS and YMRQ) for both the experimental and control group combined. At Time 1, there was a positive relationship between the MEI and relationship quality on the MYAS (r = .383, p = .019) and the YMRQ (r = .335, p = .043). This indicates that when mentees perceived that their mentors used the PRIDE skills 41 they also reported a more positive relationship with their mentor. At time 2, the results were mixed. On the MYAS, there was a small relationship with the MEI however, it was not significant (r = .240, p = .153); however, on the YMRQ there was a strong relationship with MEI (r = .623, p < .001) that was significant (see Table 4). When the relationship between the PRIDE skills and relationship quality were examined separately by group (experimental vs. control), a different profile emerged. Overall, the Experimental group (which received the attachment training) showed significant positive correlations at Time 1 and Time 2 on the MYAS. For the YMRQ there was a strong positive correlation at Time 2 however, the relationship was weaker and not significant at Time 1. The control group had small positive correlations that were not significant on the MYAS and YMRQ at Time 1 and small negative correlations that were not significant at Time 2. Hypothesis 2 It was hypothesized that participants in the experimental group would report greater relationship quality as measured by mentee and mentor reports on the MYAS and YMRQ. A 2- way mixed split plot ANOVA (SPANOVA) was conducted to asses if there were changes in relationship quality based on group assignment and over time (from Time 1 to Time 2) and if there was an interaction effect between group assignment and time. Based on mentee report of relationship quality, the SPANOVA was not significant for the YMRQ, Wilks? lambda F (1, 35) = .001, p = .978, or the MYAS, Wilks? lambda F (1, 35) = .062, p = .805. These results indicate that from the child?s perspective there were no changes in the mentee-mentor relationship from pre-intervention (Time 1) to post-intervention (Time 2) and that the quality of the mentoring relationship did not differ based on which training their mentor received (see Tables 7 & 8). 42 Paired samples t-tests confirmed the results that there were no changes from Time 1 to Time 2 on the MYAS, t (36) = -.752, p = .457) or the YMRQ, t (36) = -.841, p = .406. The SPANOVA was repeated using mentors reports of relationship quality. The interaction term was not significant for the MYAS-MV, Wilks? lambda F (1, 9) = .406, p = .540; however, there was a significant main effect for time, Wilks? lambda F (1, 9) = 49.156, p < .001. Similarly, the main effect of time was significant for the YMRQ-MV, Wilks? lambda F (1, 9) = 41.427, p < .001, whereas the interaction was not, Wilks? lambda F (1, 9) = .615, p = .453. The results indicate that mentors reported a change in relationship quality from Time 1 to Time 2; however, relationship quality did not differ by training group (see figure 3). These results indicate that the additional attachment training was not effective in facilitating relationship growth beyond the general training that all mentors received. Mentees and mentors perceived a positive relationship at Time 2 despite the type of training mentors received at the beginning of the study period. Hypothesis 3 It was hypothesized that children who were assigned to the attachment training group would show greater improvements in behavior, as measured by parent and teacher report on the BASC-2, as compared to children in the control condition. A 2-way mixed split plot ANOVA (SPANOVA) was conducted to evaluate change in BASC-2 scores based on group assignment and across time. This hypothesis was not supported. There were no relationships between the BASC-2 scores and the relationship questionnaires based on mentee report. There were no main effects of group or time and the interaction was not significant for externalizing problems, internalizing problems, adaptive skills, or resiliency (see Table 9). For social skills, the interaction term was not significant, Wilks? lambda F (1, 30) = .412, p = .526; however there 43 was a main effect for time, Wilks? lambda F (1, 30) = 9.525, p = .004.This finding was counter to other studies that reported significant changes in behavior after participation in a mentoring program. Hypothesis 4 It was also expected that the quality of the mentor-mentee relationship (as reported by the mentee) would be positively associated with child adaptive behaviors and negatively associated with child maladaptive behaviors (as measured by the BASC-2). This hypothesis was not supported: Externalizing problems (MYAS: r = .287, p = .111 and YMRQ: r = -.105, p = .568), Internalizing problems (MYAS: r = .287, p = .054 and YMRQ: r = .272, p = .139), Adaptive skills (MYAS: r = -.114 p = .540 and YMRQ: r = .280, p = .128), Resiliency (MYAS: r = -.074, p = .686 and YMRQ: r = .112, p = .540), or Social Skills (MYAS: r = -.062, p = .738 and YMRQ: r = .302, p = .093). To assess if there were any changes in BASC-2 scores (see Tables 7 & 8), a difference variable was computed using the formula of BASC-2 scale score Time 2 ? BASC-2 scale score Time 1 and then recoding the values into dummy variables. For maladaptive behaviors, negative differences reflected a decrease in maladaptive behaviors and were coded as a 1 (positive change), positive differences reflected an increase in maladaptive behavior and were coded as a - 1 (negative change), and a score of 0 indicated no change. Adaptive behaviors were reverse coded (-1 = positive change, 1 = negative change, 0 = no change). Descriptives showed that 59.2% of children showed a decrease or no change on Externalizing problems, 71.8% showed an improvement in social skills, 54.9% improvement in Adaptive skills, and 61.3% increased resiliency. However, only 41.9% showed an improvement in Internalizing problems indicating that at Time 2 children were exhibiting more internalizing problems. Overall, these scores 44 suggest that were some positive behavioral changes from pre-intervention to post-intervention however, they were not large enough to reach statistical significance (see Figure 4). Teacher report of BASC-2 scores at follow-up could not be analyzed by group due to a low response rate from teachers of children in the control group (n = 1). Additional Analysis The psychometric properties of the TAI were evaluated. Overall, it showed good internal consistency, ? = .737. This finding is similar to results for a study by Lambha (2010) in which the TAI was utilized to evaluate satisfaction for a modified CDI training (? = .70 at time 1 and ? = .88 at time 2). In this study, the TAI was also used to evaluate if mentors were satisfied with the additional trainings. A chi-square analysis revealed that there were no differences in satisfaction based on which training mentors attended, ?2 = 17.4, p = .296. The mean for the TAI indicates that mentors were generally satisfied with their training (M = 4.227, SD = .552). 45 Discussion The purpose of this study was to evaluate if mentors could learn the PRIDE skills typically taught to parents and if knowledge and use of these skills would translate to greater relationship quality with their mentors. In addition, this study sought to examine if mentee- mentor relationship quality was subsequently related to behavioral outcomes. Overall, there was little support for the hypotheses in this study. Some difficulties may have been due to methodological and/or measurement problems. First, the sample size for this study was quite small (n = 37). This resulted in low power and may have affected the ability to detect changes between groups and over time. Over the year in which participants were recruited, Project Uplift reported that 294 potential mentors attended their general training sessions. Of the 294 potential mentors, 114 (39%) were accepted and placed with a child. 120 mentors or potential mentors were provided with an additional training (86 in the attachment training and 34 in the communication training). The number of participants receiving the additional training is greater than the number of mentors placed with a child because some of the additional training sessions took place during the general training session, which was before mentors were matched. This was not the original design of the study but due to the low response rate for matched mentors signing up for the additional training, the time frame for the when the training was conducted was changed. Of the 120 mentors who received the additional training, the total number of participants decreased to 37 after some potential mentors were not approved or voluntarily withdrew from Project Uplift, or because the researchers were unable to make contact with Project Uplift clients. Therefore, at the outset, the study began with difficulties in recruiting and maintaining participants, which resulted in a smaller than anticipated sample size. Small sample size and low power have been cited as challenges by other studies on mentoring and these factors 46 were difficult barriers to overcome in the current investigation (DuBois, Doolittle, Yates, Silverthorne, & Tebes, 2006). Sullivan and colleagues provided some suggestions on overcoming barriers in community based research (Sullivan, Kone, Senturia, Chirsman, Ciske, & Krieger, 2001). First, community partners should be involved in every aspect of the project design and implementation. When they are not, conflicts may arise due to disruptions in daily operations caused by the research process (Sullivan, et al., 2001). In this project, the study design was created solely by the primary researcher and was then proposed to the community partner. The community partner agreed to the design and facilitated implementation however; the study procedures required a change in operations and the input of the community partner was not directly solicited. Had the community partner been consulted before the design was created some of the problems with recruitment may have been resolved. In addition to not including the input of the community partner, the opinion and needs of other stakeholders such as Project Uplift clients and mentors were not requested by the researcher. Chambers (2008) suggests that there are multiple levels (e.g., client, provider, supervisor, and state levels) and multiple stakeholders in community research and that these levels are interdependent and should be collaborative. A lack of a collaborative approach manifested in difficulties with recruitment and compliance to the current study?s procedures. This study also incurred problems with the psychometric properties of the YMRQ. At Time 1, the YMRQ did not show adequate internal consistency. This may be an artifact of the small sample size and a small number of items comprising the scale (Gliem & Gliem, 2003). In addition, it could suggest that at Time 1 the scale was not measuring relationship quality. This assumption is supported by the positive skew of responses (an average of 6.47 out of 7), which indicated that mentees reported a strong relationship very early in the mentee-mentor 47 partnership. Of note, many children had been on the waiting list for a mentor for six months or longer. Perhaps being paired with a mentor elicited positive feelings, which were then reflected on their responses on the relationship scales at Time 1. The YMRQ was developed for children ages 9-16 (Rhodes, Reddy, Roffman, & Grossman, 2005). It was used with a younger sample in this study, which may explain the poor reliability. In fact, this study showed a very different factor structure than what was reported by Rhodes and colleagues (2005). Instead of a four factor structure, a five factor structure fit the data better. Even with this factor structure items loaded on multiple scales and had factor loadings that were less than .32, which suggests the items were not a good measure of the factor (Tabachnick & Fidell, 2007). Despite the reason for the inconsistency on the scale, it is important to recognize that the low internal consistency may have influenced other statistical procedures that included that scale. For instance, the low internal consistency on the YMRQ may explain why the MYAS and YMRQ were not correlated as expected at Time 1. In this study, the MYAS showed good internal consistency at both time points, however, factor analyses for the mentee version revealed a different factor structure from the previous literature. Zand and colleagues (2009) suggested a one factor model, however; an Exploratory Factor Analysis resulted in a three factor model that accounted for 68.969% of the variance (Zand, Thomson, Cervantes, Espiritu, Klagholz, LeBlanc, & Taylor, 2009 Although, factor analyses are helpful in understanding the relationships between items and variables and for data reduction, there are several caveats to their use (Tabachnick & Fidell, 2007). For example, factor analyses are sensitive to sample size with smaller sample sizes being less reliable. A sample size of 300 has been suggested for obtaining reliable coefficients (Tabachnick & Fidell, 2007). In this study, factor analyses for the relationship measures deviated 48 from the reported factor structure of the measures in previous literature. However, this information should be interpreted with caution given the small number of cases included in the factor analyses. The first hypothesis posited that mentors who received the attachment training would demonstrate greater use of the PRIDE skills as assessed by mentee report on the MEI. This hypothesis was not supported, as there were no differences for the use of the PRIDE skills for mentors who had received training on the PRIDE skills and mentors who had not received the attachment training. Several reasons may account for the lack of group differences. For example, mentors in the control group may have been inadvertently primed to use the skills when they completed their bi-weekly reports. The additional questions on the biweekly report that assessed use of the PRIDE skills (e.g., Have you used the skills you learned in training? How often have you used each of the following: praise, reflections, behavior descriptions, and imitation? Are there any problems to report?) were given to all mentors regardless of the type of training they had received. These questions may have served as a prompt for mentors to use the skills violating the assumption that they had no exposure to the PRIDE skills. Even though there were no group differences in use of the PRIDE skills, correlational data suggest that there was a stronger relationship between the MEI (an indirect measure of PRIDE skills) and the relationship questionnaires for mentors that received the attachment training. These results suggest that the additional training in the PRIDE skills and use of the PRIDE skills is associated with good relationship quality. This was evident by the negative and nonsignificant correlations between the MEI, MYAS, and YMRQ for the mentors who did not receive training in the PRIDE skills, but positive correlations for mentors that did receive the training. 49 Another explanation that may explain the lack of group differences on the MEI is the use of subjective data versus objective data to measure use of the PRIDE skills. A questionnaire that assessed mentors? knowledge of the PRIDE skills after the training indicated that they learned the basic skills (an average of 94% correct). However, it is unclear if the skills were maintained or used subsequent to the training. Other recent studies have also evaluated the acquisition and application of the PRIDE skills in undergraduate populations. Lee, Wilsie, and Brestan-Knight (2011) evaluated college student?s ability to learn PCIT skills and implement them in an analog setting. Participants completed pretest and posttest quizzes to assess knowledge of PCIT and engaged in an analogy play setting before and after exposure to PCIT materials. After pretest assessment measures were collected, participants were presented a brief training that included a modified version of the CDI and PDI teach sessions as well as modeling and role plays in large group sessions. Knowledge of PCIT and related skills were assessed via pretest and posttest quizzes and application of the skills was assessed via a behavior coding system of dyad verbalizations at pretest and posttest. In regards to student knowledge, the results were similar to this study in that students increased their knowledge of the material following the teach sessions. However, participants only demonstrated an increase in praise at posttest and they were unable to meet mastery criteria (Lee, Wilsie, & Brestan-Knight, 2011). Anecdotally, mentors in this study reported that they used the PRIDE skills frequently or on every outing and in addition, Praise was the most often cited skill used. This study and the study by Lee and colleagues suggest that continued practice is needed to see an increase in skills. A recent study by Lambha (2010), which evaluated the effectiveness of a modified CDI training with community volunteers, echoes the need for additional practice 50 and exposure to the basic skills. In her study, volunteers engaged in either a group teaching session (Group Volunteer Child Interaction Training, G-VCIT) or a group teaching session plus individual training (Individual Volunteer Child Interaction Training, I-VCIT). Participants in the G-VCIT group received a modified CDI teaching session that was similar to the attachment training in this study (lecture, modeling, and role-plays). Participants in the I-VCIT group received the CDI teaching material but in addition, they participated in in-vivo practice with coaching. The results from the study suggest that volunteers in both groups were able to demonstrate knowledge of the skills as assessed by a quiz. In addition, the volunteers in the I- VCIT group were able to demonstrate some ability to apply the skills in a real setting, however, their performance was variable. Lambha (2010) suggests that additional teaching sessions, more coaching time, and coaching to criteria may have been needed to increase the volunteer?s application of the skills. The problem of skill application and maintenance was foreseen during the design phase of the current study and the researcher took steps to build in additional exposure to the attachment skills. To compensate for the constricted training time and lack of in-vivo practice, monthly booster quizzes were built into the research design in hopes that the additional written practice might increase mentors? competency with the skills. Unfortunately, this aspect of the study was completely voluntary which resulted in an overall low response rate (less than 10% overall). The low response rate from the mentors was not unique to this study. The low return rate of biweekly questionnaires that are required by Project Uplift suggests mentors generally responded inconsistently. The second hypothesis examined if there were group differences between reports of relationship quality based mentee and mentor reports on the MYAS and YMRQ and changes in 51 relationship quality over time. There was only partial support for this hypothesis. Based on mentee report, there were no differences in relationship quality by group or over time. However, mentors reported a change over time but there were no differences by group. Dose effects of the intervention may help to explain the lack of group differences. In traditional PCIT, parents are coached to criteria and must meet mastery of the skills to progress in treatment, which may take several weeks and even months. Training to mastery criteria was not feasible for this study. Instead, mentors received a significantly abbreviated version of the training that parents typically receive during clinical intervention. PCIT has been adapted to be implemented in alternative formats including abbreviated sessions. For instance, PCIT has been implemented in time limited group sessions (ranging from 8-12 weeks) and as a 2-day workshop for foster parents with good results (Wagner, 2010). These abbreviated formats keep many of the core elements of PCIT (such as the PRIDE skills and behavior management) but may not incorporate other traditional areas such as coaching and mastery criterion. Similarly, this study utilized an abbreviated training session but did not include coaching to mastery. Recently the boundaries in which abbreviated sessions are useful have been evaluated. In their 2010 study, Berkovits and colleagues evaluated the effectiveness of a 4-session abbreviated PCIT intervention versus a PCIT Anticipatory Guidance intervention (PCIT-AG), which only included written materials that explained the principles of PCIT, each phase of PCIT, and tips to use at home. The results showed no differences between the 4-session group and the Anticipatory Guidance group. The above study provides preliminary support for an abbreviated form of PCIT that time limit and does not utilize coaching and the mastery criteria; however, the abbreviated version used in this study may have excluded too many necessary components of PCIT to be sufficient for mentors to acquire the skills and put them into practice. For example, 52 there was only one session instead of four and limited handouts provided to the mentors in the attachment training condition for practice. An early study by Doley, Dorsi, and Cartelli (1976) evaluated the effectiveness of parent training techniques. There were two phases of parent training. The first phase included lectures, readings, modeling of techniques, and role plays. The second phase included parents being taped interacting with their kids and then feedback. The goals were similar to PCIT in that they hoped to increase positive statements (such as praise) while decreasing questions and commands. Parents and children were observed in a play setting at baseline, after phase 1 and after phase 2. The results showed no significant differences between baseline and phase 1. However, there were significant changes from baseline to phase 2 and from phase 1 to phase 2. The results from this study suggest that exposure to the information and even the ability to demonstrate the skills may not be enough to affect behavior change. The current study utilized a teaching method very similar to Doley et al.?s phase 1 and also did not find changes in behavior. Feedback seems to be a needed component and is a part of traditional PCIT, which has been shown to affect changes in behavior. In regards to changes in child behavior, the results of the current study suggested that relationship quality was not related to behavioral outcomes. In addition, the mentees did not show significant changes in behavior overtime as expected. The only significant change was an increase in parent reported social skills from pre-intervention to post-intervention. Other studies such as Chan and Ho (2008), also found positive outcomes for social skills over time. However, there were no changes in externalizing and internalizing behaviors, adaptive skills, or resiliency for the current sample. These results may have been due to participant characteristics that were unique to this sample. In this study, participants were classified as ?at-risk? according to Project 53 Uplift?s guidelines demonstrating a need based on disadvantages due to family circumstances (i.e., mother had first child as a teenager, poverty) or because of child?s difficulties (child is having a problem at school, experienced major trauma). This is consistent with Blechman?s theory (1992) that children from multi-problem homes (such as those with teen moms, poverty, and low SES), are less likely to have opportunities to develop appropriate coping skills which may lead to emotional and behavioral problems. In this sample, the clients can be classified as at- risk based on family circumstances such as low SES and single parent homes. Although Project Uplift reports that many parents report behavior problems in their children or children are referred because of problems at school, this sample did not exhibit maladaptive social and behavior functioning as assessed by the BASC-2. According to pretest BASC-2 scores, 8% of children had clinically significant scores on Externalizing problems scale and 3% had clinically significant scores on Internalizing problems scales. If at-risk scores are added, the percentage increased to 18% and 14% respectively. These percentages are lower than expected for an at-risk sample using a clinically-based measure. The mentoring literature reports that children who are at-risk are the most likely to benefit from mentoring (DuBois, Holloway, Valentine, & Cooper, 2002) and, in fact, children who are well adjusted may not benefit from mentoring at all. In sum, change over time and change based on group differences may not have been evident because the sample was not a true ?at-risk? sample for behavior problems. In addition, because the children?s scores were in an adaptive range at pre-intervention it was less likely that the children would display behavioral or emotional changes following their involvement in the mentoring program as there was not much room for improvement. From a community research perspective, the above findings provide important information about measurement and assessment selection. Measures that are used should be sensitive to change and sensitive enough to evaluate outcomes 54 of the intervention in the context (Chambers, 2008). In this study, that may not have been the case. BASC scores did not reflect change in behavior suggesting that the BASC may not have been appropriate for this sample which displayed subclinical levels of social and emotional problems. However, there are other reasons that children and families can be considered at-risk (other than for behavioral difficulties) such as social factors like low socioeconomic status, impoverished environment, and educational difficulties. In future studies, it may be important to adequately measure other at-risk factors and assess their impact on the child and family. Currently, there is not a consensus on how mentors should be trained (Blechman, 1992). As this study implemented a different form of training, it was important to assess mentor?s perspective of the training. The TAI has preliminary data supporting its use in a training context (Lambha, 2010). Similar to Lambha (2010) the TAI showed good internal consistency in this study. In addition, descriptive data suggests that mentors in both groups were generally satisfied with the training they received. Although mentors provided positive feedback about the training, outcome data suggests that mentors need more exposure to the skills and continued feedback on their implementation of the skills for the training to be effective. Training may need to be ongoing and use of the skills should be evaluated via objective measures (such as observation) vs. subjective report by the mentors and the mentees. Blechman (1992) has suggested building flexible manuals for mentors that operationalize the goals of the mentoring program. By doing so, mentoring programs can be studied more scientifically. In this study, it may have been helpful to have a written document similar to a manual that reviewed the PRIDE skills and provided suggestions for their use with a mentee. In addition, the researchers could have suggested that procedures of the study and handouts about the training be included in any existing manuals provided by Project Uplift. 55 Other factors, such as the length of the relationship may have been influential in the null results. The study was originally designed to assess the participants at a 12-month follow-up, however, due to the difficulty recruiting participants and the delayed start of the study, the follow-up was conducted at 9 months. The current literature suggests that 12 months or longer is an optimal amount of time for a quality mentoring relationship to develop and to affect change in child outcomes (Chao, Walz, & Gardner, 1992; Rhodes, Reddy, Hoffman, & Grossman, 2005). The shortened period of follow-up (9 months) may not have been long enough to see optimal relationship benefits from mentoring. A 12-month follow-up period may have provided more of an opportunity to see change in the mentoring relationship. Data from Project Uplift states that 62% of mentors fulfilled their 12-month commitment. In this sample, 90% were still involved at 9 months and 10% of the volunteers had ended their mentoring relationships within six months. Some literature has suggested that a shortened mentoring relationship may be detrimental to the child (Grossman & Rhodes, 2002), however, in this study, children reported a positive relationship despite these shortened periods. Other studies have also found positive reports of the mentee-mentor for relationships that lasted less than 12 months. For example, Goldner and Mayseless (2009) reported positive relationship quality between mentee-mentor dyads at an 8- month assessment. Their study found that the dyad relationship improved over time and that the positive relationship was associated with positive academic, social, and emotional functioning. Although, there were no differences in this study from pre-intervention to post-intervention on outcome measures, reports indicated some minor positive changes in social and emotional functioning as measured by the BASC-2. One major difference in the methodology of the Goldner and Mayseless study and the current study is the structure of the mentoring program. The mentoring program in the Goldner and Mayseless study (2009) was highly structured and 56 the length of the relationship, amount of contact within the dyad, and the types of activities were closely monitored. The above outcomes were not evaluated in this study but the results from Goldner and Mayseless (2009) suggests that the support and structure provided by the mentoring organization may influence the effectiveness of the program and mentoring relationship. Although there were many difficulties in the implementation of this study, there were also many strengths. Typically, very few resources are dedicated to mentoring research, which has resulted in a gap between the practice of mentorship and the scientific research for understanding mentoring (DuBois & Rhodes, 2006). This study was fortunate to be funded by a University Outreach grant. This grant allowed the researchers to provide incentives for Project Uplift clients to participate ($10 at each time point), an incentive for mentors to practice the skills after the training (raffle drawing for completing monthly booster quizzes), and the ability to purchase needed supplies (e.g. outcome measures). Although funding is important, research on community based partnerships suggest that funding can also create power imbalances in the partnership between the researcher and community organization (Sullivan, Kone, Senturia, Chrisman, Ciske, & Krieger, 2001). In this study, the researcher controlled all the funds. This is typical in partnerships with major institutions and may lead to the idea that the researcher has sole decision making power (Sullivan et al., 2001). Sullivan and colleagues suggest collaborating about how funds will be used and sharing in the decision making process about how funds will be allocated to avoid isolating community partners (Sullivan, Kone, Senturia, Chirsman, Ciske, & Krieger, 2001). For this project, funds were definitely helpful in for increasing participation from parents. However, Project Uplift staff may have also had some suggestions of how funds could have been used to help facilitate the study and further their goals 57 as an organization. For example, perhaps some funding could have been used to pay a staff person to assist the project manager Other strengths of this study is that it included many design features put forth by The National Research Agenda for Youth Mentoring in 2006, to help bridge the gap in mentoring research. One of the best practices suggested by the committee was better tracking and evaluation of mentoring programs (DuBois & Rhodes, 2006). The current study tracked mentors from pre-intervention to post-intervention and via the monthly reports that were required by Project Uplift procedures. In regards to evaluating the mentoring program, previous procedures were subjective. The success or failure of Project Uplift program outcomes was measured by parent report of the child?s contact with the Juvenile Justice System and on a 10-item satisfaction questionnaire completed by the parent. These measures did not directly evaluate the goals and purposes of the program (which are to provide positive role models and uplift children?s lives). This study used objective measures completed by a third party (i.e., parents) to assess the effectiveness of the program. In theory, a mentor that uses the PRIDE skills has the basic skills for helping children build social skills and positive self-esteem and will be competent in building an attachment with the youth, thereby fulfilling the role of a positive role model. In addition, this current study attempted to improve upon previous study in that a psychometrically sound instrument was used to provide a measure of a child?s adaptive functioning based on normative data (i.e., the BASC-2). Best practices guidelines also suggested improving recruitment, training, and retention of mentors (DuBois & Rhodes, 2006). This study attempted to provide mentors with an additional training which specifically focused on the relationship. In the standard training procedures, mentors were given limited information about how to build a warm and caring relationship with 58 children. Research suggests that in order to establish an optimal relationship that is warm, the mentoring relationship must be reciprocal and based on trust (Spencer & Rhodes, 2005). Although it is clear that mentoring works because of the relationship between the child and adult, very little research has focused on the mechanism by which the relationship works. This study looked at building a positive relationship by providing mentors with the additional skills to help them form a secure attachment with their chosen mentees. Another gap in the literature that this study tried to fill is the lack of a multi-informant approach. Loeber, Green, and Lahey (1990) stressed the importance of using multiple informants in obtaining different perspectives on often complex and multifaceted issues. However, not all informants may be appropriate for assessing every issue. In fact, Loeber and colleagues (1990) suggested that results might vary drastically based on the informant providing the report. In their 1990 study, Loeber and colleagues evaluated the usefulness of parent, child, and teacher report of childhood behavior problems according to clinicians and child researchers. The results suggested that teachers are the best informants for assessing hyperactivity and inattention, parents are most helpful for assessing oppositional defiant behaviors, and children are the best reporters of their internalizing behaviors (Loeber, Green, & Lahey, 1990). This study did not utilize child report for behavioral outcomes but perhaps self-report should be included to obtain the most comprehensive view of child behavior. The current study also sought to obtain teacher report. It was important to involve teachers in this study because the majority of referrals to Project Uplift were made by the school system (63 % in the 2008 fiscal year) in past years. For 2007, Alabama reported a high school dropout rate of 2.5%. Although, this figure may seem low it represents over 5,000 students not completing their education (Alabama Department of Education, 2007). In addition, the projected 59 dropout rate for four years is even higher at 9.76% (Alabama Department of Education, 2008). These statistics emphasize the need to include teachers as they may be the first individuals to recognize a need for a particular child to have additional help. In addition, there is support for the benefits of mentoring for improvement in attitudes and behavior at school (Rose & Jones, 2007) and academic achievement such as reading and math (Thompson & Kelly-Vance, 2003). In this study, the collection of teacher data was inconsistent therefore the impact of mentoring on behaviors at school could not be assessed. This may be due to the lower number of referrals from the school during the 2009 fiscal year (23%). Despite this fact, it will be important to continue to look at the school setting and integrate mentoring into the classroom, home, and social setting (Blechman, 1992). This study utilized parent, teacher, child, and mentor report in an effort to obtain a comprehensive view of the mentoring relationship and its outcomes. Although, the use of multiple informants can be viewed as a strength, in this study it was also a weakness. For instance, child reports of the mentoring relationship at pre-intervention were overwhelmingly positive, which may have resulted in the inability to see change over time. In addition, although teacher report was requested, very few were actually obtained. Although it was important to assess multiple informants and at multiple time points, Chorpita and colleagues (1998) have suggested that this increased rigor adds difficultly to the data collection process. For example, multiple assessment points may lead to increased noncompliance and attrition which subsequently can result in data loss (Chorpita, Barlow, Albano, & Daleiden, 1998). In this study, parents and teachers were reluctant to participate at multiple time points. This again points to the need to include the input of multiple stakeholders at the outset of project design phase. Clients of Project Uplift (parents and mentees) and teachers were integral in providing information as to 60 whether the intervention worked (the attachment training) however, they were not directly included in the intervention. Sullivan and colleagues suggest that not including all stakeholders can lead to mistrust and resistance with community partners (Sullivan, Kone, Senturia, Chirsman, Ciske, & Krieger, 2001). Implications The goal of this study was to evaluate the effectiveness of an intervention that utilized principles from an Empirically Supported Treatment (PCIT) in a community setting. There were many challenges with the implementation of this study, but also many lessons learned about conducting Community Based Partnership Research and the dissemination of interventions. Many issues arise in implementing interventions in a community setting because of the gap between scientific research and practice. The gap between research and practice is not a new phenomenon. In 1983, Chavis, Stucky, and Wandersman stated, ?Basic research has generally remained within the scientific community, responsible primarily to standards and pressures of academic institutions and funding agencies? (p. 424). Historically, the role of the scientist has been to test hypotheses, interpret observations, and make new theories based on data whereas the role of the practitioner has been to develop strategies to address problems (Chavis, Stucky, & Wandersman, 1983). Chambers (2008) pointed out that this gap has continued. Recently organizations like the National Institute of Health, Center for Disease Control and Prevention, and the Institute of Medicine have taken notice of the implications for this gap in the public health sector and have put forth guidelines to facilitate research in the community (Chambers, 2008; Wallerstein & Duran, 2010). Principles of Community Based Participatory Research (CBPR) help to bridge the gap between science and practice. CBPR has many definitions; however, ?a defining characteristic of CBPR is the degree to which community partners are 61 engaged beyond the research encounter...? (Delemos, 2006, p. 330). Below, some of the lessons learned from conducting research in the community for this study will be presented in the light of suggested practices for conducting CBPR. One of the most important issues in CBPR is the need to be planful in forming the community partnership. Bringle and Hatcher (2002) likened the initiation of the partnership to relationship development in adult interpersonal relationships. Relationships can be unplanned and happen because of convenience or need (as in a crisis situation) or they can be planned in advance. In this study, the relationship with the community organization was not new; however, it was also not well established. The relationship between the researchers and the community organization was advanced because of the project. In a planned community partnership the researcher and community organization actively decide what type of relationship they would like to pursue and openly convey their interests and intentions to the other party before the project begins (Bringle & Hatcher, 2002). This allows for better collaboration and in addition, it allows input from multiple stakeholders. In this study, some of the interests of both parties developed as the project progressed instead of being clearly stated at the outset. For example, in this study, the organization was interested in having all mentors trained in the CDI skills versus a subset. In addition, the organization expressed interest in having the researchers provide information to mentors about dealing with specific problems such as lying. Another common problem in CBPR research is the assumption by the researcher that they know the needs of the organization and the needs of the members that the organization serves (Sullivan, Kone, Senturia, Chirsman, Ciske, & Krieger, 2001). Unfortunately, this was a problem in the design of the current study. The researcher noticed an area that could use additional support (training) and the study was built around that premise. However, key stakeholders were not consulted. Because of lack of 62 community input, the researcher did not have information about what clients of Project Uplift and mentors wanted in training or in their mentoring program in general. Beyond the previous year?s fiscal report, parental perspectives of Project Uplift were not obtained. The researcher did not assess parental satisfaction with the mentoring relationship or parental perception of the mentoring process. So although the researcher perceived a need in this area, perhaps parents and mentees would have identified a different need. An initial question should have been, how can I (as a scientist) assist Project Uplift in meeting their goals and needs? For example, in the year before the study began, Project Uplift reported that a large number of children were referred by the school system; therefore, it may have been more beneficial for intervention efforts to have been focused in school settings. Two other important issues to discuss at the outset of the CBPR relationship are the procedures of the project and the resources that will be used. This allows for problem solving and an evaluation of the logistics needed to implement the desired project. In addition, both partners can assess the compatibility of the proposed intervention in the organization .Chambers points out that ?the knowledge developed by science may be incompatible for use in practice and vice versa? (Chambers, 2008, p.4). With this in mind, other questions of importance include: What are the infrastructural needs? Is the intervention compatible with treatment as usual or does it require a paradigm shift in the organization? And Is there support for the ideas at the organizational level? (Chambers, 2008). For this project, the infrastructure required additional manpower and time to track mentors and ensure mentors completed the training; forward monthly reports completed by mentors and parents to the research team, and track Project Uplift clients. As such, it would have been helpful to have a liaison who was dedicated for overseeing these types of logistical issues. 63 The compatibility of an intervention with the organizations structure also brings forth questions of treatment dissemination. Similar to the issue of bringing science into practice, dissemination of empirically based treatments (EBTs) into community/practitioner settings has been slow and often times unsuccessful (McHugh & Barlow, 2010). Also similar to CBPR, dissemination of EBTs should be planful and considered carefully before starting a project with an EBT in the community. It is important to evaluate if the treatment will be implemented in the same manner as it is in research settings and clinical trials or if modifications are needed. Factors that should be considered include: Who can conduct the intervention? What are the circumstances (or context) in which the intervention will be conducted? Will the EBT need to be adapted in the community setting? Will there be ongoing support for the intervention? And who will monitor implementation of the intervention (Schoenwald & Hoagwood, 2001)? For this study, graduate and undergraduate students conducted the intervention which is a deviation from the implementation of PCIT in clinic (and research) settings. The context for this implementation of PCIT was also very different. Only parts of the EBT were used and they were presented in a different format from what has been established through research. Additional, there were many other modifications from the intervention design of PCIT including lack of ongoing practice, lack of in vivo feedback, and a shortened teaching session for CDI. Another issue that arises in CBPR and dissemination is the sustainability of the intervention. Often times in community research (and particularly in this study) when the research is complete, intervention programs are no longer available to the community (Sullivan, Kone, Senturia, Chirsman, Ciske, & Krieger, 2001). Projects utilizing community partnerships should plan for interventions to continue beyond the research initiatives (Sullivan, Kone, Senturia, Chirsman, Ciske, & Krieger, 2001). Chambers (2008) suggests that sustainability is easier when the changes have been incorporated 64 into the organizations structure and when there are ongoing opportunities for supervision and training. Preliminary steps have been taken to offer continued trainings for mentors although the study has ended. An alternative option may be to train a staff member to criterion on the PRIDE skills such that they could continue providing the information to new mentors. After the relationship has been established it is equally important to maintain a good working relationship. As in any relationship communication of great importance, each party should be aware of the potential rewards and costs. Sullivan and colleagues (2001) also highlighted the importance of clearly communicating the benefits of the partnership to the community agency (Sullivan, Kone, Senturia, Chirsman, Ciske, & Krieger, 2001). They suggest incentives and reimbursements for time and expenses (if possible) as well as evaluating more long term benefits such as jobs for community members and continued funding of community projects (Sullivan et al, 2001). Other suggestions for maintaining the relationship include frequent contact and diversity of contact. For this study, the researchers could have been more diligent about maintaining open communication lines. Communication was problem oriented or to request assistance or information. Incorporating weekly face-to-face meetings to provide updates on progress should have been utilized as well as communication by email or telephone. Overall, there were some barriers to working with a community partner however; with more careful planning these barriers could have been overcome. Mindful planning is the key. Community organizations should be included in all phases of the study design including setting goals and procedures, implementation, follow-up and monitoring, and sustainability. Both parties should be clear on the expectations and roles of each other as well as the limitations. These types of issues are best addressed through clear and ongoing communication. With these suggestions in place, CBPR can be very successful and rewarding and the gap between science and research can 65 began to be reduced. To aid graduate students in better understanding the principles of CBPR, the Institute of Medicine has included CBPR as one of eight competencies to include in curricula (Delemos, 2006). This will be important as the standards for research and the dissemination of EBTs continue to grow in the field of mental health Limitations There were also several limitations in this study. First, because of recruitment problems and a lack of resources within the mentoring organization, the study design was changed during implementation in an effort to increase participation. The change in methodology may have introduced extraneous variables into the data (i.e., extended time between mentor training and actual contact with mentee), which may have influenced the results. The recruitment difficulties also resulted in a small sample size, which may have affected the ability to detect changes. This study also had a large of amount of missing data particularly in the follow-up phase. It was particularly difficult to obtain mentor reports and teacher reports at follow-up. There was also a significant problem with the psychometric properties of the YMRQ at Time 1. It did not demonstrate adequate reliability, which suggests that it was not a good measure of relationship quality. Although, the YMRQ was not psychometrically sound, it was still used in subsequent analyses. This is a major limitation in that the inclusion of the YMRQ in the analyses may have significantly influenced the results. In the future, it will be important to assess the reliability of a measure before incorporating it into a research study. However, due to the nature of the rolling enrollment of mentees, not all data for Time 1 was acquired at the same time, thus contributing to the difficulties assessing the appropriateness of its inclusion. This study took place in a small southern community in which the population is largely Caucasian (70.9%), followed by African American (26.3%) (US Census Bureau, 66 http://quickfacts.census.gov/qfd/states/01000.html). In addition, this particular mentoring program was housed on a college campus and thereby most of the mentors (95%) were undergraduate college students. As such, the mean age of the mentors were younger (approximately age 19) than might be expected in a mentoring organization that is more community-based. In addition, as the mentors were still in college, they likely did not have much experience parenting their own children and may not have had much experience working with children. This sample was also unique in that most mentors volunteered to work with Project Uplift in exchange for course credit. The mentor?s characteristics in this sample may not reflect the characteristics of mentors in more community- based mentoring programs. Because the participant characteristics were unique, they may not be reflective of other individuals participating in mentoring programs and, thus, the results may not generalize. One of the most important steps in planning a successful mentoring intervention will be tailoring the program to the community in which it takes place. Bogenschneider (1996?) phrased it best when he stated, ??the problems youth face, the etiology of these problems, and the most appropriate responses may differ from one community or neighbor-hood to the next. Thus, prevention efforts may be more successful if practitioners take time to identify real community or neighborhood issues? (p.130). Future Directions In future studies, it will be important to incorporate more of the best practices guidelines for mentoring and for conducting CBPR into the study design. This includes building the partnership with the community agency before beginning a research project. Important stakeholders and the needs of the organization and the larger community it serves should be taken into account in the design phase of the study. Specific to mentoring, it will be helpful to 67 include better mentor recruitment techniques, which may result in larger sample sizes and more robust data. It will also be important to be knowledgeable of participant?s characteristic prior to inclusion in the study. One problem with the current study was the lack of problem behaviors as reported by parents at pre-intervention, which resulted in very little improvement at post- intervention. If an objective of the study is to decrease maladaptive behavior, participants who are not manifesting difficult behaviors at pre-test should not be included. A pre-screening, which includes an assessment of socio-emotional and behavior functioning, would be useful in assuring the clients are appropriate for the study. Alternatively, this sample was at-risk for reasons other than behavior problems. Input at from parents and children at the outset will be useful to identify additional or different goals that the project might address. In regards to training, building a strong positive relationship with the mentee should be a clear goal in any mentoring program. PCIT principles are a good means to develop strong relationships, especially if the core features that are founded in traditional PCIT treatment outcome research (such as mastery and live coaching) are incorporated. As such, future mentor trainings might include teaching mentors the PRIDE skills and assessing them for mastery of the skills. A small pilot study may be the best starting point to assess if mentors can meet mastery in an abbreviated format. Although the session is abbreviated, the length of the training may need to be extended (2-day workshop vs. 2 hour training session) and more detailed written materials that can be referenced at home should be included to increase skill acquisition. It would also be helpful for mentors to be observed with their mentees to assure they are accurately using the skills. For example, mentee and mentor dyads may be asked to spend one outing session a month in an analogue playroom where they can be observed by the researcher. Immediate feedback on their use of the skills would help mentors to meet mastery of the PRIDE skills. With these 68 suggestions in place, a future study may be more successful in demonstrating the benefits of PCIT in building a close attachment and decreasing maladaptive behaviors in mentee and mentor dyads. Wagner (2010) suggests that Abbreviated PCIT sessions may be more of a preventive measure than intervention. As such, a future direction would be to evaluate if youth that have a mentor that consistently uses PCIT based skills show stable behavior patterns or are less likely to develop behavior problems after the mentoring relationship has ended as compared to youth that are not exposed to PCIT based skills. This would allow the researcher to focus on long-term maintenance of behavior versus outcomes measured immediately at the end of the relationship. As PCIT continues its path of dissemination and integration into to other contexts, it will be important to consider expanding into community settings that focuses on the well-being of children (like mentoring programs). 69 Table 1 Participant demographics and chi-square analyses Variable Total n = 37 Control Group n = 13 Experimental Group n = 24 Chi-Square p-value Child Age (years) M 9.0 9.0 9.0 ?2 = 5.068 p = .346 SD 1.67 1.64 1.78 Gender Male 35.3% 33.3% 38.5% ?2 = .092 p = .761 female 64.7% 66.7% 61.5% Grade M 3.30 3.25 3.38 ?2 = 1.932 p = .926 SD 1.58 1.54 1.71 Ethnicity African American 91.9% 100.0% 76.9% ?2 = 6.027 p = .014 Caucasian 8.1% 0.0% 23.1% Months on waiting list M 7.75 6.67 9.56 ?2 = 11.20 p = .130 SD 7.08 7.17 6.48 Parent Age (years) M 36.27 38.10 32.60 ?2 = 27.00 p = .079 SD 8.91 9.48 6.59 Marital status Married 30.6% 33.3% 25.0% ?2 = .521 p = .771 Single 58.3% 54.2% 66.7% Divorced 11.1% 12.5% 8.3% Income less than 10k 41.2% 30.4% 63.6% ?2 = 14.20 p = .014 10k-20k 35.2% 47.8% 9.1% 20k-30K 17.6% 21.7% 0.0% 40k-50k 2.9% 0.0% 9.1% 50K and up 5.9% 0.0% 18.2% Education > High School 31.4% 39.1% 16.6% ?2 = 9.381 p = .153 High School Diploma 34.3% 39.1% 25.0% College 34.3% 21.7% 58.3% 70 /Advanced degree Mentor Age (years) M 19.70 19.77 19.55 ?2 = 7.713 p = .173 SD 1.36 0.97 1.97 Gender (percentage) Male 18.2% 18.2% 18.2% ?2 = .092 p = .761 female 81.8% 81.8% 81.8% Ethnicity African American 6.1% 4.5% 9.1% ?2 = .750 p = .087 Caucasian 90.9% 90.9% 90.9% Multicultural 3.0% 4.5% 0.00% Course Credit yes 21.2% 18.2% 22.7% ?2 = .093 p = .763 no 78.8% 81.8% 77.3% 71 Table 2 Inter-Item Correlation Matrix for the Youth Mentor Relationship Questionnaire Item # 1 2 3 4 5 6 7 9 10 11 13 1 1.00 -0.016 0.112 0.175 0.140 0.063 -0.011 0.006 -0.201 -0.014 -0.085 2 1.00 -0.108 -0.025 -0.196 0.000 0.06 -0.152 0.044 0.022 0.277 3 1.00 0.305 0.086 -0.054 -0.082 -0.003 -0.099 0.100 -0.042 4 1.00 0.314 0.026 -0.034 0.093 -0.113 -0.014 0.225 5 1.00 -0.069 0.085 0.499 0.096 -0.169 0.149 6 1.00 0.007 0.223 -0.077 -0.054 0.294 7 1.00 -0.018 -0.117 -0.082 0.171 9 1.00 -0.028 -0.229 0.084 10 1.00 -0.099 -0.061 11 1.00 -0.042 13 1.00 72 Table 3 Inter-Item Covariance Matrix for the YMRQ Item # 1 2 3 4 5 6 7 9 10 11 13 1 0.820 -0.013 0.069 0.094 0.151 0.028 -0.007 0.008 -0.183 -0.008 -0.037 2 0.769 -0.064 -0.013 -0.205 0.000 0.038 -0.179 0.038 0.013 0.115 3 0.456 0.122 0.069 -0.018 -0.040 -0.003 -0.067 0.046 -0.013 4 0.353 0.223 0.008 -0.015 0.074 -0.067 -0.006 0.063 5 1.426 -0.041 0.074 0.800 0.115 -0.136 0.085 6 0.246 0.003 0.149 -0.038 -0.018 0.069 7 0.538 -0.018 -0.087 -0.040 0.060 9 1.805 -0.038 -0.208 0.054 10 1.010 -0.067 -0.029 11 0.456 -0.013 13 0.225 Note. Negative covariances are in boldface 73 Table 4 Correlations for Mentee Relationship Questionnaires MYAS Time 1 MYAS Time 2 YMRQ Time 1 YMRQ Time 2 MEI Time 1 MEI Time 2 MYAS Time 1 1.00 -0.035 0.162 0.130 .383* 0.164 MYAS Time 2 1.00 .338* .426* -0.064 0.24 YMRQ Time 1 1.00 .556** .335* .591** YMRQ Time 2 1.00 0.141 .623** MEI Time 1 1.00 0.279 MEI Time 2 1.00 *Correlation is significant at the 0.05 level (2-tailed). **Correlation is significant at the 0.01 level (2-tailed). Note. Mentor Youth Alliance Scale=MYAS, Youth Mentor Relationship Questionnaire = YMRQ, Pre-intervention = Time 1, and Post-intervention = Time 2 74 Table 5 Factor Loadings for Exploratory Factor Analysis With Varimax Rotation for the YMRQ Factors Items 1 2 3 4 5 When my mentor gives me advice, s/he makes me feel stupid 0.81 0.32 -0.05 -0.02 -0.12 I wish my mentor spent more time with me 0.76 0.06 -0.08 0.01 0.34 When I am with my mentor, I feel ignored -0.10 0.72 -0.13 0.11 -0.05 I wish my mentor spent more time with me -0.57 0.24 -0.06 0.06 0.05 My mentor has lots of good ideas about how to solve a problem 0.21 0.76 0.23 0.14 0.02 When something is bugging me, my mentor listens while I get it off my chest -0.30 -0.08 0.66 -0.15 -0.04 I feel that I can not trust my mentor with secrets because s/he would tell my mom or dad 0.17 0.15 0.81 0.00 0.24 When I am with my mentor I feel disappointed (feel sad) 0.24 -0.33 0.42 0.42 -0.48 When I am with my mentor I feel bored 0.13 -0.08 -0.07 0.80 -0.07 Sometimes my mentor promises/promised that we will do something and then we did not do it 0.04 0.17 -0.24 0.63 0.06 I wish my mentor asked me more what I think 0.10 -0.14 0.22 0.19 0.84 Eigenvalues 1.97 1.54 1.45 1.12 1.07 Percent Variance 17.93 12.01 13.14 10.18 9.69 Note. Factor loadings >.32 are in boldface. 75 Table 6 Factor Loadings for Exploratory Factor Analysis With Varimax Rotation for the YMRQ-MV Factors Items 1 2 3 4 When something is bugging my mentee, s/he feels bbbbcomfortable talking to me to get it off their chest 0.777 -0.036 0.05 -0.03 I ask my mentee what s/he thinks or his/her opinion bbbbabout things 0.968 -0.008 0.028 0.111 I help my mentee take his or her mind off things by bbbbdoing something with him/her 0.970 -0.036 0.030 0.069 My mentee thinks that I have lots of good ideas about bbbbhow to solve a problem 0.959 -0.031 0.029 0.093 I sometimes promise my mentee that we will do bbbbsomething and then we did not do it 0.106 0.969 -0.006 -0.079 When my mentee is with me, s/he feels ignored -0.043 0.942 -0.012 -0.045 I make my mentee feel small when I give him/her bbbbadvice 0.144 0.796 -0.039 0.063 My mentee feels that s/he can not trust me with bbbbsecrets because I would tell his/her mom or dad -0.261 0.858 -0.020 -0.003 I think my mentee fells disappointed or sad when they bbbbare with me 0.039 -0.024 0.997 0.056 My mentee wishes I were different 0.039 -0.024 0.997 0.056 When my mentee is with me s/he feels mad 0.039 -0.024 0.997 0.056 I think my mentee would like it if I spent more time bbbbwith him/her 0.312 0.122 0.064 0.833 When my mentee is with me s/he feels bored -0.444 -0.058 -0.005 -0.127 I wish I knew my mentee better -0.019 0.156 -0.084 -0.920 Eigenvalues 4.079 3.258 2.892 1.423 Percent Variance 29.136 23.275 20.660 10.165 Note. Factor loadings >.32 are in boldface. 76 Table 7 Pre-intervention and Post-intervention Data for the Experimental Group Measures Time1 Time 2 M SD M SD BASC Composites and Scales Externalizing Problems 50.62 10.12 50.09 12.17 Internalizing Problems 50.62 10.20 51.47 10.13 Behavior Symptoms Index 52.25 7.56 52.33 9.62 Adaptive Skills 42.25 8.61 43.19 9.18 Social Skills 42.17 10.72 45.23 11.02 Resiliency 45.91 9.96 45.64 10.00 Mentee Relationship and PRIDE Skills Measures Mentor Evaluation Inventory 79.24 7.47 79.70 4.96 MYAS 38.82 1.74 38.07 3.13 YMRQ 38.83 3.16 50.00 5.14 Mentor Relationship Measures TAI 4.29 0.44 - - MYAS-MV 19.95 9.23 33.17 6.61 YMRQ-MV 57.09 9.60 71.17 6.31 Note. BASC-2 = Behavior Assessment System for Children 77 Table 8 Pre-intervention and Post-intervention Data for the Control Group Measures Time1 Time 2 M SD M SD BASC Composites and Scales Externalizing Problems 55.69 16.97 54.40 19.57 Internalizing Problems 47.77 7.28 49.20 8.74 Behavior Symptoms Index 56.15 13.85 55.10 16.09 Adaptive Skills 38.61 7.87 41.30 13.79 Social Skills 36.77 7.60 43.60 13.78 Resiliency 41.75 9.98 42.90 11.38 Mentee Relationship and PRIDE Skills Measures Mentor Evaluation Inventory 79.90 3.88 81.83 2.55 MYAS 38.02 2.67 37.69 6.00 YMRQ 40.15 3.05 51.12 3.56 Mentor Relationship Measures TAI 4.10 0.73 - - MYAS-MV 17.50 9.30 34.33 1.86 YMRQ-MV 52.18 3.03 73.67 1.21 Note. BASC-2 = Behavior Assessment System for Children 78 Table 9 2 x2 Split Plot ANOVAs for Parent Reported BASC-2 Scores Degrees of Freedom F value p value BASC-2 Composites and Scales Externalizing Within subjects (Time) 1, 30 0.015 0.903 Between subjects (group) 1, 30 0.57 0.456 Interaction 1, 30 0.099 0.755 Internalizing Within subjects (Time) 1, 29 0.077 0.783 Between subjects (group) 1, 29 0.031 0.861 Interaction 1, 29 0.387 0.539 Behavior Symptoms Index Within subjects (Time) 1, 29 0.106 0.747 Between subjects (group) 1, 29 0.131 0.72 Interaction 1, 29 0.597 0.446 Adaptive Skills Within subjects (Time) 1, 29 1.418 0.243 Between subjects (group) 1, 29 0.071 0.792 Interaction 1, 29 0.385 0.54 Social Skills Within subjects (Time) 1, 30 9.525 0.004* Between subjects (group) 1, 30 0.412 0.526 Interaction 1, 30 0.396 0.534 Resiliency Within subjects (Time) 1, 29 0.165 0.688 Between subjects (group) 1, 29 0.781 0.384 Interaction 1, 29 0.779 0.385 * significant at the 0.01 level. 79 Figure 1 Flow Chart of Original Study Design Mentors continue monthly check-in with Project Uplift and Complete attachment training assignments 12 month follow-up 3 month follow-up 6 month follow-up Initial contact with clients/completion of measures Interested Volunteers Informational meeting/ Standard training Subsets of volunteers are accepted and enroll as mentors General communication training n = 22 Informed consent and Pre- intervention measures for experimental group Informed consent and Pre- intervention measures for control group Attachment training n = 27 Mentors continue monthly check-in with Project Uplift 80 Figure 2 Flow Chart of Modified Study Design Mentors in Experimental group continue monthly check-in with Project Uplift and Complete attachment training assignments 9 month follow-up Initial contact with clients/completion of measures Interested Volunteers Informational meeting/ Standard training Subsets of volunteers are accepted and enroll as mentors Communication Training n = 12 Informed consent and Pre- intervention measures for experimental group Informed consent and Pre- intervention measures for control group Attachment training n = 59 Mentors in control group continue monthly check-in with Project Uplift 81 Figure 3 Pre-intervention and Post-intervention Relationship Scores for Mentor Report 82 Figure 4 Percentage of Mentees Showing Improvement or No Change on BASC-2 Scales 83 References Allen, T. D., & Eby, L. T. (2007). Overview and introduction. In T. D. Allen & L. T. Eby (Eds.), The Blackwell Handbook of Mentoring: a multiple perspectives approach (pp. 3-22). Malden, MA: Blackwell Publishing. Ammaniti, M., Ijzendoorn, M. H., Speranza, A. M., & Tambelli, R. (2000). Internal working models of attachment during late childhood and early adolescence: An exploration of stability and change. Attachment and Human Development, 2(3), 328-346. doi.10.1080/14616730010001587. Baker, D. B., & Maquire, C. P. (2005). Mentoring in historical perspective. In D. L. Dubois & M. J. Karcher (Eds.), Handbook of Youth Mentoring (pp.14-29). Thousand Oaks, CA: Sage Publications. Bell, S. K., & Eyberg, S.M. (2002). Parent-child interaction therapy: a dyadic intervention for treatment of young children with conduct problems. In L. VandeCreek, S. Knapp, & T.L. Jackson (Eds.), Innovations in Clinical Practice: A Source Book (Vol. 20; pp. 57-74). Sarasota, FL: Professional Resource Press. Berkovits, M. D., O?Brien, K. A., Carter, C.G., & Eyberg, S. M. (2010). Early identification and intervention for behavior problems in primary care: a comparison of two abbreviated versions of Parent Child Interaction Therapy. Behavior Therapy, 41(3), 375-387. doi:10.1016/j.beth.2009.11.002 Blechman, E. A. (1992). Mentors for high-risk minority youth: from effective communication to bicultural competence. Journal of Clinical Child & Adolescent Psychology, 21(2), 160 ? 169. doi: 10.1207/s15374424jccp2102_8. 84 Bogenschneider, K. (1996). An ecological risk/protective theory for building prevention programs, policies, and community capacity to support youth. Family Relations, 45(2), 127-138. Bozeman, B., & Feeney, M. K. (2007). Toward a useful theory of mentoring: a conceptual analysis and critique. Administration and Society, 39(6), 719-739. doi: 10.1177/0095399707304119 Buckley, M. A., & Zimmerman, S. H. (2003). Mentoring Children and Adolescents: A Guide to the Issues. Westport, CT: Praeger Publishers. Cavell, T. A., & Smith, A. (2005). Mentoring children. In D. L. Dubois & M. J. Karcher (eds.), Handbook of youth mentoring (pp. 2-12). Thousand Oaks, CA: Sage Publications. Chaffin, M., Silovsky, J. F., Funderburk, B., Valle, L. A., Brestan, E. V., Balachova, T., Jackson, S., Lensgraf, Jay., & Bonner, B. L. (2004). Parent-child interaction therapy with physically abusive parents: efficacy for reducing future abuse reports. Journal of Consulting and Clinical Psychology, 72(3), 500-510. doi: 10.1037/0022-006X.72.3.500 Chambers, D. A. (2008). Advancing the science of implementation: a workshop summary. Administration and Policy in Mental Health, 35, 3-10. doi: 10.1007/s10488-007-0146-7 Chan, C. C., & Ho, W. C. (2008). An ecological framework for evaluating relationship- functional aspects of youth mentoring. Journal of Applied Social Psychology, 38(4), 837-867. doi: 10.1111/j.1559-1816.2008.00329.x Chao, G. T., Walz, P. M., & Gardner, P. D. (1992). Formal and informal mentorships: a comparison on mentoring functions and contrast with nonmentored counterparts. Personnel Psychology, 45(3), 619-636. doi: 10.1111/j.1744-6570.1992.tb00863.x 85 Chavis, D. M., Stucky, P. E., & Wandersman, A. (1983). Returning basic research to the community: a relationship between scientist and citizen. American Psychologist, 38(4), 424-434. doi: 10.1037/0003-066X.38.4.424 Chorpita, B. E., Barlow, D. H., Albano, A. M., & Daleiden, E. L. (1998). Methodological strategies in child clinical trials: advancing the efficacy and effectiveness of psychosocial treatments. Journal of Abnormal Child Psychology, 26(1), 7-16. doi: 10.1023/ A:1022626505280 Delemos, J. L. (2006). Community-based participatory research: changing scientific practice from research on communities to research with and for communities. Local Environment, 11(3), 329-338. doi: 10.1080/13549830600558838 DeWit, D. J., Lipman, E., Manzano-Munguia, M., Bisanz, J., Graham, K., Offord, D. R., O?Neil, E., Pepler, D., & Shaver, K. (2006). Feasibility of a randomized controlled trial for evaluating the effectiveness of the Big Brothers Big Sisters community match program at the national level. Children and Youth Services Review, 29, 383-404. doi:10.1016/j.childyouth.2006.09.003 Doleys, D. M., Doster, J., & Cartelli, L. M. (1976). Parent training techniques: effects of lecture- role-playing followed by feedback and self-recording. Journal of Behavior Therapy and Experimental Psychiatry, 7, 359-362. doi: 10.1016/0005-7916(76)90107-5 DuBois, D. L., Doolittle, F., Yates, B. T., Silverthorn, N., & Tebes, J. K. (2006). Research methodology and youth mentoring. Journal of Community Psychology, 34(6), 657-676. doi: 10.1002/jcop.20122 86 DuBois, D. L., Holloway, B. E., Valentine, J. C., & Cooper, H. (2002). Effectiveness of mentoring programs for youth: a meta-analytic review. Journal of Community Psychology, 30(2), 157-197. doi: 10.1023/A:1014628810714 DuBois, D. L., & Karcher, M. J. (2005). Youth mentoring: theory, research, and practice. In D. L. Dubois & M. J. Karcher (eds.), Handbook of youth mentoring (pp. 2-12). Thousand Oaks, CA: Sage Publications. DuBois, D. L., & Neville, H. A. (1997). Youth mentoring: investigation of relationship characteristics and perceived benefits. Journal of Community Psychology, 25(3), 227- 234. doi: 10.1002/(SICI)1520-6629(199705)25:3<227::AID-JCOP1>3.0.CO;2-T DuBois, D. L., & Silverthorn, N. (2005). Natural mentoring relationships and adolescent health: evidence from a national study. American Journal of Public Health, 95(3), 518-524. doi: 10.2105/AJPH.2003.031476 Eby, L. T., Allen, T. D., Evans, S. C., Ng, T., & DuBois, D. L. (2008). Does mentoring matter? A multidisciplinary meta-analysis comparing mentored and non-mentored individuals. Journal of Vocational Behavior, 72, 254-267. doi:10.1016/j.jvb.2007.04.005 Eyberg, S. M. (1993). Consumer satisfaction measures for assessing parent training programs. In L. VandeCreek, S. Knapp, & T.L. Jackson (Eds.), Innovations in clinical practice: A source book. Sarasota, FL: Professional Resource Press. Eyberg, S. M. (2004). The PCIT story: conceptual foundation of PCIT. PCIT Pages: The Parent- Child Interaction Therapy Newsletter, 1, 1-2. Eyberg, S. M., Boggs, S., & Algina, J. (1995). Parent-child interaction therapy: A psychosocial model for the treatment of young children with conduct problem behavior and their families. Psychopharmacology Bulletin, 31, 83-91. 87 Eyberg, S. M., Nelson, M. M., & Boggs, S. R. (2008). Evidence-based psychosocial treatments for children and adolescents with disruptive behavior. Journal of Clinical Child & Adolescent Psychology, 37(1), 215- 237. doi: 10.1080/15374410701820117 Frecknall, P., & Luks, A. (1992). An evaluation of parental assessment of the Big Brothers/Big Sisters program in New York City. Adolescence, 27(107), 715-718. Gliem, J. A., & Gliem, R. R. (October, 2003). Calculating, interpreting, and reporting cronbach?s alpha reliability coefficient for likert-type scales. Paper session presented at the meeting Midwest Research to Practice Conference in Adult, Continuing, and Community Education, Columbus, OH. Goldner, L. & Mayseless, O. (2009). The quality of mentoring relationships and mentoring success. Journal of Youth and Adolescence, 38, 1339-1350. doi: 10.1007/s10964-008- 9345-0 Grossman, J. B., & Rhodes, J. E. (2002). The test of time: predictors and effects of duration in youth mentoring relationships. Journal of Community Psychology, 30(2), 199-219. doi: 10.1023/A:1014680827552 Holmbeck, G. N. (2003). Toward terminology, conceptual, and statistical clarity in the study of mediators and moderators: examples from the child clinical and pediatric psychology literatures. In A. E. Kazdin (ed.), Methodological issues and strategies in clinical research (pp. 77-105). Washington, DC: American Psychological Association. Jekielek, S., Moore, K. A., & Hair, E. C. (2002). Mentoring programs and youth development: a synthesis. Washington, DC: Child trends. 88 Keating, L. M., Tomishima, M. A., Foster, S., & Alessandri, M. (2002). The effects of a mentoring program on at-risk youth. Adolescence, 37(148), 717-734. Retrieved from http://web.ebscohost.com.spot.lib.auburn.edu/ehost Keller, T. E. (2007). Youth mentoring: theoretical and methodological issues. In T. Allen & L. Eby (Eds.), The Blackwell handbook of mentoring: a multiple perspectives approach (pp. 23-46). Malden, MA: Blackwell Publishing. Lambha, M. (2010). Volunteer-child interaction training: exploring the effects of a modified child-directed interaction with volunteers (Doctoral dissertation). Retrieved from http://etd.auburn.edu/etd/ Lee, E. L., Wilsie, C. C., & Brestan-Knight, E. (2011). Using Parent-Child Interaction Therapy to develop a pre-parent education module. Children and Youth Services Review, 33, 1254-1261. doi:10.1016/j.childyouth.2011.02.024 Lentz, E. & Allen, T. D. (2007). Reflections on naturally occurring mentoring relationships. In T. Allen & L. Eby (Eds.), The Blackwell handbook of mentoring: a multiple perspectives approach (pp. 23-46). Malden, MA: Blackwell Publishing. Liang, B., & Rhodes, J. (2007). Cultivating the vital element of youth mentoring. Applied Developmental Science, 11(2), 104-107. Retrieved from http://web.ebscohost.com.spot.lib.auburn.edu/ehost Loeber, R., Green, S. M., & Lahey, B. B. (1990). Mental health professionals? perception of the utility of children, mothers, and teachers as informants on childhood psychopathology. Journal of Clinical Child Psychology, 19(2), 136-143. doi: 10.1207/ s15374424jccp1902_5 89 McHugh, R. K., & Barlow, D. H. (2010). The dissemination and implementation of evidence- based psychological treatments. American Psychologist, 65(2), 73-84. doi: 10.1037/a0018121 Nathan, P. E., Stuart, S. P., & Dolan, S. L. (2003). Research on psychotherapy efficacy and effectiveness: between Scylla and Charybdis? In A. E. Kazdin (ed.), Methodological issues and strategies in clinical research (pp. 505-546). Washington, DC: American Psychological Association. Querido, J.G., Bearss, K., & Eyberg, S.M. (2002). Theory, research, and practice of parent-child interaction therapy. In F.W, Kaslow & T. Patterson (Eds.), Comprehensive Handbook of Psychotherapy (Vol. 2, pp. 91-113). New York: Wiley. Reynolds, C. R., & Kamphaus, R. W. (2004). BASC-2 Behavior Assessment for Children Manual (2nd ed.). Circle Pines, MN: American Guidance Service. Rhodes, J. E. (2002). Stand by Me: the Risks and rewards of Mentoring Today?s Youth. Cambridge: Harvard University Press. Rhodes, J. E. (2005). A model of youth mentoring. In D. L. Dubois & M. J. Karcher (eds.), Handbook of youth mentoring (pp. 2-12). Thousand Oaks, CA: Sage Publications. Rhodes, J. E., Reddy, R., Roffman, J., & Grossman, J. B. (2005). Promoting successful youth mentoring relationships: a preliminary screening questionnaire. The Journal of Primary Prevention, 26(2), 147-167. doi: 10.1007/s10935-005-1849-8 Rhodes, J. E., Spencer, R., Keller, T. E., Liang, B., & Noam, G. (2006). A model for the influence of mentoring relationships on youth development. Journal of Community Psychology, 34(6), 691-707. doi: 10.1002/jcop.20124 90 Rose, R., & Jones, K. (2007). The efficacy of a volunteer mentoring scheme in supporting young people at risk. Emotional and Behavioral Difficulties, 12(1), 3-14. doi: 10.1080/ 13632750601135873 Schmidt, M. E., McVaugh, B., & Jacobi, J. (2007). Is mentoring throughout the fourth and fifth grades associated with improved psychosocial functioning in children? Mentoring and Tutoring, 15(3), 263-276. doi: 10.1080/13611260701201943 Schoenwald, S. K., & Hoagwood, K. (2001). Effectiveness, transportability, and dissemination of interventions: what matters when? Psychiatric Services, 52(9), 1190-1197. Solomon, M., Ono, M., Timmer, S., & and Goodlin-Jones, B. (2008). The effectiveness of parent?child interaction therapy for families of children on the autism spectrum. Journal of Autism and Developmental Disorders, 38(9), 1767-1776. doi: 10.1007/s10803-008- 0567-5 Storch, E. A., & Floyd, E. M. (2005). Introduction: innovative approaches to parent-child interaction therapy. Education and Treatment of children, 28(2), 106-110. Retrieved from http://web.ebscohost.com.spot.lib.auburn.edu/ehost Sullivan, M., Kone, A., Senturia, K. D., Chrisman, N. J., Ciske, S. J., & Krieger, J. W. (2001). Researcher and researched-community perspectives: toward bridging the gap. Health Education and Behavior, 28(2), 130-149. doi: 10.1177/109019810102800202 Tabachnick, B. G., & Fidell, L. S. (Eds.). (2007). Using multivariate statistics. Boston, MA: Pearson Education, Inc. Thomas, L. A., & Kelly-Vance, L. (2001). The impact of mentoring on academic achievement of at-risk youth. Children and Youth Services Review, 23(3), 227-242. doi:10.1016/S0190- 7409(01)00134-7 91 Zand, D. H., Thomson, N., Cervantes, R., Espiritu, R., Klagholz, D., LaBlanc, L., & Taylor, A. (2009). The mentor-youth alliance: the role of mentoring relationships in promoting youth competence. Journal of Adolescence, 32, 1-17. doi:10.1016/ j.adolescence.2007.12.006 Wagner, S. (2010). Research in PCIT. In C. B. McNeil & T. L. Hembree-Kigin (Eds.), Parent- Child Interaction Therapy, Issues in Clinical Child Psychology (pp.17-26). New York, NY: Springer. doi:10.1007/978-0-387-88639-8_2 Wallerstein, N., & Duran, B. (2010). Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity. American Journal of Public Health, 100(1), s40-s46. Zeldin, S., Larson, R., Camino, L., & O?Connor, C. (2005). Intergenerational relationships and partnerships in community programs: purpose, practice, and directions for research. Journal of Community Psychology, 33(1), 1-10. Zimmerman, M. A., Bingeheimer, J. B., & Notaro, P. C. (2002). Natural mentors and adolescent resiliency: a study with urban youth. Journal of Community Psychology, 30, 221-243. 92 Appendices 93 A. Communicating effectively with your mentee ? Show the child absolute acceptance. This is for who they are not their behavior. If the child is misbehaving it will be important to separate the behavior from the child. You don?t want to imply that your child is intrinsically bad. Showing acceptance for the child as a person will enhance his/her confidence and s/he will interact better with others. You can do this by a positive body language, appreciative looks and of course verbal motivation. ? Listen to your mentee. Show them that you are interested and want to know all about them ? When communicating use eye contact. Not just when you are asking them to do something but at all times, even when s/he?s just talking about something that happened during the day. It is important to look into his/her eyes and talk. ? Use a polite language with kids. Using words like please, sorry, thank you etc will not only make them polite but will also make them feel important. ? Always use motivating words. Do not ever discourage your child by telling him that he is not good at something. Encourage them to just try their best. ? Acknowledge your child?s feelings. Children want to feel that their opinions matter. In response to your child?s statement, you might simply say, ?I?m glad to know that,? or ?I understand.? 94 B. Exert from Project Uplift standard training Ideas to Handle Problem Situations ??When a problem develops between you and your child try to solve it by using the following six suggestions:? 1. Be unconditionally accepting of the child-not the problem. Focus on actions not the person. 2. When you speak to the child use ?I? statements and not ?You? statements: ?I feel,? ?I think,? etc. rather than, ?You did this,? or You are bad.? 3. Talk with each other about your feelings on the problem 4. Be encouraging and positive in your statements 5. Provide choices for the child and discuss what each decision means. 6. Be dependable and consistent-no matter how difficult it becomes. 95 C. Me and My Mentor 1 2 3 4 Not true at all A little true Sometimes true Very true 1. Sometimes my mentor promises/promised that we will do something and then we did not do it 1 2 3 4 2. When something is bugging me, my mentor listens while I get it off my chest 1 2 3 4 3. When I am with my mentor, I feel ignored 1 2 3 4 4. When my mentor gives me advice, s/he makes me feel stupid 1 2 3 4 5. I wish my mentor asked me more what I think 1 2 3 4 6. When I am with my mentor I feel disappointed (feel sad) 1 2 3 4 7. My mentor helps me take my mind off things by doing something with me 1 2 3 4 8. My mentor makes fun of me in ways I don't like 1 2 3 4 9. I wish my mentor spent more time with me 1 2 3 4 10. I feel that I can not trust my mentor with secrets because s/he would tell my mom or dad 1 2 3 4 11. My mentor has lots of good ideas about how to solve a problem 1 2 3 4 12. I wish my mentor was different 1 2 3 4 96 13. When I am with my mentor I feel bored 1 2 3 4 14. When I am with my mentor I feel mad 1 2 3 4 15. I wish my mentor knew me better 1 2 3 4 16. I would feel sad if something bad happened to my mentor 1 2 3 4 17. I feel comfortable with my mentor 1 2 3 4 18. I feel my mentor cares about me, even when I do things s/he does not like 1 2 3 4 19. I look forward to spending time with my mentor 1 2 3 4 20. My relationship with my mentor is important to me 1 2 3 4 21. My mentor is happy when good things happen to me 1 2 3 4 22. My mentor cares about me 1 2 3 4 23. I enjoy talking to my mentor 1 2 3 4 24. I try to follow my mentor's advice 1 2 3 4 25. I trust my mentor 1 2 3 4 97 D. Mentoring Relationship Questionnaire 1 2 3 4 Not true at all A little true Sometimes true Very true 1. I sometimes promise my mentee that we will do something and then we did not do it 1 2 3 4 2. When something is bugging my mentee, s/he feels comfortable talking to me to get it off their chest 1 2 3 4 3. When my mentee is with me, s/he feels ignored 1 2 3 4 4. I make my mentee feel small when I give them advice 1 2 3 4 5. I ask my mentee what s/he thinks or his/her opinion about things 1 2 3 4 6. I think my mentee feels disappointed or sad when they are with me 1 2 3 4 7. I help my mentee take his/her mind off things by doing something with him/her. 1 2 3 4 8. When I joke with my mentee, it hurts his/her feelings 1 2 3 4 9. I think my mentee would like it if I spent more time with him/her 1 2 3 4 10. My mentee feels that he/she can not trust me with secrets because I would tell his/her mom or dad 1 2 3 4 11. My mentee thinks that I have lots of good ideas about how to solve a problem 1 2 3 4 12. My mentee wishes I were different 1 2 3 4 98 13. When my mentee is with me s/he feels bored 1 2 3 4 14. When my mentee is with me s/he feels mad 1 2 3 4 15. I wish I knew my mentee better 1 2 3 4 16. My mentee would feel sad if something bad happened to me 1 2 3 4 17. My mentee feels comfortable with my me 1 2 3 4 18. My mentee feels that I care about him/her, even when s/he does things I does not like 1 2 3 4 19. My mentee looks forward to spending time with me 1 2 3 4 20. My relationship with my mentee is important to me 1 2 3 4 21. I am happy when good things happen to my mentee 1 2 3 4 22. My mentee believes that I care about him/her 1 2 3 4 23. My mentee enjoys talking to my me 1 2 3 4 24. My mentee tries to follow my advice 1 2 3 4 25. My mentee trusts me 1 2 3 4 99 E. Mentor Evaluation Inventory Below is a list of statements about your relationship with your mentor. Read each item carefully and indicate you level of agreement for each of the following items. Please write down your response on the rating scale because it makes it easier to answer the items Is not true at all Is Somewhat True Is Exactly true 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1. I believe my mentor likes me 1 2 3 4 5 6 7 2. I like how my mentor praises me when we work or play together 1 2 3 4 5 6 7 3. The praises my mentor gives me makes me more confident about what I can do both at home and school 1 2 3 4 5 6 7 4. My mentor pays attention to what I say and repeats what I say to him/her 1 2 3 4 5 6 7 5. I like when my mentor pays attention to me 1 2 3 4 5 6 7 6. I feel that my mentor appreciates me 1 2 3 4 5 6 7 7. My mentor and I trust one another 1 2 3 4 5 6 7 8. My mentor shows a lot of excitement and enthusiasm when we work and play together 1 2 3 4 5 6 7 9. My mentor repeats what I say to him/her, which lets me know he/she is listening to me 1 2 3 4 5 6 7 10. My mentor is there to help me in the activities I need help in when I ask for assistance 1 2 3 4 5 6 7 11. In general, I like working with my mentor 1 2 3 4 5 6 7 12. Every week, I look forward to working/interacting with my mentor 100 F. Sample Follow-up Practice Quiz Scenario: Doug, a Project Uplift volunteer, and his 6-year-old Project Uplift child Jon, are at the library. Jon is following the rules and using his inside voice while selecting books he would like to have read to him. Doug says to Jon ?Okay, you can stop picking books. I think you have enough.? Jon begins to tell Doug about his favorite book and they continue to read the books Jon selected. 1) Write a Labeled Praise statement that Doug could use to praise Jon for following the rules while in the library. ________________________________________________________________________ ________________________________________________________________________ ______________________________________________________ 2) Instead of Doug?s statement ?Okay, you can stop picking books. I think you have enough,? how can Doug describe Jon?s behavior and tell Jon that he has plenty of books in a more positive manner? Please write a statement incorporating a description and a more positive statement for Doug to say to Jon. ________________________________________________________________________ ________________________________________________________________________ ______________________________________________________ 3) As Jon begins to tell Doug about his favorite book, what PRIDE skill could Doug use: a. Labeled Praise b. Unlabeled Praise c. Description d. Reflection Please provide a reason as to why the PRIDE skill you selected could be used____________________________________________________________________ ____________________________________________________________ 101 G. Bi-weekly/Monthly Survey Date __________ 1. Your Name: 2. Your child?s name: 3. Your most recent activity with your child: 4. Are you enjoying the time you spend with your child, and do you feel as though your relationship with your child has grown in a positive direction? 5. Is there anything that Project Uplift staff can assist you with? 6. Have you used the skills you learned in training? 7. How often have you used each of the following: praise, reflections, and behavior descriptions? 8. Are there any problems to report? Thank you for your assistance with this process. Please do not hesitate to contact us with any concerns or questions. Project Uplift is a program of the Lee County Youth Development Center, Inc.