TEACHER REPORTS OF CHILDREN?S EMOTION REGULATION: IS THERE CONCORDANCE WITH TRAINED OBSERVERS Except where reference is made to the work of others, the work described in this thesis is my own or was done in collaboration with my advisory committee. This thesis does not include proprietary or classified information. _______________________________ Rachel Leah Jumper Certificate of Approval: ________________________ ________________________ Gregory S. Pettit Jacquelyn Mize, Chair Professor Professor Human Development and Human Development and Family Studies Family Studies ________________________ ________________________ Scott A. Ketring Stephen L. McFarland Assistant Professor Acting Dean Human Development and Graduate Studies Family Studies TEACHER REPORTS OF CHILDREN?S EMOTION REGULATION: IS THERE CONCORDANCE WITH TRAINED OBSERVERS Rachel Leah Jumper A Thesis Submitted to the Graduate Faculty of Auburn University in Partial Fulfillment of the Requirements for the Degree of Master of Science Auburn, Alabama December 16, 2005 iii TEACHER REPORTS OF CHILDREN?S EMOTION REGULATION: IS THERE CONCORDANCE WITH TRAINED OBSERVERS Rachel Leah Jumper Permission is granted to Auburn University to make copies of this thesis at its discretion, upon request of individuals or institutions and at their expense. The author reserves all publication rights. ________________________ Signature of Author _______________________ Date of Graduation iv TEACHER REPORTS OF CHILDREN?S EMOTION REGULATION: IS THERE CONCORDANCE WITH TRAINED OBSERVERS? Rachel Leah Jumper Master of Science December 16, 2005 (B.A., Texas A&M University, 2003) 111 Typed Pages Directed by Jacquelyn Mize The purpose of this study was to examine agreement between observers and teachers on ratings of individual children?s emotional regulation and social competence. This study examined agreement between teacher-observer pairs as a function of the domain of emotion regulation being examined (internalizing or externalizing) and as a function of teacher characteristics (education, experience, emotion beliefs, and training). Teachers and trained observers rated the emotion regulation and social competence of 324 children, 179 boys and 145 girls, using the Socio-affective Profile. Teachers also completed a set of questionnaires about their education, years teaching, continuing education classes, and beliefs about dealing with young children?s emotions. Correlations and regression analyses were used to examine concordance between teachers and observers. v Although teachers and observers agreed modestly to moderately about individual children?s internalizing, externalizing, and social competence, within-informant cross- construct correlations were often higher than cross-informant within-construct correlations, indicating only limited support for construct validity. Teacher characteristics influenced ratings assigned by both teachers and observers. Observers viewed children of teachers with more education as being more socially competent; children of teachers with less training were seen as having more externalizing problems; children of teachers with more experience were seen as having more internalizing problems; and children of teachers with more developmentally appropriate emotion beliefs were seen as having fewer internalizing problems. Teachers with more experience and teachers with more training rated children as having more internalizing problems; teachers with more appropriate emotion beliefs rated their children as having fewer internalizing problems. Both domain of emotion regulation being evaluated and teacher characteristics affected teacher and observer concordance on ratings of children?s emotion regulation. Teachers and observers were in stronger agreement about externalizing than about internalizing. Teachers with more developmentally appropriate emotion beliefs agreed more highly with observers about children?s social competence. Teachers with less experience and teachers with less training were in higher agreement with observers about children?s externalizing. Results suggest that teachers may see social competence as the absence of externalizing behavior and that internalizing is more difficult for preschool teachers to rate than externalizing. Training may help teachers better identify internalizing. vi VITA Rachel Leah Jumper, daughter of Jack and Dana Jumper, was born February 16, 1981, in Lufkin, Texas. She graduated from Woden High School in 1999. She attended Texas A&M University in College Station, Texas for four years, and graduated Magna Cum Laude with a Bachelor of Arts degree in Speech Communication in May 2003. She entered Graduate School, Auburn University, in August, 2003. vii ACKNOWLEDGEMENTS The author would like to thank Dr. Jacquelyn Mize for her help and assistance with the development and refining of the overall thesis and for assistance with statistical analysis. Thanks also to Jared Lisonbee for his help with the dataset and statistical analysis. Thanks are also due to family members Jack, Dana, Rebecca, Dan, and Emily for their support during the course of this investigation. viii Style manual or journal used: APA Computer software used: Microsoft Word XP and SPSS ix TABLE OF CONTENTS LIST OF TABLES AND FIGURES x I. INTRODUCTION 1 II. REVIEW OF LITERATURE 10 III. METHOD 35 IV. RESULTS 45 V. DISCUSSION 63 REFERENCES 74 APPENDICES 82 x LIST OF TABLES AND FIGURES Table 1. Descriptive Statistics of Demographic Variables 38 Table 2. Means, Standard Deviations, Minimum and Maximum, Alphas, and Sample Size of Teacher Characteristics and Teacher- and Observer-rated Internalizing, Externalizing, and SC. Table 3. Correlations Among Teacher Characteristics. 47 Table 4. Correlations Between Observer and Teacher Reports of Children?s Internalizing, Externalizing, and Social Competence. Table 5. Correlations of Teacher Characteristics with Observer Ratings and Teacher Ratings. Table 6. Main Effects and Interactions for Regressions for Observer Internalizing, Externalizing, and Social Competence When Paired with Teacher Internalizing, Externalizing, and Social Competence, and Teacher Characteristics. Table 7. Slopes of Teacher Ratings as Predictors of Observer Ratings at Three Levels of Teacher Training, Teacher Experience, and Teacher Emotion Beliefs. Table 8. Teacher and Observer Agreement as a Function of High and Low Education, Emotion Beliefs, Experience, and Training. Figure 1. Slopes Predicting Observer Ratings of Externalizing from Teacher Ratings of Externalizing at Three Levels of Teacher Training. Figure 2. Slopes Predicting Observer Ratings of Externalizing from Teacher Ratings of Externalizing at Three Levels of Teacher Experience. 61 Figure 3. Slopes Predicting Observer Ratings of Social Competence from Teacher Ratings of Social Competence at Three Levels of Teacher Emotion Beliefs. 46 50 53 57 58 59 60 62 1 I. INTRODUCTION Teachers are important players in children?s early social-emotional development, including in their emotion regulation (ER) and social competence (SC). Because teachers are in close contact with children on a daily basis, they usually know children?s social- emotional strengths and limitations well, and can be an excellent source of data. In the research literature, teacher ratings have been used in numerous studies of ER; however, there are few formal evaluations of the reliability of teacher judgments about children?s ER, or factors that affect teacher judgments. The purpose of this study is to determine if teachers from the Childcare Quality Enhancement Project agree with observers on ratings of preschool children?s ER. We expect that teachers and observers will agree moderately in their reporting, but that the strength of agreement will vary as a function of the domain of ER being evaluated and on individual teacher characteristics. In the following sections, research on the importance and measurement of ER will be considered in more detail. Defining the term, ?emotion regulation,? has been a topic of interest in recent studies and theoretical papers (Bridges, Denham, & Ganiban, 2004). The present research takes the position that ER is a definable and measurable construct. For the purposes herein, ER is defined in accordance with Cole, Martin, and Dennis (2004), as the response to activated emotion and is separate from actual felt emotions. Also useful 2 is a definition provided by Eisenberg and Spinrad (2004) that defines ER as, ?the process of initiating, avoiding, inhibiting, maintaining, or modulating the occurrence, form, intensity, or physiological, attentional processes, motivational states, and/or the behavioral concomitants of emotion in the service of accomplishing affect-related biological or social adaptation or achieving individual goals? (p. 338). The Cole et al. and the Eisenberg and Spinrad definitions imply that ER manifests itself in a variety of ways, one of which is through behavior in social contexts, and that ER could be measured by examining differences in social behaviors. Two manifestations of ER that have received considerable attention in the research literature are externalizing and internalizing behaviors. Externalizing refers to behaviors that are disruptive or aggressive; internalizing refers to behaviors related to anxiety or depressed mood (Bradley, 2000). Although there are probably multiple influences on the development of externalizing and internalizing, many clinicians, neurobiologists, and behavioral researchers believe that ER difficulties lie at the heart of most cases of both (Bradley, 2000; LeDoux, 2002). This report will focus on externalizing and internalizing as manifestations of ER difficulties. One of the challenges of studying ER is distinguishing it from other closely related constructs such as SC. Theoretically, ER is necessary for SC, and in social settings, the two constructs are often manifested through similar behaviors, making it difficult to distinguish the two. The most common means of evaluating ER is to use adult ratings; however, many of the behaviors found on rating scales to assess ER are similar to behavioral indicators used to assess SC. Despite the similarity of the two constructs, ER can be measured in ways that make it clear that when behavior ratings of ER are used, the 3 construct of ER is being captured reasonably well. Specifically, ER can be measured outside a social setting, and without reference to social interactions, which SC cannot be. Studies of children?s regulatory physiology (e.g., vagal tone, El-Sheikh, 2001; Gottman & Katz, 1989) demonstrate that ER has physiological components that are moderately correlated with adult ratings of ER. Laboratory challenges in which children?s responses to frustrating or emotionally arousing situations (e.g., delay of gratification or disappointing situations, Eisenberg, Cumberland et al., 2001) also provide evidence of correlation between children?s observed ER and adult reports of children?s ER. Regulatory physiology and laboratory challenges for children converge with adult ratings of children?s ER and show that adult ratings are a reasonable way to measure ER in children. However, it is not always clear whether ER and SC can be distinguished through behavioral ratings. For example, El-Sheikh (2001) found that teacher ratings of ER (specifically, internalizing and externalizing behaviors) and SC were more highly correlated than were parent and teacher ratings of ER. Within-informant correlations on different constructs that are as high or higher than between-informant correlations on the same construct indicate low construct validity (Campbell & Fiske, 1959). Because it is so important to study ER, and because informant ratings are so cost-effective and practical, efforts to examine and improve the construct validity of ER ratings would be useful. One reason studying ER is important is because teachers identify social-emotional competence as an important tool for kindergarten readiness (Blair, 2002; Lewit & Baker, 1995; Raver & Zigler, 2004). Some elementary school teachers even rank social- emotional competence as one of the most important aspects of school readiness (Blair, 4 2002; Stipek et al., 1998). It has been found that teachers rate children as more teachable if they are emotionally positive and not easily distracted (Raver & Zigler, 2004). Research suggests that there are more teachers who rate social and emotional competence as important than there are teachers who rate academic skills as important for kindergarten readiness (Blair, 2002; Lewit & Baker, 1995; Raver & Zigler, 2004). Lewit and Baker (1995) give figures from a study conducted by the National Center for Education Statistics that looked at kindergarten teachers? perceptions of school readiness. In this study, most teachers rated communication of wants and needs (84%), enthusiasm and curiosity (76%), sensitivity (58%), and not being disruptive to the class (60%) as being essential or very important. Each of these characteristics requires good ER. In contrast, fewer teachers rated use of pencil or paintbrush (21%), knowing the alphabet (10%), and ability to count to 20 (7%) as being important skills for kindergarten readiness. Each of these skills is traditionally academic in scope. These statistics suggest that ER is of great importance to current teachers. Another reason it is important to study ER is the potential for early interventions. As shown above, teachers rate skills that require ER as being of great importance for kindergarten readiness. Some clinical researchers suggest that all children, even those who are good at ER, could benefit from interventions designed to improve their emotional skills (Bierman & Erath, 2004). Several interventions have been designed specifically for children in pre-school. Each of these interventions focuses at least somewhat on ER skills. Denham and Burton (1996) developed an intervention based on the idea that emotional skills are necessary to develop healthy social relationships. This intervention showed that children?s emotional behaviors could be modified even at the 5 age of 3.5 to 4 years. Other interventions also show that teaching young children, teachers, and parents about emotion skills and how to constructively cope with negative emotions is successful in improving children?s emotional functioning (King & Kirschenbaum, 1990; Webster-Stratton, 1998). These interventions, if they do improve ER, will help children to become more equipped with the skills needed for kindergarten readiness. In order to determine school readiness and children?s need for interventions, children?s ER must be measured accurately by informants who see the children in a variety of situations. As mentioned previously, the most common way to assess children?s ER behaviors, and the way used in the current study, is to use ratings of children?s social behaviors. The most common informants for such ratings are parents, teachers, and observers. Both parents and teachers see the children in a wide range of situations, and both add a valuable perspective to the child?s behaviors (Bates, 1994). However, there are a number of reasons why teachers may be particularly sensitive and accurate informants of ER. The first reason teachers are perhaps more accurate informants of ER is that parents have been found to be biased reporters of their children?s behaviors (Gretarsson & Gelfand, 1988). Parents tend to think good and positive behaviors are part of the child?s stable personality, whereas they are likely to attribute negative behaviors to environmental factors (Gretarsson & Gelfand, 1988). Since parents attribute only certain characteristics to being part of the child?s personality, they may not accurately assess the child?s ER capabilities, especially if the child is poor at ER. The second reason to use teacher data is that children?s behavior is not necessarily consistent across different contexts (Anthony, Anthony, Morrel, & Acosta, 2005). A 6 child who is perceived to be well behaved in one setting may be seen as having behavior difficulties in other situations. In a study by Zeman and Penza (1997), it was found that children as young as preschool begin to understand that behavior changes across different interpersonal relationships. Children know that certain emotional expressions are acceptable within some relationships and unacceptable within other relationships. Since preschoolers are capable of understanding that different social relationships call for different behaviors, children are likely to behave differently at school where they are engaged in more peer relationships than they are at home. It has also been found that children behave differently when in the presence of a parent and in the presence of other non-parental adults such as teachers (Feldman & Klein, 2003). When children behave differently at school and home, it will likely be the teachers? judgments that are more relevant and predictive of potential problems for future school related issues. The final reason for using teacher data is that teachers have experience with many (often dozens or hundreds) of children with whom they can compare the child. Teachers see children in a wide range of activities and situations, many of which are frustrating and call for ER skills. Thus, teachers should be in an excellent position to provide meaningful information about a child?s ER relative to other children in similar situations. Since teachers are often used as informants about school children, it is important to determine if teachers are in agreement with other sources of data on children in the same setting. Some studies have shown that there is moderate reliability for teacher data when compared to other informants (e.g., Eisenberg, Gershoff et al., 2001; Roberts & Strayer, 1987). Several studies have reported the reliability between informants for behavior ratings of ER, but it is only a minor piece of the research findings mentioned in 7 the methods section. This is significant because studies report a wide range of reliability estimates (.20 to .75), but have not examined systematic factors that may be affecting agreement. There are at least two sets of factors that may systematically (as opposed to randomly) affect agreement between informants who observe a child in a specific setting. These factors are the domain of ER being evaluated and individual characteristics of the informants, in this case, teachers. One domain of ER that may moderate the accuracy of ratings is that of externalizing versus internalizing. Internalizing symptoms, such as those associated with depression, have proved more difficult to identify. In a study by Grietens et al. (2004), it was found that mothers and teachers had low to moderate agreement on both externalizing and internalizing domains; however, internalizing problems had the lowest agreement between raters. Eisenberg, Gershoff et al. (2001) also found high correlations between teachers? and mothers? reports of children?s externalizing ER manifestations, but reports of internalizing were not as highly correlated. The fact that internalizing is more difficult to observe can also lead teachers both to over- and under-identify internalizing symptoms. That is, studies have found that teachers not only under identify internalizing (Grietens et al., 2004), but also falsely identify children as being depressed when in fact they are not (Auger, 2004). It has also been found that studies using 11 ?15 year old children themselves as informants about their own internalizing and externalizing show that teachers are more accurate judges of externalizing than internalizing (Phares, Compas, & Howell, 1989). In addition to domains of ER, teacher beliefs about behavior and emotions and individual teacher characteristics may affect ER ratings. Teacher perceptions of children 8 are key in the identification of children with good or poor ER. However, little is known about how teacher characteristics affect ratings of children?s ER behaviors. Studies of elementary children report that regular classroom teachers are better informants on children?s ER skills than are special education teachers (Auger, 2004). However, studies also report that that there is little difference between teachers as a function of education and training (Anthony et al., 2005; Auger, 2004). When evaluating the Caregivers? Opinions about Teaching Young Children measure, which is used in the current study, Hyson and Lee (1996) found that teachers holding a degree in early childhood education significantly differed in two areas from those teachers who did not have a degree. Teachers with a degree felt that teachers should be emotionally expressive and that children should be allowed to display their emotions acceptably, whereas teachers without a degree in early childhood education were less likely to agree with these sentiments. This study will seek to determine if teachers and observers are in concordance in their ratings, and look at the characteristics that individual teachers may have that could affect the manner in which ER is rated in children. In this study, teachers and trained observers rated four-year-old children?s ER and SC using two instruments. Items from the PSP on internalizing aspects of ER, externalizing aspects of ER, and SC were identified and scales for each were created. Teacher education, beliefs about emotion, continuing education, and number of years teaching experience were obtained from questionnaires completed by teachers about themselves. Observers watched children in classrooms for a minimum of ten hours and then completed the same questionnaires about children that teachers had completed. Average teacher rating of internalizing ER behaviors, externalizing ER behaviors, and SC 9 will be examined as a function of teacher education, beliefs about ER, continuing education, and number of years teaching. Correlations between teachers and observers will be examined as a function of these teacher characteristics and as a function of domain of ratings, internalizing, externalizing, and SC. It is believed that teachers with more education, a greater number of years teaching, more continuing education hours, and more developmentally appropriate beliefs about emotions in children will agree more highly with ratings of trained observers. 10 II. REVIEW OF LITERATURE The primary purpose of this literature review is to examine existing research related to teacher reports of children?s emotional competence. As an introduction, general definitions of ER, various ways of measuring ER, and how different methods of measurement separate ER from SC will be reviewed. Next, the importance of studying ER will be addressed. The review will end with a focus on teacher report data, including why teachers are useful informants and a discussion of reliability between informants. Conceptualizing and Measuring Emotion Regulation General Definitions of Emotion Regulation As mentioned previously, no single definition of ER is agreed upon. Indeed, an entire special issue of Child Development in March/April of 2004 was dedicated to the latest developments and ideas on ER. The construct of ER is relevant for a variety of disciplines including child development, clinical psychology, education, and the medical field. The fact that so many disciplines acknowledge the importance of studying ER attests to the fact that ER is a construct that is relevant for studies and definable across disciplines (Bridges et al., 2004). The definitions used for the purposes of this paper are the ones that focus on ER as a construct that has measurable behavioral manifestations. 11 ER can best be understood as being separate from the actual, felt emotion. Rather, it is a response to an emotion, not the emotion itself (Cole et al., 2004). Cole, Martin, and Dennis (2004) argue that ER is comprised of changes as a result of an emotion being activated; this can result in a change in the emotion, or a change in psychological processes. As described previously, Eisenberg and Spinrad (2004) expand the Cole et al. definition by listing specific changes that can occur as a result of ER. The Eisenberg and Spinrad definition specifically lists a behavioral response as one manifestation of ER. The behavioral component is of most importance to the present study. Although both definitions acknowledge the additional components of ER, it is the behavioral component that teachers have direct access to. The only way that teachers could infer ER skills would be to watch the child?s reactions to classroom situations that elicit certain behavioral responses such as aggression, facial expressions, or withdrawal. Having a clear definition of what ER means for this study is important so that it is not confused with other, similar constructs, such as the closely related idea of SC. Ways to Measure ER, and Using Measurements to Separate ER from SC Since ER is so closely related to other constructs such as SC, it is important to review the ways in which ER is measured and how these measurements relate to, or distinguish ER from other constructs, especially SC. There are three common ways to measure ER: behavior ratings, behavioral responses to contrived lab challenges, and physiological measures. The constructs of ER and SC are sometimes measured in the same way, and can be difficult to distinguish. This is especially true when the two are both measured through behavior ratings. However, the use of more than one form of measurement for ER suggests that ER is a distinct and measurable construct. A brief 12 overview of three ways to measure ER and a look at evidence that ER can be measured with behavioral ratings will be discussed in this section. Measuring ER with Behavior Ratings One of the most common and cost-effective ways to gather information on children?s emotional and social characteristics is through adult ratings; however, due to the similarity of ER-relevant behaviors and behaviors that are associated with other constructs such as SC, this form of measurement provides data that make it difficult to discriminate ER. Despite difficulties in discrimination, behavior ratings are correlated with other measures of ER, including regulatory physiology and ER responses outside of the social context (Cole, Zahn-Waxler, Fox, Usher, & Welsh, 1996; Gottman & Katz, 1989). The current section explains the usefulness of, and the procedure for, collecting behavior ratings. Ratings of behavior are commonly used in research and are often more inexpensive than other methods, allowing more children?s ER to be measured. Since a questionnaire, or a standardized form, is all that is needed to perform adult ratings, this form of measurement requires the least amount of equipment. Behavioral observations are not only less expensive than other measurements, but they can also be collected from a variety of sources including parents, teachers, and observers. Most commonly used sources are parents and teachers. Later in the review, teacher observations and their reliability will be discussed in detail. When observers are used, they are usually required to complete a certain specified amount of time in the child?s classroom. A study by Eisenberg et al. (1993) nicely illustrates the use of behavior ratings. The Eisenberg et al. study consisted of two rounds of data collection over the course of two semesters of 13 preschool with Time 1 having 42 girls and 49 boys and Time 2 having 45 girls and 48 boys. Behavioral ratings of children were collected from teachers, teacher?s aides, observers, and mothers. Teachers, aides, and observers completed questionnaires for children?s social skills and negative affect, while teachers, aides, and mothers completed questionnaires for coping skills, emotional intensity, and attentional control. In order to obtain sociometric status of the children, participating children sorted pictures of the children in their classroom into piles of those most liked to those least liked. Results from this study indicated that teacher reports of children?s coping, attentional control, and emotional intensity were more highly correlated with children?s social functioning (especially sociometric status) than were mothers? reports. Another interesting finding from this study was the fact that children?s regulatory abilities were more highly correlated with adult reports of SC than were reports of their emotionality (intensity). Measuring ER with Controlled Lab Settings Outside the Social Context Another way in which ER can be measured is in a lab setting outside the social context. This involves processes that are planned for children in order to induce emotions. These procedures require training and moderate amounts of equipment in order to assure quality measurements. When compared to adult reports of children?s ER, significant correlations exist. Eisenberg, Cumberland et al. (2001) conducted a study that looked at ER outside of the social context, in a laboratory setting. This study consisted of 202 mother/child dyads that came to the lab. The procedure for this study consisted of a puzzle task that the children were asked to complete. In this task, children were asked to complete a puzzle that could not be seen by the child, but could easily be peeked at for help. First, a 14 researcher came into the room, explained the task, told the child he or she would be given an attractive prize if they were successful, and then left. The children were videotaped while they worked on the puzzle and later the tapes were coded for their regulatory abilities. After the child completed the puzzle alone, the child?s mother came into the room and was asked to help the child complete the puzzle. The child still could not see the puzzle, but the mother could, and was told to give the child clues to put the puzzle together. All the children in this study were given prizes regardless of completion or lack of completion of the puzzles. Regulation was rated according to the child?s persistence at the puzzle task without cheating. The measure of ER through persistence was significantly correlated with mother (r = .28, p < .001) and teacher (r = .24, p < .01) reports of ER. Results also found that measurements of ER based on persistence were correlated with both mother (r = .22, p < .01) and teacher (r = .25, p < .001) reports of SC. Mother and teacher ratings within domains were modestly to moderately correlated (r = .48, p < .01 for externalizing; r = .16, p < .05 for internalizing). However, within informant ratings were also moderately correlated (r = .47, p < .001 for mothers, and r = .38, p < .001 for teachers), casting some doubt on the discriminate validity of internalizing and externalizing. These data suggest that externalizing and internalizing aspects are moderately associated, perhaps stemming from some common ER problems. Alternatively, the moderate correlations between internalizing and externalizing may reflect monomethod bias. The low correlation between mother and teacher judgments of internalizing provides evidence that this aspect of ER is, as has been suggested, difficult to identify. 15 Measuring ER with Regulatory Physiology Over the past decade, a number of studies have used physiological processes to assess ER. These studies help shed light on the correlations between physiology and ratings of ER obtained from other measures. This measurement technique used by researchers follows the body?s responses to emotionally arousing situations. One measure of regulatory physiology used in this type of experiment uses changes in heart rate to mathematically compute what is known as vagal tone. This form of measurement of ER requires a lab with proper equipment, and requires the greatest amount of money and training. Scholars who use this form of measurement often create laboratory spaces that will be less daunting to the children who participate. Since the child is attached to the physiology equipment, labs are often made to appear like outer space so children feel more comfortable pretending to be astronauts (Cole et al., 1996; Gottman & Katz, 1989). A study by Gottman and Katz (2002) provides a good example of a regulatory physiology lab format designed for children. This study evaluated children who were aged 4- and 5-years old. The children entered the lab and were presented with the equipment in the room. Then they were hooked up to the equipment and given a baseline time to become familiar with their situation before the actual experiment began. Gottman and Katz used a neutral story about fly-fishing to get the baseline measurement of children?s vagal tone. Next, the children were shown a clip from a movie that was designed to be a fearful emotion-arousing situation. Ability to regulate emotion was estimated from the difference between baseline vagal and vagal during the arousing movie clip. When this information was paired with mother reports of emotion down- regulation ability at age 8, the two reports were correlated. This study also found that 16 children with high vagal tone still reacted to the fearful movie clip; however, they were able to recover more quickly. Other scholars have similar procedures to Gottman and Katz (2002) with minor changes in the films or skits used to induce an emotional response from children (Cole et al., 1996; El-Sheikh, 2001; El-Sheikh, Harger, & Whitson, 2001; Gottman and Katz, 1989). In a study conducted by John Gottman and Lynn Katz (1989), ER processes and social interactions were measured. This study consisted of 56 families, 24 with a male 4- to 5-year-old child and 32 with a female 4- to 5-year-old child. For this study, ER was measured physiologically through vagal tone and through observer coding of spontaneous expressions of ER while children watched video clips designed to elicit ER responses. SC was measured through a peer interaction home visit in which a close friend of the child was present and the pair was videotaped. Inter-rater reliability for the peer interaction was .88 for level of play, and .68 for negative interaction. Results from this study indicated that vagal tone was significantly related to observational ratings of ER (r = .37 p < .01). This suggests that observer ratings of ER are related to vagal tone measurements of ER. Another study that used physiological ratings is Cole et al. (1996), which conducted a study that examined ER and behavior problems in preschool children. This study used a sample of 51 boys and 31 girls. ER was measured in a variety of ways, including vagal tone, using a lab designed to simulate outer space so the child was at ease with the physiological equipment. The children watched videos designed to elicit emotional responses while physiological processes were being monitored, and the facial expressions of the children were coded to measure ER. Parent and teacher ratings were 17 used to evaluate behavior problems. This study, like that of Gottman and Katz (1989), found that vagal tone was correlated with other ratings of ER, and could be measured in a variety of ways. Children who were more inexpressive (had less change in facial expressions during the mood inducing periods) were found to have higher ER as measured by vagal tone than those children who showed more changes in facial expressions. When compared with both mother and teacher reports of behavior problems, inexpressive and expressive groups had more problems than those children who showed moderate changes in facial expressions. Vagal tone was also used as an indicator of ER in a study by Mona El-Sheikh (2001). For this study, ER skills were measured along with behavioral components of SC. This study used a sample of 110 boys and 106 girls ages 6- to 12-years from two parent homes. The families went to a university laboratory where the parents were given questionnaires to complete, and the child completed a physiological session and self- report measures. ER was measured by vagal tone (heart rate, heartbeat interval, and respiratory changes). Children?s emotional responses were video taped and then coded. SC was measured through mothers? reports on the Personal Inventory for Children and teacher reports on the TRF. Results showed that vagal tone was significantly related to mothers? reports of child externalizing (r = .29, p < .001), internalizing (r = .34, p < .001), and social problems (r = .18, p < 05). Vagal tone was also significantly correlated with teacher reports of externalizing (r = .36, p < .001), and social problems (r = .28, p < .01). Studies like the ones in the above sections show that when adult ratings designed to reflect children?s ER are compared with ratings outside a social setting and regulatory 18 physiology measures, the measures are correlated. This indicates that children?s ER can be measured with adult ratings, and that such ratings capture at least some of the variance in ER as assessed outside the social setting and with regulatory physiology. Why is it Important to Study ER? ER is Important for School Success as Reported by Teachers As mentioned previously, teacher survey data indicate that teachers rate ER- related skills as being of greater importance than are purely academic skills. Current statistics point to the fact that teacher? conceptualizations of what determines children?s readiness for school includes skills necessary for ER and socially competent behaviors. Lewit and Baker (1995) conducted a study to evaluate perceptions of readiness following government initiatives to ensure that children were ready for school. This study evaluated surveys from three existing national studies. First, part of the 1993 National Household Education Survey that obtained information on parents of 2,126 kindergarteners was evaluated. These parents were asked questions about the general importance of certain characteristics for determining if their children were ready for school. The second survey used in this study was The Kindergarten Teacher Survey on Kindergarten Readiness. One thousand three hundred and thirty-nine kindergarten teachers completed this study asking what skills they thought were necessary for children to be school ready. Parents were also included in this study and answered questions similar to those of the teachers. Finally, Lewit and Baker used the National Survey of Kindergarten Teachers to determine more about what makes children ready for school according to kindergarten teachers. This survey was mailed to 20,000 teachers, and 7,000 responded. The results of these three existing studies were then examined and 19 complied. The complied data indicated that teachers found the ER skills needed for effective SC strategies were of greater importance than were academic skills. Teachers rated following directions, being sensitive to other children?s feelings, and not being disruptive of the class as being of importance. Each of these skills requires children to regulate themselves. On the other hand, parents rated each of these skills as having less importance than did teachers. These findings can be interpreted as meaning that teachers want children to come to kindergarten with effective ER skills and SC skills, while parents are more concerned with academic skills. This interpretation of the data makes teachers? opinions on children?s ER abilities of great importance for ratings of ER, since it is teachers who find these skills to be important. Teachers? beliefs about school readiness were also studied by Lin, Lawrence, and Gorrell (2003). This study collected data from teachers only, making it a teacher specific survey. Findings from this study maintain the importance of ER-related skills to teachers. A national sample was used that obtained information from 3305 kindergarten teachers through a self-administered survey. Results indicated that the greatest percentage of teachers rated telling needs and thoughts, not being disruptive, taking turns and sharing, and being sensitive to others as the most important skills to have for kindergarten readiness. Academic skills were among those skills with the lowest rankings. Once again, the skills rated as being of the greatest importance by teachers were those that require children to be effective at ER. Finally, a study conducted on a much smaller scale by Wesley and Buysse (2003) found that parents and teachers both felt that ER was important, but that parents had more difficulty expressing specific ways children should be ready in terms of ER skills. This 20 study used a focus group methodology in which groups of parents, teachers, and administrators from 10 different counties in the state of North Carolina talked about themes they found important for kindergarten readiness. Total sample size for this study was 118 participants across 20 focus groups. Compilation of the conversations from the focus groups found that parents in this study felt ER skills were important, but were less articulate on the ER-related behaviors than were the teachers. The fact that parents are less likely to know the terms used for ER development suggests again that teachers are a valuable resource for studying the ER capabilities of children in the classroom setting. These studies add to the literature on the importance given to ER skills by teachers, and suggest that teachers could provide invaluable information on children because they appear to be more ?tuned in? to children?s ER capabilities in the school readiness arena. ER is Important for Early Interventions Since ER is important for school readiness and teacher perceptions, interventions that teach children how to be more emotionally competent provide adults with the opportunity to help children. Even those children who are already good at ER, and especially those who are not good at ER skills, can increase ER abilities and closely related SC abilities. It has been found that children between preschool and school age gain greater abilities to self-regulate themselves (Kopp, 1989). Researchers and practitioners are therefore presented with a time in which children are naturally gaining ER skills, and perhaps are more open to increased knowledge of ER skills and acceptable social behaviors. This section reviews interventions used to help develop ER skills in young children. Because of the closely related nature of ER and SC, many of the 21 interventions have a social-emotional component designed to increase both emotional and SC. These interventions have been found to have moderate effects on children?s behaviors. Denham and Burton (1996) implemented an intervention for four-year olds in preschools based on the belief that emotional competence is necessary to develop successful relationships. This intervention was designed to improve emotional skills. Seventy children were selected to participate in the study with a control group of 60 children from the same centers. The children participating in the study ranged in age from 3.5 to 5 years. The seven teachers who participated in the study had varying degrees of education and number of years spent teaching. Since the children in the control group were from the same centers, their daily routines and experiences were generally similar. The participating teachers attended training sessions on emotion issues. Teachers received specific training on ER skills, including techniques to help children calm themselves. Students who participated were rated by the teacher and by observers who rated affect and social interaction. Results of the study indicated that children who participated in the intervention were seen as having decreased negative emotion displays and an increase in positive social interaction with peers. In addition, children who were in the most need of the intervention were the ones who changed the most; however, children who were not at risk also showed improvements. In 1998, Carolyn Webster-Stratton conducted an evaluation of an intervention that provided parents and teachers with knowledge on increasing positive affect and social skills, and decreasing negative affect in children in Head Start. The study used 296 families to receive the experimental condition (intervention) and 130 families to be in the 22 control condition. The sample size totaled 426 families who participated, along with 45 intervention teachers and 31 control teachers. In order to complete the intervention, parents participated in classes that focused on strengthening children?s social skills and pro-social behaviors. Teachers in the intervention participated in a two-day workshop that taught them the content of the parent intervention so that classroom practices would mirror the intervention strategies and included topics like setting clear rules, teaching social skills, and teaching self-control strategies. Almost half (49%) of the children participating in this intervention were classified as ?high-risk? children. Results revealed that the intervention yielded significant changes for a variety of outcomes. Children in the intervention condition showed significant decreases in negative behaviors and negative affect, and significant increases in positive behaviors and positive affect. This outcome held true in the home setting for a year follow-up. In the school setting, children were rated more positively in the year of the intervention, but by the time the follow-up was completed a year later, the differences were not significant. This suggests that if children?s behaviors are to be changed in a specific setting, that setting should be the focus of the intervention. Teachers in this study were only given a two-day intervention whereas parents were given an 8- to 9-week intervention. This intervention showed that increased positive affect can be taught and can create long-term outcomes of positive affect in the environment that is the focus of the intervention. Another social skills intervention was evaluated by King and Kirschenbaum (1990) and found to be moderately successful. This study used a sample of 135 children ranging in age from kindergarten to 4 th grade. Children in this study were randomly assigned to receive the full intervention, a partial intervention, or no intervention at all. 23 This intervention focused on behaviors associated with ER, such as internalizing and externalizing. Children in the full intervention group were offered participation in a social skills group in which they were taught various skills to improve social behaviors. The full intervention group also included consultations for teachers and parents that provided them information on normal child behaviors, behaviors that call for referral, and teaching children behavioral management strategies. Another group of children were offered consultation only, and the final group of children were offered no intervention. Results of this study found that all the children showed some increases over time in behavioral competencies and decreases over time in behavioral problems with the use of the program; even those who were not participating showed some improvements. However, this study did find that there was slight evidence that the full-intervention program led to larger changes, especially for the internalizing symptom of depression. Why Use Teacher Data to Assess ER? As was discussed earlier, there are several ways to assess ER, the most common and easiest for researchers to gain access to are behavioral ratings that can be gathered from a variety of sources. Children?s teachers are one source of information about ER in preschoolers. The following section discusses why teacher data is useful for ER ratings. Use Teachers Because Parents are Biased Reporters Since there are multiple ways to measure ER, researchers have to decide which method is best for their studies. There are a number of reasons to believe that teachers may make particularly good informants about ER. One of the reasons that teachers may be better at rating ER is that parents can have biases when rating their own children. 24 Gretarsson and Gelfand (1988) examine the biases that mothers show in their reports of children?s behaviors. This study had a sample of 60 mothers with children enrolled in public elementary schools. Mothers were interviewed and asked questions about child characteristics and questions about specific events. The responses from the interviews were then coded. The results indicated that mothers thought that positive child characteristics largely reflected personality factors, rather than environmental factors, while negative behaviors were seen as being a function of the environment more than the child?s personality. Mothers? explanations of specific events also showed children in a more positive light. Positive events were commonly seen by mothers as being the result of child characteristics while negative events were seen as resulting from environmental factors outside the child?s control. Mothers viewed good behaviors as stable characteristics of the child while negative behaviors were seen as being out of the ordinary and a result of the situation the child was in, rather than the child?s personality. This study shows that mothers view their children as being more positive than they are likely to be, seeing only good characteristics as being stable while attributing negative behaviors to outside influences. Use Teachers Because There is not Consistency Across Contexts Even if parents were not found to have reporting biases, it has been found in research that children begin the process of acting differently according to context at a young age (Zeman & Penza, 1997). Zeman and Penza (1997) performed a study using 44 preschool children (23 girls and 21 boys). Children who participated were read stories which were designed so the children would feel negative emotion. Children were then asked questions about what they expected to happen to the protagonist in the story. 25 Parents participating in the study filled out questionnaires about the emotional climate of the home. The results of this study indicated that there was a significant main effect for the audience on expression of negative affect. In both cases, peers were perceived to be the group for which children were the most regulated, or at least believed they should be most regulated. This finding is important because it shows that children as young as preschool have already begun to realize that the interpersonal situation modifies the acceptability of behaviors. Pamela Cole (1986) developed a methodology to study children?s (3- to 9-years old) ER skills through spontaneous facial expressions in a laboratory setting. In this study, a ?laboratory? was set up outside of the classroom, but still at the child?s school. In the first wave of data collected by Cole, children were shown prizes and asked to rate the prizes according to desirability. The participating children were then told that the prizes would be given out if they answered some questions. Children were then asked to examine a set of pictures and discuss them with the researcher. After the first set of pictures was viewed, he or she was given the prize they had rated as most desirable prior to the picture viewing. The procedure was repeated with the child now receiving the prize rated as least desirable. The child then discussed the prizes and how receiving the prizes made him or her feel. The emotions of the children while receiving their prizes were rated by their facial expressions as observed on videotape. The second wave of data collected by Cole was similar to the first procedure. However, when the children were given the second prize, it was presented as broken. Children were later told that they could trade the prize and 90% of the children chose to trade for their second or third choice prize. These videos were coded and results indicated that children were more 26 likely to control their facial expressions (i.e., not act disappointed) when the researcher was in the room. However, when the children were alone, they did not regulate their facial expressions as much. Children as young as 3- and 4-years old were able to maintain more positive facial expressions when the researcher was in the room than when they were alone. In a study by Murphy, Shepard, Eisenberg, and Fabes (2004), it was found that there was little correlation between the school and home context. This study examined a variety of constructs including both ER and SC. The study was a follow-up on data collected six years previously when the children were in kindergarten or preschool. The study consisted of 33 girls and 31 boys who were part of the original study. When reports of children?s SC and problem behaviors were assessed, teacher and parent data were not correlated. However, mother and father data were correlated, suggesting there might be changes in behavior across the contexts of school and home. Teacher reports of SC were correlated with both teacher and parent reports of high ER and low negative emotion. This correlation was true for both the current time of testing and for data collected two, four and six years previously. In a study by Feldman and Klein (2003), toddlers were studied to examine their compliance to mothers, fathers, and caregivers. This study used a sample consisting of 90 toddlers. Observational data were utilized to examine mother-child interactions, teacher-child individual interactions, and teacher-child group interactions. Children were observed within a month time span both at home and at school. This study found that while there are some areas of behaviors that are similar across contexts, compliance was not always correlated between home and school. Parents? and caregivers? seem to elicit 27 different compliance levels from children. Since children?s ER (including compliance) is rated as important by teachers, the teacher?s interactions with the child may be more significant for school based interventions and ratings. Use Teachers Because They Have Experience with a Wide Variety of Children A final reason that teachers are important sources of data is that they have experience with a greater number and variety of children than parents do. While parents have a great deal of experience with their own children and know their own children better than do other adults, teachers have experience with dozens, or even hundreds, of children. The fact that teachers have such a wide range of experience should make them capable of noticing differences among children that parents might not see. Teachers may perhaps have a more realistic view of what is developmentally appropriate behavior for preschool students. Reliability of Teacher Reports Teacher Reliability in Rating ER and SC Teacher report data have been used with some success in studies of children?s ER and SC. Teacher data are used often for information on a variety of children?s behaviors and are a source of information that can provide practitioners with valuable insight into children. As will be seen, the reliability of teacher reports varies as a function of the domain being rated. Teacher report data for externalizing behaviors, defined as acting out or being aggressive, and considered a marker of poor ER, are found to be consistent with reports from mothers. Eisenberg, Gershoff et al. (2001) demonstrate the differences in correlations in a study conducted to examine the role of regulation on behavior problems 28 and SC. This study used a sample of 315 children between the ages of 4.5 and 8. Some children used in this study were of the same age as were the children in the proposed study, but the age was extended to include older children as well. Mothers for this study completed the Child Behavior Checklist and the teachers completed the Teacher Report Form. Results from the Eisenberg, Gershoff et al. study found that teachers were correlated with mothers in their ratings of children?s externalizing behaviors (r = .42, p = .001). This study also found that teacher reports were significantly correlated with mother reports of regulation behaviors (r = .48, p = .001). SC as reported by teachers was also correlated with mother reports (r = .43, p = .01). This study demonstrates that teachers? ratings can be correlated with social-emotional development relevant construct ratings made by other informants. In a study by Roberts and Strayer (1987), 35 two-parent families with a preschool-aged child participated in a study to evaluate aspects of children?s emotional distress. Families participated in a home visit, observer ratings, and self-report measures. The teachers of the children involved in the study completed a 72-item Q-sort for each child on the child?s competence. For some of the students, two teachers who knew the child well participated in completing the Q-sort. For these children who had more than one teacher completing the scale, the average correlation was .69. This implies that teachers in the same setting are rating children in a relatively similar manner, and suggests that teachers should be reliable informants of children?s classroom behaviors. Another study that found multiple teachers from the same school rate children similarly was done by Eisenberg et al. (1993). This study had a sample of 42 girls and 49 boys who participated. Children were assessed by a variety of adults including 29 parents, teachers, and college-age observers. Teacher reports of social skills were found to be correlated with teacher aide ratings of SC (r = .72 at T1 and r = .75 at T2, p < .001). It is interesting to note that the two reports were correlated at both T1 and T2 despite the fact that the aides changed from T1 to T2. Teachers? and aides? ratings of children?s popularity were also correlated (r = .48 at T1 and r = .62 at T2, ps < .001). This study also evaluated emotional intensity and found that teachers and aides once again were significantly correlated in their reports of children?s emotions (r = .59 at T1 and r = .43 at T2, ps < .001). In an analysis of a large number of studies that used data from a variety of sources on children?s emotional and behavioral problems (Achenbach, McConaughy, and Howell, 1987), it was found that teachers had the highest reliability when matched with an observer (r = .42, p < .001), and the lowest when matched with parents (r = .27, p < .001) and when matched with the child self-report (r = .20, p < .001). This article suggests that sets of informants who see the child in the same context will agree to the greatest extent. Therefore, when researchers want to evaluate children in the school setting, the groups who would provide the best, most reliable reports should be teachers and observers in the school setting. Teacher report data were also shown to be reliable in a study by Eisenberg et al. (1997). Eisenberg et al. studied children?s ER and found that teacher reports were significantly correlated with a variety of other sources. The sample consisted of 199 children in kindergarten through third grades, their parents, and their teachers. Children were measured on ER, resiliency, emotionality, and social functioning. When the results were analyzed, teacher reports were significantly correlated with those of both peers and 30 parents. Parent and teacher reports of ER measured through ego control was r = .50, p < .001. In addition, parents and teachers were also correlated in their ratings of children?s negative emotionality (r = .26, p < .001). In addition to significant correlations with parents, teachers in this study were also found to be correlated with peers in their reports of social status (r = .29, p < .001). The studies reviewed above show that teachers? reports have been found to be reliable sources of information on children. Multiple teachers? ratings are shown to correlate with each other, and in some cases, with reports from peers, observers, and parents. However, not all studies find these significant correlations between informants. Problems with Teacher Reliability Internalizing behaviors. Despite the fact that teachers have been found to be reliable informants about children?s functioning, there are some problems with using teacher report data. These problems will be discussed in the following section. Agreement between informants can differ as a function of the type of ER problem being rated. For example, internalizing behaviors are found to have lower correlations between observers than externalizing behaviors. In the Eisenberg, Gershoff et al. (2001) study mentioned previously, externalizing behaviors were correlated between mother and teacher. However, this same study found that teacher and mother reports of internalizing behaviors were not significantly correlated (r = .11). This shows differences exist as a function of the behavior being evaluated. Teacher ratings may not be as reliable for some aspects of ER, particularly for domains of ER with less observable or obvious behavioral manifestations. 31 Grietens et al. (2004) found similar results to the Eisenberg, Gershoff et al. (2001) study. Grietens et al. used two samples. The first consisted of 413 children and their parents. The second data set contained 682 parent-child dyads and 788 teachers. All the children being reported on were 5- to 6-year olds in preschool. Measures included the Child Behavior Checklist and the Teacher Report Form. While all the correlations for this study were found to be significant, those for internalizing behaviors were not as strong as correlations for other behaviors. This study found that scores for both the total problems scale and the externalizing scale were moderately correlated across raters (correlations ranged from .38 between fathers and teachers, and .40 between mothers and teachers, to .68 between fathers and mothers, all ps < .01). However, internalizing problems did not follow this trend, and were not as highly correlated. The correlations for internalizing behavior ranged from .20 between fathers and teachers, and .21 between mothers and teachers, to .57 between fathers and mothers (all ps < .01). Eisenberg, Cumberland et al. (2001) studied the effects of ER on externalizing and internalizing behaviors for 214 children. They found that internalizing behaviors were less highly correlated between teachers and mothers than were externalizing behaviors. Teacher reports of externalizing behaviors were highly correlated (r = .48, p < .001). As reported in an earlier section of this proposal, teacher and mother reports of internalizing were less highly correlated than were teacher and mother reports of externalizing behaviors (r = .16, p < .05). Once again, internalizing behaviors may be more difficult for raters to agree on, perhaps because these behaviors are less observable. Teacher characteristics. Differing domains of behavior are not the only cause of variations in inter-rater reliability. Teachers themselves have varying characteristics that 32 make them perceive children differently. Anthony et al. (2005) examined teacher and teacher assistant reports for 76 children, and examined the effects that teacher training had on ratings of ER. The authors conclude that better educated teachers were more reliable (but no correlation coefficient is provided). Importantly, the authors note that reliability between teachers and teacher assistants varied substantially across classrooms, but none of the variables assessed in this study (other than teacher education) accounted for the large variance. The authors conclude that independent ratings by trained observers are needed to more adequately evaluate teacher reliability. We may also conclude that consideration of other teacher characteristics (such as beliefs about emotions and experience) may help explain the variation. Auger (2004) evaluated teacher characteristics on the ability to rate depressive symptomology. Auger utilized a sample that was older than the current sample (6 th - 8 th grade) and consisted of 356 students and 52 teachers. When compared with child self- reports regular education teachers (r = .23, p < .001) were better at accurately rating children?s depression than were special education teachers (r = -.13). Hyson and Lee (1996) evaluated the Caregiver?s Opinions about Teaching Young Children measure, which was used in the present study. This study was based on the belief that adults? attitudes about emotions influence children?s emotional displays. Teachers play an important role in children?s lives and therefore their beliefs about emotions may influence behaviors. Using a sample of 454 caregivers, 279 U.S. and 175 Korean, Hyson and Lee evaluated what characteristics teachers held that affected their ratings of beliefs about preschoolers? emotions. Emotion-rated beliefs were divided into six areas: bonds, expressive, instruction/model, talk/label, protect, and display/control. 33 Teachers were told that all people hold different beliefs, and there were no right or wrong answers to these questions. When evaluated, no significant association was found between number of years teaching and teachers? beliefs. Amount of education was significantly correlated with four of the six subsets of emotion-related beliefs. Teachers with higher levels of education were more likely to believe that emotional bonds are important between adults and children, that it is good to talk to children about emotions, that children are able to control emotions, and that it is bad to protect children from negative emotions. Teachers with a degree in early childhood education were also found to be significantly different from other teachers in that they were more likely to believe teachers should be more expressive, and that children can display emotions in acceptable manners. Summary The research examining teacher reports about children's ER suggests that teachers can provide valid and reliable information about children they know well. However, the reliability of teacher judgments varies considerably over different research reports, and there is little systematic research on factors that contribute to teachers being more or less reliable informants. Factors that seem to affect teacher judgments include what domain or aspect of ER is being judged, with internalizing aspects of ER being more difficult to rate, or at least to agree upon, than are externalizing aspects of ER. Teachers also may vary in their sensitivity to, or accuracy in identifying ER problems among the children in their care. Previous research suggests that teacher education and beliefs or attitudes about emotions affect their judgments of children's ER. Teachers' experience with children logically could affect judgments, with more experienced teachers perhaps being able to 34 identify more sensitively individual differences. However, no research has evaluated the effect of teacher experience on judgments of children's ER. Systematic evaluation of these factors as moderators of how reliable teachers are in reporting on children's ER could strengthen research and practice that relies on teacher judgments to identify children with ER-related problems. 35 III. METHOD Overview The data used for this study were collected through a short-term longitudinal study of preschool children that was developed to examine how children?s experiences in social, physiological, and cognitive domains in preschool affect their performance and adjustment in kindergarten. The data collection for this project, the Childcare Quality Enhancement Project (CQEP), was funded by the National Science Foundation (NSF #0126584 to J. Mize). Data were collected from three cohorts of preschool children. The CQEP study was approved by IRB # 00-141 0006; the IRB for the current study has been approved by IRB #05-116 EX 0505 and is attached in Appendix A. Of specific interest for the proposed study are the measures that apply to children?s emotional and social development, especially those questionnaires that tap ER and SC. Follow-up data were collected the following year in kindergarten; however, follow-up information is not included in the present study. Participants Fourteen childcare centers were contacted about potential involvement in the project, and directors were told about the nature of the study. The directors of each center provided letters of support stating intentions to participate in the study. Due to the fact that two of the centers were more than 30 miles away, only 12 of the 14 potential 36 centers participated in the study. In order to participate, the majority of the children in a classroom were required to be entering kindergarten the following year. Cohort 1 consisted of 17 participating classrooms, Cohort 2 had 16 classrooms, and Cohort 3 consisted of 14 participating classes. The teachers and parents of the children were given questionnaires to complete at the beginning of the data collection. All children in the four-year-old classrooms met the criteria to participate in the CQEP study. There were no differences in sex or race of the children whose parents declined to participate in the study and those who agreed to participate. The majority of children in the classrooms chose to participate with the exception of one classroom with 44% participation. The average participation rate for the classrooms was 81%, with one classroom having 100% participation. Teachers indicating their agreement to participate in the study signed informed consent letters. Childcare centers were offered $5 incentive for each family and child that completed the entire study. This money was to go to purchases for the classroom. An additional monetary incentive was offered for classrooms with more than 75% participation rates. Teachers were given $20 for finishing the questionnaires about their background, education, experience, and teaching philosophy. In addition, the teachers were given $5 for completing each set of questionnaires on each participating child. Sample The larger study collected data from 502 children; however, only 324 children had both teacher and observer data. Data from these 324 children will be evaluated (145 girls, 179 boys). The average age of the children in both the full sample and the sub- sample used for this study was 52 months. Children came from a variety of economic 37 and racial backgrounds as determined by parent reports. SES was determined by the Total-Based Socioeconomic Index (Entwisle & Astone, 1994). SES data were available for 347 children. SES in the current sample ranged from 10 to 96 (mdn = 64). According to Entwisle and Astone (1994), a score of 10 would be a winding twisting machine operator, scores in the 60s include police and firemen, and scores in the 90s include dentists and veterinarians. SES was slightly negatively skewed (-.33), with half of the sample falling between 40 and 80. Of the children who participated, 99 were African American, 208 were Caucasian, and 17 were from other racial backgrounds. There were no significant differences between children who did and who did not have teacher and observer data based on age (F = .45, df = 1, 500, ns), sex (?? = 2.59, ns), race (?? = 5.13, ns), or SES (F = .31, df = 1, 346, ns). 38 Table 1 Descriptive Statistics for Demographic Variables All Participants Current Sample N 502 324 Age (SD) 52 months (4.64) 52 months (4.66) Sex Male 264 (52.6%) 179 (55.2%) Female 238 (47.4%) 145 (44.8%) Race Caucasian 304 (60.6%) 208 (64.2%) African American 168 (33.5%) 99 (30.6%) Other 30 (6.0%) 17 (5.2%) SES 1 (SD) 60.95 (21.31) 60.55 (21.57) 1 SES data were available for 347 families 39 Procedures Observer Ratings of Children Observers were trained graduate and undergraduate students. Observations in the classrooms were done throughout the course of the fall semester of each school year beginning in October. Each observer conducted a minimum of 10 hours of observations prior to completing the rating scales for each child. Observers completed a set of questionnaires about children?s ER and SC, which were identical to the questionnaires completed by teachers. All observers were asked to complete their questionnaires during the fall semester. These measures are described in a subsequent section. Teacher Ratings of Themselves and Children Teachers were asked to complete two sets of questionnaires: one on themselves, and another set on participating children in their classrooms. Questionnaires were distributed in October, and teachers were asked to return their questionnaires as soon as they were completed; completed questionnaires were handed in throughout the school year. Teacher reports about themselves included beliefs about emotion in young children, information about their education, experience, and recent continuing education training. Teachers rated children?s ER and SC using the same measures used by the observers (described in a subsequent section). Measures Socio-affective Profile (short version) Teachers and observers both completed the Socio-affective Profile (short version), herein referred to as PSP (see Appendix B) (La Freniere, Dumas, Capuano, & Dubeau, 1992) about each child participating. This measure was designed to assess the affective 40 expression, social competence, and adjustment difficulties of preschool children as they interact with adults and peers (La Freniere et al., 1992). The PSP is a 30-item scale consisting of descriptors such as ?Easily frustrated,? ?Cooperates with other children,? and ?Defiant when reprimanded.? These items are rated on a scale where 1 = never and 6 = always. This scale also provides the rater with a ?cannot evaluate? option. La Freniere et al. (1992) conducted a study with a Canadian sample to evaluate the reliability, internal consistency, and stability of the PSP. Three principle factors, SC, internalizing behaviors, and externalizing behaviors, were found to account for 67.1% of the variance in LaFreniere?s 1992 study. Alphas for the three subscales of the PSP ranged from .79 to .91. LaFreniere also evaluated concurrent validity using the Child Behavior Check List (CBCL). Moderate convergence was found between the PSP and the CBCL. Correlations of ratings on the PSP and the CBCL across the classrooms ranged from .53 for internalizing to .66 for externalizing (LaFreniere, 1992). Lindsey and Colwell (2003) conducted a study with an American sample of preschoolers and report internal consistencies for the PSP of .93 to .95. Scales for the current study were developed using procedures described in La Freniere et al. (1992) to form the following variables: teacher- rated internalizing (? = .82), teacher-rated externalizing (? = .87), teacher-rated social competence (? = .79), observer-rated internalizing (? = .83), observer-rated externalizing (? = .94), and observer-rated social competence (? = .82) (see Appendix C for list of questions included in each scale). Teacher Background Questions As part of the teacher packet, a section entitled, ?About You and Your Opinions as a Teacher,? was included for teachers to provide information about their education and 41 years of experience (see Appendix D). Items used in the proposed study included, ?How many years have you taught kindergarten or preschool, including this year?? which constituted the measure of experience. Seven questions asked about teacher education and training. From these items, three items were used in the current study: one item asked about the highest degree earned (?What is the highest degree or certification you have earned so far??), one asked about major or field of study (?If you attended college, what is/was your major??), and one asked about workshops or Continuing Education classes during the past year (?How many workshops or Continuing Education Units in ECE or child development have you completed in the past 12 months??). Responses to the degree question were assigned values ranging from 1 = High School to 5 = Graduate (see Appendix E for number of teachers in each category). Responses to the question about major were assigned values ranging from 1 = other, to 4 = Human/Child Development. Human/Child Development was assigned the most points in light of research showing that the best child care outcomes are associated with specific training in child development (Travers, 1981). The score for degree earned was then multiplied by the score for major to form a composite called education that was used in analyses for this report. The product, rather than the sum, of degree and major was used in order to give greater weight to more training in child development. For a copy of the questionnaire with values assigned for each answer see Appendix D. A teacher who had a bachelor?s degree in Child Development received a score of 16 (4 points for degree, and 4 points for major in Child Development), whereas a teacher with a bachelor?s degree in English would receive a score of 4 (4 points for degree, and 1 point for major in ?other?). The variable training 42 was derived from teachers? answers to the question about continuing education and workshops. Teachers gave a numerical response to this question, ranging 0 ? 80. The modal response to this question was 16, and answers of greater than 16 units were assumed to reflect a misinterpretation of the question. Therefore, all responses of 16 or greater were recoded to equal 16. Caregivers? Opinions about Teaching Young Children As part of the teacher packet about themselves, teachers completed a questionnaire designed to assess their personal beliefs about emotions (see Appendix D) (Hyson & Lee, 1996). The original Hyson and Lee measure included 23 items such as, ?As a teacher it is important for me to teach children acceptable ways to show their feelings,? ?Children in my class are too young for me to discuss my feelings with them,? and ?Children need to feel emotionally close to their teachers.? Teachers rated each of these items on a 6-point scale, with higher numbers representing greater agreement in the original study. Alphas for the subscales in the Hyson and Lee (1996) study were adequate (bonds = .62, expressiveness = .46, instruction/modeling = .43, talk/label = .53, protect = .41, display/control = .59). An additional 19 items were added to the measure for the current study. Additional items added for the present study were also ranked on a 6-point scale and included items such as, ?I try to get an idea each day of how children in my class are feeling,? ?When a child in my classroom is sad, I try to get him or her to talk about it,? and ?Children often act sad just to get their own way.? Added items are indicated on the questionnaire (See Appendix D) with the letters AU in parentheses after the item. For the present study, some items were reverse scored (e.g., ?Some parents worry too much about children?s feelings,? and ?Children in my class are too young for 43 me to discuss the causes of their feelings with them?) so that higher scores reflected more developmentally appropriate emotion beliefs and practices. Cronbach alphas were computed, and 4 items were deleted, yielding an internally consistent scale of developmentally appropriate emotion beliefs consisting of 37 items (? = .84). Research Questions The overall goal of the current research study was to examine the concurrence between teachers? and observers? reports of ER and whether or not teachers and observers appear to discriminate ER and SC. Concurrence will be examined as a function of domain of ER. In addition, characteristics of the teachers who participated in the study will be analyzed to determine if there are certain characteristics that affect the concordance of teachers and observers. Specifically the following questions will be addressed: 1. Are teacher reports of children?s ER behaviors correlated significantly with observer reports of children?s ER and are their reports of ER distinguishable from judgments of SC? Construct validity will be demonstrated to the extent that correlations across informants within domain are higher than are correlations within informant across domain. 2. Does agreement between teachers and observers vary as a function of ER domain? Specifically, do teachers and observers agree more strongly about externalizing than they do about internalizing? It is expected that teacher and observer ratings of externalizing will be more highly correlated than are teacher and observer reports of internalizing. 44 3. Does teacher-observer agreement about ER vary as a function of specific teacher characteristics (i.e., years experience, education, training, or personal beliefs about emotion)? It is expected that teachers with more education, more years experience, more training, and stronger beliefs in accepting and fostering children?s ER skills will agree more strongly with trained observers. 45 IV. RESULTS Overview Results from the analyses will be presented in five sections. Descriptive analyses will be presented first with subsequent sections addressing research questions posed by the study. Associations within measurement domains, associations among observer and teacher reports, and associations among teacher characteristics and teacher and observer judgments of children?s functioning will be presented. Finally, the differences in agreement between observers and teachers will be analyzed as a function of teacher education, emotion beliefs, experience, and training. Descriptive Analyses As can be seen in Table 2, teachers and observers reported considerable variation among children on internalizing, externalizing, and SC. The ratings of both observers and teachers had somewhat greater variance for externalizing (coefficient of variation (CV, computed as SD/M) = .46 and .48, respectively) than for internalizing (CV = .32 & .39, respectively) or for SC (CV = .34 and .22, respectively). Ratings of SC were slightly negatively skewed (-.45 and -.07, for teacher and observer, respectively), whereas internalizing and externalizing were positively skewed (skew ranged from 1.05 to 1.60), indicating that both teachers and observers saw children as socially competent and low in internalizing and externalizing problems. 46 Compared to observers, teachers rated children as having more internalizing (t = -2.25, df = 323, p < .025) and externalizing problems (t = -5.20, df = 323, p < .00), but paradoxically, as being more socially competent (t = -4.40, df = 323, p < .000). Table 2 Means, Standard Deviations, Minimum and Maximum, Alphas, and Sample Size of Teacher Characteristics and Teacher- and Observer-rated Internalizing, Externalizing, and SC M SD Minimum & Maximum ? N Teacher Characteristics Emotion Beliefs 4.65 .51 3.59 ? 5.62 .84 50 Experience 6.75 6.33 .50 - 27 -- 50 Education 6.64 5.00 1 - 20 -- 50 Training 7.70 5.90 0 ? 16 .88 46 Observer Ratings Internalizing 1.81 .58 1 ? 4.5 .83 324 Externalizing 1.88 .90 1 ? 6 .94 324 Social Competence 3.80 1.03 1.10 ? 6.30 .82 324 Teacher Ratings Internalizing 1.92 .76 1 ? 5.30 .82 324 Externalizing 2.20 1.01 1 ? 5.8 .87 324 Social Competence 4.08 .88 1.5 - 6 .79 324 Note. Education computed as degree X major. 47 Associations Among Teacher Characteristics In order to investigate non-redundant contributions of teacher characteristics to ratings of children?s ER and SC, it was necessary to examine the intercorrelations among teacher characteristics. As can be seen in Table 3, teachers with more education had more developmentally appropriate emotion beliefs, more experience, and more continuing education training in the past year. On the other hand, teachers with more continuing education training had less appropriate emotion beliefs. Finally, teachers who had been teaching longer had participated in more continuing education classes in the past year. Table 3 Correlations Among Teacher Characteristics Teacher Characteristics Emotion Beliefs Experience Training Teacher Characteristics Education .55** .16** .18** Emotion Beliefs -.03 -.20** Experience .37** ** p < .01 48 Associations Among Observer and Teacher Reports The first research question was whether teachers and observers would agree on ratings of children?s internalizing, externalizing, and SC, and whether there would be higher correlations within child adjustment domain across informants, than within informants across domains. Correlations among teacher and observer ratings are shown in Table 4. Good construct validity is indicated if correlations within-construct cross- informant correlations are higher than are cross-construct within-informant correlations, or cross-informant cross-construct correlations. The teacher and observer reports of internalizing, externalizing, and SC were all significantly, but moderately, correlated within child adjustment domain (absolute values of rs = .22 to .35, mdn = .25; all ps < .01). Cross-informant, cross-construct correlations were, for the most part, modest (the absolute values of r = .09 to .36, mdn = .11). However, correlations within observer, across domains absolute values were often higher than within-domain, cross-observer correlations (rs = .20 to .55, mdn = .30, all ps < .01). The highest within-informant correlations were between externalizing and SC. Both sets of informants tended to see children with more externalizing problems as less socially competent (rs = .50 and .55 for observers and teachers, respectively, both ps < .01). Thus, there was modest evidence of construct validity, particularly for externalizing and SC. The second part of this research question was whether teachers and observers would agree more strongly in regards to children?s externalizing problems than in regards to children?s internalizing problems. To address this question, the significance of the difference in two dependent correlations (the correlation between teacher and observer ratings of externalizing (r = .35) and the correlation between teacher and observer ratings 49 of internalizing (r = .22)) was computed. The correlations were significantly different, t (321) = 1.99, p < .05, indicating that teachers and observers were more in agreement about children?s externalizing problems than about children?s internalizing problems (see Table 4). There were no significant differences in the magnitude of other teacher- observer agreement coefficients (i.e., internalizing versus SC, or externalizing versus SC). Table 4 Correlations Between Observer and Teacher Reports of Children?s Internalizing, Externalizing, and Social Competence Teachers Observers Internalizing Externalizing SC Internalizing Externalizing SC Teachers Internalizing __ .30** -.30** .22** .09 -.11* Externalizing __ -.55** .10 .35** -.29** SC __ -.09 -.36** .25** Observers Internalizing __ .20** -.21** Externalizing __ -.50** SC __ * p < .05, ** p < .01 50 51 Associations Among Teacher Characteristics and Teacher and Observer Judgments of Children?s Functioning Although not posed as research questions or hypotheses, it was of interest to explore potential associations among teacher characteristics and teacher and observer ratings of children?s functioning. Examination of bivariate correlations between teacher characteristics and ratings of children?s functioning will be useful in interpreting multiple regression equations that will be presented in the subsequent section on teacher characteristics as moderators of teacher-observer agreement. The bivariate correlations revealed that teacher characteristics affected how observers rated children (see Table 5). The top half of the table shows correlations between teacher characteristics and observer rating. Children of teachers with more education were viewed by observers as more socially competent and as having fewer internalizing problems than were children of teachers with less education. Children of teachers with more years? experience were seen as having more internalizing problems. Children of teachers with more developmentally appropriate beliefs about emotion were seen as having fewer internalizing problems than were children of teachers with low emotion beliefs. Because teacher education and emotion beliefs were significantly positively correlated (see Table 3) and both were correlated with observer ratings of internalizing, a partial correlation was computed between teacher emotion beliefs and observer ratings of internalizing while controlling for teacher education. Controlling for teacher education, emotion beliefs no longer predicted observer ratings of internalizing 52 (pr = -.07, ns). Finally, children of teachers with more training were seen as having fewer externalizing problems than children in classrooms where the teacher had fewer hours of training. The bottom half of the Table 5 shows correlations between teacher characteristics and teacher ratings. Teachers with more education and teachers with more developmentally appropriate beliefs about emotion tended to see children as having less internalizing. Again, however, after controlling for teacher education, teacher beliefs about emotion no longer significantly predicted ratings of internalizing (pr = -.07, ns). In contrast, teachers with more continuing education credits and teachers with more years? teaching experience tended to see children as having more internalizing problems (see Table 5). Because experience and training were significantly correlated (see Table 3) and both were significantly correlated with teacher-rated internalizing, a partial correlation was computed between training and teacher-rated internalizing controlling for experience. Training continued to significantly predict teacher-rated internalizing (pr = .14, p < .05). 53 Table 5 Correlations of Teacher Characteristics with Observer Ratings and Teacher Ratings Observer Ratings Internalizing Externalizing Social Competence Teacher Characteristics Experience .15** .06 -.08 Emotion Beliefs -.14* -.07 .03 Training -.04 -.12* -.05 Education -.14* -.10 .23** Teacher Ratings Internalizing Externalizing Social Competence Teacher Characteristics Experience .12* .05 -.05 Emotion Beliefs -.13* -.02 .07 Training .17** -.02 .04 Education -.13 -.04 .07 *p < .05, ** p < .01 54 Differences in Agreement as a Function of Education, Emotion Beliefs, Experience, and Training To address hypotheses about whether teacher characteristics influenced teacher- observer concordance, a series of 12 multiple regression equations were computed in which teacher ratings on the relevant domain of child functioning (externalizing, internalizing, or SC), a single teacher characteristic (education, continuing education, emotion beliefs, or experience), and the multiplicative teacher characteristic by child functioning interaction term were entered. All variables were centered prior to analyses. The results of these regression equations are presented in Table 6. As would be expected based on the pattern of bivariate correlations, teacher ratings significantly predicted observer ratings in all equations, and the main effects for teacher ratings will not be discussed further in this section. In addition, in several cases, the main effect of teacher characteristics was a significant predictor of observer ratings. These results mirror precisely the results of the bivariate correlations between teacher characteristics and observer ratings described previously (see Table 5 and previous section on Associations Among Teacher Characteristics and Teacher and Observer Judgments of Children?s Functioning for these findings) and also will not be described further here. Results are presented for all regression equations in Table 6, but only regression equations in which the interaction term was significant will be discussed here. The results of these significant analyses are presented in Table 7. Regression analysis revealed three significant interactions: (a) teacher training x teacher ratings of externalizing predicting observer ratings of externalizing; (b) teacher experience x teacher ratings of externalizing 55 predicting observer ratings of externalizing; (c) emotion beliefs x teacher ratings of SC predicting observer ratings of SC. To interpret the significant interactions, procedures described by Jaccard et al. (1990) were followed. For analyses in which the interaction term was significant, the slopes were captured at high (1 SD above the mean), moderate (the mean), and low (1 SD below the mean) levels of the teacher characteristic, according to procedures outlined by Jaccard, Turrisi, and Wan (1990) and can be found in Table 7, and graphs of these slopes are presented in Figures 1 through 3. In addition, to aid in interpretation of significant interaction terms, the data were split at the median on each teacher characteristic, and correlations were computed between teacher and observer ratings at high and low levels of each characteristic. These correlations are presented in Table 8. The first significant interaction that was found was that of teacher continuing education training and ratings of externalizing. As seen in Table 6, the interaction between teacher training and teacher ratings of externalizing was significant. To explore the nature of this interaction, slopes of the regression coefficient at three levels of experience were computed. As see in Table 7 and Figure 1, the externalizing ratings of teachers with less training predicted observer ratings more strongly than did ratings made by teachers with more training. The nature of the interaction was further explored by dividing the sample at the median on teacher training. As seen in Table 8, the teacher- observer correlation on externalizing was higher for low-training teachers than for high- training teachers. The second significant interaction found was that of teacher experience and externalizing ratings (see Table 6). In order to explore the nature of the interaction, 56 slopes of the regression coefficient at three levels of experience were computed. As seen in Table 7 and Figure 2, the externalizing ratings of teachers with less experience predicted observer ratings more strongly than did ratings made by teachers with more experience. The nature of the interaction was further explored by dividing the sample at the median on teacher training. As seen in Table 8, the teacher-observer correlation on externalizing was higher for low-experience teachers than for high-experience teachers. The final significant interaction found in this study was between teacher emotion beliefs and SC ratings (see Table 6). The regression slopes at three levels of teacher emotion beliefs was computed, and as can be seen in Table 7 and Figure 3, the SC ratings of teachers with higher emotion beliefs predicted observer ratings more strongly than did ratings made by teachers with lower emotion beliefs. As can be seen in Table 8, when the sample was divided at the median on teacher emotion beliefs, the teacher-observer correlation on SC was higher for high-emotion belief teachers than for low-emotion belief teachers. Table 6 Main Effects and Interactions for Regressions for Observer Internalizing, Externalizing, and Social Competence When Paired with Teacher Internalizing, Externalizing, and Social Competence, and Teacher Characteristics Teacher Characteristics Education Emotion Beliefs Training Experience R? ? t R? ? t R? ? t R? ? t Child Characteristics Internalizing Steps: 1. Teach Reported Int. .05 .21 3.86 ** .05 .21 3.70 ** .05 .23 4.00 ** .05 .20 3.63 ** 2. Teacher Characteristic .06 -.10 -1.67 ? .06 -.11 -2.00 * .06 -.08 -1.41 .06 .13 2.35 * 3. Internalizing X Char. .06 .07 1.20 .06 -.00 -.08 .06 -.02 -.30 .06 -.02 -.40 Externalizing Steps: 1. Teacher Reported Ext. .12 .35 6.61 ** .12 .34 6.54 ** .12 .32 5.83 ** .12 .34 6.55 ** 2. Teacher Characteristic .13 -.08 -1.61 .12 -.06 -1.23 .13 -.11 -2.10 * .13 -.07 -1.40 3. Externalizing X Char. .13 .04 .71 .13 .07 1.38 .16 -.17 -3.18 ** .15 -.15 -2.81 ** Social Competence Steps: 1. Teacher Reported SC .06 .23 4.38 ** .06 .24 4.44 ** .06 .24 4.20 ** .06 .27 4.80 ** 2. Teacher Characteristic .11 .22 4.10 ** .07 -.00 -.06 .07 -.07 -1.20 .07 -.07 -1.24 3. Social Comp. X Char. .12 .10 1.72 ? .08 .13 2.43 ** .07 -.08 -1.36 .07 .07 1.28 Note. Results for equations in which the interaction term was significant are in bold. ? p < .10, * p < .05, ** p < .01. 57 58 Table 7 Slopes of Teacher Ratings as Predictors of Observer Ratings at Three Levels of Teacher Training, Teacher Experience, and Teacher Emotion Beliefs Slope SE t Training X Externalizing High Training .13 .05 2.89 Mean .28 .05 6.21 Low Training .43 .05 9.55 Experience X Externalizing High Experience .17 .05 3.90 Mean .30 .05 6.71 Low Experience .43 .05 9.52 Emotion Beliefs X SC High Emo. Bel. .43 .25 1.71 Mean .28 .06 4.43 Low Emo. Bel .13 .22 .59 59 Table 8 Teacher and Observer Agreement as a Function of High and Low Education, Emotion Beliefs, Experience, and Training Child Behavior Internalizing Externalizing Social Competence Teacher Character. Training High .21** .17* .21** Training Low .20* .46** .34** Experience High .13 .19* .33** Experience Low .30** .49** .20* Emotion Bel. High .25** .45** .37** Emotion Bel. Low .19* .23** .09 Education High .24** .37* .36** Education Low .20* .34* .15 Note. Pairs of correlations that correspond to significant interactions in the regression equations are in bold; Character = Characteristic, Bel. = Beliefs * p < .05, ** p < .01 0 2 4 6 8 10 12 1 5.8 Teacher Ratings of Children's Externalizing O b s e r v e r R ati n g s of Exte r n al i z i n g High Training Mean Low Training 60 Figure 1. Slopes predicting observer ratings of externalizing from teacher ratings of externalizing at three levels of teacher training. 0 1 2 3 4 5 6 7 8 9 10 15.8 Teacher Ratings of Externalizing O b s e rver R a t i n g s of Ext ern alizi n g High Experience Mean Low Experience 61 Figure 2. Slopes Predicting Observer Ratings of Externalizing from Teacher Ratings of Externalizing at Three Levels of Teacher Experience. Figure 3. Slopes Predicting Observer Ratings of Social Competence from Teacher Ratings of Social Competence at Three 0 1 2 3 4 5 6 7 8 16.3 Teacher Ratings of Children's Social Competence O b s erve r R a t i n g s of C h ild ren 's S o cial C o mp et en ce High Emotion Beliefs Mean Low Emotion Beliefs Levels of Teacher Emotion Beliefs. 62 63 V. DISCUSSION This study examined agreement between teachers and trained observers in judgments of preschool children?s ER and SC. Assessing teacher agreement with other informants is important for establishing the construct validity of teacher ratings of externalizing, internalizing, and SC. It was expected that teachers and observers would show greater agreement about externalizing than about internalizing, and that teacher characteristics would moderate the degree of teacher-observer agreement. Some support was found for each of these expectations. Teachers and observers agreed more about children?s externalizing behavior than about their internalizing behavior. Teachers with more developmentally appropriate beliefs about emotions agreed more highly with observers on ratings of children?s SC. In contrast, teachers with less experience and teachers with fewer continuing education credits agreed more highly with observers regarding children?s externalizing symptoms. Like the findings from other studies, only modest support was found for the construct validity of teacher or observer ratings of social behavioral (Bishop, Spence, & McDonald, 2003) and emotion-relevant (El-Sheikh, 2001) aspects of children?s functioning. As expected, cross-informant cross-construct correlations, such as between teacher-rated internalizing and observer-rated internalizing, were of lowest magnitude. Although teachers and observers agreed moderately on ratings of internalizing, 64 externalizing, and SC, all within-informant correlations were significant, and often of higher magnitude than were within-construct correlations. This was especially the case for within-informant correlations between externalizing and SC. The high correlations suggest that neither teachers nor observers clearly distinguished between externalizing and SC. This interpretation is consistent with previous research showing that observers view children who display externalizing behaviors as less socially competent (Eisenberg et al., 1993; Murphy et al., 2004). Perhaps, teachers and observers in the present study view SC as simply the lack of disruptive behavior problems. The inability to discriminate clearly between externalizing and social competence reflects broader concerns about the validity of the emotion regulation construct. Supposedly, social competence is influenced by emotion regulation, but reflects other factors as well, such as social knowledge, empathy, and social motivation (Bauminger, Edelsztein & Morash, 2005). Likewise, emotion regulation is manifested in many ways that are non-social (El-Sheikh, 2001; Gotman & Katz, 1989). Clarifying the relation and distinction among emotion regulation, externalizing problems, and social competence will require the use of multiple methods, including physiological assessments during both social and non-social tasks. In the present study, teachers and observers agreed moderately on children?s internalizing, externalizing, and SC. As expected, though, there was significantly higher agreement on externalizing than on internalizing. This is consistent with previous studies that have shown that teachers have an easier time agreeing on externalizing behaviors than they do agreeing on internalizing behaviors (Eisenberg, Gershoff et al., 2001; Grietens et al., 2004). As mentioned above, externalizing behaviors are more likely to 65 be disruptive to the daily routine of teachers and children, making them more obvious to teachers and observers. It may also be that teachers have less training in the recognition of internalizing behaviors. However, even though internalizing behaviors may not be as disruptive, they are just as important to notice (Epkins, 1993). Although probably less familiar to teachers than is research on links between early externalizing and later problem behavior, studies of young children?s internalizing also show links to later adjustment problems. For instance, children in preschool who are withdrawn (Harrist et al., 1997) or who show internalizing problems (Mesman, Bongers, Koot, 2001) are more likely to show internalizing problems in later childhood. Teacher education focusing on the manifestations and consequences of early internalizing might help preschool teachers to better recognize these behaviors. Teachers in the present study rated children as having higher internalizing and externalizing problems than did observers; however, teachers also rated children as being more socially competent than did observers. Even though observers in the current study spent 10 hours in each classroom observing prior to completing ratings, more than observers in most other studies (e.g., Coplan et al., 2001; Denham et al., 2003), teachers spend even more time with children. Teachers probably see more instances of acting out and withdrawing in the children, but they also probably see more instances of socially competent behavior, such as helping friends, asking others to play, and showing empathy. Teachers also have personal relationships with children, unlike observers. Thus, teachers probably have more complex views of children in their care that are influenced by perceptions of children?s interactions with peers and by their personal experiences interacting with children, and that include evaluations of children?s emotional intensity, 66 ability to cope, and ability to regulate attention (Eisenberg et al., 1993). This more complex picture likely makes their perceptions of children?s internalizing, externalizing, and SC different from those of observers who view the child only on a limited basis. This study is among the first to investigate perceptions of children as a function of teachers? characteristics. Children in classrooms of teachers with higher education scores and teachers with higher scores for developmentally appropriate emotion beliefs were rated by observers as having fewer internalizing problems than children in classrooms where teachers had lower education scores and less developmentally appropriate emotion beliefs. Additionally, children in classrooms where the teacher had more education were judged as being more socially competent. The fact that emotion beliefs were correlated with education, and that emotion beliefs were no longer significant predictors of observer ratings of internalizing after education was controlled, suggests that teacher education in child development may instill more developmentally appropriate beliefs about children?s emotions in teachers. It previously has been found that education that focuses on child development increases teachers? social interactions with children, and increases children?s social interactions (Travers, 1981). Teachers with education in child development may more readily recognize children who are reticent or anxious and support their attempts to engage socially with peers. Other research has shown that teachers who feel more positively about children have children in their classrooms who are viewed as more socially competent both by the observer and by the teacher (Stuhlman & Pianta, 2001). Higher education may give teachers tools for understanding children, and also may act as a selection factor, such that people who enjoy children are more likely to obtain a high level of education in a child-related field. Important in interpreting 67 these data is the definition of education used in this study. Education was operationalized as the product of the area of study, with content in child development receiving the highest ratings, multiplied by years of formal schooling. Teachers? experience score also affected observer ratings of internalizing. Observers saw children in classrooms where teachers had more years of experience as having more internalizing problems. This may reflect the time at which teachers with more experience received their education (longer in the past than less experienced teachers). Child development training in recent years has been influenced by research in ER and by publication of books on emotional competence (Philippot, 2004; Saarnni, 1999), which could influence teachers? beliefs and practices. Since the ideas of ER are relatively new, teachers who have been in the field for a greater number of years may have less education on how to deal with children?s internalizing problems and how to encourage children to participate in the classroom. Finally, observers? scores of children?s ER were influenced by teacher training. Children in classrooms where the teacher had more continuing education training were seen as having fewer externalizing problems than were children who had teachers with fewer hours of training. Training sessions have been found to give teachers a sense of control over situations they previously believed they had no control over, confidence to try new strategies, and also to provide teachers with an opportunity to discuss their feelings (Sparks, 1988). In today?s education climate, a great deal of emphasis is being placed on reducing children?s externalizing behaviors and teachers are generally eager to learn about how to handle children who are displaying externalizing problems in their classrooms (E. Abell, personal communication, June 29, 2005). These same teachers 68 worry and inquire less about children who display internalizing characteristics (E. Abell, personal communication, June 29, 2005). When these topics arise at training sessions, teachers are allowed to discuss and evaluate ways to help children who may be showing externalizing behaviors in their classrooms, which gives them new ideas on how to handle externalizing and, to a lesser extent, internalizing behaviors. Teacher characteristics not only affected observer ratings of children, these characteristics also affected teachers? own ratings of children?s ER. Teachers with more education and teachers with more developmentally appropriate emotion belief scores saw children in their classrooms as being less withdrawn. However, because of shared variance between education and beliefs, beliefs no longer predicted teacher ratings of internalizing after controlling for education. Thus, it is reasonable to think that perhaps education in fields that focus on child development influences teachers? beliefs about emotion, which then influences teachers? practices and their perceptions of children (Travers, 1981). In contrast, those teachers who had more continuing education training and those teachers with more experience saw children as having more internalizing problems. As mentioned above, training sessions often focus on emotions (E. Abell, personal communication, June 29, 2005). Even though it is usually externalizing behaviors that are focused on, an increased awareness of emotions in general may result from training sessions. Training could lead to an increased awareness of a range of emotional expressions, and teachers may become more tuned in to the more hard-to-detect internalizing behaviors. Experience may influence teachers? perceptions of internalizing in a similar way. After observing a great number of children, teachers may become more 69 aware of subtle behaviors in children. Alternatively, teachers who have been teaching for many years could experience greater levels of depression, which would influence their perceptions of children. Indeed, depressed individuals, including caregivers, have more negative or pessimistic perceptions of others (Azria, 1999; Geller & Johnston, 1995). Either sensitization from experience or pessimism could lead teachers to notice a greater number of behaviors that could be classified as withdrawn, or internalizing. A main research question for the current study was whether teacher characteristics had an effect on the concurrence of teacher and observer reports of children?s ER. A series of moderated regression analyses indicated that experience and training moderated the agreement between teachers and observers on externalizing, and that emotion beliefs moderated the agreement between teachers and observers on SC. Follow-up analyses of the slopes at different levels of the teacher characteristics and inspection of bivariate correlations at high and low levels of the three teacher characteristics were used to explore the nature of the interactions. Teachers who had less experience and less training agreed more highly with observers on children?s externalizing behaviors. This finding could reflect similar perspectives held by students with relatively little (or no) teaching experience and teachers with little experience. Lack of experience could lead the relatively inexperienced in both sets of informants to view some behaviors as evidence of externalizing problems that teachers with more experience and more training might view as within the normal range of children?s behaviors. Finally, results showed that teachers with more developmentally appropriate emotion beliefs were in higher agreement with observers? ratings of SC than were teachers with less developmentally appropriate emotion beliefs. Observers in this study had all elected to study child development, and 70 may (as a result of training or self selection factors) share views about children that are similar to those held by teachers who reported developmentally appropriate emotion beliefs. Unfortunately, no assessments of observer beliefs were available in this study, so it was not possible to investigate teacher-observer agreement on children?s behavior as a function of the concordance of teacher-observer beliefs. Implications. According to recent findings on teacher opinions, social and emotional competence is of greater importance to teachers than the traditional ?abcs? and ?123s? that many people believe children, in order to be successful, should be taught before school entry (Blair, 2002; Lewit & Baker, 1995; Stipek et al., 1998). While some people may fear that programs focusing on social development might cause a delay in children?s academic development, it has been found that programs that are child centered help children to become more socially competent and do not create a delay in basic skills knowledge (Raver & Zigler, 2004). Programs that emphasize ?basic skills? and are teacher directed do not create greater gains on test scores for kindergarten than preschool programs that are more child-centered (Stipek et al., 1998). Children?s school entry behaviors have been found to be the most significant predictor of behavior at the end of first grade, with negative behaviors especially influencing subsequent child behavior ratings (Hogulnd & Leadbeater, 2004). In addition, negative emotionality predicts teacher reports of SC concurrently and as many as six years in the future (Murphy et al., 2004). These findings point to the importance of programs that focus on social-emotional competencies for children in preschools. 71 For children who are already exhibiting social-emotional problems, preschool may be an optimal time for intervention (Bierman & Erath, 2004). Other than parents, preschool and daycare teachers are often the only adults who come into contact regularly with young children. As such, preschool teachers are probably in the best position to identify early emotional problems. However, it is not known exactly how well preschool teachers are able to screen the children in their care for early social and emotional problems. Given that early interventions have been found to be successful for helping children, particularly children with the greatest need (King & Kirscenbaum, 1990; Webster-Stratton, 1988), become more emotionally and socially adept (Denham & Burton, 1996), it is important to evaluate whether teachers can or cannot accurately identify children?s social-emotional competence. The findings of the present study bear on issues of promoting ER and SC in a number of ways. Based on observer judgments, we can conclude that children of teachers who had more education in child development had fewer internalizing problems and were more socially competent. In line with previous studies (Travers, 1981), more teacher education in child development was associated with more positive outcomes for children in social and emotional functioning. The influence of education may be at least partially a function of beliefs that more highly educated teachers hold about children?s social development (Travers, 1981). Training may also have influenced children?s ER, at least in regards to externalizing behaviors. Currently, many continuing education training sessions focus on externalizing behaviors more than on internalizing behaviors (E. Abell, personal communication, June 29, 2005), which may give teachers a wider range of ways to approach externalizing behaviors. Greater experience, on the other hand, 72 was associated with more, not fewer, internalizing symptoms. These findings provide additional evidence for the importance of training in child development. Currently, many states, including Alabama, require only a high school education to teach in a licensed preschool (Barnett, 2004). Many states and certification programs encourage or require continuing education classes in an effort to increase the quality of preschool education. Data reported here suggest, however, that continuing education training may not substitute for more intense education in child development. Whether the somewhat stronger pattern of associations between positive child outcomes and formal training in child development reflects an effect of the education per se or selection effects is unclear. That is, teachers who pursue higher education in child development may already hold attitudes and skills that promote children?s social and emotional competence. Alternatively, greater knowledge of children?s development acquired through university education may yield more appropriate practices that facilitate social and emotional competence among young children. It is also possible that continuing education training sessions are not focusing on emotion-relevant information. If this is the case, continuing education classes that focus specifically on children?s emotions might increase teachers? developmentally appropriate beliefs, which might contribute to teachers? ability to identify children with emotional problems and also teachers? ability to foster social and emotional competence. States could encourage teachers to take continuing education classes that focus on children?s social-emotional competence. Limitations One of the more serious limitations of the current study is that we were unable to demonstrate high levels of inter-observer agreement. Independent live coding of behavior 73 is considered the gold standard in behavioral research, and we have interpreted the data presented here as if our observers provided more accurate judgments than did teachers. However, we can?t be sure that this is so because we have no reliability data for the observers? ratings. We attempted to conduct reliability checks of observers? rating using undergraduates working for course credit, but the reliability data turned out to be spotty, and of dubious reliability. Future work with this data set will involve comparing observers? ratings with time-sampling observational records made by another team of observers and with children?s behaviors in contrived situations designed to test their emotion regulation abilities. 74 REFERENCES Achenbach, T.M., McConaughy, S.H., & Howell, C.T. (1987). Child/Adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213-232. Anthony, B.J., Anthony, L.G., Morrel, T.M., & Acosta, M. (2005). Evidence for social and behavior problems in low-income, urban preschoolers: Effects of site, classroom, and teacher. Journal of Youth and Adolescence, 34, 31-39. Auger, R.W. (2004). The accuracy of teacher reports in the identification of middle school students with depressive symptomatology. Psychology in the Schools, 4, 379-389. Azria, M.R. (1999). Moderators of mother-teacher disagreement on reports of young children?s behavior. Unpublished doctoral dissertation, Auburn University. Barnett, S.W. (2004). Better teachers, better preschools: Student achievement linked to teacher qualifications. [Brochure]. Retrieved online June 30, 2005. nieer.org/resources/factsheets/2.pdf Bates, J.E. (1994). Parents as scientific observers of their children?s development. In Friedman, S. L., Haywood, H. C., (Eds.), Developmental follow-up: Concepts, domains, and methods. (pp.197-215). San Diego: Academic Press. 75 Bauminger, N., Edelsztein, H.S., & Morash, J. (2005). Social information processing and emotional understanding in children with LD. Journal of Learning Disabilities, 38, 45-61. Bierman, K.L., & Erath, S.A (2004). Prevention and intervention programs promoting positive peer relations in early childhood. Tremblay, R.E., Barr, R.G., Peters, RdeV, (Eds), Encyclopedia on Early Childhood Development [online]. Montreal, Quebec: Center of Excellence for Early Childhood Development, 2004: 1-5. Avaliable at http://www.excellence-earlychildhood.ca/documents/Bierman- ErathANGex.pdf. Accessed July 4, 2005. Bishop, G., Spence, S.H., McDonald, C. (2003). Can parents and teachers provide a reliable and valid report of behavioral inhibition? Child Development, 74, 1899- 1917. Blair, C. (2002). School readiness: Integrating cognition and emotion in neurobiological conceptualization of children?s functioning at school entry. American Psychologist, 57, 111-127. Bradley, S. J. (2000). Affect regulation and the development of psychopathology. Guliford Press: New York. Bridges, L.J., Denham, S.A., & Ganiban, J.M. (2004). Definitional issues in emotion regulation research. Child Development, 75, 340-345. Campbell, D.T., & Fiske, D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105. 76 Cole, P.M., Martin, S.E., & Dennis, T.A. (2004). Emotion regulation as a scientific construct: Methodological challenges and directions for child development research. Child Development, 75, 317-333. Cole, P.M., Zahn-Waxler, C., Fox, N.A., Usher, B.A. & Welsh, J.D. (1996). Individual differences in emotion regulation and behavior problems in preschool children. Journal of Abnormal Psychology, 105, 518-529. Cole, P. M. (1986). Children?s spontaneous control of facial expression. Child Development, 57, 1309-1321. Coplan, R.J., Gavinski-Molina, M., Lagac?-S?guin, D.G., & Wichmann, C. (2001). When girls versus boys play alone: Nonsocial play and adjustment in kindergarten. Developmental Psychology, 37, 464-474. Denham, S.A., Blair, K.A., DeMulder, E., Levitas, J., Sawyer, K., Auerbach-Major, S., Queenan, P. (2003). Preschool emotional competence: Pathway to social competence? Child Development, 74, 238-256. Denham, S.A., & Burton, R. (1996). A social-emotional intervention for at-risk four year-olds. Journal of School Psychology, 34, 225-245. Eisenberg, N., Cumberland, A., Spinrad, T.L., Fabes, R.A., Shepard, S.A., Reiser, M., Murphy, B.C., Losoya, S.H., & Guthrie, I.K. (2001). The relations of regulation and emotionality to children?s externalizing and internalizing problem behavior. Child Development, 72, 1112-1134. Eisenberg, N., Fabes, R.A., Bernzweig, J., Karbon, M., Poulin, R., & Hanish, L. (1993). The relations of emotionality and regulation to preschoolers? social skills and sociometric status. Child Development, 64, 1418-1438. 77 Eisenberg, N., Gershoff, E.T., Fabes, R.A., Shepard, S.A., Cumberland, A.J., Losoya, S.L., Guthrie, I.K., Murphy, B.C. (2001). Mothers? emotional expressivity and children?s behavior problems and social competence: Mediation through children?s regulation. Developmental Psychology, 37, 475-490. Eisenberg, N., Guthrie, I.K., Fabes, R.A., Reiser, M., Murphy, B.C., Holgren, R., Maszk, P., & Losoya, S. (1997). The relations of regulation and emotionality to resiliency and competent social functioning in elementary school children. ChildDevelopment, 68, 295-311. Eisenberg, N., & Spinrad, T.L. (2004). Emotion-related regulation: Sharpening the definition. Child Development, 75, 334-339. El-Sheikh, M. (2001). Parental drinking problems and children?s adjustment: Vagal regulation and emotional reactivity as pathways and moderators of risk. Journal of Abnormal Psychology, 110, 499-515. El-Sheikh, M., Harger, J., & Whitson, S.M. (2001). Exposure to interparental conflict and children?s adjustment and physical health: The moderating role of vagal tone. Child Development, 72, 1617-1636. Entwisle, D. R. & Astone, N. (1994). Some practical guidelines for measuring youth's race/ethnicity and socioeconomic status. Child Development, 65, 1521-1540. Epkins, C.C. (1993). A preliminary comparison of teacher ratings and child self-report of depression, anxiety, and aggression in inpatient and elementary school samples. Journal of Abnormal Child Psychology, 21, 649-661. 78 Feldman, R., & Klein, P.S. (2003). Toddler?s self-regulated compliance to mothers, caregivers, and fathers: Implications for theories of socialization. Developmental Psychology, 39, 680-692. Geller, J., & Johnston, C. (1995). Depressed mood and child conduct problems: Relationships to mothers? attributions for their own and their children?s experiences. Child and Family Behavior Therapy, 19, 1-13. Gottman, J.M. & Katz, L.F. (2002). Children?s emotional reactions to stressful parent- child interactions: The link between emotion regulation and vagal tone. Emotions and the Family, 34, 265-283. Gottman, J.M. & Katz, L.F. (1989). Effects of marital discord on young children?s peer interaction and health. Developmental Psychology, 25, 373-381. Gretarsson, S.J., & Gelfand, D.M. (1988). Mothers? attributions regarding their children?s social behavior and personality characteristics. Developmental Psychology, 24, 264-269. Grietens, H., Onghena, P., Prinzie, P., Gadeyne, E., Van Assche, V., Ghesquiere, P., & Hellinckx, W. (2004). Comparison of mothers?, fathers?, and teachers? reports on problem behavior in 5- to 6- year-old children. Journal of Psychopathology and Behavioral Assessment, 26, 137-146. Harrist, A.W., Zaia, A.F., Bates, J.E. (1997). Subtypes of social withdrawal in early childhood: Sociometric status and social-cognitive differences across four years. Child Development, 68, 140-163. 79 Hoglund, W.L., & Leadbeater, B.J. (2004). The effects of family, school, and classroom ecologies on changes in children?s social competence and emotional and behavioral problems in first grade. Developmental Psychology, 40, 533-544. Hyson, M.C. & Lee, K.M. (1996). Assessing early childhood teachers? beliefs about emotions: Content, contexts, and implications for practice. Early Education and Development, 7, 59-78. Jaccard, J., Turrisi, R., & Wan, C.K. (1990). Interaction effects in multiple regression. Newbury Park, CA: Sage Publications. King, C. M., & Kirschenbaum, D.S. (1990). An experimental evaluation of a school- based program for children at risk: Wisconsin Early Intervention. Journal Of Community Psychology, 18, 167-177. Kopp, C. B. (1989). Regulation of distress and negative emotions: A developmental view. Developmental Psychology, 25, 343-354. La Freniere, P.J., Dumas, J.E., Capuano, F. & Dubeau, D. (1992). Development And validation of the preschool socioaffective profile. Psychological Assessment, 8, 442-450. LeDoux, J. (2002). Synaptic self. Penguin books: New York. Lewit, E.M., & Baker, L.S. (1995). School Readiness. The Future of Children, 5, 128- 139. Lin, H., Lawrence, F.R., & Gorrell, J. (2003). Kindergarten teachers? views of children?s readiness for school. Early Childhood Research Quarterly, 18, 225-237. Lindsey, E. W., & Colwell, M. J. (2003). Preschoolers' emotional competence: Links to pretend and physical play. Child Study Journal, 33, 39-52. 80 Lindsey, E.W., Mize, J. (2000). Parent-child physical and pretense play: Links to children?s social competence. Merrill-Palmer Quarterly, 46, 565-591. Mesman, J., Bongers, I.L., Koot, H.M. (2001). Preschool developmental pathways to preadolescent internalizing and externalizing problems. Journal of Child Psychology and Psychiatry, 42, 179-192. Murphy, B.C., Shepard, S.A., Eisenberg, N., & Fabes, R.A. (2004). Concurrent and across time prediction of young adolescents? social functioning: The role of emotionality and regulation. Social Development,13, 56-86. Phares, V., Compas, B.E., & Howell, D.C. (1989). Perspectives on child behavior problems: Comparisons of children?s self-reports with parent and teacher reports. Psychological Assessment, 1, 68-71. Philippot, P. (2004) The regulation of emotion. R.S. Feldman (Ed.), Mahwah, NJ: Lawrence Erlbaum. Raver, C.C., & Zigler, E.F. (2004). Another step back? Assessing readiness in Head Start. Young Children, 59, 58-63. Roberts, W., & Strayer, J. (1987). Parent?s responses to the emotional distress of their Children: Relations with children?s competence. Developmental Psychology, 23, 415-422. Saarnni, C. (1999). The development of emotional competence. New York: Guilford Press. Sparks, G.M. (1988). Teachers? attitudes toward change and subsequent improvements in classroom teaching. Journal of Educational Psychology, 80, 111-117. 81 Stipek, D.J., Feiler, R., Byler, P., Ryan, R., Milburn, S. & Salmon, J.M. (1998). Good beginnings: What difference does the program make in preparing young children for school? Journal of Applied Developmental Psychology, 19, 41-66. Stuhlman, M.W. & Pianta, R.C. (2001). Teachers? narratives about their relationships with children: Associations with behavior in classrooms. School Psychology Review, 31, 148-163. Travers, J.R. (1981). Federal regulations and the lives of children in day care. Prevention in Human Services, 1, 59-86. Webster-Stratton, C. (1998). Preventing conduct problems in Head Start Children: Strengthening parenting competencies. Journal of Consulting and Clinical Psychology, 66, 715-730. Wesley, P.W. & Buysse, V. (2003). Making meaning of school readiness in schools and communities. Early Childhood Research Quarterly, 18, 351-375. Zeman, J., & Penza, S. (1997). Preschoolers as functionalists: The impact of social context on emotion regulation. Child Study Journal, 27, 41-68. 82 APPENDICES 83 APPENDIX A IRB APPROVAL FORM 84 85 APPENDIX B SOCIO-AFFECTIVE PROFILE 86 87 88 APPENDIX C SCALES USED FOR THE SOCIO-AFFECTIVE PROFILE 89 Scales Used for the Socio-Affective Profile The scale for teacher and observer rated externalizing was comprised of the following questions: 1, 2, 6, 7, 8, 9, 12, 14, 21, and 23. The scale for teacher and observer rated internalizing was comprised of the following questions: 3, 4, 5, 10, 11, 16, 18, 25, 28, and 29. The scale for teacher and observer rated social competence was comprised of the following questions: 13, 15, 17, 19, 20, 22, 24, 26, 27, and 30. 90 APPENDIX D ABOUT YOU AND YOUR OPINIONS AS A TEACHER 91 92 93 94 95 96 97 98 99 100 APPENDIX E LIST OF TEACHERS WITH EACH LEVEL OF EDUCATION 101 Number of Teachers in the Current Sample with Each Level of Education Level of Education Obtained Number of Teachers High School 11 Other 3 CDA 20 1-year Child Development Certificate 3 Associates Degree 2 Bachelors Degree 10 Graduate Degree 1