ASSESSING SERVICE-LEARNING IN HIGHER EDUCATION: A CONSTRUCT VALIDATION STUDY Except where reference is made to the work of others, the work described in this dissertation is my own or was done in the collaboration with my advisory committee. This dissertation does not include proprietary or classified information. __________________________________ Shu-Ching Wang Certificate of Approval: ________________________ __________________________ Sean Forbes Margaret E. Ross, Chair Associate Professor Associate Professor Educational Foundations, Educational Foundations, Leadership, and Technology Leadership, and Technology ________________________ _________________________ David Shannon Paris Strom Professor Associate Professor Educational Foundations, Educational Foundations, Leadership, and Technology Leadership, and Technology _________________________ Joe F. Pittman Interim Dean Graduate School ASSESSING SERVICE-LEARNING IN HIGHER EDUCATION A CONSTRUCT VALIDATION STUDY Shu-Ching Wang A Dissertation Submitted to the Graduate Faculty of Auburn University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Auburn, Alabama August 4, 2007 ASSESSING SERVICE-LEARNING IN HIGHER EDUCATION A CONSTRUCT VALIDATION STUDY Shu-Ching Wang Permission is granted to Auburn University to make copies of this dissertation at its discretion, upon request of individuals or institutions at their expense. The author reserves all publication rights. __________________________ Signature of Author _________________________ Date of Graduation iii VITA Shu-Ching Wang, the daughter of Sheng-Hsiung Wang and Yuh -Mei Guo, was born on July 8, 1966, in Cha-Yi County, Taiwan. She graduated from the Cha-Yi Teachers College in 1986, and then taught in elementary schools in Taiwan for twelve years. She earned her Bachelor of Art degree in May, 1991 from Tunghai University, Taiwan. She entered the Auburn University in August, 2000, and earned her Master of Education degree in School Counseling from Auburn University in 2002. iv DISSERTATION ABSTRACT ASSESSING SERVICE-LEARNING PROGRAMS IN HIGHER EDUCATION: A CONSTRUCT VALIDATION STUDY Shu-Ching Wang Doctor of Philosophy, August 4, 2007 (M.Ed., Auburn University, 2003) (B.S. Tunghai University, 1991) 143Typed Pages Directed by Margaret Ross This was a two-phase study that aimed at re-examining the definition of service- learning and the development of an instrument for evaluating service-learning programs in higher education settings. A Delphi study was designed to achieve research objectives of the first stage. The objectives of the Delphi study were to re-visit the definition of service-leaning and to develop an instrument which operationalized the definition of service-learning. Eight (8) v reviewers of the Michigan Journal of Community Service-Learning participated in this study. At the end of this stage, a representative definition of service-learning and an instrument were developed. The objective of the second stage of this study was to examine the psychometric property of a newly developed instrument, Assessing Service-Learning Program (ASLP). An on-line survey study was designed for this stage. After an extensive literature review of service-learning, a hypothesized model of service-learning was generated for this stage as well. There were 248 participants included in this study. A Confirmatory Factor Analysis (CFA) using AMOS 7.0 was performed. Absolute Model Fit Indexes including Chi-Square statistics, Goodness-of-Fit Index (GFI), Comparative Fit index (CFI), and Root Mean Square Error of Approximation (RMSEA) were utilized to investigate the adequacy of the research results. The result indicated that the hypothesized model explained the internal structure of the data adequately. vi ACKNOWLEDGEMENTS To all of my wonderful committee members at Auburn University: this dissertation could not be accomplished without your assistance and guidance. Dr. Margaret Ross, the chair my dissertation committee: Your infinite patience and support in my dissertation writing process enriched my professional and personal growth. Dr. David Shannon: your guidance and support throughout my program made my study enjoyable and meaningful. Dr. Sean Forbes: you provided the most critical and timing assistance to me and my family during the most difficult time of my life. I could not continue my study without your unconditional support. Dr. Paris Strom: your belief in me inspired me to believe in myself. I would like to thank Mary Diamond and Anita France. Mary spent countless hours and provided her professional knowledge during the dissertation writing process. Anita?s humor and personal wisdom helped me to pass the time of frustration. My very special thanks are due to my loving husband, Yuh-Jen Guo and my wonderful children, Happy and Emily. I could not accomplish the degree without your endless love and support. vii Style manual or journal used: Publication Manual of the American Psychological Association, 5th Edition. Computer software used: SPSS 15, AMOS 7.0,Windows XP, and Microsoft Word 2003 viii TABLE OF CONTENTS LIST OF TABLES .........................................................................................................xi LIST OF FIGURES.......................................................................................................xii I. INTRODUCTION Background of the Study......................................................................................1 Research Purposes ...............................................................................................5 Significant of the Study .......................................................................................5 Delimitations .......................................................................................................7 Limitations...........................................................................................................7 Definitions of Terms ............................................................................................8 II. LITERATURE REVIEW Service-Learning................................................................................................11 Historical Background .......................................................................................13 Development of Service-Learning......................................................................18 Definitions of service-Learning..........................................................................20 Theoretical Framework of Service-Learning ......................................................27 Research and Influences of Service-Learning .....................................................33 Importance of Current Research.........................................................................42 III. METHODOLOGY Purpose of Study................................................................................................45 Research Design ................................................................................................45 Substantive Stage: Delphi Study........................................................................46 Delphi Study ..........................................................................................46 Delphi panelists......................................................................................48 Instrument ..............................................................................................49 Procedure ..............................................................................................50 Structural Stage: The Survey Study...................................................................52 Hypothesized Model...............................................................................52 Variables ...............................................................................................53 Population ..............................................................................................54 Sampling ................................................................................................ 55 ix Procedure ...............................................................................................56 Participants ............................................................................................57 IV. RESULTS Assessing Service-Learning in Higher Education: A Construct validation Study..................................................................60 V. DISCUSSION OF FINDINGS, CONCLUSIONS AND RECOMMENDATIONS Study Overview ................................................................................................88 Findings.............................................................................................................91 Discussions........................................................................................................92 Limitations of the Research................................................................................94 Recommendations for Future Research ..............................................................95 Summary ..........................................................................................................97 REFERENCES..............................................................................................................99 APPENDICES A: The instrument: ASLP.................................................................................125 B: IRB Approval Form of DELPHI Study........................................................128 C: IRB Approval Form of Survey Study ..........................................................130 x LIST OF TABLES Table 1. Items and Their Corresponding Variables and Factors....................................54 Table 2. General Information of Participant .................................................................58 Table 3. Items and Their Corresponding Variables and Factors ..................................67 Table 4. General Information of Participant ................................................................71 Table 5. Means and Standard Deviations from Round 1 & 2 ........................................74 Table 6. Reliability of Scores of Three scales.............................................................. 76 Table 7. Means, Standard Deviations of Variables and Correlations between Variables...................................................78 Table 8. Results of the Analysis for the Hypothesized Model.......................................79 Table 9. Pattern Coefficients of Variables and their Corresponding Factors .................81 Table 10. Factors and Their Corresponding Variables.................................................. 91 xi LIST OF FIGURES Figure 1. Distinctions among Service Programs............................................................18 Figure 2. The Hypothesized Model of Service-Learning ...............................................53 Figure 3. The Hypothesized Model of Service-Learning ...............................................66 Figure 4. Results of the Confirmatory Factor Analysis..................................................80 Figure 5. The Hypothesized Model of Service-Learning ...............................................89 xii 1 I. INTRODUCTION This was a two-phase study that aimed at identifying a representative definition of service-learning and the development of an instrument for evaluating service-learning programs in higher education settings. Organization of This Document This chapter includes background of this study, research purposes, and significance of this study. The chapter also addresses delimitations, limitations, and definitions of terms. More detailed information regarding this study were further organized into four categories and presented in the following four chapters, Chapter two to Chapter five. Literature related to service-learning and research findings on the involved groups are discussed in Chapter Two. A detailed research design is outlined in Chapter Three. In addition, statistical procedures performed to achieve the research purpose are also addressed. Chapter four consists of two manuscripts. Each manuscript was the product of a stage of this study. Finally, discussions on findings, implication of the research results, as well as limitation and recommendations are presented in Chapter Five. Learning through service has been practiced in American education since 1900 (Dewey, 1938; James, 1910; Kilpatrick, 1918). In recent decades, several organizations 2 such as Peace Corps, Volunteers in Service to America, Job Corps, Youth Serve America, Campus Opportunity Outreach League (COOL), and Campus Compact have been established and thrived. In the 1990s, the National and Community Service Act of 1990 signed by President George H. W. Bush and the National and Community Service Trust Act of 1993 signed by President Clinton evidenced the flourish of service-learning. The term service-learning was introduced in 1967. Since that date researchers and practitioners, alike, have struggled to define just what it is. Initially, service-learning embraced any program utilizing service and learning. Kendall (1990), for example, reported 147 different terminologies used in literature to refer programs involving students? learning and service. Eventually, leaders (Kandall, 1990; Stanton, Giles, & Cruz, 1999) in the field of service-learning identified essential elements of service- learning. A service-learning program, at its minimum, included three groups: Faculty who conduct the program, students who enroll in a class with service-learning and perform service activity in communities, and community agencies where service-learning students are hosted. Researchers also have identified four criteria among the three groups of service-learning: (1) Student learning and development through active participation in thoughtfully organized service experiences that meet actual community needs and that are coordinated in collaboration with the school and community (2) Integration between the students' academic curriculum and work with the community that provides structured time for the student to think, talk, or write about what the student did and saw during service 3 (3) Opportunity for students to use newly acquired skills and knowledge in real-life situations in their own communities; (4) Extension of student learning beyond the classroom and into the community in an effort to foster the development of a sense of caring for others. (National and Community Service Act of 1990). Without meeting these criteria, programs that simply pair students and community concerns do not qualify as service-learning. Recognition of these criteria is important--- not a semantic quibble, for to define any program that places students in communities as service-learning is to make a number of forgone conclusions as to philosophy and structure of an endeavor. Research outcomes Research results suggest positive influences of service-learning programs on all involved parties, namely, students, faculty participants, and community agencies. Research outcomes of students fall into four categories (Waterman, 1997): enhanced learning of course content (Markus et al, 1993; Strange, 2000; McCluskey-Fawcett & Green, 1992), personal development (Marchel, 2003; Eyler & Giles, 1999; Jorge, 2003), the development of civic responsibilities and other citizenship values (Abernathy & Obenchain, 2001; Giles & Eyler, 1994a; Hollies, 2004; Kane &Westheimer, 1996), and student contribution to the community (Giles & Eyler,1994a; Kane & Westheimer, 1996; Kane, Westheimer & Rogers, 2000). The improved student learning provided evidence that the integrating service-learning into course design is helpful to faculty participants in fulfilling responsibilities of both teaching and research (Bringle & Hatcher, 1996; 4 Parker-Gwin & Marbry, 1998), while community partners expressing observable improvement as a result of service-learning partnership (Jorge, 2003; Grey, 2000). Despite the positive outcomes attributed to service-learning and the increasing number of faculty participants, leaders in the field have long expressed concern that some practitioners of service-learning adopt unclear descriptions of the pedagogy (Kendall, 1990; Sheffield, 2005; Zlotkowski, 1995), a condition that gives rise to possible undesired consequences from an inappropriately structured service-learning program. That is, researchers (Holland, 2001; Kraft, 1996; Morton, 1995; O?Byrne, 2001; Troppe, 1996) emphasize that a clear description of service-learning dictates the implementation of service-learning activities and ultimately directs the evaluation on the effectiveness of service-learning programs (Eby, 1998; Pompa, 2002; Zlotkowski, 1995). If unresolved, faculty participants could possibly design and implement service related programs and then name these programs ?service-learning?, if they believed that what they were doing was, indeed, service-learning. Thus, Sheffield (2005) contends, if the definition or the description of service-learning is not identified, service-learning will be ?everything for everybody? and ultimately ?become nothing? (p. 47). Therefore, an up-to-date description of service-learning that includes necessary components of service-learning and a current instrument assessing a service-learning program are beneficial to resolve challenges faculty participants facing in the initiation and evaluation of their service-learning programs. 5 Significance of the Study Given the apparent disconnect between what service-learning should be versus what some practitioners make it to be, it is of interest to investigate whether or not the operationalization of service-learning established in the early 90s has been altered the practice of service-learning to date. Messick (1994) suggested the importance of re- examining a construct periodically to reflect the progress of the evolving human society. Thus, a clear and up-to-date description of service-learning for the design and an instrument for the evaluation of service-learning programs accordingly become important for both sustaining the endeavor of service-learning practitioners and the effectiveness of service-learning. With the up-to-date description of service-learning along with the instrument faculty practitioners will able to not only design a proper service-learning program but also the potential of service-learning will be more likely realized. In responding to these critical issues, this study was aimed at re-examining the description of service-learning and generating a current instrument to assess the inclusiveness of service-learning programs. Research Purposes The purposes of this study were to re-examine if the original depiction of service- learning still hold true today among leaders of service-learning. Or has the variability among practitioner alter the theoretical conception of service-learning? This research design included two stages of a strong program of construct validation study, the substantative stage and the structural stage (Loevinger, 1957; Nunnally, 1978). The objectives of the substantative stage were to re-examine the 6 description of service-learning and to generate an instrument to assess the inclusiveness of a service-learning program accordingly. A Delphi technique was employed to achieve these objectives. The Delphi study consisted of three iterations. A webpage was established as the communication channel between panelists and the researcher. Eight among sixty-eight reviewers listed in the Journal of Michigan Community Service Learning agreed to serve as the panelists in the Delphi study. The panelists were asked to provide their perspectives of service-learning and rate items on a proposed instrument. Their perspectives of service-learning were analyzed using qualitative data analysis approach. A descriptive analysis using SPSS 13.0 was conducted on panelists? ratings of items. Results were posted on the webpage for panelists? references and possible revision of their future responses. The completion of the substantative stage was followed by the structural stage. The objective of the structural stage was to explore the psychometric properties of the instrument developed through the Delphi study. A hypothesized model of service-learning was proposed based on the consensus obtained from the Delphi study and literature of service-learning. An on-line survey study was conducted in this stage. The instrument was posted on-line to gather responses from faculty participants around the nation. Two hundred fifty higher education institutions were identified through a stratified random sampling procedure. Seven hundred eight department heads/deans of schools/coordinators of programs were contacted to disseminate information of this study to their faculty members. As a result, two hundred forty-eight faculty participants responded to the instrument. To explore the psychometric properties of the instrument, a Confirmatory Factor Analysis (CFA) using AMOS 7.0 was employed to examine the 7 consistency between the internal structure of the data and the hypothesized model of service-learning. Delimitations Service-learning has been implemented in educational settings from kindergartens through higher education institutions. It has also involved numerous disciplines and personnel. The complexity and popularity made service-learning a rich and challenging topic of study. Explicitly, it was not easy to study the full scale of service-learning. This study was aimed at the design and evaluation of service-learning programs in higher education institutions within the United States. In addition, the scope of this study was limited to faculty participants of the College of Education only. Limitations There were limitations in this study. These limitations are presented according to the stages of this study. In the substantative stage, the limitation has partially come from the selection of communication methods among the researcher and her panel members. The Delphi study in this stage utilized the Internet and webpage for the panel discussion and information exchange. The limitation of internet communication, compared to face-to-face communication, should be addressed. The quality of communication and discussion might not be the same as that generated through face-to-face contacts. There were limitations in the structural stage. First, this study utilized a voluntary sample. Participants had freedom to decide about participation, despite the random sampling procedure used. Second, the sample for this study was a homogeneous. The 8 participants of this study were only faculty members in the Colleges of Education. The use of a homogeneous sample places a limitation on the study that may affect the generalizability of the research results. Third, this study asked for self-reported information. One of the limitations of self-reported information is social desirability. This limitation must be considered while interpreting the research results. Fourth, the sample size for this study was less than ideal for a Confirmatory Factor Analytic procedure, thus affected the statistical results may have been affected. Definitions of Terms Delphi Study: This technique systematically solicits professional knowledge and judgments from a group of expert ?panel members? regarding a variety of issues through iterations of anonymous discussion. Faculty participant: The instructor or professor who either integrates service- learning activity into his/her course design or currently engages in a service-learning program in a higher education institution is considered the faculty participant in this study. Community partners: Community partners are the agencies or facilities in the society that have cooperative relationships with faculty participants in higher education institution. Hosting students for their placement, supporting students? service-learning assignments, and supervising their activities on site are components of this cooperative relationship. Student participant: A student participant is a student enrolled in a course which contains the service-learning requirements in a higher education institution. 9 Reflection: Learning literature has suggested that students? gains would be more warranted and sustained through verbal discussion or written expression that aimed at the integration of theory learned in the classroom and empirical experience obtained from daily living. Classroom discussions and written assignments that focus on service- learning experiences are two means for reflection on the purpose of service-learning. Reciprocity: Reciprocity in this study indicates the equality of the service- learning partnership between faculty participants and the representatives of community agencies. In a service-learning partnership, faculty participants and their community partners are both benefited and equally responsible for the service-learning partnership. Substantive stage of validation: The substantative stage is first stage of a construct validation study. For a strong validation program, Loevinger (1957) and Nunnally (1978), who reflected on Chronbach and Meehl?s emphasis on the important role that theory plays in the validation process, proposed that the substantative stage involves defining the constituents and the core domains of a theory. Additionally, identifying key concepts for each domain then operationalizing these concepts for observation and measurement is also accomplished in this stage. In brief, the development of a list of concepts, consistent with theory, and an instrument, operationalizing these concepts to directly assess these observable variables or concepts, are developed in this stage. Structural stage of validation: The second stage of a construct validation study proposed by Loevinger (1957) and Nunnally (1978) is the structural stage of validation. The structural stage involves the investigation of the patterns in the relationships among observed variables (identified from the substantative stage) as well as how these observed 10 variables relate to concepts and domains of the theory under study. In short, the structure of a theory would be suggested through statistical procedures, such as internal consistency analysis, factor analysis or item response theory, in this stage. 11 II. LITERATURE REVIEW This review of the current research literature starts with a brief introduction to service-learning which is followed by a discussion of the development of service- learning, including its historical background and its evolutionary process. Research results that have been reported for specific groups are then examined. Service-Learning Service-learning was originally proposed as a way to address specific educational objectives. It has strong historical connections to the American tradition of voluntary community service, as there practices share a common goal of meeting the needs of communities. Voluntary community service has a long history reaching back centuries and found in religious activities around the world (Kraft, 1996). Various forms and types of voluntary community service have traditionally been provided by individuals or organizations to care for community needs. Educational organizations such as schools are organizations that are integral parts of their communities, and thus, share this tradition of providing voluntary service to their host communities (Prentice & Garcia, 2000). In recent decades, educational institutions have increasingly embraced the use of community service as a pedagogical tool in a variety of curriculum designs. The concept of involving students in community service activities in order to achieve specific learning goals has been accepted and adopted at all levels of the educational system, from K-12 to 12 colleges and universities, and involves a wide range of disciplinary fields (Clark, 2002; Kraft & Swandener, 1994; LeSourd, 1997). To distinguish them from the other voluntary service programs in communities, schools initiated voluntary service activities need to be defined. After decades of searching and discussion among educators, service-learning was eventually chosen as the title that represents the voluntary service activities initiated by schools. Since adopting this designation, service-learning has grown to become an important new pedagogical tool in the American education system (Carpenter & Jacobs, 1994). Higher education institutions such as colleges and universities have also embraced and endorsed service-learning. The integration of service-learning activities into the higher education curriculum has further broadened the influence of service- learning in today?s educational system (Jacoby, 1996; Kezar, 2002; Moser & Rogers, 2005). Research into service-learning has focused on investigating its effectiveness and influence since it began to gain popularity as a pedagogical tool in the American higher education system (Layfield & Flagg, 2004). The implementation of service-learning has been observed across a wide range of academic majors and disciplines (Abernathy & Obenchain, 2001; Adler-Kassner, Crooks, & Watters, 1997; Anderson & Guest, 1994; Kirk & Riedle, 2005; McCluskey-Fawcett & Green, 1992; Michalec, 1994; Reifsteck, 2002; Ropers-Huilman, Carwile, & Lima, 2005; Wade, 1997). Educational influences that have been reported include increasing students? social skills and understanding of cultural diversity (Reisfteck, 2002), their understanding of curriculum materials (Eyler, 2000; McCluskey-Fawcett & Green, 1992; Moser & Rogers, 2005; Simons & Cleary, 2005; Vogelgesang & Astin, 2000), their altruism (Marchel, 2003), their personal and 13 interpersonal development (Eyler & Giles, 1999; McCluskey-Fawcett & Green, 1992), and their improved satisfaction with their academic classes (Gallini & Moely, 2003; Giles,& Eyler, 1994a). Unexpectedly, the growing utilization of service-learning has lead to some difficulties in regulating the formats of service-learning activities (Zoltkowski, 1995) and highlighted the lack of a universal definition of service-learning (Sheffield, 2005). In order to understand service-learning and its influences in American higher education, it is necessary to examine its history and development. Historical Background The literature indicates that individuals or groups have provided voluntary service to address the needs of their communities for centuries (Kraft, 1996). Though voluntary community service has long been practiced in American society, the influence of voluntary community service has been minimized by the social and cultural environment in the United States, which generally endorses individualism and autonomy. This cultural value has hindered Americans? enthusiasm for participating in social activities and their willingness to be active in issues concerning community welfare. This indifference with regard to community involvement has been especially obvious in higher education institutions. Research suggests that college students in the twentieth century tended to pursue material needs rather than philosophical needs for their future (Astin, 1996). In the late 1960s, the Southern Regional Educational Board (SREB) started to develop community service programs that placed college students in communities for their internships (Myers-Lipton, 1998). In the1970s and 1980s, several campus organizations were formed to promote college students? community involvement by 14 engaging them in community service programs. The University Year for Action (UYA) in the early 1970s, Campus Opportunity Outreach League (COOL) in the1980s, and Campus Compact in 1985 were all prominent milestones in the endeavor to promote community involvement among college students. One of the main purposes of involving students in their communities was to counter the gradually individualized and materialized generations, according to Myers-Lipton (1998). Congress and the federal government later joined their efforts with these organizations to promote community service. In the 1990s, the National and Community Service Act of 1990 signed by senior President Bush and the National and Community Service Trust Act of 1993 signed by the President Clinton obligated the federal government to fund community service programs (Stukas & Dunlap, 2002). The Commission on National and Community Service and the Corporation for National Service were established following the enactment of these Acts to oversee and fund community service programs such as AmeriCorps and Learn and Serve America (Myers- Lipton, 1998; Stukas & Dunlap, 2002). Efforts and endeavors contributed by government, Congress, universities, and social activists to increase community involvement and service have gradually spread to include higher education institutions and communities. It has been suggested that scholars should examine the increase of individuals? community involvement. Edwards, Mooney and Heald (2001) indicated that the increasing involvement of colleges and universities in community service programs has contributed to the rising level of student involvement in community service activities. Astin (2000) reported that a record 75.3% of college freshmen were involved in volunteer work at some point during the previous year, of whom 51.1% of these students reported 15 that their community service activities were a part of class requirements. The impact and influence of community service have grown to include various dimensions of communities as well. More individuals, societal organizations, and governmental entities have joined the trend for community service to not only educates the coming generation to be active participants of community affairs, but also counteract the negative cultural influences of individualism (Stukas & Dunlap, 2002). Formats of Service-Learning As service-learning activities have gradually gained acceptance and support from individuals and organizations in recent years, the variety of services provided to the community have also become increasingly diverse. The service provided by service- learning programs took place at different levels and in different locations in communities. For example, there were service-learning programs that facilitated the connection of immigrants to the mainstream American society (Jorge, 2003), that provided help to non- profit organizations (Johnson, 2005), that worked in poor neighborhoods to assist underserved populations, and that helped out at animal shelters to relieve staff shortages (McDonald, Caso, & Fugit, 2005). These programs that addressed various issues have clearly illustrated the complexity and diversity of service-learning. They also demonstrated the difficulty of standardizing and defining service-learning. To improve the coherent understanding between researchers and practitioners reminds a critical agenda in the field of service-learning (Scheffield, 2005; Shumer, 1993, Zoltskowski, 1995). Clearly, the variety of service-learning activities increases the complexity of service-learning and adds to the difficulty for scholars in defining it and identifying 16 activities that are eligible for inclusion. Complicating the situation, the concept of delivering service to communities has included multiple terms. Terms, such as service- learning, community service, volunteerism, field education, and internships (Bickford & Reynolds, 2002, p. 230) have been used loosely to describe the practices of providing community service. Some researchers and educators use these terms interchangeably, while others insist that each term describes different activities. Among the multiple types and formats of service that are provided to communities through voluntary action, three types of community service are commonly observed in higher education, namely volunteerism, internship, and service-learning. Though all of them share the principle of providing voluntary service to their communities, there are identifiable differences among them. Volunteerism has been practiced in human society for centuries (Kraft, 1996). Recently, it has become popular on the campuses of American colleges and universities (Astin, 2000; Edward, Mooney, & Heald, 2001). The unique characteristic of volunteerism that differentiates it from other forms of community service is its lack of structured guidelines for conducting service activities. Additionally, links between this type of service activity and educational objectives are not established (Prentice & Garcia, 2000). Volunteerism has no connection to either educational philosophy or pedagogical development. Internships have been included in the curriculum designs of a variety of disciplines for decades (Apolito, 2006). Interns are required to go to community sites related to their major fields of study and provide assistance. Students are supervised in this endeavor by trained faculty members (Katula & Threnhauser, 1999). Internships 17 offer academic and career connections (Furco, 1996; Prentice & Garcia, 2000). Internships serve the purpose of orienting students to their career paths and give students valuable field experiences that are applicable to their curriculum (Apolito, 2006; Prentice & Garcia, 2000). Although internships exhibit certain characteristics of community service, they also serve the function of career orientation and preparation for students (Brown & Roodin, 2001; Prentice & Garcia, 2000; Von Das & Miller, 2002). They do not transform students? educational experiences beyond pre-established pedagogical goals. Service-learning, with its unique characteristics, is distinct from the other two types of community service activities due to its philosophical and pedagogical roots in education. Unlike volunteerism that benefits the community, or internships that focus on students, service-learning establishes a unique reciprocal relationship among students, community, and universities to benefit all involved (Furco, 1996; Long et al., 2001; Marullo & Edwards, 2000; Prentice & Garcia, 2000). To illustrate the differences among the three types of community service activities, Furco (1996) presented a diagram of ?distinctions among service programs? (p. 5) to identify the distinctive features of the three types of service activities (Figure 1) As shown in the diagram, internships and volunteerism belong to the areas of field education and community service, respectively. Service-learning, at the top of the diagram, holds a unique position that combines the best of what both community service and field education can offer. 18 Figure 1 Furco, 1996, p.5 According to McCluskey-Fawcett and Green (1998), enhanced learning in class and improved understanding of social issues were reported by students who participated in service-learning activities. Service-learning has also been reported to deliver beneficial results to participating students, faculty participants, and communities (Prentice & Garcia, 2000). Although perceptions of the structure of service-learning and its practices vary among organizations, scholars, and researchers, the popularity of service-learning in the educational system continues to increase. Development of Service-Learning A variety of service activities named service-learning are practiced in the educational system of the United States. The term ?service learning?, however, was not introduced until 1967 by Robert Sigmon and William Ramsay (Eyler & Giles, 1994a). Notably, the focus put on service or learning was different depending on each individual program. Recognizing different focuses and the connection between service and learning, Sigmon (1994) presented four types of service-learning activities in his book: service-LEARNING, focusing on learning; SERVICE-learning, focusing on service; 19 service learning, with no connection between service and learning; and SERVICE- LEARNING focusing on both and connecting the two. The term ?service-learning? as we know it today signifies that service-learning is expected to be a pedagogical practice and equally address service and learning. In addition, Eyler and Giles (1994a) argued for the indispensable function of the hyphen between service and learning. They asserted that the hyphen symbolizes reflection activities from which service experiences are transformed into knowledge learning (Eyler, 2001; Eyler & Giles, 1994a). Researchers have suggested that the special combination of service and learning, the connection to educational philosophy, and the contribution to pedagogical development, lead to the significance of service-learning in American education (Boyle-Baise & Kilbane, 2000; Lake, 2001; Vogelgesang & Astin, 2000). The mounting popularity and unique pedagogical value of service-learning has attracted researchers? attention. Research projects aimed at investigating students? academic learning have been conducted by researchers from a wide spectrum of academic studies (Eyler & Giles, 1994a). Consequently, a significant number of positive research findings have been reported. Thus, service-learning has now attracted the support of both researchers and educators. History of Service-Learning Service-learning, due to its recent high level of popularity, is now considered to be an important pedagogical tool. The rising visibility of service-learning in higher education institutions reflects its crucial influence on the American education. However, despite its close connection with the long tradition of voluntary community service, service-learning is a relatively new term in the educational system (Eyles & Giles, 20 1994a). The establishment of service-learning as a pedagogical tool in higher education has not been achieved without struggle and effort. According to the literature, education reform and the recognition of community service have been two major contributions that have accelerated the acceptance of service-learning in the education system as a pedagogical tool. Educational reform in higher education has primarily focused on revising the mission of American higher education. The establishment of Land Grant institutions in the mid-nineteenth century further elaborated the educational purposes of higher education (Annotated History of Service Learning, 2006). As a result of this reform, higher education was expected to prepare students not only for fields of academic studies and career preparation, but also to serve their communities. The other important contribution to the development of service-learning was society?s recognition of community service. In the early 1990s, nationally recognized service-learning organizations such as Campus Compact were established on campuses of colleges and universities across the nation (Myers-Lipton, 1998). Later, the U.S. federal government responded to the expanding community service movement by passing several pieces of legislation, including the National and Community Service Act of 1991 and the National and Community Service Trust Act of 1993 (Skutas & Dunlap, 2002). The enactment of federal legislations signified the prevalence and acceptance of service-learning, not only at the local community level but also at the highest level of federal government. Definitions of Service-Learning Service-learning has now made an appearance in curriculum designs across a wide range of academic majors, with an increasing influence on college and university 21 campuses. The concept of service-learning, where community service is blended into course designs to enhance students? learning and meet community needs, is a fundamentally simple idea. However, a definition with a clear description that unifies fundamental components of service-learning is still not up dated (Belbas, Gorak, & Shumer, 1993; Hollis, 2004; Kraft, 1996; Sheffield, 2005). For example, Shumer (1993) conducted a Delphi study in an attempt to identify service activities that would qualify as service-learning, contending that to meet the criteria of service-learning, individuals could not disregard social contexts in which service-learning activities were implemented. He concluded by noting that ?Service-learning still? resist[s] rigid definition and universal understanding? (p. 22). There have been repeated attempts to clearly describe service-learning and multiple definitions of the term can be found in the service-learning literature and related resources. Kendall (1990) reported that nearly 150 definitions were utilized for scholarly research after she concluded her research. Zlotkowski (1995) expresses his concern that there could be some divergence in service-learning practices as a result of these multiple definitions. Sheffield (2005) indicated that ??there simply are too many definitions of what constitutes the service-learning pedagogy --- definitions that are all accepted as valid by the service-learning community? (p.46). Sheffield argued that without a representative definition and ?by being everything for everyone, service-learning is quickly becoming nothing? (p. 47). Evidently, despite a profile of service-learning program and essential criteria repeatedly presented in literature, a congruency between researchers and practitioners is still not occurring. The following are examples that demonstrate the efforts put into diminishing the differences between service-learning 22 experts and service-learning practitioners. All of these examples include a profile of service-learning with three groups and similar essential criteria of service-learning. Two crucial definitions of service-learning are widely cited on numerous online sources posted by the service-learning organizations (Definitions of Service-Learning, 2006; Service Learning, 2006; Service Learning Definitions, 2006; Service-Learning Is ?, 2006). These two definitions were provided in two legal bills, the National Community Service Act of 1990 and the National and Community Service Trust Act of 1993. In the National and Community Service Act of 1990, service-learning was described as a program: A. under which students learn and develop through participation in? thoughtfully organized service experiences that meet actual community needs; B. that is integrated into the students? academic curriculum or provides structured time for a student to think, talk, or write about what the student did and saw during the service activity; C. that provides students with opportunities to use newly acquired skills and knowledge in real-life situations in their own communities; and D. that enhances what is taught in school by extending student learning beyond the classroom and into the community and helps to foster the development of a sense of caring others The National and Community Service Trust Act of 1993 defined service-learning as following: Service-learning means a method under which students or participants learn and develop through active participation in thoughtfully organized service that: is 23 conducted in and meets the needs of a community and is coordinated with an elementary school, secondary school, institution of higher education, [and] or community service program, and with the community; helps foster civic responsibility; is integrated into and enhances the academic curriculum of the students or the educational components of the community service program in which the participants are enrolled; and includes structured time for the students and participants to reflect on the service experiences. These two definitions describe the fundamental concepts of service-learning. It is evident that providing service to the community, enhancing students? classroom learning, and meeting the needs of their communities is important for service-learning. In addition, these two definitions have been endorsed by the Congress, therefore providing a means of regulating governmental funding to support service-learning. In addition to the two governmental definitions, researchers and scholars continue to propose different versions of service-learning definitions. Stanton (1990) suggested that Service-learning appears to be an approach to experiential learning, an expression of values ? service to others, which determines the purpose, nature and process of social and educational exchange between learners (students) and the people they serve, and between experiential education programs and the community organizations with which they work.? (p. 183). Kraft (1996) indicated that the definition provided by the Commission on National and Community Service (CNCS) might be a ?widely accepted definition? (p. 136). The Commission on National and Community Service (1993) published the 24 definition of service-learning in the promotion of community service. Its definition followed exactly the definition provided by the National and Community Service Act of 1990, as discussed above. Mendel-Reyes (1998) provided a shorter definition, indicating that service-learning could be defined as ?a pedagogical model that intentionally integrates academic learning and relevant community service? (p. 34). Definitions of service-learning generally describe the consistent characteristics of service-learning. It is therefore inevitable that these characteristics of service-learning should be investigated. Characteristics of Service-Learning Service-learning has been defined differently according to its functions, roles, and activities within the social contexts where it is implemented. The unique combination of serving community needs and achieving learning objectives distinguishes service- learning from other forms of community service activities and pedagogical practices. However, without structural guidelines, service-learning covers a wide variety of practices in the field of education. Its functions, roles, and activities range from loosely structured forms to theory-driven and well structured programs (Hollis, 2002). It is believed that different fields, motives, and purposes for participating in service-learning have contributed to these divergent definitions of service-learning (Greenberg, 2000), but the lack of standards and means with which to assess service-learning has often resulted in poor designs and executions of service-learning programs. These insufficiencies led to counterproductive influences and the failure to achieve learning objectives in given service-learning programs (Eyler, 2000; Gallini & Moely, 2003). Hence, an up-to-date comprehensive understanding of service-learning has become critical for researchers and educators. This comprehensiveness will benefit the design and implementation of 25 service-learning in colleges and universities. It will also provide necessary measures with which to assess service-learning programs and evaluate their results. In 1989, the Johnson Foundation published the Wingspread Special Report, which presented the following list of principles or criteria for an effective service-learning program: 1. Engages people in responsible and challenging actions for the common good. 2. Provides structured opportunities for people to reflect critically on their service experiences. 3. Articulates clear service and learning goals for everyone involved 4. Allows for those with needs to define those needs. 5. Clarifies the responsibilities of each person and organization involved. 6. Matches service providers and service needs through a process that recognizes changing circumstance. 7. Expects genuine, active, and sustained organizational commitment. 8. Includes training, supervision, monitoring, support, recognition, and evaluation to meet service and learning goals. 9. Insures that the time commitment for service and learning is flexible, appropriate and in the best interest of all involved. 10. Is committed to program participation by and with diverse populations. (pp. 2-3) This list of criteria for service-learning programs clearly addresses the main characteristics of service-learning. Service-learning is not a simple combination of service-providing and curriculum design. On the contrary, service-learning involves 26 multiple actions and efforts of students, faculty, and the community to pursue the ?best interests of all involved? (Johnson Foundation, 1989, p. 3). The National Service-Learning Clearinghouse site (Service-Learning Is?, 2006) provides another important list of the characteristics of service-learning. This list was generated by the National Commission on Service Learning, as quoted on the National Service-Learning Clearinghouse web pages (Service-Learning Is?,). The unique functions and roles of service-learning are described as: service-learning links academic content and standards; service-learning involves young people in helping to determine and meet real needs; it defines community needs; it is reciprocal in nature, benefiting both community and service providers by combining a service experience with a learning experience; it can be used in any subject area so long as it is appropriate to learning goals; and it works at all ages, even among young children. Notably, the site also includes descriptions of what service-learning should not be. Service-learning should not be ?an episodic volunteer program?, add-on to existing curriculum, a graduation requirement or a punishment. Eyler and Giles (1999) also provided their views of characteristics of service- learning. The important characteristics of service-learning addressed in their book include the following: 1. There is an academic connection between service activities and course content. 2. Service-learning is a ?genuine resource useful to the community? 3. Students are able to link what they are doing in the classroom to what they are experiencing in the community and vice versa. 27 4. Service-learning should include reflection activities that tie student experience in the community to academic learning. 5. Service-learning should provide the opportunity to work with people from diverse ethnic groups during the course of their service-learning. 6. Service activities should address the needs identified by members of the community. (p.167 - p.179) These characteristics of service-learning, described and summarized by various organizations and scholars, provide functional definitions that individual service-learning programs should consider as key components to be included in the process and design of service-learning activities and their related curriculums. These characteristics are also effective resources when service-learning is examined for its pedagogical value or compared to existing philosophical notions in education. The following section investigates the theoretical framework of service-learning to better understand the effectiveness of service-learning and its influence on education. Theoretical Framework of Service-Learning Service-learning has been linked to the field of education with the objective of enhancing students? learning. The implementation of service-learning has been observed at various levels and types of educational institutions within a variety of curriculums. Educators and researchers have not only attempted to examine its pedagogical effectiveness, but also to establish its theoretical links to the philosophy of education (Giles & Eyler, 1994b). In the early stage of the development process of service- learning, researchers have noted the absence of a theoretical framework (Giles & Eyler, 28 1994b). Due to his position as one of the most important theoretical and philosophical figures in education, John Dewey?s educational philosophy has attracted a great deal of attention from scholars and researchers who considered it especially appropriate for service-learning (Carver, 1997; Ehrlich, 1998; Giles & Eyler, 1994b), it thus provides an excellent basis for the establishment of the theoretical framework of service-learning (Bringle & Hatcher, 2000b; Carver, 1997; Giles & Eyler, 1994b, Fertman, 1994; Harkavy & Benson, 1998; Hatcher, 1997). John Dewey authored numerous books and articles to address his educational philosophy. Throughout his career he established a philosophical and pedagogical framework of education. Dewey?s educational philosophy relevant to service-learning included experiential education, democratic education, and education for citizenship. These concepts are closely associated with the theoretical framework of service-learning (Giles & Eyler, 1994b; Kraft, 1996). In the following section, an exploration of the literature on service-learning in relation to John Dewey?s educational philosophy and its influences on the practices of service-learning is provided. Experiential Education and Service-Learning Dewey expressed his strong support of experiential education through his educational philosophy (1938). He believed that students should learn from their real life experiences, and educators should utilize experiential education in their teaching. Dewey emphasized that true learning only occurs when learners are exercising their real world experiences (Harkavy & Benson, 1998). He wrote that ?there is an intimate and necessary relation between the processes of actual experience and education? (Dewey, p. 7), and that real world experiences are necessary and crucial to both learning and 29 education. Dewey further articulated his strong belief in the connection between experience and education, writing: The belief that all genuine education comes about through experience does not mean that all experiences are genuinely or equally educative. Experience and education cannot be directly equated to each other. For some experiences are mis-educative. Any experience is mis-educative that has the effect of arresting or distorting the growth of future experience. (p. 25) One of the important messages conveyed to education by Dewey was that genuine education originates from experience. Effective and educative experiences should not hamper future development and the growth of an individual?s learning. To explicate educative experiences, Dewey proposed two principles, the Principle of Continuity and the Principle of Interaction (Giles & Eyler, 1994b). According to Dewey, meaningful experiences have the characteristics of continuity, in which experiences build upon previous ones and lead to the development of future experiences. In addition, continuity of learning is also sustained by interactions between individuals and their experiences (Giles & Eyler, 1994b). Based on his belief in experiential education, Dewey asserted that real experiences are crucial to education and learning. To illustrate how students transform experiences into learning, Dewey highlighted reflective thinking. He believed that reflective thinking can not only facilitate students? learning and interactions among their experiences, but also transform these interactions into real learning (Giles & Eyler, 1994b). 30 Service-learning embraces a concept similar to Dewey?s experiential education. Service-learning is specifically designed to integrate a service component into curriculum designs. Service activities that students are required to provide to communities serve as students? authentic experiences and are necessary to complete their learning and education processes. In addition, various forms of reflection activities are included in service-learning to assure the transformation of students? service experiences into genuine learning (Eyler and Gelis, 1994a). These reflection activities are consistent with Dewey?s notion of reflective thinking. In the late 1980s, Boyer (1987) described the educational trend of focusing on experiential education. He wrote: The trend is clear. In just a generation, assumptions about time and location of learning that historically have guided undergraduate education have been turned on end. Undergraduate education is beginning to break loose from traditional classroom encounters and even from the notion that all learning must be completed under the formal guidance of a teacher. The nation?s colleges are discovering that the campus is as much a state of mind as a place. It exists, or at least can exist, wherever a student happens to be. (Boyer, 1987, p. 232) Scholars and researchers have applied Dewey?s philosophy of experiential education in the construction of service-learning theory (Giles, 1991; Giles & Eyler, 1994b; Kraft, 1996). Service-learning experiences are naturally continuous and interactive. The critical inclusion of Dewey?s concept of experiential education has formed a solid foundation that supports the future development of service-learning. 31 Democracy and Citizenship Education and Service-Learning In addition to experiential education, Dewey examined the important mission of education. The traditional approach of education honored learning or acquiring knowledge from textbooks or teachers? lectures in classrooms, and focused on preparing students for an unknown future. Unlike the traditional approach, Dewey proposed democratic education, embracing the important connection between education and authentic experience, and contended that education should take a democratic approach to acquaint students with the modern democratic society. Education should seek not only to prepare individuals to function in the community and societies they dwelled in (Giles, 1994b), but should also contribute to the development a democratic society (Anderson & Major, 2001). Democracy is a major component of Dewey?s philosophy, and a major concept in his educational beliefs (Eldridge, 1996). However, he realized that democracy does not necessarily result in a highly functioning society if individuals are not well versed in democratic procedures. According to Dewey, citizens of a democratic society should honor the equality of all. Individuals should be evaluated according to what they do rather than who they are. In addition, individuals? unconditional obedience to authorities is not beneficial for a democratic society. Rather, citizens of a democratic society need to be critical about community affairs. To resolve existing disagreements, in a democratic society citizens need to be able to participate in constructive public debates, which reflect observations and evidence and produce decisions regarding community affairs with which the majority of the citizens agree (Anderson & Major, 2001). Indeed, for Dewey, it was critical and fundamental for individuals to be responsible for the welfare of their 32 communities. He said that ?unless local communal life can be restored, the public cannot? find and identify itself? (1946, p. 216). He also contended that formal education should be responsible for preparing individuals to function in democratic societies, but also for educating them to become engaged citizens (Rhoads, 2000). Sheffield (2004) argued that service-learning facilitates democratic education. After conducting research on 56 students who participated in 13 weeks of service- learning activities, Giles and Eyler (1994a) concluded that service-learning played a critical role in the students? development of citizenship. Chesler (2000) reported that students were able to ascertain that components of the existing social structure contributed to the continuation of social problems. Sapp and Crabtree (2002) claimed that by participating in various types of service-learning activities, students are instilled with an awareness of the responsibility individuals have for the community in which they reside. It has been proposed that service-learning activities in higher education purposefully encourage students to exercise their critical thinking skills, not only to solve the problems they encounter but also to address the root-causes of these problems. Pompa (2002) asserted that service-learning could be a driving force for social change. However, she acknowledged that social change could be realized only if service-learning programs are well designed and implemented. Pompa?s assertion echoed Chesler?s notion of service-learning. Chesler (2000) contended that service-learning does not necessarily lead to social change. Kane and his associates (1996) were also cautious about this indefinite connection, although researchers often (Kane & Westheimer, 1996; Morton, 1995) support the possibility of its ultimate potential of leading to the 33 transformation of society. Morton (1995) identified three types of service-learning activities: charity, project, and change. As he concluded his research, he contended that any type of service-learning activity had the potential to address social injustice and lead to the transformation of a society. Later, Morton (2003) adopted a different approach to categorize a variety of objectives for conducting service-learning activities. He indicated two purposes of conducting service-learning: transactional and transformational. Kane and Westhemier (1996) realized that service-learning programs may be categorized according to their purposes and activities; each program has its own strengths and weaknesses and was possibly able to address social problems, and could lead the way to social change. This ultimate transformation of society echoes Dewey?s vision of a democratic society, in which its citizens are informed, communicate, and participate in decision making processes where the mutual interests within the community are addressed. Research and Influences of Service-Learning In addition to the discussion of the historical background, definitions, and theoretical framework of service-learning in the previous sections, it is important to review research results on the influences of service-learning in higher education. Research provides empirical data on the effectiveness of service-learning by assessing the extent to which relevant areas are improved. Research projects on service-learning have generally focused primarily on the students, especially their learning and development, with service-learning being treated as a pedagogical tool. Research designs used in investigating the influences of service-learning have involved both quantitative and 34 qualitative inquiries. The inclusion of both research methods has allowed researchers to tap into service-learning from different dimensions, thus enhancing the credibility of their research findings. As a result, there is now a more comprehensive understanding of service-learning. Numerous research articles have discussed was in which students were influenced by their service-learning experiences and what they learned from their service- learning activities. Service-learning participants from diverse academic majors have reported positive influences and supported the effectiveness of service-learning experiences (McCluskey-Fawcett, 1992; Myers-Lipton, 1998; Tucker et al., 1998); however, conflicting findings have also been reported (Simmons & Cleary, 2005). Despite the mixed results, the potential of service-learning as a pedagogical tool in higher education is widely recognized. Researchers, scholars, and an increasing number of faculty participants consider it to be both effective and crucial in achieving students? learning objectives. Integrating service-learning into instructional design is continuously and frequently observed in higher education institutions. Service-learning has drawn nationwide attention and attracted scholars from a variety of backgrounds (Zlotkowski, 1995). Scholars have investigated service-learning from various aspects according to their expertise and interests. The important participants in a service-learning program and repeatedly mentioned in publications are the students participating in service-learning programs, the community agencies hosting the service-learning students, and the faculty participants organizing the service-learning programs (Long et al., 2001; Moser & Rogers, 2005; Prentice & Garcia, 2000). To depict and emphasize the significant outcomes of integrating service-learning in higher education, Moser and Rogers (2005) asserted that ?Essentially, service learning involves 35 learning a variety of skills through the act of service. ?It is a powerful teaching strategy that can enrich student learning, enhance your teaching and revitalize your community? (p. 18). Reports in the literature indicate that service-learning has significantly influenced those involved, including faculty participants, students, and communities; however, most researchers did not design their research projects to study the influence of service- learning on the three key groups equally. Although tremendous resources have been devoted to studies investigating the influence and effectiveness of service-learning on students? learning and development, the other two groups, namely the faculty and the community, have generally been overlooked and have received less attention from researchers, despite their importance to service-learning programs. Influences of Service-Learning on Students Research on the influence of service-learning in higher education have predominantly focused on the capacity of service-learning to achieve educational objectives in diverse major study areas, namely, students? learning outcomes. The literature has shown that service-learning does indeed provide students with rich field experiences that facilitate their learning and development. A considerable volume of research results indicate a great enhancement in several areas Waterman (1997) documented four major positive influences that service-learning has on students? learning outcomes. These influences are: 1) enhanced learning of course content, 2) personal development, 3) the development of civic responsibilities and other citizenship values, and, 4) student contribution to the community. Results from research projects of Markus et al (1993), Strange (2000), McCluskey-Fawcett, and Green (1992) evidenced students? 36 positive course learning. Markus et al (1993) reported significantly higher grades for students who participated in service-learning activities in a study of 89 students enrolled in eight sections of political science classes. Strange (2000) studied the class performance of 477 students enrolled in Child Development classes from Spring 1997 to Fall 1999. Her analysis of the data revealed that students who enrolled in classes with service-learning assignments achieved significantly higher scores than students with other assignments, such as observation and essay writing. Similar findings have also been reported in other studies. McCluskey-Fawcett and Green (1992) examined the effect of incorporating service-learning into their developmental psychology classes. They found that students who participated in a service-learning program achieved a better understanding of class materials than those who did not. Tucker and associates (1998) also indicated that service-learning had assisted business students in their learning of communication skills. Johnson (2005) introduced a community food drive project as her service-learning program and reported positive learning experiences for the students. Ropers-Huilman et al (2005) reported that service-learning had been a successful pedagogical tool for achieving established learning objectives. Research results from projects of Marchel (2003), Eyler and Giles (1999) and Jorge (2003) provided evidence of students? personal development. For instance, to promote second language learning, Jorge established service-learning partnership with a Mexican community. She reported improvement of her students? cultural awareness. Research projects of Abernathy and Obenchain (2001), Giles and Eyler (1994a), Hollies (2004), Kane and Westheimer (1996) indicated students? increased civic awareness. Kane and Whestheimer compare service- learning programs adopting different approaches. As a result, they found increased civic 37 awareness and a sense of contributing to communities from students of both groups. Marchel (2003) analyzed service-learning experiences of her two undergraduate classes. She indicated that after overcoming the initial frustration and confusing stage, students experienced a meaningful connection to the community. However, some conflicting findings have been reported. Strange (2004) analyzed the same data from different aspects and reported that students who were assigned service-learning activities earned higher, but not significantly higher grades, after she compared students enrolled in the upper division of Child Development classes. Layfield and Flagg (2004) also found no difference between service-learning and non-service- learning students on subject learning after a study of 35 students enrolled in an Agricultural Applications of Microcomputers class in the Spring 2002. Rukaina and Li (2007) attempted to intervene students? ant-fat bias by integrating service-learning activities. They reported that their students? stereotypes about obesity were not improved after service-learning experiences. Moely et al (2002) studied the influence that service experience in general and service-learning have on improving students? civic awareness. They did not find that service-learning experiences significantly influenced their students? civic awareness. Vogelgesand and Astin (2000) conducted a longitudinal on students from three different types of service experiences. They analyzed data collected from 1991 to 1997 and reported that service-learning experiences did not significantly contribute to students? development of leadership and interpersonal skills. Influences of Service-Learning on Faculty In addition to its influence on students? learning and development, service- learning has impacted faculty participants at higher education institutions. Driscoll 38 (2000) and Sheffield (2004) both highlighted the essential role of faculty participants in a service-learning program. According to them, faculty participants? decisive contributions have been recognized and addressed in the service-learning literature. Faculty members are those in charge of delivering curriculum and designing instructional strategies (Bringle & Hatcher, 1998). By integrating service-learning into course designs, faculty participants are actively engaged in finding appropriate community placements for service-learning activities, designing service-learning activities for students, conducting reflection sessions in class, communicating with community partners on a variety of concerns and issues, and most importantly facilitating the processes of transforming students? experiences into knowledge (Eyler & Giles, 1999; Rubin, 1990). Parker-Gwin and Mabry (1998) also pointed out that the role of faculty participants in service-learning is to direct students? learning and facilitate students? service experiences to encourage their intellectual development. Faculty participants contribute and influence service-learning through their responsibility for curriculum design and class instruction. At the same time, faculty participants are also influenced due to their participation in service-learning. Several benefits for faculty participants have been identified. Faculty participants are likely to broaden their teaching strategies, enjoy teaching more, guide students with positive development, and embrace different teaching styles through their involvement in service- learning (Kendall et al., 1990). Easterling and Rudell (1997) indicated that participating in service-learning is one of the faculty?s important contributions to their colleges and universities. In addition, service-learning experiences also provide faculty members with new perspectives on teaching and research. In spite of these listed benefits, interactions 39 between faculty participants and service-learning programs did not come without a price. For example, Zlotkowski (1995) indicated that under the pressure of tenure and promotion, the amount of time and energy necessary for conducting service-learning programs is a major challenge for faculty participants (Praetzel, 1999; Hammond, 1994). Additionally, McDonald, Caso, and Fugit (2005) reported that locating resources that would be beneficial in designing proper service-learning programs is a common struggle for faculty participants. It is undeniable that it would be impossible to accomplish research without the involvement of faculty members. The involvement of faculty participants in research projects connected with service-learning is evidenced by the huge volume of publications and the increasing popularity of service-learning. However, faculty participants? high level of involvement in service-learning research has focused primarily on students? learning outcomes. Research devoted to faculty participants that focus on their personal and professional circumstances still remains to be done (Driscoll, 2000). Howard et al (2000) emphasized the important role of faculty participants in service-learning and called for more research into the faculty domain of service-learning. Influences of Service-Learning on Community The influence of service-learning on the community comes from the collaboration between community and faculty participants in conducting service-learning programs. Gray et al (2000) illustrated the important contributions of the community to service- learning and asserted that ?Community support is essential to the long-term success and stability of these programs? (p. 39). Service-learning honors the reciprocal partnership between faculty and community. Thus, faculty participants should focus not only on the 40 enhancement of students? learning, but also on satisfying community needs (Gray et al, 2000). By the same token, service-learning emphasizes both service and learning. Stanton (1990) described the essential perspective and stated that ?Service, combined with learning, adds value to each and transforms both? (p. 1). The reciprocity between service and learning is an essential element of service-learning that makes service- learning unique in experiential education (Cruz & Gillis, 2000; Jorge, 2003). In addition, community empowerment is a critical objective of involving the community in service- learning activities. Research indicates that to empower a community, the most important condition for fostering a sense of empowerment is benefiting the community partner through the partnership (Carpenter & Jacob, 1994; Valerius & Hamilton, 2001). In the research project ?Learn and Serve America, Higher Education? Gray et al (2000) assessed service-learning in terms of both the student and the community, concluding that the community did indeed benefit from participating in the service-learning program in numerous areas. Community agencies reported improvements in the quality, quantity, and variety of service they provided. Their reputations were also improved as a direct benefit of participating in the service-learning programs. Jorge (2003) adopted a qualitative research inquiry and designed her research project to primarily investigate the community perspective. This service-learning program involved students who were enrolled in various levels of Spanish classes and a group of Mexican women in a Mexican predominant community. To achieve the goal of improving the language skills of her students and at the same time help the community, Jorge designed service-learning activities with an emphasis on direct interactions between the students and community partners. As a result, Jorge reported numerous unexpected outcomes of this program, 41 including the awareness of diversity, empowerment of community participants, reciprocal relationships, and the importance of the equality between the faculty and community partners. According to her, the appreciation of diversity was found from both the students and the participating Mexican women. Both parties experienced ?a better understanding of such factors as cultural diversity, ethnicity, race, socioeconomic status? in a shared world? (p. 30). The empowerment of community participants was observed from community participants? willingness to take ownership of teaching activities and their enhanced self-esteem, which was the result of their recognized ability to collaborate with professors in the program and being acknowledged as critical contributors to the service-learning program. The reciprocal relationship was shown through the mutual changes and interactions of the service-learning experiences. The equality between the two parties was evidenced by community participants? sense of being ?free to negotiate and share power? (p. 30). Jorge contended that this equality was the forerunner of a reciprocal relationship. According to her, the program was able to encourage free discussion and sharing throughout the partnership, which made the relationship beneficial for all the service-learning participants, including both the students and those in the community. However, several undesired or unexpected situations have been reported among the service-learning partnerships. In addition to positive outcomes Gray et al (2000) reported some concerns of community partners. Agencies whose service programs involved individuals experiencing an emotional crisis expressed their concerns over the short duration of the service-learning programs. These agencies were concerned that service-learning programs were usually relatively short in terms of time and thus could have a negative impact on the existing psychological issues of their 42 clients. Other situations, such as low cost-effectiveness for training service-learning students and scheduling challenges, were also concerns reported by this research. However, Gray et al concluded that the advantages of service-learning partnerships ?far outweighed? (p38) the disadvantages. The community agencies involved were generally pleased by the positive outcomes from participating in service-learning. These research results supported the notion that communities can benefit from students? participation in a variety of service-learning activities (Parker-Gwin & Mabry, 1998). Despite encouraging outcomes, Zlotkowski (1995) expressed his concern and mentioned that without a precise description of the responsibility and obligations of the community agency, service-learning programs would not always realize the true equality between the community agency and its partnered educational institution. This could hamper the quality and results of service-learning. Echoing Zlotkowski?s concern, Cruz and Gillis (2000) called for a better definition of ?community? for service-learning; however, they found little service-learning research attention has been invested in examining the community perspective (Ferrari & Worrall, 2000; Layfield & Flagg, 2004), regardless of its projected critical influence on the effectiveness of the service- learning program (Gray at el, 2000; Steffes, 2004). Importance of Current Research As discussed in the service-learning literature, abundant research data has been collected regarding students? learning, but research efforts have not investigated the perspectives of faculty and community participants of service-learning to the same extent even though the literature stressed that service-learning influences all the participants, 43 namely the students, faculty, and community. However, the inadequate understanding of how service-learning has impacted faculty and community participants has resulted in some misunderstandings concerning service-learning. Faculty participants are involved in every stage of a service-learning program, including planning, implementing, and evaluating the program. In addition to delivering lessons in classes and guiding the reflection activities of their students, they also coordinate and communicate with community partners about service-learning issues. Perspectives from faculty participants are therefore crucial and essential to the investigation of service-learning. Thus, it is important to design a research project to investigate faculty participants? perspectives of engaging with service-learning. Research results from such a research design would provide essential information to assist faculty participants in planning proper service- learning programs, enhancing student learning, and maintaining positive faculty- community partnerships. The current study was designed to develop an instrument to address this issue and validate the scores gathered by the instrument. Data was collected directly from faculty participants? perspectives to reveal their perspective on the three service-learning domains, namely the student, the faculty and the community. The study included a Delphi study that involved a group of service-learning experts in the process of designing an instrument to assess service-learning programs. With this instrument, the researcher then assessed the structure of service-learning programs by surveying faculty participants. The intention of this Delphi study included investigations of the students? development and responsibility, the faculty participants?? involvement in program design and the community connection, and the community?s responses to and satisfaction with 44 service-learning programs. The reciprocity and equality between faculty and community participants were also assessed in the research to investigate the faculty-community partnership. Following the completion of the instrument, a validation process was conducted to assess the consistency between the internal structure of the data and a hypothesized service-learning model. The importance of this research lies in its intention to create a new instrument for faculty participants to assess essential information regarding their service-learning programs. The instrument was designed to include critical components of service- learning and provide necessary information for faculty participants in designing genuine service-learning programs and, eventually, the evaluation of their service-learning programs. With the assistance of this instrument, faculty participants are expected to be able to adjust their program designs and maximize the potential benefits of service- learning for all the participants. Summary This review of literature began with a brief introduction of service-learning. The development of service-learning, including its historical background and its evolutionary process were addressed. Research results on involved groups were examined as well. The next chapter will describe important information related to methodology of this study. 45 III. METHODOLOGY Purpose of Study This study consisted of two phases, the substantative stage and the structural stage, of a construct validation study. There were two objectives in the substantative stage. The first objective was to re-examine the definition of service-learning after more than a decade of service-learning practice. The second objective was to develop an instrument to assess service-learning programs that is consistent with its definition. The objective of the structural stage was to investigate the psychometric properties of the instrument developed from the substantative stage by examining the consistency between a hypothesized service-learning model and the internal structure of the data collected from the instrument developed from the substantative stage. Research Design This study included two of three components that constitute a construct validation study. A strong program of construct validation study includes three components: the substantative component, the structural component, and the external component (Benson, 1998). This study involved the substantative component and structural component. 46 Substantive Stage: Delphi Study The purpose of the substantative stage, as mentioned previously, was to re- examine the definition of service-leaning and to develop an instrument to assess service- learning programs in higher education institutions accordingly. The present study researched knowledge and opinions from experts; thus, Delphi techniques were employed to achieve this research purpose. Delphi Study A Delphi study is known for its systematic approach in collecting professional knowledge and judgments from a group of experts, known as panel members, regarding a variety of issues through iterations of anonymous discussion (Erffmeyer, Erffmeyer, & Lane, 1986; Fisher, 1978, Turoff, 1970). A consensus from panelists is expected from conducting a Delphi study (Woudenberg, 1991). The first Delphi experiment, known as Project Delphi, was designed by Norman Dalkey and Olaf Helmer for military purposes. It was undertaken in 1948 (Erffmeyer, Erffmeyer, & Lane, 1986; Fischer, 1978; Riggs, 1983; Woudenberg, 1991) and conducted by the RAND Corporation in the state of California (Fischer, 1978; Erffmeyer, Erffmeyer, & Lane, 1986; Riggs, 1983). Initially, Delphi methods were designed to avoid undesired effects embedded in the traditional group discussion and decision making process (Riggs, 1983; Rowe, Wright, & McColl, 2005). According to Woudenburg (1991), 14 Delphi research projects were conducted between 1950 and 1963 by the same group and kept secret due to their military bonds. Not until 1963 did the journal of Management Science publish the first Delphi research article, which was authored by Dalkey and Helmer ( Woudenburg, 1991). However, the Delphi techniques 47 claimed worldwide popularity the next year when Gordon and Helmer published their Delphi study (Woudenberg, 1991). Since then, Delphi techniques were applied to a broad range of research agendas (Fischer, 1978; Gordon & Pease, 2006; Woudenberg, 1991), such as forecasting, decision-making, and long-term planning (Erffmeyer, Erffmeyer, & Lane, 1986; Fischer, 1978; Gordon & Pease, 2006). In order to construct a Delphi study, a group of experts is essential. Unlike the traditional interview or focus group designs, the physical attendance of the expert participant is not required to serve as a panel member (Dijk, 1990; Erffmeyer, Erffmeyer, & Lane, 1986; Riggs, 1983). This approach granted panel members more freedom and flexibility to participate in a series of discussion on intended issues without constraints of physical location (Erffmeyer, Erffmeyer, & Lane, 1986). The implementation of mail or electronic communication added even more flexibility to experts with busy schedules (Zolingen & Klaassen, 2003). Anonymity is another essential feature of a Delphi study. The anonymity feature of the Delphi study prevents panelists from knowing the identities of other panel members (Erffmeyer, Erffmeyer, & Lane, 1986). No panel member has information about the membership of the discussion group in which he or she is involved. Dijk (1990) contended that panel member anonymity reduces the tendency of social desirability considerably. Fischer (1978) also supported and praised Delphi methods. He indicated that the combination of both no physical interaction and anonymous features contributed to the achievement of a better conclusion. According to him, under this condition, individual panel members were able to express their opinions independently without the concern of creating argumentative situations when their opinions are 48 conflicting with those of others, being intimidated by more knowledgeable members or influential individuals, dismissing their opinions because of noisy individuals in the group, or concern of losing face for holding alternative opinions (Riggs, 1983; Rowe, Wright, & McColl, 2005, Zolingen & Klaassen, 2003). Noteworthy, despite its broadened popularity, researchers acknowledged various inconsistencies regarding Delphi procedures (Dijk, 1990; Linstone & Turoff, 1975; Turoff, 1970; Erffmeyer, Erffmeyer, & Lane, 1986, Riggs, 1983), and the number of iterations was just one of them (Dijk, 1990; Turoff, 1970; Erffmeyer, Erffmeyer, & Lane, 1986, Riggs, 1983). Erffmeyer, Erffmeyer, and Lane (1986) suggested that a stable conclusion could be achieved after four iterations. Dietz (1987) stated that three rounds of iteration were found a common practice from published research reports. Various types of communication were recommended and applied for Delphi study, such as mail, fax and electronic approaches. Electronic communication is popular today for this type of research, as technology has advanced. Delphi panelists Rowe and his associates (2005) indicated that the composition of a panel had a critical impact on the outcome of a Delphi study. To develop un updated representative definition of service-learning, this Delphi study invited participation of professors and scholars with knowledge and experiences in service-learning. A purposeful sampling procedure was used for the selection of panel participants. The population of the study was faculty participants in higher education institutions within the United States. The target population was reviewers of the Michigan Journal of Community Service-Learning. The reviewer lists of this journal also 49 served as the sample frame of this study. The Michigan Journal of Community Service- Leaning was chosen because it was noted for its dedication to the development of service-learning and its involved personnel and groups (Elyer & Giles, 2000). Reviewers of such a journal, therefore, were thought to be ideal candidates for participation in this study due to their knowledge of service-learning. There were 20 issues published when sampling took place. One issue was chosen for this study. The reviewer list of the chosen issue consisted of sixty-eight members. These sixty-eight reviewers thus became the accessible sample for this study. Email addresses of these sixty-eight reviewers were found through the Google search engine. All 68 reviewers listed in the selected issue were contacted via email and invited to participate in this study. As a result, eight of the sixty-eight contacted reviewers agreed to participate and served as expert panel members for this study. The response rate in this study was 62.5%. There were five (5) responses consistently submitted throughout the three rounds of iteration. It is noteworthy that it could not be assumed that the five responses from each round of discussion were from the same respondents as panelists? identities were kept anonymous. Instrument There are important individuals who made a service-learning program possible. Students from involved institutions, instructors who design and conduct service-learning programs, and community agencies who coordinate service-learning programs with instructors from institutions of various education levels are the three major groups that have been mentioned repeatedly in service-learning literature. A tentative instrument was generated for the study after reviewing the literature. The tentative instrument consisted 50 of 34 items. These 34 items were generated to assess the three major domains of a service-learning program. Of these 34 items, 10 items were aimed at assessing the faculty domain, 15 items at assessing the community domain, and 9 of them at assessing the student domain. The first six items were dichotomous. These six items were designed to assess agreement related to experiences of participants using ?Yes? indicating agreement and ?No? indicating disagreement. The rest of the 29 items were designed to assess the three domains of a service-learning program. All of these 29 items used a five-point Likert-type scale with ?1? indicating ?not important at all?, and ?5? indicating ?very important?. Procedure The research design for this study included three rounds of discussion. Dietz (1987) and Erffmeyer et al (1986) indicated a stable consensus could be achieved after three iterations of a Delphi study. Electronic communications were selected for this study because every panel member had access to a computer. A cyberweb was designed for this Delphi study. A link to the cyberweb was sent to every participant via email. At the beginning of the study, two tasks were posted on the cyberweb to request every panel member?s reaction. The first task for panel members was to provide their own definition of service- learning. The second task asked panel members to rate the importance of individual items on the tentative instrument. A space following each item was provided for panel members to address their rationales. The details of the research procedure are presented in the following sections. 51 Round 1: For the first task, every panel member was asked to provide a definition of service-learning according to their individual perspectives. The second task was to ask panel members to rate the importance of each item on the tentative instrument. Panel members also were encouraged to provide additional items and rationales for their ratings. Round 2: All responses for the definition of service-learning from round 1 were compiled, analyzed, and then made available to all panel members on the cyberweb. Submitted definitions of service-learning were analyzed using qualitative data analysis procedures. Themes and codes were developed and organized. Additionally, panel members? ratings on each individual item were analyzed using SPSS 13.0. Statistics, such as means and standard deviation, were computed and reported on the cyberweb for panel members? reference. In addition, suggestions from panel members from round 1 were included in a revised instrument. After reviewing and referencing information posted on the cyberweb, panel members were asked to make additional suggestions for the continued improvement of the instrument and/or to revise their definitions of service- learning. Round 3: Panel members were provided with results from round 2, including the revision of the definition and the improved version of the instrument. In the third round of discussion, instead of rating each item, panel members were asked to comment on the inclusion of each item using ?yes? for inclusion and ?no? for exclusion. In addition, all panel members were asked to provide their thoughts on the revised definition of service- learning. Items that were given ?yes? from more than one-half of panel members were included in the final version of the instrument. Items receiving a ?yes? rating from less 52 than one-half of the panel members were excluded from the instrument. In addition, panel members? input regarding the definition of service-learning was organized according to qualitative data analysis procedures. As a result, although themes and codes stayed the same, the definition of service-learning was modified based on new input from panel members. The final definition of service-learning and the finalized version of the instrument were posted on the cyberweb for all panel members after changes had been made. Structural Stage: The Survey Study The second stage of this study, the structural stage, followed the completion of the Delphi study. To address the objectives of this stage, a quantitative research approach using survey inquiry was designed and implemented. Statistical procedures were carried out to explore the psychometric properties of the newly developed instrument. Hypothesized Model The hypothesized model is presented in Figure 1. This model was generated based on a thorough literature review and the Delphi study results. The model consisted of three factors and eight variables. 53 Figure 2 The Hypothesized Model of Service-Learning Faculty pedagogy 1 Slearning Community reciprocity 1 enpowerment equality Student activity 1 personal civic The three model factors are: Faculty, Community, and Student. Eight variables were investigated and included in the model. Each of the eight variables corresponds to one of the three factors. The faulty participants? attention to pedagogy and their students? learning are included in the Faculty factor. Reciprocity, equality and community empowerment are the three variables included in the Community factor. Within the student factor, students? service-learning activities; their development, which includes students? academic, personal, and professional development; and students? civic awareness are included. In addition, service-learning involves relationships among groups. Therefore, correlations between factors were expected. Variables There are two variables associated with the Faculty factor. The faculty participants? devotion to pedagogy is measured by items 1 and 4. Faculty participants? devotion to students? learning is measured by items 2, 3 and 5. 54 The Community factor consisted of three variables. Reciprocity is measured by items 6, 7, 8, 9, 19; community empowerment is measured by items 10, 15, 17 and 18; equality is measured by items 11, 12, 13, 14, 16 and 20. The Student factor is comprised of three factors. Students? service-learning activities are measured by item 21 and 26; student development is measured by items 22, 24, 27 and 28; students? civic awareness is measured by items 23 and 25. (See Table 1 for details) Table 1 Items and Their Corresponding Variables and Factors Factor Variables Items Faculty Pedagogy Student-Learning 1,4 2,3,5 Community Reciprocal partnership Empowerment Equality 6,7,8,9,19 10,15,17,18 11,12,13,14,16,20 Student Service-Learning activity Student-Development Civic Awareness 21, 26 22,24,27,28 23,25 Population The population for the substantive stage was the faculty participants in higher education institutions of the United States and participants were recruited from higher education institutions in the United States. Taking a sample from the population is a 55 common practice in a social science study because of the impossibility of studying every individual from the population (Pedhazur& Schmelkin, 1991). According to Pedhazur and Schmelkin (1991), sample size is not the only criterion that matters to research results. Representativeness of a sample is crucial. Therefore, in addition to sample size, a sampling procedure that would establish a sample with a high possibility of representing the population of the study is desired. Sampling A stratified random sampling procedure is known for its merit in collecting data in a systematic fashion. In an attempt to include variance from subgroups, a stratified random sampling procedure was employed in this study. It was impossible to obtain a list of all faculty participants in the United States, thus the stratified random sampling procedure identified only the institutions of this study. A sampling frame was found using the webpage of the University of Texas at Austin, http://www.utexas.edu/world/univ/state/index.html. From this website, a list of higher education institutions by state was identified found. Strata included all fifty states. Disproportionate allocation was decided because each state housed unequal numbers of higher education institutions. In addition, an equal allocation was decided for the number of institutions included from each stratum. Three institutions from each state were identified. Within each stratum, higher education institutions were divided into three categories using the classification of Carnegie Foundation: Doctorate-granting Universities, Master?s Colleges and Universities and Baccalaureate Colleges. One institution from each category was identified through a systematic random sampling procedure. The systematic random sampling procedures started with coding higher 56 education institutions from each category independently. The coding process started with number one and ended with ?k? which is the quantity of institutions situated in a state. A number, n, between one and 100 and another number, t, between one and k were selected for the systematic random sampling procedures. Finally, the (n + t) th institution of each category was identified and included for this study. As a result, 150 institutions were identified. After the completion of the data collection procedure, the sample size was smaller than expected. Therefore, a second data collection round was completed. Two categories, university and college, were used. The second data collection followed the same steps as the first. An additional one hundred institutions were identified. Consequently, a second email list including the email contact information of 148 contact persons was established. Finally, a total of 250 higher education institutions were identified though two sampling rounds and 708 individuals were contacted via email by the researcher. Procedure Department heads/chairs, deans of the School of Education, or the coordinators of education programs served as contact persons for this study. Using Microsoft office Access, a list of email addresses was generated by visiting the WebPages of individual departments, schools and programs of the 150 identified institutions. As a result, a list of 460 email addresses was generated. An email with a link to the research webpage was sent to the identified department heads/ chairs/coordinators using the mail merge feature of the Microsoft Office software. The purpose of this email was to request assistance in disseminating information for this research. After receiving the invitational email, the contact person 57 could forward the information along with the link to faculty members, if they agreed to provide assistance. All faculty participants who were willing to participate in this study could use the link to visit and respond to the instrument posted on the cyberweb. They also were able to submit their responses via this cyberweb. There were two follow-up emails sent to contact persons in two-week intervals. After the two follow-up emails, a thank you email was sent to them to express appreciation and inform them of the completion of the data collection. The second data collection including one hundred additional higher education institutions were identified, and 148 email addresses of contact persons were established. The second data collection attempt followed the same procedures as the first. Participants Two hundred forty-eight respondents answered the survey. Among the 248 participants, 100 (40.3%) identified themselves as female and 142 (57.3%) were male. There were six (2.4%) participants who did not indicate gender. Two hundred seven (83.5%) participants reported that they were conducting research on service-learning related issues, while 39 (15.7%) participants were not and two did not answer this question. Ninety-two (37.1%) participants indicated having less than three years involvement in service-learning activity, 77 (31.0%) having 3-6 years involvement, and 69 (27.8%) having more than 6 years involvement in service-learning activities. Among the 248 participants, 21 (8.5%) were non-tenure track assistant professors, 77 (31.0%) were tenure-track assistant professors, 49 (19.8%) were Associate professors, 36 (14.5%) were professors, and 64 (25.8%) were graduate teaching assistants or instructors. (See 58 Table 2 for detail). Note that the response rate of this study was unknown for an unknown number of faculty participants who ever received information of this study. Summary This chapter started with the purposes of the study and followed with a detailed description of the research design. Information about the population, sampling procedures, and participants were included. The instrument and the hypothesized model were discussed in this chapter as well. In the following chapter, Chapter Four, a manuscript is presented. Finally, research findings, discussions of findings, implications and recommendations for the study are discussed in Chapter Five. 59 Table 2 General Information of Participants n % Gender Male 142 57.3 Female 100 40.3 Missing 6 2.4 Service-Learning Research Involvement Yes 207 83.5 No 39 15.7 Do not respond 2 .8 Longevity of Service-Learning involvement Less than 3 years 92 37.1 3-6 years 77 31.0 More than 6 years 69 27.8 Professional ranking Professor 36 14.5 Associate Professor 49 19.8 Assistant Professor (tenure-track) 77 31.0 Assistant Professor (non-tenure track) 21 8.5 Instructor/Graduate Teaching Assistant 64 25.8 N=248 60 IV RESULTS This study was aimed at developing a definition of the representative definition of service-learning and an instrument to help faculty participants in their design and evaluation of their service-learning programs. The psychometric property of the instrument was examined in this study. There is one manuscript presented in this chapter. This manuscript includes information of the whole study. Introduction Learning through service has been practiced in American education since 1900 (James, 1910, Kilpatrick, 1918, Dewey, 1938). In recent decades, several organizations such as Peace Corps, Volunteers in Service to America, Job Corps, Youth Serve America, Campus Opportunity Outreach League (COOL), and Campus Compact have been established and thrived. In the 1990s, the National and Community Service Act of 1990 signed by President George H. W. Bush and the National and Community Service Trust Act of 1993 signed by President Clinton evidenced the flourish of service-learning. The term service-learning was introduced in 1967. Since that date researchers and practitioners, alike, have struggled to define just what it is. Initially, service-learning embraced any program utilizing service and learning. Kendall (1990), for example, reported 147 different terminologies used in the literature to refer programs involving students? learning and service. Eventually, leaders (Stanton, Giles, & Cruz, 1999, 61 Kendall, 1990) in the field of service-learning identified essential elements of service- learning. A service-learning program, at its minimum, included three groups: Faculty who conduct the program, students who enroll in a class with service-learning and perform service activity in communities, and community agencies where service-learning students are hosted. Researchers also have identified four criteria among the three groups of service-learning: (1) Student learning and development through active participation in thoughtfully organized service experiences that meet actual community needs and that are coordinated in collaboration with the school and community (2) Integration between the students' academic curriculum and work with the community that provides structured time for the student to think, talk, or write about what the student did and saw during service (3) Opportunity for students to use newly acquired skills and knowledge in real-life situations in their own communities; (4) Extension of student learning beyond the classroom and into the community in an effort to foster the development of a sense of caring for others. (National and Community Service Act of 1990). Without meeting these criteria, programs that simply pair students and community concerns do not qualify as service-learning. Recognition of these criteria is important--- not a semantic quibble, for to define any program that places students in communities as service-learning is to make a number of forgone conclusions as to philosophy and structure of an endeavor. 62 Research outcomes Research results suggest positive influences of service-learning programs on all involved parties, namely, students, faculty participants, and community agencies. Research outcomes of students fall into four categories (Waterman, 1997): enhanced learning of course content (Markus et al, 1993; Strange, 2000; McCluskey-Fawcett & Green, 1992), personal development (Marchel, 2003; Eyler & Giles, 1999; Jorge, 2003), the development of civic responsibilities and other citizenship values (Abernathy and Obenchain, 2001; Giles & Eyler, 1994a; Hollies, 2004; Kane &Westheimer, 1996), and student contribution to the community (Giles & Eyler,1994a; Kane & Westheimer, 1996; Kane, Westheimer & and Rogers, 2000). The improved student learning provided evidence that the integration of service-learning into a course design is helpful to faculty participants in fulfilling responsibilities of both teaching and research (Bringle & Hatcher, 1996; Parker-Gwin & Marbry, 1998), while research publications documented community partners? accounts on improvement as a result of the service-learning partnership (Jorge, 2003; Grey, 2000). Despite the positive outcomes attributed to service-learning and the increasing number of faculty participants, leaders in the field have long expressed concern that some practitioners of service-learning adopt unclear descriptions of the pedagogy (Kendall, 1990; Sheffield, 2005; Zlotkowski, 1995), a condition that gives rise to possible undesired consequences from an inappropriately structured service-learning program. That is, researchers (Holland, 2001; Kraft, 1996; Morton, 1995; O?Byrne, 2001; Troppe, 1996) emphasize that a clear description of service-learning dictates the implementation of service-learning activities and ultimately directs the evaluation of the effectiveness of 63 service-learning programs (Eby, 1998; Pompa, 2002; Zlotkowski, 1995). If unresolved, faculty participants could possibly design and implement service related programs and then name these programs ?service-learning?, if they believed that what they were doing was, indeed, service-learning. Thus, Sheffield (2005) contends that if the definition or the description of service-learning is not identified, service-learning will be ?everything for everybody? and ultimately ?become nothing? (p. 47). Therefore, an up-to-date description of service-learning that includes necessary components of service-learning and a current instrument assessing a service-learning program are beneficial to resolve challenges faculty participants facing in the initiation and evaluation of their service-learning programs. Significance of the Study Given the apparent disconnect between what service-learning should be versus what some practitioners make it to be, it is of interest to investigate whether or not the operationalization of service-learning established in the early 90s? has altered the practice of service-learning to date. Messick (1998) suggested the importance of re-examining a construct periodically to reflect the progress of the evolving human society. Thus, a clear and up-to-date description of service-learning for the design and an instrument for the evaluation of service-learning programs accordingly become important for both sustaining the endeavor of service-learning practitioners and the effectiveness of service- learning. With the up-to-date description of service-learning along with the instrument not only will faculty practitioners be able to design a proper service-learning program but also the potential of service-learning will be more likely to be realized. In responding to 64 these critical issues, this study was aimed at re-examining the description of service- learning and generating a current instrument to assess the inclusiveness of service- learning programs. Research Purposes The purposes of this study were to re-examine if the original depiction of service- learning still holds true today among leaders of service-learning. Or has the variability among practitioners altered the theoretical conception of service-learning? This research design included two stages of a strong program of construct validation study, the substantative stage and the structural stage. The objectives of the substantative stage were to re-examine the conception of service-learning and to generate an instrument to assess the inclusiveness of a service-learning program accordingly. The completion of the substantative stage was followed by the structural stage (Loevinger, 1957; Nunnally, 1978). The objective of the structural stage was to explore the psychometric properties of the instrument developed through the Delphi study. Method The Substantive Stage A Delphi technique was employed to achieve objectives of the substantative stage. A Delphi study is known for its systematic approach in collecting professional knowledge and judgments from a group of experts, known as panel members, regarding a variety of issues, through iterations of anonymous discussion (Erffmeyer, Erffmeyer, & Lane, 1986; Fisher, 1978, Turoff, 1970). The Delphi study consisted of three iterations. 65 A webpage was established as the communication channel between panelists and the researcher. A purposeful sampling procedure was used to form the panel of this study. Reviewers of the Journal of Michigan Community Service Learning were invited to participate in this study. Eight among sixty-eight reviewers contacted agreed to serve as the panelists in the Delphi study. The panelists were asked to provide their perspectives of service-learning and rate items on a proposed instrument. Their perspectives of service-learning were analyzed using qualitative data analysis approach. A descriptive analysis using SPSS 13.0 was conducted on panelists? ratings of items. Results were posted on the webpage for panelists? reference and possible revision. The Structural Stage An on-line survey was designed to explore the psychometric properties of the instrument developed through the Delphi study. To reach the objective of this study, a quantitative research approach using on-line survey inquiry was designed and implemented. A hypothesized model of service-learning was generated as a result of an extensive literature review. Instrument The instrument used in this study was Assessing Service-Learning programs in Higher Education Institutions (ASLP). The ASLP is comprised of 28 five point Likert- type items with 1 = Not Important at All and 5 = Very Important. ASLP was designed to assist faulty participants in designing and evaluating their service-learning programs. These items were developed from a through literature review on crucial elements of service-learning and finalized by a group of experts in service-learning through a Delphi study conducted by the researchers of the present study. The ASLP consisted of Faculty, 66 Community, and Student subscales. The Faculty subscale included five items; the Community subscale included fifteen; the student subscale consisted of ten items. Hypothesized Model The hypothesized model is presented as Figure 2. This model was generated after a thorough literature review. The model consisted of three factors and eight variables. Figure 2 The Hypothesized Model of service-Learning Faculty pedagogy 1 Slearning Community reciprocity 1 enpowerment equality Student activity 1 personal civic The three model factors are: Faculty, Community, and Student. Eight variables were investigated and included in the model. Each of the eight variables corresponds to one of the three factors. The faulty participants? attention to pedagogy and their students? learning are included in the Faculty factor. Reciprocity, equality and community empowerment are the three variables included in the Community factor. Within the student factor, students? service-learning activities; their development, which includes students? academic, personal, and professional development; and students? civic awareness are included. In addition, service-learning involves relationships between groups. Therefore, correlations between factors were expected. 67 Variables The two variables associated with the Faculty factor are faculty participants? attentions to their teaching and their students? learning. The Community factor included three variables: reciprocity, community empowerment, and equality between the faculty participants and their partnered community agencies. The Student factors included students? service-learning activities, student development and students? civic awareness. Items and their corresponding variables are presented in Table 2. Table 2 Items and Their Corresponding Variables Factor Variable Items Pedagogy 1,4 Faculty Student Learning 2,3,5 Reciprocity 6,7,8,9,19 Empowerment 10,15,17,18 Community Equality 11,12,13,14,16,20 Activity 21, 26 Learning 22,24,27,28 Student Civic Awareness 23,25 Population The population for the structural stage was the faculty participants in higher education institutions of the United States. Participants of the structural stage were recruited from higher institutions in the United States. Taking a sample from the 68 population is a common practice of social science study because of the impossibility of studying every individual from the population (Pedhazur& Schmelkin, 1991). According to Pedhazur and Schmelkin (1991), sample size is not the only criterion that matters to research result, representativeness of a sample is crucial as well. Therefore, in addition to sample size, a sampling procedure that would establish a sample with a high possibility of representing the population of the study was desired. Sampling In an attempt to include variance from subgroups, a stratified random sampling procedure was used in this stage. This sampling technique is known for its merit in collecting data in a systematic fashion. It was impossible to obtain a list of all faculty participants in the United States, thus the stratified random sampling procedure identified only the institutions of this study. A sampling frame was found using the webpage of the University of Texas at Austin, http://www.utexas.edu/world/univ/state/index.html. From this website, a list of higher education institutions by state was identified. Strata included all fifty states. Disproportionate allocation was decided upon because each state housed unequal numbers of higher education institutions. In addition, an equal allocation was decided for the number of institutions included from each stratum. Three institutions from each state were identified. Within each stratum, higher education institutions were divided into three categories: Doctorate-granting Universities, Master?s Colleges and Universities, and Baccalaureate Colleges. One institution from each category was identified through a systematic random sampling procedure. The systematic random sampling procedures started with coding higher education institutions from each category independently. The coding process started with number one and ended with ?k? which is 69 the quantity of institutions situated in a state. A number, n, between one and 100 and another number, t, between one and k were selected for the systematic random sampling procedures. Finally, the (n + t) th institution of each category was identified and included for this study. As a result, 150 institutions were identified. After the completion of the data collection procedure, the sample size was smaller than expected. Therefore, a second data collection round was completed. Two categories, university and college, were used. The second data collection round followed the same steps as the first. An additional one hundred institutions were identified. Consequently, a second email list including the email contact information of 148 contact persons was established. Finally, a total of 250 higher education institutions were identified though two sampling rounds and 708 individuals were contacted via email by the researcher. Procedure Department heads/chairs, deans of the School of Education, or the coordinators of education programs served as contact persons for this study. Using Microsoft office Access, a list of email addresses was generated by visiting the WebPages of individual departments, schools and programs of the 150 identified institutions. As a result, a list of 460 email addresses was generated. An email with a link to the research webpage was sent to the identified department heads/ chairs/coordinators using the mail merge feature of the Microsoft Office software. The purpose of this email was to request assistance in disseminating information for this research. After receiving the invitational email, the contact person could forward the information along with the link to faculty members, if they agreed to provide assistance. All faculty participants who were willing to participate in this study 70 could use the link to visit and respond to the instrument posted on the cyberweb. They also were able to submit their responses via this cyberweb. There were two follow-up emails sent to contact persons with a two-week interval. After the two follow-up emails, a thank you email was sent to participants to express appreciation and inform them the completion of the data collection. The second data collection including one hundred additional higher education institutions were identified, and 148 email addresses of contact persons were established. The second data collection attempt followed the same procedures as the first. Participants Two hundred forty-eight respondents answered the survey. Among the 248 participants, 100 (40.3%) identified themselves as female and 142 (57.3%) were male. There were six (2.4%) participants who did not indicate gender. Two hundred seven (83.5%) participants reported that they were conducting research on service-learning related issues, while 39 (15.7%) participants were not and two did not answer this question. Ninety-two (37.1%) participants indicated having less than three years involvement in service-learning activity, 77 (31.0%) having 3-6 years involvement, and 69 (27.8%) having more than 6 years involvement in service-learning activities. Among the 248 participants, 21 (8.5%) were non-tenure track assistant professors, 77 (31.0%) were tenure-track assistant professors, 49 (19.8%) were Associate professors, 36 (14.5%) were professors, and 64 (25.8%) were graduate teaching assistants or instructors. (See Table 3 for detail). Noteworthy, the response rate of this study was unknown for an unknown number of faculty participants who ever received information of this study. Table 3 71 General Information of Participants n % Gender Male 142 57.3 Female 100 40.3 Missing 6 2.4 Service-Learning Research Involvement Yes 207 83.5 No 39 15.7 Do not respond 2 .8 Longevity of Service-Learning involvement Less than 3 years 92 37.1 3-6 years 77 31.0 More than 6 years 69 27.8 Professional ranking Professor 36 14.5 Associate Professor 49 19.8 Assistant Professor (tenure-track) 77 31.0 Assistant Professor (non-tenure track) 21 8.5 Instructor/Graduate Teaching Assistant 64 25.8 N=248 Results 72 The Substantive Stage Re-examination of Service-Learning In the attempt to understand the evaluation of service-learning after more than a decade?s practice in the field of education, a Delphi study was conducted. As a result, a definition developed through the Delphi procedure was similar to those previously suggested by service-learning experts and was stated as: Service-learning is a pedagogical strategy. It is purposefully designed to enhance students? academic learning through organized community service activities. Community service activities should meet the needs of and be identified by the community. Ongoing reflection exercises, individual or in groups, are necessary for consolidating students? understanding of course content, promoting personal development, and deepening their knowledge of and commitment to the well- being of the community to which they belong. Instrument Item means and standard deviations were computed and reported to panel members. The result of this computation was presented in Table 4. Some items were reworded according to suggestions of panel members after the 1 st round of discussion. Means from the 1 st round ranged from 2.33 to 4.83, and in the 2 nd round from 2.8 to 4.8. A comparison of means between round 1 and round 2 resulted in 31 out of 34 items had larger means in round 2 than those in round 1. Exceptions included items #4, #31, and #33. Standard deviations were, for the most part, smaller in round 2 than in round 1. Items #4, # 17, #26, and #29 had slightly larger standard deviations. All these increases in values were less than .04. Item #33 had the greatest 73 increase of .39. The third iteration focused on the inclusion of each item and resulted in eliminating item #23 as it appeared to assess similar information as item to 22. Thus, the final version of the instrument consisted of 33 items. Ten items assess information about faculty domain; fourteen items assess the community partner domain; and nine items assess the student domain. The first five items were reformatted to be binary, yes or no, as suggested by panel members. The response formats for the remaining twenty-eight items is a five-point Likert-type scale with 1 indicating not important at all and 5 indicating very important. Table 4 74 Means and Standard Deviations from Round 1 and Round 2 of Discussion Item Number Mean _________________ Round 1 Round 2 Standard deviation _____________________ Round 1 Round 2 1 3.00 3.8 1.50 .447 2 2.5 3.6 1.76 .548 3 2.33 2.8 1.75 .837 4 4.67 4.6 .52 .548 5 4.33 4.8 1.63 .447 6 3.8 4.4 1.47 .548 7 3.5 4.6 1.23 .548 8 4.0 4.8 1.67 .447 9 3.83 4.5 1.60 .577 10 3.83 4.8 1.47 .447 11 4.0 4.0 1.27 .707 12 3.5 4.60 1.76 .548 13 3.83 4.80 1.47 .447 14 4.0 4.6 1.67 .548 15 4.33 4.6 .82 .548 16 4.5 4.75 .55 .500 17 4.67 4.6 .52 .548 18 3.83 4.2 1.60 .837 Table 4. (continued) Means and Standard Deviations from Round 1 and Round 2 of Discussion 75 Mean _________________ Round 1 Round 2 Standard deviation _____________________ Round 1 Round 2 19 3.83 3.8 1.94 1.3 20 4.5 4.6 .84 .548 21 4.5 4.8 .55 .447 22 4.17 4.6 .98 .548 23 3.67 3.8 1.63 .837 24 4.67 4.8 .516 .447 25 3.83 4.2 1.17 .837 26 4.83 4.8 .41 .447 27 4.17 4.5 1.17 .577 28 4.67 4.8 .52 .477 29 4.17 4.0 .75 .816 30 4.67 4.6 .516 .894 31 3.33 3.25 1.97 .957 32 3.5 3.6 .55 .548 33 4.5 4.4 .55 .894 34 4.33 3.6 1.03 1.34 The Structural Stage Data analysis 76 Confirmatory Factor Analysis (CFA) using Amos 7.0 was utilized to investigate the consistency between the data and the hypothesized model, shown as Figure 1, generated from literature. Model fit indexes used to exam the research results of this study were classified as absolute. Absolute fit indexes assess how well the overall model explains the observed variance or correlation matrix. The absolute fit indexes used in this study were: Chi-Square, Goodness-of-Fit Index (GFI), Comparative Fit index (CFI), and the Root Mean Square Error of Approximation (RMSEA). Reliability There were three subscales included in this instrument. Cronbach?s Alpha was used to examine the reliability of scores from the three subscales. Cronbach?s Alpha coefficients ranged between .545 and .875 (See Table 5). Table 5 Reliabilities of Scores of Three Subscales Faculty (n = 5) Community ( n= 15) Student (n= 8) Cronbach?s Alpha .545 .875 .750 Validity This instrument was developed based on the input of experts in service-learning using a Delphi study. Evidence of content validity was provided based on both the service-learning literature and these experts. Additionally, the correlation coefficients between factors of the hypothesized model of service-learning were between .46 and .60. 77 These correlation coefficients were less than .85, thus provided evidence of discriminant validity as suggested by Kline (1998). Bivariate Correlations Items addressing each variable within each domain were summed in order to calculate means, standard deviations, and intercorrelations between variables (see Table 6). The bivariate correlation coefficients were statistically significant except for the one between reciprocity and civic awareness. The range of bivariate coefficients was between .106 and .679. No correlation coefficient exceeded .8, thus multicollinearity was not considered a concern (Hair et al, 1998). 78 79 Model Fit The results of this analysis are presented in Table 7. As observed in Table 6, a statistically significant Chi-Square (? 2 = 46.567, p< .001) resulted from CFA procedure. The Chi-Square statistic is known for its sensitivity to sample size, thus, it is important to examine other model fit indexes. The magnitudes of the GFI (.951) and the CFI (.949) exceeded an insignificant value of .90 or greater (Meyers, Gamst & Guarino, 2005). In addition, the value of RMSEA was .090, which was below the accepted value of .1 or less (Meyers, Gamst & Guarino, 2005). Thus, the hypothesized model of this research adequately explained the variance of the data. Table 7 Result of the Analysis for the Hypothesized Model N Chi-Square p DF GFI CFI RMSEA 217 46.567 < .001 17 .951 .949 . 090 Correlation Coefficients The correlation coefficients between any two factors were all statistically significant. In CFA, discriminant validity between factors is indicated by the correlation coefficients between factors. According to Kline (1998), two factors with correlation coefficient higher than .85 could be conceptually perceived as the measuring the same information. The analysis of this CFA resulted in a correlation of .56 between the Faculty factor and the Students factor; .46 between the Student factor and the Community factor; and .60 between the Faculty factor and the Community factor (see Figure 3). All of these correlation coefficients were less than .85, and thus evidence of discriminant validity. 80 Figure 4 Results of the Confirmatory Factor Analysis Chi-Square=46.562, DF=17, p=.000, GFI=.951, CFI=.949, RMSEA=.090 Faculty pedagogy e1 .41 Slearning e2 .78 Community reciprocity e3 .69 enpowerment e4 .84 equality e5 .81 Student activity e6 .72 personal e7 .81 civic e8 .73 .60 .46 .56 Pattern Coefficients The pattern coefficients between each individual observed variable and its corresponding factor were statistically significant. See Figure 4 for correlation coefficients and pattern coefficients. The pattern coefficients of this study were all statistically significant. The pattern coefficients of paths to the Faculty factor were between .41 and .78; they were between .69 and .84 for the Community factor; and between .72 and .81 for the Student factor. In addition, the correlation coefficients between two factors were between .46 and .60. (See Table 8) Table 8 81 Pattern Coefficients between Variables and Their Corresponding Factors Factor Variable Pattern Coefficients Faculty Pedagogy .41 Faculty Student-learning .78 Community Reciprocity .69 Community Empowerment .84 Community Equality .81 Student Service-learning activity .72 Student Personal Development .81 Student Civic Awareness 73 Discussions Guidelines in designing a service-learning program and a tool for its evaluation are crucial to keep the perceptions of service-learning experts and the practices of practitioners in coherence. First, faculty participants need guidelines in designing a proper service-learning program because they are not necessarily experts in service-learning. Faculty participants come from a variety of backgrounds with diverse expertise. Their motives for being involved in service-learning also are diverse. In addition, service-learning is not a simple service enterprise and should not become ?everything for everybody? (Sheffield, 2005, p. 47). It is important to have guiding principles for faculty participants to conduct service- learning programs described by Zlotkowski (1995) as ?real service-learning? (p.129). 82 Second, faculty participants need guidelines to relieve the pressure of seeking balance between time availability and professional responsibilities. Service-learning is a very time consuming activity, which requires a great deal of communication and coordination among those involved (Holland, 2001). A dependable instrument that includes core elements of service-learning reduces the amount of time faculty spend in designing proper service-learning programs, thus faculty participants become more effective in conducting service-learning programs. Third, faculty participants need guidelines to conduct high quality service- learning. The literature indicated that the quality of service-learning programs would have significant effects on the outcomes of student learning (Giles & Eyler, 1994). To achieve effective teaching and enhance student learning, it is critical for faculty participants to follow guiding principles when they participate in service-learning. Fourth, faculty participants need data to evaluate the outcomes of their programs to provide evidence of the effectiveness of these programs. Faculty participants are not necessarily experts in program evaluation (Gelmon, 2003), and an instrument that includes essential components of service-learning could be a valuable tool to them. Evaluation results from such an instrument would be beneficial to faculty participants in initiating service-learning programs, evaluating student-learning, and maintaining the faculty-community partnership. In the short term, evaluation results inform the extent to which faculty participants have achieved their objectives of integrating service-learning into their teaching. In the long term, the evaluation result may guide future efforts, thus improving the quality of service-learning as well as enhancing teaching effectiveness. 83 However, faculty participants still have difficulties in the design and evaluation of service-learning programs. The researcher undertook this challenge and conducted the current research. In this research project, a hypothesized model was proposed and an on- line survey was implemented. The research results indicated that the model is adequate in explaining the data and the hypothesized model was supported by the results. To be precise, the model could be used as a guideline in designing a service-learning program, and the instrument could be used as a tool for evaluating the implementation of a service- learning program. As the model illustrated, a service-learning program consists of three domains: the Faculty, the Student, and the Community. Within each domain, there are facets of service-learning that must be addressed. In the Faculty domain, faculty participants? attention to their teaching and their students? learning need to be assessed. Within the Community domain, the faculty participants should take the reciprocal partnership, community empowerment, and the equality between themselves and their partnered community agencies into consideration. In the Student domain, students? service- learning activities, their development and their civic awareness are important aspects of service-learning to be addressed. In addition, the instrument is a valuable tool for assessing a service-learning program. The evaluation results reflect the inclusiveness of service-learning programs, and thus not only faculty participants? effort could be sustained, but also the quality of service-learning could be improved. 84 Limitations of the Research There were limitations in the study. Future research designs should consider these limitations to improve the psychometric properties of this instrument. First, a voluntary sample was generated for this study. The sample, therefore, may not represent the population adequately despite the fact that a stratified random sampling procedure was used. Second, a homogeneous sample was used for this study. Characteristics of a sample will affect the extent to which a research result can be generalized (Meyers et al, 2005; Pedhazur, & Schmelkin, 1991). Participants of this study were recruited from the College of Education only. The composition of this sample may bias the result of this study in relation of other fields of endeavor. Third, a small sample was used in this study, considering the statistical analyses employed. There is no definite criterion for sample size (Hair et al, 1995; Meyers et al, 2006; Thomson, 2004). Hair et al (1995) suggested that a ratio of 10 respondents to one measured variable is acceptable for CFA. A ratio of 15 participants to one item was suggested by Stevens (2002) for confirmatory analytic procedures. The sample size for this study was relatively small according the recommendations made by Hair et al and Stevens. All of the limitations mentioned above would diminish the generalizability of the research results. Fourth, the on-line survey included in this study required self-reported information. One of the most troublesome shortcomings of a self-reported survey study is the issue related to social desirability (Phillips & Clancy, 1972; Groves, 1987). Social desirability, also known as self-desirability (Nunnally, 1978), is the tendency with which 85 individuals tend to respond to survey questions according to their perceived expectations of the society or researchers (Pedhazur & Schmelkin, 1991). Therefore, responses from faculty participants were possibly biased, and consequently could have skewed the result. Recommendations for Future Research The instrument was developed in this study to assess service-learning. This study represents the first attempt to test the psychometric properties of the instrument. The recommendations presented below would be beneficial for future studies assessing the psychometric properties of this instrument. First, future research should refine sampling procedures to ensure the representativeness of the sample in relation to the population from which it is drawn. A stratified random sampling procedure was used for this study; however, the sampling procedure was only for the selection of institutions rather than actual participants. In order to have a representative sample, sampling procedures in future research should target potential participants rather than institutions. Second, this study focused on examining the conceptualized structure of a service-learning model. Future validation studies could focus on convergent validity by examining the extent to which the results of this instrument correlates with results from other instruments assessing service-learning in higher education. A focus on the discriminant validity also is recommended. Although discriminant validity between factors was indicated by correlation coefficients, which were less than .85 as suggested by Kline (1998), future research could aim at the exploration of discriminant validity 86 across instruments by comparing results from the instrument to those produced from instruments assessing different but similar constructs. Third, this study included a homogeneous sample. Participants were invited from the College of Education only. This sample characteristic influences the accuracy in applying research results to the population (Meyers et al, 2005; Pedhazur, & Schmelkin, 1991). Future research on the psychometric properties of this instrument could include participants from diverse discipline backgrounds to further examine the hypothesized model in relation to other fields of study in higher education. Fourth, the reliability of the scores from the Faculty subscale was low. Further research might improve the subscale by increasing number of items and/or revising items to reduce ambiguity to better assess the variables included in this scale, and thus solicit intended information. Summary This study was conducted to solicit information directly from faculty participants concerning their practice of service-learning. As discussed in service-learning literature, abundant research data had been collected regarding students? learning; however, the research data on faculty and community participants in service-learning have not been investigated with equivalent intensity when compared to that on students? learning. It was shown in the literature that service-learning influences all participants, namely, the students, faculty, and community. However, there is inadequate understanding of how service-learning has impacted faculty and community participants. The insufficient knowledge about and study of faculty and community has resulted in misunderstandings 87 and haphazard implementation of service-learning. We posit that in order to assess the influences of service-learning on all participants, researchers must access faculty as resources. Faculty participants are involved in the initiation of service-learning, student reflection and learning evaluation, and community partners. Thus, it is important to design research projects to assess faculties? perspectives of students? learning, program design, and community reactions. Results from such research projects will be beneficial to faculty participants in initiating their service-learning programs, evaluating student learning, and maintaining faculty-community partnership. This study was designed to explore the definition of service-learning and to develop an evaluation instrument of service-learning programs. The psychometric properties of the instrument were examined in this study as well. A Delphi study was designed to explore the definition and to develop the instrument. Experts in service- learning were invited to serve on the expert panel for the Delphi study. A representative definition of service-learning and an instrument were developed as the results of the Delphi study. An on-line survey study was designed to explore the psychometric property of the instrument. Faculty participants were invited to offer their perspectives by responding to the instrument posted on-line. The data collected from the instrument was examined against a hypothesized model of service-learning generated through a thorough literature review. The results from a Confirmatory Factor Analysis indicated that the model explained the internal structure of the data adequately. Explicitly, the model could be used as a guideline for designing a service-learning program and the instrument could be used as an evaluation tool for assessing the implementation of a service-learning program. However, the reliability from the faculty subscale was less 88 than ideal. Therefore, caution should be exercised in the interpretation of results from the Faculty subscale when the subscale is used for both program evaluation and research purposes. 89 V. DISCUSSION OF FINDINGS, CONCLUSIONS AND RECOMMENDATIONS This chapter presents an overview of the study including research purpose, research design and results. Limitations are discussed and followed by recommendations for future research on the psychometric properties of the newly developed instrument. Study Overview This research included two stages of a construct validation study: the substantive stage and the structural stage. The purpose of the substantive stage was to develop a comprehensive definition of service-learning and an instrument to assist faculty participants with their design and evaluation of service-learning programs. In response to these research purposes, a Delphi study with three iterations was designed and conducted. Eight service-learning experts participated in this study. As a result, a representative definition of service-learning and an instrument were developed. Analyses of the structural stage were conducted following the completion of the Delphi study. The structural stage was aimed at examining the psychometric properties of the instrument by investigating the consistency of a hypothesized service-learning model (Figure 5) and the data collected from the instrument. 90 Figure 5 The Hypothesized Model of Service-Learning Faculty pedagogy 1 Slearning Community reciprocity 1 enpowerment equality Student activity 1 personal civic This Figure was also presented as Figure 2 in Chapter III The hypothesized model of service-learning was generated after an extensive literature review. There were three subscales and eight variables included in this hypothesized model. Variables included in the Faculty subscale were pedagogy and student-learning. These two variables assessed faculty participants? attention to their teaching and student-learning. There were three variables included in the Community subscale. The variable of reciprocity assessed the unique partnership between the faculty participants and the community agencies in service-learning; the variable of empowerment assessed the critical objective of service-learning; and the variable of equality assessed the equality between the faculty participants and the community agencies. The variable of service-learning activity, the variable of student development and the variable of students? civic awareness were the three variables included in the Student subscale. This information is organized and presented in Table 9. All 91 twenty-eight items used a five-point Likert-type scale with 1= not important at all and 5= very important. Table 9 Factors and Their Corresponding Variables Factor Variables Faculty Pedagogy Student?s learning Community Reciprocal partnership Empowerment Equality Student Service-learning activity Student development Civic awareness The structural stage was aimed at probing the psychometric properties of the instrument by examining the consistency between a hypothesized model of service- learning and the data collected from the instrument. An on-line survey study was conducted. The sample of the on-line survey was gathered as a result of a stratified random sampling process. Two hundred fifty higher education institutions were identified. A total of 602 persons in the 250 higher education institutions were contacted via email. To examine the consistency between the hypothesized model and the data collected from the instrument, a confirmatory factor analysis was performed. After 92 excluding 31 incomplete surveys, the statistical procedure analyzed information from 217 participants. Findings Eight service-learning experts participated in the Delphi study. As a result, a comprehensive definition developed through this study was: Service-learning is a pedagogical strategy. It is purposefully designed to enhance students? academic learning through organized community service activities. Community service activities should meet the needs of and be identified by the community. Ongoing reflection exercises, individual or in groups, are necessary for consolidating students? understanding of course content, promoting personal development, and deepening their knowledge of and commitment to the well- being of the community to which they belong. This definition was consistent with suggestions made by service-learning experts in the 1990s. This finding indicated that after more than a decade?s practice, the underlying criteria of service-learning still hold true today. In addition, an instrument to assess service-learning programs was developed. The instrument consisted of three subscales: faculty, community and student, and twenty-eight items (See Appendix 1). The accomplishment of the substantive stage was followed by the structural stage of a construct validation study. The objective of the structural stage was to examine the psychometric properties of the newly developed instrument by examining the consistency between the instrument and a hypothesized model of service-learning. The outcomes of the statistical procedure indicated that there was a consistency between the hypothesized model and the internal structure of the data collected from the instrument. In short, the 93 hypothesized model explained the data adequately. Notably, the reliability of scores from the Faculty subscale was .54. This result is less than the suggested value of .7 or higher (Nunnally, 1978). Caution is advised when this subscale is used for both research and program evaluation purposes. Discussions Guidelines in designing a service-learning program and a tool for its evaluation are crucial to preserve the essence of service-learning while maintaining its popularity. First, faculty participants need guidelines in designing a proper service-learning program because they are not necessarily experts in service-learning. Faculty participants come from a variety of backgrounds with diverse expertise. Their motives for being involved in service-learning also are diverse. In addition, service-learning is not a simple service enterprise and should not become ?everything for everybody? (Sheffield, 2005, p. 47). It is important to have guiding principles for faculty participants to conduct service-learning programs described by Zlotkowski (1995) as ?real service-learning? (p.129). Second, faculty participants need guidelines to relieve the pressure of seeking balance between time availability and professional responsibilities. Service-learning is a very time consuming activity, which requires a great deal of communication and coordination among those involved (Holland, 2001). A dependable instrument that includes core elements of service-learning reduces the amount of time faculty spend in designing proper service-learning programs, thus faculty participants become more effective in conducting service-learning programs. Third, faculty participants need guidelines to conduct high quality service- learning. The literature indicated that the quality of service-learning programs would 94 have significant effects on the outcomes of student learning (Giles & Eyler, 1994). To achieve effective teaching and enhance student learning, it is critical for faculty participants to follow guiding principles when they participate in service-learning. Fourth, faculty participants need data to evaluate the outcomes of their programs to provide evidence of the effectiveness of these programs. Faculty participants are not necessarily experts in program evaluation (Gelmon, 2003), and an instrument that includes essential components of service-learning could be a valuable tool to them. Evaluation results from such an instrument would be beneficial to faculty participants in initiating service-learning programs, evaluating student-learning, and maintaining the faculty-community partnership. In the short term, evaluation results inform the extent to which faculty participants have achieved their objectives of integrating service-learning into their teaching. In the long term, the evaluation result may guide future efforts, thus improving the quality of service-learning as well as enhancing teaching effectiveness. However, faculty participants still have difficulties in the design and evaluation of service-learning programs. The researcher undertook this challenge and conducted the current research. In this research project, a hypothesized model was proposed and an on- line survey was implemented. The research results indicated that the model is adequate in explaining the data and the hypothesized model was supported by the results. To be precise, the model could be used as a guideline in designing a service-learning program, and the instrument could be used as a tool for evaluating the implementation of a service- learning program. As the model illustrated, a service-learning program consists of three domains: the Faculty, the Student, and the Community. Within each domain, there are facets of 95 service-learning that must be addressed. In the Faculty domain, faculty participants? attention to their teaching and their students? learning need to be assessed. Within the Community domain, the faculty participants should take the reciprocal partnership, community empowerment and the equality between themselves and their partnered community agencies into consideration. In the Student domain, students? service- learning activities, their development and their civic awareness are important aspects of service-learning to be addressed. In addition, the instrument is a valuable tool for assessing a service-learning program. The evaluation results from this instrument reflect the inclusiveness of crucial components of service-learning programs, and thus provide faculty participants a reference for their future service-learning efforts. Furthermore, not only faculty participants? effort could be sustained, but also the quality of service- learning could be improved. Limitations of the Research There were limitations in the study. Future research designs should consider these limitations to improve the psychometric properties of this instrument. First, it was a voluntary sample generated in this study. The sample, therefore, may not represent the population adequately despite the fact that a stratified random sampling procedure was used. Second, a homogeneous sample was used for this study. Characteristics of a sample will affect the extent to which a research result can be generalized (Meyers et al, 2005; Pedhazur, & Schmelkin, 1991). Participants of this study were recruited from the 96 College of Education only. The composition of this sample may bias the result of this study in relation of other fields of endeavor. Third, a small sample was used in this study, considering the statistical analyses employed. There is no definite criterion for sample size (Hair et al, 1995; Meyers et al, 2006; Thomson, 2004). Hair et al (1995) suggested a ratio of 10 respondents to one measured variable is acceptable for CFA. A ratio of 15 participants to one item was suggested by Stevens (2002) for confirmatory analytic procedures. The sample size of this study was relatively small according the recommendations made by Hair et al and Stevens. All of the limitations mentioned above would diminish the generalizability of the research results. Fourth, the on-line survey included in this study required self-reported information. One of the most troublesome shortcomings of a self-reported survey study is the issue related to social desirability (Phillips & Clancy, 1972; Groves, 1987). Social desirability, also known as self-desirability (Nunnally, 1978), is the tendency with which individuals tend to respond to survey questions according to their perceived expectations of the society or researchers (Pedhazur & Schmelkin, 1991). Therefore, responses from faculty participants were possibly biased, and consequently could have skewed the result. Recommendations for Future Research The instrument was developed in this study to assess service-learning. This study represents the first attempt to test the psychometric properties of the instrument. The recommendations presented below would be beneficial for future studies assessing the psychometric properties of this instrument. 97 First, future research should refine sampling procedures to ensure the representativeness of the sample in relation to the population from which it is drawn. A stratified random sampling procedure was used for this study; however, the sampling procedure was only for the selection of institutions rather than actual participants. In order to have a representative sample, sampling procedures in future research should target potential participants rather than institutions. Second, this study focused on examining the conceptualized structure of a service-learning model. Future validation studies could focus on convergent validity by examining the extent to which the results of this instrument correlates with results from other instruments assessing service-learning in higher education. A focus on the discriminant validity also is recommended. Although discriminant validity between factors was indicated by correlation coefficients, which were less than .85 as suggested by Kline (1998), future research could aim at the exploration of discriminant validity across instruments by comparing results from the instrument to those produced from instruments assessing different but similar constructs. Third, this study included a homogeneous sample. Participants were invited from the College of Education only. This sample characteristic influences the accuracy in applying research results to the population (Meyers et al, 2005; Pedhazur, & Schmelkin, 1991). Future research on the psychometric properties of this instrument could include participants from diverse discipline backgrounds to further examine the hypothesized model in relation to other fields of study in higher education. Fourth, the reliability of the scores from the Faculty subscale was low. Further research might improve the subscale by increasing number of items and/or revising items 98 to reduce ambiguity to better assess the variables included in this scale, and thus solicit intended information. Summary This study was conducted to solicit information directly from faculty participants concerning their practice of service-learning. As discussed in service-learning literature, abundant research data had been collected regarding students? learning; however, the research data on faculty and community participants in service-learning have not been investigated with equivalent intensity when compared to that on students? learning. It was shown in the literature that service-learning influences all participants, namely, the students, faculty, and community. However, there is inadequate understanding of how service-learning has impacted faculty and community participants. The insufficient knowledge about and study of faculty and community has resulted in misunderstandings and haphazard implementation of service-learning. We posit that in order to assess the influences of service-learning on all participants, researchers must access faculty as resources. Faculty participants are involved in the initiation of service-learning, student reflection and learning evaluation, and community partners. Thus, it is important to design research projects to assess faculties? perspectives of students? learning, program design, and community reactions. Results from such research projects will be beneficial to faculty participants in initiating their service-learning programs, evaluating student learning, and maintaining faculty-community partnership. This study was designed to explore the definition of service-learning and to develop an evaluation instrument of service-learning programs. The psychometric 99 properties of the instrument were examined in this study as well. A Delphi study was designed to explore the definition and to develop the instrument. Experts in service- learning were invited to serve on the expert panel for the Delphi study. A representative definition of service-learning and an instrument were developed as the results of the Delphi study. An on-line survey study was designed to explore the psychometric property of the instrument. Faculty participants were invited to offer their perspectives by responding to the instrument posted on-line. The data collected from the instrument was examined against a hypothesized model of service-learning generated through a thorough literature review. The results from a Confirmatory Factor Analysis indicated that the model explained the internal structure of the data adequately. Explicitly, the model could be used as a guideline for designing a service-learning program and the instrument could be used as an evaluation tool for assessing the implementation of a service-learning program. However, the reliability from the faculty subscale was less than ideal. Therefore, caution should be exercised in the interpretation of results from the Faculty subscale when the subscale is used for both program evaluation and research purposes. 100 REFERENCES Abernathy, T.V., & Obenchain, K. M. (2001). Student ownership of service-learning projects: Including ourselves in our community. Intervention in School & Clinic, 37(2), 86-95. Aberson, C. L., Berger, D. E., Healy, M. R., Kyle, D. J., & Romero, V. L. (2000). Evaluation of an interactive tutorial for teaching the central limit theorem. Teaching of Psychology, 27(4), 289-291 Adams, M. (2004). Faculty matters. Nursing Education Perspectives, 25(5), 216-217. Adler-Kassner, L., Crooks, R., & Watters, A. (1997). Writing the community: Concepts and models for service-learning in composition. Washington, DC: American Association for Higher Education. Amtmann, J. (2004). Perceived effects of a correctional health education service-learning program. Journal of Correctional Education, 55,4, 335-348. Anderson, J. B., & Guest, K. (1994). Service learning in teacher education. In R. J. Kraft & M. Swadener (Eds.), Building community: Service learning in the academic disciplines (pp. 139-148). Denver, CO: Colorado Campus Compact. Anderson, D.J. & Major, R. L. (2001). Dewey, democracy, and citizenship. The Clearing House, 75,2, 104-107. Annotated history of service learning. (n.d.). Retrieved May 15, 2006, from http://www.servicelearning.org/welcome_to_service-learning/history/index.php 101 Apolito, T. (2006). Why take a criminal justice internship? Retrieved May 18, 2006, from http://academic.udayton.edu/crimjust/intern/whytake.htm Astin, A. (1996). The American Freshman: National norms for fall 1996. Los Angeles, CA: University of California at Los Angeles, Higher Education Research Institute. Astin, A. W., & Sax, L.J. (1998). How undergraduate are affected by service participation. Journal of college student development, 39 (3), 251-163. Astin, A. (2000). The American Freshmen: National norms for fall 1999. Los Angeles: University of California at Los Angeles, Higher Education Research Institute. Belbas, B., Gorak, K., & Shumer, R. (1993). Service Summary. Scotts Valley, CA: National Service-Learning Clearinghouse. Benson, J. (1998). Developing a strong program of construct validation: A test anxiety example. Educational Measurement: Issue and Practice, 10-22 Bickford, D. M., & Reynolds, N. (2002). Activism and service-learning: Reframing volunteerism as acts of dissent. Pedagogy, 2(2), 229-252. Boyer, E. L. (1987). College: The undergraduate experience in America. New York: Harper & Row. Boyer, E. L. (1994). Creating the new American College. Chronicle of Higher Education, 9, 48. Boyle-Baise, M., & Kilbane, J. (2000). What really happens? A look inside service- learning for multicultural teacher education. Michigan Journal of Community Service-Learning, 7, 54-64. 102 Bradford, M. (2005). Motivating students through project-based service learning. THE Journal, 32(6), 29-31. Bringle, R., & Hatcher, J. (1995). A service learning curriculum for faculty. Michigan Journal of Community Service Learning, 2, 112-122. Bringle, R., & Hatcher, J. (1998). Implementing service learning in higher education. Journal of Research and Education, 29(4), 31-41. Bringle, R.G., Games, R., & Malloy, E. A. (1999). Colleges and universities as citizens. Needham Heights, Mass: Allyn & Bacon. Bringle, R., & Hatcher, J. (2000a). Institutionalization of service-learning in higher education. Journal of Higher Education, 71(3), 273-290. Bringle, R., & Hatcher, J. (2000b). Meaningful measurement of theory-based service- learning outcomes: Making the case with quantitative research. Michigan Journal of Community Service Learning, Special Issue, 68-75. Brown, L. H., & Roodin. P. A. (2001). Service-learning in gerontology: An out-of- classroom experience. Educational Gerontology, 27, 89-103. Burns, L. (1998). Make sure it?s service-learning, not just community service. The Educational Digest, 64, 2, 38-41. Carpenter, B. W., & Jacobs, J. S. (1994). Service learning: A new approach in higher education. Education, 115(1), 97-98. Carver, R. L. (1997). Theoretical underpinnings of service-learning. Theory into Practice, 36(3), 143-149. 103 Cauley, K., & Canfield, A. (2001). Service learning: Integrating student learning and community service. Education for Health: Change in Learning & Practice, 14(2), 173-181. Chesler, M., & Scalera, C. V. (2000). Race and gender issues related to service-learning research. Michigan Journal of Community Service-Learning, special edition, 18- 27. Clark, C. T. (2002). Unfolding narratives of service learning: Reflections on teaching, literacy, and positioning in service relationships. Journal of Adolescent & Adult Literacy, 46(4), 288-297. Clasen, C., Dobbins. J., Hemphill, S., Jaballas, E., Walbroehl, G. (1993). Commission on National and Community Service. What you can do for your country. Washington, DC: Government Printing Office. Connor-Greene, P. A. (2002). Problem-based service learning: The evolution of a team project. Teaching of Psychology, 29(3), 193-197. Copeland, S. R., Hughes, C., Carter, E. W., Guth, C. Presley, J. A., Williams, C. R., & Fowler, S. E. (2004). Increasing access to general education. Remedial & Special Education, 25(6), 342-352. Cottrell, S. A., & Jones, E. A. (2002). A snapshot of scholarship of teaching and learning initiatives: Using assessment results to improve student learning and development. Assessment Update, 14(3), 6-7. Cowell, P. (2005). Practicing democracy: Early American authors in twenty-first-century communities. Early American literature, 40(2), 363-368. 104 Cruz, N., & Gilis, W. E. (2000). Where?s the community in service-learning research? Michigan Journal of Community Service-Learning, special edition, 28-34. Cummings, C. K. (2000). John Dewey and the rebuilding of urban community: Engaging undergraduates as neighborhood organizers. Michigan Journal of Community Service-Learning, 7, 97-109. Daynes, G., Esplin, P., & Kristensen, K. (2004). Learning as an epidemic: The tipping point, freshman academy, and institutional change. Perspectives: Policy & Practice in Higher Education, 8(4), 113-118. Definitions of service-learning. (2005). Retrieved May 05, 2006, from http://www.compact.org/faculty/definitions.html Definitions of service learning. (2006). Retrieved May 05, 2006, from http://www.campuscares.org/serve/definition2.html Dewey, J. (1916). Democracy and Education. New York: Free Press. Dewey, J. (1938). Experience and Education. New York: Collier Books. Dewey, J. (1946). The public and its problems (2 nd ed.) Chicago: Henry Holt and Company. Dietz, T. (1987). Methods for analyzing data from Delphi panels: Some evidence from a forecasting study. Technological forecasting and social change, 31, 79-85. Dijk, J. (1990). Delphi questionnaires versus individual and group interviews: a comparison case. Technological forecasting and social change, 37, 293-304. Driscoll, A. (2000). Studying faculty and service-learning: Directions for inquiry and development. Michigan Journal of Community Service-Learning, special edition, 35-41. 105 Dudderar, D., & Lois, T. (2003). Putting service learning experiences at the heart of a teacher education curriculum. Educational Research Quarterly, 27(2), 18-32. Easterling, D., & Rudell, F. (1997). Rationale, benefits, and methods of service-learning. Journal of Education for Business,73(1), 58-61. Eby, J. (1998). Why service-learning is bad. Retrieved July 2006, from http://www.messiah.edu/agape/learning. html#Resources Edwards, B., Mooney, L., & Heald, C. (2001). Who is being served? The impact of student volunteering on local community organizations. Nonprofit and Voluntary Sector Quarterly, 30(3), 444-461. Ehrlich, T. (1998). Reinventing John Dewey?s ?Pedagogy as a University Discipline.? The Elementary School Journal, 98(5), 489-509. Eldridge, M. (1996). Dewey?s faith in democracy as shared experience. Transactions of The Charles S. Peirce Society,32(1), p11. Emanoil, P. (2000). Real-world learning. Human Ecology, 28(3), 20-24. Enos, S. & Morton, K. (2003). Developing a theory and practice of campus-community partnerships. In B. Jacoby (Ed.), Building partnerships for service-learning (pp. 20-41). San Fransisco: Jossey-Bass. Erffmeyer, R. C., Erffmeyer, E. S., & Lane, I.M. (1986). The Delphi Technique: An empirical evaluation of the optimal number of rounds. Group & organization Studies, 11, 120-128. Eyler, J. S., & Giles, D. E. (1999). Where?s the learning in service-learning? San Francisco, CA: Jossey-Bass. 106 Eyler, J. S. (2000). What do we most need to know about the impact of service-learning on student learning. Michigan Journal of Community Service-Learning, special edition, 11-17. Eyler, J. S. (2001). Creating your reflection map. New Directions for Higher Education, 114, 35-44. Eyler, J. S., Giles, D. E., & Braxton, J. (1997). The impact of service-learning on college students. Michigan Journal of Community Service-Learning, 4, 5-15. Eyler, J. S., & Giles, D. E. (1999). Where?s the learning in service-learning? San Francisco: Jossey-Bass. Farrari, J. R. & Worrall, L. (2000). Assessment by community agencies: How ?the other side? sees service-learning. Michigan Journal of Community Service-Learning,7, 35-40. Fertman, C. I. (1994). Service-learning for all students. Bloomington, IN: Phi Delta Kappa educational foundation. Ferrari, J. R. (2000). Assessments by community agencies: How ?the other side? sees service-learning. Michigan Journal of Community Service-Learning, 7, 35-40. Fisher, R. (1978). Review and Criticism. The journal of Academic Librarianship, 65-70. Furco, A. (1996). Service-learning: A balanced approach to experiential education. In Corporation for National Service (Ed.), Expanding Boundaries: Service and Learning (pp. 2-6). Columbia, MD: Cooperative Education Association. Furco, A. (2002). Institutionalizing service-learning in higher education. Journal of Public Affairs, 6, 39-67. 107 Gallini, S. M. & Moely, B. E. (2003). Service-learning and engagement, academic challenge, and retention, Michigan Journal of Community Service-Learning,10, 5- 14. Gelmon, S. (2003). Assessment as a means of building service-learning partnerships. Building partnerships for service-learning. In B. Jacoby (Ed.), Building partnerships for service-learning (pp. 42-64). San Francisco, CA: Jossey-Bass. Gilchrist, L. Z., Mundy, M.E., Felton, P., & Shields, S.L. (2003). Care ethics, service- learning, and social change. Michigan Journal of Community Service-Learning,1, 51-58. Giles, D. (1991). Dewey?s theory of experience: Implications for service-learning. Journal of Cooperative Education, 27(2), 87-90. Giles, D. E., & Eyler, J. (1994a). The impact of a college community service laboratory on students? personal, social, and cognitive outcomes. Journal of Adolescence, 17, 327-339. Giles, D. E., & Eyler, J. (1994b). The theoretical roots of service-learning in John Dewey: Toward a theory of service-learning. Michigan Journal of Community Service Learning, 1, 77-85. Giles, D. E., & Eyler, J. (1998). A service learning research agenda for the next five years. New Directions for Teaching & Learning, 73, 65-72. Giuliano, Traci A. (2001). Student presentations of the life and work of prominent social psychologists. Teaching of Psychology, 28(4), 269-271. 108 Gordon, T. & Pease, A. (2006). RT Delphi: An efficient, ?round-less? almost real time Delphi Method. Technological forecasting & social change, 73, 321-333 Gose, B. (1997). Many colleges move to link courses with volunteerism. The Chronicle of Higher Education, 44(12), A45-46. Gray, M. J. (2000). Making the commitment to community service. About Campus, 5(2), 19-24. Gray, M., Ondaatje, E. H., Fricker, R. D., & Geschwind, S. A. (2000). Assessing service- learning: Results for a survey of LEARN and Serve America, Higher Education. Change, 32(2), 30-39. Groves, R. M. (1987). Research on survey data quality. Public opinion quarterly, 51, 156-172. Gutek, G. L. (1988). Philosophical and ideological perspectives on education. Englewood Cliffs, NJ: Prentice Hall. Hair, J. F; Anderson, R.O.; Tatham, R. L.; Black, W.C. (1998). Multivariate data analysis (5 th ed). Upper Saddle River, NJ: Prentice Hall. Hammond, C. (1994). Integrating Service and Academic Study: Faculty Motivation and Satisfaction in Michigan Higher Education. Michigan Journal of Community Service Learning, 1, (1), 21-28. Hatcher, J. A. (1997). The moral dimension of John Dewey?s philosophy: Implication for undergraduate education. Michigan Journal of Community Service Learning, 4, 22-29. Harkavy, I., & Benson, L. (1998). De-Platonizing and democratizing education as the bases of service learning. New Direction for Teaching and Learning, 73, 11-20. 109 Hartlep, K. L., & Forsyth, G. A. (2000). The effect of self-reference on learning and retention. Teaching of Psychology, 27(4), 269-271. Hodge, G., Lewis, T., Kramer, K., & Hughes, R. (2001). Collaboration for excellence: Engaged scholarship at Collin County Community College. Community College Journal of Research & Practice, 25(9), 675-690. Holland, B. A. (2001). A comprehensive model for assessing service-learning and community-university partnerships. In M. Canada and B.W. Speck (eds), Developing and implementing service-learning programs. New directions for higher education, no 114. San Francisco, CA: Jossey-Bass. Hollis, S. A. (2002). Capturing the experience: Transforming community service into service learning. Teaching Sociology, 27, 360-372. Hollies, S. H. (2004). Blaming me, blaming you: Assessing service-learning and participants? tendency to blame the victim. Sociological Spectrum, 24, 575-600. Howard, J. P. F., Gelmon, S. B., & Giles, D. E. (2000). From yesterday to tomorrow: Strategic directions for service-learning research. Michigan Journal of Community Service-Learning, special edition, 5-10. Jackson, F.M. (1993). Evaluating service-learning. In T. Y. Kupiec (Ed.), Rethinking Tradition (pp. 129-135). Providence, RI: Campus Compact. Johnson, R. M. (2005). A case study in service learning. Topics in Clinical Nutrition, 20(4), 351-356. Jacoby, B. (2003). Fundamentals of service-learning participants. In B. Jacoby (Ed.), Building Partnerships for Service-Learning. San Francisco, CA: Jossey-Bass. 110 Jacoby, B., & Associates. (1996). Service-Learning in Higher Education: Concepts and practices. San Francisco, CA: Jossey-Bass. Johnson Foundation. (1989). Wingspread special report: Principles of good practice for combining service and learning. Racine, WI: Author. Johnson, R. M. (2005). A case study in service learning. Topics in Clinical Nutrition, 20(4), 351-356. James, W. (1910). A Pluralistic Mystic. The Hibbert Journal, 8, 739-759. Jorge, E. (2003). Outcomes for community partners in an unmediated service-learning program. Michigan Journal of Community Service-Learning, 1, 28-38. Kahne, J., & Westheimer, J. (1996). In the service of what? The politics of service-learning. Phi Delta Kappan,77, 592-599. Katula, R. A., & Threnhauser, E. (1999). Experiential education in the undergraduate curriculum. Communication Education, 48(3), p238. Kendall, J. (1990). Combining service and learning: An introduction. Combining service and learning: A resource book for community and public service. 1. I., 1-33. Raleigh, NC: National Society for Internships and Experiential Education. Kendall, J., Duley, J., Little, T., Permaul, J., & Rubin, C. S. (1990). Increasing faculty involvement. Combining service and learning: A resource book for community and public service. l. I., pp. 137-160. Raleigh, NC: National Society for Internships and Experiential Education. Kendler, P. B. (2000). Bringing real world interaction into the classroom. Curriculum Administrator, 36(10), 55-58. 111 Keller, J., Nelson, S., & Wick, R. (2003). Care ethics, service-learning, and social change. Michigan Journal of Community Service-Learning, 1, 39-50. Kezar, A. (2002). Assessing community service learning: Are we identifying the right outcomes? About Campus, May-June, 14-24. Kezar, A. (2005). Moving from I to We. Change, 37(6). 50-57. Kielsmeier, J. C., Scales, P. C., Roehlkepartain, E. C., & Neal, M. (2004). Community service and service-learning in public schools. Reclaiming Children & Youth, 13(3), 138-143. Kilpatrick, W. H. (1918). The project method. Teachers college record, 19, 319-335. Kirk, R., & Riedle, L. (2005). Creating lifelong learners and lifelong givers. The Delta Kappa Gamma Bulletin71(4),32-36. Kivinen, O., & Ristel?, P. (2002). Even higher learning takes place by doing: From postmodern critique to pragmatic action. Studies in Higher Education, 27(4), 419- 430. Kirkpatrick, M. K., & Brown, S. (2004). Narrative pedagogy: Teaching Geriatric content with stories and the ?Make a Difference? project (MADP). Nursing Education Perspectives, 25(4), 183-187. Kline, R. B. (1998). Principles and practice of structural equation modeling. New York, NY: Guilford Press. Kolb, D. A. (1983). Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall. Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice-Hall. 112 Kraft, R. J. (1996). Service learning: An introduction to its theory, practice, and effects. Education and urban society, 27(2), 131-159. Kraft, R. J., & Swadener, M. (Eds.). (1994). Building community: Service learning in the academic disciplines. Denver, CO: Colorado Campus Compact. Kretchmar, M. D. (2001). Service learning in a general psychology class: Description, preliminary evaluation, and recommendations. Teaching of Psychology, 28(1), 5- 10. Kuh, G. D., Douglas, K. B., Lund, J. P., & Ramin-Gyurnek, R. (1994). Student learning outside the classroom: Transcending artificial boundaries. Washington, DC: George Washington University School of Education and Human Development. Lakes, R. D. (2001). The social foundations classroom: Volunteerism in social foundations courses. Educational Studies, 32(1), 53-59. Layfield, K. D., & Flagg, S. (2004). The impact of service-learning: A quasi- experimental assessment of student performance in an introductory microcomputer course. Journal of Southern Agricultural Education Research, 54(1), 147-155. Leonard, A. (2004). Service learning as a transgressive pedagogy. Cross Currents, 54(2), 61-72. LeSourd, S. J. (1997). Community service learning. Theory Into Practice, 36(3), 134- 135. Long, A. B., Larsen, P., Hussey, L., & Travis, S. S. (2001). Organizing, managing, and evaluating service-learning projects. Educational Gerontology, 27(1), 3-21. 113 Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychology reports, 3, 635-694 Manzo, K. K. (2005). Portland, Ore., District selected for Chinese-Language program. Education Week, 25(5), 4-5. Marchel, C. A. (2003). The path to altruism in service-learning classes: Big steps or a different kind of awkwardness? Michigan Journal of Community Service- Learning, 1, 15-27. Markus, G. B., Howard, J. & King, D. (1993). Integrating community service learning and classroom instruction enhances learning: Results from an experiment. Educational evaluation and policy analysis, 15, 410-419. Marlin?Bennett, R. (2002). Linking experiential and classroom education: Lessons learned from the American University ? Amnesty International USA Summer Institute on Human Rights. International Studies Perspectives, 3(4), 384-395. Martin, J. (2003). The education of John Dewey. New York: Columbia University press. Marullo, S., & Edwards, B. (2000). From charity to justice: The potential of university- community collaboration for social change. American Behavioral Scientist, 43, 895-912. Masucci, M., & Renner, A. (2004). Reading the lives of others: The Winton homes library project a culture studies analysis of critical service-learning for education. High School Journal, 84(1), 36-47. McCluskey-Fawcett, K. (1992). Using community service to teach developmental psychology. Teaching of Psychology, 19(3), 150-152. 114 McDonald, T. W., Caso, R., & Fugit, D. (2005). Teaching and learning operant principles in animal shelters: Perspectives from Faculty, students, and shelter staff. Journal of Instructional Psychology, 32(4), 310-321. McKenna, M.W., & Rizzo, E. (1999). Student perceptions of the "learning" in service- learning courses. Journal of Prevention & Intervention in the Community, 18, 111-123. McNally, M. D. (2004). Indigenous pedagogy in the classroom: A service learning model for discussion. American Indian Quarterly, 28(3/4), 604-617. Mendel-Reyes, M. (1998). A pedagogy for citizenship: Service learning and democratic education. New Direction for Teaching and Learning, 73, 31-38. Messick , S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational researcher, 23 (2), 12-23. Meyers, L. S., Gamst, G., & Guarino, A. J. (2005) Applied multivariate research: Design and interpretation. Thousand Oaks. CA: Sage Publication, Inc. Michalec, P. (1994). Service learning and the multicultural education of pre-service teachers. In R. J. Kraft & M. Swadener (Eds.), Building community: Service learning in the academic disciplines (pp. 169-176). Denver, CO: Colorado Campus Compact. Moely, B. E., Mercer, S. H., Ilustre, V., Miron, D., & McFarland, M. (2002). Psychometric properties and correlates of civic attitudes and skills questionnaire (CASQ): A measure of students? attitudes related to service-learning. Michigan Journal of Community Service-Learning, 1, 28-38. 115 Morgan, W., & Streb, M. (2001). Building citizenship: how student voice in service- learning develops civic values. Social Science Quarterly,82(1), 154-169. Moloney, J., Dion, S., Hickey, C., & Siccama, C. (2004). Transforming graduate students into leaders through service learning. Academic Leader, 20(11), 4-5. Moore, S. D., Brennan, S., Garrity, A. R., & Godecker, S. W. (2000). Winburn Community Academy: A University-Assisted community school and professional development school. PJE: Peabody Journal of Education, 75,3, 33-50. Morton, K. (1995). The irony of service: Charity, project and social change in service- learning. Michigan Journal of Community Service-Learning,2, 2, 19-32. Moser, J. M., & Rogers, G. E. (2005). The power of linking service to learning. Tech Directions, 64, 7, 18-21. Myers-Lipton, S. J. (1998). Effect of a comprehensive service-learning program on college students? civic responsibility. Teaching Sociology, 26, 243-258. National Community Service Act of 1990. (1990). National and Community Service Trust Act of 1993. (1993). Ngai, S. S. (2006). Service-Learning, personal development and social commitment: A case study of university student in Hong Kong. Adolescence,41,165-175. Nunnally, J. (1978). Psychometric theory (2 nd . Ed.). New York: McGraw-Hill. O'Bryne, K. (2001). How professors can promote service-learning in a teaching institution. New Directions for Higher Education, 114, 79-87. O?Toole, T. P., Hanusa, B. H., Gibbon, J. L., & Boyles, S. H. (1999). Experiences and attitudes of residents and students influence voluntary service with homeless populations. JGIM: Journal of General Internal Medicine, 14(4), 211-216. 116 Parker-Gwin, R., & Mabry, J. B. (1998). Service learning as pedagogy and civic education: comparing outcomes for three models. Teaching sociology, 26, 276- 291. Pedhazur, E. J. & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Lawrence Erlbaum Associates. Phillips, D.L. & Clancy, K. J. (1972). Some effects of social desirability in ?survey studies?. The American Journal of Sociology, 77, (5), 921-940. Phoenix, T., Miller, K., & Schleien, S. (2002). Better to give than receive. Parks & Recreation, 37,10, 26-32. Pompa, L. (2002). Service-Learning as crucible: Reflections on immersion, context, power, and transformation. Michigan Journal of Community Service-Learning, 7, 67-76. Porter, M. & Monard, K. (2001) Ayni in the Global Village: Building Relationships of Reciprocity through International Service-Learning. Michigan Journal of Community Service-Learning,8,1, 5-17. Praetzel, G. (1999). Pedagogical Recommendations for Internationalizing the Undergraduate Business Curriculum. International Advances in Economic Research, 5, (1), 137-146. Prentice, M., & Garcia, R. M. (2000). Service learning: The next generation in education. Community College Journal of Research & Practice, 24,1, 19-26. Price, C. A., Zavotka, S. L., & Teaford, M. H. (2004). Implementing a University community retail partnership model to facilitate community education on universal design. Gerontologist, 44,5, 697-702. 117 Prietula, M.J., Feltovich, P. J., & Marchak, F. (2000). Factors influencing analysis of complex cognitive tasks: a framework and example from industrial process control, Hum. Factor, 42, 1 56-74. Reifsteck, J. (2002). Straight from the heart: Integrating service values with the study of psychology. North American Journal of Psychology, 4,3, 489-496. Rhoads, R. A. (2000). Democratic citizenship and service-learning: Advancing the caring self. New directions for teaching and learning. 82, 37-44. Riggs, W. E. (1983). The Delphi Technique: An experimental evaluation. Technological Forecasting and Social Change, 23, 89-94. Rowe, G., Wright, G., & McColl, A. (2004). Judgment change during Delphi-like procedures: The role of majority influence, expertise, and confidence. Technological Forecasting and Social Change, 72, 377-399. Rockquemore, K. A. & Schaffer, R. H. (2000). Toward a theory of engagement: A cognitive mapping of service-learning experiences. Michigan Journal of Community Service-Learning, 7, 14-25. Ropers-Huilman, B., Crawile, L., & Lima, M. (2005). Service-learning in engineering: A valuable pedagogy for meeting learning objectives. European Journal of Engineering Education, 30(2), 155-165. Rubin, S. G. (1990). Service Learning: Education for democracy. Liberal Education, 76, 12-17. Saltmarsh, J. (1996). Education for critical citizenship: John Dewey?s contribution to the pedagogy of community service-learning. Michigan Journal of Community Service-Learning, 3, 13-21. 118 Saltmarsh, J. (1998). Exploring the meanings of community/university partnerships. NSEE Quarterly, Summer, 6-7 & 21-22. Sapp, D. A., & Crabtree, R. (2002). A laboratory in citizenship: Service learning in the technical communication classroom. Technical Communication Quarterly, 11(4), 411-431. Schmidt, M. E., Marks, J. L., & Derrico, L. (2004). What a difference mentoring makes: Service learning and engagement for college students. Mentoring & Tutoring: Partnership in Learning, 12(2), 205-217. Schulze, P. A., & Witt, S. (2004). Everybody wins: How service learning benefits students and the community. Education & Society, 22(3), 35-45. Service learning. (2006). Retrieved May 05, 2006, from http://westcgi.west.asu.edu/publicaffairs/volunteerservices/service.cfm Service-learning definitions. (n.d.). Retrieved May 05, 2006, from http://www.txcsl.org/definitions.html Service-learning is ? (n.d.). Retrieved May 15, 2006, from http://www.servicelearning.org/welcome_to_service-learning/service- learning_is/index.php Sheffield, E. C. (2004). Service-learning as a democratic, public affair: Some necessary philosophical ingredients. Journal of Public Affairs, 7(1), 1-15. Sheffield, E. C. (2005). Service in service-learning education: the Need for philosophical understanding. The High School Journal, 89, 46-53. Shumer, R. (1993). Describing service-learning: A Delphi study. Minneapolis, MN: University of Minnesota. 119 Simons, L., & Cleary, B. (2005). Student and community perceptions of the ?value added? for service-learners. Journal of Experiential Education, 28(2), 164-188. Singer, J. A., King, L. A., Green, M. C., & Barr, S. (2002). Personal identity and civic responsibility: ?Rising to the Occasion? Narratives and generativity in community action student interns. Journal of Social Issues, 58(3), 535-556. Skilton-Sylvester, E., & Erwin, E. K. (2000). Creating reciprocal learning relationships across social-constructed borders. Michigan Journal of Community Service- Learning, 7, 65-75. Stanton, T. (1987). Service-learning: grouping toward a definition. Experiential education, 12, 1, Raleigh, NC: National Society for Internship and Experiential Education, 2, 4. Stanton, T. K. (1990). Service-learning: Groping toward a definition. In J.C. Kendall and Associates (Eds.), Combining Service and Learning: A Resource Book for Community and Public Service, Vol. I, (pp 65-68). Raleigh, NC: National Society for Internships and Experiential Education. Stanton, T. K. (1991). Liberal arts, experiential learning and public service: Necessary ingredients for socially responsible undergraduate education. Journal of Cooperative Education, 27(2), 55-68. Stanton, T. K. (1994). The experience of faculty participations in an instructional development seminar on service-learning. Michigan Journal of Community Service Learning, 1(1), 7-20. 120 Stanton, T. K., Giles, D. E., & Cruz, N. I. (1999). Service-learning: A movement?s pioneers reflected on its origins, practice, and future. San Francisco, CA: Jossey- Bass. Steffes, J. S. (2004). Creative powerful learning environments beyond the classroom. Change, 36(3), 46-50. Steinke, P., & Buresh, S. (2002). Cognitive outcomes of service-learning: Reviewing the past and glimpsing the future. Michigan journal of community service-learning, 1, 5-14 Stevens, J.P. (2002). Applied multivariate statistics for the social science (4 th ed.). Hillsdale, NJ: Erlbaum. Strage, A. (2000). Service-learning: Enhancing student learning outcomes in a college- level lecture course. Michigan Journal of Community Service-Learning, 7, 5-13. Strage, A. (2004). Long-term academic benefits of service-leaning: When and where do they manifest themselves? College Student Journal, 38(2), 257-261. Strand, K. (2000). Community-based research as pedagogy. Michigan Journal of Community Service-Learning, 7, 85-96. Stukas, A. A., & Dunlap, M. R. (2002). Community involvement: Theoretical approaches and educational initiatives. Journal of Social Issues, 58(3), 411-427. Subranmony, M. (2000). The relationship between performance feedback and service- learning. Michigan Journal of Community Service-Learning, 7, 46-53. The National and Community Service Act of 1990. (1990.). Retrieved May 15, 2006, from http://www.nationalservice.gov/about/role_impact/history.asp The National and Community Service Trust Act of 1993. (1993). Retrieved May 15, 121 2006, from http://www.nationalservice.gov/about/role_impact/history.asp Thompson, R. J., & Serra, M. (2005). Use of course evaluations to assess the contributions of curricular and pedagogical initiatives to undergraduate general education learning objectives. Education, 125(4), 693-701. Troppe, M. (1996). Two cases of institutionalizing service-learning: How campus climate affects the change processes. Providence, RI: Campus Compact. Tovey, J. (2001). Building connections between industry and university: Implementing an internship program at a regional university. Technical Communication Quarterly, 10(2), 225-239. Tucker, M. L., McCarthy, A. M., Hoxmeier, J. A., & Lenk, M. M. (1998). Community service learning increases communication skills across the business curriculum. Business Communication Quarterly, 61(2), 88-99. Turoff, M. (1970). The decision of a policy Delphi. Technological Forecasting and social change, 21, 149-171. Valerius, L., & Hamilton, M. L. (2001). The community classroom: Service to learn and learning to serve. College Student Journal, 35(3), 339-345. Varlotta, L. (2000). Service as text: Making the metaphor meaningful. Michigan Journal of Community Service-Learning, 7, 76-84. Vickers, M., Harris, C., & McCarthy, F. (2004). University-community engagement: Exploring service-learning options within the practicum. Asia-Pacific Journal of Teacher Education, 32(2), 129-141. Vogelgesang, L. J., & Astin, A. W. (2000). Comparing the effects of community service and service-learning. Michigan Journal of Community Service-Learning, 7, 25-34. 122 Von Dras, D. D., & Miller, K. M. (2002). Learning outside the classroom: The undergraduate gerontology internship. Educational Gerontology, 28(10), p881- 894. Wade, R. C. (1997). Empowerment in student teaching through community service learning. Theory Into Practice, 36(3), 184-191. Waterman, A. S. (1997). Service-Learning: Applications from the research. Mahwah, NJ: Lawrence Erlbaum Association. Weinberg, A. S. (2002). The university: An agent of social change? Qualitative Sociology, 25(2), 263-272. Wells, C. V., & Grabert, C. (2004). Service-learning and mentoring: Effective pedagogical strategies. College Student Journal, 38(4), 573-578. Wentland, D. (2004). A guide for determining which teaching methodology to utilize in economic education: Trying to improve how economic information is communicated to students. Education, 124(4), 640-648. Wielkiewicz, R. M., Prom, C. L., & Loos, S. (2005). Relationships of the Leadership Attitudes and Beliefs Scale with student types, study habits, life-long learning, and GPA. College Student Journal, 39(1), 31-44. Williams, N. R., & Reeves, P. M. (2004). MSW students go to burn camp: Exploring social work values through service-learning. Social Work Education, 23(4), 383- 398. Woundenberg, F. (1991). An evaluation of Delphi. Technological forecasting and social change, 40, 131-150. 123 Zieren, G. R. & Stoddard, P.H. (2004). The historical origins of service-learning in the 19 th and 20 th centuries: The transplanted and indigenous traditions. In B. W. Speck, & S. L., Loppe (Eds.), Service-Learning: History, Theory, and Issues (pp. 23-34). Westport, CT: Greenwood. Zlotkowski, E. (1995). Does service-learning have a future? Michigan Journal of Community Service-Learning,2,123-133. Zlotowski, E. (1996). Linking service-learning and the academy. Change, 28 (1), 20-27. Zlotowski, E. (1997). Service-learning and the process of academic renewal. Journal of Public Service & Outreach, 21, 80-87 Zlotkowski, E. (1999). Pedagogy and engagement. In R. Bringle, R. Games, & E. A. Malloy (Eds.), Colleges and universities as citizens (pp. 66-120). Boston, MA: Allyn & Bacon. Zolingen, S & Klaassen, C. (2003). Selection processes in a Delphi study about key qualifications in senior secondary vocational education. Technological forecasting & social change, 70, 317-340 124 APPENDICES 125 APPENDIX A Assessing Service-Learning Programs in Higher Education Institutions 126 Assessing A Service-Learning Program in Higher Education Institutions Respond to the following items by selecting the number that best describes your experiences of conducting a service-learning program. Faculty Practitioner Domain Not Aagree ----------------------Very At all Agree 1. My cooperating site is a good match to the course content of my service-learning class. 1 2 3 4 5 2. My students? professional development is important to me when I conduct my service-learning program. 1 2 3 4 5 3. My students? acquisition of course content is important to me when I operate my service-learning program. 1 2 3 4 5 4. My pedagogical practice is an important factor in my decision to integrate service into my course design. 1 2 3 4 5 5. I provide class discussion on service-learning issues. 1 2 3 4 5 Community Domain 6. I discuss the difficulties and needs faced by the community agency with my community partner. 1 2 3 4 5 7. I share my needs for conducting a service-learning program with my community partner. 1 2 3 4 5 8. I take the needs of the community partner into consideration when I design my service-learning program. 1 2 3 4 5 9. I discuss service-learning activities that will be implemented with my community partners at the designing stage. 1 2 3 4 5 10. I ask my community partner to coach my students during on-site service-learning activity when necessary. 1 2 3 4 5 11. I am aware of the assets of the community partner. 1 2 3 4 5 12. I appreciate my community partners? expression of their concerns on the ongoing service-learning activities. 1 2 3 4 5 13. I regularly communicate with my community partner regarding service-learning issues. 1 2 3 4 5 127 14. I formally introduce my community partner to my students. 1 2 3 4 5 15. My community partner provides on-site supervision for service-learning students from the university to maintain the quality of service provided. 1 2 3 4 5 16. I make program modifications in response to suggestions from my community partner. 1 2 3 4 5 17. After service-learning, my community partner gains insight about resolving problems they encountered. 1 2 3 4 5 18. My community partners gain insight in relation to how to improve their situation after the service- learning partnership. 1 2 3 4 5 19. My community partners express positive comments on the service-learning partnership after the program is concluded. 1 2 3 4 5 20. My community partners express their goals for our service-learning program at the planning stage. 1 2 3 4 5 Student Domain 21. My students understand the requirements for their service-learning activities. 1 2 3 4 5 22. My students are able to apply their service-learning experiences to facilitate their understanding of information presented in the classroom. 1 2 3 4 5 23. My students demonstrate a comprehensive understanding of problems faced by the community after participating in service-learning activities. 1 2 3 4 5 24. My students are able to apply alternative explanations for difficulties encountered by individuals after their service-learning experiences. 1 2 3 4 5 25. My students are more aware of the democratic procedure in the community after participating in service-learning activities. 1 2 3 4 5 26. My students are able to come up with effective strategies to resolve problems that arose unexpectedly after service-learning experiences. 1 2 3 4 5 27. My students are more able to listen to different opinions after I integrate service-learning into my course design. 1 2 3 4 5 28. My students express more concerns about their chosen profession after participating in service- learning activities. 1 2 3 4 5 128 APPENDIX B IRB APPROVAL FORM OF DELPHI STUDY 129 130 APPENDIX C IRB APPROVAL FORM OF ON-LINE SURVEY STUDY 131