USING HYPERLINKED SCAFFOLDING TO SUPPORT STUDENT WORK WITH TEXT-BASED SOURCE DOCUMENTS AS PART OF A PROBLEM-BASED HISTORICAL INQUIRY LESSON Except where reference is made to the work of others, the work described in this dissertation is my own or was done in collaboration with my advisory committee. This dissertation does not include proprietary or classified information. Linda A. Mitchell Certificate of Approval: Andrew M. Weaver John W. Saye, Chair Professor Professor Curriculum and Teaching Curriculum and Teaching David Carter Jada Kohlmeier Associate Professor Assistant Professor History Curriculum and Teaching Joe F. Pittman Interim Dean Graduate School USING HYPERLINKED SCAFFOLDING TO SUPPORT STUDENT WORK WITH TEXT-BASED SOURCE DOCUMENTS AS PART OF A PROBLEM-BASED HISTORICAL INQUIRY LESSON Linda A. Mitchell A Dissertation Submitted to the Graduate Faculty of Auburn University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Auburn, Alabama December 15, 2000 iii USING HYPERLINKED SCAFFOLDING TO SUPPORT STUDENT WORK WITH TEXT-BASED SOURCE DOCUMENTS AS PART OF A PROBLEM-BASED HISTORICAL INQUIRY LESSON Linda A. Mitchell Permission is granted to Auburn University to make copies of this dissertation at its discretion, upon request of individuals or institutions and at their expense. The author reserves all publication rights. _________________________________ Signature of Author _________________________________ Date of Graduation iv DISSERTATION ABSTRACT USING HYPERLINKED SCAFFOLDING TO SUPPORT STUDENT WORK WITH TEXT-BASED SOURCE DOCUMENTS AS PART OF A PROBLEM-BASED HISTORICAL INQUIRY LESSON Linda A. Mitchell Doctor of Philosophy, December 15, 2006 (M.S., Jacksonville University, 2001) (B.S., Jacksonville University, 1992) 271 Typed Pages Directed by John W. Saye Although many social studies education experts recommend the use of resource documents as part of history education, many social studies teachers are reluctant to utilize these historical resources. One reason for this reluctance is the belief that students will have difficulty understanding and using the resources in a meaningful way. This is especially true for students with reading skills that are below average. When teachers do not regularly include the use of resource documents in their history classes, students are iii unable to develop first-hand knowledge of the past and instructional methods such as inquiry become difficult to successfully implement. In order to allow teachers to utilize historical resource documents techniques must be employed that enable all students to successfully read and use the documents within the classroom setting. Including hyperlinked scaffolding may be one way to provide the definitional, historical background, and metacognitive assistance many students need as they read complex historical resource documents. This study examines the ways students with various levels of reading skills utilize scaffolded historical resource documents within a problem-based inquiry lesson. The study was completed in two seventh grade social studies classrooms using a design experimental method. Students read three resource documents that included hyperlinked scaffolding and data analysis was completed using qualitative methods. Differences were seen in the ways above average, average, and below average readers utilized the hyperlinked scaffolding as they read the online documents. Many of the findings suggest that students utilize these texts in ways that are similar to the ways they utilize print text. Average readers, however, tended to become frustrated with the processes involved in both the reading and the inquiry-based lesson. Data suggests that the use of the hyperlinked scaffolding was successful and assisted the students in utilizing the documents as part of the problem-based inquiry lesson. Another aspect of the study indicated that the teachers involved in this study and in a pilot study were able to successfully anticipate the areas where students would need assistance as they read the documents. iii ACKNOWLEDGEMENTS The author would like to thank Dr. John W. Saye for his assistance throughout the dissertation process. Without his support, suggestions, and encouragement the process would have been more difficult and less successful. iv Style manual used is Publication Manual of the American Psychological Association, 5 th Edition. Computer software used is Microsoft Word X for Mac. TABLE OF CONTENTS v LIST OF TABLES????????????????????????? x LIST OF FIGURES??????????????????????..?? xi I. INTRODUCTION?????????????????????? 1 Introduction???????????????????????..? 1 Research Question????????????????????.?... 9 II. REVIEW OF LITERATURE???????????????...?? 12 Introduction???????????????????????..? 12 Inquiry Learning??????????????????????.. 14 Problem-based Historical Inquiry???????????????? 21 Complexity of historical source documents???????????? 22 Use of textbooks?????????????????????..? 26 Use of Source Documents??????????????????? 32 Issues related to student literacy???????????????..? 38 Expert v. Novice behavior??????????????????? 52 Scaffolding??????????????????????.??.. 58 Use of technology in education????????????????? 66 Using technology to provide scaffolding?????????.???? 71 Conclusion????????????????????????? 80 III. Methodology???????????????????????? 82 Introduction?????????..??????????????? 82 Project design??????????????????...????? 83 Development of research questions???????????????. 85 Study timeline???????????????????????.. 87 Project setting and description of participants???????????. 88 Study Phases????????????????????????. 91 Phase I: Development of Design Intervention????????. 92 Phase II: Coding of Documents?????????????? 103 Phase III: Analysis and preparation of hyperlinked source documents??????????????????????. 106 Phase IV: Study lesson?????????????????. 114 Phase V: Interviews???????????????.???.. 117 iii Data analysis procedures??????????.??????.???. 119 Study limitations?????????????.??????.???. 123 Conclusion?????????????????????????. 126 IV. FINDINGS??????????????????????..??.. 128 Introduction?????????????????????.???.. 128 Preliminary analysis to refine design intervention?????????.... 128 Final analysis: Study questions????????????????.? 146 Conclusion????????????????????????? 175 V. SUMMARY, IMPLICATIONS, FURTHER STUDY?????.??? 179 Summary of findings????????????????????? 179 Implications???????????????????.?????. 187 Suggestions for further study????????????...?????.. 194 Conclusion????????????????????????? 196 REFERENCES??????????????????????????. 198 APPENDICES????????????????????????.?.? 217 Appendix A: The Code of Hammurabi, excerpted text read by students?. 217 Appendix B: Justinian Code, excerpted text read by students?????. 220 Appendix C: Magna Carta, excerpted text read by students????...? 224 Appendix D: Introductory page from student CD?????????? 226 Appendix E: Group work problem descriptions??????????.. 228 Appendix F: Teacher Interview Guide??????????????. 230 Appendix G: Student Interview Guide??????????????. 232 Appendix H: Reading Guide for Code of Hammurabi????????. 234 Appendix I: Reading Guide for Justinian Code??????????... 237 Appendix J: Reading Guide Magna Carta????????????.. 239 Appendix K: Group work decision-making scaffold.????????.. 241 Appendix L: Description of pilot study?????????????? 243 Appendix M: Copy of study lesson plan?????????????.. 251 Appendix N: Magna Carta, excerpted text at the end of Phase I???? 253 Appendix O: Hyperlinked scaffolds: Magna Carta?????????. 255 Appendix P: Original Group Problem descriptions?????????. 256 Appendix Q: Scaffolding and document alteration?????????.. 258 LIST OF TABLES Table 1: Research project timeline?????????????...???.? 89 Table 2: Data sources???????????????????????. 92 Table 3: Concepts within the legal codes emphasized in the study lesson.??... 99 Table 4: Readability statistics for excerpted documents: Phase 1??????. 130 Table 5: Student coding for historical background information??????? 137 Table 6: Comparison of expert and student coding????????...??? 140 Table 7: Comparison of readability statistics??????...???????. 144 Table 8: Comparison of metacognitive scaffolding responses by reading skill level??????????????????????????.. 169 iii iii LIST OF FIGURES Figure 1: Concept map?????????????...?????.???? 97 Figure 2: Example of source documents with hyperlinks??????..??? 110 1 CHAPTER 1: INTRODUCTION Introduction Inquiry into social or historical problems has been advocated as one way to interest learners and develop the competencies necessary for participation in a pluralistic, democratic society (Newmann, 1992; Sexias, 2000; Thornton, 1998). Despite this, teachers are often reluctant to adopt this method of instruction for a variety of reasons, including the fear that students will be unable to adequately make use of the complex texts that make up much of the material used in this type of instruction. This fear is not unfounded. Reading assessments across the country indicate that only a small percentage of students read at an advanced or proficient level (Donahue, Daane, & Griss, 2003), making it little wonder that teachers are reluctant to attempt an instructional strategy that requires students to make use of an array of complex social texts. Problem-based historical inquiry uses the problems societies have faced in the past and continue to face today as a basis for history instruction (Newmann and Oliver, 1970; Saye and Brush, 2004a; Shaver, 1992). Societal problems are posed within a specific historical context and students use the search for information and possible solutions to the problem as a catalyst for learning about the past, as well as understanding of how the knowledge and skills they are learning apply to the present. Students must use a variety of complex source documents as part of this type of instruction because in 2 order for students to authentically struggle with problems in history they must understand these problems from the points of view of the societies and individuals involved. If our goal is to have students authentically consider historical issues the use of comprehensive history textbooks does not solve the problems related to the use of source documents. To alleviate the need for numerous source documents, comprehensive history textbooks combine multiple stories and points of view into a single unified narrative that can be contained in a relatively small space. In doing this, individual historical voices are seldom heard, opinions and biases are largely eliminated, and differing accounts are combined in a comprehensive story that is acceptable to the largest number of people (Wade, 1993). While these texts can serve an important informational purpose in history education, they tend to leave students disengaged from the study of history and do not provide the in-depth understanding of historical events that can be achieved through the use of multiple authentic historical source documents (Barton and Levstik, 2003; Paxton, 1999). Use of authentic source documents allows students to experience past events through the voices of diverse historical characters allowing them to develop empathy with the individuals on all sides of a historical conflict or event. This also allows students to practice skills required for effective democratic citizenship, such as genuine consideration of opposing points-of-view, effective social discourse, and analogical reasoning. Historical source documents include the diverse range of material related to the human experience including, but not limited to, print text. Although any material produced by humans can be considered a historical source, the purpose behind a text?s creation and the ways in which it will be used are important considerations for the teacher 3 planning authentic instructional activities. Historical source documents such as journals, tax records, literature, maps, and religious texts are created for purposes other than instruction and their use requires students to understand subtext, infer meaning, and grasp the complexities inherent in material produced for an audience other than the student. Textbooks and other didactic texts are differentiated from source documents because they are specifically written with students as the intended audience and cannot authentically provide students with an understanding of the complexities involved in a historical event or issue. For this reason the letters a soldier writes to family and friends from a war zone would be considered source documents related to the war, whereas a textbook account of the war would not be a source document. Although both provide information about the war, the soldier?s letters provide the reader with first-hand understanding of personal experiences and beliefs, while the textbook tells about events of the war in a way largely disassociated from individual understanding or experiences. Determining what is a source document and what is not can be complicated by the fact that a text could be a source document in one context and a didactic text in another. The history textbook that tells about a war is didactic in that context, but it could be a source document if it is used to understand the beliefs individuals had about the war in the time period when it was written. In actuality, although some texts are recognizable as source documents in all instances, any text might be a source document in the proper context. The teacher planning an authentic instructional activity must identify texts that will be effective source documents and must then find ways to assist students as they deal with the inherent complexities of these texts. 4 The complexity of source documents and the issues this complexity raises for a teacher using them in the classroom is a serious concern. Source documents may be complex for a variety of reasons including complex language. Texts written in the distant past may include arcane words and phrases that make understanding difficult for modern readers, and even modern text may contain complicated or specialized language, such as legal terms, that can be difficult. Some texts that have no unfamiliar terms or phrases may still be written in a manner that is difficult and make independent reading impossible for many students. Source documents may require the reader to access prior knowledge to gain complete understanding, and when this knowledge is inadequate understanding will be impossible (Afferback & VanSledright, 2001). Adding to the complexity of source documents is the need to situate the text within its current use, such as in an instructional activity. A teacher must consider text complex when it includes any of these individual or combined criteria for a student, group of students, or an entire class. In addition to the problems posed by complex text and insufficient literacy skills, reading historical texts includes the added complexity of social literacy. Social literacy is the understanding individuals have of the world, people in the world, and interactions between people. A student?s ability to empathetically understand the point of view of another individual, to recognize not only the perspective of another, but also the ability of individuals to hold very different, yet valid, understandings of the same situation are vital aspects of social literacy (Lowenthall, 2000; VanSledright, 2002b). The complexity involved in this type of literacy increases as students struggle to understand and identify with individuals separated from them by time, space, or both. For this reason, a thirteen- year-old student may have trouble understanding the world through the eyes of a fifty- 5 year-old woman, but this same student will have even more difficulty understanding the perspectives of or developing any level of empathy with a fifty-year-old woman in 14 th Century Asia. Although source documents are by their nature complex, the level of difficulty for any given text is dependent on two factors, the text itself and the reader. Students with below average literacy may find a specific text difficult to engage, whereas students with adequate literacy skills will have little or no trouble accessing and engaging the source. Literacy combines the abilities to decode words, make meaning of bodies of text, apply appropriate complementary knowledge to understanding the text, and use it within its current purpose. In this study students? literacy skills will be considered in two ways that directly correspond to the two ways text complexity will be considered. The first classification is the readability of the text and the reading skill levels of students. This is often expressed in grade level increments, and I refer to basic readability in this manner. Although students? reading skill levels can also be described using grade level designations, for the purposes of this study students are divided into three categories, average, above average, and below average. These are determined by a comparison of each student?s literacy skills compared to the average for students in the same grade level. The second classification of text complexity is domain specific complexity. Domain specific complexity is a result of factors that, while not unique to the social studies domain, have a considerable influence on the complexity of historical source documents. In order to determine the domain specific complexity for source documents in this study, I considered three criteria: the use of archaic or exceedingly complex 6 language, the need for historical background information to understand the context of the source, and factors within the source document or the reason for reading the document that encourage or discourage motivation in the reader. Unlike readability levels, there are no formal assessments for determining domain specific complexity. Determination is situational and based on these factors along with factors unique to the individual or group that will be reading the material. Complex texts, insufficient literacy skills, and the need for students to have social literacy skills add to the difficulties faced by teachers as they incorporate source documents into their instructional activities. These difficulties can overwhelm teachers when they find that the assistance needed by a classroom full of students is greater than one teacher can meaningfully provide. The frustration of not being able to provide the assistance necessary for students to meaningfully engage source documents can sometimes lead teachers to choose other, less meaningful activities and texts in place of the more effective and difficult authentic ones. Unfortunately, students are unable to engage in authentic, problem-based activities if they do not use authentic materials including complex source documents. Additionally, students are unable to develop and practice democratic citizenship skills, such as consideration of multiple perspectives and the ability to have meaningful dialogue about meaningful societal issues, if they are unable to participate in authentic activities designed for this purpose. Helping teachers provide the types and levels of assistance that will enable students to actively engage in the study of complex source documents is essential if teachers are expected to routinely make use of them in authentic, problem-based, history instruction. One possible way to provide this assistance is through the use of technology. 7 Although technology in the educational setting has too often been used in inappropriate and/or underproductive ways, interactive, multimedia technology has the potential to provide scaffolding for students with insufficient literacy skills as they engage complex texts. Scaffolding refers to the supports that enable students to accomplish more and attain higher goals than would be possible if they worked without these supports (Brush & Saye, 2001; Keselman, 2003; Taylor, 1999; Vygotsky, 1978). For most tasks some of this scaffolding can be built-in so that students have access to it at the moment they need it. When scaffolding is not built into the activity, the teacher must provide it, and since teachers cannot possibly provide assistance in every instance when it is needed by each individual student in a class, finding effective ways to build scaffolding into instructional activities, especially those that utilize complex texts, is essential. It is this type of built- in, just-in-time scaffolding that might be effectively provided through the use of interactive technology. Interactive technology may be used to provide those types of scaffolding the teacher knows in advance that students will need. Providing definitions and explanations to individual students can take up a significant amount of a teacher?s time. Interactive technology may be able to instantaneously provide this type of individual scaffolding so that the student who needs significant levels of assistance does not become frustrated with constantly asking for and waiting on help, while the student with adequate literacy skills is not burdened with unneeded definitions and explanations. In addition to scaffolding that provides basic information, interactive scaffolds can link students to resources that provide background information they may lack. This may be as simple as a brief explanation or as complex as links to additional documents 8 and websites to provide additional background knowledge and understanding. Technology may also be used to help students think about and use text in novel ways. Built-in scaffolding using interactive technology can direct students? attention, pose questions for consideration, provide instructions, and guide the learning process (Saye & Brush, 1999). Each type of scaffolding that can be provided through interactive technology can, of course, be provided in person by a teacher or through non- technological means. However, the ability to provide students with immediate assistance and support for issues involved in dealing with complex source documents may make this type of scaffolding invaluable to the teacher whose students? literacy skills may range from very high to very low. Purpose of the study Students today have been exposed to technology throughout their education careers. Despite these years of use, many promising applications for educational technology have yet to be thoroughly researched in order to provide teachers with a foundation of wise practice recommendations and realistic expectations (Penuel, Means, & Simpkins, 2000; VanFossen & Shiveley, 2003). The purpose of this study is to gain a better understanding of the effects of using interactive technology, specifically hyperlinked texts, to assist students as they engage historical source documents as part of problem-based historical inquiry learning activities. Need for the study All phases of this study were guided by a need to establish guidelines for the effective use of technology to help teachers manage the use of complex texts as part of effective instruction. To establish these guidelines, research must consider two types of 9 questions. First are questions regarding decisions about when, where, and how to include hyperlinks as effective scaffolding within historical source documents. Second are questions about the effects hyperlinks have on text readability, students? use of hyperlinks to analyze text and develop problem solutions, and students? ability to undertake the authentic learning activities within which these texts are set. Significance of the study This study examines students? use of hyperlink scaffolded historical source documents beginning with the decisions made by the teacher regarding the choices of documents and placement of hyperlinks and ending with the students? use of the information gained from the texts to solve problems posed in the lesson. The classroom setting provided information regarding students? use of the hyperlink scaffolds in a normal school environment. Additional examination of the students based on their reading skill levels provided valuable information about the differences in the ways students with varying ability levels utilize this type of scaffolding. Research Question This study began with the broad question: How may students be supported in working with complex, text-based source documents as part of a problem-based historical inquiry lesson? Although there are many paths that a researcher might take to consider this question, I chose to focus on the possibility that interactive technology might be one means of providing this support. Five research sub-questions were used to provide additional focus for my study: Sub-Question 1: How do students interact with source documents presented in an on-line format? 10 Sub-Question 2: How do students use hyperlinked scaffolds in analyzing source documents? Sub-Question 3: How do students use scaffolds to develop problem solutions? Sub-Question 4: Are there similarities and differences in the use of online documents and hyperlinked scaffolds for students with different literacy levels? Sub-Question 5: How closely do educators? expectations of students? needs match students? actual needs? Methodology To explore these questions, I examined the efforts of two seventh grade classes as they worked with a set of complex source documents within a civics lesson. The students, the teacher, and I identified areas of source documents where the students were or might have become confused. Using this information, hyperlinks were added to the documents and the students encountered these texts on-line as part of an instructional lesson. Data collection was completed through observations of the students as they participated in the instructional unit, collection of student work products, and interviews with the students and the teacher about the use of the hyperlinked documents and the effects of using them as part of the overall instructional unit. Analysis of the data was completed for two purposes. Through the early aspects of the study data were analyzed in order to guide the development of the design intervention. Qualitative analysis of all data was completed in order to address the study questions. Limitations of the study Although the classroom setting of the study provided valuable information regarding the use of complex historical source documents and the inclusion of 11 hyperlinked scaffolding, it created some limitations for the study as well. Some aspects of the study had to be altered in order to accommodate the needs of the teacher, the students, and the school schedule. While this affected the way the study was conducted and the information that was gained from the study, the more realistic setting meant that the problems and concerns of the teacher and students were part of the study?s findings. Another limitation was the lower level of participation by students with below average reading skills. The findings from the study very difficult to generalize beyond the study class and in some cases within the class as well because fewer students from this group chose to participate by having data collected. Despite these limitations the study provided valuable information regarding the use of hyperlinked scaffolding to assist students as they utilize complex historical sources as part of problem-based inquiry instruction. 12 CHAPTER 2: REVIEW OF LITERATURE Introduction Studies show that students engaged in actively seeking knowledge are more motivated to learn, more likely to retain information, and better able to apply the knowledge they gain in meaningful ways (Dewey, 1933; Newmann, Wehlage, and Lamborn, 1992). Inquiry learning, when it involves the critical examination of historical events and the individuals involved, has been shown to produce these positive benefits (Shaver, Davis, & Helburn, 1979; Thornton, 1991), yet a majority of history and social studies teachers have failed to adopt this method of instruction. In some cases teachers and others with influence over curriculum have philosophical disagreements with the use of inquiry learning, especially if they are examining events that involved the United States (Nash, Crabtree, & Dunn, 1997; Windschitl, 2002; Wineburg, 1999). Other teachers and researchers cite a lack of sufficient time to allow for the in-depth study of individual events or topics and a lack of knowledge about how to use inquiry effectively as factors that prevent their adoption of this method (Brush & Saye, 2001; Levstik & Barton, 2003). Other social studies teachers, who do not have a philosophical conflict with the use of inquiry, who understand its value to student learning, and who have the knowledge needed to plan for and use it in the classroom still do not use inquiry-learning strategies routinely (Levstik & Barton, 2003, 2005; Wenglinsky, 2004). Although reasons exist 13 for this reluctance that are outside those issues that can be adequately addressed through classroom research, many issues can be addressed through research directly related to classroom practice. This research must seek solutions for those problems that teachers who are otherwise able and willing to use inquiry have for avoiding its use as a routine part of their instructional practice. One challenge social studies teachers face as they plan for instruction is the issue of students with literacy skills that are inadequate to allow them to successfully engage in reading and use of difficult texts. Because history is an academic discipline that is heavily dependent on reading, student literacy is a significant concern for teachers, especially those teachers dedicated to the use of inquiry as a means of preparing students for the demands of citizenship in a pluralistic democracy such as the United States (Mcdaniel, 2004; Wineburg & Martin, 2004). Many teachers believe that only advanced students have the literacy and cognitive skills needed to successfully engage in inquiry learning and that students perceived as less-skilled or lower-achieving are better served through the use of direct instruction and rote learning activities (Gamoran & Nystrand, 1992; Oakes, Gamoran, & Page, 1992; Onosko, 1991). Because inquiry in the social studies classroom makes greater demands on the literacy skills of students than these lower-level instructional activities, researchers must address the relationship between basic student literacy and the use of complex source documents within the context of inquiry learning in these classes and find ways to help teachers and students overcome the obstacles presented by an inquiry-based instructional environment if this is to become a routine strategy in a majority of social studies classrooms. 14 Inquiry Learning It is important to first understand what inquiry learning is and why many social science education professionals recommend its use (Levstik & Barton, 2005; Thornton, 1998; Wineburg, 2001). Inquiry learning is a type of instruction in which students are presented with a problem that they must solve, explain, or clarify and for which there is often no agreed upon solution. Through the research and study students do as they work on these problems they learn basic knowledge, concepts, and skills as well as practice the concrete application of this knowledge. This application reinforces what the students have learned and improves the likelihood of long-term knowledge and skills retention (Brush & Saye, 2001; Renzulli, Gentry, & Reis, 2004; Savery & Duffy, 1995; Saye & Brush, 1999). Because inquiry learning is also likely to involve elements that increase motivation to learn, such as personal relevance, interesting subject matter, and the opportunity to consider questions that extend beyond rote memorization, students are likely to be more motivated to participate in these types of learning experiences (Dewey, 1938; Levstik & Barton, 2005; VanSickle, 1996; Wineburg, 2001). Inquiry learning has become a staple in the science classroom, and a significant amount of research has been done in that setting (Sonmez & Lee, 2003). Science teachers using inquiry as an instructional strategy find that students are more motivated, take greater responsibility for their own learning, develop a greater understanding of the ways newly acquired knowledge complements previous knowledge and skills, and have greater retention of knowledge and skills (Chang 2001; Keselman 2003). These benefits, the relative lack of conflict over the use of inquiry in the science classroom, and the ease 15 with which inquiry can be incorporated into this setting have resulted in significant gains in the numbers of science teachers routinely incorporating this instructional method. In a study of elementary, middle, and high school math and social studies classes across the nation, Marks, Newmann, and Gamoran (1996) also found that inquiry learning yielded improved results, but they noted variations among different populations of students. They found that scores on both traditional and authentic assessments were improved for students who had participated in inquiry learning activities, and that ?regardless of race or gender, an average student would increase from about the thirtieth percentile to about the sixtieth percentile as a result of experiencing high versus low authentic pedagogy? (p. 58). Although the researchers? findings indicate that there is a comparable increase in the achievement for all students, those who began with higher educational achievements tended to derive a slightly greater benefit than those who began with lower educational achievements. They speculate that this may be a result of the cumulative nature of learning in inquiry instructional environments. They also found that teachers are more likely to use this type of instruction with higher performing students and speculate that this is due to the belief that poorer performing students are not capable of participating in rigorous inquiry learning and because teachers often do not know how to help them bridge the gaps in their skills and knowledge. Unfortunately this means that lower-performing students often do not get the very instruction that research suggests can be highly beneficial for student motivation and learning (Rossi & Pace, 1998). Inquiry learning in social studies classes. In social studies classrooms many teachers have been more reluctant to routinely incorporate inquiry into their instruction. Often this reluctance is based on the belief that history teachers should concentrate on 16 teaching only historical facts in an attempt to ensure that students learn all of the historical information they must know. The question of whether students are learning enough historical information, however, is a persistent one that has been asked by social science educators as well as the public in relationship to almost every method of social studies instruction. Although studies sometimes find that students do not know a sufficient amount of historical information, the solutions proposed are routinely centered on teaching more facts and information and seldom consider the issue of the method through which this information is learned (McTighe, Seif, & Wiggins, 2004; Parker 2001a, 2001b; Thornton 1998). Research suggests that it is the method used to teach history that determines how successful most students will be at learning the relevant facts and skills, retaining these for later use, and applying what they have learned to their lives as citizens of a democratic society (Weinstock, 2005). Wenglinsky?s (2004) analysis of instructional methods and NAEP scores found that when students in reading and civics classes had to apply the historical knowledge and skills they were learning in class in meaningful ways, they tended to score better than students in other instructional environments. This suggests that if students are expected to both learn and use relevant historical facts and skills, they must learn those facts and skills through active participation in activities that cannot be successfully completed without this knowledge. VanSledright (2002a, 2002b) directly involved fifth-grade students in an inquiry- based instructional environment. This type of instruction proved to be a significant motivator for the students in his study. According to VanSledright, the students became self-directed learners as they interpreted and analyzed historic events that had become 17 personally meaningful as a result of their participation in the inquiry activities. VanSledright?s results corroborate what other researchers have found, that students, even younger students, are motivated by the process of investigating much more than memorizing, and that students involved in this type of study are more likely to have a deep understanding of the historical events they study, retain the information they use and learn, and be able to apply it to other situations. After completing an extensive study of the literature on issues-oriented classrooms in which inquiry instruction was used, Rossi (1995) identified four common characteristics of in-depth instruction: (1) the use of knowledge that is complex, thick, and divergent about a single topic, concept, or event using sources that range beyond the textbook, (2) essential and authentic issues or questions containing ambiguity, doubt, or controversy, (3) a spirit of inquiry that provides opportunities, support, and assessment mechanisms for students to manipulate ideas in ways that transform their meaning, and (4) sustained time on a single topic, concept, for event. Although the issue of instructional method is not always at the forefront of the debate over what to do about students? lack of historical knowledge, it has not been completely ignored by researchers. Newmann (1992) found that in order for students to engage in higher-order thinking they must struggle with challenging questions or tasks, have sustained time to engage in this challenge, and must be required to explain and defend their conclusions. It is through the active participation involved in inquiry learning that students become engaged ?active interpreters of knowledge, rather than docile recipients? (p. 184-5). Like Newmann, Josten (1996) and Beck, McKeown, and Worthy (1995) found that it was students? active participation in the learning process that 18 resulted in the most significant gains in knowledge and skills. In a study of students reading historical texts, Beck et al, found that the more engaged a student is in a problem related to the text, the more likely they are to be highly attentive to the information and ideas presented, actively consider those ideas, and deal with them in order to make them personally meaningful. Josten?s study of poor readers in a history course for college freshmen, found that students engaged in inquiry learning out performed those in the teacher-centered, direct- instruction class in several key areas. Although Josten found that there was almost no difference in scores for the two groups on questions requiring low-level thinking, there were significant differences between the two groups on questions requiring higher-level thinking, such as analysis, prediction, or evaluation. Additionally, students in the inquiry group indicated that they believed the instructional method helped them maintain their focus and was a more interesting approach to history instruction. These statements by the students corroborate the beliefs of the teacher that the students in the inquiry class were more on task, including normally disengaged students, and they were more likely to think creatively about what they were learning. Authentic instruction as a type of inquiry instruction. Newmann, Wehlage, and Lamborn (1992) describe instruction as authentic when students are asked to perform tasks that ?are considered meaningful, valuable, significant, and worthy of one's effort, in contrast to traditional instructional activities that are often considered nonsensical, useless, contrived, trivial, and therefore unworthy of effort" (p. 23). Through inquiry instruction students use information in ways that promotes deeper thought and learning. When inquiry instruction has students learn through the consideration of problems and 19 performance of tasks that are similar to those considered and performed outside of the academic setting, inquiry instruction can be called authentic instruction (McTighe, Seif, & Wiggins, 2004; Wiggins, 1998). Although authentic instruction can be highly motivating for students who may be struggling with other academic issues, researchers have found disturbing gaps between the levels of authentic instruction provided for higher and lower performing students (Cohen, 1992; Hofer & Gamoran, 1993; Josten, 1996). Wehlage and Smith (1992) identify at-risk students, no matter the reason they are at-risk, as harder to engage in academic tasks. They cite case studies in which schools have been restructured to work with at-risk students, and they argue that these schools need to be based on authentic instruction in order to successfully engage students in the study of complex, often abstract, knowledge and skills. These researchers define ?authentic? instruction as activities that emphasize ?production of socially useful, personally meaningful, and aesthetically valuable knowledge" (p. 111). They identify the necessity of creating authentic tasks that utilize each student?s strengths and interests and that recognize the capacity of all students to exercise multiple forms of intelligence in order to ensure that students understand the relevance of the academic tasks they are performing. Rather than reproducing knowledge, students in these schools would be involved in solving rich problems that allow them to construct their own meaning and thereby give significance and coherence to abstract concepts. This ability to motivate difficult students was important in their study since these students may benefit significantly from the additional reading and instructional interaction that takes place in authentic instructional environments. 20 Marks, Newmann, and Gamoran (1996) and Dochy, Segers, Van den Bosche, and Gijbels (2003) found positive long-term results of authentic instruction on assessment outcomes. Marks, Newmann, and Gamoran considered the effects of authentic instruction on both authentic and standard assessments in elementary, middle, and high school math and social science classes. Their results at these levels were much the same as the results Josten (1996) had at the college level and those of the meta-analysis by Dochy, et al. Like Josten, Dochy, et al. found that there was a short-term drop in the amount of factual knowledge acquired by students in the authentic instruction classrooms, but this deficit was reversed after two years of instruction. After the two year point, students in the authentic instruction based classrooms were no longer at a disadvantage in terms of factual knowledge recall, and actually exhibited a significant advantage over those students in the direct instructional environments in terms of their ability to apply their knowledge and skills in unique situations. Wehlage, Newmann, and Secada (1996) also found that authentic instruction based on inquiry yielded the greatest results in student achievement. They recommended that students construct meaningful knowledge through disciplined inquiry that exhibits three main features: use of a prior knowledge base, striving for in-depth understanding as opposed to superficial awareness, and expressing ideas and understanding through ?elaborated communication? (p. 24-5). Taken as a whole these researchers describe authentic instruction as instruction that: (a) requires students to gain an in-depth understanding of an event, topic, or concept through the use of multiple sources over a sustained period of time, (b) is centered around essential and authentic issues, questions, and tasks that contain ambiguity, doubt, or controversy, (c) provides opportunities, 21 support, and assessment mechanisms for students and they make use of prior knowledge, gather new information, and manipulate ideas in ways that transform their meaning, and (d) requires students to perform tasks that are meaningful, valuable, significant, and worthy of effort as they develop solutions and defend these through ?elaborated communication.? Problem-based Historical Inquiry Problem-based historical inquiry (PBHI) is a type of authentic inquiry learning in which most of the academic work in a social studies classroom is centered on a set of enduring historical problems (Saye & Brush, 2004a). In social studies classes using PBHI students thoroughly examine specific historical events through which they consider larger persistent issues. PBHI requires students to conduct disciplined research, deal with conflicting interpretations of history and strive to understand their significance, and construct knowledge based on information from a variety of sources. Additionally, it is focused on an ill-structured problem, requires collaborative work and a public defense of arguments in support of a position, and has value beyond school (Brown, Collins, & Duguid, 1989; Newmann & Wehlage, 1995; Saye & Brush, 1999). Essentially PBHI is social discourse taking place in the classroom. Newmann and Oliver (1970) identify four types of social discourse: social opining, persuasion, transmitting information, and clarification, and they indicate that the clarification type of discussion is most important since this is the most useful type in the context of citizenship education. They identify value conflicts as problematic issues because even the resolution of factual and definitional conflicts does not always resolve continuing 22 sources of disagreement based on value conflicts and it is these value disagreements around which PBHI instruction is centered. More recently, Saye and Brush (2004a) looked specifically at the issue of problem-based historical inquiry and its effects on student learning in the social studies class. They suggest that "text interpretation and narrative construction as school history activities should be a means to a civic end. That end should be reasoned decision making about enduring social problems? (p. 127-8). In order to reach the goal of developing students with the skills and knowledge needed to perform as reasoned decision makers, students should acquire their historical knowledge and social studies skills during the investigation of historical events through which they seek to understand persistent societal issues. Through these investigations they should develop foundational knowledge, clarify key concepts, make use of historical analogy, and ?propose problem solutions [that are defended] with historical evidence? (p. 128). Finally, Saye and Brush identify ?student motivation and students? readiness to handle the cognitive challenges posed by social inquiry? as significant challenges that must be overcome in a PBHI centered classroom (p. 130). Although student motivation is less likely to be a challenge in a class where problem-based historical inquiry is a routine part of the instructional activities (Shaver, 1992), the ability of students to handle the cognitive challenges of this type of instruction are a serious issue for many teachers attempting to motivate students through the use of authentic instruction such as problem-based historical inquiry. Complexity of historical source documents What happens when students cannot move beyond the factual or definitional issues as mentioned by Newmann and Oliver (1970) or cannot overcome the cognitive 23 obstacles discussed by Saye and Brush (2004) in order to deal with the value conflicts involved in the social questions? Although at times this may happen as a result of strongly held individual beliefs such as a nationalistic attitude that precludes a belief in the fallacy of a specific national action, in the social studies classroom it is also likely to happen as a result of students? difficulty in comprehending the written material in which relevant information is to be found. Making-sense of the social sciences, especially history, is an activity heavily dependent on the use of language. An empathetic understanding of the past is often best achieved through careful consideration of the writings of individuals living in that time period. These writings make up much of the vast array of source documents produced by all societies with a written language and include official records, personal narratives, religious and literary works, correspondence, and all other materials that historians refer to as primary sources. When inquiry or problem-based instruction is used in the social studies classroom these source materials provide students with insight into the historical world they are studying. They help students develop a personal connection to the past, allow them to consider the various points of view of people involved in the same historical event, and require that they wrestle with the biases and prejudices of these individuals. If students are going to be expected to develop a deep understanding of the past in order to consider the problems of those times and apply what they learn to the problems of the present, the experience of using text-based historical sources is vital. Researchers and teachers understand, however, that source documents of all types are incredibly complex. Afflerback and VanSledright (2001) examined the ways students read diverse historical texts, specifically historical source text presenting multiple 24 perspectives imbedded within the traditional text. They studied fifth graders reading a textbook section about the Jamestown Colony that contained two embedded texts, a poem and a diary entry and found that while some students were able to use the text to understand the differing points of view about a particular historical event, others struggled with making sense of the additional text and this struggle became their primary focus. Afflerbach and VanSledright (2001) identify the need for students to be specifically instructed in the use of this type of historical texts if they are to develop the understanding that history is a human construction and that different people may view the same event and/or time period differently. The students in their study seemed more willing and able to respond in a personal way to the source materials, but they found that while the differing accounts caused some students to begin to focus on how history is made, for other students the fact that history is interpretive and that these interpretations are frequently contested remained unclear. This continuing confusion made it almost impossible for them to fully understand that history is not one unified narrative but a result of the accumulation of multiple stories and points of view. Students experienced varying levels of success in developing understanding and appreciation for the texts they read, and for some this proved to be an extremely demanding task that required a significant level of teacher assistance for the student to experience true success. When students became confused by archaic vocabulary and complicated syntax, the embedded text seemed to inhibit construction of meaning instead of enriching their understanding. This was also true when text included seductive details that interested the students but ultimately led them astray, such as a student upset by a reference to Native Americans as 25 savages in a diary entry from the 1600s. The student was unable to move beyond this issue to fully incorporate the source material into her understanding of the event she was studying. Afflerbach and VanSledright suggest that teacher assistance in providing on- time meanings and synonyms for unfamiliar words and phrases, as well as helping students understand how interesting details should be considered as part of the whole were essential for students as they made use of the texts. The need for domain specific instruction identified by Afflerbach and VanSledright mirrors the findings of other researchers studying the specialized literacy demands of differing educational disciplines. Unsworth (1999) found that functional grammatical knowledge differs depending on the literacy demands of the specific text. The literacy demands of everyday language are distinctly different from the literacy demands of domain specific language. Reading historical source documents requires an understanding of the structure of a text?s genre (Goldman & Rakestraw, 2000), the ability to connect new information to previously learned information as well as to commonly known references, a willingness to critically analyze previous scholarship (Lowental, 2000; Rosenblatt, 1969, 1985), the ability to identify and consider multiple perspectives (Barton & Levstik, 2003, 2005; Holt, 1990; VanSledright, 2002b), and the ability to make sense of archaic and unfamiliar language and cultural norms and expectations (Afflerbach & VanSledright, 2001; Tovani, 2000; VanSledright, 2002b). It is necessary to identify the specific demands of historical and scientific language if students are going to be successful reading and comprehending texts in these areas. Additionally, students who develop this content area grammatical knowledge are at a distinct advantage over 26 students who may have a high level of general grammatical knowledge, but who do not have the domain specific knowledge they need. Use of textbooks In order to avoid the problems inherent in the use of source documents, history instructors often rely heavily on history textbooks, and for many the textbook becomes the only source used in their class (Britt, Perfetti, Van Dyke, & Gabrys, 2000; Thorton, 1991). Because textbooks condense large amounts of historical information into a small amount of space they would seem to be the solution to the problem of students having to deal with the complexity of historical source materials. Although history is, essentially, the collective story of a group of people, individuals do not understand each story the same nor do historians interpret each the same, but textbooks eliminate any indication that there is disagreement among historians or uncertainty on the part of the historians (Wineburg, 1999). Problems with textbook construction. Many of the problems with the use of textbooks are a result of the way textbooks are constructed. Wade?s (1993) meta-analysis of content analysis studies of social science textbooks examined 25 studies published either in Theory and Research in Social Education, Social Education, or The Social Studies in the ten years from 1983 to 1993. Of the 25 studies, eighty-eight percent concluded that there was limited coverage of topics, and thirty-two percent identified factual errors in the textbooks under review. Over half, fifty-six percent, of the textbooks reviewed tended to avoid controversy and controversial issues. Forty percent of textbooks that researchers examined contained incidences of stereotypical or biased presentation of material, although twelve percent noted that there was less bias than in 27 previous studies. There is some indication that specific areas, such as inclusion of bias, improved during this ten-year period, but it is important to note that the overall trend was toward the continuation of these problems, not toward their elimination. Wade?s (1993) conclusion that textbooks tend to avoid controversy is extremely important for the educator who believes the purpose of social science instruction is to prepare students to actively participate in a free, pluralistic, democratic republic. Because controversy is innate in a democratic society, the avoidance of controversial issues in social science textbooks is problematic at best and renders them obsolete for any purpose except as reference material at worst. Wade?s conclusion that forty percent of texts contained material that impeded comprehension is also a cause for concern for most classroom teachers because factors that would impede the comprehension of students with at least adequate literacy skills and domain specific social science literacy skills would eliminate any chance that students with less than adequate literacy could comprehend the textbook material. Finally, it is important that despite changes to social science textbooks over a ten-year period, significant problems with these texts remained constant, including inadequate coverage, bias and stereotyped presentation, inaccurate information, poor organization, and conservative presentation of historical materials and concepts. Allington (2002) identifies another significant problem with the exclusive or near exclusive use of textbooks in the classroom, the reading levels of these books compared to the reading levels of the students utilizing them. By the fourth grade many students begin exhibiting frustration with domain specific reading. This is largely because the difficulty level increases significantly and at the same time textbooks begin to be used 28 more extensively. Many of these are written at levels well above many students' independent reading level, they contain vocabulary that begins to be more specialized and abstract, the language and syntax becomes more difficult and less conversational, and students are expected to independently make connections to prior knowledge. All of these changes tend to make textbooks more complex for all students as they move from the elementary grades into the secondary grades, but they are especially large problems for students with less than adequate literacy skills. Problems with student use of textbooks. Researchers considering the use of history textbooks in the years since the conclusion of Wade?s study have continued to find significant problems related to students? ability to read, comprehend, and use textbooks to provide information that allows them to consider historical issues in a meaningful way. While Paxton (1999, 2002) and Taylor (1999) found the third-person expository writing style and the linear presentation of most textbooks to be an inhibition to students? ability to comprehend and use the material, Tunnell and Ammon (1996) and Nelson and Nelson (1999) found that students? understanding of multiple historical perspectives was inhibited by the single perspective presented by most textbooks and by the overall lack of interest students show in reading textbooks. When historical information was presented in the form of a first-person narrative, as is found for example in literature and most historical works written for adults as opposed to textbooks typically written for young people, these studies found that students were more likely to ask pertinent questions related to the text, search for and utilize additional material to add to their understanding, consider alternative perspectives, and remain engaged in the study of the texts. Additionally, Paxton found that the students reading the first-person narratives 29 thought more deeply about the information and constructed more reasoned arguments related to the information than those students reading the third-person versions. The way history textbooks present information is also a problem for many readers. The better a text allows the reader to draw connections among ideas, and the more engaged a student is in a problem, the more likely they are to be highly attentive to the information and ideas presented in a text, react to those ideas, and deal with them in order to make them meaningful (Beck & McKeown, 1991; Beck, et al., 1995). Beck, et al. suggests that the first consideration should be engaging the student with the purpose their reading. A student reading to locate information is going to be much less engaged than the student who is reading in order to give a personal interpretation of the text. Next, consideration must be given to making the text itself engaging. Beck and colleagues found that since most students? spoken vocabulary and listening comprehension levels are much higher than their reading level, text written in first-person, narrative style, which is more like spoken language, is more successful for engaging students and keeping the focus on the important information in the text. Therefore, if text can speak to the reader as opposed to simply presenting information, as if it were divorced from any human author, the reader is more likely to become engaged in the reading and, consequently remember more. When a text is written using first-person voice, the interaction between reader and author is much more like the interaction between a speaker and a listener. Weak readers often feel more comfortable learning by listening and may, therefore, be more willing to become engaged in the text if the language is in a format they are comfortable with. Beck, et al. point out that the third-person style of writing used in most textbooks accentuates the differences between the written and 30 spoken word making it more difficult for all readers to adequately relate to the information in the textbook. Another serious problem with many textbooks is the assumption that students have background knowledge or that they can or will make necessary connections between important pieces of information. When expository text is modified to enhance students? ability to make connections and develop a clearer understanding of cause and effect, Beck, et al. found measurable improvements in students? ability to comprehend. A study by Voss and Wiley (2000) looked at the issue of instructional method and use of a single, unified text, such as a textbook, in order to better understand how to combine instruction and text to achieve the best student performance. This research studied the differences between students reading about a historical event presented as a single unit of text and students reading about the same event presented as multiple, smaller units of text. At the conclusion of the study some students were assigned narrative essays and others were to write an argumentative essay. The researchers found that students reading in the multi-segment condition processed the information more deeply than those reading the single narrative unit. They posit this was a result of students having to integrate the multiple sets of content in order to develop a coherent text structure, unlike the single-text condition in which the text structure was already present. The assignment of the essay was also important because it provided students with a purpose for reading. Those students who were assigned an argumentative essay were more likely to have developed greater causal connections between dissimilar events than those writing a narrative essay. In the study, the groups that developed the most in- depth understanding of the material were those who read the multiple text versions and 31 were assigned argumentative essays. Voss and Wiley?s results suggest a need for instructors to provide students with authentic tasks related to their research. Their results also seem to indicate, as many other studies have, that textbooks are seldom the best medium through which students can gain historical knowledge or social science skills. The problems with textbooks make students? understanding of historical information and events more difficult and increase the likelihood that students will skip key terms, fail to understand key concepts, and be unable to make necessary connections as they struggle to make sense of text that they find incomprehensible (Hirsch, 2003). Allington (2002) found that exemplary teachers who overcome the difficulties imposed by textbooks use multiple source documents as part of their instructional plans. These teachers used documents from multiple genres and at multiple reading and instructional levels to support the general framework and information provided by textbooks. Bain?s (2000) research into the ways multiple historical sources improve students? abilities to understand how historical accounts are created indicates that while history textbooks inhibit students? abilities to see history as stories created by historians, use of historical sources improves this ability. Like many history educators, Bain strongly criticizes the near exclusive use of textbooks as the instructional text in history classrooms because, like all skilled artisans, ?historians polish [their] final products [textbooks and monographs], intentionally removing signs of the struggles and strategies along the way? (p. 333). It is this very polish that complicates the instructional problem for teachers and students. Teachers find it hard to model and students have difficulty practicing the types of historical thinking that go into the creation of a historical narrative when that thinking is all but hidden in the final polished product. To avoid the tendency 32 of students to think of history as a series of events to be memorized and encourage them to interpret history, develop the habits of expert thought, and arrive at a narrative, Bain argues that students must utilize a wide variety of documents. Use of Source Documents History textbooks are an imperfect solution to the complexity of text-based source documents, and research into the use of source documents suggests that the very complexity of these documents helps students better understand the complexity of history (Afflerbach & VanSledright, 2001; Britt, et al., 2000; Levstik & Barton, 2001; Otten, 1998; Paxton, 2002; VanSledright, 2002b; Wineburg, 2001). The use of source documents is recommended by professional organizations, such as the NCSS (1994). The 2001 NAEP U. S. History Assessment (Lapp, Grigg, & Tay-Lin, 2002), a history assessment given to 4 th , 8 th , and 12 th graders, indicated a clear advantage for students in grades eight and twelve that use source documents as a regular part of their history instruction. Eighth graders that used source documents at least once a week had higher scores than those who used them with less frequency, and twelfth graders that reported reading source materials regularly throughout the school year also scored better than those that did not read these materials regularly (Lapp, et al., 2002 ). Use of source documents is also recommended in the standards for historical thinking established by the National Center for History in the Schools. In these standards use of source documents is described as an essential aspect of an instructional environment in which students can develop historical thinking skills. The ?Overview of Standards in Historical Thinking? in the standards discusses the need for students to make extensive use of source documents: 33 [T]rue historical understanding requires students to engage in historical thinking? to go beyond the facts presented in their textbooks and examine the historical record for themselves; to consult documents, journals, diaries, artifacts, historic sites, works of art, quantitative data, and other evidence from the past, and to do so imaginatively--taking into account the historical context in which these records were created and comparing the multiple points of view of those on the scene at the time (National Center for History in the Schools, 2005). Benefits of using source documents. The use of various source documents from which students can construct historical meaning is not only effective for immersing students in the past (Lowenthall, 2000), but is also motivational (Pang, Muaka, Bernhardt, & Kamil, 2003; Rosenzweig, 2000). In a study about the attitudes of Americans toward history, Rosenzweig found that despite a common perception that Americans are disengaged from the past, most are significantly engaged with knowing the past and constructing meaning from it, both the national past and their personal past. A cross-section of Americans was surveyed for the study and Rosenzweig reports that a significant percentage indicated an interest in history and historical knowledge. Overwhelmingly, however, respondents in the study indicated a lack of interest in histories that give them information without allowing them to directly engage with the past. Many said that while they like museums, they did not like history in school because of this lack of engagement. When asked to ?pick one word or phrase to describe your experiences with history classes in elementary or high school,? negative descriptions significantly outweighed positive. It is also important to note that the only place the words boring or boredom appeared in connection to the pursuit of the past was in 34 connection with school (p. 273). In the survey ?the most powerful meanings of the past come out of the dialogue between the past and the present, out of the ways the past can be used to answer pressing current questions about relationships, identity, mortality, and agency? (p. 280). The participants in this study exhibited a clear preference for those activities where they could develop a meaningful historical narrative based on their analysis of the information available and a clear disregard for those activities, such as reading a textbook, in which someone else analyzed the important evidence and simply explained what they should believe. It is significant that in Rosenzwig?s (2000) study history was only identified as boring when it was history as it is often taught in school. Typical history instructional methods provide few opportunities for students to engage history in an authentic way. More authentic instruction might provide students with the opportunities to think about and use history the way real people do, through deliberation, debate, and the analysis and use of a variety of source materials. There are many types of source documents that history teachers can incorporate into their instructional strategies such as: works of art, photographs, movies, public and private records, poems and other literature, and archeological artifacts. Few, however, are as important when students are considering historical events through authentic, empathetic, pedagogically sound instructional practice as the wide variety of text-based sources. Through text-based source documents students gain insight into the lives and beliefs of the individuals who were part of the history they read about in their textbooks (Afflerback & VanSledright, 2001; Billman, 2002; Otten, 1998). Through these texts, historical characters become individuals who struggled with issues common to all human 35 societies and history becomes the combined struggles of these individuals instead of the sterile recitation of the historical facts that a group of historians collectively concluded create the best representation of a specific event or time period. The value of these types of sources also extends beyond the classroom. Levstik and Barton (2003, 2005) indicate that if the goal of social science education is to educate students as citizens, they must learn to value and understand multiple perspectives, including those that are radically different from their own. They further suggest that to be an effective citizen one must be able to distinguish myth from grounded assertions. In the history classroom these skills, while seldom acquired through study in a textbook, can be cultivated through the use of multiple historical sources. Use of these types of sources, however, present multiple levels of complexity that the teacher must be able to help students negotiate if they are going to be effectively used and if this model of social science education is going to replace the content coverage model in which the teacher attempts to mention as many historical facts as possible during the course. Like Barton and Levstik (2003,2005), Holt (1990) is critical of the use of authoritative historical textbooks and instructional methods that leave students believing the purpose of history is to learn historical facts that are completely disconnected from their personal lives. Holt?s study of college freshmen found that students make clear distinctions between school history and personal history and most believe the two have little in common. While many students believe personal narratives can be told many ways, depending on the purpose of the narrative and who is doing the telling, they believe that historical narratives can only be told one way, the correct way, and that the only purpose of a historical narrative is to convey the facts. This disconnection between 36 students? beliefs about personal historical narratives and real historical narratives speaks to a misunderstanding between students? understanding of how historical narratives are constructed by historians and how their own historical narratives are created. According to Holt, the use of historical source texts allows students direct access to see and hear multiple historical perspectives and personal narratives for themselves and thus to formulate their own questions and find their own answers. Such a practice may effectively eliminate the belief that school history is different from personal history in some fundamental way. The students in Holt?s study indicated that the contradictions in and comparisons of historical documents were not confusing, but engaging, and like Barton and Levstik, Holt indicates that using source documents to teach history is more likely to make students ?better consumers of the history written by others, as well as more critical thinkers and citizens? (p. 29). This assertion seems to be supported by research indicating that students benefit from considering history through sources other than history textbooks. Researchers such as VanSledright (2002a, 2002b), Wineburg (2001), and Levstik and Barton (2003, 2005) indicate that students at grade levels from elementary school through high school are able to successfully use a variety of source materials as they learn historical information and learn to use that information in ways that benefit students as future citizens. Other research considers the ways the use of source materials affect student learning. Lee and Ashby?s review of the Chata Project (2000) considers the ways students develop an understanding of historical evidence when they learn using source documents, and Bain?s (2000) research sheds light on the ways using multiple sources helps students develop the habits of thought that allow them to effectively analyze and consider historical events. 37 In the review of the Chata Project, which studied the progression of children?s understanding of historical evidence from the second to the eighth grade, Lee and Ashby (2000) found that having students use source texts resulted in a fairly linear progression of students? social studies skills. In the study, students? development of historical reasoning moved from seeing history as ready-made stories that are simply retold by historians to understanding history to be stories told by historians who find, compile, and collate information, and finally to an understanding that historians take the information and create stories which are influenced by a variety of factors that may distort their telling of the story. The study concludes that as students examined controversial issues using multiple sources they were more likely to refer to sources when they encountered differing versions of a story, they seemed to become aware of the power of new ideas, and history was increasingly valued as a difficult but worthwhile subject. Teacher use of source materials. Although a body of research indicates that the use of source materials as a regular part of history classes provides benefits for student learning, many teachers are still reluctant to use these sources and rely on textbooks for the majority of their instructional materials (Paxton, 1999). Grant and Gradwell (2005) studied two teachers who reported using source documents extensively in their classes in order to understand the influences that led these teachers to choose the sources they used. Each teacher reported using a variety of criteria to support the choice of source documents including student interest and reading ability, students? prior knowledge and ability to successfully utilize the documents chosen, and curriculum requirements. Although the researchers suggest that these two teachers may not be typical in the number of source documents they choose to utilize, the criteria they use to chose these 38 documents is important in determining the types of considerations other teachers may also have when choosing to incorporate these types of materials in their instructional activities. This may also provide researchers with information regarding the concerns teachers have as their students use source documents. Although there is evidence that many teachers are using historical source documents as part of their instructional strategies, care should be taken to avoid assuming that all use is equally valuable. Hicks, Doolittle, and Ewing (2004) found that while a high percentage of teachers used both classroom based and web-based source documents regularly in their classrooms, less than half of the teachers in their survey had students analyze source materials in order to participate in activities that would move the students toward more expert historical thinking, such as consideration of context, authenticity, or corroborating accounts. Most teachers indicated that their use of source materials was to reinforce more traditional forms of history instruction by having students read or view a source that corroborated the textbook?s story or simply added an interesting piece of information. This was often true even when their stated goal was to foster inquiry-based instruction in their classroom. The routine use of source materials to accomplish or complement lower-level learning activities, even when higher-level learning is the goal, seems to reinforce the notion that the use of source materials in the classroom does not always provide activities for students that accomplish the goal of authentic, higher-level learning. Issues related to student literacy Convincing social studies teachers to use complex, text-based source documents as an integral part of their history instruction does not mean that they will have success 39 using these materials. Regular use of source documents, especially when that use is part of authentic, inquiry-based instruction, can help a teacher maximize students? motivation to engage in these types of activities, but for many teachers a frequent issue of concern is basic student literacy. The NAEP 2003 Reading Report Card (Donahue, et al., 2003) found that of the students tested throughout the United States, only thirty-one percent of fourth graders and thirty-two percent of eighth graders scored at or above the proficient level. When sixty-eight percent or more of the students in a classroom are not proficient readers, teachers face significant challenges, and because of the complexity of most text- based source documents, history teachers face an even greater challenge if they are going to include these as part of social studies instruction. Basic student literacy. Because the study of history is largely a literary endeavor, it is little wonder that many of the characteristics that make a student a good reader are closely related to characteristics that make someone a good history student (VanSledright, 2002a, 2002b; Wineburg & Martin, 2004). Strong, Silver, Perini, and Tuculescu (2002), lists five characteristics of good readers that clearly show this parallel. They state that good readers: (1) know how to organize ideas and information to fit the task at hand, (2) know how to use questions to filter out the most important information and to clarify points of confusion, (3) know how to use their imagination to make predictions, draw inferences, and create pictures that mirror important concepts in the text, (4) know how to use conversation, dialogue, and retelling to deepen their understanding of the texts they read, (5) recognize when their understanding of texts is confused or mistaken and use strategies to repair their comprehension. 40 The characteristics identified by Strong, et al. (2002) for good readers are aligned closely with the skills identified by VanSledright (2002b) in a continuum he used to rate students? skills as he investigated fifth-graders using historical source documents. Although Strong, et al. considered reading from a basic literacy point of view and VanSledright is considering reading from a domain specific point of view, their lists of skills are largely the same. However, each begins with an assumption that students at a certain level will possess the basic literacy skills needed to begin using these skills. Unfortunately, many teachers have students whose literacy skills place them in a position below VanSledright?s lowest level and make the use of advanced skill difficult if not impossible. Research indicates that three principles have useful implications for improving students? reading comprehension: fluency, breadth of vocabulary, and domain knowledge (Hirsch, 2003; Pang, et al., 2003). Each of these has implications for the researcher looking to improve reading comprehension in a specific subject area, such as history. Fluency, which allows the mind to complete the mechanics of reading almost automatically, is of extreme importance if a student is going to comprehend a passage of text as opposed to simply decoding it. When a student reads a passage of text, yet lacks the fluency required to comprehend the material easily, he or she is much like a person listening to a film in a foreign language in which they are knowledgeable but not fluent. By the time the words have been decoded, their meaning has been lost because everything that came immediately after the first phrase was lost as they decoded the meaning of the words. 41 Breadth of vocabulary is a major contributing factor in how easily and well a student comprehends while reading. Hirsch (2003) points out that a high-performing first grader tends to have twice as many words in his or her vocabulary as a low-performing first grader. This discrepancy in vocabulary tends to widen over time and by the time the low performer and the high performer reach the 12 th grade the high performer may know four times as many words. This means that text-heavy subject areas, like history, become increasingly difficult for the student who does not have or does not develop a broad vocabulary base. Domain knowledge, a ?threshold level of knowledge about the topic being discussed,? is essential if a student is going to make the necessary connections between the words that are used and the way they are put together in a specific domain. The student that possesses a reasonable vocabulary, but does not possess a high degree of domain knowledge, may understand all the words in a given text and yet have no idea what they mean when they are put together as part of a domain-specific text. Fluency, breadth of vocabulary, and domain knowledge are important if history activities and texts are to be utilized and allow students of various backgrounds and ability levels comprehend any subject at an optimum level (Hirsch, 2003; Pang, et al., 2003). Cunningham and Stanovich (1998) studied the cumulative effect reading has on students. Their work indicates that the more students read the better readers they become and the less they read the more they fall behind their peers in reading ability. This is referred to as the ?Matthew effect?, the notion that the rich-get-richer and the poor-get- poorer, an applicable analogy since students who can read reasonably well tend to read more and, therefore, improve their literacy skills. As reading skills improve and student?s vocabulary increases, the reading process becomes more automatic enabling students to 42 spend less energy on decoding language and more energy on comprehending text. The opposite is true for poor readers who often lag further and further behind as they encounter text that is increasingly advanced and above their level to successfully comprehend and use. The researchers found that students who are less able readers tend to get a higher percentage of their information from non-print materials because this provides more ready access to the information than a text they may have a hard time deciphering. Although many teachers try to solve the problem of students with inadequate literacy skills with pictures or film in order to help them understand the necessary historical information and concepts, this study suggests they are actually exacerbating the problem because the students are falling further and further behind and making the classroom teacher?s job increasingly difficult. Cunningham and Stanovich also discovered that the use of non-print material tends to make the overall problem worse because the average spoken language, whether person-to-person communication, television, radio, etc. has the verbal complexity of books written for preschoolers. When a teacher allows students to substitute non-print material for text-based sources, they are not only failing to be exposed to a literacy building activity, but are acquiring the little knowledge they are getting through sources that fail to provide even the slightest challenge to their vocabulary or literacy skills. This study suggests the importance of having all students read as often as possible and as much as possible since this is the only way to ensure they acquire the vocabulary and skills to be successful in areas such as history that rely heavily on reading. Effects of student literacy on teacher behavior. While Cunninghan and Stanovich (1998) considered the ways student literacy affected the ability of the students to perform 43 in the social studies classroom, Gamoran and Nystrand (1992) considered the effects student literacy has on the classroom teacher?s behavior. They studied the ways authentic instruction affects student engagement and achievement in both high- and low-ability classes. They focused on teachers? use of authentic questions, defined as questions that asked students to think in a real-world way, in the classroom. They found that although teachers in both types of classes asked authentic questions about the same percent of the time, they were asking the two groups to perform different tasks. In the high-ability classes the teacher routinely asked the students authentic questions that were directly related to the material they were expected to master. However, in the low-ability classes the teachers tended to ask questions that, while authentic, were not related to in-depth thinking about the course material and could, therefore, not be expected to produce improved performance in those students. In looking at literature classes they found that in high-ability classes sixty-eight percent of the questions related directly to students? thinking about a specific piece of literature, whereas in low-ability literature classes only about twenty-five percent of authentic questions related to the literature that was read. They concluded that when the teacher was asking authentic questions that were unrelated to the content of the reading the questions had a negative effect because, although they were indeed authentic, they had little or nothing to do with what they students were supposed to be learning. Although Gamoran and Nystrand do not propose a reason for these differences, it might be hypothesized that teachers of the low-ability classes are reluctant to ask questions related to the text if they have reservations about the students? ability to comprehend the material. 44 The changes in teacher behavior identified by Gamoran and Nystrand are not, however, the only ways teachers attempt, consciously or unconsciously, to deal with poor literacy skills among their students. For some teachers denying a problem exists or failing to acknowledge the depth of the problem is a way of dealing with the issue of literacy when they do not know how to solve the related problems. Tovani (2000) found that many teachers indicate that they know some of their students have problems with literacy, but because many of these students still tend to perform adequately in the class they do not feel the need to address the issue. Tovani indicates that this is because these students are doing what she calls ?fake-reading?, when a student skims text, reads important parts, and then waits for the discussion or test review to actually understand the material. These tend to be students who refuse to read, are good word-callers but have poor comprehension skills, and who still manage to make good grades on traditional school assessments. According to Tovani the ?two types of struggling readers most often encountered in secondary schools [are]?resistive readers and word callers. Resistive readers can read but choose not to. Word callers can decode the words but don?t understand or remember what they?ve read.? Like Hirsch (2003) and Cunningham and Stanovich (1998), Tovani found that these students tend to fall further behind the longer they are able to avoid in-depth reading and even when they try to really read they either do not have the advanced skills they need or they had them but have lost them through lack of use. These students get stuck and expect the teacher to fix the problem. They have no idea that they can fix the problem themselves because they have not developed the strategies to do so. 45 Domain specific literacy. Although the problems related to general literacy are of concern at every level of education and in every discipline, social studies teachers face additional issues when using text-based source documents in the classroom. Unlike textbooks and other reference materials, these texts are seldom created as instructional materials. They are the written record of human existence and as such, they are texts set within the time and place where they were written. Personal texts, such as diaries, journals, and letters often only have significance if the reader understands the context in which they were written and the person who wrote them. Without this knowledge the reader has little way of understanding the relevance of the text within either its micro- or macro- historical setting. Even public documents, such as the United States Constitution, that are written with the intention that they will be part of the public sphere for a significant period of time, can be difficult to understand and are differently interpreted even by experts. It is little wonder then that students experience difficulty with source documents of all kinds, and yet research indicates that it is the study of these very documents that engages students in the study of history (Rosenzweig, 2000). Reading, especially reading that is associated with learning history through the use of source documents, is an endeavor that requires the reader to understand more than the meaning of words. Rosenblatt (1969, 1985) suggests that a transaction takes place between a reader and the material being read each time an interaction occurs. She further indicates that this transaction is a dynamic encounter in which both the reader and the material being read change as a result. Because a text was written in a specific time and place, for a specific reason, and by a person with feelings and beliefs, it had a particular meaning at the time of its creation. As readers bring their own understanding and beliefs 46 to the interpretation of a text (Rosenblatt, 1969, 1985) and as readers from different cultures understand the text through their own cultural perspectives (Pang, et al., 2003), the meaning readers develop regarding that text is altered. This transaction is required if students are going to read, comprehend, and make use of historical text sources. It is much the same as the transaction that takes place in a personal meeting between people of different backgrounds. Without some knowledge of the other person?s background, cultural experiences or expectations, and language, a meeting between individuals, either in person or through written language will have little chance of success. Unsworth (1999) found that students with the specialized language skills required for a specific domain are more successful in using text sources from that domain and suggests that when students develop these specialized skills they are better able to conduct those transactions Rosenblatt (1969, 1985) discusses. Unsworth suggests that the distinctly different language requirements of domains such as history or science make it possible for students to fully comprehend the words and sentences in text from that domain and still be unable to fully understand and ?transact? with that text. This problem is compounded by the inability of a majority of students to recognize when they are confused, diagnose the problem(s) leading to their confusion, or decide on strategies to repair their faulty understanding (Strong, et al, 2002; Tovani, 2000). In addition to the issue of domain specific language is the issue of text structure. Goldman and Rakestraw (2000) point out that there are clear processes through which readers make sense of text. These processes are dependent on the structure of the text and can be divided into text-driven processing and knowledge-driven processing. "Text- driven processing refers to the use of the content and organization of the text as a basis 47 for the construction of mental representations? (p. 312). In text-driven processing, structural cues are used by the reader to guide comprehension of the text. If these are missing or changed and the reader is unable to supply them as they read, comprehension will be impaired. "Knowledge-driven processing refers to the important role that prior knowledge plays in the ways readers use what they already know to construct mental representations of what they read? (p. 313). Much of text processing in knowledge- driven processing is done through the use of common or expected structural elements within the text. When these elements are changed or missing, comprehension will suffer just as with text-driven processing. Goldman and Rakestraw point out that readers learn to expect certain structural cues from specific genres of text, such as biographies, stories, textbooks, etc. The more a person reads, particularly in school, the more familiar they become with the structures common within the various literary genres, and this familiarity becomes more advanced over time. They also found that instruction is helpful in making readers aware of structure and the role it plays in making sense of text. Research suggests a variety of methods that may be employed to help students develop their domain specific language skills. In a discussion of the effect of reading young adult historical fiction as part of a history curriculum, one of the benefits listed by George and Stix (2000) was the development of domain specific vocabulary and language skills. They and others (Nelson & Nelson, 1999; Paxton, 2002; Tunnell & Ammon, 1996) suggest that text written in a more literary style, such as historical fiction, biography, and other types of trade books, as opposed to the authoritative style of a textbook encourages students to remain focused longer, identify key moral and ethical issues, and comprehend different sides and issues involved, resulting in greater gains in 48 achievement. Additionally, the use of these types of literature, because they are written to entertain as well as inform, are likely to keep students? interest long enough to draw them into a greater study of the subject. These types of sources can increase interest in a subject because of the personal interaction between the student and the characters in the text, and students who are interested in a topic are more likely to want to read historical source documents from similar historic individuals instead of the generalized information found in a textbook (Elliot & Dupuis, 2002; Nelson & Nelson, 1999; Tunnell & Ammon, 1996). The complexities involved in understanding source documents, even for students whose general literacy skills are adequate to proficient, seem to indicate that additional skills or other types of literacy are needed to make this interaction successful. VanSledright (2002a, 2002b) suggests a continuum of literacy requirements that begin with basic literacy skills and end with domain specific literacy skills. In VanSledright?s continuum the first and second levels center on the students? ability to successfully interact with the text on a basic level. In the first step of this process, a student ?checks details, re-reads, summarizes, and/or predicts developments in the source?, a process that enables the student to make initial sense of text. In the second step the students judge aspects of the source in order to determine if its various elements make sense. In the final two steps of the continuum students continue their analysis of the texts, but the analysis extends beyond the actual text. For steps three and four, students must move beyond the ability to read and understand the text as a stand-alone document and begin to understand and analyze the text within a larger context. The students consider the text?s author?s point of view and place in history as an important aspect of understanding the text. This 49 leads to an evaluation of the ?sources? reliability, subtext, and agent intentions.? This continuum helps clarify the reasons many students that may have sufficient literacy skills in general remain unable to successfully utilize the complex text-based source documents that must be part of any historical investigation. Social literacy. While basic literacy skills allow a reader to make sense of text and domain specific skills allow a reader to make sense of text written for a specific domain, another type of literacy is needed for readers of historical sources to be able to understand and utilize these source documents in ways that will allow them to meaningfully participate in historical inquiry. Students need to have literacy skills related to social interactions to take them beyond understanding the text of source documents and allow them to understand the social context of these materials. This type of social literacy includes such skills as moral reasoning, empathy, and the willingness to look at individuals from the past through the lens of the past without imposing pre-conceived notions of the present on them. In addition to the general literacy demands of complex text-based source documents, students must also develop specific social literacies directly related to the study of history if they are going to fully utilize source documents as part of historical study. Kohlberg (1976) found that the development of moral reasoning skills occurs in stages and that individuals can only be expected to perform at their current level of development, although some individuals were capable of limited functioning one stage above their current level. The ability to reason morally as described by Kohlberg is an aspect of social literacy. If, as Kohlberg contends, this reasoning develops in stages, 50 student reasoning may develop more successfully when teachers provide assistance to help them move through the stages more quickly and successfully. Historical empathy, like moral reasoning, is an aspect of social literacy and is a necessary skill for students if they are going to be able to understand the past and the people who inhabited it. VanSledright (2002a) found three distinct problems in students? development of historical empathy. First, both the teacher and students come to any consideration of history with preconceived ideas about the past and their relationship to events and people from the past. Second, students must pay close attention to historical context if they are going to be able to successfully develop an empathetic understanding of historical agents. However, the level of attention required is significant, especially for novice historians such as students. Third, the urge to view the past through the lens of the present must be avoided in order to ensure that present positions and beliefs are not imposed on people and events of the past. Ogawa (2000) also considered students? development of historical empathy and, like VanSledright, found that students exhibited greater historical empathy when they considered and analyzed a variety of sources and perspectives from which to develop the understanding of a particular event. Lowenthall (2000) identifies five ?special demands of historical understanding?, and although historical empathy is not specifically named as part of the five, the characteristics that make up empathy are included in this list as well. Most closely related to historical empathy, as it is described by VanSledright and Ogawa, is Lowenthall?s third demand, the ?awareness of manifold truths. This is the ?ability to understand why different viewers are bound to know the past differently.? In order to understand how individuals may see, understand, and interpret the same events 51 differently a student must possess many of those characteristics required for empathy. Lowenthall goes on to describe four other demands. Two of these are comparative judgment and familiarity. Comparative judgment is the ability to use and critique a wide variety of sources, many of which will be contradictory in their treatment of the same events. Comparative judgment, in part, depends on familiarity, the ability to make use of common historical information in order to place people and events in the broader contexts of their time and place. The fourth demand discussed by Lowenthall is the ability to appreciate the value of the work and traditions of those who have considered the same events, but Lowenthall explains that this appreciation should not result in ?blind veneration? nor should the student believe that they should accept these previous views without question. This willingness to question the beliefs and assumptions of those before us is also an aspect of Lowenthall?s final demand, hindsight. Hindsight as described here is the ?awareness that knowing the past is not like knowing the present and that history changes as new data, perceptions, contexts, and syntheses go on unfolding? (p. 64). Any discussion of having students work to acquire these social literacies must also address the question of the value of these literacies to the average person. The value of these social literacies stretches well beyond the history classroom, however, when one considers the value of historical empathy (Ogawa, 2000; VanSledright, 2002a), the ability to recognize differing understandings of the past, the ability to appreciate the value of previous historical interpretations, the understanding that the past is uniquely different from the present (Lowenthall, 2000), and the ability to understand and critique conflicting sources, connect these to commonly understood historical references, and make use of 52 them in the present (Holt, 1990; Lowenthall, 2000; Sexias, 2000; Wineburg & Martin, 2004). Each of these literacies has value beyond the classroom because each is ?meant to make all students, even the majority who will not become historians, better consumers of the history written by others, more critical thinkers and citizens? (Holt, 1990, p. 29), as well as provide students with the frameworks they will need to continually add to and revise their understanding of the past and its relation to the present (Lee & Ashby, 2000; Levstik & Barton, 2001). Expert v. Novice behavior Those habits, behaviors, and skills that make up both basic and social literacy are specialized knowledge that historians, as experts, possess and students, as novices, must learn if they are going to successfully interact with complex source documents. Developing these expert habits of mind enable students to engage in the reasoned decision-making that is necessary for citizens as they participate in the government of a pluralistic, democratic society such as the United States (VanSledright, 2004, Wineburg, 1999). However, in order to move students toward the goal of becoming expert thinkers, educators must first understand the differences in expert and novice behavior, habits, and skills in both basic literacy and in the domain specific social literacies. In order to develop strategies to make this interaction more successful for students with inadequate literacy skills, it is important to understand the differences between the behaviors of experts and that of novices since, ?by definition, [novices] do not employ the heuristics and strategies that experts do? (VanSledright, 2002a, p. 160). Students must develop specific areas of comprehension if they are going to move from novice to more expert historical thinking. These include an understanding of historical time or the 53 individual?s existence within history (Downey & Levstik, 1991), development of a pattern within which the information makes sense specific to the task required (Downey & Levstik, 1991; Khong, 1992; Martorella, 1991; VanSickle, 1996; VanSickle & Hoge, 1991;), and the ability to construct a narrative understanding of historical events (Levstik & Barton, 2005). Novice v. expert behavior: General literacy. It is also important to understand the differences in the ways students read complex texts as they move from novice readers to expert readers. Studies find that even students who appear to be expert readers often do not exhibit those qualities associated with expert domain-specific readers, resulting in an inability to fully comprehend and utilize texts encountered in a history classroom (Goldman & Rakestraw, 2001; Tovani, 2000; Unsworth, 1999; VanSledright, 2002a, 2002b;). Additionally, Fallace and Neem (2005) argue that many teachers do not exhibit or possess the skills and dispositions needed to think historically either. They contend that both teachers and students must have the skills that make it possible for them to engage in historical thought as well as judge the value of the historical thought exhibited by others in order to adequately engage in a study of history that will prepare young people to meet the challenges of citizenship in our society. Bain?s (2000) study into students? use of textbooks found that the use of history textbooks masks the work done by experts, in this case historians, and prevents students from understanding and duplicating those behaviors common among these experts. When dealing with historical information experts consider conflicting accounts, interpret the meaning, analyze a variety of sources, and develop a narrative that is open for further interpretation and change. Historians actually expect others to analyze their work, 54 question it, and change it as new information emerges or new interpretations are made. Novices, including a majority of history students and some adults, do not exhibit these qualities and most do not even understand that these are the behaviors exhibited by the experts in this field. Marks, et al. (1996), Goldman and Rakestraw (2001), and VanSledright (2002a, 2002b) found that the level of expertise students bring to a learning environment significantly affects their interaction with and retention of the information being presented. Marks, et al., examined the use of authentic instruction in elementary, middle, and high school settings and found that, although students at all levels showed measurable improvement with the use of authentic instruction, those students that began with greater knowledge and skills benefited at a higher level than those that began with lower knowledge and skills. The greater initial knowledge and skills allowed these students to access prior knowledge and skills and add to this knowledge as they experienced the authentic instruction that was the basis of the study (68-9). Goldman and Rakestraw (2000) indicate that the expert knowledge students have in a subject influences the effectiveness of specific text presentations. While students with higher levels of expertise in the area being studied benefited from a less linear style of presentation, such as is generally seen in hypertext, students with novice levels of expertise benefited from a more ?linear, singular presentation?, as generally seen in more traditional text. In VanSledright?s research with fifth-graders in a history course, those students with lower reading and related skills exhibited more difficulty and progressed both slower and not as far as those students that began the study with greater skills. Goldman and Rakestraw, as well as VanSledright found that those students without prior 55 knowledge had significant difficulty creating the new mental files needed to process complex and unfamiliar information. Because this information was unrelated to knowledge and skills the student had previously acquired and become comfortable with, they were unable to learn as quickly or retain as much as those students for whom the new information was a continuation of previously learned material. Although many researchers have found that a student?s level of expertise is extremely important as they interact with new information and learning environments (Antoniette, Imperio, Rasi, & Sacco, 2001; Marks, et al., 1996; VanSledright, 2002a, 2002b), Cousin (1989) found that when learning environments were adjusted to accommodate students with novice levels of knowledge and skills, all students tended to benefit. Cousin did not, however, consider the ways these accommodations were used by students with varying levels of expertise. Therefore, while this study helps the teacher concerned that they not limit the potential of student experts while assisting student novices, it does not provide any information concerning the differing ways these two groups of students utilize the material. Novice v. expert behavior: Historical domain. When students and historians consider source documents, their behaviors are substantially different in several areas (Wineburg, 1991a, 1991b). Historians tend to use a variety of sources and identify important information about that source, such as who wrote or created it, when was it created, and why was it created. They use these different sources to corroborate each other, and they consider the context of the source. These behaviors are considered standard for historians when dealing with any type of source document, so that even when they are considering completely unfamiliar documents or events outside their 56 specialty area, they continue to make use of the schema they have developed as part of their expertise. These behaviors are not, however, readily apparent to novices, such as students. In Wineburg?s study, when students considered the same source documents as the historians, they behaved very differently. While ninety-eight percent of the historians actively considered the source for each document they used, students only did this thirty- one percent of the time. Students tended to view text as a "vehicle for conveying information", whereas historians viewed each text as a means for understanding a particular point of view of a particular person at a particular time and place. While historians regularly compared sources and used them for corroboration, students tended to consider each one independently and move on. Throughout this study the students maintained a stance in which they assumed there was a ?correct? answer, and they were on a mission to determine which of the sources gave it. Finally historians were much more likely to consider and search for the context for each document. The differences in the behaviors of the historians and students in Wineburg?s (1991a) study are important because they resulted, not only in the groups coming to different conclusions, but in the two groups understanding the purpose of the task differently as well. In the study both groups read a series of documents about an event in American history and then studied several paintings depicting the event. Each participant was asked to consider each source and then choose the painting they believed was the best depiction of the event. Although historians applied what they knew from the texts to determine the best answer, they clearly expressed reservations about choosing one depiction when all had discrepancies. The students took more of a test-taking stance and acted as if there were one right answer on a multiple-choice test. They chose the "correct 57 answer" and did not feel the need to qualify their choice with doubt regarding the task. Wineburg?s results suggest that while there are clear differences between the behaviors of experts and those of novices, these differences influence not only the short term behavior, such as analyzing a document, but also influence the overall understanding each group had for the domain. The broader implication suggested by Wineburg?s work is seen in Sexias?s (1994) study. He found that students differed from historians in their understanding of the significance of historical events. He found that students often attribute significance to specific events for several reasons: masses of people or large geographic areas affected, the scale of the historical event, importance to personal history, significance to a specific group, and the ability of the event to directly inform the present and/or add to an enlightened present. Sexias points out that in this study the students? ways of determining relevance was often random and individualized with little direction and few unifying considerations. As with Wineburg?s findings, students? inability to perform in a more expert-like manner impacts their ability to understand historical events and concepts as well as their ability to utilize their knowledge of history in a meaningful way. Stahl, Hynd, Britton, McNish, and Bosquet (1996) found that when students read multiple source texts and attempt to construct meaning about a specific event from these texts students differ from experts in another key area. Where experts tend to expand their knowledge base with each document they utilize as they study a particular event, students tend to become more consistent in their understanding but do not necessarily expand that understanding in a noticeable or consistent way. This could be due to the nature of the texts used in this study, and Stahl, et al. suggest that the contradictory texts may have 58 caused students to look for similarities instead of look for new information. Either of these explanations suggest that if students are completing a complicated task, appropriate levels of assistance might be used to encourage students to perform expert tasks, such as constructing meaning from contradictory texts, additional assistance may encourage them to achieve results that are more like those from experts. Like Stahl, et al., Hynd, Hubbard, Holschuh, Reinking, and Jacobson (2000) found that even when students are performing tasks that experts perform, such as sourcing, contextualization, and corroboration, the level at which they are performing these tasks if often well below a level that will allow them to achieve more expert results. Scaffolding When considering the goals for successful use of complex social texts in the social science classroom, the ultimate goal is to have students, as novices, interact with the text using the strategies and methods historians or expert history readers would. In order to achieve this students must have continuous access to assistance throughout their use of the text to help them understand difficult words or phrases, gain and comprehend relevant historical and social background information, and to instruct and assist them as they struggle to develop the metacognitive skills needed to move from novice to expert (Abbott, 1997; Afflerback & VanSledright, 2001; Bain, 2000; Hannafin, Land, & Oliver, 1999; Parker, Mueller, & Wendling, 1989; Saye & Brush, 2004b; Stearns, 2000; Taylor, 1999; VanSledright, 2004; Vygotsky, 1978). Such assistance is likely to increase students? confidence in the likelihood of success, and as a result students are more likely to be motivated to strive for greater achievement (Hicks, Doolittle, & Ewing, 2004; Levstik & Barton, 2005; VanSickle, 1996). 59 Types of scaffolding. Scaffolding can be differentiated by the type of assistance it provides for students. Definitional scaffolding would provide students with definitions for unfamiliar words and phrases. Students may have difficulty understanding words and phrases for a variety of reasons such as: inadequate vocabulary, inadequate literacy skills, domain specific terms or phrases the student is unfamiliar with, and archaic words or phrases the student is unfamiliar with. When students are unable to understand words or phrases in text they have several options, skip the word or phrase and hope it is not essential to the material, guess the meaning and hope they are correct or almost correct, or stop and look up the word in a dictionary or glossary if it is not too archaic or obscure to be included (Stahl, 2003; Tovani, 2000). Unfortunately, each of these solutions breaks up the continuity of the text for the student hindering cohesiveness and comprehension. If students find or ask for assistance and then continue reading, continuity is still disrupted but the student may be able to understand the word or phrase. If, however the student skips the word or guesses the meaning incorrectly, not only will continuity be affected, but understanding will be as well (Allington, 2002; Cunningham & Stanovich, 1998; Gunning, 2003; James, Black, & McCormick, 2003; Meyer, 2003; Tovani, 2000). When scaffolding can be provided to give students on-time assistance with definitional problems, it is possible that students would be able to read more difficult material without losing continuity or hindering basic understanding. Historical source material often requires that the reader have a basic understanding of events from the past that have some relevance to the text. When the reader is familiar with the necessary background information they will be more likely to make necessary historical connections and understand references the author may be 60 making. However, when the reader does not understand these references and cannot make those connections, entire source documents may make little or not sense to the reader (Strong, et al., 2002). It is often difficult for students to understand the confusion that occurs as a result of needing background information, especially students with poor reading skills. These students will often indicate that they understand all of the words but the text still does not make sense. This can be very frustrating to students as they struggle to understand difficult material and unless the student is engaged in the reading to the point that they are willing to find the necessary information and then incorporate this into their overall study, it is unlikely they will be able to make the historical connections needed to understand the source within its historical context (Afflerbach & VanSledright, 2001; Goldman & Rakestraw, 2000; Wineburg, 1991a, 1991b). Helping students learn to think and reason at advanced levels requires assistance as they read complex source materials and consider difficult issues but do not have the metacognitive skills needed to do this effectively. Taylor (1999) and Kesselman (2003) discussed the difficulties associated with students learning to use metacognitive skills, such as thinking and reasoning skills, at an advanced level. Taylor contends that because most students see themselves as passive learners, thinking and reasoning skills should be taught as part of the educational curriculum enabling students to increase their ?capacity to think critically, solve complex problems, act in a principled manner, be dependable, read, write, and speak effectively, have respect for others, adapt to change, and engage in lifelong learning.? When metacognitive processes are taught as part of the instructional process learning is more durable and students are better able to transfer the knowledge gained. Metacognition allows students to appraise their own thinking, knowledge, 61 motivation, and abilities, as well as manage the mental processes they are using and need to use for problem solving. Teaching students to successfully manage their metacognitive processes requires that instruction in questioning techniques take place including both general questioning as well as domain specific questioning. The goal of this instruction is for students to be able to formulate their own questions and intuitively manage their metacognitive processes. Taylor concludes that teaching students to manage their metacognitive processes is best done through repeated practice in self-interrogation, introspection, and interpretation of ongoing experiences. Students must be able to connect concepts and the answers to their internal questions with prior knowledge and patterns if they are going to acquire lasting knowledge that is transferable to other applications. Although the types of scaffolding needed by students as they engage in the use of text-based source documents are most often considered something a teacher must provide in a one-on-one interaction with the students, this is not always the case. Saye and Brush (2002) divide scaffolding into two types, hard and soft. Soft scaffolding is that assistance that must be provided in person by the teacher. This assistance is done on an as-needed basis and may be provided for an individual, group, or the entire class, but is always done based on a need that becomes apparent only after students have begun working. Hard scaffolding, on the other hand, can be used to provide that assistance a teacher can anticipate all or most students needing during the course of the instruction. For example, a teacher may know in advance that many students will be unable to place a particular passage of text in its historical context and can then find a way to make this assistance available for all students. Hard scaffolding is provided in such a way that students access 62 it on their own with little or no in-person assistance from the teacher, such as by referring to a definition or question at a critical point in their reading. Atkinson, Renkl, and Merrill (2003) also found that scaffolding, particularly scaffolding that Saye and Brush would call ?hard scaffolding?, can successfully allow students to access assistance that provides examples and allows students to develop increasing levels of expertise. As students? expertise increases, the use of scaffolding can strategically decrease over time. Scaffolding inquiry-learning. Conducting research within science classrooms, Keselman (2003) found that students do not naturally consider the ?multiple causality? of many scientific problems, nor do they actively question their metacognitive processes as they engage in any type of inquiry or problem-solving activity. The students in this study were involved in an inquiry-learning activity related to earthquakes and flooding in which they were expected to consider multiple causal factors and make predictions based on these factors. The study found that students used analysis skills randomly and often identified no logical causation even when situations were similar. Keselman suggests that sufficient scaffolding could assist students in developing an understanding of multiple causality as well as metacognitive skills as they engage in inquiry activities. These recommendations are applicable to history instruction as well because, like science, historical study is by nature multi-causal, and students in all disciplines are frequently unfamiliar with the concept of considering their metacognitive processes as part of the problem-solving process. In his study of fifth-graders participating in historical investigation, VanSledright (2002a, 2002b) found that even students in upper elementary school are capable of high levels of complex historical thinking and reasoning. He found that these students not only 63 engaged in this type of in-depth study, but also benefited in more substantive ways from their participation in a class that allowed them to take some control over their own learning. "Taking some responsibility for one's own learning can be daunting. However, in the safety of the classroom, where a teacher can assist should efforts be sidetracked, I can think of few more important experiences we would want to provide for our children on their way to becoming fully functioning adult learners " (p. 151). VanSledright?s analysis of these students? learning indicates the significance of reading and analyzing a variety of source documents, most of which are text-based and require a high level of literacy skill. Students without these advanced literacy skills are likely to find their efforts frustrating unless they can be provided with significant levels of scaffolding to support them as they acquire and perfect those skills. As VanSledright points out, these efforts are most likely to have positive outcomes when they are conducted in the ?safety of the classroom.? If, however, the number of students needing assistance and the levels of assistance needed are overwhelming for the teacher, other methods of providing support besides personal or soft scaffolding by the teacher must be found or many students will find themselves as frustrated as they would be in a class with no support at all. Additionally, the scaffolding that is provided must require students to move beyond the novice level of historical thinking if the scaffolding is going to benefit students as they develop their historical thinking capabilities, although providing this type of scaffolding can be difficult for many teachers (Saye & Brush, 2002b). Scaffolding Complex Text Sources. While scaffolding can provide students with needed support as they develop advanced level metacognitive skills, specific types of 64 scaffolding are needed to assist students as they interact with complex texts. Bean and Ericson?s (1989) study into students? use of textbooks indicates that teachers must find ways to ?project the students actively into the context of their reading? and provide scaffolding in three categories: factual information, assistance connecting prior knowledge to new knowledge, and problem-solving assistance. Bean and Ericson, Strong, et al (2002), Elliott and Dupuis (2002), Beck, McKeown, Hamilton, Kucan (1997), and Manifold (1997) all indicate that students must be provided with various types of assistance that enable them to actively seek information, grasp prior connections to new information, understand complex and often abstract concepts, and understand how the material in the text is important and relevant to what they are doing, Stevens, Slaving, and Famish (1991) also found that when students are provided with assistance in these areas their performance improves significantly. Although a history teacher may need to provide a variety of types of scaffolding, Unsworth?s (1999) findings that functional grammatical knowledge differs depending on the literacy demands of the specific text further indicate the need for special consideration when providing scaffolding specifically for students as they read and use complex text sources. Unsworth indicates that the literacy demands of everyday language are distinctly different from the literacy demands of content area language, making it is necessary to address the specific demands of historical and scientific language if students are going to be successful reading and comprehending texts in these areas. Students who are able to develop this content area grammatical knowledge are at a distinct advantage over students who may have a high level of general grammatical knowledge, but who do not have the domain specific knowledge they need. 65 Tovani (2000) also found that students need specialized types of assistance if they are going to move from superficial reading of text sources to successfully utilizing text materials for specific purposes. According to Tovani there are several reasons this assistance is needed. Teachers often do not recognize that students have reading difficulties since many students have learned to mask difficulties while accomplishing the requirements of traditional classrooms (p. 14). A majority of students also do not have the level of expertise needed to know how to recognize when their reading has become ineffective or how to fix problems with their comprehension once a problem has occurred (p. 16). Tovani suggests several types of assistance, called cues, which should be provided for students to successfully interact with complex text sources. These include cues that: help students understand the basic information regarding specific words and phrases, assist students in developing understanding of complex grammatical structures, help students understand subtle definitions and nuances, help students connect new information to prior knowledge, and help students understand the ?social construction of meaning? and develop their own meaning as well. While scaffolding students? use of complex text sources may take many forms, there is evidence that this type of assistance has positive effects. Working with a middle school student with poor literacy skills, McConnell (2003) found that sustained literacy intervention and scaffolding led to a significant improvement in all of the student?s literacy skills. Although these findings are from a study of only one student, they indicate that individualized scaffolding that is provided as the student needs it may result in improved general literacy skills and possibly in improved domain specific literacy skills as well. McConnell?s work with one student might also indicate that when available 66 scaffolding can be utilized by students in the way best suited to their individual needs, they are more likely to be successful in their understanding and use of the source documents for which the scaffolding was provided. This level of personalized assistance, however, is very difficult for a teacher with a class of students with varying levels of literacy skills. Use of technology in education How can teachers bridge the gap between the types of expert behavior historians exhibit when they encounter a complex social text and the behavior of student novices when they are reading these same texts? The ability to bridge this gap is potentially the key to overcoming the reluctance of many teachers to include these texts in the sources they are comfortable incorporating into their instruction. One answer to the question of how to bridge this gap is through the use of technology, although this solution produces its own apprehension on the part of many teachers who have seen technology used in many classrooms in ways that were inconsistent with wise instructional practice (Berson, Lee, & Stuckart, 2001; Burke, 2002). The NAEP Nation's Report Card: U. S. History 2001 (Lapp, et al. 2002) found that students in grades four, eight, and twelve that report using a computer daily in social studies class had lower than average scores than those that reported less usage. The report states that this statistic was from a very small percentage and there is no indication of the types of activities these students were doing with computers. These types of reports add to the apprehension of some teachers about the use of technology. There is general agreement that the Internet is, or at least could be, a useful tool for the social science classroom. Despite the negative report on overall student use of 67 technology in social studies classes, the NAEP Nation's Report Card: U. S. History 2001 (Lapp, et al. 2002) found ?a strong positive correlation between students in the 8 th and 12 th grades using computers for conducting research and for writing reports and performance? (xii). VanFossen and Shiveley (2003) found, however, that many social studies teachers choose not to use the Internet and that there is a significant lack of research into effective use of Internet resources. In an attempt to understand why more teachers do not make use of the Internet, the researchers looked at the content of Internet sessions presented at NCSS annual meetings between 1995 and 2002. Their analysis suggested a lack of research in best practices for using technology resources that presents teachers with a significant challenge. This dearth of research relating to wise instructional practices utilizing technology resources is potentially a limiting factor for teachers who would like to plan and implement successful technology-based instruction. Lack of a sufficient research base also means that much current research must be based on anecdotal evidence or research from other fields that are related to a chosen topic. As students become familiar and comfortable with specific computer programs, research indicates that they begin to exhibit more expert behavior (Beaufils, 2000; Brosnan, 1998; Chou & Lui, 2005). Brosnan (1998) reported that the greater a student?s anxiety is regarding their ability to perform in a technology-based learning environment, the lower their performance will be. In this study, students that were more comfortable with the learning environment and tools were able to exhibit more expert behaviors and better performance. Beaufils (2000) found that students using note-taking software to facilitate their use of a hypermedia learning environment exhibited behaviors such as repeating the use of strategies that had proven successful in the past and the use of 68 multiple tools at the same time. These behaviors are generally associated with more expert use of these tools. In a study on the effects of learner control in technology-based instructional environments, Chou and Lui (2005) also found that the more familiar and comfortable students became with the technology they were using, the more likely they were to exhibit patterns of use similar to those of expert users. Although these studies were not unique to the history domain, the findings are applicable in a consideration of students in a history class developing the habits of experts in the use of content material and in the metacognitive processes needed to successfully use that information and those skills. Comparing traditional and technology-based learning environments. Chang (2001) also studied students in science classrooms using a problem-solving computer assisted instructional environment. In this study students in the computer assisted instructional environment were compared to students in a more traditional environment that consisted of lecture, Internet investigation, and discussion. The scores from these two groups were compared on pre-/post-tests in order to determine what, if any, differences existed between the achievements of the two groups. The study results indicate that the computer assisted problem-solving instructional environment was "marginally more effective in promoting students' achievement than the lecture-internet- discussion? environment. Additionally, the computer-based instruction produced significantly better results at the knowledge level and slightly better results at the comprehension level when compared with the more traditional method. The researcher points out that this study's findings correspond to the findings of other studies that suggest computer assisted problem-solving can improve students' abilities to remember 69 and use factual information. Chang?s finding that computer assisted instruction resulted in improved comprehension as well as improved knowledge level information retention indicates that using technology to scaffold students working with complex text sources may provide successful assistance on both the informational levels and the metacognitive level. Chang, Sung, and Chen (2001) studied seventh graders as they created concept maps in a biology class. Students in the study completed their concept maps in one of three ways: with pen and paper, on a computer, or on a scaffold using a computer. In each environment students had access to feedback and hints as they worked, but when working on the computer the feedback was more instantaneous and controlled by the student. The results of the study showed little difference in the level at which the two groups working without a scaffold learned the biology concepts, however the students working with the scaffold outperformed both groups. The results also showed that the two groups working with computers were better able to complete the concepts maps, but as stated above, only the group working with the scaffold exhibited a positive effect on the ability to learn the biology concepts being studied. Chang, et al. concludes that in this study the combination of scaffolded-learning and on-demand access to hints and assistance provided students with the most successful learning environment. Other researchers have also examined the use of technology-based instruction in comparison to traditional instructional methods as they considered how student control within a technology-based learning environment affects student learning. In studies with one group using traditional instructional methods and the other using technology-based learning environments, students in the technology-based environments tended to out 70 perform those in the traditional environment, they were more satisfied with the learning experience, and the students believed they learned more than the students in the traditional environment believed they had learned. This finding was consistent in a study of general technology (Chou & Liu, 2005), as well as in studies of technology use in history classes (Britt, et al., 2000; Brush and Saye, 2001; Saye & Brush, 1999). Student learning in a technology-based environment. Chou and Liu (2005) found that as students became more comfortable with their use of the technology they were more confident in their abilities and began to exhibit more expert patterns of behavior. These findings are consistent with those of Brosnan (1998) who found that greater computer anxiety resulted in lower student performance. Additionally, Brosnan found that student anxiety about reading and writing abilities also played a significant part in their level of anxiety regarding technology-based learning environments. Both Chou and Lui and Brosnan found that student performance is enhanced as they become comfortable with the technology they are using. The methods and strategies students utilize when they use technology as part of instructional environments affects their ability to learn the material being presented. Some teachers mistakenly believe that students are comfortable with technology and therefore can be allowed to freely access technology-based instruction. Wiedenbeck, Zavala, and Nawyn (2000) found, however, that when students receive little or no guided instruction using the technology or receive only directed instructional activities, the level of learning is significantly less than when students are given guided instruction followed by free exploration. In this study of experienced computer users, the group for whom directed instructional activities were followed by free use of the instructional technology 71 resource produced more innovative final products and was more willing to venture into unfamiliar areas within the source than either the free exploration group or the directed activities group, and the combined group produced fewer errors than the free exploration group although slightly more than the directed activities group. These findings have implications for the use of scaffolds in technology-based learning environments. An instructional model based on this research would include guidance and support for students at every phase of a technology-based instructional activity. Using technology to provide scaffolding Using scaffolding can be an effective means of assisting students as they acquire and practice the skills exhibited by expert readers of complex text-based source documents (Bain, 2000), but the problem remains that it is impossible for a teacher to provide this level of individualized scaffolding for an entire class. Research indicates that this is an area where the use of technology may be valuable. If scaffolds can be added to any document that is accessed electronically, much of the individualized scaffolding students need might be provided in this way. Hyperlinks are interactive links to additional materials that may assist students as they interact with a text. These links can provide definitions and examples, background or historical information, or metacognitive assistance in the form that is most appropriate for the specific needs related to the text (Berson, et al., 2001; Britt, et al., 2000; Brush & Saye, 2001; Hannafin, Land, et al., 1999). Although the ability to incorporate scaffolds into complex texts could potentially provide immediate assistance as students interact with complex social texts and move from novice to expert strategies, there are concerns that this strategy could result in additional problems as a result of increased cognitive requirements. 72 In this section, three terms are used to describe hyperlinked scaffolds: hypermedia, hypertext, and hyperlink. Although each of these terms refer to slightly different types of hyperlinked scaffolding, for the purposes of this discussion they are exemplary of a type technology use and, as such, will be used as each of the researchers discussed used them without drawing a distinction between each since this distinction is not relevant to the discussion here. Comparing hypertext and text scaffolding. In a discussion of research regarding students reading hypertext compared to traditional text, Goldman and Rakestraw (2000) point out that reading and using hypertext is different from traditional text in several key ways that present challenges to readers who are generally more familiar with the linear structure of traditional text. However, they point out that there are many aspects of both traditional and hypertext that are similar. The major differences between these two is that traditional text presents material in a linear fashion and largely controls the amount of information the reader has access to and the structure through which that information is presented, whereas hypertext requires the reader to make decisions about which information is needed and access it based on that assessment. These researchers found that one of the most advantageous uses of hypertext is to support ill-structured and complex instructional approaches. They speculate that this is likely due to the need for learners to use information from a variety of sources in order to fully comprehend a complex domain. Reisslein, Atkinson, and Reisslein (2004) also found that computer-based support can be beneficial for students working in a complex, ill-structured learning environment. Their findings indicate, however, that students benefit more from textual rather than 73 pictorial prompts in this type of setting where the students were unfamiliar with the subject matter. In addition to finding that textual prompts were more beneficial, they also found that students did better when prompts were externally regulated, as opposed to student regulated. Although they do not speculate, this may be a result of decreased cognitive load as the students struggle to work in an unfamiliar learning environment on an unfamiliar subject matter and/or do not have a level of expertise that allows them to know when they need assistance. This study also indicates that those students using the externally regulated, textual prompts were more likely to indicate a desire for further study on the subject. The researchers speculate that this is likely a result of their initial success working with the subject. Reisslein, et al, indicate that their results are only applicable for students with no previous knowledge of the topic they were studying and suggest that further study should be done to determine if these results would be duplicated with students that have greater previous domain specific knowledge. One area not discussed by the researchers is the reading skills of the students in the study. Although these students were novices in the domain specific knowledge, reading ability could play a significant role in their preference for and use of textual versus pictorial prompts. Using hypertext scaffolding. There is evidence that hypermedia may have positive effects on learners and learning environments. Spoehr and Spoehr (1994) found that the use of hyperlinked scaffolding may support students in a variety of activities that help them consider history in more expert ways and that supports teachers? use of effective teaching strategies. They also suggest using hypertext scaffolding may help support different learning and instructional styles. As a result of a six-year study of students using 74 a program that utilized hyperlinked scaffolding in history instruction, students? conceptual representations of historical concepts improved. This improvement was seen in activities as varied as written assignments, Socratic discussions, and the creation of concept maps using the information gained from they hyperlinked materials. Students in this study were better able to construct historical knowledge and understand historical connects as a result of their use of scaffolding in this setting. These positive indications seen by Spoehr and Spoehr (1994) are not unconditional however. While instruction that gives the learner such a large measure of control can make hypermedia environments more motivating for some learners, this is not universally true. When learners lack adequate levels of metacognitive knowledge and skills they may be unable to learn in instructional environments with high levels of learner control. Some studies also indicate that students experience significant levels of cognitive overload as they attempt to negotiate instructional environments that have less hierarchical structure, such as a hypermedia instructional environment (Shapiro, 1999; Saye & Brush, 2002, 2004b). Influence of prior knowledge on use of hypertext scaffolding. Researchers have found that the effectiveness of hypertext scaffolding is influenced by a variety of factors. One of these is the users? prior knowledge of the topic and/or the technology being used. Antonietti, Imperio, Rasi, and Sacco (2001) examined students? use of hypermedia to access information related to instructional text in an undergraduate engineering course and found that hypertext scaffolding was used most successfully by the students in their study when they had already been exposed to the instructional material and were using the hypertext scaffolding to clear up misconceptions or gain further knowledge. These 75 researchers posited that this was a result of the students? ability to create mental concept maps of the material being discussed in the scaffolding because of their earlier experiences with instructional environment. Although this finding held true for students in the overall study, when students were divided by degree of experience, students with higher levels of previous knowledge found that using the hypermedia scaffolding prior to the instructional environment led to increased learning, whereas inexperienced students found that using hypermedia scaffolding prior to the instructional environment resulted in a decrease in learning. Last, O?Donnell, and Kelly (2001) also found strong links to prior knowledge and successful use of hypertext scaffolding. In their study students with greater prior knowledge experienced less confusion in using the scaffolds, were more comfortable with the structure of the scaffolding, and exhibited greater gains of knowledge than students that began the study with low prior knowledge about the topic. Like Last, et al., Lee and Clark (2003) found an important link between prior knowledge of historical information and skills with successful use of technology resources, although their study considered students? use of Internet resources instead of hyperlinked resources. In each of these studies, students that experienced the greatest successes had begun with the greatest prior knowledge. Additionally, Last, et al. found that students with greater prior knowledge were more likely to identify when they were using ineffective strategies in the use of the scaffolds and alter their strategy to one more successful. In a study by Stephens, Lehr, Thorp, Ewing, and Hicks (2005), students in an undergraduate history course used a hypermedia-learning environment to analyze text- based documents. The technologies used by these researchers provided scaffolding for 76 students as they utilized source documents and encouraged their development of historical thinking strategies such as sourcing, contextualizing, and corroboration. They found that while many students were familiar with the technological aspects of a hypermedia-learning environment, they were still unfamiliar with the basics of historical analysis. For these students, the cognitive load was greatest as they attempted to use the strategies that would be common for expert historical thinkers and not as a result of the technology. The technology used by the researchers was specifically designed to assist students as they interacted with text-based source documents, however, and it can be hypothesized that this increased the ability of students to use the intervention without becoming frustrated. In this study the scaffolding did seem to assist students in their use of historical analysis strategies. These researchers deliberately created an environment that considered students? potential lack of prior knowledge and attempted to accommodate the increased needs of these students. Their successes suggest that when prior knowledge is considered as part of an intervention the results can be improved for students who might not otherwise be as successful. Shapiro (1999) also found that prior knowledge influenced students? learning in a hypertext environment. In a study of four hierarchical structures that may be used to construct a hypertext environment, the undergraduate students in her study who were working in an unstructured condition experienced limited success as they worked with the hypertext materials. Shapiro suggests that the limited information this group received coupled with their limited prior knowledge is likely to have influenced this outcome. These findings may indicate that students with lower levels of prior knowledge need a more structured technology environment in order to be successful. 77 Influence of motivation on use of hypertext scaffolding. In addition to the influence of prior knowledge, Last, et al. (2001) found that for undergraduate students in a psychology course, motivation was a key factor in their successful use of hypertext scaffolding. Students who were identified as having a high degree of motivation generally were more successful in their use of the hypertext scaffolds, especially those students with high prior knowledge and high motivation. Students in their study with low prior knowledge and low motivation were least successful and simply moved through the use of the scaffolding with little or no direction. An interesting finding in their study was the frustration indicated by students who were in the low prior knowledge/high motivation group. These students while motivated to achieve at a high level became very frustrated when their lack of prior knowledge prevented them from accomplishing this goal within the hypertext scaffolded environment. Like Last, et al. (2001), Neiderhauser and Shapiro (2003) also found that motivation affects students? use of hypertext scaffolding. They found that students? reading patterns changed depending on the purpose for doing the reading, and that as the students? purpose became more engaging they exhibited increased success as they used hypertext-supported materials. Although they found a strong link between motivation and successful use of hypertext scaffolding, they also found that the cognitive requirements for students using hyperlinked text is much greater than the requirements for non-hyperlinked text. This may act as a disincentive for some students and would work against the motivation provided by having an engaging reason to take part in the instructional activity. Finally, they found that when students were expected to use hyperlinked text to search for low-level, knowledge-based information, the increased 78 cognitive requirements needed to accomplish the task was counter-productive and created significant problems for the students. In a study of college students, Hynd, et al. (2000) found that when students working in a hypermedia environment were motivated to complete a task that required the use of expert skills and behaviors, these students exhibited these behaviors. However, additional consideration of the data indicated that these students were exhibiting these behaviors and skills at levels well below that of experts. This would seem to indicate that while motivation can encourage students to exhibit specific behaviors and hypermedia environments may be able to assist students in accomplishing the task, the teacher must ensure that the students are developing and using expert levels of expert behaviors. Hynd, et al. suggest that this may be done through additional use of technology resources. Influence of text structure. The structure of the text and the hypertext scaffolding also influences the ability of students to use the material effectively. Neiderhauser and Shaprio (2003) found that when text has a clear structure that allows students to understand the importance of various aspects of the material students were better able to organize and use the information. The positive effect of this greater structure was strongest in students that were considered lower-performing which seems to indicate that these students need the increased structure to do their best when using hypertext supported sources. This seems to indicate that students with less prior knowledge benefit from a hypertext environment that has built in structure as opposed to structure the students must create as they use scaffolds and read the material. This would reinforce the finding by Antonietti, et al. (2001) and Last, et al. (2001) that students with less prior 79 knowledge experienced more difficulties using the hypertext and had more difficulties understanding the structure of the text they were studying. In a study of high school students using a multimedia supported, ill-structured instructional unit, Brush and Saye (2001) found that some students benefited from the inclusion of a summarizing document within which the hyperlinked multimedia scaffolds were set. This summarizing document, in the form of an essay with embedded hyperlinked scaffolds, added structure to the material. Students using them tended to make greater use of the embedded scaffolds, used the scaffolds in a non-linear manner that provided more meaningful contextual information, and were more satisfied with their learning experience and the level of learning it provided. The researchers found however, that scaffolding guides provided for the students were seldom utilized at any point during the lesson. They hypothesize that despite the additional structure these guides would have provided for the students as they were working in the hyperlinked environment, students did not utilize these guides due to a lack of sufficient class time to complete them. Influence of cognitive requirements. When working in this type of environment, learners must be able to assess their own metacognitive strengths and weaknesses, recognize the metacognitive requirements of a task, determine the specific metacognitive strategy for the situation, and plan for using those strategies in order to accomplish a specific goal (Brush & Saye, 2001; Hannafin, Hill, & Land, 1997; McLoughlin & Hollingworth, 2002; Saye & Brush, 1999; Shapiro, 1999). Lin (1994) and McLoughlin and Hollingworth (2002) found that cognitive overload became an issue when learners were trying to incorporate new knowledge and skills as well as metacognitive strategies 80 into their use of a learning environment, especially a hypermedia or technology rich environment. They found that when students understand the metacognitive process and the context within which a problem is set, must use advanced metacognitive knowledge and skills to solve a problem, and are required to explain their metacognitive processes while they are working on the problem, they are more likely to outperform students that do not perform these tasks. Beaufils (2000) found that the inclusion of hyperlinks and other multimedia tools presented specific ?ergonomic problems.? As students manipulated the original text, the hypermedia scaffolds, and the note-taking tools that were part of the study some students experienced a significant level of cognitive overload. Additionally, many students had trouble simply manipulating the multiple windows in which the information and scaffolds were found, especially after the note-taking tool was added. This study suggests that any utilization of hypermedia scaffolding must consider that although the scaffolding may provide needed assistance, it also adds to the difficulties of the tasks students are asked to complete. Conclusion Although the use of complex, text-based source documents is a necessary aspect of authentic instruction such as problem-based historical inquiry, the difficulties presented by the use of these types of documents in a class that contains students with varying literacy levels makes their regular use by a majority of teachers unlikely. Without the regular use of these documents, teachers will be unable to embrace the types of authentic instruction that many social studies education professionals believe to be necessary if history classes are going to provide students with the knowledge and skills 81 required for full participation as citizens of a pluralistic, democratic society. Scaffolding may provide students with the structure needed to engage in the study of complex source documents, but this scaffolding works best when its use is tailored to the needs of the individual student, an expectation that is unrealistic for most classroom teachers. Through the use of technology, in the form of hypermedia, scaffolding may provide the individualized assistance students need in order to successfully engage in the study of numerous source documents, but the use of hyperlinked scaffolds raises several questions such as: How do students interact with source documents presented in an on-line format? How do students use scaffolds in analyzing source documents? How do students use scaffolds to develop problem solutions? Are there similarities and differences for students with different literacy levels? If educators are to provide hard scaffolding in advance, how closely do educator?s expectations of students? needs match students? actual needs? 82 CHAPTER 3: METHODOLOGY Overview of the study Introduction This study was conducted in two seventh grade civics classes using an instructional unit and lesson chosen by the teacher. Throughout the study, plans occasionally changed and aspects of the study had to be renegotiated to accommodate the realities of a public school classroom. As a result, the methodology for my study evolved to accommodate these realities just as any teacher?s lessons change to accommodate some unexpected events and to take advantage of others. Some of the adjustments resulted in limitations to the study and these are noted. Other adjustments resulted in changes in the expectations of the study that were not limitations, but simply differences in the outcomes of the study that reflect the realities of working within a school setting. My study was set within a social studies classroom and examines the issue of students working with complex text-based source documents as part of a problem-based historical inquiry lesson. Most social studies instructors deal with the issues of students? literacy skills, and this is especially true for teachers using problem-based historical inquiry because of the many source documents students must utilize during this type activity. In order to investigate students engaging in the study of text-based source documents, a significant amount of the research must also involve issues of literacy and reading. Throughout this chapter, I refer to student literacy levels. This term indicates 83 the approximate skill level a student has in reading. Most often this is expressed in grade level increments that more or less correspond to the skill level that can be expected from a U.S. student at a specific grade level. Because of privacy concerns, I did not examine the standardized test scores for the students in my study, but instead asked the teacher to use his knowledge and expertise to identify the students? reading skill levels by indicating if each student was reading above grade level, at grade level, or below grade level. In addition to reading level, reference is regularly made to the readability of texts. Readability is used to describe the complexity of a text. Traditionally readability is calculated using a measure of the difficulty of words and/or phrases as well as the length and/or complexity of the sentences within a passage of text. The readability is then described using some indicator, most often one that gives the grade level at which an individual should be able to independently read and comprehend the text. In this study I expand the use of readability to include issues that are specific to the social studies domain, such as the need for historical or background information and a need to understand archaic or unfamiliar language usage. These issues, however, do not have a standard measure, and when readability is used in the study to include these domain specific issues I describe how I used them and why I made those choices. Project design This project was conducted as a design experiment in which an intervention is planned and carried out in ?average classrooms operated by and for average students and teachers, supported by realistic technological and personal support? (Bogdan & Bilken, 2003; Brown, 1992). The purpose of this type of research is to understand as many aspects of an intervention within the complex environment of a classroom as possible 84 (Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003; Shavelson, Phillips, Towne, & Feuer, 2003) and to overcome a ?credibility gap? resulting from the perception that educational research is far removed from the realities of classroom life (The Design- Based Research Collective, 2003). This type of environment is multiply-confounded, meaning that a change in one aspect of the classroom is likely to cause unforeseen changes in aspects of the classroom that the researcher did not anticipate (Brown, 1992; Cobb, et al, 2003; James, et al., 2003; Shavelson, et al, 2003). The design experiment model is a process by which the researcher seeks to understand as much as possible about a specific educational situation. The goals of the research are guided by the researcher?s agenda, but also by the context of the learning environment including the teacher?s goals, the students? needs, and the realities of the classroom setting. In order to accomplish this type of in-depth, wide-ranging investigation, a design experiment study is an iterative process that can involve any or all of the following: collaborative planning during one or more study phases, implementation that is often followed by refinement and additional implementation phases, and in-depth analysis of formative and summative findings in order to better understand the educational situation within its real-world context (Brown, 1992; Marshall & Rossman, 1999; Shavelson, et al, 2003). As part of the design experiment, data is collected from a variety of sources in order to identify and consider the greatest number of possible effects brought about as a result of the experiment being conducted. The assumption in this type of research design is that while theory should guide the hypotheses on which an intervention is based, the application of those interventions and the analysis of the results of the interventions must 85 be based on the realities of the educational setting. The design experiment model requires the researcher to work closely with the teacher to design and implement the study in a way that allows for the greatest level of realistic classroom participation by the students, the teacher, and the researcher. The combination of theory-based interventions and classroom experience is likely to produce findings that inform both the development and refinement of theory as well as practice related to an educational setting (Brown, 1992; Cobb, et al, 2003; Creswell, 1997; Design-Based Research Collective, 2003; Shavelson, et al, 2003). As I worked with the participating teacher this emphasis on designing a study that adequately captured the effects of a specific treatment within the confines of the real classroom setting with real classroom requirements and limitations proved to be a challenge. Some aspects of the study were altered to fit the realities of the school, the classroom, and the students. Other aspects of the study retained their original form but yielded unexpected data and results. Overall, however, the use of the design experiment model allowed me to conduct this study in an authentic setting and many of my findings are a result of this authenticity (James, et al., 2003). Development of research questions I began this research project with several hypotheses that guided the development of the questions I chose to concentrate on as I designed, implemented, and analyzed the research (Marshall & Rossman, 1999). Through my research and experience in the use of problem-based historical inquiry (PBHI) models in social studies classrooms, I knew that teachers? concerns about students? literacy skill levels often hindered their willingness to utilize complex, text-based source documents in their classrooms. My main research 86 question arose from this concern and from my hypothesis that there are ways to assist students in this task. Research Question: How may students be supported in working with complex, text-based source documents as part of a problem-based historical inquiry lesson? This question, however, needed to be narrowed in order to provide additional focus for the purposes of my study. The tendency of educators in the last several years to look to technology to solve common educational problems led me to consider how technology might be used to provide the scaffolding students might need in order to engage in the study of text-based historical source documents, how this scaffolding might be developed, what this scaffolding might look like, how various groups of students within a classroom might use this scaffolding, and how benefits from this scaffolding might be different for different student groups. As with the main research question, specific hypotheses guided the development of the research sub-questions that narrowed the focus of this study. First, I hypothesized that students interact with online source documents in ways unique to the online format and that understanding this interaction would help me better understand the benefits various approaches to the use of online scaffolding can have for different student groups (Burke, 2002; Goldman & Rakestraw, 2000; Larkin, 2002; McLoughlin, 1999). My second hypothesis was that educators may be able to develop scaffolds that anticipate students? needs as they read online documents, and that in an online setting these scaffolds can provide on-time assistance for students in the analysis of source documents and in the application of the knowledge they gain from this analysis to develop problem 87 solutions (Burke, 2002; Larkin, 2002; Perkins-Gough, 2002; Saye & Brush, 1999, 2002; Taylor, 1999). My final hypothesis was that students' interaction with source documents presented in an online format and their use of hyperlinked scaffolds would differ depending on their literacy skill level (Afflerbach & VanSledright, 2001; Grady, 2002; Hirsch, 2003; Hoffer & Gamoran, 1993; Jenkins, et al., 2003; Larkin, 2002; Saye & Brush, 1999; VanSledright, 2002a). These hypotheses led to the development of the five sub-questions that refined my research. Sub-Question 1: How do students interact with source documents presented in an on-line format? Sub-Question 2: How do students use hyperlinked scaffolds in analyzing source documents? Sub-Question 3: How do students use scaffolds to develop problem solutions? Sub-Question 4: Are there similarities and differences in the use of online documents and hyperlinked scaffolds for students with different literacy levels? Sub-Question 5: How closely do educators? expectations of students? needs match students? actual needs? Study timeline The research question and the five sub-questions guided the project during each phase and provided focus for the planning, implementation, and analysis of the research. My goal was to investigate one intervention method for assisting students as they used complex, text-based source documents and to determine if this method was useful for all students in an average classroom setting. To do this I had to consider multiple factors 88 that were influenced by the use of this method during the planning, implementation, and analysis phases of the project in order to identify ways this intervention might inform classroom practice as well as further research. Table 1 provides a timeline of the phases of the research project. Project setting and description of participants The study was conducted in two seventh grade civics classes in a suburban middle school in the southeast United States. The school has approximately 1065 students in grades six through eight, and approximately fifty-seven percent of these students receive free or reduced lunch. The two classes that participated in the primary study had forty- eight total members. Of these, twelve students did not return permission forms, and their data was not included in this analysis. All students, however, participated in all phases of the lessons without regard for their participation in the research study. Teacher. The teacher who participated in the study had eight years of middle school teaching experience in reading and social studies. He was certified in both subjects and at the time of this study, he had recently completed his Ph.D. in reading education. This teacher?s familiarity with both reading and social studies education and his interest in the project made his participation preferable to that of some other teachers that might have participated. Students. The teacher was asked to identify each participating student as either an above average reader, indicating literacy skills above the norm for their grade level, an average reader, indicating the student was reading at or near grade level, or a below average reader, indicating the student was reading below grade level. Of the thirty-six students who participated in the study, seventeen were identified as above average 89 Table 1 Research project timeline Pilot Study March ? July 2004 Pilot Study used to refine coding process and develop procedure to test readability of hyperlinked documents Step 1: Develop instructional unit/lesson and identify source documents Step 2: Excerpt documents for classroom use and test readability of excerpts. Phase I: Design Refinement August 2004 - January 2005 Step 3: Refine excerpts Phase II: Coding February 2005 Student, teacher, and researcher code documents Step 1: Refine document excerpts based on coding Step 2: Add hyperlinks to documents to support student understanding Step 3: Test readability and domain specific complexity of document excerpts after hyperlinks were added Step 4: Refine hyperlinked documents based on analysis Phase III: Document Analysis & Preparation February - April 2005 Step 5: Prepare technology-based instructional materials Phase IV: Study Lesson May 2005 Implementation of instructional lesson using hyperlinked sources Phase V: Interviews May 2005 Student and teacher interviews are completed readers, fifteen were identified as average readers, and four were identified as below average readers. Because of his training, experience with these students, and knowledge of their performance on assessments, I felt confident in the teacher?s ability to make the determinations of students? reading skills. 90 One source of concern regarding the participation of students was the disproportionately low number of below average readers that returned permission forms and therefore participated in the study by having data collected. While ninety-five percent of above average readers and seventy-five percent of average readers returned permission forms, only fifty percent of below average readers returned their permission forms. In order to understand the effect these different participation levels might have on the findings of the study, the teacher and I examined student information to better understand the students who would be part of the data set and those who would not. The teacher determined that with one exception the students in both the average and below average reader categories who did not participate were in the lower socioeconomic level. Additional analysis revealed that most of the students who did not participate were male and eighty percent were African-American. The majority of the students who did participate were in the middle and upper socioeconomic level with average to above average reading skills, and a disproportionate number of them were white and female. Although no definitive conclusions can be drawn from this information regarding why these students did not participate, the exclusion of data from specific groups of students means that the study?s findings are less generalizable, especially regarding students in those groups with lower participation rates. Therefore, while this study?s findings may reveal tendencies among below average, average and above average readers, these findings are more likely to indicate areas for further study than generalizable findings. 91 Study Phases Overview of study phases Introduction The study consisted of five phases beginning with my initial work with the teacher and culminating with the student and teacher interviews that took place after the study lesson was complete. Each phase of the study influenced the following phase, as is often the case with design experiment studies. In Phase I the teacher and I identified the study lesson, selected and excerpted the source documents, and tested the excerpts to help us understand the readability and domain specific complexity of each. In Phase II the students, the teacher, and I coded the excerpted documents in order to identify areas of confusion for students. During Phase III the coded documents were analyzed and, based on the results of this analysis, the document excerpts were adjusted through additional excerpting and the addition of hyperlinked scaffolds. At the end of this phase the finalized source documents were analyzed again to determine the effect the hyperlinks had on the readability and domain specific complexity of the texts. The study lesson using the hyperlinked source documents was taught in Phase IV and interviews with students and the teacher took place after the study lesson was complete. (Table 2 shows study phases and data sources from each phase). Phase I: Development of Design Intervention Initial planning. During the summer before the study was to be completed I identified the teacher I would work with and through the fall of that school year he and I developed the instructional unit and study lesson, including selecting the source documents that would be used in the study. Although the teacher had used inquiry 92 Table 2 Data sources Project Phase Phase Purpose Data Sources Data Gathered Phase I: Design Refinement ? Unit/lesson planning and document selection ? Lesson planning materials ? Excerpted source documents ? Indicate teacher?s beliefs regarding student abilities and use of source materials ? Readability scores for excerpted documents Phase II: Coding ? Coding of source documents by students, teacher and researcher ? Coded excerpted source documents ? Researcher observation data ? Indicate areas of confusion for students ? Indicate areas the teacher and researcher believed would result in confusion Phase III: Document Analysis & Preparation ? Analysis and preparation of instructional materials ? Create individual CDs of hyperlinked source materials for each student ? Excerpted source documents ? Excerpted hyperlinked documents ? Readability analysis of hyperlinked source documents ? Analysis of domain specific complexity of hyperlinked source documents Phase IV: Study Lesson ? Students use hyperlinked documents as part of the study lesson ? Individual CDs containing the hyperlinked documents ? Reading guides used by students ? Group problem- solving scaffolds ? Researcher observation data ? Track hyperlinks students used while reading hyperlinked documents ? Indicate student understanding while reading the documents and achievement in the problem-solving task ? Indicate the ways students used the information from the documents during the rest of the lesson Phase V: Interviews ? Conduct interviews with individual students ? Conduct interview with teacher ? Student interviews ? Teacher interview ? Better understanding of student work and attitudes about using the hyperlinked documents ? Better understanding of teacher?s perceptions of students? use of hyperlinked documents 93 learning in the past and was familiar with problem-based historical inquiry (PBHI), he did not use either as a regular part of his classroom instruction. He was not, however, opposed to having the project lesson follow PBHI guidelines, and he indicated that he believed this would make the lesson more interesting for the students. During these discussions he also indicated that he intended to use the PBHI instructional method at times during the school year. After these initial discussions I developed the lesson and the teacher then provided feedback based on his knowledge of the students? previous lessons and the knowledge he believed they needed when they finished the lesson. Selection of the study lesson. In order to select an instructional unit that the study lesson could be part of, the teacher identified several units that he normally taught using source documents. Based on the amount of time allotted to the teaching of each and when they would be taught during the school year, we decided to use a unit on the history of the modern legal code. This unit was taught near the end of the school year and prepared the students to study the development of the United States legal code, although at that point they would have already studied the United States Constitution and some basic information about the U. S. legal code. We decided that I would plan the study lesson, which would be an introduction for the unit, and the teacher would plan the rest of the unit. Both the lesson and the unit were intended to use the PBHI instructional model. Early in the fall I developed the study lesson, an analysis of several source documents. Several hypotheses guided the lesson?s development. First was the hypothesis that students are more likely to read complex source materials when they are given a relevant purpose for understanding the information (Burke, 2002; Levstik & Barton, 2005; Perkins-Gough, 2002; VanSledright, 2004, 2002a, 2002b). This led to the 94 development of the lesson?s culminating activity that posed several legal dilemmas that the students were to address using the information from one or more of the legal codes they had read. A hypothesis that students would be more likely to successfully engage in the lesson if they were supported throughout the process, led to the development of the scaffolds students used during each phase of the lesson, including the hyperlinked scaffolds that were one of the primary focuses of the study (Beck & McKeown, 1991; Burke, 2002; Chang, 2001). In the lesson students were to read excerpts from three legal texts. These would be presented in an online format and would include hyperlinked scaffolds that provided the students with definitional information, background information, and metacognitive assistance. After reading the source documents, students would work in groups to develop solutions for three legal dilemmas using the information from the documents they had read and from previous lessons about types of legal systems in the United States. In the final part of the lesson, student groups would present their solutions to the dilemmas and defend the decisions they had made (See Appendix M for the lesson plan). Selection of documents to be used in the study lesson. Although the teacher indicated that he had used source documents as part of this lesson in the past, he did not believe that the lesson was generally effective or that the students gained an adequate understanding of the importance of these historical documents. One problem with the source documents the teacher had used in the past was that they were expected to add interest to the lesson but beyond their ability to interest the students, there was no real purpose behind their inclusion. He believed the difficulty level of the excerpts as well as the lack of purpose for their use made it difficult to motivate most of the students to 95 attempt to utilize them in any meaningful way. When teaching this lesson in the past the teacher had included short excerpts from these six historical texts: The Code of Hammurabi, the Ten Commandments, the Torah, The Justinian Code, The Napoleonic Code, and the Magna Carta. In addition to the lack of purpose in these document excerpts, the teacher expressed a significant level of concern about the students? ability to understand the documents even though the excerpts were very short. He indicated that when the class was reading the excerpts in the lesson most of the below average readers, as well as many of the average and above average readers simply sat and looked at the pages until he ?got ready to tell them what they meant and then moved on.? As a result of these discussions, the teacher and I decided to have the students do an in-depth analysis of three source documents, The Code of Hammurabi, The Justinian Code, and the Magna Carta. We chose these three because we felt that they would allow the students to see codes of law from three very different regions and time periods. Analysis of sources for domain specific complexity. Both readability and domain specific complexity were concerns as we began to consider how to excerpt the three documents. However, because readability would change with each alteration, domain specific complexity was the key consideration during this phase of the project and readability was analyzed after the documents had been excerpted for the first time. I use the term domain specific complexity to indicate those issues that add to the complexity of historical source documents, such as the need for historical background information and the complexity of archaic or unfamiliar language usage. Unfortunately, there is no standard measure for computing domain specific complexity because this is dependent on a number of factors that cannot be standardized. Students in different 96 regions of the country are introduced to specific historical information at varying points in their educational careers and so, any attempt at establishing a standard score for the complexity of a document based on these domain specific issues would be incredibly difficult and open for constant revision by each teacher in each classroom that attempted to use them. For this study, I used the following criteria as I considered the domain specific complexity of the documents (Afflerbach & VanSledright, 2001; Bean & Ericson, 1989; Beck & McKeown, 1991, 2002; Britt, et al., 2000; Fry, 2002; Goldman & Rakestraw, 2000; Gunning, 2003; Hicks, et al., 2004; Levstik & Barton, 2005; Mayer, 2003, Rosenblatt, 1969, 1985; Stahl, 2003; Tovani, 2000; Unsworth, 1999; Voss & Riley, 2000): 1. The need for background information that would allow a reader to understand the document within its historical context. (See Figure 1 for an example of domain specific content information a student would need in order to understand a passage in The Code of Hammurabi, one of the documents used in the study lesson) 2. The difficulty posed by unfamiliar use of language 3. The motivational factor of the document based on the presentation of the text and the uses to which the text would be put in the lesson. Research suggests that when students have an engaging purpose for reading source material and the presentation of the text complements that purpose, students are better able to comprehend the material in the text. Figure 1. Concept Map Figure 1. Briefly illustrates the concepts students need to comprehend in order to understand the meaning of this phrase from The Code of Hammurabi: ?When Marduk sent me to rule over men, to give the protection of right to the land?? Example: The excerpts from The Code of Hammurabi include numerous references to the issue of theft. Students read excerpts from this text that were grouped so that they could more easily comprehend the Babylonian concept of theft and punishment. In order to set a purpose for students? use of the text, they were instructed that when they finished reading the documents they would use the information to solve problems using this legal code. Both the way in which the texts were presented and the uses to which the information was to be put should affect the domain specific complexity of the texts for the students reading them. ??give the protection of the right?? Sovereign?s right to change laws Few people knew the laws Lack of written laws Belief in the infallibility of the ruler Marduk, King of the gods Responsibility of the ruler to the people Mesopotamian Religious beliefs Divine Right Polytheism 97 98 In addition to the consideration of domain specific complexity, the teacher and I established three guidelines for the excerpting process: 1. We would try to keep each no longer than one and a half pages in length (Strong, et al., 2002; Tovani, 2000) 2. The excerpts would reflect those aspects of each code that we believed had been the most influential on modern codes of law 3. Each excerpt would contain some elements we believed would be of interest to seventh grade students. Excerpting the source documents. Because the texts were lengthy legal codes, I began the process of excerpting them using the third criteria in domain specific complexity as a primary consideration. In order to identify those areas that would be the most motivational based on the purpose for the reading and the presentation of the text, the teacher and I considered our second excerpting guideline to be the most important because this would determine much about the rest of the lesson. In looking at the three codes we identified the areas we believed were the most influential (See Table 3). One element that we believed was important for students in all of the documents was each society?s concept of the right to rule and so this was included in each excerpt. Beyond this concept, we believed that while there were similarities between each of the codes, we wanted to emphasize different concepts within the different texts in order to allow the students to have broader knowledge of these texts than might be gained from the examination of a more limited list of topics. Using these concepts as the basis for excerpting each of the codes, I selected passages of text that were relevant and organized these so that, while the excerpts could 99 Table 3 Concepts within the legal codes emphasized in the study lesson Code of Hammurabi ? The concept of divine right to rule ? Issues involving proof of guilt when accused of a crime ? The concept of ?an eye for an eye? Justinian Code ? The concepts of natural, common, and civil law ? Roman concept of family law, paterfamilias Magna Carta ? The concept of the consent of the governed ? Rights and responsibilities of a judicial system still be located within the larger document if desired, the overall excerpted text would be usable for the students within the context of the lesson. These first excerpts were longer than the one and a half page limit the teacher and I set as one of our guidelines so he and I worked together to identify additional passages that could be eliminated or shortened and still allow the students to access the information needed to complete the lesson. At the end of this cycle of excerpting we were satisfied that the documents contained the information students would need, that they contained elements that the students would find interesting, and that they were a length appropriate for students at this grade level. The final length of the documents at this point was: The Code of Hammurabi, one and three-quarters pages (See Appendix A); Justinian Code, almost two pages (See Appendix B); Magna Carta, one and one-quarter pages (See Appendix N). Analysis of sources for readability. After the initial excerpts were completed at the end of Phase I, I analyzed the excerpted documents and found that they had readability levels ranging between ninth and twelfth grades. The Flesch-Kincaid Readability Test and the Flesch Reading Ease Measurement were used together because although they use the same criteria to measure the reading difficulty of a text they do not always provide identical scores. By using the two together I sought to get a clearer 100 picture of the basic readability of each document. In most instances use of the two formulas produced very similar results, however, in the few instances where the two formulas produced results that were not close, the discrepancy was noted in analysis of the data and the higher grade level readability was used to make any decisions about the document (Gunning, 2003; Zakaluk & Samuels, 1998). Both the Flesch-Kincaid Readability Test and the Flesch Reading Ease Measurement use the difficulty of the words and the complexity of the sentences in a passage of text to determine the level of difficulty involved in reading that text. The Flesch-Kincaid Readability Test indicates this difficulty in the form of a grade level score that approximates the level of education in the United States a student would need to have completed in order to be able to read the text with little or no difficulty. A readability grade score of 7.7 would indicate that a student in the seventh month of the seventh grade could read that document with little or no difficulty based on the complexity of the language used. The Flesch Reading Ease Measurement also measures the difficulty level of text, but the score for this measure is presented in the form of a percentage. These percentages are generally considered to be more useful than the readability scores and can be roughly estimated to equate to grade levels. In the Flesch Reading Ease Measurement scoring the lower the percentage score, the more difficult a text is, so a passage of text with a score of 0% would be at a college reading level. A score of between 90 and 100% on the reading ease measurement indicates a reading level at approximately the fifth grade. By using these two measurements together, I hoped to more reliably determine the readability levels of the texts used in each part of the study and the effects that adding hyperlinks to those texts had on basic readability. 101 At the end of this process the teacher and I were satisfied that the excerpted documents meet the criteria we had established and that readability and domain specific complexity had been addressed as much as possible through the excerpting process. Completing the lesson plan. Once the documents had been chosen and analyzed I completed the lesson plan. Because the lesson plan used the problem-based historical inquiry model, I identified four problems the students would consider at the end of the lesson. One problem posed a question about where each group believed their right to rule originated. The other three questions posed a problem that required the students to use information from one of the source documents to develop a solution (See Appendix P). After the teacher considered these problems he raised several concerns. First, he was concerned that the students would not have time to adequately consider all four questions thoroughly in the time we had allotted for the lesson. Second, he was concerned because he felt that at this point in the lesson, the final phase, the students would do well to consider these codes using some comparison to the U.S. legal code that they had already studied and that this lesson was preparing them to study further. His final concern centered on the problem that used the Magna Carta, specifically on his continuing concern about the students? abilities to read the document and understand it at a level that would allow them to use the information in a meaningful way. Ultimately the negotiations between the teacher and me resulted in the elimination of the individual question that used the Magna Carta, but we retained the document as one of the texts because the teacher believed it would benefit the students to read the document and use it in the question that compared all of the legal codes. By eliminating one of the original questions, the problem-solving aspect of the lesson was shortened, 102 alleviating the teacher?s concerns over time. Additionally, comparisons to the U. S. legal code were added to each of the questions. These negotiations with the teacher over the structure of the problems to be used in the study lesson resulted in the final three problems the students would consider in the lesson. We were satisfied that they required students to consider the information in meaningful ways that would allow them to better participate in the remainder of the unit. The teacher believed that his major concerns had been addressed and he was comfortable with the structure of the lesson at this point. (Appendix E shows the study lesson problems as the students used them) After finalizing the problems the students would consider in the study lesson, I completed the plan for the lesson and the scaffolds that students would use during their group work activities as they considered the problems and worked out solutions (See Appendix M for the study lesson plan and Appendix K for the group work decision making scaffold). After reviewing the materials the teacher believed they would accomplish the goals of the lesson listed below: 1. Students will develop an understanding of the role past codes of law have played in influencing the development of modern codes of law, including in the United States. 2. Students will identify the basis those in power used to legitimize their rule in four different societies, ancient Mesopotamia, ancient Roman society, thirteenth century England, and the modern United States of America. 3. Students will develop and defend positions regarding three problems dealing with the following issues involving ancient codes of law: 103 a. The concept of ?an eye for an eye? and the meaning and punishment of theft in The Code of Hammurabi and compared to the U.S. code of law. b. Family law in Justinian Rome compared to modern America. c. The methods used to change laws by the societies represented by these codes of law. Phase II: Coding of Documents At the conclusion of the initial phase of the project the design of the study lesson was complete and initial source document excerpts had been chosen and analyzed for readability and domain specific complexity. In order to identify those areas of the excerpted documents where scaffolding would be most effective, the students, the teacher, and I coded the documents using coding categories I had refined during the pilot study (See Appendix L). I hypothesized that most of the time experienced educators would be able to identify the areas of difficulty for students as they read complex source documents allowing them to place scaffolding in the areas where students were most likely to need assistance. The ability of educators to correctly identify these areas of confusion is an indication of their ability to support student understanding through the choice of excerpts, provide personal scaffolding, and create hyperlinked scaffolding when and where appropriate. Effective hyperlinked assistance must be available at the moment the student becomes confused, and if educators cannot identify these areas correctly a majority of the time it will be very difficult for them to provide this type of assistance in a meaningful way. Phase II was intended to identify how effectively the educators in the study, the teacher and I, to identified the areas the students in the study experienced 104 confusion as they read the study documents (Burke, 2002; Larkin, 2002; Perkins-Gough, 2002). Student coding was completed by having the students read and code each of the excerpted documents. This was done during reading class with the same teacher and student classes. This teacher taught both classes reading and social studies, so early in the second semester we had the students code the documents during reading class. Because the teacher and I wanted coding to be part of a meaningful class activity, he taught a lesson on ways students can recognize when they are experiencing comprehension problems and strategies to correct the problems. This lesson was based on the lesson I taught during the pilot study (See Appendix L for a description of the pilot study). Following the lesson, students were given three colored highlighters and instructions for coding the documents excerpts. During the instructions the students were reminded of the purposes of the study and they were also told that although they would not be discussing the documents at that time in order to clear up confusion, the classes would be using these documents as part of a social studies lesson later in the year and they would be able to better understand them at that time. We felt these instructions were necessary because we wanted to alleviate the anxiety some students would feel when they read the documents and did not have a chance to develop an understanding of the information as part of the lesson. Student coding was intended to identify areas where students experienced confusion while reading the source documents. Because the teacher taught the same groups of students for reading and social studies, the students were able to complete the coding of the documents as part of a reading lesson on recognizing areas of confusion 105 and understanding how to deal with different types of confusion while reading. The students were given the following list of three reasons for confusion: 1. I don?t know the meaning of this word or phrase 2. I think I need some historical information 3. I?m confused but I don?t know why. As they read each document excerpt, students were asked to highlight each area where they experienced confusion in the specific color assigned for the type of confusion they were experiencing. During this part of the lesson, I acted as a co-teacher and assisted students by answering questions related to coding and identifying the reasons for their confusion as they coded the texts. The students had been instructed that the teacher and I would not tell them which color to highlight areas of confusion and, therefore, they were not upset if the teacher or I instructed them to use their best judgment. Some of the questions the students asked dealt with how to determine where to highlight, such as when they believed they needed background information for an area they wanted to know if they needed to highlight the entire area or just a few words. Another frequent question regarded how to highlight if there was more than one type of concern regarding their confusion, such as if they needed to highlight for background information but they also recognized a need for definitional assistance in the same area. During this time the teacher and I would answer these types of questions and if a question arose that we had not discussed we would briefly decide on the best method and inform the students. We did that to ensure coding consistency among the students. 106 Prior to the students coding the documents, the teacher and I coded the documents to identify the areas we believed the students would need definitional or background information. Our coding also included areas where we believed the students would need metacognitive assistance in order to comprehend the texts within the context of the lesson. The student coded documents, as well as the documents coded by the teacher and me, were collected as data and used in the third phase of the study to help determine where hyperlinked scaffolds could be used to provide the most effective assistance for students. Phase III: Analysis and preparation of hyperlinked source documents Addition of hyperlinks and alterations of documents. During the third phase of the study, I analyzed the student, teacher, and researcher coding of the documents using descriptive quantitative analysis in which I identified the number of times students highlighted specific passages of text and compared this to the passages coded by the teacher and me. This analysis was used to determine where to add scaffolding for definitional and historical background based on areas of confusion indicated by the coding. Using the data gathered from the comparative analysis of the coded documents, I added hyperlinked scaffolds to the documents to address the definitional, historical background, and metacognitive needs of the students as they read the texts as part of the study lesson. Although I wanted to include hyperlinks to help students with any areas where they needed assistance, this was unrealistic if the documents were to retain the cohesiveness that would make them easier to read, something the teacher had worried about when using source materials from the textbook. 107 In order to provide scaffolding that would benefit the greatest number of students I established guidelines for the inclusion of a scaffold in a particular place based on the coding of that area by the students, teacher, and me. In order to establish these guidelines I considered the amount of time the teacher would need to work with an individual student in order to answer a question regarding a definitional or historical background issue. Based on my own experience and a discussion with the teacher, I decided to use thirty seconds as an average for answering a definitional question. I used one and a half minutes as an average for answering questions about historical background. Using these averages the teacher could be expected to spend approximately forty-eight minutes during each of the class periods answering student questions if each student in the class had one definitional question and one historical background question. Because part of the purpose of the hyperlinked scaffolding is to enable students to get on-time assistance when reading these documents and because these averages indicate that a significant portion of the teacher?s time will be spent answering predictable student questions if students do not have this assistance I decided on criteria for including a hyperlink in the documents that would allow the teacher to eliminate most of the student questions and allow him to concentrate on assisting those students that needed greater assistance. The areas coded by 20 percent or more of the students made up 75 percent of the coding in the documents, therefore, I estimated that the teacher would eliminate 75 percent of the individual questions from students as they read if scaffolds were used to address each of these problem areas. This would decrease the amount of time spent on answering questions from approximately forty-eight minutes during a class period to twelve minutes. 108 Once a determination was made regarding where to add definitional or historical background scaffolding or alter the document to best assist the students as they read the text, I identified four types of changes or scaffolds that I could use to alter the source documents: 1. Provide a definitional scaffold 2. Provide a historical background scaffold 3. Delete a portion of the text 4. Replace a portion of the text with a summary or explanation Using these I was able to identify the best ways to adjust the documents in order to assist students as they read the texts during the lesson (See Appendix Q for examples of each type of scaffolding or document alteration). In addition to definitional and historical background information, I added metacognitive scaffolding to the source documents. In order to provide students with this scaffolding in the areas the teacher and I identified during the coding phase, I compared his coding and mine. There were few differences in our coding for metacognitive assistance and after a brief discussion we were able to determine where these scaffolds would be the most useful. The purpose of the metacognitive scaffolding would be to help students think about the texts as they read them and help ensure that they gather the information they will need in order to consider the problems later in the lesson. These criteria were the basis for the coding and consequently guided the inclusion of metacognitive scaffolding hyperlinks (See Appendix Q for examples of metacognitive hyperlinks). 109 Once the decision was made regarding where hyperlinks would be placed in the documents and which additional text adjustments needed to be made, I created the documents as online documents and added definitional, historical background, and metacognitive scaffolding hyperlinks in those areas. Figure 2 is an example of the scaffolding that was included in a section of The Code of Hammurabi. This section of The Code of Hammurabi contains two hyperlinks with three scaffolds, a definitional scaffold, a historical background and context scaffold, and a metacognitive scaffold in the form of a question to help students connect this information with other relevant information in the text and the lesson. The word ?accusation? is a definitional scaffold that students could use if they were unsure of the meaning of the word or of how the word is being used. Students were instructed to use these links if they needed them. The globe icon provided students with historical background information that helped explain something about this section of the text. In this case the link is intended to place the Mesopotamian practice of using water to determine guilt or innocence within a larger historical context. This link also contains a question students would respond to on their individual data gathering sheets. This question asks students to consider another piece of information from the text that directly relates to this passage. Later they will use both pieces of information to complete one of the problems at the end of the lesson. A few areas of the excerpted text were altered or further excerpted in order to accommodate areas of confusion that could not be effectively addressed solely through the use of hyperlinked scaffolding. An example of this type of change was in the excerpts from the Magna Carta. Initially the first paragraph from the original text Figure 2 Example of source document with hyperlinks 2 If any one bring an accusation against a man, and the accused go to the river and leap into the river, if he sink in the river his accuser shall take possession of his house. But if the river prove that the accused is not guilty, and he escape unhurt, then he who had brought the accusation shall be put to death, while he who leaped into the river shall take possession of the house that had belonged to his accuser. described at length the background reasons for the nobles? decision to write the document. During coding most of the students became so confused while trying to read this paragraph that a majority gave up trying to identify areas of confusion within the paragraph and simply drew a circle around the entire paragraph using all three colors. Students I spoke with during the coding asked if they could code the paragraph with all three colors because they felt that there were elements of all types of confusion. After witnessing this confusion during the coding and during analysis of the coding finding that one hundred percent of the students had coded this paragraph as confusing, I decided to eliminate the paragraph and replace it with a hyperlink that provided a brief description Accusation To say someone did something that was illegal. To accuse them. Until modern times, many societies used water to decide if someone was guilty of a crime. According to this law, if a person was accused of a crime he could throw himself in the river. If he drowned, he was guilty and the person that accused him of the crime would get to take his house. If he does not drown, they believed he was proven innocent. ? What does it say will happen if the person accused of the crime is not guilty? 110 111 of the background information about the conditions that led to the creation of the Magna Carta. Analysis of source documents with hyperlinks. The addition of hyperlinks to a text-based source document potentially alters the difficulty level as a result of the additional text and might, therefore, increase or decrease the readability level of the text. Hyperlinks might also increase or decrease the domain specific complexity as a result of providing readers with information such as historical background, assistance in understanding unfamiliar and/or archaic language and word usage, and support utilizing the document within the context of the task. In order to ensure that adding hyperlinks did not increase the level of reading difficulty to a point that students would be unable to utilize the documents, I tested the readability of each document before and after the addition of hyperlinks using two measures of readability. I retested the readability of each of the source documents after the inclusion of hyperlinks using the Flesch-Kincaid Readability Test and the Flesch Reading Ease Measurement. In order to test the hyperlinked documents I used a method I developed during the pilot study that allowed me to test the reading level of passages of text with documents that contain both standard text and hyperlinks (See Appendix L). To do this each of the basic documents and the text from the hyperlinks associated with that document were analyzed together to determine the readability of the text as a whole. I felt it was important to understand how each of the basic documents and associated hyperlinks worked together as a cohesive unit and, therefore, believed this method would best provide that information. The domain specific complexity was reconsidered after the addition of hyperlinked scaffolding as well. 112 Addressing the teacher?s concerns. In our discussion about the links the teacher was pleased with the definitional links and the historical background links. He believed the students would find that these were similar to links they were familiar with and they would be comfortable with their use. He was concerned, however, about the students? ability to successfully use the metacognitive scaffolds. At this point these scaffolds were linked questions that the students were expected to consider as they read the documents, but he had two concerns regarding their use. His first concern was that if the students did not have any requirements for using the questions they would simply skip over them in an effort to finish reading the documents more quickly. To address this concern we decided to create a data-gathering scaffold the students would use to record their answers to the scaffolding questions. The students would then take this data-gathering scaffold into their group work sessions and use the information to assist with their group decision- making. The teacher?s second concern regarded the data-gathering scaffold for the students to use in connection with the metacognitive scaffolds. The data-gathering sheet was intended to allow the students to record important information and considerations about the sources and then have this information available during group deliberations. The way the scaffolds were included in the texts the students would have access to their responses on their data-gathering sheet, but they would not have access to the questions once they were finished reading the documents. The solution for this had to accommodate several concerns from both the teacher and me: the concern that students have some requirements that they could be held accountable for as they worked with the metacognitive scaffolds, the concern that students be able to access most of the metacognitive scaffold questions 113 during group deliberations in order to recall the context of their responses, and my concern that if metacognitive scaffolds were connected to a historical background scaffold students would become confused if they could read the question on the data- gathering sheet but then did not see the corresponding number on the main text of the source document. In order to accommodate all of these concerns I decided on a system for the metacognitive scaffolds and the data-gathering sheet that worked together using these elements: 1. If a metacognitive scaffold was connected to the main body of the source document text, a number was placed in the area the scaffold question referred to and the question was written on the data-gathering sheet. These scaffolds were not linked because the teacher and I did not believe there was value in having the students link to a question that was already written on their data-gathering sheet. 2. If a metacognitive scaffold was linked to a historical background information scaffold, the number was listed on the data-gathering sheet, but the question was left blank so that students knew to look for the question in the links as they read the historical information. When using these questions, students would be encouraged to write their answers completely enough to ensure that they understood their response when they were not looking at the questions. These solutions addressed the teacher?s concerns about the metacognitive scaffolds and of the data-gathering sheet. A final concern was that the students would think of the data-gathering sheet as a list of knowledge level or factual questions that they must complete to prove they had read the material and they would fail to use the questions to help them think about the material. 114 Preparing study lesson materials. When the hyperlinked documents were finalized, a compact disk that contained an introductory page (Appendix D) and all of the linked documents was created for each student. By having each student use a personal copy of the CD, I could track which links students had used when I analyzed the data from the final phase of the study. I also made this decision based on a concern that if students were dependent on an Internet connection to access the documents, the potential for confusion and problems would increase because of potentially unreliable Internet access at the school. Phase IV: Study lesson In the fourth phase of the study, students participated in the study lesson that included the use of the hyperlinked documents (See Appendix M for the lesson plan). During this phase of the project my role within the class can best be described as that of co-teacher. Both the participating teacher and I conducted aspects of the lesson, both were available to provide individual assistance for students and student groups, and both of us were equally likely to be asked questions regarding the lesson. Although I tried to perform this role in a way that would minimize the impact it had on the study?s findings, this is a difficult stance to maintain and, therefore, the threat remained that I may have influenced the results of the study by participating in the ways that I did. During this phase of the study I recorded observational data at the end of each class and regularly discussed my observations with the teacher in order to gain additional information and to check my understanding and memory. Although ongoing observation notes would have been preferable, my role as co-teacher did not allow this. I also found that the students were reluctant to speak when they thought I was taking notes, so while 115 some incidents may not have been recorded I believe it is possible my observations were more authentic. Student use of hyperlinked sources. In order to understand the students? use of the hyperlinked documents within the instructional lesson as completely as possible, several methods of data collection were used during this phase of the study. The compact disk that contained all of the lesson materials allowed me to identify the links each student had used as they read the documents. Students worked with laptop computers provided by the school and used the CDs to access an introductory page that described the types of links contained in the documents and gave them instructions. From this page the students linked to each of the hyperlinked documents. While reading, the students used a data- gathering scaffold for each document to record their responses to the metacognitive scaffold questions throughout the texts and to help them gather information for use in the problem solving part of the lesson (Appendices H-J). These data-gathering scaffolds were collected at the end of the lesson and were used to provide data about the students? understanding of the documents as they were reading them and using the hyperlinks. Student work in decision-making groups. After students finished reading the hyperlinked documents they worked in decision making groups to create solutions to several contemporary legal problems that required them to use the information they had learned while they read the hyperlinked documents. Having the students work on contemporary issues increased the likelihood that they would be motivated to do the hard work required to develop these problem solutions because they are more likely to see the issue as relevant to their lives. For example, student groups were given a scenario in which a man lost his dog and another man found the dog and returned it after a reward 116 had been offered. Using The Code of Hammurabi the groups had to decide if the actions of the man that found the dog amounted to theft and what would have been done to resolve this issue based on that code of law. After making that decision, the groups compared the actions that would have taken place under The Code of Hammurabi to the way this would be handled under an American code of law, and then they made a decision about which they thought was the most fair. At the end of group work, each group explained and defended their decisions for each of the three problems (See Appendix E for the group work problems). During this part of the lesson students did not have access to the laptops and the hyperlinked documents, but they could use the responses on the data-gathering scaffolds they had used while reading the documents on-line. Additionally, each group had access to paper versions of the three documents, although these did not include the hyperlinks that had been part of the online documents. While working in groups, students used decision-making scaffolds to guide their collaborative work (Appendix K). These scaffolds were collected as data at the end of the lesson to provide insight into the students? understanding and use of the information in the documents. The conclusion of the lesson was a presentation by each group describing and defending the decisions of their group about the problems they were presented with during the group work. As in the earlier parts of the study lesson, I recorded my observations at the end of each class and often discussed these with the teacher to check my facts and record his observations as well. 117 Phase V: Interviews Student interviews. The final aspect of data collection was in the form of interviews with the teacher and the students (Appendix F: Teacher Interview Guide; Appendix G: Student Interview Guide). I originally intended to conduct student interviews as think-alouds in which the student and I would sit together, and the student would read one or more of the documents and describe for me his or her thoughts as they were reading the documents and using or not using the hyperlinks. This, however, proved not to be a feasible method for conducting the student interviews in this setting. Time was the primary issue in completing the interviews using the think-aloud model. This type of interview requires a significant block of uninterrupted time, and although I had planned on spending an entire class period for each interview, the reality was that I had underestimated the level of cognitive load this type of activity would be for the students and therefore underestimated the amount of time that would be required. It quickly became apparent that I was either going to have to use significantly more time for the interviews or I would need to change the format. Unfortunately, it was impossible to have more time. The study lesson, which had originally been planned for six weeks before the end of the school year, had been moved several times by the teacher due to scheduling conflicts. Because of this rescheduling, the study lesson was begun three weeks before the end of the school year, so that the lesson took place during that week and the interviews took place during the week before school was dismissed for the summer. It was impractical to extend the interview process into the final week of the school year and so I made the decision to change the format of the student interviews from the think-aloud format that would have given me in-depth information about a 118 limited number of students to a general interview guide approach (Patton, 1987). Using this approach I established a set of open-ended questions to ask the students and then asked relevant follow-up questions based on their responses. During these interviews students had access to the hyperlinked documents so they could refer to these if they wanted to show me something or if they were unable to remember a detail and wanted to refresh their memory. I conducted interviews with twenty of the students in the classes. These students were chosen based on their reading ability, and I attempted to interview students with a wide range of literacy skill levels. Other considerations for choosing students to interview included: (1) availability since many of the students were on trips and other end of the year school business during this time or (2) some aspect of the lesson in which a particular student stood out in some way that either the teacher or I noticed, such as a student who chose to read paper documents while utilizing the online scaffolds. Teacher interview. The teacher interview was conducted using the general interview guide approach. Email was used for follow-up questions with the teacher and these continued through the analysis process as issues would arise where I believed his perspective would be valuable. Both the teacher and student interviews provided valuable data that helped me to understand how students utilized the hyperlinked source documents and how the use of these documents fit into the broader picture of a problem- based historical inquiry lesson. 119 Data analysis procedures Analysis to refine the design intervention The process of analysis I used can best be described as a recursive cycle of data collection and analysis (Brown, 1992; Patton, 1987). Throughout the study on-going analysis of data being collected affected the next phase of the study. In Phase I of the study, the teacher?s beliefs about his students? abilities and interests affected the choices that were made regarding the formation of the lesson, the outline of the unit that he would develop to accompany the study lesson, and the choices of the documents that would be used. Analysis of domain specific complexity influenced the initial excerpting decisions that were made during Phase I and readability analysis was used to refine these excerpts. These analyses were completed using the methods and measurements described earlier in this chapter. During the third phase of the study, the documents coded by the students, the teacher, and me during Phase II were analyzed using descriptive quantitative analysis. Student coding was analyzed to determine passages a majority of students identified as areas where they needed some type of assistance in order to understand and use the document. Student coding was also compared to teacher and researcher coding in order to identify those places in the three documents that hyperlinks could be placed most effectively. Additionally I analyzed this coding in order to address my fifth research sub- question, ?How closely do educators? expectations of students? needs match students? actual needs?? To complete this aspect of the analysis, I used descriptive quantitative analysis that compared the words and phrases the teacher and I identified as areas in which the students would need assistance with the words and phrases the students 120 identified in their coding. I also compared the reasons students believed they were confused in specific areas to the reasons the teacher and I believed they would be confused. Once the initial analysis of the student, teacher, and researcher coding was complete at the beginning of Phase III, hyperlinks were added to the documents based on the results of the analysis. After the inclusion of the hyperlinks another analysis of the readability was completed to ensure that the inclusion of hyperlinks did not significantly increase the complexity of the documents. The results of this analysis led to further revisions of the hyperlinked documents. After the final revisions were made to the hyperlinked source documents, a CD that contained the instructional materials was made for each student so that I could track the links each used as they read the documents during the study lesson. Final analysis for study questions Analysis on data from the study lesson was principally conducted using recursive thematic analysis (Creswell, 1997; Denzin & Lincoln, 2003; Lather, 1986; Patton, 1987), although at times descriptive quantitative analysis was used if this provided a clearer description of an aspect of the data set. When descriptive quantitative analysis was used as part of the final data analysis, thematic analysis was also done in order to understand how this data fit with other data in the study. Thematic analysis for data from the study began with the expectation that themes related to specific topics important to the study would be present. The themes I began the analysis with were: 121 a. Students? interaction with source documents presented in an on-line format b. Students? use of scaffolds in analyzing source documents c. Students? use of scaffolds to develop problem solutions d. Similarities and differences for students with different literacy levels e. Educator?s expectations of students? needs compared to students? actual needs The initial analysis was completed by reviewing data and identifying examples of the five themes listed above. Following this first analysis, I reviewed the data again to identify additional themes that might be present but that I had not anticipated initially. Once these themes had been identified, I reviewed the data again to find additional examples of the themes that were identified prior to the beginning of the analysis as well as those identified during the second review of the data. During this third review I also began to identify sub-themes that related to the themes already identified in the previous reviews. I conducted a final review of the data in order to identify additional examples of the themes and sub-themes that had been previously listed, as well as to identify any additional sub-themes that might have been overlooked in previous reviews. After all data was reviewed and themes and emergent sub-themes were identified, analysis was completed by identifying commonalities and differences among the data and identifying the conclusions that could be drawn from the analysis of the data. As I completed this part of the analysis and developed explanations for the conclusions I had drawn from the analysis of the data, I deliberately sought to identify alternative explanations for my findings. This search for alternative ways of explaining my conclusions allowed me to better understand the multiple layers of data that are a product of the use of the design experiment model. When considering a conclusion I had formed, 122 I intentionally considered data sources that did not seem to corroborate the conclusion I had drawn because understanding how this type of conflicting data works as part of the overall study is paramount to understanding the multiple layers of complexity that are inherent in any classroom setting (Creswell, 1997; Denzin & Lincoln, 2003; Lather, 1986; Patton, 1987). Although the presence of alternative explanations for my study?s findings meant that I had to question each conclusion made about the data, the multiple layers of data that were produced during the course of the entire study allowed me to compare my data from a variety of sources before reaching my conclusions. For example, when I considered the how students used the hyperlinked scaffolds that provided background information I used the following data to draw conclusions: teacher observations and interview, researcher observations, student interviews, student data gathering scaffolds used during reading, student group scaffolds used during group deliberations, and group presentations. The triangulation of the data from these sources increased confidence in the credibility of my findings allowing me to draw reasonable conclusions from the multiple data sources available while acknowledging those instances when the data suggests that an alternative explanation may be plausible (Bogdan & Bilken, 2003; Denzin & Lincoln, 2003; Lather, 1986). Study limitations The decision to conduct this project as a design experiment resulted in both advantages and disadvantages to the overall study. The disadvantages that resulted from the realities of working in a school setting as opposed to a lab setting and from the necessity of negotiating specific areas of the study with the teacher in order to fit the 123 study to his instructional needs limited my control over aspects of the study in ways that I had to consider as I analyzed data and reached conclusions about my findings. I believe, however, that most of these disadvantages are offset by the advantages afforded to the study by the authenticity of conducting my research in a real classroom setting (Brown, 1992; Design-Based Research Collective, 2003; Lather, 1986; Marshal & Rossman, 1999). The structure of the classes I worked with resulted in both advantages and limitations to the study. Although my original intention was to have one class participate in the study and the other class act as a control group, discussions with the teacher regarding the make-up of the classes indicated that this would not be feasable. One class, called the ?advanced class? by the school, was made up of students whose reading abilities were medium to high. The other class, called the ?grade level? class by the school, was made up of students whose reading abilities were low to medium. Because of this division of students between the two classes it was unrealistic to establish either as a control group for the other since there was a clear difference in their ability and skill levels. This altered the structure of the study so that instead of studying two intact class groups with one being the control group and one participating in the study, both classes participated equally in the study, and my analysis of data compared reading level groups made up of students from both classes in the three categories of reading ability (Denzin & Lincoln, 2003; Lather, 1986). The two classes that were part of the study were taught both reading and social studies by the participating teacher. This structure allowed the first phase of the study to be conducted with the students in the reading class and the second phase to be conducted 124 later in the school year as part of the social studies classes. I hoped that for many of the students the three months between reading the documents in Phase I of the study and using them in Phase III was sufficient to allow them to largely forget the material. Additionally, the teacher and I made the decision not to discuss the documents with the students when they read them the first time in the reading class. By not discussing the documents at this time we hoped the possibility of the students? later use being affected would be minimized. To eliminate students? apprehension over reading something and not discussing it, they were told by the teacher that later in the year they would be using these same documents in another part of my study and they would have a chance to learn about them and understand them better at that time. Despite these precautions however, this must be considered a limitation since it is not possible for me to ensure that the students? original interaction with the texts did not affect their reading of the on-line texts in the final phase of the study. Another limitation of the study was the low number of students identified as lower level readers that returned permission forms and could, therefore, have data collected as part of the study. While most of the students identified as higher level and middle level readers returned permission forms, a more limited number of lower level readers returned forms despite the teachers? regular encouragement to do so. Although I believe the study?s comparisons of students with different reading ability levels offer valuable insight into issues teachers and curriculum designers should consider, I also believe that the limited number of lower ability level students, lower socioeconomic level students, and racial minority students that participated in the study means that the 125 findings cannot be generalized beyond the classes or even within the classes with any degree of confidence. The format of the student interviews was also a limitation for the study, although I believe this change altered the nature of the findings more than the quality (Bogdan & Bilken, 2003; Lather, 1986; Patton, 1987). My original intent, to conduct the interviews as think-alouds, would have given me more in-depth knowledge about a limited number of students? thought processes as they read each document and made the decision to use or not use each hyperlink. Although this format proved to be impossible due to unforeseen time constraints and student difficulties understanding the task, I was able to use the more traditional interview format to gather data from a large number of the students that had participated in the study. While I feel that the change in the interview format is a limitation for the study?s findings, I believe that by changing to the more traditional format I was able to collect more and better data than I would have collected if I had insisted on continuing with the think-aloud format as originally intended. During the coding and study lesson phases of the study I acted as a co-teacher and in this role I provided assistance to the students in the same ways the teacher did such as answering students? questions and clarifying instructions. Although this role may have had some benefits it also created limitations as I moved from the role of observer to participant observer (Denzin & Lincoln, 2003; Wolcott, 1995). In this role, I spent most of the class time assisting students and so I was unable to keep an in-depth observation log throughout each lesson. I attempted to minimize any negative impact of not having this type of observation log by writing extensive notes at the end of each class period, 126 discussing the class with the teacher and comparing what each of us observed from the lesson, and attempting to match the information in my notes to information from other sources such as student work and interviews. Although the reality of the design experiment model requires the researcher to be involved in the activities going on in the classroom making objectivity very difficult, my role as co-teacher increased my involvement with the participants and, as a result, increased the likelihood that this would impact my findings. I believe my awareness of this as a potential impact on my study and the comparison of data that I describe previously helped address this concern, although it should continue to be noted and considered (Bogdan & Bilken, 2003; Denzin & Lincoln, 2003; Lather, 1986; Marshall & Rossman, 1999; Wolcott, 1995). Conclusion Use of the design experiment model allowed me to conduct my study in a way that produced results firmly based in the realities of a regular classroom setting. While this was a positive factor in many ways, it also resulted in limitations as the realities of the classroom environment forced unforeseen changes on the study. The data collected and the analysis of that data reflect these changes. The analysis of my study data can be divided into two parts, that used to refine the design intervention and that used to address the study questions. Analysis used to refine the design intervention, while based on theory and research, tended to be pragmatic in that it was done to address a specific issue that could then be acted upon immediately to alter the next phase of the study. Aspects of this ongoing analysis were also included in the summative analysis used to address the study questions. In these cases data should be considered both formative and 127 summative. Analysis of data to address the study questions is used to help draw conclusions about the data where possible and identify the need for further study where necessary. 128 CHAPTER 4: FINDINGS Analysis of the data for my study served two broad purposes. Data collected in each of the first four phases of the study was analyzed and used to inform decisions about the following phase. This recursive pattern of data collection and analysis followed by informed decision-making in the next stage of the study is common in design experiments and allowed me to refine aspects of my study as needed in order to better understand the needs of the students and the teacher as they participated in the study. Following the conclusion of the study lesson and interviews, analysis of the data was completed in order to address the study sub-questions and thereby, address the study?s primary research question. Preliminary analysis to refine design intervention Phase I: Study lesson and document selection During the first phase of my study, I worked closely with the cooperating teacher to identify a unit, plan the study lesson, choose the source documents, and excerpt those documents. In this phase of the study data was collected from the following sources: lesson planning materials and the completed lesson plan, the source documents and the resulting excerpted documents, analysis of the documents for readability and domain specific complexity, teacher interview, and researcher observations. During this phase the teacher?s beliefs about his students? literacy skill levels played a substantial role in 129 decisions the teacher and I made regarding the lesson we chose and the materials we selected to use. This phase of the study produced data that was predominately used to inform the next phases of the study. Chapter three contains a complete description of the processes involved in planning the lesson, selecting the documents, and completing the initial excerpting process. Following the selection of the final excerpts I analyzed the excerpted documents to determine the readability and domain specific complexity of each document. The results of that analysis informed the next steps of the study and provided comparative data for the analysis that was done after the inclusion of the hyperlinked scaffolding Initial analysis of source document excerpts: Readability. Once the teacher and I were satisfied with the source document excerpts, I analyzed each to determine its readability. The purpose of this readability analysis was to establish the baseline readability score that would help guide the inclusion of hyperlinked scaffolding into the online documents (See Table 4 for readability statistics for each document). The Flesch- Kincaid Readability Test and the Flesch Reading Ease Measurement use the complexity of the words and sentences to determine the difficulty level of a passage of text. The Flesch-Kincaid Readability Score is expressed in grade level terms that approximate the level of education a student in an American school would need in order to comprehend the text. The Flesch Reading Ease Measurement is expressed in a percentage that can approximate a grade level similar to the Flesch-Kincaid Readability Score (Zakaluk & Samuels, 1998). I believe using these two measures together provided a more accurate assessment of the difficulty of the three texts. 130 Table 4 Readability statistics for excerpted documents: Phase I The Code of Hammurabi Number of words: 730 Flesch-Kincaid Readability Score: 11.2 grade level Flesch Reading Ease Measurement: 64.9% or approximately 8 th grade Justinian Code Number of words: 1278 Flesch-Kincaid Readability Score: 9.6 grade level Flesch Reading Ease Measurement: 59.4% or approximately 9 th grade Magna Carta Number of words: 935 Flesch-Kincaid Readability Score: 12.0 grade level Flesch Reading Ease Measurement: 39.5% or approximately 11 th grade Based on these readability analyses, the Magna Carta excerpt was the most difficult document with a readability score of between twelfth and eleventh grades. Because neither of these readability assessments can give an exact grade level score, a discrepancy of one grade level between the two is common and simply provides a realistic range for the readability score. The Justinian Code excerpt was the longest of the three documents at this point, but it had the easiest readability level. Both of the assessments placed it at a ninth grade readability level. Unlike the Magna Carta and the Justinian Code, The Code of Hammurabi, had a large difference between the two readability scores. The Flesch-Kincaid score indicated a readability in the early eleventh grade range while the Flesch Reading Ease indicated a readability in the eighth grade range. Although this likely indicates that the document had a readability level between these two extremes, in the ninth to tenth grade level range, when I considered these scores in the third phase of the project, I based my 131 assessment of the difficulty students would have reading the source on the higher grade level score. I chose to use this score because I felt that this was the more cautious course and would allow me to be more cautious in the choices I made as I inserted hyperlinks in the documents. I wanted to ensure that the inclusion of these hyperlinks did not increase the level of difficulty students would have as they interacted with the documents. Therefore, beginning that process using the more difficult readability helped to ensure this. Initial analysis of source document excerpts: Domain specific complexity. Analysis of the domain specific complexity of the three documents was more difficult because no standard assessment exists for testing this type of complexity in historical source materials. In order to analyze each document, I used the three criteria listed in Chapter 3: need for background information, complexity of language, and motivational factors, to determine the level of domain specific complexity. I relied on my ten years of experience teaching social studies and reading to seventh and eighth graders and the teacher?s expertise and experience with his students to apply these criteria and determine the level of complexity for each document. Each document was analyzed by considering the three categories separately. For each category the teacher and I rated the documents based on the level of complexity we believed it posed for the students in his class. The levels we used were: 1. Too difficult for any of the students in the class 2. Accessible for only the above average readers 3. Accessible for average readers and above 4. Accessible for almost all of the students in the class. 132 Later in the study I would make adjustments to each document based on the three separate categories I used to determine domain specific complexity: need for background information, complexity of language, and motivational factors. Therefore, I did not combine the analyses of these three categories, but instead, analyzed each document and assigned an appropriate complexity level for each of the three categories. I made a final determination regarding this analysis after the teacher and I individually considered each of the documents and then compared our assessments in order to determine the level of complexity we believed was appropriate for each of the categories for each document. The first factor we used in assessing each document?s domain specific complexity was the difficulty posed for students by the need for background information. Using this criterion, both the teacher and I determined that each of the three documents was too difficult for any of the students in the class to comprehend without significant assistance. Each document would require students to create complex mental concept maps that utilized a substantial amount of information and that made connections among multiple historical concepts. For example, in The Code of Hammurabi students would need to have a basic understanding of polytheistic religious beliefs, the right to rule through divine will, hierarchical structure of Mesopotamian civilization, commerce and agriculture in ancient societies, and the role of written language in early societies. The Justinian Code required students to understand Roman familial and societal structure, the Roman belief in the logic and order of society, and to be able to comprehend several difficult issues involving inheritance and custodial arrangements. In order to understand the Magna Carta students would need to understand how a feudal system of government worked, the power of the English monarchy prior to the signing of the charter, the 133 realities of travel in England during the thirteenth century, and the judicial system in England at this time. The difficulties posed by the historical background information required for students to understand each of these documents convinced the teacher and me that all three should be considered well beyond the abilities of students in the seventh grade. When we considered each of the documents based on the second domain specific complexity factor, the difficulty posed by unfamiliar use of language, the teacher and I again came to similar conclusions. We both concluded that The Code of Hammurabi would be accessible for the above average readers and that the average readers would find most of the text accessible. We believed that most of the text was clearly written and that with the exception of some words that would be unfamiliar most of the students in these categories would be able to understand the excerpt. As with The Code of Hammurabi, the teacher and I were in agreement regarding the level of complexity due to language in the Magna Carta. We believed that the majority of this excerpt would be difficult for all of the students including the above average readers because the Old English style of language usage made this document extremely difficult to understand. Our analysis of the Justinian Code yielded slightly different results for the teacher and me. Despite this document having the lowest readability grade level we both thought it would pose a considerable challenge for many of the students. However, in our initial assessment of the document to determine the level of domain specific complexity it posed due to unfamiliar language, the teacher believed few of the above average readers would find the document accessible while I believed most of the above average readers would be able to understand the document but few of the average readers would. As we 134 discussed the difference in our assessment we decided that the teacher had placed more emphasis on the use of unfamiliar words in the text, such as curator and tutor, while I had considered those words as part of the background information complexity and had not included them in my analysis of the complexity due to language. We decided that it was best to only consider these in one category and that we would use them in the analysis of the need for background information. We reasoned that even if a student understood the words themselves, an understanding of the modern meaning of these and similar words in the text would be of little value because those meanings are considerably different from the meanings in this text. After this discussion we agreed that based on the complexity due to language, most of the above average readers would be able to comprehend the text but the average and below average students would be unable to access the information in the document. Because we had not yet developed the study lesson, at this time we did not analyze the final factor in domain specific complexity: the motivational factor of the text based on the presentation of the text and the uses to which it would be put. This factor is heavily dependent on the lesson the documents are situated within and the way the information is presented as students use it in that context. Without considering the context of the document excerpts as part of a problem-based historical inquiry lesson, each of the documents would have to be considered too complex for any of the students. Once the lesson planning was complete, however, and a problem had been identified, the motivational aspect of domain specific complexity could be assessed since a motivational problem might prompt a more careful reading of the texts. 135 When the analyses of these documents, based on both readability levels and domain specific complexity, was complete we determined that few of the students would be able to meaningfully access the texts as they were at this point. Each had a readability level well above that of almost all of the students in the classes and each would be extremely difficult for most of the students based on their domain specific complexity. Phase II: Coding of documents The second phase of the study was devoted to gathering data for use in the third phase. During Phase II, the students, the teacher, and I coded each of the three excerpted documents in order to identify the areas in each document that would benefit the most from definitional or historical background information, or from metacognitive scaffolding. Coding by the teacher and me was completed separately using the same method. I assigned colors to three types of assistance the teacher and I might believe students would benefit from as they read the document excerpts 1. Student will need a definition for this word or phrase 2. Student will need historical background information for this word, phrase or section 3. Student will need metacognitive assistance in this area. The first two items in this list were the same as the choices given to students as they coded the documents and would be used in Phase III to determine how well teacher and researcher coding compared to the students? coding. The students did not have the third category because it is unlikely they would be able to recognize when they were having metacognitive difficulties or when they could improve their understanding using this type of scaffolding. 136 Phase III: Analysis of coding and document preparation When coding was complete I did a comparative analysis of the document coding from the students, teacher, and myself. In this comparison, I considered the similarities and differences between the areas where the teacher and I believed the students would need assistance and the areas where the students believed they needed assistance. I also compared the type of assistance students thought they needed to the type of assistance we believed they needed. This analysis provided data that I used for two separate purposes. The analysis of this data was first used to inform my inclusion of hyperlinks in the excerpted source documents. The second purpose was to determine the success the teacher and I, as experts, had in identifying the areas students, as novices, would need assistance. My intention with this comparison was to determine how effective the teacher and I would have been without student input in determining where to add scaffolding in order to have the greatest positive impact on the students? ability to utilize the source documents. Analysis of coding for including hyperlinks. I began my analysis of the coded documents by counting the number of times specific words, phrases, or sections were highlighted in a specific color. I established guidelines for considering the different categories of student coding. When I was considering student coding that indicated a need for historical information I often combined differing areas of coding if it was clear that the students needed assistance with the same information but had identified that need in different places. As Table 5 shows, the students often coded passages in ways that made it almost impossible to differentiate each variation as I counted the number of students needing a particular type of assistance in an area. To solve this problem I 137 Table 5 Student coding for historical background information Coding by Student 1 Need historic information: ?Marduk sent me to rule over men? and ?brought about the well-being of the oppressed? Need definition: ?protection of right? Coding by Student 2 Need historic information: ?When Marduk sent me to rule over men, to give the protection of right to the land, I did right and righteousness in? and brought about the well-being of the oppressed.? Need definition: ?Marduk? and ?protection of right to the land? Coding by Student 3 Need historic information: ?Marduk sent me? and ?I did right and righteousness?and brought about the well-being of the oppressed? Coding by Student 4 Need historic information: ?When Marduk sent me to rule over men? Confused but do not know why: to give the protection of right to the land, I did right and righteousness. . . , and brought about the well-being of the oppressed Need definition: ?righteousness? Original section of text from The Code of Hammurabi: When Marduk sent me to rule over men, to give the protection of right to the land, I did right and righteousness. . . , and brought about the well-being of the oppressed. counted closely related passages as the same coding. When students identified a need for assistance but indicated that they were unsure of the reason for their confusion, I indicated this in my analysis. However, when I used the analysis to determine the best places to provide scaffolding, I determined the type of assistance most likely to alleviate the confusion in that area based on coding done by other students who were clearer on the cause of their confusion and on the coding completed by the teacher and I. 138 For the example in Table 5, I counted any student coding indicating a need for historical information anywhere in this passage as a need for historical information for the entire passage. In this case the passage was the entire sentence relating to Hammurabi?s right to rule based on divine intervention and good works. In other cases, a passage might be a phrase, a sentence, or an entire paragraph. In each case I determined the length of an individual passage based on whether or not the section presented a unified idea, concept or event that could be addressed using one historical background scaffold. In the example in Table 5, I counted any coding for background information anywhere in the passage as an indication that the student needed historical information. Students one, two, and four also coded a portion of this section as needing a definition, although when I was adding scaffolding to the excerpted document I determined that historical background information would serve both purposes. Finally, Student four indicated a large part of the passage as causing confusion but was not sure why the confusion existed. I counted this as confusion for an unknown reason, but when I used my analysis to include scaffolding, I included that count as an indication of a need for historical information because the majority of students, as well as the teacher and I, had coded the passage in that way. After compiling the data from the student coding I compared this data to the coding done by the teacher and myself. Because student coding categories and the categories used by the teacher and I were not identical, I compared the students? three coding categories, definitional, historical background, and unknown confusion, with the two categories used by the teacher and me that corresponded to these, definitional and historical background. For each document I identified the areas coded by students, the 139 teacher, and me and I determined the number of times students coded those areas for each given reason. Areas the teacher and I coded as in need of metacognitive assistance were treated differently because the students did not code for that category. I then used this information to determine the best places to include hyperlinked scaffolding. Comparative analysis of students, teacher?s, and researcher?s coding. For the comparative analysis of the students? coding and the teacher?s coding, I will discuss the teacher?s coding and my coding together as expert coding. Coding by the teacher and me differed less than ten percent for each document, and where differences occurred our discussion of our rationale for our coding decisions resolved the differences. There were no instances in which we firmly disagreed about where students could best use definitional, historical background, or metacognitive scaffolding after a discussion about the section. This similarity in the teacher?s coding and my coding indicates a degree of continuity among the coding by the experts, the teacher and researcher in this study. However, the need for discussion to clarify discrepancies also indicates the value of having more than one expert coder in order to validate the coding of the other. The comparison of the students? coding to the teachers? coding indicated that the expert and student coding matched between 80 and 94 percent of the time. As Table 6 shows, when 20 percent or more of the students indicated a need for assistance the expert coding matched student coding 90 percent or more of the time. The largest discrepancies between expert and student coding occurred in areas coded by twenty percent or fewer of the students. In these areas the teacher coding matched the student coding less than ten percent of the time. Although this discrepancy would not affect the inclusion of hyperlinks in this study because of the percentage threshold established for when 140 Table 6 Comparison of expert and student coding Percentage of student coding a passage Percentage of match between expert and student coding 30 % or more 100 % matching 20 to 30 % 90 to 98 % matching 20 % or less Less than ten % hyperlinked scaffolds would be included, a review of individual students? coding indicated that areas coded by twenty percent of the students or less were coded almost entirely by students classified as below grade level readers. This means that in areas where the students with the greatest literacy challenges experienced difficulties that were not experienced by the average and above average readers, it was unlikely the teacher and I identified this as an area where students would need assistance. In this study the need for a cut-off point when deciding where to place hyperlinks had the unintended result that no scaffolds were provided for these areas. Because of the high overall correlation between coding by the teacher and me and that done by the students, I felt confident that for this study I would be able to provide hyperlinked scaffolds that should address the needs of a majority of the students as they read these source documents. Additionally, because the teacher and I agreed on the areas where metacognitive scaffolding would be useful, I also had confidence that we could provide scaffolding to meet the students? needs in this area as well. Although the comparative analysis of the coded source document excerpts was done primarily to compare expert coding to novice coding, I also compared the coding by students in the three different reading skill level categories. A comparison of the average 141 and above average readers indicated few differences in the types of coding done by students in these two groups. Although the students in the above average category had fewer areas of confusion and fewer words or phrases they believed they needed defined, these two groups tended to code for definitional and historical background information at approximately the same levels. The only difference in the two would be that the average readers tended to code for areas where they were confused but were unsure why approximately fifteen percent more often than students in the above average reading group. There were, however, more substantial differences in the coding done by below average readers than those in the average and above average groups. Below average readers tended to code that they needed definitional assistance more often than either average or above average readers. This was expected since students with below average reading skills tend to have significantly smaller vocabularies and would, therefore, identify more words that they do not understand. However, students in this group tended to code that they needed historical background information less often than either of the other groups. This difference tended to be between forty and sixty percent less often than average or above average readers on the three documents. Below average readers were also much more likely to code areas as confusing for an unknown reason than either of the other groups. These differences are likely the result of students in this group having not reached a level of reading proficiency that will allow them to move from attempts to decode the text at a basic literacy level to attempting to understand the text within the context of a history class. 142 VanSledright (2002b) established a continuum that can be used to identify students? level of proficiency when reading historical sources. In this continuum, VanSledright suggests that the two lower levels apply to students? more general reading abilities and includes activities such as ?questioning the document/image? and ?assessing text language/image depiction effectively.? The two upper levels apply directly to students reading within the history domain and doing activities such as ?assess and judging the subtext against other subtexts? and ?checking where source(s) come from, identifying the nature of a source(s) relative to other sources.? I found this continuum helpful when thinking about students and the level of proficiency they were exhibiting as they read these documents, but I found that it was inadequate for use with the below average readers because many of these students were reading below the lowest level on VanSledright?s continuum. Analysis of hyperlinked documents: Readability. After finalizing the hyperlinked documents I analyzed the texts for readability. Because there are no standard measures or methods for testing the readability of hyperlinked documents I decided to use the same measurements I had used for the texts before the hyperlinks were added, the Flesch- Kincaid Readability Measurement and the Flesch Reading Ease Test. I also had to devise a method for analyzing the hyperlinked texts. There were two possibilities to accomplish this analysis, I could test the main text and the hyperlinks separately and then combine the results in order to determine a final readability score or I could test the main text and the hyperlinks together as one text. I decided that the second choice would be the best in this situation because my goal was to better understand the readability of the entire source documents after the inclusion of the hyperlinks. 143 In order to test the readability of the texts I combined the main text and the hyperlink texts for each of the source documents. I then completed the two readability tests on each document with its hyperlinks. Table 7 shows the results of the analyses of the hyperlinked source documents and provides a comparison of these findings with the analyses of the original excerpted texts. Based on the readability analysis of the hyperlinked source documents at the end of Phase III, the addition of hyperlinks altered the readability of the texts in several important ways. Two of the texts, The Code of Hammurabi and the Justinian Code, increased in length significantly with The Code of Hammurabi more than doubling in length. This increase in length is a source of concern because students, especially students with below average reading skills, are often intimidated by exceptionally long documents. It is possible, however, that because the increased length would not be immediately obvious to the students since the additional words were in the hyperlinks that students did not see, that the additional text would not negatively affect student motivation. The only source document that did not significantly increase in length was the Magna Carta. This was due to the decision to replace the entire first section of the original excerpted text and replace it with the shorter description in a historical background hyperlink. For each of the source documents the addition of the hyperlinked text lowered the readability grade levels and also eliminated any large discrepancies between the two measurements. The Code of Hammurabi was originally rated at a ninth- to tenth-grade readability level and after the inclusion of the hyperlinks both measurements place it at a 144 seventh grade level. The addition of hyperlinks to the Justinian Code produced the least changes in readability among the three texts. The Justinian Code was originally rated at Table 7 Comparison of readability statistics for original excerpted source documents and hyperlinked source documents. Number of words: Original text: 730 Hyperlinked text: 1,546 Flesch-Kincaid Readability Score: Original text: 11.2 grade level Hyperlinked text: 6.9 grade level The Code of Hammurabi Flesch Reading Ease Measurement: Original text: 64.9% or approximately 8 th grade Hyperlinked text: 74.8% or approximately 7 th grade Number of words: Original text: 1278 Hyperlinked text: 2,225 Flesch-Kincaid Readability Score: Original text: 9.6 grade level Hyperlinked text: 8.5 grade level Justinian Code Flesch Reading Ease Measurement: Original text: 59.4% or approximately 9 th grade Hyperlinked text: 62.6% or approximately 8 th grade Number of words: Original text: 935 Hyperlinked text: 955 Flesch-Kincaid Readability Score: Original text: 12.0 grade level Hyperlinked text: 8.5 grade level Magna Carta Flesch Reading Ease Measurement: Original text: 39.5% or approximately 11 th grade Hyperlinked text: 70.4% or approximately 8 th grade 145 a ninth-grade readability level and the inclusion of the hyperlinks brought this rating down to an eighth-grade level. The Magna Carta, had the greatest change in readability scores due to the inclusion of hyperlinks, moving from an eleventh- or twelfth-grade readability level to an eighth-grade level. Overall the inclusion of the hyperlinks, while altering the length of each of the documents, greatly reduced the readability grade level for each so that the combined effect should have been that the source documents were easier to utilize for all of the students. Analysis of hyperlinked documents: Domain specific complexity. After the addition of hyperlinks to the source documents and the completion of the study lesson, I reevaluated the domain specific complexity in addition to the readability of the texts. To do this I used the three criteria discussed previously, 1) the need for background information, 2) the difficulty posed by unfamiliar use of language, and 3) the motivational factor of the text based on the presentation of the text and the uses to which it would be put. Using the first two factors, the domain specific complexity of each document should have decreased with the inclusion of the hyperlinks. Additionally, the motivational factor, which was not considered in the previous analysis of the texts because the study lesson was not completed at that time, could reasonably be expected to have lowered the domain specific complexity of each of the documents. The completion of the study lesson, which set the purpose for students reading the documents, should directly affect the motivation of students by establishing a real-world purpose for their reading. Therefore, using these measures and comparing the results to those from the end of Phase I, it was reasonable to assume that the domain specific complexity would be at a level acceptable for most of the average readers in the class. 146 Analysis of Phase II and Phase III data. Analysis of data from these preliminary phases of the study informed the final design intervention that I used to examine the study questions. In the next section I will discuss the study?s summative findings that address these questions. Final analysis: Study questions The guiding question for this study was how may students be supported in working with complex, text-based source documents as part of a problem-based historical inquiry lesson? Five research sub-questions were used to focus the study: Sub-Question 1: How do students interact with source documents presented in an on-line format? Sub-Question 2: How do students use scaffolds in analyzing source documents? Sub-Question 3: How do students use scaffolds to develop problem solutions? Sub-Question 4: Are there similarities and differences for students with different literacy levels? Sub-Question 5: How closely do educator?s expectations of students? needs match students? actual needs? After completing the final phases of the data collection and beginning to analyze the data in relationship to these sub-questions I realized that sub-question four would best be addressed as part of each of the other questions. During the study and early aspects of the analysis, similarities and differences in the ways students with different literacy skill levels interacted with online source documents, made use of scaffolds in analyzing documents, and developed problem solutions were often obvious. The expectations of 147 student needs by educators also proved to be different for students with different literacy skill levels. For this reason, research sub-question four will not be discussed separately but will, instead, be part of the discussion of each of the other four sub-questions. Data from Phase IV, the study lesson, was used primarily as summative data, unlike data from Phases I through III that was used primarily for formative purposes. The only exception to this was that data from Phase IV was used to partially guide the selection of students to participate in the student interviews during Phase V. During the fourth phase of the study, when students were participating in the study lesson, data were collected from a variety of sources including: researcher observation, student work products, student presentations, and individual student compact disks. The student interviews, completed during Phase V, were largely related to the students? work during the study lesson, although the teachers? interview related to his participation in the entire study. Students interacting with on-line source documents The first research sub-question, ?How do students interact with source documents that are presented in an on-line format?? is important because although much is understood about students? interactions with source documents presented in paper format, many questions remain about how this may change when the documents are in an electronic format and especially how this interaction may be different for students with varying literacy skill levels. In my study I used several sources of data to consider this question: researcher observation notes, teacher interview, student interviews, and student work products. Analysis of the data revealed findings related to motivation, the relationship between literacy skills and the use of hyperlinked documents, and teachers? 148 capabilities in predicting areas of potential confusion for students reading complex source documents. Students are often enthusiastic about working in a technology rich environment and this enthusiasm can result in high levels of motivation in the students (Last, et al., 2003; Lee & Clark, 2003). In this study the teacher and I noted that the students were very enthusiastic at the thought of completing part of a lesson on the computer. Although this enthusiasm did not completely disappear, we both noticed that the students quickly developed a more realistic attitude regarding the work as they realized that they were expected to accomplish specific tasks using specific materials. During the interview the teacher noted the students? enthusiasm as he discussed their responses to the lesson. ?Without the documents, the lesson would have been simply another ?history? lesson based upon lecture and the assumption that something ?was because the teacher said it was?. The hyperlinked documents made the lesson real?and [the students] were given tools that allowed them to decode the texts rather than relying on others to interpret [the texts] for them.? This statement makes it clear that the teacher believed the documents were vital to the students? enthusiasm for the lesson. Additionally, all of the students interviewed indicated that they enjoyed working with the online documents and would recommend this type of activity. These comments seem to indicate that many of the students did maintain their enthusiasm for the lesson beyond their initial excitement, and it is possible this influenced the work that they did throughout the lesson. One of the strongest findings regarding students? use of the online source documents in this study was the differences in their use based on literacy skill levels. Student interaction with the online hyperlinked texts was very similar to the interaction 149 that could be expected with students at a specific reading level reading text in any format (Goldman & Rakestraw, 2000; Jenkins, et al., 2003; Tovani, 2000). The below average readers tended to read less for comprehension and more to get through the task at hand. At the beginning of the study lesson students were given instructions about how to read the documents and use the hyperlinks. They were instructed to read the documents once without clicking hyperlinks or being concerned with answering questions. After this initial reading, the students were told to reread the document making use of the hyperlinks as they came to them and needed to use them. Based on both my observations as the students were reading the online documents as well as on students? statements during the student interviews, the below average readers did not follow this reading format. A few of the students read the questions on the data-gathering sheet before reading because that way they would be better able to ?find the answers.? This was, from their point of view, the focus of any lesson. Although students? approaches to hyperlinked documents were similar to their approaches to conventional texts, some of the affordances of hypermedia did seem to affect their ability to work with the texts. One finding regarding students? use of the source documents was the difference in how they perceived the length of the online source documents compared to how they seem to have perceived source documents in other formats. Although the teacher and I deliberately kept the excerpted documents relatively short, one to two pages with a significant amount of white space along with the text, the actual total length of the sources was much longer than this (See Table 4). The Code of Hammurabi and the Justinian Code were each doubled or almost doubled after the inclusion of the hyperlinks. Several students commented about their perceptions of 150 the documents? length during their interviews. One below average reader discussed her willingness to attempt to read the Magna Carta, ?I looked at it and it had big words, but it wasn?t very long so I figured I could read it.? Another below average reader mentioned the longer The Code of Hammurabi and why he was willing to read it, ?It was like, two pages, but there were lots of short sentences and a lot of the words were short so I thought it wouldn?t be too hard, and I saw lots of underlined words so I knew I could find out what they meant.? These students seem to have based their willingness to attempt to engage the in the study of the source documents on their perception of the amount of work that would be required for this effort. Their perception that the documents were short, and therefore easy, allowed these students to believe they could be successful. When this perception was combined with their initial enthusiasm about working on the computer and their understanding that difficult aspects of the documents had hyperlinked assistance built in, many of the students seemed willing to put forth a greater effort than might have otherwise been the case. The teacher noted this increased motivation in his interview and other researchers have identified increased motivation with increased effort and the potential for greater success in both technology-based environments and in more traditional environments as well (Britt, et al., 2000; Dewey, 1938; Last, et al., 2001; Levstik & Barton, 2005; VanSickle, 1996; Wineburg, 2001). Although most of the students did not seem to recognize that they had actually read far more than they believed they would be reading based on the visible length of the excerpted source documents, one of the above average readers did recognize this. In the interview she was answering a question about which document she found to be the most 151 difficult, ?I thought the Justinian Code was the hardest. It didn?t look hard at first ?cause they all looked short and I thought they would just be easy, but it was like you tricked us because I?d be reading and clicking on stuff and reading what was there and there was a lot of reading, but most of it was hidden and that was kind of sneaky.? This student went on to say that she would recommend this method of reading difficult text for a variety of reasons, seeming to indicate that her recognition that there was more reading than she originally thought did not dissuade her from wanting to use hyperlinked text in the future. This student makes an important point when she talks about much of the work of utilizing the documents being hidden and unexpected. This may be related to the trouble some students had handling the cognitive load of using multiple sources of information in the online format, a difficulty that has been noted by several researchers (Beaufils, 2000; Brush & Saye, 2001; Hannafin, et al., 1997; Lin, 1994; McLoughlin & Hollingworth, 2002; Saye & Brush, 1999; Shapiro, 1999). Related to the issue of cognitive load is the issue of students? preference for reading paper or online text. Many students and non-students indicate that they do not like reading text from a computer screen (Milson, 2001). During the study lesson when the students were reading the documents online I had several copies of the excerpted texts available in case there was a problem with a computer or another need for a paper copy arose. One student asked if she could read part of the text from the paper copy and she took the pages back to her seat. Later I realized she was still reading from the paper copy so I observed her use of the source documents in the two formats, and I also asked her about this later in the interview. She said that she was uncomfortable doing all of her reading on the computer, but she liked having the hyperlinked scaffolds available when 152 she needed them. As she read from the paper document she developed a fairly elaborate system in which she would read from the paper and utilize the links from the online documents as she came to them on paper. When I asked her why she read this way she said that she got tired of trying to read online and would sometimes lose her place, so she read on paper and made use of the advantages the technology afforded her as well. In addition to the cognitive load of mentally managing information presented in the online format, students also had to develop a mental map of the complex information in these documents and create meaning from text that was written for a purpose other than to inform students. This is an aspect of problem-based historical inquiry and is not unique to students utilizing online hyperlinked source documents. During the group activities many of the students indicated that they wished they could look back at the on- line, hyperlinked documents because they had found some parts of them hard to understand. Some students also commented during the interviews that the reading task had been harder than they thought it would be. Because all of the students interviewed indicated that they preferred to work with source documents in this way rather than working with only paper documents, including the student that had so deliberately read from the paper document and then used the technology to access the scaffolds, I concluded that difficulties inherent in the source documents or created by reading online may have been offset by the benefits students believed they received from working in the hyperlink scaffolded environment. The teacher also believed that the students? reading abilities impacted their ability to use the online source documents successfully, although he concluded the students benefited from the hyperlinked documents overall. ?As with many historical texts, some of the texts were above the reading level of many of my 153 students. This was a problem, but I felt that the hyperlinks helped decrease the level of frustration for my students.? Students using hyperlinked scaffolds to analyze documents Analysis of students? use of online source documents leads to the second research sub-question, ?How do students use hyperlinked scaffolds in analyzing source documents?? As in the analysis of students? overall use of the source documents, students? use of the hyperlinks to assist in their analysis of these documents was often differentiated by students? reading skill levels with below average readers and above average readers often utilizing the hyperlinks in markedly different ways. When students read the online source documents, my analysis indicated that the below average readers tended to read these documents with less concern for comprehension than for accomplishing the associated tasks, such as filling in all of the blanks on their data-gathering scaffold. The students tended to read less strategically than they would have done if they had been reading to comprehend the documents at a holistic level. This lack of strategic reading in order to understand the documents as a whole was evident in their use of the hyperlinked scaffolding as well (Milson, 2001; Tovani, 2000). During the interviews I asked students to describe how they used the hyperlinks and in most cases I had them show me as well. There was a clear difference in the way below average readers used hyperlinks compared to the majority of average and above average readers. Each of the below average readers used the hyperlinked scaffolds in an almost identical manner. This quote is from a student whose description of his use of all three types of hyperlinked scaffolds is remarkably similar to the descriptions from the other below average readers. He discussed his use as he showed me on the computer: 154 I kind of used them all at the same time. I had clicked on the globe [historical background link] and read what it said and then I kept reading and clicked on the underlined word [definitional link] ?cause I thought I knew what it meant but I wasn?t sure. Then I read the rest of the sentence and read the question at the first of the next sentence before I started it because the question was at the front. This student was typical of each of the below average readers in the study in that he utilized the hyperlinked scaffolds to get additional information that he needed to improve his comprehension, but the manner in which he used the links most likely prevented him from developing a thorough understanding of the source documents. In follow-up questions I also asked each of these students to compare their use of hyperlinked texts to source materials accessed in a textbook or other paper source, specifically how they would handle a word they did not understand or a reference they did not understand. One of the below average readers indicated that she would look it up in a dictionary if she did not know a word because her mother would make her, but each of the others indicated that in a different reading situation they would simply skip words they did not know or references to information they did not understand and hope that they would ?get it later or the teacher would tell us what it was talking about.? Although this approach would allow for more cohesion in reading the material because the students were not stopping to click on definitions, there would be a loss of understanding and they would comprehend less because of the number of times words were omitted and background information not understood. As with the below average readers, the above average readers tended to respond to the question about how they used the hyperlinked scaffolding as they were reading 155 with remarkable similarity. Although there were more differences in specifics indicated by these students, their use of hyperlinks as they read was generally the same. Where the below average readers tended to read until they found a link, stop to read the link, and then pick up reading where they had left off, the above average readers had a variety of strategies that allowed them to retain the cohesion of the text and still get the information from the hyperlinked scaffolds as they needed it. Several students used a sentence by sentence approach in which they would read until they came to a link they needed to use, they would finish reading the sentence and then go back to click the link, after reading the link they would then reread the sentence and continue on with the document. Several others would click on all of the links connected to a paragraph or section before they started reading. This allowed them to make sure that they understood all of the words and had all of the background information needed before they read the text. The others tended to do the opposite of this approach in that they would read the paragraph or section, they would go back and click the links they needed, and then they would reread the paragraph. This was the least used approach by the above average readers, perhaps because it involved more additional reading than the other two. Despite these differing approaches several areas of continuity existed in the use of the links by the above average readers. First, each of these readers used the links in relationship to their reading of the documents in a way that allowed them to maintain text cohesiveness as they were reading. Second, each had a strategy for using the hyperlinks as opposed to simply using them as they appeared in the text. Third, each utilized the hyperlinked scaffolds as they needed them so that they did not click on definitional scaffolds if they were certain they knew the meaning of the word. Average readers 156 tended to use the hyperlinks at various places on a continuum between the below average readers? use of the hyperlinks with little or no strategy and the above average readers? use with well planned strategies. Another finding related to reading skill levels was that below average readers tended to utilize the definitional scaffolds more than the historical background scaffolds. This is despite the instructions students received explaining that the historical background scaffolds should be used each time and the definitional scaffolds were to be used only when needed. My observations suggested that this happened most often when slower reading students began to fear they were going to run out of time to finish reading. At that point they would begin reading the documents using only the definitional scaffolds and attempting to obtain the information they needed to complete their data-gathering scaffolds. During interviews these observations were reinforced because several of the below average students made comments about feeling rushed at the end of the class period and being afraid they would not be able to finish reading. These same students often indicated that they would sometimes skip the historical background scaffolds and instead concentrate on the metacognitive scaffolds that were connected to their data- gathering sheets. Based on this data from the student interviews, these students took a pragmatic approach to the use of the scaffolds. Even though they had been instructed to use all of the historical background hyperlinks, they determined that completing the data gathering sheets was the more important task based on their beliefs about schoolwork. Therefore, if they believed they understood enough to put a response on the data- gathering sheet, reading the historical background information was unnecessary. Many of these same students indicated later in the lesson that they would like to go back and 157 look at the source documents and hyperlinked scaffolds because they believed they needed information they contained. I hypothesized that the use or lack of use was believed to be a pragmatic issue at first but after the students were motivated by a need for the information their understanding of the usefulness of the links changed. Data from several sources suggest that even students who did not skip the historical background links did not always utilize them fully. My observations of group decision-making activities, student responses on individual data-gathering sheets, and student interviews indicate that during the individual analysis of the source documents, many of the students did not utilize the historical background links effectively. During group work activities, when the students no longer had access to the online documents, data from my observations and student interviews indicated that a majority of the students would have liked to have access to these links in order to finish finding information they had not believed important when they read the documents the first time. Analysis of these students? individual data-gathering sheets also suggested that these students failed to utilize the historical background links effectively. Many of the students responded to the data-gathering and metacognitive scaffolds in ways that suggested hurried responses to questions that they believed were unimportant. At times these students simply copied phrases or sentences directly from the text, even when this was a totally inappropriate response to the question. Although this data suggests that some students failed to adequately utilize these links, why this happened is not clear. I hypothesize that for many students a lack of motivation, due in part to their unfamiliarity with the problem-based historical inquiry lesson format, may have contributed to this (Last, et al., 2003; Levstik & Barton, 2005; Neiderhauser & Shapiro, 2003; Thornton, 158 1998; VanSickle, 1996; Wineburg, 2001). I do not, however, believe this is an adequate explanation for all of the students because some of the students whose use of the scaffolds seemed to indicate a lack of motivation were actually very motivated based on data from their interviews and my observations. Student reading skill levels may provide another possible explanation for the ineffective utilization of the historical background scaffolds by some students. During interviews I asked the students about their ability to remember the material from the links while they were participating in the rest of the lesson and found that of the students that participated in the interviews there were almost equal percentages that said yes, no, and sometimes when asked if they could remember the information from these links. Among the students answering in each of these categories there was a great deal of continuity. All of the below average readers in the study took part in the interviews and all of them indicated that they either did not believe they could remember the information from the historical background hyperlinks or they did not think they could most of the time. In the other two categories all but one of the average readers said they could remember the historical background information sometimes with one average reader indicating that she could remember most of the time. All of the above average readers indicated that they could remember this information all or most of the time except for one student that indicated she could remember part of the time. These very different findings for the students in the different reading level categories suggest that students? retention of the material is likely connected to the reading skill level they begin the activity with. Data from the individual data-gathering sheets reinforced this belief because when student reading level was used to divide these, 159 there were clear differences in the responses of the students with above average readers having the most thorough and thoughtful responses, below average readers having responses that were often either inappropriate or inaccurate, and average readers? responses falling somewhere between these two extremes. These findings suggested that even if hyperlinked scaffolding can help all students improve their use of source documents, differences in reading skill levels continue to play a large role in the ability of students to adequately access text-based information in any format. One piece of evidence that I had hoped would provide some insight into the students? use of the hyperlinks as they analyzed the data was the individualized compact disks created for each student. Although I was able to see the links that students had used, a review of the links for the students indicated that all of the students had utilized almost all of the links. Later in the interviews I asked students to explain this, especially in light of the students often contending that they had not used all of the links. The explanation seemed to be that the below average readers tended to use all of the links because they thought they were supposed to, the average readers tended to use all of the links because they wanted to check their understanding, and the above average readers, as well as some average readers that did not use all of the links while reading, finished reading the text before the rest of the class and went back to the links and used them to ?see what they said.? For this reason, tracking the links students used was not possible in this study. Despite the difficulties some students faced as they used the hyperlinked scaffolds to analyze the three source documents, in the interviews all of the students believed the hyperlinks had helped them understand the information better. When asked if she would 160 want to read other documents using the hyperlinked scaffolds, a below average reader said, ?Yes. You can understand it more. Once you click on a link you can know the word so you know more.? During the lesson this student struggled with the metacognitive scaffolds and near the end of the class period she would abandon the use of the historical background scaffolds and concentrate on utilizing the definitional scaffolds in order to develop a basic understanding of the documents. For her, the ability to receive assistance as she struggled to understand the documents at their most basic level was important. Another student, an average reader who had become frustrated several times by the questions that asked him to respond based on what he thought was the best or most appropriate answer, nevertheless indicated that he believed reading this way was better than reading from the textbook. ?[In the book] if I was really confused and couldn?t figure it out, I?d keep reading and maybe later get a dictionary. When I was in the computer I just did it right then?I think it was better because you had the links to help you as you went along. I would want to read like this again.? Based on my observations and the students? comments, each of these students found aspects of the reading activity difficult, but each suggested that the benefits they derived from this type of reading activity outweighed the negative aspects of the reading. The teacher also indicated that the inclusion of the hyperlinked scaffolds had assisted the students? analysis of the texts. ?Without the hyperlinks, I found myself trying to tell students everything they needed to know. With the hyperlinks, though, one of the most successful aspects [of the lesson] was that each student was empowered with control of his or her learning experience. Rather than relying on others to interpret the texts, they were given tools that allowed them to decode the texts.? 161 Students using scaffolds to develop problem solutions Students? use of scaffolds to develop solutions to the historical problems presented to them during the last part of the lesson is an aspect of my analysis that seemed to indicate clear differences among all three groups of readers, above average, average, and below average. Overall my analysis of observations during the group decision-making activities and group problem solution presentations indicated that the students had utilized the hyperlinked scaffolds to develop understandings of the source documents, and the group decision-making scaffolds helped the students synthesize this information into a useable form. The teacher?s interview corroborated my beliefs regarding this overall effectiveness of both types of scaffolds. ?The depth of the answers received from the use of hyperlinked texts was astounding. I guess if I could figure out how to put my entire textbook and supporting documents online, my students would likely do better. With the hyperlinks, though, they were given the power to answer their own questions before immediately relying on the teacher.? A comparison of individual data-gathering sheets, my observational data from the group work activities, and student interviews suggested that the students were able to use the hyperlinked scaffolds to analyze the source documents and create appropriate solutions to the lesson problems. However, the students with different reading skill levels did this with differing degrees of success. Above average readers tended to have more thorough responses on their data gathering sheets, in interviews they responded that they could usually remember the information in most of the hyperlinks, and observations of these students indicated that they were more likely to take a leadership role in the group discussions and check facts as the group developed responses to the problems. 162 Average readers were often the most frustrated during the data-gathering aspect of the lesson when they couldn?t ?find the answers.? During group work when these answers were needed they were also more likely to be frustrated and ask if there was a way for them to return to the online documents and scaffolds to find additional information. The average readers were often more uncomfortable taking a position on the problems if they didn?t believe they had sufficient facts, but once students in this group believed they had gathered sufficient information they were able to use the group work scaffolds to help develop a response. The below average readers were the least successful at using the scaffolds to gather relevant information and at applying their knowledge to develop responses to the problems. During interviews they indicated that they could seldom remember the information in the historical background scaffolds, although this may be because they did not utilize them effectively and in some cases did not read them at all during the lesson. The individual data gathering sheets from this group did not reflect thoughtful responses and in many cases their responses suggested that they were unable to comprehend the source at a level that would allow in-depth answers. In group work, however, it became obvious that the students with below average reading skills had developed some understanding and could occasionally explain responses to their group orally much more effectively than they had on their written responses on the data-gathering sheet. Although the students in each of the reading groups performed at very different levels as they used the information in the scaffolds to solve problems, each group was able to do this activity with a level of success that suggested they had attained a higher level of success than they might have done without the scaffolds. 163 In the previous section I discussed the differences in the ways students with different levels of reading skills utilized the historical background hyperlinked scaffolds. These differences also appear to have impacted students? abilities to individually utilize the other scaffolds including the metacognitive scaffolds. As discussed previously, individual data-gathering sheets indicated differences among students in the three reading level groups and these differences suggest differences in the students? ability to use this information for problem solving. As I analyzed group problem-solving scaffolds, individual data-gathering sheets, group presentations, and my observations, the data suggested that these differences in reading ability may have been mitigated by other aspects of the lesson. These analyses suggest that all of the students participated fully and made meaningful contributions to the group deliberations and presentations even if their individual data gathering had not been completely successful. This would indicate that even those students whose reading skill level abilities may have hindered their use of the source documents were able and motivated to contribute to the group activities. I hypothesize that this may have been a result of the problem-based historical inquiry task that motivated the students to complete the group work and develop defensible solutions. There are several possible reasons for the possible increase in student motivation including the novelty of the lesson?s structure, the authentic nature of the activities, the comfort of knowing that they were able to work with others and receive assistance if they encountered difficulty, or some combination of these potential reasons. Above average readers. As with students? ability to utilize the hyperlinked scaffolds to access information, differences were apparent among the three groups of students as they used the scaffolds to think about the information they were reading and 164 develop solutions to problems. As in previous aspects of the study?s analysis, I found that the above average readers tended to use the scaffolds strategically to think about the issues and then transfer that understanding to the problems for which they were developing solutions. Based on this group?s responses on their individual data-gathering sheets, during interviews, and my observations during both individual and group work, I concluded that students in the above average reading group were better able to handle the cognitive load of synthesizing the text in both the source document excerpts and the hyperlinks. They were also better able to combine this information with other material during the group decision-making activity. This group?s success with the scaffolds was likely a result of their ability to read and comprehend at a level that made them able to understand any text rather than to understand this text in particular (Milson, 2001; Perkins-Gough, 2002; Stahl, 2003; Tovani, 2000; VanSledright, 2002a, 2002b). Below average readers. Below average readers tended to be unable to successfully utilize the scaffolds for solving problems. The data from all sources seems to suggest that the below average readers continued to struggle with the cognitive load required for reading the text and using the hyperlinked scaffolds to make meaning of the source documents. Considering this continuing struggle, it seems reasonable to assume that this did not allow them to then apply any knowledge in novel ways to develop problem solutions. This does not mean that the below average readers were unable to develop solutions to the problems at the end of the lesson, simply that their solutions tended to be more general and less likely to directly utilize the information they had access to through the hyperlinked scaffolds. In their use of the data-gathering scaffolds that were connected with the metacognitve scaffolds these students usually reacted as if 165 the scaffolds were simply something to complete in order to be finished with the task. Their answers tended to be text copied directly from the source documents and often seemed to reflect a lack of confidence in their ability to understand the material and develop answers or solutions on their own. This group struggled to comprehend the source documents, to combine and synthesize information from hyperlinked sources, and to consider this combined information in complex ways in order to develop defensible problem solutions. While the cognitive load for all of the students was great in this lesson due to unfamiliarity with the problem-based inquiry format, the data suggests that it became overwhelming at times for the below average readers (Allington, 2002; Milson, 2001; Perkins-Gough, 2002; Stahl, 2003; Tovani, 2000; VanSledright, 2002a, 2002b). Average readers. The analysis of this aspect of the study data suggests that the average readers also tended to perform in a way that was unique to the majority of that group. In previous aspects of the analysis these students tended to move on a continuum between the above average readers and the below average readers. In the use of scaffolds to develop solutions to problems, however, many students in this group exhibited characteristics that were unique to the average readers. These students often became very frustrated with the search for solutions to problems. Although many had done well in their use of the online source documents and went on to successfully develop solutions to the problems, this frustration and resistance to using the scaffolds for this purpose was of interest. During the interviews most of these average readers tended to describe themselves as ?good? readers and most said they enjoyed the lesson activities, but the resistance many of them exhibited during each step of the lesson suggested that they were 166 frustrated by work requirements that were outside the boundaries of what they had become comfortable with. This is not unique to my study, but instead reinforces what others have seen when introducing authentic instruction into a curriculum. Tovani (2000) indicates that average students often become comfortable with their ability to ?do school.? They have developed strategies and patterns of thought and behavior that allow them to successfully complete each of the tasks normally assigned in a classroom setting. The responses of the students in this group indicated that they were excited about the prospect of working on the computers but became distressed when the activities they were assigned to do using the online documents, as well as the group problem solving activities, were outside the realm of activities they were familiar with. As a result, this group of students tended to show the most resistance to working with the online source documents and the scaffolds in the ways they were instructed and, based on my observations and student interviews, they were more likely to try to fit the study lesson activities into a paradigm they were comfortable with. One student that I observed during the first part of the study lesson became frustrated to the point of almost not being able to continue with his work. He called the teacher and me to his seat repeatedly wanting to know what a particular question was asking for and where to ?find the answer.? I knew he could understand the questions, because when I asked him to tell me what he thought it was asking he could accurately interpret the questions, but he would immediately indicate that he knew what it was asking for but the ?answer? was nowhere to be found. One of these exchanges happened as the student was reading The Code of Hammurabi and was centered on a metacognitive scaffold that asked the students to consider why Hammurabi compared himself to 167 Shamash. Historical background links provided the students with information about Mesopotamian religion and that Shamash was a god believed to uphold truth and justice who was also a lawgiver. Because Hammurabi was giving laws to the people, the metacognitive question was intended to have students consider why he would compare himself to a god that gave laws and how, if this comparison were true, it would validate his right to rule through divine right. As I spoke with the student about this question his agitation was quite apparent. He pointed out to me that he had found all the words in the question in the text, but ?the answer? was not ?near any of them.? When I pointed out to him that the question asked him to express what he thought after reading the section of the document, he explained to me that when a question asked you to say what you thought all you ?really have to do is find where the answer is and write it in your own words.? In reviewing this student?s response to that metacognitive scaffold question the student did just that, responding ?because he is the god of Justic[e]?, even though he knew that this was not an adequate response and despite the fact that in my conversation with him he could articulate a much better response than the one he chose to put on his individual data-gathering sheet. I also interviewed this student and asked him about his use of the metacognitive scaffolds. His response was, ?I was confused by some of them because I was trying to find the answers where the [links] were. It would have been better to just put it on the side or at the end and say this is the area where the answer is so I wouldn?t think the answer was in that exact place.? Later as I discussed this incident, as well as several similar instances, with the teacher he related that he had similar conversations with this student and with several other students. In my notes I recorded the students we recalled having these types of 168 conversations with and a review of these indicated that with one exception, a below average reader who was borderline average to below average according to the teacher, all of the students expressing this type of frustration were average readers who usually did well in their class work. As is evidenced in the description of the experiences of the student in my example and less dramatically in the experiences of many other students with average reading skill levels, the metacognitive scaffolds were an added layer of complexity that they had extreme difficulty dealing with. These scaffolds asked them to use information in novel ways that they were not familiar with and not comfortable with. In reviewing the students? individual scaffolds as well as their group work scaffolds, I believe most of the students managed to overcome this to a degree (See Table 5 for a comparison of individual scaffold answers from students in the three reading level skills groups). Based on their ability, motivation, and reaction to the lesson, these average readers seem similar to ?low knowledge/high motivation? students in Last, et al.?s (2001) study. Last and colleagues found that students they classified as ?low knowledge/high motivation? exhibited the greatest frustration during the lesson. Like those students, the average readers in my study were highly motivated to do well because they saw themselves as ?good students? and ?good readers.? Their reading skill level, lack of familiarity with the lesson format, and with the use of technology for unfamiliar purposes, however, placed them in the low knowledge category. The frustration they exhibited is very similar to that described by Last, et al. and suggests that this group of students behaves in unique ways that should be considered when developing scaffolding to assist students as the struggle with the cognitive load of this type of activity. 169 The ways students used the scaffolds to develop problem solutions must be tied to their use of the scaffolds in general and my analysis suggests that this was influenced heavily by the students? reading skill level. As I have discussed previously, each group exhibited certain characteristics that, while not exhibited by each member of the group in each instance, were consistent overall among the students at each reading level. In Table 8, students one, two, and three are above average readers and each responds to this scaffold question with a brief response that they think is appropriate after reading the source document and the connected historical background hyperlinks. Although the response from Student 2 is perhaps the least original, this student could explain why he answered as he did during the interview. The below average readers responded by simply copying directly from the text, and it is reasonable to assume this is a result of the difficulty they experienced as they tried to comprehend the text. When I asked Student 8 about her response to this question as well as the other metacognitive questions she said that when she came to the place where a metacognitive scaffold was she would ?go over and write the answer from there?, a response that was largely the same for each of the below average readers. This type of response from below average readers is typical of students who have become accustomed to struggling to make meaning of text (Allington, 2003; Strong, et al., 2002; Tovani, 2000). Last, et al. (2001) saw similar results in their study with the ?low knowledge/low motivation? group. Findings suggest that these students have accepted that the work is going to be beyond their abilities and are therefore not motivated to put forth much effort. In my study, a majority of the below average readers maintained this Comparison of metacognitive scaffolding responses by reading skills level 170 Table 8 Metacognitive scaffold from The Code of Hammurabi Question: Why do you think Hammurabi compares himself to Shamash? Above average readers Student 1: Student 2: Student 3: Because both would be ruling over the Mesopotamians He wants to uphold the law and truth Because he was a lawgiver too Average readers Student 4: Student 5: Student 6: He compares himself with Shamash because Shamash decreed the fate of the land and he do not fear god he believe to uphold truth, and justice. Because he is a god that makes laws and Hammurabi wanted to rule of the righteousness. Because he is the god of justic Below average readers Student 7: Student 8: Student 9: Because enlighten the land and further the well-being of mankind To be a god ? to believed uphold truths He was supposed to be the ruler attitude until they began working in their decision-making groups. At that time most seemed to believe they might be able to do the work and many became motivated to contribute meaningfully to this aspect of the activity. This might suggest that finding ways to motivate these students and help them believe they can achieve is imperative if they are going to meaningfully utilize scaffolds put in place to help them. This seems especially true because the below average readers I interviewed indicated that they believed the scaffolds helped them understand the source documents better than they would have without them. The average readers, as discussed in the previous example and exemplified in Table 8, were unwilling to simply copy what was in the text, but they also wanted to be able to ?find the answer.? The data suggests that these students responded to the metacognitive scaffolds using response criteria that had worked for them in the past, 171 specifically to find the area where the answer is located and then respond ?in their own words.? After talking with these students about their use of these scaffolds during interviews as well as in discussions during the lesson, I was unsure if this was because of a deficit of skills or because this was what they believed was the appropriate way to respond. Both Student 4 and Student 5 were careful to respond using complete sentences, but neither these two students nor Student 6 were comfortable responding without including text from the source document despite being able to articulate a response to the question that did not include a quote from the text during the interview. Several contradictory findings are suggested for the average readers. All of the students in this group believed the hyperlinked scaffolds allowed them to understand the source documents better. They were motivated to do well on the lesson and they liked and recommended the instructional strategy. Although a majority indicated some level of frustration during the lesson the reasons for this frustration are not always clear and most of the average readers performed adequately during the group decision-making, although several expressed frustration at this point as well because they were uncomfortable with the ambiguity of the problems. These findings are similar to those from Lee and Clarke (2003) who found that students in their study also became frustrated when their expectations of successful schoolwork strategies failed to provide successful results in a learning environment they were unfamiliar with, suggesting that additional scaffolding or other strategies may need to be employed when using this type of activity in order to increase the likelihood of success for these students. Influence of motivation. I also found that the use of the scaffolds for the problem- solving tasks was influenced heavily by the students? motivation related to the task they 172 were being asked to perform. Although the problem-solving task was explained at the beginning of the lesson, the students were unfamiliar with this type of activity and, therefore, many did not understand the importance of using their scaffolds effectively during the data-gathering phase. For this reason, when they began the problem-solving phase many students realized that they lacked the knowledge they needed and asked if they could return to the online documents to find this information. Often these students would be looking at the paper documents and would recall that one of the hyperlinked scaffolds gave information they now believed was important, but they were unable to remember the details and had not remembered or recorded this when they were using the scaffolds during the earlier part of the lesson. Although the students often indicated that they would ?pay more attention next time? that did not alleviate the frustration of not having the information for this lesson. While several aspects of my analysis of student use of the scaffolds have indicated differences among students based on reading skill levels, my analysis of students? motivation to use the scaffolds did not appear to be related to skill levels. Based on my observations students in all groups expressed this same frustration equally as they moved into the later parts of the lesson, and although the students? use of the scaffolds would likely have been heavily influenced by their reading skill levels if they had been using the scaffolds after having been motivated by a need for the information, it is important that all of the students recognized this need and responded to it similarly. Although most students expressed some level of frustration that they had not responded to the metacognitive scaffolds at a level that allowed them to address the problem-solving tasks in the lesson, they were motivated to use the information they had 173 more effectively when they were working on a problem that they found interesting. For example, most of the students and student groups indicated that the problem that required them to determine if a man was guilty of stealing a dog was their favorite of the three problems. This problem required the groups to consider the problem from the point of view of The Code of Hammurabi. During the group problem-solving part of the lesson more groups asked for assistance in remembering details about the hyperlinked scaffolds in The Code of Hammurabi, as well as determining if their responses to the metacognitive scaffolds completed during the individual work with this document were appropriate than they did for any of the other documents. Students? motivation to find an appropriate response to a problem they were interested in seemed to make them more willing to make more effective use of the scaffolds and also to recognize that they had failed to effectively use these scaffolds during the earlier parts of the lesson. The single student who indicated that he was more interested in the Justinian Code than in The Code of Hammurabi illustrates the motivating potential of relevant problems. Most of the students had become more interested in The Code of Hammurabi as a result of the personal interest they took in the problem of the potential theft of the dog. As I asked this student to explain further why he was more interested in the Justinian Code, I also asked him which of the problems he liked working on the most and why. He responded that he preferred the problem related to what would happen to two young people whose parents had passed away. He further explained that his parents, both were older and in poor health, might not live until he finished high school and so he could personally relate to this problem. Therefore, while the other students were motivated to learn about The Code of Hammurabi because they were interested in a problem involving 174 a dog, this young man was motivated to learn more about the Justinian Code because of his personal interest in the problem dealing with family law, the focus of the excerpt of the Justinian Code. This student?s experience with the lesson strengthens the argument that relevant, personally engaging problems are likely to encourage greater effort on the part of students working with complex source documents. Educators? expectations of students? needs The final research sub-question regarding how closely educators? expectations of students? need match students? actual needs when reading complex source documents is the only question that relied exclusively on data from the early phases of the study. I asked this question because if educators are going to assist students in using source documents it is imperative that they be able to predict where students will struggle. For that reason I had the students code the source documents just as the teacher and I coded the documents and I compared these to determine how our expectations of the students? needs as they read these documents matched the students actual needs in these three source documents. The comparative analysis of the teacher?s, researcher?s, and students? coding indicated that for most average and above average students the teacher and I were able to identify those areas where students were likely to be confused and need assistance more than ninety percent of the time. This level of similarity in coding indicates that in this study the experts, the teacher and I, were able to successfully identify when novices, the students, would need assistance approximately ninety percent of the time for all students and almost one hundred percent of the time when students were reading at an average or above average level. It is important to note that the times when the teacher and I 175 consistently failed to identify students? needs were instances when only the below average readers needed a particular type of assistance and/or assistance in a particular place. The success the study teacher in this study and I had in predicting students? needs as they read these documents also closely matched the rates of success the teachers in the pilot study and I had. This would indicate that in these two instances the teachers and I could have successfully predicted students? needs and included definitional and historical background scaffolding successfully without input from the students. Caution should be used in generalizing these findings due to the limited number of teacher experts involved. However, the success the study teacher, the two pilot study teachers, and I had anticipating students? needs suggests that teachers with a variety of training and experience levels may be able to successfully identify areas of complexity in historical source documents. Conclusion The data from my study suggests that hyperlinked scaffolding may assist students as they engage complex, text-based, historical source documents within a problem-based historical inquiry lesson. Data from the students suggests that they enjoyed using these documents and both the students and the teacher believed their understanding had been enhanced. Although the scaffolded assistance seems to have been a positive influence on students? ability to read and utilize the material, this was not the case in each instance. In some cases the hyperlinks worked to distract students because they did not read the sources as cohesive units of information as was seen with many of the below average readers. Another instance was when many of the average readers concentrated on the 176 metacognitive scaffold questions to the point that they were unable to focus on reading the documents for meaning. Students? interaction with the online source documents suggests that the strategies students use to understand and use these sources is dependent on the students? reading skill level and is very similar to strategies used by these students when reading sources that are not online. Above average readers tended to read more strategically and with more consideration for understanding the document as a whole, whereas below average readers tended to read using much less strategy and with very little consideration for their ability to understand the overall document. Average students tend to perform in this respect at various levels between these two extremes. None of the students read the documents using the strategies the study teacher and I suggested, possibly because of their lack of familiarity with the requirements of the problem-based historical inquiry instructional lesson. Use of the hyperlinked scaffolds to assist students in understanding the documents and in making decisions also seemed strongly influenced by students? reading skill levels. While below average readers tended to respond to the metacognitive scaffolds with indifference and often seemed resigned to not being able to respond adequately, average students became very frustrated by the fact that they did not believe they were responding appropriately to the metacognitive questions. There were also differences in the links students used. Above average and average readers tended to use all of the links, often even if they did not need the definitional links to understand the text. Below average readers tended to use the definitional links more and the historical background links less as they struggled to develop a basic understanding of the text. 177 While there were some situations in which students at differing reading levels experienced difficulty utilizing the source documents and hyperlinked scaffolds, the students and the teacher reported that overall they believed their use was successful. Most of the students believed the hyperlinked scaffolds enabled them to successfully engage the source document excerpts at a level higher than they would have been able to do without this assistance. Although the lack of a control group and the lower percentage of below average readers in the study means that the findings cannot be generalized beyond the classes and in most instances within the class, the positive findings suggest that further study is needed to better understand these results. The issue of motivation remained a problem for many students at the average and below average reading levels. Some aspects of the intervention such as perceived length of the text and hyperlinked scaffolding that allowed the students to make meaning of the text without asking for assistance seemed to act as motivating factors for all students. Motivation in this study was also a result of the authentic, relevant problems used in the study lesson. There appeared to be a stronger motivation for students at all reading levels after they became familiar with the requirements of the study lesson. Lack of familiarity with this type of lesson meant that the students were unable to understand the requirements for information to complete the problem solving aspect of the lesson. As the students realized their need for the information in order to be able to solve the problems, their motivation to use the documents increased. In some instances, motivation was also influenced by the reading skill of the students, such as when the average readers became frustrated with the unfamiliar work, their motivation to continue seemed at times to falter. 178 The ability of teachers to predict where students would need assistance was also related to students? reading skill levels. Comparative data of the student novices coding and that of the teacher experts suggested that although the success with which we predicted when students would need assistance was high, this rate was lower for below average readers. However, the overall rate at which the teachers in the pilot study and the primary study anticipated student needs suggests that teachers may be capable of creating excerpts of source documents and hyperlinked scaffolds with a high degree of success. The findings from this study indicated that using hyperlinked scaffolds with historical source documents may be one way to help make understanding more available for students. I do not believe the findings suggested, however, that these scaffolds work alone to accomplish this goal. Several other factors seem to be important considerations, such as student motivation to complete the task of which the sources are a part and student familiarity with using technology and with the problem-based inquiry method. The findings also support the hypothesis that students? reading levels influence the use of source documents with hyperlinked scaffolding. 179 CHAPTER 5: SUMMARY, IMPLICATIONS, AND SUGGESTIONS FOR FURTHER STUDY Summary of findings I began this study with the question, ?How may students be supported in working with complex, text-based source documents as part of a problem-based historical inquiry lesson?? The focus of the research lesson was on the effectiveness of using hyperlinked scaffolding as one way of providing this support. In order to consider the effectiveness of this type of scaffolding I conducted a design experiment in which I worked with a teacher to design a lesson that asked students to use hyperlinked scaffolded source documents within a problem-based historical inquiry context. I sought to understand how students interacted with source documents that were presented in an online format, how students used the hyperlinked scaffolds as they analyzed the source documents, and how students used the scaffolds to develop problem solutions. As part of these considerations I looked at how differing literacy skill levels among students affected each of these considerations. Finally, I considered teachers? abilities to anticipate students? needs because without the ability to adequately anticipate students? needs it would be impossible to add scaffolds or effectively excerpt source documents. Student use of online documents. In this study several factors appeared to affect students? use of the online documents. My findings suggest that students? motivation played a role in the level of success students had in using the online source documents. 180 These findings suggest that motivation was affected in several ways throughout the lesson. Early in the study lesson, both the teacher and I noted that the students were highly motivated because of the prospect of using computers for part of the lesson. As the study lesson progressed, however, the reality of using complex source documents seemed to frustrate some students as the requirements of an authentic inquiry lesson became obvious. Lee and Clarke (2003) found that students in their study also became frustrated when strategies that had proven successful in the past failed to provide successful results in a technology rich learning environment. They went on to suggest that additional scaffolding may be needed to ensure these students experience success in online instructional activities. I found this to be true in this study as well because although many students? motivation waned as the reality of completing a difficult task competed with the excitement of using the technology, most students did seem to retain at least part of their initial enthusiasm if they seemed to believe they were likely to be successful. During the early part of the study lesson, students? perceptions of the online documents contributed to increased motivation as well. Because the additional text added by the hyperlinks was not initially apparent to the students, many indicated that they were confident in their ability to successfully engage in reading these source documents. Other students indicated that the hyperlinked scaffolds also added to their confidence and therefore their motivation. These students believed that because they could get assistance with the documents by using the hyperlinks the task of reading these documents would be easier. This combined with the students? perception of document 181 length and enthusiasm over computer use to promote greater motivation on the part of many students. While the students were initially motivated by the prospect of using the technology and their belief that they could be successful because the source documents did not appear intimidating, after they began working with the online documents the increased cognitive requirements of this type of instructional environment caused some students to develop a more realistic attitude regarding the difficulty of the work. Additional text included in the hyperlinked scaffolds and the difficulties involved in creating mental concept maps of complex information from several sources made the use of these documents much more difficult that many students originally believed. As students dealt with these difficulties some experienced decreased motivation. Many researchers have found differences in the performance of students with various reading levels as they worked in technology rich instructional environments (Milson, 2001; Niederhauser & Shapiro, 2003; Last, O?Donnell, & Kelly, 2001). Student interaction with source documents in this study suggests that there are distinct differences in the ways students with above average, average, and below average literacy skills interact with these types of documents. My findings suggest that these differences may closely resemble the differences seen when these students engage in the study of print sources within a PBHI instructional environment. I found that students could be placed on a continuum with above average readers reading strategically and for meaning on one end and below average readers using little strategy and failing to read for meaning on the other. Average readers tended to fall somewhere between these two extremes. These reading patterns are very similar to patterns seen in students reading any complex text, 182 suggesting that teachers may be able to structure the assistance they give students in one instructional environment on patterns that have been successful in other environments (Afflerbach & VanSledright, 2001; McConnell, 2003; Meyer, 2003; Niederhauser & Shapiro, 2003; VanSledright, 2002b). The findings suggest that student sometimes develop strategies that allow them to be successful working specifically in the online instructional environment. In some cases reading skill levels seemed to strongly influence the preferences students had for the ways they utilized the sources. Some of the preferences students developed were not beneficial as in the case of below average students using the hyperlinks in ways that prevented them from maintaining text cohesion and comprehending the text adequately. Other preferences, such as the manner in which the students would read the documents, electronically or on paper, and how they would make use of the technological affordances may be personal preferences with little influence on students? ability to effectively utilize the source documents presented in this way. All students found that they wanted to return to the documents during group decision-making because they felt they had not gathered enough information during the earlier parts of the lesson. This failure to use data-gathering scaffolds effectively when analyzing online documents has been seen by other researchers as well (Saye & Brush, 1999, 2002). Overall, the students and the teacher believed that the online format was a successful way for students to access these sources. While the increased cognitive requirements made the task more difficult, those elements that added to the cognitive load such as non-linear presentation of information also may have contributed to better understanding of the sources. Additionally, the increased confidence some students felt 183 about working in the hyperlink-scaffolded environment may have added to the motivation of the students. Students using scaffolds to analyze documents. In many ways this study reinforces what others have found about students working in technology rich environments. Studies have shown that students who begin with greater prior knowledge often achieve the best results when working in technology rich environments (Last, O?Donnell & Kelly, 2001; Neiderhauser & Shapiro, 2003). The finding of these researchers held true in my study as well and the students who were the most successful at using the scaffolds to help them solve problems were the ones that had begun the study with the greatest skills, especially reading skills. Like some other studies, I also found that there were clear differences in the ways students used online sources and hyperlinked scaffolding based on the students? reading skill levels (Milson, 2001). My findings suggest that while adding hyperlinked scaffolding may improve students? abilities to engage in the study of complex source documents, my findings suggest that improvements may be different for students with different skill levels. This does not suggest that each student must have differentiated scaffolds, but that different students may be expected to achieve benefits from the scaffolds based on the skills they begin the lesson with (Last, et al., 2001). In my study the below average readers often began the lesson with skills that were below the lowest level on VanSledright?s (2002b) continuum of reading strategies. These students could not be expected to begin using skills listed on the highest level of the continuum as a result of hyperlinked scaffolding, but their use of skills listed on the lowest level of this continuum would indicate a real improvement. Above average readers should also show 184 improvement by moving from one level to the next. Finally, if these students were able to utilize scaffolds to better comprehend and use source documents the teacher might be more willing to include these types of sources as a routine part of instruction. The cumulative effect of this additional reading would be that all students would likely improve their general literacy as well as their domain specific literacy (Cunninham & Stanovich, 1998). My findings suggest that both average and below average readers had trouble using some types of scaffolds. Below average readers seemed to have difficulty connecting information from the source document text and the historical background text. These difficulties contributed to the problems these students had with the metacognitive scaffolds and the individual data-gathering sheets. Average readers also experienced difficulties with the metacognitive scaffolds and the individual data-gathering sheets. Their difficulties, however, seemed to have a different origin from that of the below average readers. The difficulties the below average readers experienced are similar to those that other researchers have seen. The complexity of the source documents strained their limited literacy skills while the complexity of the hyperlinked scaffolding environment provided assistance in some cases and exacerbated the problem in other. From these findings it would seem that hyperlinked scaffolding may need to be combined with other strategies to help these students who are in need of the most assistance if they are going to successfully engage in the study of source documents with hyperlinked scaffolding. This does not, however, imply that the scaffolds that were in place were ineffective, but that other factors besides these scaffolds were important to student success and may be important in other situations as well. Students at all levels 185 experienced additional cognitive requirements that made the texts difficult for many of the students. The students had little experience with this type of text presentation and struggled to make sense of the non-linear pattern used to present the information. Overall the pattern of student use of the scaffolds was much, like their use of the online documents, dependent on their reading abilities. Motivation was a key factor in the response of the below average and average students to the hyperlinked scaffolds that were the focus of this study. All of the students, including the below average students were motivated at the beginning of the lesson because of the prospect of using the laptop computers. Both the below average and the average readers struggled with remaining motivated as they read complex documents and as they worked in an instructional format that was unfamiliar. The above average readers in this study seemed more able to alter the strategies they used in order to experience success, a trait suggested in other research as well (Hynd, et al, 2000; Last, et al. 2001; Lee & Clark, 2003; Milson, 2001). Last, et al. (2001) found distinct differences in the ways students they identified as ?low knowledge/low motivation? and ?low knowledge/high motivation? worked in a complex, technology rich learning environment. Their category of ?low knowledge/low motivation? is comparable to the below average reader category in my study, and the students in these groups exhibited many of the same characteristics, including a willingness to leave work undone if it became too difficult and an unwillingness to expend excessive amounts of energy to strive for a goal they did not believe was unattainable. The below average readers, however, lost some of their motivation as they 186 began to work with the source documents and realized they were experiencing many of the same frustrations they experienced with any other documents. Last, et al.?s ?low knowledge/high motivation? category was comparable to the average reader category in my study. These students were motivated to do well for some reason, but they did not have the knowledge or skills required to accomplish that goal. The frustration these researchers noted in this group is much like my findings for the average readers as they used the online source documents to solve the PBHI problems. The average students, on the other hand, lost some of their motivation as they realized that methods of study they were comfortable with were not being successful in this instructional environment and they had few alternatives to replace these ineffective strategies. The ability of teachers to predict where students might need assistance is a consideration in the use of hyperlinked scaffolding to assist students as they read complex source documents. My findings suggest that teachers may be successful a majority of the time when determining where students would struggle, but may be less successful at determining when the below average readers would need assistance. Several limitations to the study exist, and the findings from the study cannot be generalized beyond the class and in some instances within the class. The setup of the two classes used in the study did not allow for a study group and a control group, therefore, all of the finding are based on comparisons across groups, predominantly reading groups, within these two classes. The low participation rate of below average readers in relation to other groups affects any comparisons among these groups. Because the participants are skewed toward the above average and average readers in the class, the findings 187 cannot be generalized to the entire class. Although the study findings may not be generalized to a larger group, many of the findings provide indications of the ways students in the study used these hyperlinked documents. The factors that seemed to contribute to their successes and failures may inform other studies in order to better understand how to use technology rich environments to best serve students? needs. These findings lead to suggestions for further study and possible consideration by teachers and researchers whose student populations may be similar to the students that participated in this study. Implications This study has implications for understanding and improving the uses of hyperlinked scaffolding for students as they utilize historical source documents and, thereby, possibly improving students? abilities to engage in the study of complex, text- based source documents. Motivation. In this study, student motivation was influenced by a number of factors throughout the study lesson. At the beginning of the study lesson, students were positively motivated by the use of computers, perceived length of the online texts, and the possibility of assistance through the use of the hyperlinked scaffolds. Motivation did not, however, remain consistent throughout the lesson. As students found that the cognitive requirements of reading the online sources and working in the problem-based historical inquiry (PBHI) lesson were greater than they were used to, motivation for some students was negatively impacted. The lack of familiarity with the PBHI lesson format resulted in some students being motivated to perform in ways that were ineffective for this type of lesson. Students who were accustomed to receiving high scores for simply completing 188 work found themselves struggling to change their understanding of what was important in the lesson. Despite the apparent negative impact the cognitive requirement may have had on some students as they read and analyzed the sources and completed the data-gathering sheet, the decision making group activities that followed seemed to positively impact motivation for several reasons. Below average readers were able to contribute meaningfully to these group decisions and all of the students were motivated to improve the work they had done individually. Although it was frustrating for some students to realize their analysis of the sources was inadequate, several of the students indicated that they were encouraged to use source material more strategically in the future. These findings regarding motivation may suggest that students need instruction in the technology scaffolds if they are going to be successful in their use, and that if they are going to be successful in the a PBHI learning environment the teacher needs to make this type of instruction a routine aspect of the class instruction. Additionally, the authentic problems that are part of the PBHI learning environment may need to be introduced at the beginning of a lesson to provide motivation for all of the students as they begin the reading task. When these documents are accessed in an online format, issues of motivation can be affected both negatively and positively. Another implication from this study is the effect of apparent text length on students? motivation to engage in the use of that source. In my study students indicated that the appearance of a shorter document motivated them to attempt to read because they believed they had a better chance to be successful. When students receive sources that are excessively long they may become discouraged; 189 therefore, I believe the ability of hyperlinked scaffolds to provide scaffolding without obviously increasing the length should be considered as teachers determine the feasibility of using this method of assisting their students. Increased cognitive requirements. Researchers have found that both technology rich learning environments and PBHI instruction can substantially increase the cognitive requirements for students (Antonetti, et al., 2001; Britt, et al., 2000; Bosnan, 1988; Brush & Saye, 2001; Last, et al., 2001; Mayer, 2003; Reisslein, et al., 2004; Saye & Brush, 2004b; Spoehr & Spoehr, 1994). Use of online documents and hyperlinked scaffolding present information in ways that require students to construct their own meaning and PBHI requires them to apply these constructs to create defensible responses to authentic meaningful problems. These are difficult tasks that often necessitate students working outside of the traditional ?doing school? mentality (Tovani, 2000). Students in these instructional environments must develop a deep understanding of the sources that they will use to consider the problems, consider why these sources are important, determine the value they have for the task at hand, and understand how they can use the information they are gathering in the early parts of these types of lessons as they begin to consider the problems that are the focus of the lesson. Using source documents in this way has been shown to produce deeper, longer-lasting knowledge, but students often need considerable encouragement and assistance in order to operate effectively in this environments (Hynd, et al., 2000; Saye & Brush, 1999, 2002, 2004b). All of the students in my study, especially the below average and average readers commented on the difficulty of working with this text and of creating meaning through its use. 190 Adding hyperlinks seemed to provide benefits for the students reading these source documents but added substantially to the cognitive requirements of the lesson. This implies that the scaffolds might be much more effective if some aspects of the cognitive load that can be reduced. Students will most likely always find that creating understanding from multiple sources and developing defensible responses to problems is difficult, but struggling with these difficulties has been shown to increase student learning (Levstik & Barton, 2005; Rossi, 1995; VanSledright, 2002a, 2002b; Wineburg, 1991, 2001). The findings from this study and others suggest that this is most likely to happen when students are not struggling with other aspects of reading the sources or completing the lesson. Some of the aspects of the lesson that were sources of increased cognitive load for the students in this study that might be improved and thereby improve student performance in other areas are lack of familiarity with the lesson format and lack of familiarity in working effectively in the online format. All of the students in my study could have benefited from direct instruction and practice in effective use of online resources and hyperlinked documents. This would, however, require that teachers understand these strategies, know how to teach them, and be willing to take the time to teach and allow students the time to practice these strategies. Another suggestion from the study?s findings is the need for the students to understand the requirements of the lesson within which the source documents are being used. This implies more than the teacher simply telling the students what is expected. In this study both the study teacher and I did this on numerous occasions throughout the individual document analysis part of the lesson without notable effect. For students to truly understand the requirements of a problem-based historical inquiry lesson, they must 191 be familiar with this strategy through routine use. By the end of my study lesson, many students indicated that if they did this type of activity again they would know what to expect and how to ?do the reading.? At that point these students were motivated and understood how to correct some of cognitive requirement problems on their own, but for the purposes of this lesson it was too late for them to use the documents more successfully. The implication from my findings as well as from other studies is that if students are going to be motivated to read complex source documents and make sense of these sources in meaningful ways they must be motivated by the lesson within which it is situated, and this is most likely to be successful when they are familiar with the instructional strategy through experience (Levstik & Barton, 2005; Thornton, 1998; VanSledright, 2002a, 2002b; Wineburg, 1991, 2001). Teachers must use these strategies on a regular basis if students are going to see the value of putting forth the effort and if they are going to be comfortable with the learning strategies involved. One finding from this study suggests that another way of relieving some of the strain of students using hyperlinked online source documents is to allow for flexibility in the ways students read these documents. An obvious example of this is the decision of whether to allow students the choice of which medium they want to use to access the text, online or paper. If students are allowed to make a choice regarding the method of reading the text and accessing the technological affordances, care must be taken that they do not ignore the hyperlinked scaffolds as the below average readers sometimes did in my study. Other issues of flexibility involve students? use of the hyperlinks as they read the overall source documents. In this study the above average readers developed several 192 strategies for effectively utilizing the hyperlinked scaffolds while maintaining the cohesiveness of the source document. While most students ignored instructions regarding the best way to read the documents and use the hyperlinks, many students developed effective strategies of their own. The below average readers and some of the average readers, however, failed to do this and even developed reading strategies that were counterproductive. This suggests that while some flexibility may be good, teachers likely need to provide students with specific boundaries for that flexibility. This suggests that teachers should make modeling a routine aspect of their class in order to allow students to take full advantage of all of the scaffolds available. Influence of student literacy skills. My findings suggest that students? reading skill levels affect not only their ability to read the online sources and the hyperlinked scaffolds, but also the ways students interact with source documents and use them in the PBHI instructional environment. Most of this study?s data was considered using student reading skill level as a possible factor in the findings. In many instances where there were clear differences in the ways above average and below average reading students performed, while average readers performed between the two extremes with characteristics of both. In one particular instance, however, students in the average reader group were unique in their performance. Analysis of the data on students? use of the hyperlinked scaffolds and the online sources to assist in problem solving indicated that the average readers experienced considerable levels of frustration. I believe their response to this aspect of the lesson could be compared to the ?low knowledge/high motivation? group described by Last, O?Donnell, and Kelly (2003). I hypothesized that these students experienced greater cognitive load in this aspect of the lesson because of 193 their desire to achieve but their belief that they were not able to do so because of the unfamiliarity of the lesson format and the text analysis requirements. Ensuring that above average readers are challenged at an appropriate level and below average readers receive appropriate levels of assistance are necessary in order to successfully include all students in the instructional environment. However, I suggest that the average students in a technology rich history class using PBHI instructional methods present a unique challenge and helping them overcome the cognitive overload many of them experienced in this study is imperative. Teachers? ability to predict areas where students will have difficulties. If teachers are going to include historical source documents as part of problem-based historical inquiry instruction, it is imperative that they are able to anticipate the areas where students might need assistance and how to create effective documents and document sets from all of the historical documents they have to choose from. This study suggests that in most cases the teachers were successful at anticipating where students would find source documents difficult. They were less successful at anticipating when the below average readers would need assistance however. In the pilot study the teachers were also effective at anticipating areas where students would have difficulties. The similarities between the findings from these teachers suggests that teachers with a variety of experience and training levels may successfully anticipate student difficulties when dealing with complex source documents. This may allow them to successfully select sources and excerpts as well as create scaffolds that support students? routine use of source documents. 194 Effectiveness of hyperlinked scaffolds. The overall impression of the students and the teacher regarding the effectiveness of including hyperlinked scaffolds to the online source documents was favorable. Both teachers and students indicated that they believed these scaffolds helped the students perform at levels above where they would have performed without this assistance. As discussed previously the findings from the study suggest that the scaffolds were not equally effective. Additionally, because I was unable to have a control group in this study I cannot claim that these findings indicate a measurable improvement based on the design intervention. However, based on the data that was available from the study, recommendations of all of the students, the recommendation of the study teacher, and the findings from other researchers there is reason to believe that the use of hyperlinked scaffolding may be effective in assisting students as they engage in the study of complex, text-based source documents as part of a problem-based historical inquiry lesson (Britt, et al., 2000; Brush & Saye, 2001; Chang, et al., 2001; Hannafin, et al., 1997; Larkin, 2002; Last, et al., 2001; Lee & Clark, 2003; Mayer, 2003 McLoughlin, 1999; Milson, 2001; Saye & Brush, 1999, 2002, 2004b; Shapiro, 1999; Stahl, et al., 2003; Stephens, et al., 2005). Suggestions for further study Students with different reading skill levels. Additional study into the specific ways students with differing reading levels work with online source documents and with hyperlinked scaffolds would allow for greater understanding of students? use of these texts. Although the original think-aloud format of the interviews might have begun to provide this information, after completing this study I do not believe think-alouds alone would have provided the focused information that is needed in order to understand the 195 differences in the ways these groups use online source documents and hyperlinked scaffolding to analyze source documents and develop problem solutions. I believe reading skill level must be the focus of future studies into students? use of hyperlinked documents in order create technology rich instructional environments that address the needs of all students (Milson, 2001; Niederhauser & Shapiro, 2003; Last, O?Donnell, & Kelly, 2001). An aspect of further study should be the differing cognitive needs of students in the various reading levels. Although studies have considered the cognitive requirements of students as they work in technology rich environments and utilize online resources, my findings suggest that the cognitive needs for students with different reading skill levels can in some instances be very different, not similarly different by varying degrees (Last, et al., 2001; Lee & Clark, 2003). Study needs to continue into ways to motivate students with varying reading skill levels as they work with online sources and hyperlinked scaffolds. The students in this study were motivated at the beginning of the lesson by many of the same things, especially use of the computers and the novelty of the PBHI lesson. The below average readers and the average readers began to lose some of their initial enthusiasm and motivation as they worked on the lesson, although some became motivated again by the end of the lesson. A substantial body of research suggests that authentic, problem-based inquiry is motivational for students, but it would be beneficial for researchers and teachers to understand how those motivational factors affect students differently based on their reading skill levels (Dewey, 1938; Levstik & Barton, 2005; VanSickle, 1996; Wineburg, 2001). 196 How does student perception of a document length affect motivation? After observing students in this study and analyzing student interviews I suggest that teachers would benefit from such knowledge. The findings from this study suggest that for students at every level, the length they perceive an online document to be was a factor in how motivated students were to begin the analysis of those documents. This was not a focus in my study, although I suggest that a clearer understanding of the length at which students begin to experience a drop in motivation as a result of the length and perceived difficulty of a document would guide teachers in making choices regarding the use of online source documents. Additionally, an investigation into how long an online document and connected links can be before students begin to become frustrated would be beneficial as well. This would contribute to an understanding of the effects of domain specific complexity on students? motivation to engage source documents as well. Conclusion Can online, hyperlinked scaffolded documents be used to support students as they work with complex, text-based source documents as part of a problem-based historical inquiry lesson? The findings from this study suggest that the use of these types of scaffolded documents may be useful in this endeavor. However, these findings also suggest that many other factors should be considered because text presentation and scaffolding are only part of the larger classroom experiences involved in teaching any lesson. Finally, the findings from this study suggest that any type of support should be considered based on the effect it has for students with varying degrees of reading proficiency. 197 If history education is going to be used to help achieve the goals of preparing students to be citizens in a pluralistic, democratic society, the instructional methods used must provide students with practice using those habits of thought and citizenship skills necessary for this role. Authentic, inquiry-based, historical instruction has been indicated as one way to do this. The findings from this study suggest that technology may be able to support the use of complex source documents that are a vital part of this type of study. 198 REFERENCES Abbott, J. (1997). To be intelligent. Educational Leadership, 54(6), 6-10. Afflerbach, P. & VanSledright, B. (2001). Hath! Doth! What? Middle graders reading innovative history text. Journal of Adolescent & Adult Literacy, 44(8), 696-707. Allington, R. L. (2002). You can?t learn much from books you can?t read. Educational Leadership, 60(3), 16-19. Antonietti, A., Imperio, E., Rasi, C., & Sacco, M. (2001). Virtual reality and hypermedia in learning to use a turning lathe. Journal of Computer Assisted Learning, 17, 142-155. Atkinson, R. K., Renkl, A., & Merrill, M. M. (2003). Transitioning from studying examples to solving problems: Effects of self-explanation prompts and fading worked-out steps. Journal of Educational Psychology, 95(4), 774-783. Bain, R. B. (2000). Into the breach: Using research and theory to shape history instruction. In P. Seixas & S. Winburg (Eds.), Knowing, teaching & learning history (pp. 331-352). New York: New York University Press. Barton, K. C. & Levstik, L. S. (2003). Why don?t more history teachers engage students in interpretation? Social Education, 67(6), 388-61. Bean, T. W. & Ericson, B. O. (1989). Text previews and three level study guides for content area critical reasoning. Journal of Reading, 32(4), 337-41. 199 Beaufils, A. (2000). Tools and strategies for searching in hypermedia environments. Journal of Computer Assisted Learning, 16, 114-124. Beck, I. L. & McKeown, M. G. (1991). Research directions: Social studies texts are hard to understand: Mediating some of the difficulties. Language Arts, 68, 482-90. Beck, I. L. & McKeown, M. G. (2002). Questioning the author: Making sense of social studies. Educational Leadership, 60(3), 44-7. Beck, I. L., McKeown, M. G., Hamilton, R. L., & Kucan, L. (1997). Questioning the author: An approach to enhancing student engagement with text. Neward, DE: International Reading Association. Beck, I. L., McKeown, M. G., & Worthy, J. (1995). Giving a text voice can improve students? understanding. Reading Research Quarterly, 30(2), 220-238. Berson, M. J., Lee, J. K., & Stuckart, D. W. (2001). Promise and practice of computer technologies in the social studies: A critical analysis. In W. B. Stanley (Ed.), Critical issues in social studies research for the 21 st century (pp. 209-229). Greenwich, CN: Information Age. Billman, L. W. (2002). Aren?t these books for little kids? Educational Leadership, 60(3), 48-51. Bogdan, R. C. & Biklen, S. K. (2003). Qualitative Research for Education, 4 th Edition. Boston: Allyn-Bacon. Britt, M. A., Perfetti, C. A., Van Dyke, J. A., & Gabrys G. (2000). The sourcer?s apprentice: A tool for document-supported history instruction. In P. Seixas & S. Winburg (Eds.), Knowing, teaching & learning history (pp. 437-470). New York: New York University Press. 200 Brosnan, M. J. (1998). The impact of computer anxiety and self-efficacy upon performance. Journal of Computer Assisted Learning, 14, 223-234. Brown, A. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141-178. Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educational Researcher, 13, 32-41. Brush, T. & Saye, J. W. (2001). The use of embedded scaffolds with hypermedia- supported student-centered learning. Journal of Educational Multimedia and Hypermedia, 10(4), 333-356. Burke, J. (2002). The Internet reader: We can help students read Internet materials by teaching questioning strategies and by creating useful digital resources. Educational Leadership, 60(3), 38-42. Chang, C. Y. (2001). A problem-solving based computer-assisted tutorial for the earth sciences. Journal of Computer Assisted Learning, 17, 263-274. Chang, K. E., Sung, Y. T., & Chen, S. F. (2001). Learning through computer-based concept mapping with scaffolding aid. Journal of Computer Assisted Learning, 17, 21-33. Chou, S. W. & Liu, C. H. (2005). Learning effectiveness in a Web-based virtual learning environment: A learner control perspective. Journal of computer Assisted Learning, 21, 65-76. Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in education research. Educational Researcher, 32(1), 9-13. 201 Cohen, E. G. (1992). Restructuring the classroom: Conditions for productive small groups. Washington, D. C.: Office of Educational Research and Improvement. (ERIC Document Reproduction Service No. ED363952) Cousin, P. T. (1989). Content area textbooks: Friends or foes? Washington, D. C.: Office of Educational Research and Improvement. (ERIC Document Reproduction Service No. ED321249) Creswell, J. W. (1997). Qualitative inquiry and research design: Choosing among four traditions. Sage Publications: Thousand Oaks, CA. Cunningham, A. E. & Stanovich, K. E. (1998). What reading does for the mind. American Educator, 22(1-2), 8-15. Denzin, N. K. & Lincoln, Y. S. (Eds.). (2003). Collecting and interpreting qualitative materials (2 nd Edition). Thousand Oaks, CA: Sage Publications. Design-Based Research Collective (2003). Design-based research: An emerging paradigm for education inquiry. Educational Researcher, 32(1), 5-8. Dewey, J. (1933). How We Think. Boston: D. C. Heath and Company. Dewey, J. (1938). Experience and education. New York: The Macmillan Company. Dochy, F., Segers, M., Van den Bossche, P., & Gijbels, D. (2003). Effects of problem- based learning: a meta-analysis. Learning and Instruction, 13(5), 533-568. Donahue, P. L., Daane, M. C. & Grigg, W. S. (2003). The nation?s reading report card: Reading highlights 2003. Washington, D. C.: Institute of Educational Sciences. National Center for Educational Statistics. (NCES 2004-452) 202 Downey, M. T. & Levstik, L. S. (1991). Teaching & learning history. In J. P. Shaver (Ed.), Handbook of research in social studies teaching & learning (pp. 400-410). New York: Macmillan Publishing Co. Elliot, J. B. & Dupuis, M. M. (2002). Young adult literature in the classroom: Reading it, teaching it, loving it. Newark, DE: International Reading Association. Fallace, T. & Neem, J. N. (2005). Historiographical thinking: Towards a new approach to preparing history teacher. Theory and Research in Social Education, 33(3), 329- 346. Fry, E. (2002). Readability versus leveling: Both of these procedures can help teachers select books for readers at different stages. The Reading Teacher, 56(3), 286-291. Gamoran, A. & Nystrand, M. (1992). Taking students seriously. In F. M. Newmann (Ed.), Student engagement and achievement in American secondary schools (pp. 40-61). New York: Teachers College, Columbia University. George, M. A. & Stix, A. (2000). Using multilevel young adult literature in middle school American Studies. Social Studies, 91(1), 25-31. Goldman, S. R. & Rakestraw, J. A., Jr. (2000). Structural aspects of constructing meaning from text. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of Reading Research (Volume III). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers. Grady, K. (2002). Adolescent literacy and content area reading. Bloomington, IN: Clearinghouse on Reading English and Communication. (ERIC Document Reproduction Service No. 469930) 203 Grant, S. G. & Gradwell, J. M. (2005). The sources are many: Exploring history teachers? selection of classroom texts. Theory and Research in Social Education, 33(2), 244-265. Gunning, T. G. (2003). The role of readability in today?s classrooms. Topics in Language Disorders, 23(3), 175-189. Hannafin, M. J., Hill, J. R., & Land, S. M. (1997). Student-centered learning and interactive multimedia: Status, issues, and implications. Contemporary Education, 68(2), 94-97. Hannafin, M., Land, S., & Oliver, K. (1999). Open learning environments: Foundations, methods, and models. In C. Reigeluth (Ed.), Instructional design theories and models (Vol. II). Mahway, NJ: Erlbaum. Hicks, D., Doolittle, P. E., & Ewing, E. T. (2004). The SCIM-C strategy: Expert historians, historical inquiry, and multimedia. Social Education, 68(3), 221-227. Hirsch, E. D. (2003). Insights into the fourth-grade slump and the nation?s stagnant comprehension scores. American Educator, 27(1), 10-13, 16-22, 28-29, 48. Hoffer, T. B. & Gamoran, A. (1993). Effects of instructional differences among ability groups on student achievement in middle-school science and mathematics. Washington, D. C.: National Science Foundation, Office of Educational Research and Improvement. (ERIC Document Reproduction Service No. ED363 509) Holt, T. (1990). Thinking historically: Narrative, imagination, and understanding. New York: College Entrance Examination Board. Hynd, C., Hubbard, B., Holschuh, J., Reinking, D., & Jacobson, M. (2000). Reading like a historian: Critical reading of multiple texts in a hypertext environment. Paper 204 presented at the annual meeting of the American Educational Research Association, New Orleans, LA, April 2000. James, M., Black, P., & McCormick, R. (2003, April). Deepening capacity through innovative research design: Researching learning how to learn in classrooms, schools and networks. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL. Jenkins, J. R., Fuchs, L. S., van den Broek, P., Espin, C., & Deno, S. L. (2003). Sources of individual differences in reading comprehension and reading fluency [Electronic version]. Journal of Educational Psychology, 95(4). Josten, D. (1996). Students rehashing historical decisions ? and loving it? Journal of Adolescent & Adult Literacy, 39(7), 566-574. Keselman, A. (2003). Supporting inquiry learning by promoting normative understanding of multivariable causality. Journal of Research in Science Teaching, 40(9), 898- 921. Khong, Y. G. (1992). Analogies at War. Princeton, New Jersey: Princeton University Press. Kohlberg, L. (1976). Moral Reasoning. In D. Purpel & K. Ryan (Eds.), Moral education?It comes with the territory (pp. 176-195). Berkeley, CA: McCutchan Publishing Corporation. Lapp, M.S., Grigg, W. S., & Tay-Lim, B. S.-H. (2002). The Nation?s Report Card: U. S. History 2001. Washington, D. C.: Office of Educational Research and Improvement. National Center for Education Statistics. (NCES 2002-483) 205 Larkin, M. (2002). Using scaffolded instruction to optimize learning. Arlington, VA: Clearinghouse on Disabilities and Gifted Education. (ERIC Document Reproduction Service No. ED4474301) Last, D. A, O?Donnell, A. M., & Kelly, A. E. (2001). The effects of prior knowledge and goal strength on the use of hypertext. Journal of Educational Multimedia and Hypermedia, 10(1), 3-25. Lather, P. (1986). Issues of validity in openly ideological research: Between a rock and a soft place. Interchange, 17(4), 63-84. Lee, J. K. & Clarke, W. G. (2003). High school social studies students? uses of online historical documents related to the Cuban Missile Crisis [Electronic version]. The Journal of Interactive Online Learning, 2(1). Lee, P. & Ashby, R. (2000). Progression in historical understanding among students ages 7-14. In P. Seixas & S. Winburg (Eds.), Knowing, teaching & learning history (pp. 199-222). New York: New York University Press. Levstik L. S. & Barton, K. C. (2001). Doing history: Investigating with children in elementary and middle grades. Mahwah, NJ: Lawrence Erlbaum Associates, Publishers. Levstik L. S. & Barton, K. C. (2003). Doing history: Investigating with children in elementary and middle grades (2 nd Edition). Mahwah, NJ: Lawrence Erlbaum Associates, Publishers. Levstik L. S. & Barton, K. C. (2005). Doing history: Investigating with children in elementary and middle grades (3 rd Edition). Mahwah, New Jersey: Lawrence Erlbaum Associates, Publishers. 206 Lin, X. (1994, February). Metacognition: Implications for research in hypermedia-based learning environments. Paper presented at the 1994 National Convention of the Association for Educational Communications and Technology, Nashville, TN. Lowenthal, D. (2000). Dilemmas and delight of learning history. In P. Seixas & S. Winburg (Eds.), Knowing, teaching & learning history (pp. 68-82). New York: New York University Press. Manifold, M. C. (1997). Picture books as a social studies resource in the elementary school classroom. Washington, D. C.: Office of Educational Research and Improvement. (ERIC Document Reproduction Service No. ED412168) Marks, H. M., Newmann, F. M., & Gamoran, A. (1996). Does authentic pedagogy increase student achievement? In F. M. Newmann & Associates (Eds.), Authentic achievement: Restructuring schools for intellectual quality (pp. 49-73). San Francisco: Jossey-Bass Publishers. Marshall, C. & Rossman, G. B. (1999). Designing Qualitative Research. London: Sage Publications. Martorella, P. H. (1991). Knowledge & concept development in social studies. In J. P. Shaver (Ed.), Handbook ob research in social studies teaching & learning (pp. 370-384). New York: Macmillan Publishing Co. Mayer, R. E. (2003). The promise of multimedia learning: Using the same instructional design methods across different media. Learning and Instruction, 13, 125-139. McConnell, D. L. (2003, May). Reading intervention for struggling adolescents. Paper presented at the 48 th Annual Meeting of the International Reading Association, Orlando, FL. 207 Mcdaniel, C. (2004). Critical literacy: A questioning stance and the possibility for change. The Reading Teacher, 57(5), 472-481. McLoughlin, C. (1999, June). Scaffolding: Applications to learning in technology supported environments. Paper presented at the 11 th World conference on Educational Multimedia and Hypermedia & World Conference on Educational Telecommunications, Seattle, WA. McLoughlin, C. & Hollingworth, R. (2002, June). Bridge over troubled water: Creating effective online support for the metacognitive aspects of problem solving. Paper presented at the 14 th World Conference on Educational Multimedia, Hypermedia, and Telecommunications, Denver, CO. Meyer, B. J. F. (2003). Text coherence and readability. Topics in Language Disorders, 23(3), 204-224. McTighe, J., Seif, E., & Wiggins, G. (2004). You can teach for meaning: Teaching for meaning is an engaging idea, but many teachers find it problematic in this age of mandates and standardized tests. Educational Leadership, 62,(1), 26-30. Milson, A. J. (2001, November). Engaging students in historical inquiry using Internet resources. Paper presented at the 81 st Annual Meeting of the National Council for the Social Studies, Washington, D. C. Nash, G., Crabtree, C., & Dunn, R. (1997). History on trial: Culture wars and the teaching of the past. New York: Knopf. National Center for History in the Schools, UCLA (2005). Overview of standards in historical thinking. Retrieved April 15, 2005, from http://nchs.ucla.edu/standards/thinking5-12.html. 208 National Council for the Social Studies (1994). Expectations of excellence: Curriculum standards for social studies. Washington, D. C.: NCSS Publications. Nelson, L. R. & Nelson, R. A. (1999). Learning history through children?s literature. Washington, D.C.: Office of Educational Research and Improvement. (ERIC Document Reproduction Service No. ED435586) Newmann, F. M. (Ed.). (1992). Student Engagement and Achievement in American Secondary Schools. New York: Teachers College, Columbia University. Newmann, F. M. & Oliver, D. (1970). Clarifying Public Controversy: An Approach to Teaching Social Studies. Boston: Little Brown. Newmann, F. M. & Wehlage, G. G. (1995). Successful School Restructuring: A Report to the Public and Educators. Washington, D.C.: Center on Organization and Restructuring of Schools. Newmann, F. M., Wehlage, G. G. & Lamborn, S. D. (1992). The significance and sources of student engagement. In F. M. Newmann, (Ed.), Student engagement and achievement in American secondary schools. New York: Teachers College, Columbia University. Niederhauser, D. S. & Shapiro, A. (2003, April). Learner variables associated with reading and learning in a hypertext environment. Paper presented at the Annual Meeting of the American Educational Research Association, Chicago, IL. Oakes, J., Gamoran, A., & Page, R. N. (1992). Curriculum differentiation: Opportunities, outcomes, and meanings. In P. W. Jackson (Ed.), Handbook of research on curriculum (570-608). New York: Macmillan Publishing Company. 209 Ogawa, M. (2000, November). Sixth grade students? development of historical perspective: World War II and the atomic bombing of Hiroshima and Nagasaki. Paper presented at the 80 th Annual Meeting of the National Council for the Social Studies. San Antonio, TX. Onosko, J. J. (1991). Barriers to the promotion of higher order thinking in social studies. Washington, D. C.: Office of Educational Research and Improvement. (ERIC Document Reproduction Service No. ED340640) Otten, E. H. (1998). Using primary sources in the primary grades. Washington, D. C.: Office of Educational Research and Improvement. (ERIC Document Reproduction Service No. ED419773) Pang, E. S., Mauka, A., Bernhardt, E. B., & Kamil, M. L. (2003). Teaching reading. Brussels, Belgium: International Academy of Education. (ERIC Reproduction Service No. ED481186) Parker, W. C. (2001a). Educating democratic citizens: A broad view [Electronic version]. Education into Practice, 40(1). Parker, W. C. (2001b). Toward enlightened political engagement. In W. B. Stanely, (Ed.), Critical Issues in social Studies Research for the 21 st Century (pp. 97-117). Greenwich, CT: Information Age. Parker, W. C., Mueller, M., & Wendling, L. (1989). Critical reasoning on civic issues. Theory and Research in Social Education, (17)1, 7-32. Patton, M. Q. (1987). How to Use Qualitative Methods in Evaluation. Newbury Park, CA: Sage Publications. 210 Paxton, R. J. (1999). A deafening silence: History textbooks and the students who read them. Review of Educational Research, 69(3), 315-339. Paxton, R. J. (2002). The influence of author visibility on high school students solving a historical problem. Cognition & Instruction, 20(2), 197-249. Penuel, W. R., Means, B., & Simkins, M. (2000). The multimedia challenge. Educational Leadership, 58,(2), 34-38. Perkins-Gough, D. (2002). RAND report on reading comprehension. Educational Leadership, 60(3), 38-42. Reisslein, J., Atkinson, R. K., & Reisslein, M. (2004, October). Exploring the presentation and format of help in a computer-based electrical engineering learning environment. Paper presented at the Association for Educational Communications and Technology, Chicago, IL. Renzulli, J. S., Gentry, M., & Reis, S. M. (2004). A time and a place for authentic learning [Electronic version]. Educational Leadership, 62(1). Rosenblatt, L. M. (1969). Towards a transactional theory of reading. Journal of Reading Behavior, 1(1), 31-49. Rosenblatt, L. M. (1985). Viewpoints: Transaction versus interaction ? A terminological rescue operation. Research in Teaching of English, 19(1), 96-107. Rosenzweig, R. (2000). How Americans use and think about the past: Implications from a national survey for the teaching of history. In P. Seixas & S. Winburg (Eds.), Knowing, teaching & learning history (pp. 262-283). New York: New York University Press. 211 Rossi, J. A. (1995). In-depth study in an issues-oriented social studies classroom. Theory and Research in Social Education, 23(2), 88-120. Rossi, J. A. & Pace C. M. (1998). Issues-centered instruction with low achieving high school students: The dilemmas of two teachers. Theory and Research in Social Education, 26(3), 380-409. Savery, J. R. & Duffy, T. M. (1995). Problem based learning: An instructional model and its constructivist framework. Educational Technology, 35(5), 31-38. Saye, J. W. & Brush, T. (1999). Student engagement with social issues in a multimedia- supported environment. Theory and Research in Social Education, 27(4), 472- 504. Saye, J. W. & Brush, T. (2002). Scaffolding critical reasoning about history and social issues in multimedia-supported learning environments. Educational Technology Research and Development, 50(3), 77-96. Saye, J. W. & Brush, T. (2004a). Promoting civic competence through problem-based history learning experiments [Electronic version]. In G. E. Hamot, J. J. Patrick & R. S. Leming (Eds.), Civic learning in teacher education: International perspectives on education for democracy in the preparation of teachers (pp. 123- 145). Saye, J. W. & Brush, T. (2004b). Scaffolding problem-based teaching in a traditional social studies classroom. Theory and Research in Social Education, 32(3), 349- 378. 212 Sexias, P. (1994). Students? understanding of historical significance. Theory and Research in Social Education, 22(3), 281-304. Seixas, P. (2000). Schweigen! die Kinder! or, does postmodern history have a place in the schools? In P. Seixas & S. Winburg (Eds.), Knowing, teaching & learning history (pp. 19-37). New York: New York University Press. Shapiro, A. M. (1999). The relevance of hierarchies to learning biology from hypertext. The Journal of the Learning Sciences, 8(2), 215-243. Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of education design studies. Educational Researcher, 32(1), 25-28. Shaver, J. P. (1992). Rationales for issues-centered social studies education. Social Studies, 83(3), 95-99. Shaver, J. P., Davis, O. L., & Helburn, S. W. (1979). An interpretive report on the status of pre-collegiate social studies education based on three NSF-funded studies. Washington, D. C.: National Science Foundation. Sonmez, D. & Lee, H. (2003). Problem-based learning in science. Columbus, OH: Clearinghouse for Science, Mathematics, and Environmental Education: (ERIC Reproduction Service No. ED482714) Spoehr, K. T. & Spoehr, L. W. (1994). Learning to think historically. Educational Psychologist, 29(2), 71-77. Stahl, S. A. (2003). Vocabulary and readability: How knowing word meaning affects comprehension. Topics in Language Disorders, 23(3), 241-247. 213 Stahl, S. A., Hynd, C. R., Britton, B. K., McNish, M. M., & Bosquet, D. (1996). What happens when students read multiple source documents in history? Reading Research Quarterly, 31(4), 430-456. Stearns, P. M. (2000). Getting specific about training in historical analysis: A case study in world history. In P. Seixas & S. Winburg (Eds.), Knowing, teaching & learning history (pp. 419-436). New York: New York University Press. Stephens, R. P., Lehr, J. L., Thorp, D. B., Ewing, E. T., & Hicks, D. (2005). Using technology to teach historical understanding: The digital history reader brings the possibilities of new technology to the history classroom. Social Education, 69(3), 151-155. Stevens, R. J., Slavin, R. E., & Famish, A. M. (1991). The effects of cooperative learning and direct instruction in reading comprehension strategies on main idea identification. Journal of Educational Psychology, 83(1). Strong, R. W., Silver, H. F., Perini, M. J. , & Tuculescu, G. M. (2002). Reading for academic success: Powerful strategies for struggling, average, and advanced readers, Grades 7-12. Thousand Oaks, CA: Corwin Press, Inc. Taylor, S. (1999). Better learning through better thinking: Developing students? metacognitive abilities. Journal of College Reading and Learning, 30(1), 34-45. Thornton, S. J. (1998, April). The persistent problem of method in social studies teaching. Paper presented at the Annual Meeting of the American Education Research Association, San Diego, CA. 214 Thornton, S. J. (1991). Teacher as curricular-instructional gatekeeper in social studies. In J. P. Shaver (Ed.), Handbook of research on social studies teaching and learning (pp. 237-248). New York: Macmillan. Tovani, C. (2000). I Read It, But I Don?t Get It: Comprehension Strategies for Adolescent Readers. Portland, ME: Stenhouse Publishers. Tunnell, M. O. & Ammon, R. (1996). The story of ourselves: Fostering multiple historical perspectives. Social Education, 60(4), 212-215. Unsworth, L. (1999). Developing critical understanding of the specialized language of school science and history texts: A functional grammatical perspective. Journal of Adolescent & Adult Literacy, 42(7), 508-521. VanFossen, P. J. & Shiveley, J. M. (2003). A content analysis of Internet sessions presented at the National Council for Social Studies Annual Meeting, 1995-2002. Theory and Research in Social Education, 31(4), 502-521. VanSickle, R. L. (1996). Questions of motivation for achievement in social studies. In B. G. Mausialas & R. F. Allen (Eds.), Crucial Issues in Teaching Social Studies, K- 12 (pp. 81-105). New York: Wadsworth Publishing Company. VanSickle, R. L. & Hoge, J. D. (1991). Higher cognitive thinking skills in social studies: Concepts and critiques. Theory and Research in Social Education, 19, 152-172. VanSledright, B. A. (2002a). Fifth graders investigating history in the classroom: Results from a reasearcher-practitioner design experiment. The Elementary School Journal, 103(2), 131-220. VanSledright, B. A. (2002b). In Search of America?s Past: Learning To Read History in Elementary School. New York: Teachers College Press. 215 VanSledright, B. A. (2004). What does it mean to think historically?and how do you teach it? Social Education, 68(3), 230-235. Voss, J. F. & Wiley, J. (2000). A case study of developing historical understanding via instruction: The importance of integrating text components and constructing arguments. In P. Seixas & S. Winburg (Eds.), Knowing, teaching & learning history (pp. 375-389). New York: New York University Press. Vygotsky, L. S. (1978). Mind in society: the development of higher psychological processes. Cambridge, MA: Harvard University Press. Wade, R. C. (1993). Content analysis of social studies textbooks: A review of ten years of research. Theory and Research in Social Education, 21(3), 232-256. Wehlage, G. G., Newmann, F. M., & Secada, W. G. (1996). Standards for authentic achievement and pedagogy. In F. M. Newmann (Ed.), Authentic Achievement: Restructuring Schools for Intellectual Quality (pp. 21-48). San Fransisco: Jossey- Bass publishers. Wehlage, G. G. & Smith, G. A. (1992). Chapter in F. M. Newmann (Ed.), Student engagement and achievement in American secondary schools (92-118). New York: Teachers College, Columbia University. Weinstock, M. P. (2005). Cognitive bases for effective participation in democratic institutions: Argument skill and juror reasoning. Theory and Research in Social Education, 33(1), 73-102. Wenglinsky, H. (2004). Facts or critical thinking skills? What NAEP results say [Electronic version]. Educational Leadership, 63(1). 216 Wiggins, G. (1998). An exchange of views on ?semantics, psychometrics, and assessment reform: A close look at ?authentic assessments.? Educational Researcher, 27(1), 19-21. Wiedenbeck, S., Zavala, J. A., & Nawyn, J. (2000). An activity-based analysis of hands- on practice methods. Journal of Computer Assisted Learning, 16, 358-365. Windschitl, M. (2002). Framing constructivism in practice as the negotiation of dilemmas: An analysis of the conceptual, pedagogical, cultural, and political challenges facing teachers. Review of Educational Research, 72(2), 131-175. Wineburg, S. S. (1991a). Historical problem solving: A study of the cognitive processes used in the evaluation of documentary and pictorial evidence. Journal of Educational Psychology, 83(1). Wineburg. S. S. (1991b). On the reading of historical texts: Notes on the breach between school and academy. American Educational Research Journal, 28(3), 495-519. Wineburg, S. S. (1999). Historical thinking and other unnatural acts. Phi Delta Kappan, 80(7), 488-499. Wineburg, S. (2001). Historical thinking and other unnatural acts: Charting the future of teaching the past. Philadelphia: Temple University Press. Wineburg, S. & Martin, D. (2004). Reading and rewriting history [Electronic version]. Educational Leadership, 62(1). Wolcott, H. F. (1995). The Art of Fieldwork. Walnut Creek: AltaMira Press. Zakaluk, B. L. & Samuels, S. J. (1998). Readability: Its Past, Present, and Future. Newark, DE: International Reading Association. 217 APPENDIX A Code of Hammurabi: excerpted text read by students 218 Code of Hammurabi When Anu the Sublime, King of the Anunaki, and Bel, the lord of Heaven and earth, who decreed the fate of the land, assigned to Marduk, the over-ruling son of Ea, God of righteousness, dominion over earthly man, and made him great among the Igigi, they called Babylon by his illustrious name, made it great on earth, and founded an everlasting kingdom in it, whose foundations are laid so solidly as those of heaven and earth; then Anu and Bel called by name me, Hammurabi, the exalted prince, who feared God ?, to bring about the rule of righteousness in the land, + to destroy the wicked and the evil- doers; so that the strong should not harm the weak; so that I should rule over the black- headed people like Shamash ? , and enlighten the land, + to further the well-being of mankind ?. When Marduk sent me to rule over men, to give the protection of right to the land? 2 If any one bring an accusation against a man, and the accused go to the river and leap into the river, if he sink in the river his accuser shall take possession of his house. But if the river prove that the accused is not guilty, and he escape unhurt, then he who had brought the accusation shall be put to death, while he who leaped into the river shall take possession of the house that had belonged to his accuser. 3 If any one bring an accusation of any crime before the elders, and does not prove what he has charged, he shall, if it be a capital offense charged, be put to death. 8 If any one steal cattle or sheep, or an ass, or a pig or a goat, if it belong to a god or to the court, the thief shall pay thirtyfold; if they belonged to a freed man of the king he shall pay tenfold; if the thief has nothing with which to pay he shall be put to death. ? 9 If any one lose an article, and find it in the possession of another: if the person in whose possession the thing is found say "A merchant sold it to me, I paid for it before witnesses," and if the owner of the thing say, "I will bring witnesses who know my property," then shall the purchaser bring the merchant who sold it to him, and the witnesses before whom he bought it, and the owner shall bring witnesses who can identify his property. The judge shall examine their testimony?both of the witnesses before whom the price was paid, and of the witnesses who identify the lost article on oath. The merchant is then proved to be a thief and shall be put to death. The owner of the lost article receives his property, and he who bought it receives the money he paid from the estate of the merchant. ? 10 219 If the purchaser does not bring the merchant and the witnesses before whom he bought the article, but its owner bring witnesses who identify it, then the buyer is the thief and shall be put to death, and the owner receives the lost article. ? 11 If the owner do not bring witnesses to identify the lost article, he is an evil-doer, he has traduced, and shall be put to death. ? 21 If any one break a hole into a house, he shall be put to death before that hole and be buried. 22 If any one is committing a robbery and is caught, then he shall be put to death. 195 If a son strike his father, his hands shall be hewn off. 196 If a man put out the eye of another man, his eye shall be put out. 197 If he break another man's bone, his bone shall be broken. 200 If a man knock out the teeth of his equal, his teeth shall be knocked out. 206 If during a quarrel one man strike another and wound him, then he shall swear, "I did not injure him wittingly," and pay the physicians. 207 If the man die of his wound, he shall swear similarly, and if he (the man that died) was a free-born man, he shall pay half a mina in money. ? 220 APPENDIX B The Justinian Code: excerpted text read by students 221 The Justinian Code I. Justice and Law ? JUSTICE is the constant and perpetual wish to render every one his due. The maxims of law are these: to live honesty, to hurt no one, to give every one his due. II. Natural, Common, and Civil Law. The law of nature is that law which nature teaches to all animals. For this law does not belong exclusively to the human race, but belongs to all animals, whether of the earth, the air, or the water. Hence comes the union of the male and female, which we term matrimony; hence the procreation and bringing up of children. We see, indeed, that all the other animals besides men are considered as having knowledge of this law. Civil law is thus distinguished from the law of nations. Every community governed by laws and customs uses partly its own law, partly laws common to all mankind. The law which a people makes for its own government belongs exclusively to that state and is called the civil law, as being the law of the particular state. But the law which natural reason appoints for all mankind obtains equally among all nations, because all nations make use of it. The people of Rome, then, are governed partly by their own laws, and partly by the laws which are common to all mankind. We will take notice of this distinction as occasion may arise. 9. The unwritten law is that which usage has established; for ancient customs, being sanctioned by the consent of those who adopt them, are like laws. 11. The laws of nature, which all nations observe alike, being established by a divine providence, remain ever fixed and immutable. But the laws which every state has enacted, undergo frequent changes, either by the tacit consent of the people, or by a new law being subsequently passed. ? IX. The Power of Parents. Our children, begotten in lawful marriage, are in our power. 1. Marriage, or matrimony, is a binding together of a man and woman to live in an indivisible union. 2. The power which we have over our children is peculiar to the citizens of Rome; for no other people have a power over their children, such as we have over ours. 3. The child born to you and your wife is in your power. And so is the child born to your son of his wife, that is, your grandson or granddaughter; so are your great- 222 grandchildren, and all your other descendants. But a child born of your daughter is not in your power, but in the power of its own father. XIII. Guardianship. Let us now proceed to another division of persons. Of those who are not in the power of a parent, some are under a tutor, some under a curator, some under neither? 1. Tutelage, as Servius has defined it, is an authority and power over a free person, given and permitted by the civil law, in order to protect one whose tender years prevent him defending himself. 2. Tutors are those who have this authority and power, and they take their name from the nature of their office; for they are called tutors, as being protectors and defenders? XX. Appointing of Tutors If any one had no tutor at all, one was given him, in the city of Rome by the praetor urbanus, and a majority of the tribunes of the plebs ?; in the provinces, by the praesides ... 6. It is agreeable to the law of nature that the persons under the age of puberty should be under tutelage, so that persons of tender years may be under the government of another. 7. As tutors administer the affairs of their pupils, they may be compelled to account ? when their pupils arrive at puberty. ? XXI. Authority of Tutors. In some cases it is necessary that the tutor should authorize the acts of the pupil, in others not. When, for instance, the pupil stipulates for something to be given him, the authorization of the tutor is not requisite; but if the pupil makes the promise, it is requisite; for the rule is, that pupils may make their condition better, but may not make it worse, without the authorization of their tutor. And therefore in all cases of reciprocal obligation, as in contracts of buying, selling, letting, hiring, bailment, and deposit, if the tutor does not authorize the pupil to enter into the contract, the person who contracts with the pupil is bound ?, but the pupil is not bound ?. XXII. Freedom from Guardianship. Pupils, both male and female, are freed from tutelage when they attain the age of puberty. The ancients judged of puberty in males, not only by their years, but also by the development of their bodies. But we, from a wish to conform to the purity of the present times, have thought it proper, that what seemed even to the ancients to be indecent towards females, namely, the inspection of the body, should be thought no less so towards males; and, therefore ? we have enacted that puberty in males should be considered to commence immediately on the completion of their fourteenth year; while, as to females, we have preserved the wise rule adopted by the ancients, by which they are esteemed fit for marriage on the completion of their twelfth year. XXIII. Curatorship. Males arrived at the age of puberty, and females of a marriageable age, receive curators, until they have completed their twenty-fifth year; for, although they have attained the age of puberty, they are still of an age which makes them unfit to protect their own interests. 1. Curators are appointed by the same magistrates who appoint tutors? 2. No adolescent is obliged to receive a curator against his will, unless in case of a lawsuit, for a curator may be appointed for a particular special purpose. 3. Madmen and prodigals, the Twelve Tables . But, ordinarily, curators are appointed for them? after inquiry into the circumstances has been made. 4. Persons who are of unsound mind, or who are deaf, mute, or subject to any perpetual malady, since they are unable to manage their own affairs, must be placed under curators. ? 223 224 APPENDIX C The Magna Carta: excerpted text read by students The Magna Carta Excerpts from the Magna Carta: These excerpts talk about how courts should be set up 17. Common pleas shall not follow our court, but shall be held in some fixed place. 18. Inquests ?, shall not be held elsewhere than in their own county courts and that in manner following,--We, or ? our chief justiciar, will send two justiciars through every county four times a year, who shall, along with four knights of the county chosen by the county, hold the said assize in the county court, on the day and in the place of meeting of that court. 19. And if any of the said assizes cannot be taken on the day of the county court, let there remain of the knights and freeholders, who were present at the county court on that day, as many as may be required for the efficient making of judgments ... ? 20. A freeman shall not be amerced for a slight offense, except in accordance with the degree of the offense ?; and for a grave offense he shall be amerced in accordance with the gravity of the offense, ? --if they have fallen into our mercy: and none of the aforesaid amercements shall be imposed except by the oath of honest men of the neighborhood. ? 21. Earls and barons shall not be amerced except through their peers, and only in accordance with the degree of the offense. 39. No freeman shall be taken or imprisoned or disseised or exiled or in anyway destroyed, ? except by the lawful judgment of his peers or by the law of the land. 40. (Part 1) To no one will we sell, (Part 2) to no one will we refuse or delay, (Part 3) right or justice. ? ? ? 225 226 APPENDIX D Introductory page from student CD Instructions page: Day 1 and Day 2 As you read you will see different symbols on the pages. Each of these has a special meaning. If a word or phrase is underlined, you can click on it and a definition or explanation will appear to help you understand what it means. You only have to use this if you aren?t sure what the word means or if you want to check to be sure you understand. When you see a small globe, this means that there is some information you need to read. ? When you see a number like this, it means there is a question about this section to answer on the information gathering page. Some of the questions are written on the information gathering page and some are written next to the number on one of the pages you are reading. Write all of your answers on the information gathering sheet. If you aren?t sure about an answer, write what you think the answer is. First: Read the Code of Hammurabi and use the Code of Hammurabi: Information Gathering Chart to answer the questions you find as you read. Your chart is in your folder. Click on this icon to read the Code of Hammurabi. Second: Read the Justinian Code and use the Justinian Code: Information Gathering Chart to answer the questions you find as you read. Your chart is in your folder. Click on this icon to read the Justinian Code. Third: Read the Magna Carta and use the Magna Carta: Information Gathering Chart to answer the questions you find as you read. Your chart is in your folder. Click on this icon to read the Magna Carta. 227 228 APPENDIX E Group work problem descriptions 229 The problems: Group Work Problem #1 Many of the citizens believe a law is unfair. What would they have to do to try to get it changed in each of the civilizations you have read about? Which of these does your group think is the best way? Explain why. To work on this problem, you need to: ? Explain what would have to happen for the Babylonians to change Hammurabi?s Code ? Explain what would have to happen for the Romans to change the Justinian Code ? Explain what would have to happen for the English to change the laws in the Magna Carta ? Explain what do U. S. citizens do if they want to change a law? Problem #2 One morning a man found a dog sitting on his front porch. It didn?t have tags to let him know whose dog it was, and he decided to take it to his backyard and put it in a fence. Later that morning the owner realized the dog was missing and put signs all over town with pictures of his dog, but the man who had the dog didn?t answer the ads. Several days later the owner changed the signs and offered a reward. The next day, the man who had the dog called and offered to return the dog if he was paid the reward. After this happened the owner of the dog paid the reward and got his dog back, but he was so angry that the man had kept the dog, he accused the other man of theft. Facts of the case: ? The man who owned the dog lived across the street from the man who had dog. ? The owner of the dog is often seen around the neighborhood with the dog. ? One of the signs advertising for the dog was on a pole right in front of the man who found the dog?s front door. ? The man who found the dog entered his house from the side and seldom went to his front yard. Was this theft? How would it be handled by the groups listed below? If the man was cleared of theft, what can he do because he was falsely accused of theft? For this question use Hammurabi?s Code and U.S. law to answer the questions. Figure out what the Babylonians would have done and what would be done in the U. S. Which of these is the best way to decide who is in the wrong? Why? Problem #3 Shawn, a 12-year-old boy, and Mary, a 10-year-old girl, became orphans when both of their parents died. Their family owned a lot of land and other property. Who will decide what will be done with these children? According to the Justinian Code, what are the guidelines for making this decision? For this question use the Justinian Code and U. S. law to answer the question. Explain what would have happened under Roman law and what would happen under U. S. law. Which one do you think is the best way to handle this situation? Why? 230 APPENDIX F Teacher interview guide 231 Teacher interview 1. Describe the aspect of the instructional unit your class completed as part of this study that you think was the most successful. 2. Describe the aspect of the instructional unit your class completed as part of this study that you think was the least successful. 3. Thinking about the hyperlinks that were added to the documents, how successful do you think each of the three types were: a. Hyperlinks that gave the students definitions or explanations? b. Hyperlinks that gave the students historical information or told you how something fit into history? c. Hyperlinks that helped the students understand what they needed to do to use the document in the history lesson? 4. Describe your observations of the students? use of the hyperlinks as they read the texts. 5. To answer this question, think about other times you have had your class has read difficult documents in history class. Did inclusion of the hyperlinked documents change what you did as a teacher while students worked with the documents? If so, describe how. 6. Describe how the students integrated their knowledge of the documents into the overall lesson. 7. Compare your use of these or similar non-hyperlinked documents in past lessons to the use of these hyperlinked documents in this lesson. 8. Describe problems you saw associated with the use of these documents in this lesson? 9. Describe the most successful aspects of using these documents? 10. Overall, how successful was the use of these documents in the lesson? Why? 11. In the future, if you have difficult text you want to include in a lesson will you consider using hyperlinks to provide scaffolding for the students as we did in this lesson? Explain why or why not. 232 APPENDIX G Student interview guide 233 Student interview (During the interview, students should either have a computer available to provide on-line access to the documents or have print versions of the documents available if it is not possible to have access to computers.) 1. Describe your favorite part of the instructional unit your class completed as part of this study. 2. Describe your least favorite part of the instructional unit your class just completed as part of this study. 3. When you read documents on the computer some places are underlined to let you know that you can click there to find out more information. These are called hyperlinks. Looking at the hyperlinks that were added to the documents you read on the computer, describe how you used each type of link and tell me how successful or unsuccessful you think each was in helping you understand the document: a. Hyperlinks that gave you definitions or explanations? b. Hyperlinks that gave you historical information or told you how something fit into history? c. Hyperlinks that helped you understand what you needed to do to use the document in the history lesson? 4. Describe how you used the hyperlinks as you read the texts. (Example: Did you read the entire text and then go back and look at the hyperlinks or use each link as you came to it?) Explain why you choose to read that way. 5. To answer this question, think about other times your class has read documents in history class. Do you think having the hyperlinked documents in the lesson changed what your teacher did while students worked with the documents? If so, describe how. 6. After you finished reading the documents, how did you use your knowledge from them to help you complete the lesson? 7. Compare your use of non-hyperlinked documents in past lessons to the use of these hyperlinked documents in this lesson. 8. Describe problems you had or that you saw classmates have using these documents in this lesson? 9. Describe the most successful aspects of using these documents? 10. Overall, how successful was the use of these documents in the lesson? Why? 11. In the future, if your teacher wants you to read difficult text as part of a lesson would you want him or her to use hyperlinks to provide assistance link in this lesson? Explain why or why not. 234 APPENDIX H Reading guide for the Code of Hammurabi 235 Code of Hammurabi: Individual Information Gathering Chart Why do you think this paragraph begins by talking about how powerful Marduk is and then Hammurabi calls himself ?the prince who fears God?? Why do you think Hammurabi compares himself to Shamash? What things does Hammurabi say he wants his laws to do? + + + + + According to this law, what happened if someone stole something that belonged to a church or the king? 236 What if it belonged to a free citizen? What happened if the thief didn?t have enough money to pay the fine? If you bought a watch from someone and a week later you were at a party and someone there saw your watch and said it was theirs, what would they have to do to prove the watch was theirs? What would you have to do to prove you were not a thief? If you did what you had to do and proved your innocence, what would happen to the person that sold you the watch? Would you get to keep the watch or would it be returned to the original owner? How would the person who doesn?t get the watch get their money back? What would happen if you didn?t prove that you bought the watch? What would happen if the person that said the watch was theirs didn?t prove it? In 206, what does it say happens if one man hurts another during a quarrel? In 207, what does it say should happen if one man kills another during a quarrel? 237 APPENDIX I Reading guide for The Justinian Code 238 Justinian Code: Individual Information Gathering Chart ? After reading the entire first section, explain what were the different types of laws Justinian said were part of his code of laws? ? This section is talking about laws changing. What are two ways laws can change? ? Look at the link at the end of IX. 3. and then answer this question. Which other family members did the paterfamilias have control over and which family members did he not have control over? ? If a tutor as control of a young person?s affairs until the young person reaches puberty, what do you think it means that the tutor may be ?compelled to account? when they stop being the tutor? ? If a young person makes a business agreement that the tutor believes is good, does the other person have to go by that agreement since they made it with a child? ? If a young person makes a business agreement that the tutor believes is bad, does the young person have to go by the agreement anyway? ? Describe the difference between a tutor and a curator by answering these questions: Who is assigned a tutor? Who is assigned a curator? What is a tutor?s job? What is a curator?s job? 239 APPENDIX J Reading guide for The Magna Carta 240 Magna Carta: Individual Information Gathering Chart Read the link at the end of the title and answer this question. List some of the reasons the English noblemen would have wanted the Magna Carta. Read the link at the end of the title and answer this question. How do you think the king felt about the Magna Carta? This section tells what will happen if court can?t be held on the day it is scheduled. What does it say will happen? This section says a person can only receive a penalty that is in line with the crime they committed. What would be an example of a penalty that would be unfair? What would be an example of a penalty that is fair? Who does this section say can make someone pay an amercement? The three parts of this section are marked with +. To understand what this section means: Read the first and third parts together. What do these parts mean when you read them together? Read the second and third sections together. What do these parts mean when you read them together? Why do you think the earls and barons thought Number 40 was very important? 241 APPENDIX K Group work decision-making scaffold 242 Data Gathering Chart: Groups 1. When the Code of Hammurabi was written, why did he say he had the right to make laws? 2. Why was the Justinian Code written? What types of laws were included in this code? 3. Why was the Magna Carta written? 4. How do we make laws in the United States? 5. Some people call the Code of Hammurabi the ?eye for an eye and a tooth for a tooth? laws. Why do you think they say that? List one example of when U. S. law is the same as this. List one example of when U. S. law is different from this. 6. What did Hammurabi say would happen if someone accuses someone else of a crime they didn?t really do? Tell why that is fair or not fair. How do you think that effected people accusing someone of a crime 7. According to Hammurabi?s Code, what was one way a person could prove they were innocent of a crime? Is this fair? Why or why not? 8. According to the Justinian Code, what are the differences between a tutor and a curator? When does a young person stop having a tutor? Do you think this is a good way to help young people who don?t have parents? Explain why or why not. 243 APPENDIX L Description of pilot study Purposes of the pilot study The pilot study took place in three sixth-grade social studies classes in a small middle school in a suburban, southeastern town. It was intended to address two areas of concern in planning my primary study: 1) the coding of documents to identify areas where scaffolding could assist students and 2) determining what effects the addition of hyperlinks have on the readability of online source documents. Two issues guided the pilot study: 1. Is it feasible to have students code areas of confusion as they read complex text-based source documents? If so, what coding categories should be used? How closely does coding by teachers match coding done by students? 2. What effect does the addition of hyperlinks have on the readability of text- based source documents used in a history class? For the first issue I wanted to ascertain how well students could identify areas of source documents where they became confused and the reasons for their confusion, as well as how well educators can identify those times when students will become confused. Additionally I sought to understand which coding categories would be most effective to allow students to identify areas where they experienced confusion. 244 The pilot study was also intended to determine the effects on readability of adding hyperlinks to source documents. The addition of hyperlinks to a text-based source document potentially alters the complexity as a result of the additional text and might, therefore, increase or decrease the reading level of the text. Hyperlinks might also increase or decrease the domain specific complexity as a result of providing readers with information such as historical background, assistance in understanding unfamiliar and/or archaic language and word usage, and support utilizing the document within the context of the task. (For a more complete description of the readability measures see Chapter 3.) Description of the pilot study In the pilot study I worked with a veteran history teacher and a Master?s level graduate student to plan and implement the study. Two classes with a total of 43 students took part in the pilot study. The pilot study lesson was intended to prepare students to do oral histories of individuals they knew who had lived through the African-American Civil Rights era. In this lesson the students read and analyze written accounts of several oral histories. The teachers chose four texts that they believed complimented the instructional unit the lesson was part of and that represented several levels of reading difficulty. The beginning of the pilot study lesson was a discussion about the reasons students may become confused as they read historical source documents. During this part of the lesson I was a guest teacher and discussed the following reasons for potential confusion with the students: 1) unfamiliarity with the definitions of words or phrases, 2) the need for additional background information, 3) not fully understanding the language being used because of archaic or unfamiliar dialects, and 4) being confused but not understanding the reason for the confusion. After this discussion the students were given the documents 245 one at a time and asked to code them in order to identify areas where they were personally confused. In order to code the documents, students were given instructions about which colors to code the different types of confusion, and these colors and instructions were written on the board to provide reinforcement. The coding categories were: 1. I do not understand the meaning of this word or phrase 2. I do not understand the language or dialect being used 3. I need historical information 4. Confusion has occurred but I do not know why I?m confused. The teachers and I also coded the documents to identify areas we believed the students would be confused and the reasons we believed the confusion would take place. Following coding, the history lesson continued with the students using the documents as part of the lesson. Prior to the students? coding I had analyzed each of the documents to determine the reading grade levels and the domain specific complexity of each document. Readability analyses were completed using the Flesch-Kincaid Readability Test and the Flesch Reading Ease Measurement. Analysis was also completed to determine the domain specific complexity of each document using criteria I established because there is no standard measure for determining domain specific complexity . These analyses were repeated after hyperlinks were added. This allowed me to understand the effect of adding the hyperlinks to each document. I also did a comparative statistical analysis to determine how closely the teachers? coding and my coding matched the students? coding. 246 Results of the pilot study: Student coding As a result of the analysis of the pilot study data, several important findings emerged that guided key aspects of the later study. Coding by the students, the teacher, and me was the most similar in areas where the students seemed to understand that they needed assistance and the type of assistance needed. Overall between 60% and 90% of students coded areas the same as the teachers and I with some documents having greater similarities in coding than others. This indicated that while students were often able to recognize the areas where their understanding of complex text sources breaks down and what they need to fix their understanding, the teachers and I were able to accurately predict when students would experience these difficulties a majority of the time. In considering the success of the different coding categories, I concluded that having four categories was overly complicated and contributed to the confusion some of the students experienced. Students tended to make clear distinctions between the need for definitional assistance and the need for historical background information. Additionally, students were willing to indicate when they were confused but unable to understand the source of their confusion. However, the category asking them to identify when the language being used caused them confusion was completely unsuccessful. Students were unsure how to differentiate between a problem that was definitional and one that was a language problem. Many simply didn?t use that category and some who did use the category were unable to explain why during the follow-up discussion at the end of the lesson. Overall, students were able to identify areas of confusion adequately and accurately identify the reasons for their confusion a majority of the time. Three of the 247 categories proved to be easily understood by the students and because the category for confusion due to the language proved to be too confusing for most students to use successfully, this category was eliminated from the primary study. Results of the pilot study: Effects of hyperlinks on readability After analyzing the coded documents, I added hyperlinks that would provide definitional, historical background, and metacognitive scaffolds to the documents. This allowed me to determine the effects adding hyperlinks had on their readability. The addition of hyperlinked scaffolding, while providing useful information, also added considerable text to the original documents. This additional text, when considered along with the original text, affected both the basic readability and the content specific complexity of the documents. In order to determine the readability of each document with the hyperlinks added, the document text along with the text from all of the associated links were analyzed together as one document. Although the students would access the text in the hyperlinks separately from the text of the source documents, I believed that this method would provide the most accurate information about the ways the hyperlinks and the source documents worked together as a single document. Table A1 illustrates the changes that took place in the basic readability of each document after the inclusion of hyperlinks: The overall effect of the addition of hyperlinks to these documents was to make the readability of the texts more level. Those documents that began with extremely high readability scores had their scores lowered by the inclusion of hyperlinks that included more simplified language. One document that began with a readability in the eighth grade range, above that of most of the students, remained in the same range after hyperlinks 248 Table A1 Changes in readability after inclusion of hyperlinked scaffolding Before addition of hyperlinks After the addition of hyperlinks Document 1: Document 1: Words ? 472 Words ? 872 Flesch-Kincaid ? 11.3 reading level Flesch-Kincaid ? 9.4 reading level Reading Ease ? 57.7% or approx. 9 th grade level Reading Ease ? 61.3% or approx. 8 th grade level Document 2: Document 2: Words ? 439 Words ? 834 Flesch-Kincaid ? 10.7 reading level Flesch-Kincaid ? 9.2 reading level Reading Ease ? 54.9% or approx. 9 th grade level Reading Ease ? 63.5% or approx. 8 h grade level Document 3a: Document 3a: Words ? 296 Words ? 569 Flesch-Kincaid ? 8.1 reading level Flesch-Kincaid ? 8.6 reading level Reading Ease ? 71.6% or approx. 7 th grade level Reading Ease ? 70.4% or approx. 7 th grade level Document 3b: Document 3b: Words ? 109 Words ? 239 Flesch-Kincaid ? 5.5 reading level Flesch-Kincaid ? 6.3 reading level Reading Ease ? 79.6% or approx. 6 th grade level Reading Ease ? 76.2% or approx. 7 th grade level Document 4: Document 4: Words ? 495 Words ? 1134 Flesch-Kincaid ? 3.8 reading level Flesch-Kincaid ? 5.5 reading level Reading Ease ? 84.3% or approx. 6 th grade level Reading Ease ? 77.1% or approx. 7 th grade level were added, and the two documents that began with readability scores lower than the grade level of the students had their readability increased to a range close to that of most of the students. Based on analysis after adding hyperlinked scaffolding I concluded that the addition of hyperlinks allowed important scaffolding to be added without negatively impacting students? ability to read and successfully use the text. However, readability does not measure the skills needed to access the historic information contained in the documents, the level of motivation or lack of motivation provided by the text and its presentation, the difficulty posed by the use of unfamiliar or archaic language, or the 249 ability of students to understand and use the text within the context of the lesson, and unfortunately there is no standard instrument that measures these aspects of source documents. I used several criteria to consider as I analyzed the effects of adding hyperlinks on domain specific complexity. I considered the effect of the additional text on students? overall ability to use the documents, the effect of having access to historical background information and the motivational factor of these documents as students use them as part of a history lesson. The number of words in a document may be a factor in determining whether or not some students will attempt to read the document (See Table A1 for changes in the length of each document). Extremely long documents can be daunting for many students whose experience with reading in school has not always been positive. Results of the pilot study. The pilot study indicated that students were able to recognize where and why they experienced confusion when reading source documents and that in this study students?, teachers?, and researcher?s coding was very similar. This study also suggested that the process of student coding should be as simple as possible. In the pilot study students were asked to code documents based on four criteria. The category asking students to identify areas where they were confused as a result of the use of language was not understood by the students and as a result coding done using this criteria was of little use, leading to the elimination of this choice in the my primary study. The pilot study also provided guidance in identifying the effects of adding hyperlinks to text-based, historical source documents. The process used for determining readability proved successful and was repeated in the final study. The leveling of readability was a positive outcome because it meant that the final readability of all of the 250 documents was closer to the reading ability of the majority of the students in the class. The domain specific complexity was also improved through the use of the hyperlinks that provided students with the on-time assistance which research suggests is a positive influence on students? willingness to continue to actively engage in the study of difficult source materials. Implications of the primary study. The pilot study helped me refine some aspects of the process of my primary study and informed my analysis and conclusions for that study as well. The process of student coding in the pilot study indicated that having students code documents to identify areas of confusion could be successful and helped me establish the codes I could use to help ensure this success. I also used this study to refine the measure of domain specific complexity I was using so that the criteria and the determinations were more clearly articulated in my primary study (See pages 4-6). The coding in this study helped me establish a starting point for determining how successfully teachers, acting as experts, can identify the needs of students, acting as novices, when they are reading complex historical source documents. This gave me a basis of comparison for the findings of the primary study. Similarly, the finding on the changes in readability for the documents in the pilot study as a result of adding hyperlinked scaffolding provided me with a point of comparison for the changes I saw in the primary study and again allowed me to consider those findings and the reliability of using the methods for testing readability as I used them. 251 APPENDIX M Lesson goals: 1. Students will develop an understanding of the role past codes of law have played in influencing the development of modern codes of law, including in the United States. 2. Students will identify the basis those in power used to legitimize their rule in four different societies, ancient Mesopotamia, ancient Roman society, thirteenth century England, and the modern United States of America. 3. Students will develop and defend positions regarding three problems dealing with the following issues involving ancient codes of law: a. The concept of ?an eye for an eye? and the meaning and punishment of theft as it is presented in Hammurabi?s Code and compared to the U.S. code of law. b. The enforcement of family law in Justinian Rome compared to modern America. Activity: (1) Students read each of the documents individually and complete a scaffold. (2) In small groups students complete a group scaffold meant to combine and clarify the individual scaffolds. (3) Groups are presented with legal problems related to each of the types of law covered by the documents and the groups determine how the different civilizations would deal with each of the problems. (4) The groups are presented with legal problems and are to look at the issues from the perspectives of two or more of the cultures they examined. The group determines two things: a) how each of the cultures would have handled this problem, and b) which of these is the most fair OR if none of them are fair, what would be fair? In this part of the activity students should explain why they do or do not think the perspectives they looked at are or are not fair and just. (5) Each group will be asked to present their work from one problem in #4 (more if there is time) to the class and the class members can question or challenge their decisions during the discussion/question session after they are finished presenting. Problems: 1. Many of the citizens believe a law is unfair. What would they have to do to try to get it changed? ? In the scaffolding, students should have to: Describe the basis for each of the four sets of laws (where does the power to make laws come from?). How are the four different? How are they the same? 252 For this question, groups will use all three codes and ?Legal Responsibilities?, Chapter 16 on page 388 as well as other information they have learned about how laws are made in the US. 2. One morning a man found a dog sitting on his front porch. It didn?t have tags to let him know whose dog it was, and he decided to take it to his backyard and put it in a fence. Later that morning the owner realized the dog was missing and put signs all over town with pictures of his dog, but the man who had the dog didn?t answer the ads. Several days later the owner changed the signs and offered a reward. The next day, the man who had the dog called and offered to return the dog if he was paid the reward. After this happened the owner of the dog paid the reward and got his dog back, but he was so angry that the man had kept the dog, he accused the other man of theft. a. Facts of the case: ? The man who owned the dog lived across the street from the man who had dog. ? The owner of the dog is often seen around the neighborhood with the dog. ? One of the signs advertising for the dog was on a pole right in front of the man who found the dog?s front door. ? The man who found the dog entered his house from the side and seldom went to his front yard. Was this theft? How would it be handled by the groups listed below? If the man was cleared of theft, what can he do because he was falsely accused of theft? For this question use Hammurabi?s Code and US law 3. Shawn, a 12-year-old boy, and Mary, a 10-year-old girl, became orphans when both of their parents died. Their family owned a lot of land and other property. Who will decide what will be done with these children? According to the Justinian Code, what are the guidelines for making this decision? For this question use the Justinian Code and US law 253 APPENDIX N Magna Carta 61. Since, moreover, for God and the amendment of our kingdom and for the better allaying of the quarrel that has arisen between us and our barons, we have granted all these concessions, desirous that they should enjoy them incomplete and firm endurance for ever, we give and grant to them the underwritten security, namely, that the barons choose five-and-twenty barons of the kingdom, whomsoever they will, who shall be bound with all their might, to observe and hold, and cause to be observed, the peace and liberties we have granted and confirmed to them by this our present Charter, so that if we, or our justiciar, or our bailiffs or any one of our officers, shall in anything beat fault toward any one, or shall have broken any one of the articles of the peace or of this security, and the offense be notified to four barons of the foresaid five-and-twenty, the said four barons shall repair to us (or our justiciar, if we are out of the realm) and, laying the transgression before us, petition to have that transgression redressed without delay. And if we shall not have corrected the transgression (or, in the event of our being out of the realm, if our justiciar shall not have corrected it) within forty days, reckoning from the time it has been intimated to us (or to our justiciar, if we should be out of the realm), the four barons aforesaid shall refer that matter to the rest of the five-and-twenty barons, and those five-and-twenty barons shall, together with the community of the whole land, distrain and distress us in all possible ways, namely, by seizing our castles, lands, possessions, and in any other way they can, until redress has been obtained as they deem fit, saving harmless our own person, and the persons of our queen and children; and when redress has been obtained, they shall resume their old relations toward us. And let whoever in the country desires it, swear to obey the orders of the said five-and-twenty barons for the execution of all the aforesaid matters, and along with them, to molest us to the utmost of his power; and we publicly and freely grant leave to every one who wishes to swear, and we shall never forbid any one to swear. All those, moreover, in the land who of themselves and of their own accord are unwilling to swear to the twenty-five to help them in constraining and molesting us, we shall by our command compel the same to swear to the effect aforesaid. And if any one of the five-and-twenty barons shall have died or departed from the land, or be incapacitated in any other manner which would prevent the foresaid provisions being carried out, those of the said twenty-five barons who are left shall choose another in his place according to their own judgment, and he shall be sworn in the same way as the others. Further, in all matters, the execution of which is intrusted to these twenty-five barons, if perchance these twenty-five are present, that which the majority of those present ordain or command shall be held as fixed and established, exactly as if the whole twenty-five had concurred in this; and the said twenty-five shall swear that they will faithfully observe all that is aforesaid, and cause it to be observed with all their might. And we shall procure nothing from any one, directly or indirectly, whereby any part of these concessions and liberties might be revoked or diminished; and if any such thing has been procured, let it be void and null, and we shall never use it personally or by another. 17. Common pleas shall not follow our court, but shall be held in some fixed place. 254 18. Inquests of novel disseisin, of mort d'ancester, and of darrein presentment, shall not be held elsewhere than in their own county courts and that in manner following,--We, or, if we should be out of the realm, our chief justiciar, will send two justiciars through every county four times a year, who shall, along with four knights of the county chosen by the county, hold the said assize in the county court, on the day and in the place of meeting of that court. 19. And if any of the said assizes cannot be taken on the day of the county court, let there remain of the knights and freeholders, who were present at the county court on that day, as many as may be required for the efficient making of judgments, according as the business be more or less. 20. A freeman shall not be amerced for a slight offense, except in accordance with the degree of the offense; and for a grave offense he shall be amerced in accordance with the gravity of the offense, yet saving always his "contenement;" and a merchant in the same way, saving his "merchandise;" and a villein shall be amerced in the same way, saving his "wainage"--if they have fallen into our mercy: and none of the aforesaid amercements shall be impsed except by the oath of honest men of the neighborhood. 21. Earls and barons shall not be amerced except through their peers, and only in accordance with the degree of the offense. 39. No freeman shall be taken or imprisoned or disseised or exiled or in anyway destroyed, nor will we go upon him nor send upon him, except by the lawful judgment of his peers or by the law of the land. 40. To no one will we sell, to no one will we refuse or delay, right or justice. 255 APPENDIX O Magna Carta Introduction: In England before the Magna Carta was written in 1215, the king could do almost anything he wanted. If he didn?t like something one of his barons or earls did, he could take all of their lands or have them killed. The king could force the barons and earls to raise huge amounts of money any time he wanted to. If a baron or earl died, the king could decide to give someone else his lands, even if he had children who should get them. He also had the right to sell the noblemen?s children and the widow into marriage if he wanted to. This meant that he could force them to marry whoever paid him the highest price and that person would then get control of all of the lands. To make all of this worse, there was no way for the noblemen to complain if they believed they were being treated unfairly. The English noblemen felt that the powers the kings had were more than he deserved. In order to fix this they wrote the Magna Carta and forced the king to agree to it. The Magna Carta set up a system that would allow all of the noblemen to be treated fairly and gave them the right to have their complaints heard by other noblemen instead of just the king. Only other noblemen could decide what was fair and what should be done with other noblemen. The Magna Carta also set up a system for what should happen if a nobleman died so his wife and children would be treated fairly. ? List some of the reasons the English noblemen would have wanted the Magna Carta. ? How do you think the king felt about the Magna Carta? 256 APPENDIX P Original Problems Problem 1: Many of the citizens believe a law is unfair. What would they have to do to try to get it changed in each of the civilizations you have read about? Which of these does your group think is the best way? Explain why. (Using all three documents) Problem 2: One morning a man found a dog sitting on his front porch. It didn?t have tags to let him know whose dog it was, and he decided to take it to his backyard and put it in a fence. Later that morning the owner realized the dog was missing and put signs all over town with pictures of his dog, but the man who had the dog didn?t answer the ads. Several days later the owner changed the signs and offered a reward. The next day, the man who had the dog called and offered to return the dog if he was paid the reward. After this happened the owner of the dog paid the reward and got his dog back, but he was so angry that the man had kept the dog, he accused the other man of theft. Was this theft? How would it be handled by the groups listed below? If the man was cleared of theft, what can he do because he was falsely accused of theft? (Using Hammurabi?s Code) Problem 3: An Englishman nobleman?s son is accused of stealing and murder. Before the day he was supposed to have court his father paid an official to declare him innocent and let him go. 257 Other people in the community found out about this and became angry. When they went to the other noblemen of the region, what should their complaints have been and what do you think should have been done about them based on the laws in the Magna Carta? (Using the Magna Carta) Problem 4: Shawn, a 12-year-old boy, and Mary, a 10-year-old girl, became orphans when both of their parents died. Their family owned a lot of land and other property. Who will decide what will be done with these children? According to the Justinian Code, what are the guidelines for making this decision? (Using the Justinian Code) 258 APPENDIX Q Scaffolding and document alteration based on student, teacher and researcher coding Provide a definitional scaffold Original text from Hammurabi?s Code: ?he is an evil-doer, he has traduced, and shall be put to death? Reason for the text adjustment: One hundred percent of students coded that they needed definitional assistance with the word ?traduced? Text adjustment: The word ?traduced? was linked to the following definition: ?Traduce: To tell a lie? Provide a historical background scaffold Original text from the Justinian Code: ?Tutelage, as Servius has defined it, is an authority and power over a free person? Reason for the text adjustment: This section of text included a definitional scaffold to assist students with the word ?tutelage?, but a majority of students, as well as the teacher and I, also coded this section as needing historical background information because of the need to understand the importance of Servius to this section of the code. Text adjustment: The following historical background link was included at the end of the phrase ?as Servius has defined it?: Servius Tullius was a Roman king. While he was king, a Roman constitution was written. That constitution explained what tutors and curators were supposed to do. When the Justinian Code was written this was one of the sets of laws that were part of it. Justinian used Servius? definitions of tutors and curators when he put the Justinian Code together. Replace a portion of the text with a summary or explanation Original text from the Magna Carta: ?61. Since, moreover, for God and the amendment of our kingdom and for the better allaying of the quarrel that has arisen between us ? and if any such thing has been procured, let it be void and null, and we shall never use it personally or by another.? See Appendix N for the full text of this section. Reasons for the text adjustment: Although many students began reading this section and attempting to code specific words and phrases where they thought they needed definitional or historical background information, all of the students eventually marked the entire section as confusing. Many of the above average readers indicated that they needed historical background information for the section, but most of the average and below average readers coded the entire section as confusing without knowing why they were confused. Delete a portion of text Original text from Hammurabi?s Code: ?When Marduk sent me to rule over men, to give the protection of right to the land, I did right and righteousness in . . . , and brought about the well-being of the oppressed.? Reasons for the text adjustment: A majority of the students coded this excerpt as needing historical background information, but approximately thirty percent also coded that the last half, beginning with ?I did right?? was confusing and they were not sure why they were confused. After considering the passage I decided that the first half of the excerpt was vital to the students? understanding and could not only be used alone but was more easily understood without the second half of the passage. As a result the passage was excerpted and historical background and metacognitive scaffolds were added. Text adjustment: Excerpted text: ?When Marduk sent me to rule over men, to give the protection of right to the land?? Historical background scaffold: ?Stele of the Code of Hammurabi? (picture of the stele included) This rock is eight feet tall and is called a stele. The picture carved into the top shows Shamash sitting down giving Hammurabi the laws. All of the laws in the Code of Hammurabi were carved into the bottom and then the stele was put in a public place for everyone to see. This stele is now in the Louver Museum in Paris, France.? Metacognitive scaffold included at that end in this hyperlink: ?Why was it important for Hammurabi to put these laws in a place where everyone could see them?? 259 260 Text adjustment: The entire section was replaced with a historical background scaffold that explained the reasons the noblemen of England wrote the Magna Carta. The scaffold also included two metacognitive scaffolding questions to help the students consider the establishment of the Magna Carta. See Appendix O for the entire text of this hyperlink. Examples of metacognitive scaffolds Hammurabi?s Code Metacognitive scaffold: ?What does it say will happen if the person accused of the crime is not guilty?? Purpose of the scaffold: This scaffold was included in a section that indicated an accused person could use a ?trial by water? to prove their innocence or guilt. The teacher and I believed that students would be intrigued by this concept and fail to notice the other part of the section where it discussed the fate of someone that makes a false accusation. This scaffold was intended to encourage them to consider both aspects of this section. Justinian Code Metacognitive Scaffold: ?Which other family members did the paterfamilias have control over and which family members did he not have control over?? Purpose of the scaffold: This scaffold was intended to help students better understand the importance of the paterfamilias in Roman families and society. The teacher and I were concerned that the students would fail to understand the complete power this arrangement gave the head of a family if they were not required to consider the situation in more depth. Magna Carta Metacognitive scaffold: List some of the reasons the English noblemen would have wanted the Magna Carta. How do you think the king felt about the Magna Carta? Purpose of the scaffold: These are the two metacognitive scaffolding questions that were included at the end of the section describing why the English noblemen insisted on the Magna Carta. The teacher and I wanted to ensure that the students didn?t apply the modern concept of democratic rule to this situation and fail to understand the fundamental changes this document had on the lives of the noblemen and the king. In the first question students are expected to use information from the historical background scaffold to identify reasons the nobles wanted more control over the affairs of the government. The second question is asking the students to imagine how the king would feel after being forced to give up the powers he gave up in agreeing to the Magna Carta.