SIMBUILDER SCIENCE: AN APPROACH TO ENHANCING READING LITERACY THROUGH VISUAL PROGRAMMING Except where reference is made to the work of others, the work described in this thesis is my own or was done in collaboration with my advisory committee. This thesis does not include proprietary or classified information. ____________________________________ Alexandria L. Williams Certificate of Approval: _______________________________ ______________________________ Kai Chang Juan E. Gilbert, Chair Professor Associate Professor Computer Science and Software Computer Science and Software Engineering Engineering _______________________________ ______________________________ Cheryl Seals Stephen L. McFarland Assistant Professor Acting Dean Computer Science and Software Graduate School Engineering SIMBUILDER SCIENCE: AN APPROACH TO ENHANCING READING LITERACY THROUGH VISUAL PROGRAMMING Alexandria L. Williams A Thesis Submitted to the Graduate Faculty of Auburn University in Partial Fulfillment of the Requirements for the Degree of Master of Science Auburn, Alabama August 7, 2006 iii SIMBUILDER SCIENCE: AN APPROACH TO ENHANCING READING LITERACY THROUGH VISUAL PROGRAMMING Alexandria L. Williams Permission is granted to Auburn University to make copies of this thesis at its discretion, upon the request of individuals or institutions and at their expense. The author reserves all publication rights. ________________________ Signature of Author ________________________ Date iv VITA Alexandria Williams is the daughter of Alfred and Joanne Williams. She was born on July 23, 1980, in Jacksonville, FL. She graduated from Stanton College Preparatory School in Jacksonville. Alexandria entered South Carolina State University immediately following her high school graduation and received a Bachelor of Science in Computer Science in May 2002. Immediately following graduation from South Carolina State University Alexandria returned to Jacksonville, Florida to teach seventh grade mathematics. Alexandria entered Auburn University in 2004, after two years of a successful teaching career. She is currently working diligently to meet the requirements of a Master of Science in Computer Science at Auburn University. v THESIS ABSTRACT SIMBUILDER SCIENCE: AN APPROACH TO ENHANCING READING LITERACY THROUGH VISUAL PROGRAMMING Alexandria L. Williams Master of Science, August 7, 2006 (B.S., South Carolina State University, May 2002) 54 Typed Pages Directed by Juan E. Gilbert Reading literacy has been a major problem for American students for the past several years. Educators have struggled to find a solution to this problem that will help students while at the same time maintain their interest in the subject matter. Educational video games and other e-learning technologies have made there way into classrooms and homes all around the world as an innovative solution to children?s learning deficiencies. This study suggests that the Squeak SimBuilder e-learning technology can be used to enhance reading literacy in a group of fourth and fifth grade students. The SimBuilder Science Project came into existence as an outreach program which uses visual programming to assist in enhancing reading literacy. Solving a science challenge was the overall goal for the students in the program, while reading science vi articles and writing related journal responses allowed the students to unknowingly improve reading comprehension and writing skills. An experiment was conducted and results were compiled based on an ongoing assessment of the students and a final survey. A careful analysis of the experimental data reveals that it is possible for Squeak SimBuilder to be effectively used as a tool to help improve reading literacy. To explain these results, a summarized discussion is given. Moreover, conclusions of this research are presented and future work that aims to improve the SimBuilder Science Project is also included. vii ACKNOWLEDGMENTS First and foremost, I would like to thank God who is my life and strength and with whom I can do all things but fail. I would like to thank Dr. Juan Gilbert for his support and guidance throughout my graduate career thus far. I would like to thank my parents, Mr. & Mrs. Alfred & Joanne Williams, who have supported me in every endeavor of my life. To Dr. Kai Chang and Dr. Cheryl Seals, my graduate committee members, thanks for all the reviewing and revising advice. To Mrs. Ramona Lindsey, Mr. Kenneth Rouse, and the entire SimBuilder Science Team, thank you for all of your hard work and assistance on this project. To my friends, thanks for supporting me in all that I do. Finally, exceptional thanks to my Damon, who always knows how to calm me down, who put up with the whining and complaining, and who supported and motivated me more than he knows. viii Style manual or journal used Journal of SAMPE Computer software used Microsoft Word XP ix TABLE OF CONTENTS LIST OF FIGURES........................................................................................................xi 1. INTRODUCTION......................................................................................................1 2. LITERATURE REVIEW ...........................................................................................3 2.1 Intelligent Tutoring System ............................................................................3 2.2 Interactive Books ...........................................................................................5 2.2.1 Features of Interactive Books...........................................................7 2.3 Educational Video Games ..............................................................................9 2.4 Reading Software.........................................................................................11 2.5 Summary......................................................................................................14 3. PROBLEM STATEMENT .......................................................................................15 4. EXPERIMENT AND ANALYSIS ...........................................................................18 4.1 Introduction..................................................................................................18 4.2 Development of SimBuilder Science Program..............................................18 4.3 Method.........................................................................................................19 4.3.1 Participants....................................................................................19 4.3.2 Procedures.....................................................................................20 4.3.3 Materials........................................................................................23 4.4 Analysis .......................................................................................................26 x 4.4.1 Measurement .................................................................................26 4.4.2 Results and Discussion...................................................................27 5. CONCLUSION AND FUTURE WORK ..................................................................31 5.1 Conclusion ...................................................................................................31 5.2 Future Work.................................................................................................32 REFERENCES..............................................................................................................34 APPENDICES...............................................................................................................36 APPENDIX A End of Program Evaluation ........................................................37 APPENDIX B End of Program Evaluation Results ............................................39 APPENDIX C Evaluation Item Descriptions and Score Summary......................41 APPENDIX D Results of Paired t-test................................................................42 xi LIST OF FIGURES Figure 1 Intelligent Animated Agent................................................................................6 Figure 2 3D GameStudio Images ...................................................................................10 Figure 3 Squeak SimBuilder Game................................................................................11 Figure 4 Example of Fourth Grade Student Work ..........................................................23 Figure 5 Paint Palette.....................................................................................................24 Figure 6 Painting with Handles......................................................................................24 Figure 7 Car Script ........................................................................................................25 Figure 8 Reading and Response Assignment Results .....................................................28 Figure 9 Averages of Final Grades.................................................................................29 1 1. INTRODUCTION It is documented that more than 8 million U.S. students in grades 4-12 struggle to read, write, and comprehend adequately (Guensburg, 2006). Many ideas have been set forth on how to break this seemingly never ending cycle. On January 8, 2002, President George W. Bush signed the No Child Left Behind Act with the goal of encouraging proven education methods (U.S. Department of Education, 2005). Reading programs such as Reading First and Early Reading First are examples of the many adolescent literacy initiatives that resulted from this act being passed. Other initiatives, such as Reading Next and Striving Readers, incorporate scientific research to identify and replicate strategies that improve reading literacy in adolescents (U.S. Department of Education, 2005). Reading deficiencies in elementary aged students across America have inspired many teachers to take their own initiatives to ensure that their children are not left behind. The following study was brought about by a teacher in Auburn, Alabama who recognized a problem with reading literacy in her students and set out to find a solution. In order to combat an obvious lack of interest and improvement in reading literacy, the SimBuilder Science Project was created for a group of nineteen fourth and fifth grade students who were deficient in reading, but immensely enjoyed and were successful at science, math and technology. The program was designed to teach reading comprehension and writing skills by using visual programming, computer simulations, and science as somewhat of a 2 decoy. The hypothesis was that by intriguing the students? interests in science and technology, reading and writing skills could be taught in a manner in which the students were able to relate and have fun without focusing on their deficiencies. The SimBuilder Science Project set out to investigate the extent to which the e- learning tool, Squeak SimBuilder, could be effectively used across the curriculum to teach reading comprehension and writing, as well as science, technology, and problem solving skills. To assist the nineteen fourth and fifth grade students, the program?s team consisted of six students from Auburn University?s Samuel Ginn College of Engineering and an elementary school technology teacher. The findings of the study were compiled by recording the students? success on reading and related response questions along with computer lab work, including simulations and animations. The students were also given surveys at the end of the program to further evaluate their progress. Overall, over 80% of the students successfully completed the program with documented improvements in reading and writing skills. The rest of this thesis is organized as follows: Chapter 2 consists of a literature review, which briefly discusses current technologies that are being used to teach reading comprehension, such as reading tutors and educational video games. A definition of the research problem along with a synopsis of the literature review is present in Chapter 3. The experiment details and analysis are included in Chapter 4. Chapter 5 houses future works and conclusions. 3 2. LITERATURE REVIEW 2.1 INTELLIGENT TUTORING SYSTEMS An intelligent tutoring system (ITS) is educational software containing an artificial intelligence component (Hafner, 2004). Most often the software tracks students? work giving feedback and hints along the way. The software can make inferences about a particular student?s strengths and weaknesses by collecting information on their performance, it can then suggest additional work if necessary (Hafner, 2004). Intelligent tutoring systems allow the emulation of a human teacher in that an ITS knows what to teach and how to teach it. It is also capable of learning certain teaching relevant information about the student that is being taught. This requires a combination of three models, the Expert Model, which is a representation of a domain expert?s knowledge, the Instructional Model, a representation of the instructor?s knowledge, and the Student Model, the particular student that is being taught (Human Performance Center, 2002). In the Expert Model the guidelines for acceptable performance are clearly identified. If the input of the student does not meet the guidelines, the computer will identify the student?s mistake and recommend a possible solution. There is a very simplistic dialogue between the student and the ITS. A detailed explanation of the computer?s reasoning is not provided. 4 The Student Model establishes the structure for identifying the student?s misconceptions and below average performance. The structure of the student model can be derived from four main attributes, (1) the problem solving behavior of the student, (2) direct questions asked from the student, (3) historical data, based on assumptions of the student?s assessment of his skill level, novice to expert, and (4) the difficulty level of the content domain (Barr & Feigenbaum, 1981). ITS compares the actual performance of the student to the student model to determine if the content domain has been mastered. The intelligent tutor goes about solving problems similarly to the human. The tutor predicts student performance and if the performance does not meet the prediction the system must then determine if the deficiency is due to a misconception or a missing conception. Once the tutor makes this determination, it forms an analysis and prescribes instructional remediation. In the Instructional Model, ITS actively interacts to student inputs and diagnoses the student?s level of understanding or misunderstanding of the knowledge domain (Human Performance Center, 2002). The selection and sequencing of information is controlled by the tutorial by responding to student questions related to the subject area. Selection and sequencing is also controlled by determining when the student needs help and what kind of help is needed (Halff, 1988). An effective ITS will meet the constantly changing needs of a student. The ITS adapts the instruction of the student according to the diagnoses of the student?s weaknesses. Ideally, the ITS will conform to the evolving skill level of the student as the level of proficiency increases. As the novice evolves into a subject matter expert, the ITS continues to adapt. 5 Traditionally, computer assisted instruction provides a constrained format of individualized instruction. Current research has been geared towards intelligent tutoring systems which assume the active role of instructor by responding directly to the student. 2.2 INTERACTIVE BOOKS Interactive books are made up of perceptive intelligent animated agents (Center for Spoken Language Research, 2006). Researchers believe that a comprehensive reading program can be provided by a combination of interactive books and tutors. Interactive books help children learn to recognize words, read fluently and understand what they read. They help identify lacking or weak foundational reading skills, and provide an individualized sequence of lessons that assess and teach these skills (Center for Spoken Language Research, 2006). In an interactive book environment, students use natural communication skills to interact with intelligent animated agents. Human communication technologies such as speech recognition, natural language processing, speech generation, character animation, computer vision, and dialogue modeling, allow students to engage in natural face-to-face spoken dialogue interaction with perceptive animated characters that behave much like effective teachers. 6 Figure 1 Intelligent Animated Agent Interactive books are designed to serve as learning tools, authoring tools, and test beds. As a learning tool, interactive books help children learn to read fluently and effortlessly, help them develop effective strategies to understand what they read, and allow them to apply knowledge gained from reading to real world situations. Authoring tools are necessary for effective and immersive learning experiences. These experiences include dialogue interaction with animated characters such as comprehension training and sounding out words, having students read aloud with real time feedback on the pronunciation of each word, story narrations by animated characters, and the ability of the students to create or change their own animated stories. Interactive books also serve as test beds for research and development of other educational technologies. These books have led to further research and development of perceptive animated interfaces, research, development, and evaluation of core technologies in learning tasks including, speech recognition, speech generation, natural language processing, computer vision, computer animation, and dialogue modeling and 7 management. They have also contributed to education research in reading instruction, second language learning, knowledge acquisition in content areas, and comprehension training, among other things. (Center for Spoken Language Research, 2006) 2.2.1 FEATURES OF INTERACTIVE BOOKS In order to provide engaging and immersive learning experiences, interactive books incorporate leading edge speech, language, computer vision and character animation technologies. The main features of interactive books in terms of the learning experiences they enable and the technologies that enable them are as follows: animated speech, speech recognition, natural language processing, computer vision, and face-to- face conversations (Center for Spoken Language Research, 2006). Animated speech is defined as a three dimensional character?s ability to produce natural speech, a wide variety of facial expressions and emotions, and natural body movements (Center for Spoken Language Research, 2006). The animated character can serve as the book?s narrator or can be used to engage the user in conversational interaction. For example, an animated character can narrate an entire story while individual words are highlighted. Along with telling stories, the character can pronounce words that are selected by the student, or give hints to help the student decode the word. Finally, the character can train and test comprehension by engaging in dialogue with the student. Speech recognition allows students to read aloud while receiving feedback about their word recognition or pronunciation accuracy (Center for Spoken Language Research, 2006). In an application created by the Center for Spoken Language Research (2006) the 8 recognizer determines if words are pronounced accurately and provides immediate visual feedback about correct and incorrect pronunciations. Speech recognition systems are also used to process speech, as illustrated in the following examples: the animated agent may ask specific questions like, ?What did Mary eat for breakfast?? Or, the agent may attempt to guide the student into making inferences by asking a question like, ?Why did Mary get angry?? The agent may also ask the student to simply summarize what he just read. Reading aloud, providing specific answers to questions, producing open-ended responses and summarizing stories all allow the speech recognition system to process the speech. Utterances that are transcribed by the speech recognition system or typed by the student are processed to interpret semantic content; this is known as natural language processing (Center for Spoken Language Research, 2006). When students converse with animated agents, semantic parsing techniques are used to assign word strings to semantic frames and interpret the users intended meaning. Latent Semantic Analysis is used to analyze the summaries produced by students and provide feedback about their quality and conciseness (Center for Spoken Language Research, 2006). Computer vision enables interactive books to locate and identify the student, track the student?s movements and interpret his visual behaviors (Center for Spoken Language Research, 2006). Dr. Javier Movellan of UC San Diego (2002) has developed an accurate face tracking system which has been incorporated into interactive books. This face tracking system makes it possible for the animated agent to orient to the student and move its head or eyes to follow the student?s movements. Face recognition algorithms can be used to identify individuals, to recognize visible speech, and to interpret facial 9 expressions and gestures once the location of the student?s face is determined (Center for Spoken Language Research, 2006). Finally, face-to-face conversational interaction with animated characters occurs through real-time integration of the following techniques: speech recognition, natural language understanding, speech generation, facial animation, computer vision and dialogue modeling techniques (Center for Spoken Language Research, 2006). 2.3 EDUCATIONAL VIDEO GAMES It has been very well documented that children spend hours upon hours after school playing video games that are not at all related to what is being taught in school. With students? lack of interests in academics and teachers? lack of up to date resources, the educational system in the United States has found it hard to compete with others around the world. Educational researchers are attempting to take video games, which are perceived as a problem with children and education, and convert them into an academic solution. Game authoring software such as 3D Game Studio, Alice 3D, and Squeak SimBuilder is being used by educators nationwide to enhance students? interests in learning. 3D GameStudio combines the C-Script programming language with a high end 3D engine, a 2D engine, a physics engine, level, terrain and model editors, and huge libraries of 3D objects, artwork and ready-assembled games (3D GameStudio, 2006). Users can create a simple action game in one afternoon with no previous programming experience. GameStudio provides three levels of games creation: the user can click together simple games from pre-assembled script templates, or program commercial 10 games in C-Script using the built-in compiler and debugger, or the user can include the GameStudio engine into their own software using the VC++ or Delphi interface (3D GameStudio, 2006). The usability of this software makes it ideal for classroom usage. Figure 2 3D GameStudio Images Alice is a 3D interactive graphics programming environment that was initially developed at the University of Virginia. Most recently Carnegie Mellon University has continued with version 2.0 of this project (Alice, 2006). The focus of the Alice project is to provide the best possible first exposure to programming for students ranging from middle school age students to college students. Alice is made freely available as a public service. It is designed to make the usually difficult task of computer programming easy and fun. As a free service, Alice 3D is widely used to teach beginning programming skills in classrooms across the U.S. (Alice, 2006). Squeak SimBuilder is a user interface platform that provides a multipurpose application programming environment (Squeak, 2006). It is designed to encourage text input, graphical object creation and manipulation, code generation, testing, information storage and retrieval (Williams et al., 2006). Although Squeak is widely used to develop games and other online media, it is commonly used as an educational tool to help teach 11 students how the world works. While some students work with media created in Squeak by their teachers, others are creating their own simulations and models to test their theories and deepen their understanding of math and science (Squeak, 2006). Figure 3 Squeak SimBuilder Game 2.4 READING SOFTWARE Retention, special education placement, and poor self-esteem can all be linked to failing to learn to read and write in the early years (Casey, 2001). It has been seen in many cases that a student has been found to have a behavior problem, attention deficit disorder (ADD), or a learning disability simply because he has difficulties reading and writing. Or the child may simply be written off as being lazy, belligerent, or inattentive. Educators in the U.S. and Europe have conducted fourteen years of research to create a solution to this problem that students face daily (Casey, 2001). Computers with talking 12 word processors now make it possible for a student to express ideas that he has been unable to express with a pencil and paper. Daily writing on the computer eventually leads to the student?s words growing into sentences and stories that are familiar to him. These words are added to the child?s vocabulary and become words that he can now recognize and read. The student has now gone from being a writer to a reader. Several programs are available today that make this process possible for students. Research has found that good readers read strategically while struggling readers lack critical comprehension strategies to make sense of the text. According to the National Reading Panel (2006), comprehension is dependent on active and thoughtful interaction between the text and the reader. Auditory feedback alone is not enough to ensure comprehension. Read:OutLoud is a program that was developed by the Don Johnston Company (2006) which uses strategic comprehension supports to further thoughtful interactions between reader and text. These supports improve comprehension skills so readers gain true understanding of concepts in social studies, science and language arts (Don Johnston Company, 2006). Read:Outloud allows the student to increase knowledge in a variety of different subject areas by creating an information outline to organize and clarify content understanding. A Franklin Dictionary is available to offer the student word definitions that he can understand, therefore building his vocabulary knowledge. There is also available assistance with homonym awareness (Don Johnston Company, 2006). Research-based comprehension strategies such as guided reading practice are used to further enhance students? recognition and comprehension. This program is easy to implement in a classroom environment and ranges in interest level from age three to age twelve. 13 JumpStart Reading with Karaoke is a program developed by Knowledge Adventure to help children develop important learn-to-read skills. This program caters more to early readers between the ages of four and seven years old. Val, the animated JumpStart cat, encourages the student to speak each new letter, sound the word out loud, and provides instant feedback and encouragement. After the student masters new words, he uses them to read and sing up to twenty original Karaoke songs in the interactive Karaoke Clubhouse (Knowledge Adventure, 2006). The student becomes an enthusiastic and confident reader by participating in eight beginning reading activities while working along with Val and other JumpStart characters. Some of the highlights of this software are the previously mentioned eight learn-to-read activities, more than 200 early reading words, 24 original read-along Karaoke songs and progress reports to track reading success. Several skills are emphasized by JumpStart including gaining phonemic awareness, identifying consonant sounds, vowel sounds, digraphs and blends, blending letters together to sound out words, reading word families and sight words, building vocabulary and practicing comprehension. Reading Blaster is another Knowledge Adventure product that provides over seventy five mystery reading passages which develop critical thinking skills (Knowledge Adventure, 2006). These skills include reading for details, finding the main idea, making inferences, and drawing conclusions. The student is able to discover word meanings and build comprehension skills over increasing levels of difficulty by applying context clues and grammar rules (Knowledge Adventure, 2006). Reading Blaster is correlated to state and national educational standards and emphasizes skills such as developing critical- 14 thinking skills, building reading comprehension, using antonyms and synonyms, working with analogies, improving grammar skills, and building vocabulary. 2.5 SUMMARY Teaching reading has proven to be a daunting task. For teachers, it is a struggle to evaluate the ability of available products to address instructional needs. Also, in most situations it is difficult for teachers to find funding for these products and to get their students motivated to use them. For students, the inability to read and comprehend can cause self-esteem issues which leads to isolation from peers. While there are many programs and technologies available for teaching reading, many parents and teachers do not have the necessary training to use these resources properly. To combat this problem, educators have conducted extensive research in order to develop products that are easy to use and understand, affordable, and fun for students. Even products that were not developed specifically for educational use have found their way into the classroom. By making learning a game in which every student is able to participate, teaching a difficult subject such as reading has become more fun for both the teacher and the student. 15 3. PROBLEM STATEMENT Reading literacy has been a problem across the United States for years. According to the National Assessment of Educational Progress (2004), only three out of ten eighth graders read at or above grade level. Many Americans tend to place the blame on their local educational systems, while local educators blame the federal government. Parents point the blame at teachers and teachers feel that parents should be taking more responsibility for their child?s development in the school system. With so many fingers being pointed, the result is a lack of progress in actually finding a solution to the problem at hand. Readers who fall significantly behind risk failure in school and in the workplace (Guensburg, 2006). Therefore, reading is not only a problem that must be dealt with in the educational system, but it is a problem for society as a whole. In 2003, only three- fourths of high school students graduated in four years; the previous year, just over half of African American and Hispanic students graduated at all (National Center for Education Statistics, 2004). With the percentage of students completing high school and being able to make a significant contribution to society dropping, programs to help improve reading literacy have become imperative. It is well known that books have taken a back seat to technology (Squire, 2003). However, even in a technological based world, reading skills are still essential. In higher 16 education and the workplace, young people must handle an array of complex texts across technologies, such as narratives, repair manuals, scholarly journals, maps, graphics, and more (Guensburg, 2006). With the majority of information that is taken in on a daily basis being presented using some form of technology, it seems that it is only natural to teach students how to read and comprehend this information by also using technology. After just over thirty years on the scene, video games have become one of the most pervasive and profitable forms of entertainment in the United States and across the world (Squire, 2003). By observing this billion dollar entertainment industry and the effect it is having on children and adults alike, educators have decided to take something that was initially designed for entertainment purposes and turn it into something that could be used for education. Therein lies the creation of the educational video game. From familiar game systems that incorporate education to programs that allow children to create their own games and use them for educational purposes, the educational video game is quickly becoming as enticing and lucrative as video games that are intended for entertainment (Squire, 2003). This research attempts to use an educational video game as an engine to improve reading literacy. During this research a group of participants were selected due to their deficiencies in reading, but great interest and acceleration in math, science, and technology. The students? scores on the 4 th ? 6 th Stanine on the Stanford Achievement Test ? 10 and Alabama Reading and Math Test were used to assist in making the determination of the students? eligibility for the program. The students? primary teachers also contributed their recommendations based on observations of difficulty in written expression of 17 science concepts and steps in scientific problem solving. The teachers also observed interests in technology with a fundamental understanding of computer usage. The Squeak SimBuilder software was used to allow the selected group of fourth and fifth grade students to create computer simulations and animations that would help them to solve a specific science challenge. The science challenge, which was assigned according to grade level, forced the students to incorporate critical thinking and problem solving skills. In addition to enhancing these particular skills, the students were given related reading materials and journaling assignments in order to enhance reading literacy, comprehension and writing skills. The students were observed and their progress was recorded throughout the program. The students were also surveyed at the end of the program to evaluate their personal outlook on their progress. From these results, a determination was made as to the effectiveness of this particular e-learning technology in the improvement of reading literacy. To summarize, the purpose of this research was to: (1) create a program that introduces an innovative solution to the overwhelming problem of reading literacy in elementary aged students who are deficient in reading; (2) establish shortcomings and thus future improvements. 18 4. EXPERIMENT AND ANALYSIS 4.1 INTRODUCTION With reading literacy being such a huge problem with American students today, it is necessary for educators and researchers to find creative solutions to combat this overwhelming quandary. The SimBuilder Science Project was created with the intention of making reading fun by relating it to topics in which the students felt confident and successful. By employing unique and exciting activities to shape the students? learning experiences, the SimBuilder Science Project set out to successfully enhance reading literacy and writing skills without the students ever realizing what was actually taking place. 4.2 DEVELOPMENT OF THE SIMBUILDER SCIENCE PROJECT A technology teacher at an elementary school in Auburn, Alabama was interested in using technology to improve reading comprehension in a group of struggling fourth and fifth grade students. This idea was immediately viewed as a great opportunity to study techniques for the improvement of reading literacy using an e-learning technology for novice programmers. The collaborating teacher in this project, who will be referred to as Rose, approached an Auburn University faculty member from the Computer Science and Software Engineering Department about starting a program that would teach students 19 computer programming while also focusing on improving reading literacy. A team of Auburn University graduate students and faculty was assembled to create a program that addressed the specific needs of Rose and her students (Williams et al, 2006). The collaboration of Rose and the Auburn University team brought about the SimBuilder Science Project. This project allowed a group of nineteen fourth and fifth grade students to come together with six Auburn University students and learn to create computer based simulations using the Squeak SimBuilder software. The students were found to be deficient in reading, but above average in math and science. The SimBuilder software was chosen because in previous studies, when it was used by teachers to build their own educational materials, the participating teachers found that the environment was easy to use and gave them the ability to have more flexible lesson plans (Seals, 2002). Because the majority of educational software has fixed behavior and mainly supports drill and practice routines, most often it does little to improve students? higher- level problem solving skills. By allowing students to bring to life their own understanding of a topic using a constructivist style of learning, the students were able to create their own educational framework and therefore have a better perception of the particular topic (Williams et al, 2006). 4.3 METHOD 4.3.1 PARTICIPANTS Six Auburn University graduate students were chosen to take part in the SimBuilder Science Project. The author, a Ph.D. student in the Computer Science and Software Engineering (CSSE) Department at Auburn University, was chosen as the 20 graduate student leader of the project and the lead instructor for the fourth grade group. The lead instructor for the group of fifth graders is also a Ph.D. student in the CSSE Department at Auburn University. The remaining four members of the team consisted of two undergraduate students, an Aerospace Engineering major and a CSSE major, and two CSSE graduate students. The team was assisted by the school?s technology instructor, and overseen by two CSSE professors. The group of nineteen student participants consisted of nine fourth grade students and ten fifth grade students. 4.3.2 PROCEDURES The two group leaders initially met with Rose, the technology instructor, to discuss the SimBuilder software, classroom rituals and routines, and lesson plan outlines. First, the software was installed on all of the computers in the school?s computer lab. Next, the group met several times to discuss Rose?s goals and expectations for her students. It was decided that the class would meet twice weekly for forty five minutes each class period. The group also decided on the format of the program which consisted of two overall science problems, based on grade level, that the students would be solving using the Squeak software, weekly reading assignments related to the science problems, and journal writings that allowed the students to analyze and better comprehend their reading. The topics that were selected for the students were aligned with guidelines created by the Alabama State Department of Education (ALSDE). The fourth grade students explored the topics of electricity, forces and motion. The overall science problem that the fourth graders were challenged to solve was to create an economical way for civilians to 21 travel to different locations in space. The state objectives that were directly related to this project were as follows: ? Explain how force affects speed and direction of motion (ALSDE objective I-1, VIII-1) ? Differences between conductors and non-conductors (ALSDE objective I-1, II-3) Similarly, the fifth grade students focused on topics related to the Alabama State Department of Education guidelines, mainly energy and ecosystems. Their science challenge was to create a mode of transportation that would assist in decreasing air pollution and other environmental hazards. The following state objective was the focus of this challenge: ? Relate natural forces to slow changes in the Earth?s surface (ALSDE objective I- 1, VII-2) After establishing the science topics that were to be covered an outline of the daily class schedule was made. It was decided that the science articles and the related journal writings would be given as take home assignments. Rose designated one day per week during the students? lunch time to discuss their articles in more detail and review the journal entries. The forty five minutes of class time were spent as follows: ? Discussion of current assignment 5 minutes ? Computer lab work 25 minutes ? Journaling activity related to current assignment 10 minutes ? Wrap up 5 minutes Now that the daily rituals and routines for the program had been initiated, the entire team of six met to finalize the plan and decide who would work with each group. 22 The team of Auburn University students was split into two teams of three in order to allow for more individualized instruction. The Auburn University team began working with the students on February 23, 2006. The first four weeks of the program were dedicated to training the students on how to use the Squeak SimBuilder software. Three tutorials were used to teach the students how to manipulate the software including the Painting Tutorial, the Handles Tutorial, and the Make Your Own Car Tutorial. A more detailed discussion of the tutorials follows in the section discussing the materials that were used. Throughout the completion of the tutorials the students read articles and wrote journal entries in preparation for solving their science challenge. They were also given in class journaling assignments related to each tutorial. After completing the tutorials the students were now able to apply everything they had learned and begin solving their science challenge. The students worked diligently for the remaining four weeks of the program creating and animating simulations related to their challenges. The fourth grade students used the visual programming skills they were taught to create and animate space vehicles, simulating the vehicle slowly escaping Earth?s gravitational pull and going on into space. The fifth graders created environmentally conscious vehicles that helped to decrease pollution and other environmental problems. Throughout the entire process the graduate student team formally evaluated the students and tracked their progress in reading comprehension and writing. 23 Figure 4 Example of Fourth Grade Student Work 4.3.3 MATERIALS As stated previously, in order to teach the students how to properly use the software, three tutorials, which were provided on the SqueakLand website, were utilized. The Painting Tutorial was the first that was introduced to the students. This tutorial allows students to explore how to create a painting in the Squeak environment. The software provides a paint palette which gives the student the opportunity to select the type and size of paint brush to be used and the desired colors. The student is also able to keep or toss the painting after it is complete. 24 Figure 5 Paint Palette The second tutorial introduced the student to what is referred to in the Squeak environment as ?handles?. The handles are used to bring the painting to life. Now that the student knows how to create a painting, the Handles Tutorial teaches him how to manipulate that painting by doing things such as rotating, making copies, repainting, or resizing. Figure 6 Painting with Handles 25 The final introductory lesson for the students was the Make Your Own Car Tutorial. This tutorial shows students how to animate their paintings and create simulations. The students drew a car and learned how to ?drive? it around a track. The tutorial focuses on teaching the student how to script the actions that he wants his car to perform. This is the initial introduction to visual programming. Figure 7 Car Script Aside from the tutorials, other materials that were used consisted of the weekly science articles and the student journals. The articles were chosen from various sources such as children?s magazines and science related websites. Along with each article the students were given a ?Thinking Like a Scientist? question. This question allowed students to reflect on the information presented in the article and write about how it directly related to their own experiences, thus enhancing their reading comprehension skills. 26 Each student was issued a folder to use as their journal. The journal was made up of sheets of grid paper which the students were required to use for standardized testing. This type of paper was provided with the journals so that the students could have additional practice on properly using the grids. The students were required to copy the specified question and write in paragraph format a response to that question. This process helped to enhance the students? writing skills. 4.4 ANALYSIS 4.4.1 MEASUREMENT The Auburn University students used a grading rubric to formally assess the students. The rubric was made up of five criteria each worth ten points. The grading criteria were as follows: (1) completion of journal drawing; (2) completion of computer drawing; (3) completion of computer animation; (4) proper saving of animated project; and (5) completion of daily in class journal entry. The grades were converted to a 100- point scale. The students were also evaluated at the completion of the program using a ten item survey. 1 The students were asked various questions about the effectiveness of the program in teaching problem solving skills, reading comprehensions skills, and new computer skills. The survey results, along with results from the grading rubrics, were used to summarize the findings of this study. 1 The entire end of program evaluation appears in Appendix A. 27 4.4.2 RESULTS AND DISCUSSION Of the original nineteen participants, two students decided to withdraw from the program, leaving a total of seventeen students, nine fourth graders and eight fifth graders. The progress of the students was measured by (1) the students? average scores on the initial reading and response assignment versus the average scores on the final reading and response assignment; (2) the students? overall grade average which was calculated at the end of the program, 70% being a passing grade and 80% being above average; (3) the results of the survey. All grades are on a 100-point scale. The first grade the students received was on February 23, 2006, the first day of the program. As previously stated the students were given their reading and writing assignments as homework. The reading and writing grades were also calculated using a fifty point rubric. The rubric consisted of six criteria which were as follows: (1) completed and turned in Thinking Like a Scientist question when due (5 points); (2) used several details from the reading and/or student?s own life to support solution (10 points); (3) entire response is written inside the answer grid (10 points); (4) answer is well organized and easy to understand (10 points); (5) copied the Thinking Like a Scientist question and wrote date at the top of the page (5 points); (6) used science words to explain solution (10 points). The average grade of the fourth graders on the initial reading and response assignment was 73%, with the lowest grade being 2% and the highest 100%. The average of the fifth graders was 15%, with eight students scoring 2%, the lowest of the grades, and the highest grade being 80%. From this initial assessment it was determined that while the fourth graders average score was a passing grade, several of the students 28 still needed improvement on reading comprehension and writing skills. The fifth grade students obviously had a huge barrier to cross. The results of the final reading and response assignment, which was graded on April 26, 2006, looked much more promising and showed the vast improvements that were made during the course of the eight week program. The fourth grade average was 83%, a 10% increase from the first week. While the fifth graders made an outstanding 77% increase with an average of 92%. Grade Level Reading & Response 2/23/06 Reading & Response 4/26/06 Improvement 4 73% 83% 10% 5 15% 92% 77% Figure 8 Reading and Response Assignments Results A paired t-test was conducted to decide if there was a significant difference in the students? improvement from week one to week eight. 2 The results suggest that the fourth graders did not show a significant improvement. Because the fourth grade students began week one with a passing average and completed week eight with only a 10% improvement, the results of the paired t-test were to be expected. On the other hand, the results of the fifth graders? paired t-test showed quite a significant difference in the students? scores from week one to week eight. With a p-value far less than 0.05, the test suggests that the fifth graders? improvement was in fact very significant. 2 Complete results from the paired t-test appear in Appendix D. 29 Significant progress was also noted by the students? final grade in the program. The final grade was calculated using the students? reading and response grades, computer lab grades, and total attendance grades for five working lunch sessions and thirteen computer lab sessions. The fourth graders? final average was 84%, well above passing and also above average. The final average of the fifth graders was 79%, also above passing. Grade Level Average of Final Grades 4 84% 5 79% Figure 9 Averages of Final Grades Finally, the students participated in a survey for further evaluation of the program. 3 The survey consisted of ten items of which the students were asked if they (5) strongly agree; (4) agree; (3) disagree; (2) strongly disagree; or (1) don?t know. The average scores of selected items from the survey are as follows. The first item stated that the SimBuilder Science Project helped the student to become a better problem solver. The average score for this statement was 3.93, with the fourth grade average being 3.71 and the fifth grade average 4.14. While the fourth graders on average did not agree that the program improved their problem solving skills, the fifth graders agreed that the program was helpful in this area. 3 Complete results of the survey appear in Appendix B. 30 The second item stated that the program made it easier for the student to read and understand science articles. The overall score for this item was 4.36, with the fourth grade score of 4.29 and the fifth grade score of 4.43. All of the students agreed that the program made reading and comprehending an easier task. The statement that meeting with Rose for lunch really helped the student to understand the science articles was the third item on the survey. Overall, the students agreed that the lunch meetings were helpful with a score of 3.43. However, the fifth graders expressed strong disagreement on this item with a score of 2.43. The fourth item on the survey stated that the student learned new skills using the computer. While the fourth graders declared strong disagreement of this statement with a score of 2.71, the fifth graders strongly agreed with a score of 4.57. The score of the entire class was 3.64 which expressed partial agreement on this item. Item five stated that Squeak was an easy program to learn and use. Collectively the students showed only partial agreement to this item with a score of 3.57. Apparently, the fourth grade students found Squeak to be a little more difficult than did the fifth graders. The fourth grade score was 3.43 and the fifth grade score was 3.71. Item six discussed the convenience of the meeting time and seven asked the students to express their interest in an after school program instead of missing class time during the school day. Items eight and nine both discussed the program being offered to fourth and fifth grade students during the next school year. Finally, item ten stated that the student felt successful while in the program. All of the students agreed to this statement with an overall score of 4.71. The fourth graders strongly agreed with a score of 4.86 and the fifth graders also expressed agreement with a score of 4.57. 31 5. CONCLUSION AND FUTURE WORK 5.1 CONCLUSION This study brought about the creation of the SimBuilder Science Project. This program was designed for a group of fourth and fifth grade students who struggled with reading literacy but excelled in science and math. The program specifically targeted the students? success and interest in science in order to enhance reading comprehension and writing skills. The program began on February 23, 2006 and continued for eight weeks. The students were challenged to solve science problems related to electricity, motion, forces, energy and the ecosystem. The beginning of the program introduced students to the Squeak SimBuilder Software and reading and responding to science related articles. The second half of the program allowed students to apply what they had learned in the introduction and begin using visual programming to create and animate simulations while continuing to enhance their reading comprehension skills through reading and response activities. The SimBuilder Science Project concluded on April 26, 2006. With seventeen participants fully completing the program and fifteen of those participants finishing with a final grade average above 70%, 88% of the students successfully completed the entire program. The results of the study suggest that the students greatly enhanced their reading 32 comprehension and writing skills, while also learning new computer skills and further advancing their science and problem solving skills. The students all expressed feelings of success while in the program. At the school?s end of the year awards ceremony twelve students from the SimBuilder Science Project were awarded for their above average performances. Ten students, six fourth graders and four fifth graders, were given Achievement Awards for Innovation, Imagination, and Technology. The remaining two students, a fourth grader and a fifth grader, were given Founder?s Awards for Outstanding Leadership and Commitment to Science. 5.2 FUTURE WORK The future plans of the project are to continue this program as an outreach activity for the local elementary schools. Based on the findings of this study, the materials will be refined to support reading reinforcement, science and e-learning and improved computer literacy. Creating a more age appropriate minimalist tutorial (Seals, 2002) is the first step that will be taken. This will facilitate the fourth and fifth grade groups in their understanding of visual programming and integrate materials to support science learning and reading improvement. In future iterations of the program the research team would like to pose the following questions to further evaluate the students? progress: (1) How did the SimBuilder Science students perform versus their counterparts? performance in science class? (2) Did the program affect the students? future occupational choices? For 33 example, after completing the program does the student now want to be a computer game designer rather than a veterinarian? Work will be done in the next year to expand the program and provide materials to support many different age levels such as middle and high school. Furthermore, research will be done to support the use of Squeak SimBuilder on a PDA with the intent of porting it to a device that may be more accessible to all students. The Auburn University Team has already created a prototype Web space for area teachers to share resources. During the next year work will be done to refine the Web space to support computer supported collaborative work of teachers and their students in an online e- learning community (Williams et al, 2006). 34 REFERENCES 3D Game Studio (2006). Retrieved June 3, 2006 from http://www.3dgamestudio.com/. Alabama State Department of Education (2006). Retrieved June 9, 2006 from http://www.alsde.edu/. Alice (2006). Retrieved June 3, 2006 from http://www.alice.org/. Barr, A. and Feigenbaum, E.A. (1981). The Handbook of Artificial Intelligence, William Kaufman, Inc., Los Altos, CA. Bartlett, M. S., Movellan, J. R. and Sejnowski, T. J. (2002). Face recognition by independent component analysis. IEEE transactions on neural networks, 13(6):1450? 1464, 2002. Biancarosa, G. and Snow, C. (2004). Reading Next ? A Vision for Action and Research in Middle and High School Literacy: A Report to Carnegie Corporation of New York. Alliance for Excellent Education. Casey, J. (2001). Technology empowers reading and writing of young children. Reading Online, 5(3). Available: http://www.readingonline.org/electronic/elec_index.asp?HREF=casey/index.html Center for Spoken Language Research (2006). Retrieved June 1, 2006 from http://cslr.colorado.edu/beginweb/reading/reading.html. Don Johnston Company (2006). Retrieved June 9, 2006 from http://www.donjohnston.com/. Guensburg, C. (2006). Why Johnny (Still) Can?t Read? Edutopia Magazine. Hafner, K. (2004). Software Tutors Offer Help and Customized Hints, the New York Times (September 16, 2004). Halff, H.M. (1988). Curriculum and instruction in automated tutors. In: Polson, M.C.; Richardson, J.J. (eds.). Foundations of Intelligent Tutoring Systems. Hillsdale NJ: Lawrence Erlbaum. 35 Human Performance Center (2006). Retrieved June 1, 2006 from http://www.3dgamestudio.com/. Kay, A. (2000). Dynabook. Retrieved March 24, 2006, from http://en.wikipedia.org/wiki/Dynabook. Knowledge Adventure (2006). Retrieved June 9, 2006 from http://www.knowledgeadventure.com/. National Assessment of Educational Progress (2006). Retrieved June 7, 2006 from http://nces.ed.gov/nationsreportcard/ National Center for Education Statistics (2006). Retrieved June 3, 2006 from http://nces.ed.gov/ National Reading Panel (2006). Retrieved May 29, 2006 from http://www.nationalreadingpanel.org/. Seals, C., Rosson, M., Carroll, J., Lewis, T., and Colson, L. (2002). Fun learning Stagecast Creator: an exercise in minimalism and collaboration, in Proc. IEEE Symp. on Human-Centric Computing 2002 (Arlington VA, Sept. 2002), IEEE, 177-186. Squeak (2006). Retrieved April 15, 2006, from http://www.squeakland.org/. Squire, K. (2003). Video Games in Education. U.S. Department of Education (2006). Retrieved May 29, 2006 from http://www.ed.gov/. Williams, A., Seals, C., Rouse, K., Gilbert, J. (2006). Visual Programming with Squeak SimBuilder: Techniques for E-Learning in the Creation of Science Frameworks. 36 APPENDICES 37 APPENDIX A End of Program Evaluation 1. I believe this program helped me to become a better problem solver. Strongly agree Agree Disagree Strongly Disagree Don?t Know 2. This program made it easier for me to read and understand science articles. Strongly agree Agree Disagree Strongly Disagree Don?t Know 3. Meeting with Mrs. Lindsey for lunch really helped me to understand my science articles. Strongly agree Agree Disagree Strongly Disagree Don?t Know 4. I learned new skills using the computer. Strongly agree Agree Disagree Strongly Disagree Don?t Know 5. Squeak was an easy program to learn and use. Strongly agree Agree Disagree Strongly Disagree Don?t Know 6. Our meeting time was a good time for me. Strongly agree Agree Disagree Strongly Disagree Don?t Know 7. If getting home was not a problem, I would like to stay after school for this program instead of missing time away from my class. Strongly agree Agree Disagree Strongly Disagree Don?t Know 38 8. Fourth and fifth grade students should be allowed to participate in this program next year. Strongly agree Agree Disagree Strongly Disagree Don?t Know 9. Fourth and fifth graders should meet at the same time and in the same room. Strongly agree Agree Disagree Strongly Disagree Don?t Know 10. I felt successful while in this program. Strongly agree Agree Disagree Strongly Disagree Don?t Know Tell us anything that you think would help to make this program better. 39 APPENDIX B End of Program Evaluation Results Student Item 1 Item 2 Item 3 Item 4 Item 5 4a 5 5 5 1 5 4b 5 5 1 3 2 4c 5 4 5 4 4 4d 5 4 5 4 1 4e 1 4 5 2 4 4f 4 4 5 1 3 4g 1 4 5 4 5 5a 5 4 5 5 4 5b 5 4 4 4 4 5c 3 4 1 5 3 5d 4 5 4 4 4 5e 4 5 1 5 1 5f 4 5 1 5 5 5g 4 4 1 4 5 Avg Score 3.93 4.36 3.43 3.64 3.57 4 th Avg 3.71 4.29 4.43 2.71 3.43 5 th Avg 4.14 4.43 2.43 4.57 3.71 Rating Scale 5 Strongly Agree 4 Agree 3 Disagree 2 Strongly Disagree 1 Don?t Know 40 Student Item 6 Item 7 Item 8 Item 9 Item 10 4a 45545 b 45515 4c 12125 d 42545 4e 52145 f 44534 4g 51525 5a 52555 b 45555 5c 25525 d 54544 5e 31554 f 25525 5g 54334 Avg Score 3.79 3.36 4.29 3.29 4.71 4 th Avg 3.86 3.00 3.86 2.86 4.86 5 th Avg 3.71 3.71 4.71 3.71 4.57 Rating Scale 5 Strongly Agree 4 Agree 3 Disagree 2 Strongly Disagree 1 Don?t Know 41 APPENDIX C Evaluation Item Descriptions and Score Summary Item Description Overall Average Score 4th Grade Average Score 5th Grade Average Score 1 I believe this program helped me to become a better problem solver. 3.93 3.71 4.14 2 This program made it easier for me to read and understand science articles. 4.36 4.29 4.43 3 Meeting with Mrs. Lindsey for lunch really helped me to understand my science articles. 3.43 4.43 2.43 4 I learned new skills using the computer. 3.64 2.71 4.57 5 Squeak was an easy program to learn and use. 3.57 3.43 3.71 6 Our meeting time was a good time for me. 3.79 3.86 3.71 7 If getting home was not a problem, I would like to stay after school for this program instead of missing time away from class. 3.36 3 3.71 8 Fourth and fifth grade students should be allowed to participate in this program next year. 4.29 3.86 4.71 9 Fourth and fifth grade students should meet at the same time and in the same room. 3.29 2.86 3.71 10 I felt successful while in this program. 4.71 4.86 4.57 Rating Scale 5 Strongly Agree 4 Agree 3 Disagree 2 Strongly Disagree 1 Don't Know 42 APPENDIX D Results of Paired t-test 4th Grade Week 1 Week 8 Difference 2 100 98 72 0 -72 98 98 0 100 98 -2 84 100 16 60 94 34 80 58 -22 96 98 2 66 100 34 Average of Difference 9.777777778 Standard Deviation 46.08084683 Standard Error 16.29203964 t-ratio 0.600156763 p-value 0.282505304 43 5th Grade Week 1 Week 8 Difference 2 96 94 80 94 14 2 98 96 2 96 94 2 92 90 2 98 96 58 92 34 2 68 66 Average of Difference 73 Standard Deviation 32.24903099 Standard Error 11.40175425 t-ratio 6.402523541 p-value 0.000104286