A META ANALYSIS: THE EFFECTIVENESS OF THE USE OF MOBILE COMPUTERS ON THE ATTITUDE AND ACADEMIC OUTCOMES OF K?12 STUDENTS Except where reference is made to the work of others, the work described in this dissertation is my own or was done in collaboration with my advisory committee. This dissertation does not include proprietary or classified information. __________________________________ Kathleen M. Cassil CERTIFICATE OF APPROVAL: David M. Shannon Margaret E. Ross, Chair Professor Associate Professor Educational Foundations, Leadership, Educational Foundations, Leadership, and Technology and Technology Paris S. Strom Stephen L. McFarland Assistant Professor Dean Educational Foundations, Leadership Graduate School and Technology A META ANALYSIS: THE EFFECTIVENESS OF THE USE OF MOBILE COMPUTERS ON THE ATTITUDE AND ACADEMIC OUTCOMES OF K?12 STUDENTS Kathleen Cassil A Dissertation Submitted to the Graduate Faculty of Auburn University in Partial Fulfillment of the Requirements for the Degree of Doctor of Education Auburn, Alabama August 8, 2005 iii A META ANALYSIS: THE EFFECTIVENESS OF THE USE OF MOBILE COMPUTERS ON THE ATTITUDE AND ACADEMIC OUTCOMES OF K?12 STUDENTS Kathleen M. Cassil Permission is granted to Auburn University to make copies of this dissertation at its discretion, upon request of individual or institutions at their expense. The author reserves all publication rights. Signature of Author Date of Graduation iv VITA Kathleen Marie Cassil, daughter of Thomas and Gertrude Harvie, was born January 21, 1945, in Alma, Michigan. She is married to Charles E. Cassil, son Charles Cassil Sr. and Lois Cassil. She graduated from Bethel College in St. Paul, MN with honors in 1967 with a major in Speech Communication and from California State University, Fresno in 1980 with a Masters of Arts in Education. From 1971 to 2001 she taught special education and regular education classes (1 st ?12 th grades) in California schools. During that time she obtained the following credentials: Standard Elementary, Learning Handicapped Specialist, Resource Specialist, Administrative, and Community Colleges Instructor Credential. She entered Graduate School, Auburn University in September, 2001. v DISSERTATION ABSTRACT A META ANALYSIS: THE EFFECTIVENESS OF THE USE OF MOBILE COMPUTERS ON THE ATTITUDE AND ACADEMIC OUTCOMES OF K?12 STUDENTS Kathleen M. Cassil Doctor of Philosophy, August 8, 2005 (M.A., California State University?Fresno, 1980) (B.S., Bethel College-St. Paul, 1967) 172 Typed Pages Directed by Dr. Margaret E. Ross, Associate Professor Statistical meta analyses performed for this study included 32 primary studies conducted between 1993?2005. Two independent meta analyses were conducted regarding student attitudes and academic outcomes. The overall meta analysis mean by author was .23, indicating that student use of mobile computers had a small and positive effect on student attitudes and academic outcomes. The consistent pattern of positive effect size results indicated that student use of mobile computers was effective in improving student attitudes and academic outcomes. The small number of samples in the independent meta analyses suggests a need for further research regarding mobile computers. vi The overall meta analysis had three purposes. The first purpose was to assess the effectiveness of student use of mobile computers on attitude and academic outcomes. The second purpose was to explore the effect of specific demographic and methodological characteristics on the measures of effect size. The third purpose was to suggest new directions for research and practice in conducting and evaluating statistical meta-analyses. This dissertation was written as an alternative dissertation. Chapters one, two, three, and five follow a traditional format. Chapter four is comprised of three different but related potential journal articles addressing student use of mobile computers. Each potential journal article was written with advisement of a different committee member. The first is an independent statistical meta analyses of student attitudes. The second is an independent meta analysis of academic outcomes related to student use of mobile computers. The third article is a practitioner article. Submission of the articles for publication was required. Publication was not. vii ACKNOWLEDGEMENTS I would like to express sincere appreciation to my husband, Charles Cassil, whose sacrifices and dedication gave me the opportunity to complete this degree. I would like to express sincere appreciation to Dr. Margaret Ross for her guidance in the preparation of this dissertation. As chair of my committee, her personal interest and support of my efforts in completing this study were invaluable. In addition, I wish to thank Dr. David Shannon and Dr. Paris Strom for giving their time, insight, and valuable expertise in serving as members of my committee. I accept full responsibility for any errors or omissions. To my father, Thomas Harvie, I offer my gratitude for your encouragement and support. In memory of and in loving tribute to my mother, Gertrude Harvie. viii Style manual or journal used: Publication Manual of the American Psychological Association, 5 th editon Computer software used: Microsoft Word XP, and the Statistical Package for the Social Science (SPSS) 11.0 (2003) ix TABLE OF CONTENTS LIST OF TABLES..................................................................................................... xii LIST OF FIGURES ................................................................................................... xiii I. INTRODUCTION ......................................................................................... 1 Importance of the Study................................................................................. 5 Research Questions........................................................................................ 12 Definition of Terms........................................................................................ 12 Limitations of the Study................................................................................. 14 Delimitations of the Study ............................................................................. 15 Assumptions of the Study .............................................................................. 15 Summary of Chapter One .............................................................................. 15 II. LITERATURE REVIEW .............................................................................. 16 Attitudes of Students Concerning Stationary Computers.............................. 17 Student Achievement Concerning Stationary Computers............................. 19 Outcomes Regarding Student Behavior on Stationary Computers ............... 21 Teacher Attitudes Regarding Stationary Computers..................................... 21 Introduction to Use of Mobile Computers in K?12 ...................................... 22 Positive Student Attitudes ............................................................................. 25 Student and Teacher Beliefs about Technology............................................ 28 Benefits of Longitudinal Observations.......................................................... 29 Negative Attitudes Regarding Mobile Computers or Mobile Computing Programs ........................................................ 29 Student Academic Outcomes Regarding Mobile Computers........................ 30 Student Behavioral Outcomes Concerning Mobile Computers .................... 32 Negative Findings Regarding Student Use of Mobile Computers ................ 35 High Access Mobile Computers in the Classroom........................................ 35 Summary of Chapter Two ............................................................................. 36 III. METHODS.................................................................................................... 39 Coding the Studies......................................................................................... 42 Data Analysis................................................................................................. 43 Formulae........................................................................................................ 43 x Summary of Chapter 3................................................................................... 47 IV. RESULTS...................................................................................................... 49 Study 1: A Statistical Meta Analysis of the Effectiveness of Mobile Computers on Student Academic Outcomes......................... 49 Purpose of the Study .......................................................................... 53 Method ............................................................................................... 54 Search Procedures Used in Obtaining Studies....................... 54 Criteria for Inclusion.............................................................. 55 Coding of Important Study Characteristics............................ 56 Database Description ............................................................. 57 Instruments............................................................................. 58 Results ................................................................................................ 59 Description of Study Characteristics ..................................... 59 Effect Sizes ............................................................................ 61 Discussion.......................................................................................... 65 Limitations ............................................................................. 69 Implications for Intervention ................................................. 71 Conclusions........................................................................................ 72 Study 2: A Statistical Meta-analysis of the Attitudes of K?12 Student Mobile Computer Users.............................................. 72 Research Questions............................................................................ 77 Methodology...................................................................................... 77 Rationale for Meta Analysis .................................................. 77 Literature Search.................................................................... 78 Criteria for Inclusion.............................................................. 79 Criteria for Exclusion............................................................. 79 Literature Search Results ....................................................... 79 Calculations of Effect Sizes................................................... 80 Data Analysis......................................................................... 81 Results ................................................................................................ 82 Discussion.......................................................................................... 89 Limitations ......................................................................................... 94 xi Suggested Interventions and Recommendation for Research............ 95 Conclusion ......................................................................................... 96 Study 3: Effects of the Distribution of $100 Laptops to Students in Third World Countries.................................................... 96 School Use of Personal Laptops ........................................................ 97 Personal Use........................................................................... 98 E-nexees Will Enrich the Internet.......................................... 99 Student Self-Esteem Will Increase ........................................ 101 Implications for Practitioners............................................................. 103 School Boards and Administrators ........................................ 103 A National Technology Assessment...................................... 104 Conclusion ......................................................................................... 107 V. DISCUSSION, RECOMMENDATIONS, LIMITATIONS, AND CONCLUSIONS ................................................................................. 108 Discussion ..................................................................................................... 108 Attitude Outcomes ............................................................................. 109 Academic Outcomes .......................................................................... 114 Recommendations ......................................................................................... 115 Inverse Relationships......................................................................... 117 Negative Effect Sizes......................................................................... 118 Limitations and Need for Further Research .................................................. 119 Conclusions ................................................................................................... 121 REFERENCES .......................................................................................................... 123 APPENDICES ........................................................................................................... 149 Appendix A: Example of E-Mail Request Regarding Unpublished Research ........................................................... 150 Appendix B: Sample Survey ....................................................................... 152 xii LIST OF TABLES Tables 1 Effect Sizes Based on Meta-analyses of Academic Outcomes on Technology .............................................................................. 51 2 Academic Outcome Sample Effect Sizes by Sample/Author (N = 21) ......... 62 3 Academic Outcome Effect Sizes by Category (N = 21 articles) ................... 63 4 Academic Outcomes by Sample (N of samples = 21) ................................... 64 5 Mean Effect Size Comparisons across Meta Analyses Reported in Table 1 and the Current Study ................................................... 65 6 Summary Chart of Student Attitude Effect Sizes by Instrument................... 83 7 Types of Samples Reported ........................................................................... 84 8 Survey of Study Characteristics and Corresponding Effect Size Means....... 86 9 Descriptive Profile of Attitude Toward Mobile Computers Effect Size Means by Category (No significance test attached).................... 88 10 Preferences of Digital Natives (Prensky, 2001) Compared to Assessment Characteristics............................................................................ 105 xiii LIST OF FIGURES Figure 1 Relationship Between Technology and Area Tested By NCLB Standardized Tests.................................................................. 116 1 I. INTRODUCTION As technology transforms K?12 education, schools adopting new technologies often do so without benefit of systematic research results on the effective uses of the new devices or services, such as wireless mobile computing. Although numerous studies have been conducted in the area of mobile computers, different results have created confusion. A statistical meta analysis is designed to integrate findings of multiple studies so as to be able to make sense out of apparent disorder and contradiction. The goal of this study to is report the results of a meta analysis regarding educational research in the area of mobile computing. As emerging technologies in business and communication are introduced into the classroom, extensive research is needed to investigate the effects on students, teachers and the process of learning. Beasley and Waugh (1996) warned that educational research is lagging far behind advances in the capabilities of the new technologies (Siegle & Foster, 2001). This research is needed to help educators better prepare students for a highly technological workforce. U.S. Labor Secretary Robert Reich states that workers in a technological society need skills of abstraction, systems thinking, experimentation, and collaboration (Reich, 1991). In The New Basics: Education and the Future of Work in the Telematic Age, David Thornberg identifies the additional skills of digital-age literacy, inventive thinking, effective communication, and high productivity as necessary for the 2 new class of workers (2002). Educational research is needed to determine how technologies can be implemented in the classroom so as to foster the attitudes and traits of this new class of workers. Research in this area has built on that of the past. Researchers in the area of the use of stationary computers in education laid the foundation for this meta analysis. Multiple studies, meta analyses, and meta analyses of meta analyses have reported empirical research regarding the use of stationary computers in K?12 education. Before 1993, computer assisted instruction (CAI) was conducted primarily on stationary computers in the classroom, computer labs, or school libraries. Major topics included attitudes of students, attitudes of teachers, effects on different grade levels, effects on different subject matter, and outcomes of the uses of computers. Studies of stationary computers were not designed to answer questions regarding the mobility, miniaturization, and wireless communication of mobile computers. First mobile computers had to be conceptualized, created, manufactured, and introduced to educators. The mobile computer was first conceptualized by Alan Kay of the Xerox Palo Alto Research Center in the 1970s (Gasch, 1996). Kay?s creative thinking envisioned a new paradigm in the computer world: a personalized mobile computer. Kay is credited with being the first to envision a universally affordable notebook-sized, portable computer capable of providing for the owner?s total information needs (Kay, 1992). William Moggridge of Grid Systems Corporation is credited with designing the first laptop computer in 1979. The Gavilan Computer and the Osborne 1 vie for the claim of 3 the first truly functional laptop (History of Laptop Computers, 2003). The IBM PC Convertible, a genuine laptop computer, was released in 1986. The IBM PC Convertible and clones soon dominated the market for laptops. These first mobile computers were nick-named ?luggables? and often weighed nine to twelve pounds. Miniaturization of electronic components has enabled laptops to shrink into book-sized computers and the even smaller handheld computers. Handheld computers have been a subject of interest to educators since 1993 when they were first introduced. In August 1993, Apple and Sharp produced the Newton. When U.S. Robotics introduced the Palm Pilot 1000 and 5000 in 1996, the market for handheld computers began to intensify. By 1996, Microsoft Windows CE provided a common operating system for a variety of different microprocessors, including handheld computers. As mobile computers entered the schools, educators began research into the effects of mobile computers on teachers, students, and the learning process. Educators were introduced to laptops, handhelds, and other mobile computers by technology-oriented friends, technology coordinators, specialists at technology centers, and speakers at computer conferences. Initially administrators, technology directors, and teachers used mobile computers as address books, calculators, calendars, and for memos as software was primarily limited to business applications. The early laptops were far too expensive for most teachers to afford. Some of the earliest mobile computers were donated to schools by Apple Computers and by Kaypro Computers. As the prices dropped, teachers wanting to use a computer at home as well as 4 at school were attracted to laptops. The excessive weight of the early laptops was a primary disadvantage. The early handheld computers dealt with the problem of weight. They were portable, cheaper, had a lower learning curve, but could provide only limited information. At approximately $500 per unit, handhelds were considered too expensive to entrust to students. Educators also hesitated to put handhelds into the possession of students because no one knew whether the handhelds would benefit learning or cause problems. In 2001, Palm Computers began donating Palm handhelds to what would eventually total more than175 private and public schools. The Palm Education Pioneer (PEP) program awarded $2.3 million dollars in competitive technology grants as part of a three-year research program to ascertain the effect of a one-to-one ratio of students to handhelds in the classroom (Study: PDA?s Enhance, 2001). These studies dispelled many unwarranted concerns. As the price of handhelds has fallen, use has increased. Specialized software has enabled students to turn the handheld into a graphing calculator, a science-class measurement tool (with probes attached), and a silent communication device using infrared beaming. More recently, handhelds have allowed students to access the Internet. Wireless technology provides Internet access for mobile computing. Field trips, team messaging, joint and simultaneous input on one document, and instantaneous teacher monitoring of student progress on a mobile computer assignment are all approaches to use of wireless technology in K?12 education. 5 Importance of the Study Educational leaders need to answer questions regarding the justification of the expense of mobile computers while trying to prepare future citizens for a future technology-oriented workplace. A meta-analytic report of the synthesis of the effects of studies conducted on mobile computers in education will provide educational decision- makers with statistical insights as to the effective uses and issues in K-12 education. With the introduction of Apple Classrooms of Tomorrow (ACOT) in 1985, Apple Computers initiated the first longitudinal research project in educational computer history (Apple Computer, 2003). Early research related to attitudes of students, outcomes for students, attitudes of teachers, and outcomes for teachers regarding stationary computers. Generally, researchers found positive student attitudes towards the use of computers in the schools and positive changes in students' senses of self-efficacy (Coley, 1997; Ringstaff, Haymore, & Dwyer, 1991). When students engaged in academic tasks, student attitudes towards academic performance improved in areas such as remedial reading (Nixon, 1992), writing (Kurth, 1987; Riel, 1990), and math (Funkhouser, 1993). Other researchers reported that when students were assigned computer-based instruction activities, the effect sizes of the attitudes of the students toward the subject matter were near zero (Coley, 1997). In general, positive effects have been found for student achievement when students engage in learning activities on computers (North Central Regional Educational Laboratory (NCREL), 2002; Schacter, 2001; Sivin-Kachala, 1998; Wenglinsky, 1998). In the elementary schools, positive results were found for beginning reader computer 6 assisted instruction (CAI) programs (Blok, Oostdam, Otter, & Overmatt, 2002) and for CAI programs in general (Kulik & Kulik, 1991; Ouyang, 1993). In high school, positive effects have been found for the use of CAI in biology (Fletcher-Flinn & Gravatt,1995; Lazarowitz & Huppert; 1993; Liao,1992; Lu, Voss, & Kleinsmith, 1997). The use of computers in the classroom supports positive student behaviors such as being more engaged and more independent, assuming more academic responsibility, producing higher levels of work (Coley, 1997), and increasing student time-on-task more than students in traditional classrooms (MacArthur, Haynes, & Malouf, 1986; Schofield & Verban, 1988; Waxman & Huang, 1996; Worthen, Van Dusen, & Sailor, 1994). The introduction of computers into the classroom generated profound pedagogical challenges for computer-using teachers (Gess-Newsome et al., 2003; Riel & Becker, 2000). Teacher attitudes, beliefs, and perceptions about computers in the classroom became the focus of many studies. Many of these studies found constructivist teacher behaviors to be characteristic of teachers most willing and able to adapt to computerized instruction in the classroom (Becker & Riel, 2000). Moderate teacher incorporation of technology in the classroom resulted in much less whole-class instruction and more student independent work and on-task behavior (MacArthur et al., 1986; Schofield & Verban, 1988; Waxman & Huang, 1996; Worthen et al., 1994). When teachers saw a positive impact on learning, they were more likely to implement innovations (Sandholtz et al., 1992). These early student and teacher studies were conducted on desktop or stationary computers. 7 Stationary computers had a number of drawbacks such as bulky size, the limited number per class and the tendency to disorganize classrooms. Mobile computer advantages, such as lighter weight and lower cost, impressed many teachers, administrators, and school boards. One-to-one ?ownership? of a mobile computer by each student was encouraged by decreasing costs, increasing computer power, and capabilities, growing wireless capabilities, increased access to the Internet, and public awareness of the need for a technology-proficient workforce (Smith, 1995). A number of laptop and handheld computer companies launched educational research projects. Apple Computer, Microsoft, Toshiba, and Palm Computers designed studies that focused on the strengths of their products and the needs of educators (Rockman, 2000; Stevenson, 1998; Vahey & Crawford, 2002). The results of these studies were mixed and are reported in Chapter Two. The National Science Foundation supported the Concord Consortium (a nonprofit educational research and development organization) in conducting a two-year research project, the Technology Enhanced Elementary and Middle School Science (TEEMSS) project, during 2000 and 2001. Handheld Palm computers helped students reduce their misconceptions, and increase their graph-reading skills, and understand the lessons (Metcalf & Tinker, 2003). In general, students who ?owned? mobile computers tended to have positive attitudes towards the mobile computers and their use (Gardner, Morrison, Jarman, Reilly & McNally, 1994; Pfeifer & Robb, 2001; Siegle & Foster, 2001; Stevenson, 1998). When compared to students not using mobile computers, mobile computer-using students liked 8 school better, learned more, and exhibited higher motivation and speed in acquiring information technology literacy (Fouts & Stuen, 1997; Gardner et al., 1994; Vahey & Crawford, 2002). A majority of the laptop-using students perceived that their spelling skills, writing skills, math scores, and reading scores were improved by laptop use (Stevenson, 1998). Mobile computer-using students experienced increased self-esteem, assessed their computer-related skills more highly, and had higher confidence levels concerning productivity software than students not using mobile computers (Rockman, 2000; Vahey & Crawford, 2002; Waker, 2001). The value of computer skills in future vocational tasks was clear to nine out of ten mobile computer-using students (Newhouse, 1997). Many mobile computer programs have encouraged students to take the laptops home by assigning homework (Newhouse, 1999; Waker, 2001). Home use of mobile computers appeared to encourage more positive attitudes towards school (Waker, 2001). Use of mobile computers helped students to improve the quality of their schoolwork, made their schoolwork easier to do, made it more fun and/or interesting, and helped them understand their classes better (Rockman, 2000). Mobile computer-using students expressed some negative attitudes. Examples of negative attitudes include type of access to mobile computers, teachers who never or infrequently assigned laptop lessons, teachers unskilled in the use of computers, lack of computer training for students, perceptions that computers were not useful, and student worry or lack of confidence about using the laptops (Newhouse, 1997). Mobile computer mechanical problems such as excessive weight, lack of battery duration, limited memory, 9 faulty screens, and prolonged startup time contributed to negative student attitudes (Newhouse, 1997; Rockman, 2000; Stevenson, 1998; Vahey & Crawford, 2002). Handheld mechanical problems involved occasional breakage, cracked screens, difficulty with text input, and screen size (Vahey & Crawford, 2002; van?t Hooft, 2003). Students using laptops maintained a sustained level of academic achievement while in middle school while laptop-using at-risk students and laptop-using special education students caught up with and exceeded non-laptop-using fifth to seventh graders within two years (Stevenson, 1998). Mobile computers had to be used in a sustained and academically relevant manner if results were to be accurately measured (Newhouse, 1997). Standardized measures of academic achievement were inconclusive for the three year Rockman study of Toshiba laptop computers. When compared to non-laptop-using students, a Rockman survey of active learning strategies found that strategies such as highlighting a main idea, re-reading reports before turning them in, and asking questions to make sure they understood what they had read, were employed more frequently by laptop-using students (Rockman, 2000). Laptop-using students also performed better than non-laptop-using students on writing assessments conducted over the three year study (Rockman, 2000). Students using mobile computers exhibited responsibility, independence, achievement, and quality in their products (Fouts & Stuen, 1997; Gardner et al., 1994; Siegle & Foster, 2001). These students increased their levels of computer literacy (Newhouse, 1997; Rockman, 2000), made intensive and frequent use of the mobile 10 computers, and accomplished a wide variety of tasks (Rockman, 2000). Student behavior changed when student gained ?ownership? of mobile computers. When compared to stationary computer-using students, laptop-using students used computers more at home (Rockman, 2000; Stevenson, 1998), increased their homework completion (Vahey & Crawford, 2002), and spent more time doing homework (Rockman, 2000). These students also used computers more at school than stationary computer-using students (Ashmore, 2001; Blumenthal, 2003; Newhouse, 1997; Rockman, 2000; Stevenson, 1998; Vahey & Crawford, 2002). Mobile computer-using students also increased their use of the Internet more both in frequency and duration (Rockman, 2000), and they were more likely to use transference of computer skills to complete work for non-computer-using classes (Rockman, 2000). In some schools, wireless capabilities allowed students using laptop computers and handhelds to devise and implement strategies to increase academic collaboration. (Pfeifer & Robb, 2001; Vahey & Crawford, 2002). Non-laptop-using students using other types of technology engaged in collaboration less than the students using handhelds (van?t Hooft, 2003). The efficiency promoted by the mobile computers created higher student motivation (Newhouse, 1999; Siegle & Foster, 2001; van?t Hooft, 2003). Students using handheld computers experienced continuous improvement in motivation over the span of the study (Vahey & Crawford, 2002). Mobile computers also helped students improve their organizational skills (Pfeifer & Robb, 2001; van?t Hooft, 2003). Mobile computer-using students had higher school attendance and less tardiness than students not using mobile computers (Stevenson, 1998). Negative findings regarding 11 student use of mobile computers tended to be concerned with students using the computers for the wrong reasons. Students attending to games rather than lessons, beaming messages at inappropriate times, beaming inappropriate messages, or wanting to use the handhelds for every task whether it was the best tool or not, caused problems in classrooms (Vahey & Crawford, 2002). Chapter three presents the method of procedure, a statistical meta analysis. Multiple sources of data were found in dissertations, journal research articles, reports from companies, conference archives, government research sites on the Internet, organizational research sites, and unpublished articles from individuals such as researchers, journal editors, and conference archivists. Experimental and quasi- experimental research was reported. Criteria for excluding inappropriate data, coding the appropriate studies, computing effect sizes, and treatment of the data were described. The approach to grouping the features of individual studies and analyzing their outcomes were discussed. Experimental research statistics such as numbers, means, standard deviations, and effect sizes contributed the foundation of this meta analysis. Research regarding the effects of mobile computers on the attitudes and outcomes of students and teachers had not been evaluated with a meta analysis at the inception of this study. This meta analysis opens a new line of research by providing a better understanding of the effects of mobile computers on the attitudes and outcomes of K?12 international students and teachers. Chapters four and five present the results of the findings in journal article form. A discussion of the distribution of the effect sizes include the number of studies in the meta 12 analysis, the total number of subjects in each study, reported values of effect sizes, and the range of the effect sizes. Continuous variables are presented and interpreted for significance. Accounted-for and not-accounted-for variability will be discussed and interpreted for significance. The diversity of the studies in the sample and the implications of that diversity for the findings are discussed. Explanations for significant moderators are provided. Theoretical and applied implications of major findings for K?12 educational use of computers are discussed in detail, and the generalizability of the results is considered. Specific recommendations are made for future research. Research Questions Research questions included: 1. To what extent have the attitudes of students been impacted by the use of mobile computers in the educational environment across relevant studies reviewed? 2. To what extent have student academic outcomes been impacted by the use of mobile computers in the educational environment across relevant studies reviewed? Definition of Terms Computer Assisted Instruction (CAI): Computer Assisted Instruction is an approach to presenting educational materials to a learner and to guiding the student?s learning via a computer program. Effect size: Effect size is ?the treatment mean minus the control mean divided by the control group?s standard deviation? (Glass, McGaw & Smith, 1981, p. 29). 13 Handheld computers: Handheld computers (personal digital assistant or PDA) are lightweight, handheld computer designed for use as a personal organizer with communications capabilities (Encyclopedia.com, 2004). Internet International: The Internet International is a computer network linking together thousands of individual networks at military and government agencies, educational institutions, industrial and financial corporations of all sizes, and commercial enterprises (called gateways or service providers) that enable individuals to access the network (Encyclopedia.com, 2004). Meta analysis: ?A meta analysis seeks a full, meaningful statistical description of findings of a collection of studies, and this goal typically entails not only a description of the findings in general but also a description of how the findings vary from one type of study to the next? (Glass, McGaw, & Smith, 1981, p. 79). Personal computer (PC): A personal computer is a small but powerful computer primarily used in an office or home without the need to be connected to a larger computer. In the 1980s the first low-cost, fully assembled units were mass marketed. Smartphone: A smartphone is a device that combines phone, pager, and computer (Darwinmag.com, 2003). Text messaging: Text messaging is an abbreviated form of English used in devices such as the short message service (SMS) that comes with many mobile phones. Ubiquitous computing: Ubiquitous computing is ?the method of enhancing computer use by making many computers available throughout the physical environment, but making them effectively invisible to the user? (Weiser, 2004). 14 Limitations of the Study Effect sizes gathered for the purpose of this meta analysis depended on the accuracy of the original studies and the willingness of researcher to produce effect sizes as opposed to percentages. Many educational journals reported percentages rather than means, standard deviations, and effect sizes. Highly correlated variables limited statistical analysis. Simultaneous reform movements may have confounded the results. The pace of change in information and computer technology is so rapid that it is difficult for educators to keep abreast of the change (Warning Over Schools? Use of Computers, 1999). Educational research lags the production of new mobile devices. The information in this meta analysis was not able to address the newest of technologies. The study relied on the author?s ability to classify types of computers into the classification of mobile computing and types of educational environments into the classification of educational environments. While differences might exist among different types of mobile computers and programs used on mobile computers, it was impossible to determine causality of differences. This study will contribute to further insights in the effectiveness of mobile computers in the classroom. Delimitations of the Study The following guidelines were used to determine inclusion of studies in this meta analysis: 15 1. Experimental, quasi-experimental or pre-experimental design must have been used. 2. Data acquired from the study must be sufficient for meta analytic calculations. 3. Boundaries for inquiry were determined by negation. Stationary computer studies, computer-enabled phones, and land-line Internet connections were not included in the meta analysis. Assumptions of the Study Meta analysis relies on the soundness of the incorporated studies. The findings of these prior studies were assumed to be methodologically sound. These studies were also assumed to be representative of all studies related to educational mobile computing. Summary of Chapter One The need for improved use of technology in the schools has compelled many researchers to study the effects of the use of computers in the classroom. Researchers have found positive effects for student attitudes, academic achievement and behavior. This research will be discussed in greater detail in Chapter two. 16 II. LITERATURE REVIEW The focus of Chapter two is a literature search of the attitudes and outcomes of computer-using students regarding educational uses of computers. Attitudes and outcomes regarding stationary computers (desktop computers) provide historical insight into the transition from no computers to multiple computers in K?12 classrooms. Attitudes and outcomes regarding mobile computers (laptops, e-tablets, and handhelds) suggest predictions for the transition into approaches to education based on mobile computers. No statistical meta analysis of mobile computers in K?12 education has been conducted. This meta analysis is needed to assist educational decision makers. Computers have reshaped our world and are continuing to do so. We use computers and computerized technology to extend our senses, increase our speed, facilitate our mental capacities, improve efficiency, increase communication, and by proxy to go places we cannot otherwise go. We generate, store, and analyze huge amounts of data. We access the Internet to increase the speed of communication, research vast data bases, exchange goods and services, play games, and socialize. Computerized satellites, missiles, planes, communications, and planning enhance the security of our country. Business, industry, transportation, the communications industry, and the government are totally dependent on computers. To successfully live, work, and play in this computer- driven world, individuals must develop expertise in computer languages, 17 programs, and technology relevant to their environments. To prepare students to succeed in this world, many school decision makers have sought research-driven information to help them provide the most effective, cost-efficient, and relevant use of computers in the schools. Computer research in education came of age with the arrival of the personal computer in K?12 classrooms. Apple Computer Incorporated supported early longitudinal research (1985?1998) in the use of computers in the schools. Apple pioneered a Research-Based Staff Development Approach and created a number of staff development models to improve teaching and learning practices with computers. Apple researchers sought to understand the process of infusion of innovation into the learning process and to help educational professionals better implement innovation with computers. Although some consider research conducted by a computer company to be biased, the early Apple Classrooms of Tomorrow, ACOT, research results reported in Changing the Conversation About Teaching, Learning, and Technology ? A Report on 10 Years of ACOT Research, have generally been supported by independent research (Waxman, Connell & Gray, 2002). In addition to the ACOT longitudinal research, multiple studies, meta analyses, and meta analyses of meta analyses have reported empirical research regarding the use of stationary computers in K?12 education. Attitudes of Students Concerning Stationary Computers Research in the literature relevant to this study include topics related to attitudes of students, attitudes of teachers, achievement effects at different grade levels, and achievement effects on different subject matter. Generally, researchers found positive 18 student attitudes towards the use of computers in the schools and positive changes in students' senses of self-efficacy (Ringstaff et al., 1991). When computers are used for instruction, students develop more positive attitudes towards both their classes and the computers (Coley, 1997). Student attitudes were found to be more positive when students were given opportunities to work in groups with the computers rather than working individually with the computers (Lou et al., 2001). Middle school remedial reading students develop improved attitudes toward reading when using computer reading games to develop comprehension skills (Nixon, 1992). Middle school students? attitudes toward writing improve when they participate in writing activities involving telecommunications (Riel, 1990). Additionally students using a variety of technologies developed a positive self-image (Persky, 1992). High school students have more positive attitudes about math and about themselves as mathematicians when they use computers (Funkhouser, 1993). Their attitudes improve towards science when the computer is used as a tool to do what could not easily be done without using computers (Lavoie, 1998). Student attitudes towards self, their writing, and teachers significantly improve when students use word processing to write (Kurth, 1987). Results of studies related to attitudes of student about use of computers were not always positive. ?In 34 studies that examined students? attitudes toward subject matter, for instance, the average effect size of computer-based instruction was near zero? (Coley, 1997). These findings suggest that computers cannot be expected to change negative attitudes regarding courses or subjects disliked by students. 19 Results of studies regarding student attitudes about computer-based simulations are inconsistent. According to Lee, computer-based simulation had a negative effect on student attitudes (Lee, 1999). Research regarding Computer Assisted Instruction (CAI) simulations and tutorials, CAI supplements to traditional instruction, and individual use of CAI on an individual computer, have all produced small positive effects for CAI (Bayraktar, 2001). Student Achievement Concerning Stationary Computers In general the major studies and meta analyses have documented the positive effects of the use of computers on student achievement (NCREL, 2002; Schacter, 2001; Sivin-Kachala, 1998; Wenglinsky, 1998). Student efficiency in learning usually is increased when the students receive computer-based instruction (Kulik, 1994). The North Central Regional Educational Laboratory (NCREL) cautions that student achievement should be judged by student improvements in higher-order thinking skills as well as standardized test scores (NCREL, 2002). In the elementary schools, research regarding beginning reader CAI programs found small positive gains (Blok et al., 2002). Earlier meta analyses by Kulik and Kulik (1991) and Ouyang (1993) also found positive but small effects for CAI (NCREL, 2002). Elementary students participating in CAI gained three additional months of learning for every year of CAI instruction (Ryan, 1991). In 1993, the meta analysis by Ronghua Ouyang yielded a strong effect size for effectiveness of computer-assisted instruction in elementary classes kindergarten through sixth grade (Ouyang, 1993). Based on the means of effect sizes, Ouyang found 20 effectiveness for K?12 activities (in descending order) to be keyboarding skills, spelling, arts, mathematics, problem solving, vocabulary, writing, grammar, comprehension, language arts, music, science, reading in general, and social studies (Ouyang, 1993). On the secondary level, Ouyang also explored the effects of CAI instructional time on achievement. While instructional time of four weeks or less resulted in significantly positive achievement (Ouyang, 1993), the longer the period of instructional time, up to 30 weeks, the less effective the CAI (Chen, 1994; Ouyang, 1993). Similar results were found for mathematics achievement (Kulik, Kulik, Bangert-Drowns, & Williams, 1983). However, once the instructional time passed 30 weeks, gains in achievement suggest a strong increase in the relative effectiveness of the CAI (Chen, 1994; Lee, 1990). A meta analysis investigating the effects of achievement scores by subject area yielded small positive effects in some areas (Christmann, Badgett, & Lucking, 1997), e.g., the effect was small and positive for biology. Meta-analyses (Fletcher-Flinn & Gravatt, 1995; Liao, 1992) and studies (Lazarowitz & Huppert, 1993; Lu, Voss, & Kleinsmith, 1997) supported these findings in high school biology classes. Hounshell found statistically significantly higher student achievement scores for CAI students using supplemental computer simulations in comparison to the traditional classroom approach (Hounshell & Hill, 1989; Siegle & Foster, 2001). When provided with access to computers, students not only made gains academically, but their behavior also improved. 21 Outcomes Regarding Student Behavior on Stationary Computers Educators consistently look for ways to support positive student behaviors. Computers have the potential to encourage those behaviors. Students in high-access computer classrooms became more engaged and more independent than students in traditional classrooms. They assumed more academic responsibility and produced a higher level of work (Coley, 1997). In addition, ACOT research found computer-based instruction increased student time-on-task (MacArthur et al., 1986; NCREL, 2002; Schofield & Verban, 1988; Waxman & Huang, 1996; Worthen et al., 1994). Teacher Attitudes Regarding Stationary Computers As access to computers increased, many students were profoundly affected by learning opportunities afforded them by classroom computers; however, teachers were the educational group most profoundly affected by conversion from computer-less classrooms to multi-computer classrooms. Educational reformers predicted that technology would initiate a pedagogical revolution (Gess-Newsome et al., 2003). Constructivist educators hailed the computer as being one of the most powerful tools in education (Riel, 2000). This constructivist agenda has driven much of the research into teacher attitudes, beliefs, and perceptions about computers in the classroom. Constructivism in high- technology educational environments has encouraged the use of multi-sensory technologies, e.g., video and audio clips, voice mail, voice data entry, and MP3 technology. Constructivism supports exploration of ubiquitous technologies such as mobile computers, mobile phones, and indigenous computerized devices. Constructivism 22 thrives on the ubiquitous availability of computer technology anywhere and at all times. Constructivist curriculum values depth of subject matter over breadth. Depth is supported by greater student access to the Internet for virtual field trips, online research, and simulations. Constructivist curriculum is supported when computerized text, graphs, tables, graphics, audio, and video are merged by students when preparing reports. Interdisciplinary student projects encourage students to analyze, synthesize, and build new knowledge (Blumenthal, 2003; NCREL, 2003). Stationary computers had a number of drawbacks. Their size limited the number of stationary computers that could fit into one classroom. They had to be shared among the students and therefore could not be personalized. When they were to be used, teachers had to break the class into smaller groups and rotate the groups between the computer(s), lessons and other group work. Many teachers, administrators, and school boards began to consider the advantages of purchasing mobile computers such as laptops. Introduction to Use of Mobile Computers in K-12 The more visionary school districts studied the concept of one-to-one ?ownership? of a mobile computer by each student. The movement to provide a mobile computer to each student was encouraged by decreasing costs, increasing computer power and capabilities, growing wireless capabilities, increased access to the Internet, and public awareness of the need for a technology-proficient workforce (Smith, 1995). The promise of less expensive mobile computers was enticing. Student use of mobile computers would extend learning beyond the classroom, with access at any time and any place (Rockman, 2000). Teachers with 100% student access to identically configured 23 mobile computers would have the potential to seamlessly incorporate technology into their classrooms (Siegle & Foster, 2001). Aware of state and district research-based decision processes in the acquisition of computers, a number of laptop and handheld computer companies launched research projects within several years of introducing their products. ACOT research was divided into a professional development strand; a development and integration of cutting technologies strand; and dozens of short-term classroom research partnerships. Apple Computer continued to form partnerships with research centers, the Public Broadcasting System (PBS), the San Francisco Exploratorium, universities, software writers, counties, districts, and schools (Apple Computer, 1995). Results were mixed depending on outcomes addressed. Apple also partnered with schools and researchers outside the United States. The Australian researcher, Paul Newhouse conducted a three-year study to investigate the impact of Macintosh portable computers in grades 8 through 11 at a girls? school. Each student either owned or leased a portable computer. Newhouse found the best indicators of the success of the laptop program to be the amount and control of use of the laptops and the range of applications to curriculum tasks (Newhouse, 1997). Newhouse and a number of other Australian researchers have contributed to the understanding of uses of laptops in education through substantial and sophisticated empirical studies. Palm Pilot initiated a three-year study, the Palm Education Pioneers (PEP) program, conducted by SRI International from February 2001 through August 2002. PEP offered competitive grants to be awarded to teachers with innovative plans to integrate 24 handheld technology into instructional activities. Reports were written based on data collected from over 102 classroom teachers (Vahey & Crawford, 2002). Results were mixed according to the outcome addressed and will be discussed in the following sections. Microsoft Corporation and Toshiba American joined forces to hire the independent research group, Rockman ET AL, to conduct a three-year study. The short- term studies were completed at many research sites from 1997 through 1999. The study included private schools and public school districts; elementary, middle, and high school students; and hundreds of teachers. Many of the public schools had small pre-existing technology programs and served students with no home computers. The goals of the project were to document impacts on types of school technology models, teaching, and learning. Again results were mixed according to outcomes addressed and these will be addressed below. The U. S. government supported some research that was independent of company research. The National Science Foundation supported the Concord Consortium (a nonprofit educational research and development organization) in conducting a two-year research project, the Technology Enhanced Elementary and Middle School Science (TEEMSS) project, in the Spring of 2001 and Fall of 2002. Handheld Palm computers were used to introduce probeware into fifth through eighth grade science, mathematics, pre-engineering, and technology (SMET) classrooms. When students used the probeware and saw the real-time graph on the Palm handhelds, they reduced their misconceptions and increased their graph-reading skills and understanding of the lessons (Metcalf & 25 Tinker, 2003). The researchers in these studies were consistently concerned about the attitudes and perceptions of students toward mobile computers. Positive Student Attitudes In general, students in high access classes tended to have positive attitudes towards laptops and laptop use (Gardner et al., 1994; Pfeifer & Robb, 2001; Siegle & Foster, 2001; Stevenson, 1998). When surveyed, 80% to 90% of the students liked school better than non-laptop using students, perceived that computers helped them learn more than they had learned before, or would recommend their project to others (Fouts, 1997). Laptop-using students were also attributed with higher motivation and speed in acquiring information technology literacy (Gardner et al., 1994; Vahey & Crawford, 2002). These students also believed the laptops would help them learn more (Fouts, 1997). In the Stevenson study, a majority of the laptop-using students perceived that their spelling skills, writing skills, math scores, and reading scores were improved by laptop use (Stevenson, 1998). Students experienced increased self-esteem due to their self-perceived increase in status associated with devices usually used by professionals and business people (Vahey & Crawford, 2002). Laptop-using students? perceptions of their computer-related skills were higher than non-laptop users (Rockman, 2000; Vahey & Crawford, 2002; Waker, 2001). Statistically significant differences were found in confidence levels between the two groups for word processing, presentation software, Internet use, spreadsheets, databases, and web-page design (Rockman, 2000). Student use of computers was moderated by student perceptions of the value of computers as a learning facilitator 26 (Newhouse, 1997). The value of computer skills in future vocational tasks was clear to laptop-using students with 90% of them perceiving that computers would make workplace tasks easier (Newhouse, 1997). Student attitudes varied from study to study and group to group within longitudinal studies. In the three-year Newhouse study, positive attitudes varied from 38% to 10% depending on the group and the circumstances (Newhouse, 1997). Many researchers used surveys and questionnaires to determine students? preferences regarding the use of laptops and characteristics of laptop programs. Ownership issues arose concerning the acquisition of the laptops, with students preferring to be given choices between the purchase, lease, long-term loan, short-term loan, or classroom availability of a laptop (Newhouse, 1997). This issue seemed to be related to frequency of use in the classroom and the weight of the early laptops. Teachers who never or infrequently assigned laptop lessons were a factor in several studies. Students preferred that these teachers inform them as to when the laptops were needed so that the students would not unnecessarily have to carry the laptops. Many of these students expressed the desire to increase the use of computers in these classrooms (Newhouse, 1997). In the Newhouse second-year cohort of eighth graders, 61% of the students stated a desire to use their laptops more frequently (Newhouse, 1997). Students preferred teachers skilled in the use of computers and felt more secure when these skilled teachers assisted them (Newhouse, 1997). Many students expected the school to provide systematic computer training (Newhouse, 1997). 27 On surveys of self-rated level of proficiency, students reported increased self- confidence and rated their computer proficiency more highly than non-laptop using students (Rockman, 2000; Pfeifer & Robb, 2001; Vahey & Crawford, 2002; Waker, 2001). These increases were supported by researcher and teacher observations (Pfeifer & Robb, 2001). Students using laptops in the early 1990s gained sophistication at tasks such as independently opening documents, copying files, creating text and graphics, using a spreadsheet, creating graphs, and using packages such as MacGlobe? (Newhouse, 1997). Students using mobile computers in high access environments were expected to demonstrate skills in areas such as remotely accessing relevant class materials, submitting work on line, communicating via e-mail, and sharing information via infra-red beaming (Carrucan, 1999; Pfeifer & Robb, 2001). In the early 1990s, Apple Computer produced two short films that followed mobile computer-using students on class field trips. These films, Wireless Coyote and Cloud Forest Classroom, describe technical aspects as well as pedagogical aspects of wireless field trips. Researchers have found that students using handheld wireless technology benefit from a strong sense of personal empowerment (Pfeifer & Robb, 2001). Many laptop programs promoted 24-hours-a-day, 7-days-a-week access (often referred to as ?high-access?). by encouraging students to take the laptops home and by assigning laptop homework (Newhouse, 1999; Waker, 2001). No relationship was found between student attitudes towards school and use of the laptop at school. A positive relationship was found between student attitudes towards school and students who used 28 their laptops at home more frequently than students who did not use their laptops at home frequently (Waker, 2001). Student and Teacher Beliefs about Technology Laptop-using students? attitudes toward computers are more positive than non- laptop-using students? attitudes (Rockman, 2000). For five statements about the benefits of computers, laptop-using students indicated greater levels of agreement, and differences between laptop-using and non-laptop-using students were statistically significant. Laptop-using students agreed more strongly that computers helped them improve the quality of their schoolwork, made their schoolwork easier to do, made it more fun and/or interesting, and helped them understand their classes better. Laptop-using students also indicated that they more strongly preferred doing their schoolwork on the computer. In fact, the only statement with which non-laptop-using students indicated a greater level of agreement than laptop-using students was ?I enjoy playing games on the computer? (the difference was not statistically significant). Laptop-using students seem to feel more enthusiastic about the benefits of computer use for their schoolwork (Rockman, 2000). Both laptop-using and non-laptop-using students perceived specific benefits from computer use. When asked the open-ended question, ?How would your schoolwork be different if you didn?t use computers,? both groups of students perceived benefits from computer use. These included greater productivity in their schoolwork (primarily in writing and research), the ability to create more professional products, an increase in creative opportunities, and increases in the skill set they feel they will need in the workplace (Rockman, 2000). 29 Benefits of Longitudinal Observations At the 1999 Western Australian Institute for Educational Research (WAIER) Forum, Newhouse reflected upon his three-year longitudinal study of laptops of Year 7 through Year 10 students at an Australian-girls? school. Strong gains were made during the first two years. During the third year many teachers were concerned with preparing the tenth grade students for the tertiary entrance exams (TEE). The use of computers decreased considerably in Year 10 with the laptops primarily perceived to be used for technology development classes (Newhouse, 1999). Students entering the Year 8 classes in the third year of the study benefited from the experience of teachers involved in the program during the previous years of the study and had a much higher perception of their computer-related skills (Newhouse, 1997). The students participating in this longitudinal study for three years provided complex insights into the attitudes of students. Studies lasting only a year or two missed these insights. Negative Attitudes Regarding Mobile Computers or Mobile Computing Programs Reported negative attitudes ranged from no overall negative attitudes (Apple Computers, 2003; Newhouse, 1997; Vahey & Crawford, 2002) to high negative attitudes among the majority of the students in the third year Cohort B of the Newhouse longitudinal study (Newhouse, 1997). During interviews, the latter students expressed dissatisfaction with the laptop program due to teacher decisions not to use computers in the classroom. Negative student attitudes also were generated by a lack of teacher expertise; a lack of student mobile computer training; issues related to the transport of the heavy early laptops; issues related to the requirement to bring laptops to class when 30 teachers were not going to teach lessons using them; a sense of wasted money and resources when teachers did not teach lessons using them; perceptions that computers were not useful; student worrying; and a lack of confidence about using the laptops (Newhouse, 1997). Mechanical problems contributed to negative student attitudes (Newhouse, 1997). Lack of robustness of computers resulted in loss of work on computers and loss of access time to computers in all of the studies of mobile computers (Apple Computers, 2003; Vahey & Crawford, 2002). In the early 1990s, technical problems such as excessive weight, lack of battery duration, and limited memory, faulty screens, and prolonged startup time tended to discourage use (Newhouse, 1999). Handheld mechanical problems involved occasional breakage, cracked screens, difficulty with text input, and screen size (Vahey & Crawford, 2002; van?t Hooft, 2003). In spite of these problems, when asked how satisfied they were with their Palm handheld, approximately 60% were ?extremely satisfied,? 34% were ?somewhat? satisfied and 6% were ?not at all satisfied? (Vahey & Crawford, 2002). Student attitudes towards mobile computers effect student academic outcomes when using mobile computers. Student Academic Outcomes Regarding Mobile Computers Difficulty in obtaining reliable and valid standardized test scores for purposes of statistical analyses was a consistent theme in the literature of academic outcomes for laptop-using students (Fouts, 1997; Newhouse, 1997; Rockman, 2000). Rockman conducted studies in 13 schools but only eight of those schools yielded useful and reliable state and/or nationally normed assessment data. Some schools were not permitted to 31 provide data, schools conducted assessments with different standardized tests, and tests such as the PSAT or ACT were not mandatory and therefore did not provide enough useful data (Rockman, 2000). The Copernicus Project got started later than planned and the researchers could not collect enough meaningful data (Fouts, 1997). Many researchers resorted to research tools such as surveys, questionnaires, interviews, prompted writing samples, or logs of computer use by teachers and students. Students using laptops maintained a sustained level of academic achievement while in middle school (Stevenson, 1998). In Beaufort County the standardized test scores of the majority of laptop-using fifth to seventh grade students increased. The scores of laptop-using at-risk students and laptop-using special education students caught up with and exceeded non-laptop-using fifth to seventh graders within two-years. Scores for non-laptop-using students declined during the three-year project (Stevenson, 1998). The Beaufort County laptop-using girls consistently outperformed laptop-using boys throughout the study (Stevenson, 1998). For his program, Newhouse found that the contributions of laptop computers to student learning were statistically significant when the computers were used in a sustained and academically relevant manner (Newhouse, 1997). For three non-laptop-using groups, non participation in Beaufort County?s laptop program was associated with negative achievement. Non-laptop-using free-and-reduced lunch students experienced declines in test scores (Stevenson, 1998). Non-laptop-using boys and racial minorities (African American, Hispanic, and ?other?) experienced a ?significant? decline in standardized test scores (Stevenson, 1998). 32 In the three-year Rockman study, standardized test score comparisons were inconclusive. However, a Rockman student survey of active learning strategies showed pronounced differences in comparisons between the externally matched sites: active learning strategies were employed more frequently by laptop-using students then non- laptop-using students. These strategies included highlighting a main idea, re-reading reports before turning them in, outlining papers and information they read, looking up additional information as they read, and asking questions to make sure they understood what they had read (Rockman, 2000). Laptop-using students also performed better on writing assessments conducted over the three years (Rockman, 2000). Students using mobile computers have achieved technological literacy, responsibility, independence, achievement; and quality in their products (Fouts & Stuen, 1997; Gardner et al., 1994; Siegle & Foster, 2001). Students using laptops consistently showed increased levels of computer literacy and more sophisticated uses of technology than non-laptop groups (Newhouse, 1997; Rockman, 2000). Laptop-using students continued to show utilization patterns that were more intensive, more frequent, took place over longer periods of time, and included a wider variety of tasks than the utilization patterns recorded in baseline research (Rockman, 2000). Student Behavioral Outcomes Concerning Mobile Computers Mobile computer-using students used computers more at home than stationary computer-using students (Rockman, 2000; Stevenson, 1998). Many schools, such as in Beaufort County and Henrico County, had policies encouraging students to feel ownership and to take the mobile computers home. As perceived ownership increased, 33 homework completion increased (Vahey & Crawford, 2002). Stevenson reported an increase from 30% to 97% in the numbers of students using computers at home to do homework (Stevenson, 1998). Rockman reported laptop students spent more time doing homework on computers than non-laptop students (2000). All students used computers at home for more varied types of tasks and subjects than they did at school (Rockman, 2000). For the Newhouse study, the estimates of computer use at home ranged from five hours to one and a half hours per week (1997). Mobile computer-using students used computers more at school than stationary computer-using students, both compared to participant previous usage and compared to stationary-computer-using students (Ashmore, 2001; Blumenthal, 2003; Newhouse, 1997; Rockman, 2000; Stevenson, 1998; Vahey & Crawford, 2002). For the three-year Newhouse study, the estimates of computer use at school ranged from five hours to three hours per week. Stevenson saw laptop computer usage increase from 15% to more than 75% of laptop-using students reporting using the computer extensively in school (Stevenson, 1998). Mobile computer programs saw an increase in the number of students using mobile computers (Newhouse, 1997; Rockman, 2000; Stevenson, 1998; Vahey & Crawford, 2002); student task-oriented usage for homework and school work (Newhouse, 1997; Rockman, 2000; Stevenson, 1998; Vahey & Crawford, 2002); frequency and duration of use of the Internet (Rockman, 2000); and transference of computer skills to complete work for classes in which computers were not used within school (Rockman, 2000). 34 Students using laptop computers and handhelds devised and implemented strategies to increase their academic collaboration. Wireless capabilities allowed students to interact with students in other classrooms or areas when working on joint projects (Pfeifer & Robb, 2001; Vahey & Crawford, 2002). Students using other types of technology did not collaborate as much as students using handhelds (van?t Hooft, 2003). Flexible thinking was encouraged by wireless access to the Internet. Data not in the textbook, very recent data, and multiple perspectives could readily be found, copied, and discussed by students regardless of their location at the school (Pfeifer & Robb, 2001; Rockman, 2000). Laptop students reported that laptops either enhanced or maintained interaction and had neutral effect on student behavior (Stevenson, 1998). Motivation increased when students discovered they could work more efficiently (Newhouse, 1999; Siegle, 2001; van?t Hooft, 2003). For example, when the teacher beamed lessons to them or placed assignments on an electronic bulletin board, students could access the information without having to transcribe by hand (Pfeifer & Robb, 2001). Vahey and Crawford (2002) reported a continuous improvement in the motivational effects over time for students using handhelds. Organizational skills increased as students used calendar features, clarified assignment questions by e-mailing teachers and other students, and worked together to complete group projects (Pfeifer & Robb, 2001; van?t Hooft, 2003). These skills were especially useful for students usually thought of as being low achievers (Vahey & Crawford, 2002). Student responses to the Palm Students? Evaluation of Handhelds for 35 Learning Survey showed that 37% of the high school seniors and 22% of the other grade levels cited organizing as a favorite use of the Palm handheld (Vahey & Crawford, 2002). Laptop-using students had statistically significant differences in attendance patterns from non-laptop-using students: They had higher school attendance and less tardiness than non-laptop-using students (Stevenson, 1998). Negative Findings Regarding Student Use of Mobile Computers Inappropriate use of handhelds tended to cause a high level of distraction, e.g., when instructions for new software were presented, some students chose to play games rather than follow the instructions. Student responses to the Palm Students? Evaluation of Handhelds for Learning survey showed that 80% of the eighth and ninth graders, 60% of the eleventh graders, and 29% of the twelfth graders cited game playing as their favorite use of the Palm handhelds. Infra-red beaming enabled students to send homework or inappropriate messages to other students. Some students wanted to use the handhelds for every task whether it was the best tool or not (Vahey & Crawford, 2002). High Access Mobile Computers in the Classroom Apple Computer, Palm Computer, Microsoft, Toshiba, and other computer companies invested in research partnerships in what are often referred to as ?high-access? classrooms. The goal was to observe the results of mobile computer use on teaching and learning when every student ?owned? a computer 24 hours a day, seven days a week. In some research programs, wireless technology provided new high-access educational approaches. 36 Teachers with high-access classes tended to spend less time planning compared to other non-laptop teachers. They were also more likely to assign students to small group projects or presentations; teach a large number of topics in greater depth; and engage in constructivist approaches to teacher and student roles than were teachers in other models. They also were more likely to practice constructivist approaches to instructional strategies; assign assessment tasks of a constructivist nature; assess homework according to constructivist criteria; and credit individual improvement, effort, and participation when assessing homework activities, than were teachers in other models (Ashmore, 2001). A county-wide example of high-access teaching was researched in Henrico County. The Henrico County study was unique in several ways. All high school teachers and students in the county were given ubiquitous access to laptops during the study. No other reform-based interventions occurred during this study. Teacher-level factors and system-level factors impacting the effective use of laptops were investigated regarding student-centered, inquiry-driven, and collaborative learning. Many of the findings in this study support the research previously reported in this literature search. Summary of Chapter Two Studies investigating the effects of stationary computers generally have yielded positive student attitudes and student outcomes. These benefits were only available to students with access to computers and with teachers willing to utilize computers in the learning process. Many researchers perceived teacher attitudes to be the key to successful 37 implementation of computers and technology in the classroom. Teachers implementing student-centered, inquiry-driven, and collaborative pedagogical approaches were found to possess more positive attitudes towards the use of computers in the classroom than teachers exhibiting instructivist behaviors. Student attitudes towards mobile computers are more positive than the attitudes of students using stationary computers. Students reported increased self-confidence, rated their computer proficiency more highly, felt mobile computers helped them to be better students, were more motivated, and perceived improvements in academic areas. The effect sizes produced in these studies will be used in Chapter four. Effect sizes were the primary statistical focus of the sources for this literature search. Studies reporting percentages were used only to support initial findings reported as effect sizes. Research regarding the effects of mobile computers on the attitudes and outcomes of students had not been evaluated with a meta analysis at the inception of this study. This meta analysis will provide a new line of research by providing a better understanding of the effects of mobile computers on the attitudes and outcomes of K?12 students and teachers. Chapter three, the methods section, will define the theoretical relationship of interest. Theoretical constructs for the explanatory and response variables will be stated. Chapter three will also describe the data collection procedures, coding procedures, relevant formulas used for calculating effect sizes, and data analysis. 38 III. METHODS The focus of chapter three is a description of the methodology, a statistical meta analysis, implemented in this study. The theoretical relationship of interest will be described as well as the theoretical constructs for explanatory and response variables. Approaches to collecting data, criteria for excluding inappropriate data, coding the appropriate studies, computing effect sizes, and treatment of the data will be described. Contradictory results are often found between studies on the same education topic, frustrating researchers and public policy makers alike. Variations among treatments, environments, methods, and instruments confound comparisons. Overwhelming amounts of theory and background information further confuse attempts to seek clarity as to the effects of a program, study, or treatment. An approach designed to resolve apparent contradictions in research findings was devised by Gene Glass in 1976. ?Meta-analysis is a collection of systematic techniques for resolving apparent contradictions in research findings. Meta-analysts translate results from different studies to a common metric and statistically explore relations between study characteristics and finds? (Bangert-Drowns & Rudner, 2003, p.1). Gene Glass proposed a philosophical justification for meta analysis. ?Glass argued that literature review should be as systematic as primary research and should interpret the results of individual studies in the context of distributions of finding, 39 partially determined by study characteristics and partially random? (Bangert-Drowns & Rudner, 2003, p.1). Studies of mobile computing contain frequent contradictions and inconsistencies. The number of investigators, the many types of mobile computers, an overwhelming number of statistics, potential bias, different methodologies, and inconsistent approaches to reporting statistics, make interpretation of research findings difficult. The common thread in these studies is that they seek to assess the attitudes, knowledge, and skill of students. Theoretical relationships of interest are student attitudes toward mobile computing and student academic outcomes when mobile computers are used by the students. The four reasons to analyze the effects from the accumulated studies are to ?establish the presence of an effect, determine the magnitude of an effect, resolve differences in a literature, and to determine important moderators of an effect? (DeCoster, 2002, p. 4). Initially studies regarding teacher attitudes were of major interest, but this line of investigation was minimized due to an insufficient number studies producing statistically useful data. Explanatory variables help explain complex relationships in meta analyses. An example of an explanatory variable in this context might occur regarding teacher ratings of students. If one group of students were to use mobile computers; have one-on-one access; and were to use their computers anytime and anywhere; teachers might respond with more positive attitude ratings for the mobile computer-using students than for students with only partial access to stationary computers or with no access at all. 40 Limits have been set to include and exclude research studies. Studies were included if the study yielded Cohen?s d effect sizes or statistics that were transformed to Cohen?s d effect sizes. Research articles concerned with academic and attitude outcomes for K?12 student use of mobile computers were included. The statistical analysis excluded qualitative studies as well as anecdotal accounts. Stationary (non-portable) computer studies were also excluded. Education programs outside of K?12 education were excluded. Literature searches included Auburn University academic databases, Inter-Library Loan, and Google; a citation search; and expert responses. A comprehensive and exhaustive effort was made to gather all published and unpublished reports and studies on the subject of mobile computing K?12. No restrictions ? dates, languages, amount of text, or type of source ? were applied during database searches. Two categories of key words were searched: mobile computing and K?12 education. Key words in mobile computing searches included mobile computing, mobile computers, portable computers, laptops, personal digital assistants, Palm computers, PDAs, pocket computers, notebooks, iBooks, iPaqs, ubiquitous, and wireless. Keywords for K?12 education searches included K?12, education, elementary, middle school, high school, science, and math. Academic data bases searched included Digital Dissertations, ERIC (Ovid), Psyc INFO, World Cat, Aubiecat, and IEEE Xplore and Google. Google Scholar became available during the course of this project and was used extensively. A citation search was also conducted for each of the authors of the primary studies. 41 A request for expert assistance was made via e-mail to researchers, editors of technical educational research journals, directors of technical education conferences, and leaders of technical educational associations or special interest groups (Appendix A). E- mail was used rather than postal mail. ?The deployment of any devices designed to decrease human effort in the person-computer interaction-and hence increase the rewards for participation, will result in higher levels of comprehension and may subsequently improve the quality of response? (Dillman & Christian, 2000, p. 2). Effort was made to keep the request simple as plain versus fancy web survey designs yield higher completion rates (Dillman & Christian, 2000; Dillman, Tortora & Bowker, 1998). An electronic request for expert responses is appropriate for the target audience as this population is highly connected, is computer literate, and routinely communicates via e-mail. Coding the Studies A form was used to help organize study information and determine relevant categories for coding. This form, Assessing Methodological Quality of Journal Articles for a Statistical Meta Analysis (see Appendix B), evolved from a North Central Regional Educational Laboratory form to a form more representative of the issues and concerns of the primary studies obtained for this meta analysis (Waxman et al., 2002). Demographics, student technology, instructional, reliability, and validity characteristics were collected using this modified form. Coding rules were established for academic outcomes and attitude outcomes. Inter-rater agreement was obtained based on these rules. 42 Data Analysis After the data was collected and coded, many statistics (e.g. F test values) needed to be transformed to Cohen?s d effect sizes. Calculation of effect sizes is crucial in the establishment of statistical meaning in a meta analysis. The formulae for calculating effect sizes were primarily found in LeJeune?s A Meta-analysis of Outcomes From the Use of Computer-Simulated Experiments in Science Education (2002), Ouyang?s A Meta- analysis: Effectiveness of Computer-assisted Instruction at the Level of Elementary Education (1993), Furr?s Effect Sizes and Significance Tests (2004) and the original Smith and Glass meta analysis (1976). Formulae Formula 1: Cohen?s formula for effect size M Exp ? M Con ES?= ___________________ SD pooled In the formula for effect size: ES? is the effect size measure, M Exp is the estimated mean of the experimental group, M Con is the estimated mean of the control group, and SD pooled is the estimated pooled within-group standard deviation. Formula 2: Formula for computing d 2 1 2 r r d ? = 43 In the formula for computing d r = the correlation coefficient Formula 3: Formula for effect size for studies reporting t-scores. ES? = ConExp ConExp nn nn t ? + In the formula for effect size for studies reporting t-scores: ES? is the effect size, t is the reported t score, n Exp is the number in the experimental group, and n Con is the number in the control group. Formula 4: Formula for effect size for studies reporting F-statistic. ES? = N F2 In the formula for effect size for studies that reported the F-statistic: ES? is the effect size, F is the F statistic, and N is the number of subjects of both the experimental and control groups when these numbers are equal. Formula 5: Formula for effect size using pretest and posttest data (M Exp ?post ? M Exp ?pre ) ? (M Con ?post ? M Con ?pre ) ES? = _________________________________________ SD Pooled 44 In the formula for effect size using pretest and posttest data: M Exp-post is the mean posttest score of the experimental group, M Exp-pre is the mean pretest score of the experimental group, M Con-post is the mean post-test score of the control group, M Con-pre is the mean of pretest score of the control group, and SD? Pooled is the pooled within-group standard deviation. Formula 6: Formula for pooled within-group standard deviation. SD pooled = 2 )1()1( 22 ++ ?+? ConExp Con Con Exp Exp nn SDnSDn In the formula for pooled within-group standard deviation: SD pooled is the pooled within-group standard deviation, n Exp is the number of experimental students, n Con is the number of control students, SD Exp is the standard deviation of the experimental group, and SD Con is the standard deviation of the control group. Formula 7: Formula for effect-size error correction. ES = (ES?) 1 ? 94 3 ?N In the formula for effect size error correction: ES is the corrected effect size considering sampling error due to sample size differences, 45 ES? is the uncorrected effect size, and N is the total number of subjects in the study. The effect sizes calculated from the above formulas were used to determine the means and standard deviations of study samples, academic outcomes, attitude outcomes, and instruments used to measure the outcomes. Correlational studies were conducted. Results from technology studies in general and the current mobile computer study were compared. Summary of Chapter Three Chapter three discussed the history and theory of meta analysis; approaches to collecting data, coding data, and computing effect sizes; and criteria for inclusion and exclusion of data. Formulas for calculating Cohen?s d effect sizes were presented. Data analyses were discussed. The next chapter contains three journal articles: A statistical meta analysis of academic outcomes, a statistical meta analysis of attitude outcomes, and a practitioner article. Theoretical relationships of interest have been examined for the presence of a statistically significant effect, the magnitude of that effect, and differences related to studies. Sources of research were discussed. The studies were coded and effect sizes computed with the goal of clarifying the true nature of the complex relationships involved among the variables. 46 IV. RESULTS Study 1 A Statistical Meta Analysis of the Effectiveness of Laptop Computers on Student Academic Outcomes In A National Education Technology Plan: The Future is Now, the U.S. Department of Education (2005) states, ?There is no dispute over the need for America?s students to have the knowledge and competence to compete in an increasingly technology-driven world economy. This need demands new models of education facilitated by educational technology? (p. 1). Among other recommendations, the Seven Major Action Steps and Recommendations in the plan encourage districts to provide both ubiquitous access to computers for each student and the connectivity required for students to utilize fully mobile computers. Many legislators, educators, and students perceive adoption of mobile computers in the schools to be a sound investment that will result in the high academic achievement required for collegiate and vocational training. Mobile computers are credited with increasing productivity, accuracy, and employee morale. The use of mobile computers in business is steadily growing with portables accounting for 54% of the $500 million spent for computers in May of 2003 (Thornton, 2003). Many educational leaders believe that student use of mobile computers will help prepare students for the world of work, but 47 they want supportive academic research results before committing scarce resources. ?Education is the only business still debating the usefulness of technology,? according to former U.S. Education Secretary Paige (U.S. Department of Education, 2005, p. 1). School boards are continuing to debate the usefulness of mobile computers and to demand proof of beneficial results before committing to extensive uses of such technology. The cost of purchase, maintenance, Internet access, security, and trained personnel forces administrators and board members to make choices based on reliable and valid information. Some computer studies appear to be contradictory or paradoxical. Often the studies differ in ways that make it difficult to compare results. A study examining the effect of mobile computers on an elementary math test would appear to be difficult to compare to the effect of mobile computers on high school SAT verbal test results. When research is conducted using different analyses (e. g., correlation, t-tests, and Analysis of Variance), results are reported using different types of test results. Survey research, observation, teacher-produced tests, and standardized tests all would appear to be difficult to compare. But these difficulties can be resolved by the use of the effect size, a standardized measure that enables the analysis, evaluation, and integration of results by means of a statistical meta analysis. A number of meta-analyses have been conducted regarding academic outcomes for students using technology (see Table 1). In these studies, educational technology is used as a broad term which includes many areas, e.g. use of computers, multi-media, voice communication, mobile computers, and the Internet. These meta analyses produced 48 effect sizes that form a baseline to which mobile computer effect sizes will be compared. By using this baseline as a context of decision and comparative value, the benefits of mobile computers for academic outcomes can be assessed (Glass, McGaw, & Smith, 1981). Table 1 Effect Sizes Based on Meta Analyses of Academic Outcomes on Technology Meta Analysis Author Year Pub. Number of Studies in Meta Analysis Category Mean Effect Size Chen 1994 75 Math .50 Blok, Oostdam, Otter, & Overmaat 2002 42 Reading .19 Goldberg, Russell, & Cook 2002 15 Writing .50 Christman and Badgett 1999 11 Science .27 A review of the literature suggests that integration of curriculum, teaching methods, and mathematics software increase standardized mathematics test scores (Bain & Ross, 2000; Bain & Smith, 2000; Center for Applied Research in Educational Technology [CARET], 2005; Mann, Shakeshaft, Becker, & Kottkamp, 1998). Student mathematics test scores benefit from the skill and knowledge of teachers regarding the use and instruction of mathematics software such as Logo, computer-assisted instruction microworlds, and algebra and geometry software (Hillel, Kieran, & Gurtner, 1989; McCoy, 1996; Simmons & Cope, 1990, 1993). The length of the experiment, length of 49 each treatment, and frequency in the use of mobile computers are significantly related to mathematics achievement. Chen also found Computer Based Instruction (CBI) is more effective than traditional teaching approaches in teaching arithmetic, computation, and when instructing primary students (1994). An integrated approach to curriculum, teaching methods and reading software demonstrated increases in scores on standardized reading tests (Bain & Ross, 2000; Bain & Smith, 2000; CARET, 2005; Mann, Shakeshaft, Becker, & Kottkamp, 1998). For remedial students, computer assisted instruction (CAI) is effective in the teaching of basic reading skills in computer laboratories (Zollman, Oldham, &Wyrick, 1989). Software designed to improve skills in phonemic awareness (Barker & Torgesen, 1995; Mitchell & Fox, 2001), phonics (Mitchell & Fox, 2001), vocabulary (Anderson-Inman & Horney, 1998; McKenna & Watkins, 1996), and text comprehension (Higgins & Boone, 1991; Medwell, 1996) produce positive impacts on reading acquisition (Sherman, 2004). Although over 200 studies have been conducted regarding student use of computers when writing (Goldberg, 2003), predominately modest effect sizes have been reported for better quality (Bangert-Drowns, 1993; Goldberg, 2003; Hannafin & Dalton, 1987; Owston, 1991) and increased length and more extensive editing (Dauite, 1986; Etchinson, 1989; Vacc, 1987). Middle school writing scores for students in a technology- rich language arts curriculum were no better for the computer-use program than for the traditional program (Cramer & Smith, 2002). Disadvantaged students such as ethnic minorities (Lerew, 1997) and mildly handicapped adolescents (Sitko & Crealock, 1986) appear to benefit the most from the use of word processing. 50 For purposes of this study, the construct and academic outcomes are operationally defined as major information aggregates of statistics regarding academic tests, subjects, higher order thinking skills, and knowledge of computers found in the relevant mobile computer studies. Academic tests and academic subjects are reported individually and by collapsing them into one measure. Academic subjects are reported by categories (math, reading, writing, science, and communication/language). Purpose of the Study The purpose of this meta analysis is to synthesize the effect sizes from an exhaustive search of empirical research projects on academic outcomes of student mobile computer use. Six research questions are addressed by this study: 1. How extensive is the experimental and quasi-experimental body of research related to the effects of mobile computer use on K?12 academic outcomes? 2. What is the magnitude and direction of the means of the effect sizes by sample? 3. What is the magnitude and direction of the means of the effect sizes by category? 4. What is the magnitude and direction of the means of the effect sizes by academic subjects and tests? 5. What is the magnitude and direction of the overall (total) effect from these academic studies? 51 6. How do the effect sizes obtained for the use of mobile computers compare with the effect sizes obtained for the use of technology in general? Method The statistical meta analysis is characterized by an exhaustive literature search, exclusion of irrelevant studies, coding of important study characteristics, calculations of effect sizes, and appropriate statistical analysis of those effect sizes. Meta analyses depend on research project results that produce statistics appropriate for determining effect sizes. Glass defined meta analysis as ?the statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings? (Glass, 1976, p. 3). The following literature search was focused on finding appropriate statistics for the purpose of determining relevant effect sizes. Search Procedures Used in Obtaining Studies An exhaustive, systematic search of literature for mobile computer use by K?12 students was conducted by multiple means. Literature searches included academic databases, public search engines such as Google, Yahoo, and Google Scholar; and a citation search. Many for-fee empirical articles were found in Google Scholar. A citation search was conducted for each of the authors of statistically-driven studies. Academic databases searched included Digital Dissertations, ERIC (Ovid), Psyc INFO, World Cat, and IEEE Xplore. All database searches were conducted with no restrictions on dates, languages, amount of text, or type of source. Researchers, authors, and journal editors 52 were queried by e-mail in a comprehensive and exhaustive effort to gather all published and unpublished empirical studies on the subject of mobile computing K?12. Key words searched for mobile computing included mobile computing, mobile computers, portable computers, laptops, personal digital assistants (PDAs), Palm computers, pocket computers, notebooks, iBooks, iPaqs, ubiquitous, and wireless. Key words searched for K?12 included K?12, education, elementary, middle school, high school, reading, writing, science, and mathematics. Criteria for Inclusion Literature search criteria for inclusion and exclusion were established. Literature was included only if it produced effect sizes or statistical data that yielded effect sizes and concerned mobile computers, student academic outcomes, or K?12 education. Research projects (91) were excluded if they had no effect sizes or failed to produce statistics necessary for calculation of effect sizes. Any study producing percentage results was excluded if it did not also produce effect sizes or sample number, mean, and standard deviation. Studies concerning pre-school children and post-secondary students were excluded. Qualitative projects (10) such as case studies were excluded as were articles on mobile computers limited to use of dedicated software (2). A number of studies were inaccessible (23). Unpublished studies were requested from authors and editors involved in the publication of research projects. An e-mail request was sent to these experts regarding the name, location, and e-mail address of any researcher with knowledge of an unpublished article regarding a true experimental or quasi-experimental K?12 educational mobile 53 computer study (see Appendix A). Of the 47 e-mails sent, eight brought responses, with six of these responses providing positive feedback and links. Although the e-mail survey failed to produce usable unpublished studies, it resulted in further links to usable published research projects. The literature search on the effects of mobile computing on K?12 students produced 30 research projects. Of these projects, 21 were concerned with academic outcomes. The academic outcome studies were concerned with academic subjects and test results, higher-order thinking skills, and knowledge of computers and ancillary technologies. Coding of Important Study Characteristics Study characteristics were coded by means of a form, Assessing the Quality of Journal Articles for a Statistical Meta analysis (see Appendix B), which is a modified version of a North Central Regional Educational Laboratory (NCREL) form (Waxman, 2002). The NCREL form was modified to make it more sensitive in reporting information specific to mobile computer studies. Information was collected regarding demographic, student, technology, instructional, reliability, and validity characteristics by using this modified form. 1. Demographics included student gender, grade level, ethnicity, socioeconomic status, country, US geographical region, school type, and community type. Academic content area, unit of analysis, student sample size, number of classes, and number of schools in the study were also collected. 54 2. Student characteristics contained patterns of student mobile computer use, ownership of mobile computers, percentage of students using mobile computers in the experimental group, percentage of students in the control group, technology experience, special education, and types of academic outcomes. 3. Technology characteristics included types of mobile computers, software, tools for other tasks, technology resources, focus of technology, quantity of technology, task difficulty, type of learning task, and time of establishment of laptop program. 4. Instructional characteristics and teacher characteristics included teacher training in technology, teacher experience with technology, joint productive activity, collaboration, language and literacy development, contextualization, challenging activities, instructional conversation, setting, mode of instruction, role of teacher, learning responsibility, and teacher qualifications. 5. Statistical characteristics of each study focused on evidence of reliability and validity. Database Description Two databases were created: a demographics database and an effect size coding database. The demographics database included author name, title of research project, year of study, policy level, focus of study, effect size, study mean effect size, and the demographic categories described above. This database also collected information on the methodological quality characteristics of each primary study. The effect size coding database included author name, academic outcome effect size, page on which effect size was found in each article, codes, and subcodes. The codes 55 included the academic outcome effect sizes for academic tests, higher order thinking skills, academic subjects, knowledge of computers, and grade point average (GPA). The effect-size subcodes were useful in developing a better understanding of the coded variables (see Appendix C). Rules determined membership in the four coded groups. For an effect size to be coded as an academic test, the author was required to state that the effect size was obtained from a test. SAT, standardized academic achievement, district, and teacher test scores were included in this category. Any effect size reported as an academic subject but not connected to a test was coded as academic subject, e.g., grade point averages, holistic scoring and project-based scoring. Higher order thinking skills not associated with testing or academic subjects were coded as higher order thinking skills, e.g., analysis, synthesis, and evaluation. Knowledge of computers, the Internet, and technical information not connected to an academic test or academic subject was coded as knowledge of computers. Inter-rater agreement on the coding systems for academic outcomes was determined by percent of agreement between two evaluators. The percent of agreement represented the times that the journal article evaluators assigned the same score when using the evaluation forms (Shannon, 2001). Inter-rater agreement was achieved on the descriptors (81%), quality (83%), and academic outcomes coding (96%). Instruments The purpose of the evidence-based Assessing the Quality of Journal Articles for a Statistical Meta analysis was to gather information regarding characteristics reported in 56 the selected studies. Demographics, type of technology, instructional approaches, and measures of quality and invalidity were recorded. The methodological quality form changed as new studies and new evidence were added. Evidence-based assessment based on the authenticity of the findings was the purpose of these changes. Results In the results, descriptive statistics and effect sizes are reported. The descriptive statistics include descriptions of study, population, student, technology, and instructional characteristics. Mean effect sizes are reported for the overall meta analysis and for sub- codes of academic outcomes (academic tests, higher-order thinking skills, and academic subject areas). Totals greater than 100% occurred when studies reported multiple variables for a category. Description of Study Characteristics The first research question asked, ?How extensive is the experimental and quasi- experimental body of research related to the effects of mobile computer use on K?12 academic outcomes?? An extensive literature search produced 21 research projects which formed the foundation of this meta analysis (noted in the reference section). A total of 349 effect sizes were reported for academic outcomes for students using K?12 mobile computers. Publication dates ranged from 1993 to 2004, with 80% of the articles published after 1999. Journal articles (48%) and dissertations (14%) were the major forms of publication with 38% of articles published in peer-reviewed journals. 57 In over half of the studies, mobile computers were provided at no cost to the student by the district (58%). In other studies, the parents were expected to lease or purchase the computer (42%) as a pre-requisite for participating in the mobile computer project. Student gender was predominantly mixed (86%), exclusively female (5%) or unspecified (10%). No exclusively male population was reported. Participating grade levels ranged from grade one to grade twelve: grade one (4%), grades two through four (2% each), grade five (9%), grade six (16%), grade seven (18%), grade eight (16%), grade nine (11%), grade ten (9%), grade eleven (7%), and grade twelve (5%). The units of analysis were the class (42%), school (25%), individual (13%), grade level (8%) and state (8%). Student sample sizes ranged from 24 to 426. Class sample sizes ranged from 1 to 73. The number of schools per study ranged from 1 to 223 in a state-wide project. The types of schools were predominantly public (76%) and private (10%), with some studies not specifying the school type. The United States (68%) was the primary country of residence for the students followed by Australia, Austria, Canada, Chile, Germany, and Northern Ireland (5% each). Laptops and notebook computers (90%) were the major types of mobile computer used. Handheld computer projects (10%) tended to be qualitative or reported with percentages rather than experimental statistics. These studies emphasized ubiquitous access (24 hours a day and 365 days a year) by mobile computer students. The classroom was the school setting for the use of mobile computers (100%). Academic test scores (64%), higher order thinking skills (27%), and computer knowledge (10%) were reported for academic outcomes. Academic outcomes by subject 58 matter overlapped. Subjects included reading (14%), math (48%), language arts (29%), writing (52%), science (19%), oral communication (6%), SAT Math (5%), SAT Verbal (10%), and the world (5%). For coding purposes, a distinction was made between academic test scores of subjects and non-test subject matter. Math and language test scores were coded as test scores. Math and language semester grades, observer scores regarding student language activities on the mobile computers, or student attitudes toward academic subjects taught by mobile computer were coded as non-test subject matter. The academic subjects reported in these research projects were math (19%), writing (40%), language arts (14%), science (8%), reading (4%) and oral communication (11%). Only three experimental research studies (14%) were found, probably because random sampling/assignment is rarely conducted in school settings. Quasi-experimental designs (48%) and pre-experimental design (38%) were employed most frequently. Only 14% of the designs were true experimental (Ary et al., 2002). Most of the authors of the qualifying articles reported statistics (number, mean, and standard deviation) which enabled the calculation of Cohen?s d effect sizes (Cohen, 1988). Some authors reported effect sizes (43%). Effect Sizes The second research question regarding effect sizes asked, ?What is the magnitude and direction of the means of the effect sizes by sample?? The number of effect sizes, mean of the effect sizes (using Cohen?s d), and standard deviations are reported for each outcome. The magnitude and direction of each outcome are discussed 59 and compared. Longitudinal studies were identified by year, e.g., Ross1, Ross2, and Ross3. The magnitudes of the academic author effect size means (see Table 2) are predominantly positive in direction (95%) with one negative effect size (5%). The means are predominantly moderate with four large effect sizes: Schieber (-1.91), Ross sample 1 (.96), Siegle (.91), and Lowther (.81). The smallest effect size was contributed by Gardener (.05). Table 2 Academic Outcome Sample Effect Sizes by Sample/Author (N = 21) Sample Number of Relevant Effect Sizes Per Article Mean ES for Study Chang (1998) 18 .30 Gardner (1993) 35 .05 Gasque (2000) 2 .07 Gulek1 (2005) 10 .19 Gulek2 (2005) 8 .12 Gulek3 (2005) 10 .12 Kessel (1999) 116 .30 Lewis (2004) 8 .12 Lowther (2003) 13 .81 Martin (2004) 2 .47 McTeer (2004) 34 .66 Muir (2004) 12 .12 (table continues) 60 Table 2 (continued) Sample Number of Relevant Effect Sizes Per Article Mean ES for Study Owston (2001) 20 .13 Ross1 (2003) 12 .96 Ross2 (2001) 17 .60 Ross3 (2000) 11 .44 Schaumburg (2001) 8 .68 Schieber (1999) 6 -1.91 Siegle (2001) 3 .91 Trimmel (2004) 2 .34 Zurita (2004) 2 .74 Sample Total = 349 Overall Sample Mean = .29 The third research question asked, ?What is the magnitude and direction of the academic outcome effect sizes by category?? Academic outcome effect sizes by category are presented below (see Table 3). The academic outcome effect size means are positive in direction. The highest effect size for knowledge of computers and the Internet (.58) is moderate and positive. The remaining two effect sizes are small and positive: Academic tests and subject areas (.44) and Higher Order Thinking Skills (.26). The average academic outcome effect size mean (.43) is small and positive. 61 Table 3 Academic Outcome Effect Sizes by Category (N = 21 articles) Number of Effect Sizes Mean of Effect Sizes (Cohen?s d) Academic test outcomes 141 .44 Higher Order Thinking Skills 59 .26 Knowledge of Computers/Internet 22 .58 Total = 222 Mean = .43 The fourth research question asked, ?What is the magnitude and direction of the academic outcome effect sizes by academic subject?? The academic subject effect size means are positive in direction and have a small magnitude. When academic outcomes by test and subject matter (see Table 3) are examined separately, the mean effect size for academic test outcomes (.43) is larger than for academic subject areas. The largest effect size for communication and language (.33) was obtained from measures of language arts, visual and performing arts, speaking out, communicating, and presenting information orally. Mobile computer academic outcomes by subject are reported in Table 4. 62 Table 4 Academic Outcomes by Subject (Number of Samples = 21) Laptop Academic Outcomes Number of Effect Sizes Mean Effect Size by Study Math 54 .20 Reading 10 .22 Writing 112 .27 Science 24 .37 Communication and Language 73 .33 Total ES = 273 ES Mean = .29 The fifth research question asked, ?What is the magnitude and direction of the overall (total) effect from these academic studies?? The average sample mean effect size represents effect sizes well. The average author mean effect size is positive but small (.29) (see Table 2). When the negative effect size (-1.91) is removed, the average sample mean effect size approaches is small but approaching moderate (.41). The sixth research question asked, ?How do the effect sizes obtained for the use of mobile computers compare with the effect sizes obtained for the use of technology in general?? Table 1 contains subject effect sizes of meta analyses regarding the use of technology in math, reading, writing, and science. In the following table (Table 5), the results of the technology meta analysis mean effect sizes (Table 1) are compared to mobile computer subject effect sizes in Table 4. The purpose of the comparison is to determine if a subset of technology (mobile computers) produces higher effect sizes for 63 any subject(s) than are produced by technology in general. In Table 5, number (N) equals the number of studies in each meta analysis (see Table 5). Table 5 Mean Effect Size Comparisons Across Meta Analyses Reported in Table 1 and the Current Study Subject Table 1 Study Current Study Math .50 (N = 75) .21 (N = 22) Reading .19 (N = 42) .22 (N = 22) Writing .50 (N = 15) .27 (N = 22) Science .27 (N = 11) .37(N = 22) The results section reported percentages describing the studies, effect size means and standard deviations for various study categories and variables, and numbers of studies and effect sizes. In the following section these results will be examined. Limitations, implications for intervention, and a conclusion will be discussed. Discussion The first research question asked, ?How extensive is the experimental and quasi- experimental body of research related to the effects of mobile computer use on K?12 academic outcomes?? The initial intent of the research was to find empirical research studies on the effects of varied types of mobile computers on academic outcomes for U.S. K?12 students. In spite of a diligent research for different types of mobile computers, the 64 vast majority of the effect sizes found concern laptop computers (90%). A search for international studies became a necessity when it became apparent that the limited number of U.S. empirical studies was not sufficient for a meta analysis on mobile computers. Studies by researchers in Australia, Austria, Canada, Chile, Germany, and Northern Ireland tended to support the occurrence of common experiences regarding both problems and solutions in dealing with laptop computers. Researchers outside the U.S. tended to be more forthcoming in their reports of hardware problems than U.S. researchers (Gardner, 1993; Newhouse, 1997). The second research question asked, ?What is the magnitude and direction of the means of the effect sizes by sample?? The magnitude and direction of the means of the effect sizes by sample was small and positive (.29). Since some of the control groups had access to stationary computers, e.g., Ross1, Ross2, and Ross3, this mean effect size suggests benefit of laptops for academic achievement beyond the benefit of stationary computers. Four commonalities exist among three of the four studies yielding the highest effect sizes: grade level, subject matter, multiple instruments, and triangulation. The measures of academic achievement by Schieber (fifth and sixth grades), Ross1 (fifth and sixth th grades), and Lowther (sixth and seventh grades) overlapped in relation to same grade level, 6 th grade. Additionally, all three evaluated mobile computers using the same subject matter, writing. Each of these authors also used multiple instrument(s) i.e., student focus groups, student surveys, school observation, and measures of attitudes and problem solving ability. 65 Triangulation ? the practice of confirming data using multiple data-gathering procedures (Ary, 2002) ? was used by Schieber, Ross, and Lowther. The use of information from multiple measures enabled the authors to produce complex results and insightful discussions. An academic measure, an attitude survey, and a focus group were most frequently used by researchers in these meta analyses. Two important differences in the four studies yielding high effect sizes were observed: The large negative effect size obtained by Schieber (ES = ?1.91) and the differences in population size. Schieber suggests that the large negative effect size resulted from deterioration of the classroom environment as the novelty of the laptop immersion program wore off. Students and teachers were overwhelmed by both the amount and complexity of the change (Schieber, 1999). The range in total population size (experimental plus control groups) was from 27 to 522 students. The third research question regarding effect sizes asked, ?What is the magnitude and direction of the means of the effect sizes by category (academic test and subject matter, higher order thinking skills, and knowledge of computers and the Internet). The academic outcome and higher order thinking skills effect size means are small and positive. Only the effect size for knowledge of computers and the Internet was positive and moderate. These results suggest that learning on mobile computers is more highly associated with knowledge about technology than with other academic concerns. These results support an emerging concept that the study of technology should be a fourth basic literacy (Langraf, 2005). The average academic outcome effect size mean is positive and 66 small suggesting that mobile computers are only slightly more academically useful than stationary computers and/or no computers. The fourth research question asked, ?What is the magnitude and direction of the outcome effect sizes by academic subject?? Academic subject is a subset of academic test/subject matter and includes math, reading, writing, science, communication and language. The relatively small value of the academic subject effect size mean (.29) suggests that students using mobile computers find them to be less useful for academic subjects than for other purposes. This finding supports previous research. Earlier studies investigating the effects of achievement scores by subject matter yielded small positive effects (Christmann, Badgett, & Lucking, 1997; Coley, 1997). The fifth research question asked, ?What is the magnitude and direction of the overall (total) effect from these academic studies?? The academic outcome effect size means are positive in direction with three out of four moderate effect sizes. The highest effect size (.58) for knowledge of computers and the Internet suggests that learning on mobile computers is more highly associated with technology than with academic concerns. That the effect size for knowledge of computers is the highest effect size in this study provides further support for the promotion of technology as a fourth basic literacy. The average academic outcome effect size mean by category (.37) is moderate suggesting that mobile computers are more academically useful than stationary computers (Schieber, 1999; Siegle, 2001) or no computers at all (Anderson-Inman, 1996; Gardner, 1993). The sixth research question asked, ?How do the effect sizes obtained for the use of mobile computers compare with the effect sizes obtained for the use of technology in 67 general?? Both mobile computer scores and technology effect sizes are positive. Mobile computer effect sizes are smaller than technology scores for three of four comparisons. Two observations may help explain these differences. Available technology effect sizes were based on older research which compared experimental technology-using students to control non-technology-using students. The mobile computer research often compared experimental mobile computer-using students with control desktop-using students. The second observation concerns overlap of categories: mobile computers are a subset of computers, and computers are a subset of technology as previously explained. Limitations Meta analyses rely on primary research studies to report sufficient statistics to calculate effect sizes. Glass suggests that effect sizes be calculated whenever possible, even the effect size should be estimated (Bangert-Drowns & Rudner, 1991). This study was limited to effect sizes calculated on the basis of mean, standard deviation, and number; reported Cohen?s d effect sizes; or transformations from t tests, F tests, and correlations. Studies reporting only percentages were excluded due to lack of rigorous statistical analysis. ?The validity of meta analysis is dependent on the degree to which the collected data represent the total research? (LeJeune, 2002, p. 15). Although extensive and exhaustive statistical research was conducted to collect every possible relevant research project, the possibility exists that studies were missed. This study was limited to a small sample size because of the limited availability of empirical articles. Due to the limited number of U. S. K?12 articles on mobile computers, 68 international research studies were included. It is possible that differences in language, culture, and education systems could influence the effect size scores. The validity of the results may be threatened by confounding factors such as concurrent Federal, state, district, or school reforms (Penuel et al., 2002). The varied amount of computer time available to the one-on-one mobile computer participants as opposed to the amount of time available to classroom-only mobile computer participants; access to wireless, broadband, or other technologies; and changes in computer technology since the mid-1980s; all could confound the results. This threat to validity is shared by most educational research. The academic effect sizes reported in this study are predominantly small to moderate, if Cohen?s (1988) regions of the effect-size metric are applied (.2 is small, .5 is medium, and .8 is large). Glass suggests that ?particular magnitudes of effect gain meaning by reference to what is typical in similar circumstances? (Glass, 1981, p. 104; Welkowitz, Ewen, & Cohen, 1982). He further states that ?dissociated from a context of decision and comparative value, there is no inherent value to an effect size of 3.5 or .2? (Glass et al., 1981, p. 104). The ?context of decision and comparative value? in this study is to be found in Table 1, ?Effect Sizes Based on Meta analyses of Academic Outcomes on Technology.? The technology effect sizes in this table of meta-analyses range from .19 (reading) to .50 (math achievement) suggesting that a useful region of effect size metric for technology effect sizes in K?12 education would be .1 to .3 (.1 being small, .3 medium, and .5 large). If this context of decision and comparative value is applied to the results of the current 69 study, the effect sizes for academic category mean and communication and language would be medium, and the effect sizes for knowledge of computers would be large. Many researchers failed to use the unique characteristics and strengths of mobile computers in the design of their research studies. No empirical study was based on a study of mobility per se, such as student research conducted while on a biology, art, history, or music field trip. Implications for Intervention Kurt Landgraf (2005), President and CEO of Educational Testing Service, describes technology as the ?fourth basic literacy.? Work, play, communication, and defense of the nation all have become centered on technology. Given the widespread use and importance of technology, it seems that using academic outcomes as the sole rational for justifying the use of computers in the classroom is shortsighted. It is recommended that legislators, school board members, administrators, and teachers need to unite in the support of technology as a basic literacy for all children. Innovations in technology are creating new challenges for U.S. students and schools. The implications of some of these changes may be profound for the future of U.S. workers, e.g., the Massachusetts Institute of Technology?s Media Lab anticipates the production and distribution of millions of $100 laptop computers to poor school children in third world countries by early 2007 (Lewis, 2005). It is recommended that the implications of this development for U.S. students entering the job market need to be considered by U.S. national technology policy planners, state legislators, and school board members. 70 Conclusions The meta analysis reported here examined academic outcomes for academic tests, higher order thinking skills, academic subjects, and computer knowledge. This study suggests that laptops are effective across all four academic outputs examined, a wide variety of student characteristics, and varied laptop interventions. This meta analysis suggests that laptops are more effective in academic test preparation and when knowledge of the computers and the Internet is being studied. The potential of these powerful educational tools has yet to be fully explored. Optimism for the future of mobile computers in education is warranted. Study 2 A Statistical Meta analysis of the Attitudes of K?12 Student Mobile Computer Users In 2002, the Pew Internet and American Life Project published a report describing the behaviors and attitudes of U.S. K?12 students. The Digital Disconnect: The Widening Gap Between Internet-Savvy Students and Their Schools (Levin & Arafeh, 2002) addresses the frustrations, pressures, concerns, and challenges faced by students who often are more adept at technology than their teachers. The behavior of these Internet- savvy students is markedly different from pre-Internet student behavior. Communication behaviors (e.g., via chat rooms, blogging, e-mails, and instant messaging), research behaviors (e.g., via search engines, on-line libraries, and friends), entertainment (e.g., surfing for fun, games, music, and movies), economic behaviors (e.g., via ebay and 71 company web pages), and creativity (e.g., writing, art, and music programs) have fostered a digital perception of how to operate in an increasingly complex and changing world. This digital perception has produced student attitudes regarding the usefulness of technology markedly different from those of parents and teachers who were not raised in a digital environment (Prensky, 2001). It appears that the digital disconnect between the generations is becoming more pronounced as students develop further skills in using increasingly powerful miniaturized mobile electronic devices. Educators need to know about student attitudes toward mobile computers and understand how to use this knowledge to improve education. This study seeks to answer questions regarding mobile computer use effects on K?12 student affective outcomes from experimental studies from 1993 to 2004. Experimental research is supported by the U.S. Department of Education?s No Child Left Behind (NCLB) Act. Two factors are increasing researcher access to experimental education research projects. One is the Department of Education?s demand for experimental results. The other is increased access to international educational research data via the Internet. The resulting pool of data is useful for meta analytic analysis of topics such as attitudes of K?12 students towards the usefulness of mobile computers. Although the NCLB Act encourages the scientific rigor associated with true experimental studies, the effects reported in this paper are based on pre-experimental and quasi-experimental research. No true experimental studies focusing on attitudes of students have been conducted. The academic focus of the NCLB supported research may account for the recent dearth of attitude outcome studies. 72 From the initial introduction of computers into the classroom to the present, students have expressed excitement, enthusiasm, and keen interest in using school computers. Positive student attitudes are described in the literature by terms such as self- esteem, motivation, and enhanced attitude toward subject matter. These positive student attitudes have been reported by students, teachers, parents, and observers (Baker, 1989; Repman, 1993; Riel & Becker, 2000). One of the earliest research results regarding positive student attitudes towards computers was reported by the Apple Classrooms of Tomorrow (ACOT) Project (Baker, Gearhart, Herman, & Kulik, 1994). Student attitudes toward school are more positive among computer-using students than non-computer- using students (Baker, Gearhart, Herman, & Kulik, 1994). Many elementary students prefer learning by computer instead of traditional instruction (Clements, Nastasi, & Swaminathan, 1993; Kinzie, Sullivan, & Berde, 1998). Elementary student perception of computers as tools useful for increasing school success encourages positive attitudes towards computers (Breakwell & Fife-Schaw, 1987). Mobile computer-using students experience an increased sense of self-esteem on the elementary (Robertson, Ladewig, Strickland, & Boshung, 1987) and middle school levels (Breakwell, 1987; Repman, 1993). Increased academic self-esteem (Apple Classrooms of Tomorrow Library, 1996), improved problem solving (Tyler & Vasu, 1995), and increased self-esteem gains from collaborative learning (Repman, 1993) were determined to be associated with the use of computers in the classroom. DeGraw (1990) found that the use of computers both at home and at school increased student self-esteem. 73 Researchers in the 1980s and 1990s found that attitudes toward computers and use of technology differed by gender. Females were more likely to obtain lower scores than males on computer aptitude tests and attitude surveys (Hattie & Fitzgerald, 1987; Kay, 1992; Shade, 1993). Gender differences in communication and relationships (Mulvaney, 1994), in use of computers (Hattie & Fitzgerald, 1987), in achievement (Jackson, & Yamanaka, 1985), and on gender equity issues (Baran, 1987; Clarke, 1992; Filipczak, 1995) generated researcher interest. A plethora of research project results on the same topic are useful for a sophisticated analysis known as a statistical meta analysis. A statistical meta analysis is used for three reasons: to better understand apparently conflicting results; to explore relationships between the experimental variable(s) and other variables; and to provide a standardized approach to meaningful synthesis of ?different designs, data collection techniques, dependent variables, and statistical analyses? (Thomas & Nelson, p. 238). Such a meta analysis is needed to provide a better understanding of the effects of laptop use on the attitudes of students. This study is different from other mobile computer research studies in three ways: 1. It is different from studies such as the Pew Internet & American Life Project case study approach in that this study is a quantitative meta analysis. 2. Whereas some mobile computer studies used estimates from percentages to calculate effect sizes, in this study effect sizes were not calculated from percentages thus improving the likelihood of increased validity and 74 reliability in study results. Only effect sizes reported as Cohen?s d effect sizes or transformed by Cohen?s d formulas (LeJeune, 2002) were used in this study. 3. Whereas most meta-analyses in the U.S. report only U.S. research project results, this study reports both U.S. and international experimental research data. Laptop research results in Australia, the UK, Canada, and Germany were excellent contributions to this study. Research Questions Four research questions were investigated in this independent meta analysis regarding attitude outcomes: 1. How extensive is the experimental and quasi-experimental body of research related to instruments used to measure student attitudes toward mobile computers? 2. How extensive is the experimental and quasi-experimental body of research related to study characteristics used to measure student attitudes toward mobile computers? 3. Do the study characteristics in this meta analysis predict significant relationships with the effect sizes? 4. Does the independent variable methodological quality predict significant relationships with the effect sizes? 75 Methodology In this section the theory, advantages, and criticisms of meta analysis are discussed. A description of the literature search, criteria for inclusion, criteria for exclusion, and coding of the studies are described. The rationale and creation of two databases are discussed. Rationale for Meta Analysis Meta analysis is a statistical analysis of summary statistics from published empirical research reports (Cooper & Hedges, 1994). This methodology was defined by Glass: ?Meta analysis refers to the analysis of analyses ? the statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings? (Glass, 1976, p. 3). The primary strength of meta analysis lies in its use of effect sizes. The conversion of empirical research findings to this common metric allows comparison of result across multiple studies (Penuel et al., 2002). The primary weaknesses of meta analysis lies in its dependence on the accuracy of the primary research (Christman, 1995). Literature Search A comprehensive and exhaustive effort was made to gather all published and unpublished research reports on the subject of K?12 mobile computing. The literature search included using search engines such as Yahoo, Google Scholar, and Google; free databases such as Education Resources Information Center (ERIC) and World Cat; and for-fee data bases such as Gale Group and UMI Proquest. All database searches were conducted with no restrictions as to dates, languages, amount of text, or type of source. 76 Keywords searched for mobile computing included mobile computing, mobile computers, portable computers, laptops, laptop computers, personal digital assistants, PDA, Palm computers, pocket computers, notebooks, iBooks, iPaqs, ubiquitous, and wireless. Keyword searches for the student population included K?12, education, primary school, elementary school, middle school and high school. A citation search was conducted for each of the authors of statistically driven studies. Unpublished studies that fulfilled the criteria for this meta analysis were sought from authors and editors involved in the publication of research projects in the area of this meta analysis. An e-mail request for information regarding ?the name, location, and/or e-mail address of any researcher who has produced an unpublished article regarding an experimental or quasi-experimental K?12 educational mobile computer study? (see Appendix A) was sent to these experts. Of the 47 e-mails sent, eight elicited responses, with six of the respondents providing positive feedback and links. Although the survey did not provide direction to usable unpublished articles, respondents provided links and references that led to relevant research articles. Criteria for Inclusion Statistics produced by research projects regarding K?12 student use of mobile computers were included. Only attitude outcomes were included. Only results producing effect sizes or statistical data that yielded effect sizes were included. Criteria for Exclusion Non-parametric statistics and results reported as percentages were excluded because they do not produce accurate effect sizes. Statistics produced by research 77 projects regarding teacher use of mobile computers and teacher behaviors and attitudes when students use mobile computers were excluded. Studies of educational mobile computers that used only producer-dedicated software were not included. Literature Search Results Both criteria for inclusion were met by 15 out of 91 research projects obtained. These projects formed the basis of this statistical meta analysis. To analyze the data produced by research projects the statistical meta analysis required a system for organizing and synthesizing the effect sizes. Creating Databases Two master databases were created for this meta analysis. An effect size database consisted of author name, effect size, codes, and subcodes. Authors, codes and subcodes were dummy coded for sorting purposes. A methodological quality database was created to catalogue research project characteristics. Evidence of different characteristics, i.e., study, student, technology, research quality, and sources of invalidity, was coded as present or not present. Coding of Studies Two coding systems were developed for this study. Effect size descriptors were coded as attitude outcomes. Coding rules were as follows: Effect sizes produced by instruments containing the words attitudes or attitudinal in the titles were included in the study. If the instrument did not contain the word attitude in the title, all effect sizes were excluded for that instrument unless the test/survey category title was concerned with words such as attitude or self-esteem. Categorical subcodes for attitude outcomes were 78 also developed. For each of these codes and subcodes, effect size descriptors were also reported. Research project descriptors were coded as demographics, technology, student, and instructional characteristics. The number of participants, classes and schools were measured on a continuous scale. Variables ? gender, country, and type of school ?were measured with categorical scales. Membership in the category was coded as one, and lack of membership was coded as zero. Inter-rater agreement on the codes for attitude outcomes was based on percent of agreement between the two evaluators. Inter-rater agreement represented the number of times the evaluators assigned identical scores when using the evaluation forms. Inter-rater agreement was achieved on the descriptors (81%), quality (83%), and attitude outcomes (85%). Calculation of Effect Size Effect size descriptors were calculated using the Cohen?s d effect size formula. In general when evaluating the magnitude of an effect size, a .2 is indicative of a small effect, .5 of a moderate effect, and .8 of a strong effect; however, both Cohen and Glass encourage researchers to reference what is typical under similar circumstances before assigning values to effect sizes (Glass et al. 1981; Welkowitz et al. 1982). Formulas for transposing correlation, t-tests, and F-test statistics were obtained from three sources (DeCoster, 2002; Furr, 2001; LeJeune, 2002). 79 Data Analysis Titles of instruments used; instrument effect size means and standard deviations; and sample, attitude outcome, and characteristic means and standard deviations are summarized. Percentages of study characteristics are reported. Categories of samples are summarized. Results The results section reported results for each research question. Descriptive statistics, types of samples, and effect size means were reported. Tables were used to summarize and illustrate the data. The first research question asked: ?How extensive is the experimental and quasi- experimental body of research related to instruments used to measure student attitudes toward mobile computers?? A summary of the effect sizes of student attitudes toward mobile computers, categorized by instrument, is presented below. The largest effect size (2.24) was produced by the School Observation Measure (Ross, 2001); and the smallest (.09) by the student survey: What is Happening in this Class? (Kessel, 1999; see Table 6). 80 Table 6 Summary Chart of Student Attitude Effect Sizes by Instrument Author Instrument Title Number of Effect Sizes Effect Size Mean Anderson-Inman (1996) Learning and Study Strategies Inventory ? High School Version (LASSI-HS). 9 .36 Dyson (2002) Student Self-Appraisals (not standardized). 1 .57 Haynes (1996) Attitudinal Survey - researcher developed. 7 .13 Kessel (1999) Student Survey: What is Happening in this Class? (WIHIC) 11 .09 Knezek (2004) The Computer Attitudes Questionnaire 2 .49 Newhouse, Dissertation, Year 1 (1997) New Classroom Environment Instrument (NCEI) Computer Attitudes Scale (CAS). 79 2 -.45 -.83 Newhouse, Dissertation, Year 2 (1997) NCEI Computer Attitudes Scale (CAS) ? Explored negative attitudes of small group only. No effect sizes reported. 127 -.48 Newhouse, Dissertation, Year 3 (1997) NCEI classroom environment instrument Computer Use Questionnaire 24 8 -.66 .27 Priest ( 2001) Researcher-developed survey to measure perceptions/attitudes of disabled students regarding the use of laptop computers. 10 .18 Raaflaub, (2002) What is Happening in this Class? (WIHIC) 51 .82 Ross, Year 2 (2002) School Observation Measure ? (SOM) 1 2.24 Schaumburg (2001) Researcher developed a computer literacy test that included attitude items: ? Confidence in using computers: Rating scale for self-assessment of the students? subjective level of confidence in using computers. ? Computers as tool or toy: Rating scale to measure student attitudes towards computers and the Internet (tool or toy/critical reflection). 9 .66 (table continues) 81 Table 6 (continued) Author Instrument Title Number of Effect Sizes Effect Size Mean Schieber (1991) My Classroom Inventory (MCI) 11 1.00 Stolarchuk (2001) Science Classroom Environment Questionnaire (SCES Scales) 12 .11 Trimmel (2004) Questionnaire of School and Classroom Atmosphere Hermans? Questionnaire of Achievement Motivation 4 3 .74 .42 The second research question asked: ?How extensive is the experimental and quasi-experimental body of research related to study characteristics used to measure student attitudes toward mobile computers? An extensive search of the literature found 15 research projects, which yielded 379 effect sizes concerning student attitudes toward mobile computers. Pre-experimental designs accounted for 60% of the research designs and quasi-experimental designs accounted for the remaining 40% of the research designs. No experimental designs were found. Student sample sizes ranged from a minimum of 32 to a maximum of 863 (M = 173, SD = 239.47). Class sample sizes ranged from four to 73 (M = 12.8, SD = 22.34). The number of schools involved in the study ranged from one to 14 (M = 2.53, SD = 3.78) (see Table 7). 82 Table 7 Types of Samples Reported Samples Minimum Per study Maximum Per study Mean Standard Deviation Students 2 863 173 239.47 Class or Group 4 73 12.8 22.34 Schools 1 14 2.53 3.78 Study characteristics included unit of analysis and duration of study. The units of analysis were predominantly the individual student (33%), school (22%), class (17%), and grade level (11%). Duration of the study was most frequently specified as less than one year (34%), between one and two years (33%), or more than two years (33%). Student characteristics included gender, grade level, types of schools, and nationality. Gender was limited to females in 27% of the studies while mixed gender accounted for 73% of the studies. No exclusively male studies were conducted. Since many studies examined more than one grade level, a total of 54 grade levels were specified. Grade levels ranged from Grade 1 through Grade 12: grades 1 through 4 (2% each), grades 5 and 6 (8% each), grade 7 (15%), grade 8 (21%), grade 9 (15%), and grades 10 through 12 (8% each); some studies did not specify grade level. Some studies did not specify type of school, but out of the studies that did, 47% were taxpayer supported and 40% were private. Students were primarily from Australia (53%), the U.S. (33%), Austria (7%), and Germany (7%). 83 Student attitudes toward academic outcomes were reported by academic subject (26%), test scores (17%), and higher order thinking skills (17%). Reported academic content areas using mobile computers included technology (23%), mathematics (13%), science (4%), writing (4%), reading (2%), and language arts (2%). Many studies did not report academic outcomes. Applied outcomes were primarily concerned with knowledge of computers (53%), word processing (46%), the Internet (33%), and e-mail (27%). Word processing was the primary software (35%) used by students on these computers. Attitude outcomes for students when using mobile computers were reported toward mobile computers for 73% of the studies, course content or subject for 47% of the studies, and classroom learning environment for 33% of the studies. Free use of mobile computers was provided by the school district in 73% of the studies while the remaining 27% required that families lease or purchase the mobile computer for the student to be a participant in the laptop using group. Laptops or notebook computers were used by all experimental students in all studies of attitudes of students toward mobile computers. Question three asks, ?Do the study characteristics in this meta analysis predict significant relationships with the effect sizes?? The sample size was not sufficient to examine multiple predictors by regression analysis. Instead, the effect sizes means for many of the previous categories are reported in Table 8, ?Survey of Study Characteristics and Corresponding Effect Sizes?. Data was not reported when the subcategory numbers were small. The asterisk (*) indicates the influence of one large positive outlier. The caret (^) indicates the influence of negative numbers. When present, these numbers tended to 84 skew the data in opposite directions. These numbers had no influence on 15 of 19 effect size means. Table 8 Survey of Study Characteristics and Corresponding Effect Size Means Study Characteristics Number of Studies Mean Effect Size Gender Mixed Gender 11 .45 Females Only 4 -.33^ Type of School Taxpayer supported (public) 7 .46* Parents pay tuition (private) 6 -.08^ Unit of Analysis Individual 6 .23 Grades (overlap) Six?Eight 10 .39*^ Nine?Twelve 8 .32 Duration of Study Less than a year 3 .39 More than a year but less than two years 5 .22 More than two years 5 .15 Attitudes Toward Applied Outcomes Internet 5 .63 Knowledge of Computers 8 .36 Word Processing 7 .32 (table continues) 85 Table 8 (continued) Study Characteristics Number of Studies Mean Effect Size Duration of Study More than one year but less Than two years 5 .22 Three years 5 .15 Reporting the way the sample was described Target population 6 .30 Accessible population 8 .19 Sampling procedure 12 .13 Rationale provided 7 .48 *One study contained a positive outlier (2.24). ^ One or more studies contained negative effect scores (-.14, -.46, -.48, and -.66). For purposes of better understanding the data, a descriptive profile of the attitude outcomes, number, effect size mean, and the number of contributing studies for each category is presented in Table 9. Categories with the highest effect sizes tend to have limited sample sizes, e.g., attitude of enjoyment and attitude toward self. The category with the largest sample size, e.g. attitude toward classroom environment, has a small effect size. Although 15 primary studies contributed to these attitude effect sizes, 9 out of 10 categories obtained effect sizes from only one or two studies (see Table 9). 86 Table 9 Descriptive Profile of Attitude Toward Mobile Computers Effect Size Means by Category (No Significance Test Attached) Attitude Outcome Categories Total Number of Effect Sizes Reported Effect Size Mean Number of Studies Enjoy 13 .61 2 Toward Self 2 .53 2 Toward Work 11 .40 2 Toward Instructor 8 .37 2 Toward Classroom Environment* 289 ?.29 7 Toward Mobile Computer 3 .28 2 Negative attitude toward mobile computers 4 ?.27 2 Toward School 3 .25 2 Toward Equity 4 .22 1 Toward Subject Matter 26 .21 2 *All of these scores were obtained from one instrument, the New Classroom Environment Instrument (NCEI). Question four asks, ?Does the independent variable (methodological quality) predict significant relationships with the effect sizes?? The magnitude of the correlation between the effect sizes and the measure of methodological quality yielded one of the larger effect sizes in the study (-.56). Pre-experimental designs were coded as 0. Pre? experimental designs were coded as 1. The negative direction of the effect size suggests 87 that quasi-experimental designs are more likely to obtain high effect sizes than quasi- experimental designs. Results from the four research questions indicate that a more extensive empirical body of research for student attitudes is needed. Because of highly correlated variables, univariate analysis of variance and multiple regression were inappropriate. A closer examination of these results is needed. Discussion The discussion section examines the results for each of the questions. Limitations and suggestions for further research are discussed. The results of this meta analysis suggest a small, positive effect of use of mobile computers on student attitudes. The overall mean effect size of the samples was .23. The first research question asked, ?How extensive is the experimental and quasi- experimental body of research related to instruments used to measure student attitudes toward mobile computers? Instruments measuring quantitative results varied considerably by authorship, title, and resulting effect sizes. Of 10 instruments employed, three instruments were researcher-developed, and only one instrument, What is Happening in this Class? (WIHIC), was used by more than one researcher (Kessel, 1999; Raaflaub 2002). The effect sizes for all the instruments ranged from .09 to 2.24, suggesting that these instruments may have been measuring varied constructs or populations. A list of other instruments used to measure student attitudes towards computers can be found in Kessel?s report (Kessel, 1999, p. 14). 88 The second research question asked, ?How extensive is the experimental and quasi-experimental body of research related to the effects of the attitudes of K?12 students toward the use of mobile computers?? Although the number of samples was limited, the studies produced extensive descriptive information. Items of interest included gender, grade levels, and outcomes. Results for gender were skewed: Males were not the exclusive focus in any study; females were the exclusive focus for four of the samples (Newhouse, 1997; Schaumburg, 2001). Results report females compared to mixed groups, but not females compared to males. The correlation of females to effect size yielded a negative correlation (?.57) whereas the correlation of mixed groups to effect size yielded a positive correlation (+.57). Because the sample number is small, further research is recommended. Grades commonly associated with middle school (e.g. grades six, seven, eight, and nine) accounted for 59% of the grade levels studied. Fluctuating hormones levels, differences in maturity, and emotional instability characterize students in these grades. The attitudes of these adolescent students are not representative of the attitudes of all students. The generalizability of results based primarily on middle school students is limited. Many of the descriptive reports for outcomes observed that students learn more about computers than about the subject matter in the first phase of a computer program (Stolarchuk & Fisher, 2001). In describing research performed for the French Landes Initiative, Jaille (2004) compared the logs for the high school?s proxy server with student reports of Internet usage. He observed evidence of an emphasis on non-academic uses of 89 the Internet in an initial stage and a subsequent emphasis on academic uses in a second phase. Table 8, Summary of Study Characteristics and Corresponding Effect Size Means, reports effect size means for many of the study characteristic percentages. The overall effect size mean for this group of study characteristics (.24) is positive and small. The asterisk (*) indicates the influence of a large positive outlier that affected two categories: Type of school and grades six?eight. The caret (^) indicates the influence of negative numbers that affected three categories: female only gender, private school, and grades six?eight. Of the 19 categories in Table 8, 15 categories were unaffected by these numbers. A comparison of Table 8 (Survey of Study Characteristics and Corresponding Effect Size Means) and Table 6 (Summary Chart of Student Attitude Effect Sizes by Instrument) contributes to an understanding of how the effect sizes were obtained. Four instruments contributed the most positive effect sizes: the School Observation Measure (2.24), My Classroom Inventory (1.00), What is Happening in this Class? (.82), and the Questionnaire of School and Classroom Atmosphere (.74). Two instruments contributed the largest negative effect sizes: The Computer Attitudes Scale (?.83) and the New Classroom Environment Instrument (average of ?.53). These instruments may have contributed to effect sizes across categories. An inverse relationship exists between duration and methodological quality of design (Ary, Jacobs, & Razavieh, 2002). With an effect size mean of .28, eight studies contributed to the duration subcategory, 0 to one year. For this subcategory, 75% of the 90 studies were pre-experimental. With an effect size mean of .24, two studies contributed to the subcategory, one to two years, with 50% of the studies being pre-experimental. With an effect size mean of .15, five studies contributed to the duration subcategory, two or more years, with 60% of the designs being quasi-experimental. The order of the effect size means is hierarchical (.28, .24, and .15). The mean effect sizes decrease as the duration of the studies increase, suggesting that shorter studies tend to obtain higher effect sizes. Duration has an inverse relationship to methodological quality of design, with pre-experimental designs occurring most frequently during the studies with shorter duration and yielding higher effect sizes. Quasi-experimental designs occur most frequently in the studies with longer duration and yield lower effect sizes. This suggests that studies based on pre-experimental designs and short duration tend to yield higher effect sizes. Question three asked, ?Do the study characteristics in this meta analysis predict significant relationships with the effect sizes?? The planned multiple regression was not appropriate for three reasons: where significance was found, the variables were highly correlated (multicollinearity); the dummy-coded variable contained too few numbers; or both. Multicollinearity is a problem for three reasons: it complicates interpretations, increases likelihood of Type I error, and decreases ability to predict. In this case, it complicates the interpretation of the relationship between the dependent variable and independent variables owing to their highly correlated interrelationships. Multi- collinearity increases the likelihood of saying that a difference exists when one does not, otherwise known as a Type I error. Highly correlated independent variables decrease the 91 ability of independent variables to predict unique parts of the dependent variable, and thus make multiple regression inappropriate for this study. Table 9, ?Descriptive Profile of Student Attitudes Effect Toward Computers?, presented ten attitude categories. The highest effect size category, attitudes of enjoyment when using a mobile computer, has a positive, moderate effect size (.61). This result suggests that student attitudes become more positive when the students derive benefit from the computers. Two studies and thirteen effect sizes contributed to this mean, making one of the more substantial contributions to this profile. The category, attitudes toward self when using mobile computers, produced a positive and moderate effect size mean but was based on only two effect sizes and two studies. The category, attitudes towards laptop computer programs evaluated by the New Classroom Environment Instrument (NCEI), produced a negative, small effect size (-.27). Newhouse reports that students in this program were frustrated when traditional teachers did not incorporate the laptops into classroom instruction (1994). The NCEI category contains 289 effect sizes; was produced by seven primary studies; is negative in direction; has magnitudes that are moderate (?.66) or approaching moderate (?.45 and ?.48); and represents student frustration with the failure of the teachers to incorporate the laptops into their lessons. The remainder of the categories contained both small numbers and small effect size means and thus are inappropriate for regression. Question four asked, ?Does the independent variable (methodological quality) predict significant relationships with the effect sizes?? The negative correlation (-.56) suggests that pre-experimental designs are more likely to obtain higher effect sizes than 92 quasi-experimental designs. Many of these studies were associated with other reforms, e.g., the research for Ross was conducted during the reforms of the No Child Left Behind Act. The lack of rigor in design may have allowed the pre-experimental design studies to obtain significance from the noise of the dependent variable created by one or more simultaneous interventions and resulting in an inflated effect size. Further research is needed to determine if the Ary designs can consistently predict high effect sizes (Ary et al., 2002). Limitations This study had at least four major limitations: Small sample size, highly correlated data, designs lacking methodological quality, and confounded results. A small sample size has a tendency to increase error and limit the generalizability of the results. For multiple regression with two predictor variables, a sample size of 20 or more was needed (Shannon, 2000). The data was highly correlated and therefore was not useful for explaining unique portions of the dependent variable, effect sizes. The validity of a meta-analyses is limited by the validity of the primary studies. The pre-experimental and quasi-experimental designs of the primary studies limited the methodological quality of this meta analysis. Many of the primary studies were embedded in larger reform movements (Penuel et al., 2002). Early mobile computer research was embedded in teacher training and curriculum reform movements. Recently published mobile computer research is embedded in the No Child Left Behind reform movement. 93 Suggested Interventions and Recommendations for Future Research There is a need for researchers to report data according to the professional standards of the American Psychological Association (APA). Information useful in conducting research synthesis should be reported. Failure to report effect sizes and the direction of the effect are referred to as a defect in designing and reporting research (American Psychological Association, 2001). Although the reported effect sizes are small, there are at least three good reasons why the value of mobile computer programs should not be disregarded: digitally-oriented students need to prepare for an electronically mobile workplace, mobile computing is in its infancy, and lower costs can be expected. Students prepared to work in a mobile digital environment will be better prepared for the workplace of the future. Laptops, palm pilots, cell phones, and similar tools are being miniaturized, integrated, and produced at lower cost, e.g., MIT predicts production of a $100 laptop computer by early 2007 (Lewis, 2005). Further empirical research is needed on the use of one-on-one computers for educational purposes. Inequities in the ownership and distribution of mobile computers need to be addressed. Biased results can occur when researchers require families to lease or purchase computers as a requirement for participation in the experimental group. Long-term (5 or 8 year) longitudinal research projects should be initiated in the area of mobile computing, Three-year longitudinal programs note important changes in student uses and attitudes related to the passage of time. Empirical information after the third year is lacking on student attitudes in K?12 education towards mobile computers. 94 Conclusion This statistical meta analysis examined the relationship between the effects of mobile computer use and K-12 student attitudes as reported by primary empirical research studies. The goal of the study was to improve educational technology by providing precise information for educators. Although the sample size of primary studies was limited, the overall results are positive in direction. The small effect size is similar to the results of other meta analyses in technology. The study of mobile computers is not a mature field. Optimism is warranted for future investigation. Study 3 Effects of the Distribution of $100 Laptops to Students in Third World Countries Imagine a world where every school age child ?owns? a personal laptop computer with access to the Internet. According to Nicholas Negroponte of the MIT Media Lab, a $100 laptop computer being created by IBM researchers may make this dream come true for many children around the world. Negroponte projects the first distribution of these $100 laptops by the end of 2006 or early 2007 (Lewis, 2005). MIT researchers are also developing inexpensive communication technologies, e.g., mesh net-working technology and WI-FI wireless networking for these laptop students and their communities (Bray, 2005). 95 Characteristics of Students Receiving the $100 Laptops Who are the students who will receive these computers? Pilot project schools will distribute laptops to their students. Each student and laptop will become a nexus and will act as a connection between individuals or groups in their schools and communities. A web of student laptops will enable communication via mesh technology for the village. With inexpensive Internet access, a laptop-proficient student will become an informal librarian, ebay trader, a conduit to the outside world, and a bridge to an alternate future. In this paper, these students will be referred to as E-nexees. This paper will explore some of the social and educational implications of laptops for the E-nexees and for the distribution of millions of $100 laptop computers in third world countries. School Use of Personal Laptops E-nexees will probably be expected to bring their laptops to school every day. Since every student will have a laptop computer, teachers will need to accommodate their instruction to the use of the laptops. Research from international studies on the school use of laptop computers suggests that laptops will be most effective for student academic outcomes if the students are learning about knowledge of computers and the Internet (Cassil & Ross, unpublished; Jaillet, 2004). The laptops will also be effective if they are used for preparing for academic tests and for higher order thinking skills (Cassil & Ross, unpublished). The E-nexees and their teachers are likely to experience two initial phases: In the first phase, the students will be more concerned with learning technology than academics. In the second phase, the students will use their technological skills to focus on academic concerns (Jaillet, 2004; Stolarchuk, 2001). Laptop-using students tend to have 96 positive attitudes if they can use the laptops in their lessons. Students tend to have negative attitudes towards the laptop program if teachers ignore the presence of the laptops and teach in a traditional lecture approach only (Newhouse, 1997). Personal Use International research suggests that many E-nexees will exhibit new digital modes of thinking (as opposed to their parental/societal modes of thinking). Inventiveness, higher self-esteem, and high frequency and duration of use of the laptops will characterize E-nexee use of laptops. They will have preferences for their own indigenous language and music. It is possible that many of these students will develop a preference for or become supporters of the open software movement. Immersion in digital technology will reshape the way the students think and learn. According to Mark Prensky, author of Digital Natives, Digital Immigrants, children raised in a digital environment think and learn differently than those raised in non-digital environments (Prensky, 2001). Current digital natives are children born to technological privilege. Having been immersed in a digital environment, these children prefer/demand the following: ? Receiving information rapidly, ? Random access (like hypertext), ? Parallel processing and multi-tasking, ? Graphics before their text, ? Being networked, ? Educational electronic/online games rather than routine assignments, and 97 ? Instant gratification and frequent rewards (Prensky, 2001). International research on educational uses of laptop computers supports several of the digital native characteristics. Rapid receipt of information; parallel processing and multi-tasking; online games; and instant gratification and frequent rewards were supported by a report on the Landes initiative, the first large-scale laptop research program in France and Europe (Jaillet, 2004). Jaillet?s research reads like a precautionary tale of what happens when bright, creative French high school students are each given laptops and access to broadband and traditional teachers ignore the laptops. Although these research results are relevant for current digital natives, E-nexees were not born into a digital environment. It is possible they will develop some unique characteristics of their own. As a result of their use of laptops and access to the Internet, it is likely that E-nexees will enrich the Internet; improve communication systems and economic opportunities in their communities; demonstrate inventiveness; improve their self-esteem; engage in frequent use of laptops; increase the use of their native languages and music on the Internet; and possibly contribute to the open software movement. E-nexees will Enrich the Internet What will happen when millions of students from the developing world access and take up residence on the Internet? The number of languages and dialects used on the Internet and the frequency of those languages will increase with the distribution of $100 computers to third world countries. Although Asia accounted for 56.3% of the world?s population as of April 2005, the Asian penetration of the Internet is only 8.4% (Miniwatts International, Inc., 2005). Along with their languages, many E-nexees will contribute 98 their unique cultural perspectives and ways of thought to the Internet. These perspectives will enrich the shared human knowledge base on the Internet (Bushweller, 2005). Economic Opportunities Communication systems and economic opportunities will improve for E-nexee families and communities. Changes in local communication patterns will result from the mesh network. ?In a mesh network, every machine acts as a relay point, sending data to every other machine? (Bray, 2005, p. 1). E-mail and instant messaging will provide a communication system in villages with no current electronic communication. E-nexees will teach their siblings, parents, and extended family members how to access this technology. Local economic change is likely as the home use of the laptop will affect the development of the family, neighborhood, and entire village (Siddle, 2005). Connection to the Internet and services like ebay will increase opportunities for economic development. E-nexees will function as the nexus between the economies of the past and the future. E-nexees will demonstrate inventiveness and creative problem solving. Research tells us that when teachers fail to structure laptop activities for students ?owning? laptops, the students will appropriate the laptops and Internet resources for their personal interests at school as well as at home. As previously mentioned in Jaillet?s study of three one-on- one laptop schools, teachers ignored the laptops and the broadband Internet access. The students used the Internet ?very excessively? for electronic messaging, downloading videos and music, zapping (many sites viewed in a short time), and to escape from boring 99 classes. The students seldom visited educational sites, did not communicate electronically with teachers, and made minimal use of the laptops for educational purposes. An evaluation of proxy server logs found chat, private sites (e.g., sites hosted by Internet Service Providers), and hobby/sport/leisure sites accounted for the most data transfer time. The researcher hypothesized that all these distractions would negatively impact national final exam scores. Interestingly enough the national final exam scores for the experimental laptop schools were no different than for the non-laptop schools (Jaillet, 2004). Research is needed to determine the effects of student creativity and problem- solving ability when channeled into Internet-engaged problem-solving educational activities in a laptop saturated school. Student Self-Esteem The self-esteem of laptop-using children will increase proportionately to their skill in operating the computer and in finding useful information/services on the Internet. If the future imitates the past, these young E-nexees will become the teachers of their parents, teachers, and peers (Strom & Strom, to be published; Tapscott, 1997). These changes may have the greatest impact on paternalistic societies. Duration and Frequency of Use The duration and frequency of use will probably be higher for developing world students than U.S. students. E-nexee students, teachers, and families are more likely to use the laptops more frequently and for longer periods if only because they will not be as distracted by other gadgets as U.S. students. In U.S. homes, television, music, video games, and movies distract students from educational uses of the Internet. According to 100 the Kaiser Family Foundation, U.S. students spend approximately eight and a half hours per day under the influence of various forms of media (Elias, 2005). Some U.S. schools have computer labs and classroom computers that are under-utilized for various reasons (Bushweller, 2005). Technology Counts ?04 Global Links: Lessons From the World reports that Mexican schools had less access to these services than U.S. schools but exhibited proportionately greater use than U.S. schools (Bushweller, 2005). Open Software Movement E-nexees may develop a commitment to the open software movement. The free Linux operating system, a sophisticated type of open software, will be provided on the laptops. It is possible that E-nexees will develop a commitment to the use, support, and creation of new open software. Whereas Microsoft has copyrighted and standardized the Microsoft operating system, an international open-software movement is challenging the entire copyright mentality. A new digital divide could emerge: international, young, poor, open-software supporters versus western, older, richer Microsoft customers. Native Language and Music E-nexees will look for their own language and music. When Sugata Mitra, a New Delhi physicist, built a touch-screen laptop into a wall frequented by poor New Delhi children, the children quickly taught themselves computer basics such as drawing on the computer and browsing the Internet. When Mitra played a MP3 digital music file for them but provided no instructions, the students swiftly ?located all the Hindi music and pulled it out? (Judge, 2000). Wide distribution of $100 laptops into highly populated countries such as India and China will change the types of languages, services, and 101 products sold on the Internet. One billion Indians speak Hindi. More than 1.3 billion Chinese speak Chinese dialects. Students receiving these laptops will be a force in re- shaping the Internet. Implications of E-nexee Use of Laptops for U.S. Practitioners It is time for an increased sense of urgency in the U.S. regarding student technology skills. Perhaps because we created computers and the Internet, we assume we are the experts and will remain the experts. This assumption is a mistake. This mistake is compounded by the fact that educators in the U.S. and around the world have no means of measuring K?12 student technological expertise (Kozma, 2003). The U.S. has committed billions of dollars to educational technology (Summary of E-Rate Bill in the 106 th Congress, 1999) without requiring measurement of results. The authors propose a national test of K?12 educational technology skills for students. For this assessment, paper and pencil testing are inappropriate; an Internet- based problem-solving assessment of student technology skills is required. The format, tempo, and complexity of the assessment should reflect the preferences of Digital Natives as defined by Prensky (2001). The assessment might be in the form of a simulation or a game with a problem-solving focus. The competitive element would be much like a decathlon, suggesting a program name of Cyberathelon. Electronic data collection from student teams, pre-programmed instant feedback, sophisticated statistical analysis, and online posting of results are essential to the success of this program. 102 In The Second Century of Ability Testing: Some Predictions and Speculations, Susan Embretson provides a statistical/measurement rationale for the proposed assessment procedures, e.g., continuous test revision, automated validity studies, web- based delivery of tests, flexible mixtures of evidence for ability, and broad conceptualization of what constitutes a ?test item? (Embretson, 2001). The philosophical foundation for this assessment is based on Sugata Mitra?s self-organizing systems and collaborative or minimally invasive approaches to learning (Mitra, 2000). A National Technology Assessment The authors propose the following concepts for the Cyberathelon as a national assessment of K?12 student technology programs. This approach of using a competitive event to assess skills would have many of the characteristics of The Interscholastic Future-Oriented Problem-Solving Bowl developed by Torrance in 1979 (as cited in Strom & Bernard, 1982). State, county, district, and school programs would be assessed, not individual students. The assessment would not be standardized for three reasons: Technology changes rapidly; schools own different amounts and types of technology; and assessments based on hands-on problem-solving must make use of timely problems, e.g. the problems must change. The Cyberathelon would be inclusive rather than exclusive: public school, private school, and home-schooled students would be invited to participate. Students would need to collaborate with team members to achieve higher criterion levels. The Cyberathelon topic would demand higher order thinking skills involved in solving complex problems. Although the questions will be kept secret until the moment 103 of transmission on the assessment day, students will be able to make general preparations in advance. The assessment would be criterion referenced with the criterion for technological skills and abilities being established well in advance for each grade level. Criteria for classification of participants (wizard, master, journey person, and newbee) would be determined and publicized before the assessment date. The question content would involve current, relevant problems in a game-based format. Recognition and rewards would be provided in a post-event ceremony. Unlike standardized tests that guarantee failure for students in the bottom quartile, the criterion referenced approach offers potential reward for students who prepare for the specific criterion. The above Cyberathelon assessment characteristics match the preferences of today?s Digital Natives (see Table 10 below). Table 10 Preferences of Digital Natives (Prensky, 2001) Compared to Assessment Characteristics Digital Native Preferences Assessment Characteristics Receiving information rapidly Timed tests and the nature of the problem-solving tasks would require information to be exchanged rapidly. Random access (like Hypertext) Material could be approached in any order as opposed to a specific order or set of steps. Parallel processing and multi-tasking Videos, text, music, cell-phone calls, and text messages could be simultaneously accessed and processed. Graphics before their text Topic would present graphics before text. (table continues) 104 Table 10 (continued) Digital Native Preferences Assessment Characteristics Being networked Networking technologies, e.g., Internet and voice transmission would be implemented in the assessment. Educational electronic/online games rather than routine assignments Online problem would be presented as a simulation or game. Instant gratification and frequent rewards Pre-programmed instant feedback would be sent to students when they did something right/smart/profitable. A central administrative organization would be necessary to provide electronic collection, analysis, and reporting of statistical results. This organization would also provide a web site to clarify the rules, control rumors, and accumulate public suggestions for future questions/problems. State and local educational administrators would work with their local university(s) in analyzing and reporting data. County practitioners would assist in finding and assigning outside monitors and analyze monitor reports. District and school administrators would provide outreach to home-schooled students, assign teams, provide training on the criterion theme and skills, and arrange for the physical and electronic aspects of the hands-on problem-solving competitions. The date would be established in the first semester of the school year. This is not meant to be a cumulative test of one year?s teaching: It is meant to be an overall assessment of students? abilities to access information, use higher order thinking skills, write the final report, and demonstrate deep knowledge of varied technologies and programs. Sustainability for the Cyberathelon may be most likely if the assessment is sponsored by businesses rather than government. 105 Standardized test publishers have a vested interest in keeping K?12 assessment dollars focused on standardized tests of the three basic literacies of reading, writing, and arithmetic. Public schools and universities have a vested interest in promoting low cost empirical assessment of the fourth basic literacy, technology. Statistical results would be useful for establishing cost-benefit assessment for program evaluation, planning, budget proposals, and grant applications. Legislatures tend to support programs demonstrating positive cost benefit. These results would form the basis of an annual national reassessment of efficient use of technology resources in education. Partnerships between schools and technological companies for the purpose of pilot testing and developing innovations would keep these annual reassessments informed regarding possible new directions for Cyberathelon assessments. Criterion referenced testing would enable flexibility as technology changed. The on-line availability and minimal cost of this approach would make it an excellent assessment approach for third world countries. As the distribution of $100 laptops increases, the need for program evaluation will increase also. Non-Government Organizations (NGOs) contributing to the funding of the $100 laptops need to be able to efficiently assess the benefit of their contributions via electronic feedback. Conclusion Distribution of the $100 laptops offers unprecedented opportunities for third world students. It also will create a vast pool of connected low-cost labor. We can choose to ignore this development at the peril of future generations of U.S. workers. The 106 alternative is to choose to take stock of our national progress in educational technology and to commit to a rigorous program of improvement of technology education programs. 107 V. DISCUSSION, RECOMMENDATIONS, LIMITATIONS, AND CONCLUSIONS Discussion This study employed a statistical meta analytic technique to synthesize findings across multiple primary studies for the purpose of systematically examining the effects of student use of mobile computers. A review of the literature found a concentration of empirical research studies in the areas of attitude and achievement outcomes. Two independent meta analyses were conducted. The results will be synthesized and evaluated in the following discussion. Additionally, results will be compared to and contrasted with the findings of other researchers. Limitations, recommendations, and a conclusion will be presented. The overall effect of mobile computers on the attitudes and academic outcomes was small and positive, suggesting that student use of mobile computers is more effective than student use of stationary computers or no computers at all. The two independent analyses each yielded small and positive effect sizes. These findings are consistent with previous meta analyses in technology (Waxman, 2003) and with meta analysis findings regarding educational interventions in general (Penuel et al., 2002). Characteristics of the primary studies were studied for the purpose of determining relationships between student use of mobile computers and student attitudes and 108 academic outcomes. In How Science Takes Stock: The Story of Meta analysis, Glass is quoted as saying: What I?ve come to think meta-analysis really is ? or rather, what it ought to be ? is not little single-number summaries such as ?This is what psychotherapy?s effect is? but a whole array of study results that show how relationships between treatment and outcome change as a function of all sorts of other conditions ? the age of the people in treatment, what kinds of problems they had, the training of the therapist, how long after therapy you?re measuring change, and so on. That?s what we really want to get ?a total portrait of all those changes and shifts, a complicated landscape rather than a single central point. That would be the best contribution we could make. (Hunt, 1997, p. 163) ?Complicated? ? with ?changes and shifts? is a good description of the relationships evaluated in this study. In this meta analysis, change was measured with Cohen?s d effect sizes, with .8 as large, .5 as moderate, and .2 as small. The description of the effect size relationships were organized by magnitude of effect size. Magnitude of Effect Sizes Largest Effect Sizes The largest effect sizes were found for three areas: characteristics of authors, year of publication, and instruments used to measure attitudes. The largest effect sizes for the entire meta analysis were produced in an analysis of instruments used to measure attitudes of students using mobile computers. These surveys, questionnaires, self- 109 appraisals, inventories, scales, and measures contained at least one question regarding attitudes specific to the mobile computers or to the associated program. Certain authors had studies that tended to yield higher effect sizes. The categories of instrument and author mean for the Anytime, Anywhere Learning: Final Evaluation Report of the Laptop Program ? 2001 yielded the highest effect sizes (Ross, 2001). Effect size results from the School Observation Measure (SOM) and the Ross author mean suggest that the Anytime Anywhere Learning program was more effective for yielding larger positive effect sizes for both student attitude and academic achievement than any other study in the meta analysis. Large positive mean effect sizes found for five authors (Lowther, 2003; Raaflab, 2002; Ross, 2000, 2001; Siegle, 2001) suggest that the intervention reported in these studies, mobile computers, was successful in improving student attitudes and academic outcomes. Of the remaining author mean effect sizes, moderate effect sizes were found for four studies, small effect sizes were found for five studies, and minimal effect for eight studies. One negative author mean effect size was inconsistent with the above findings and suggests that the mobile computer interventions in those studies were not successful (Schieber, 1999). Moderate Effect Sizes Moderate and positive effect sizes were found for student use of mobile computers for Internet use, e-mail, chat, and instant messaging; student knowledge of computers; enjoyment; and self-esteem. Student attitudes toward the Internet were found to be moderate and positive. This measure was the highest effect size magnitude found 110 for the independent meta analysis regarding student attitudes. Internet-driven communication, multi-media, e-mail, chat, and instant messaging were all highly correlated in the independent meta analysis regarding attitudes. These results are consistent with the findings of another study of student use of the Internet (Jaillet, 2004). These results also support earlier findings that students using laptops increased the frequency and duration of their use of the Internet (Rockman, 2000). Student knowledge of computers was found to have moderate and positive effects. This finding supports non-experimental results regarding increased levels of computer literacy for laptop-using students (Rockman, 2000). On surveys of self-rated level of proficiency, laptop students from non-experimental studies considered themselves to be highly proficient (Pfeifer & Robb, 2000; Vahey & Crawford, 2002; Waker, 2001). Use of mobile computers is moderately effective in increasing student perceptions of enjoyment. These results support findings of non-experimental researchers (Christensen & Knezek, 2001; Rockman, 1997; Vahey & Crawford, 2002). These outcomes are consistent with the results of another study concerning mobile computer- using students? use of the Internet (Jaillet, 2004). Use of mobile computers is effective in increasing students? positive attitudes towards themselves. These findings are supported by earlier research on desktop computers (Apple Computer, 1996; Breakwell, 1987; Repman, 1993; Robertson, Ladewig, Strickland, & Boshung, 1987). By contrast, Newhouse (1997) found that approximately 5% of every group of students appears to have feelings of anxiety, fear of 111 damaging the computer, or some other type of negative self-perception when in the presence of computers. Small Effect Sizes Small positive attitude effect sizes were found for student use of mobile computers for the following categories: mixed gender; female gender only; taxpayer- supported mobile computer purchases; and for student attitudes towards work, mobile computers, school, equity, and subject matter. Small and positive academic effect sizes were found for academic test outcomes and higher order thinking skills. Academic outcomes by subject matter ? exclusive of testing ? were found to have small positive effects: math, reading, writing, science, and communication. Gender attitude differences related to ownership of a mobile computer were investigated with small and negative effect sizes found for student use of mobile computers by female-only students. Mixed-gender student effect sizes approached moderate. A substantial body of research suggests that female attitudes towards computers are less positive than males (Hattie & Fitzgerald, 1987; Kay, 1992; Schaumburg, 2001; Shade, 1993). Further research is needed regarding effects produced by all-male student populations. Results for taxpayer-supported laptop loans to students were positive and approached moderate. This result should be viewed with caution as this measure was influenced by small sample size and by large positive and negative values in the control group. More research is needed to determine the effects of requiring parents to purchase 112 laptops as a condition of the student participation in an experimental laptop group. This issue appears not to have been previously researched by empirical studies in the U.S. Mobile computer-using students? attitudes towards work were positive and approached moderate effectiveness. This outcome was consistent with the findings of improved laptop student motivation towards homework (One-to-one Laptops, 2004). These results are consistent with results of earlier stationary computer research indicating that computer-using students were more engaged, independent, and produced higher levels of work than non computer-using students (Coley, 1997). Small and positive effects were found for student attitudes toward mobile computers. Public schools in the laptop research programs frequently loaned laptop computers to all the students on a given grade level. Positive student attitudes towards the laptops often resulted from the self-perceived increase in status gained from the use of devices often associated with business people and professionals (Vahey & Crawford, 2002). This finding is consistent with non-experimental findings that one-to-one ?ownership? of mobile computers promotes positive attitudes towards them (Gardner, 1994; Pfeifer & Robb, 2001; Siegle, 2001; Stevenson, 1998). Mobile computer-using students? attitudes toward school were found to be more positive than non mobile computer-using students. This effect was small and positive and is consistent with non-experimental research that suggests that mobile computer-using students had higher rates of school attendance and lower rates of tardiness than non- laptop-using students (Stevenson, 1998). This result supports the non-experimental 113 finding that the ability to take mobile computers home appears to encourage more positive attitudes towards school (Waker, 2001). The category of equity concerned special needs, minority and female students. In research on computers, lack of equity is primarily based on lack of access to computers. The finding of small and positive effect sizes regarding attitudes toward equity supported non-experimental research findings of positive academic gains for laptop-using special education students and females (Apple Computers, 1996; Christensen & Knezek, 2001; Stevenson, 1998). By contrast, Stevenson found that non-laptop-using free- and reduced- lunch students, males, and racial minority students experienced declines in academic test scores (Stevenson, 1998). Small and positive effects were found for attitudes toward subject matter. These results were in contrast to an earlier finding that attitudes of desktop-using students toward subject matter were virtually zero (Coley, 1997). This suggests that mobile computers were useful in achieving a benefit in addition to that already obtained from stationary computers. Small and positive effect sizes found for academic test outcomes were consistent with the non-experimental findings regarding the sustained and academically relevant use of computers (Stevenson, 1998). Populations containing many students and schools frequently had difficulty in the collection of data. Small and positive findings were not consistent with the findings of researchers who had difficulty obtaining reliable and valid standardized test scores for mobile computer studies (Fouts, 1997; Rockman, 2000). 114 Small and positive effect sizes were found for higher order thinking skills (knowledge, comprehension, application, analysis, synthesis, and evaluation). These results were consistent with non-experimental research involving problem solving (Rockman, 1998) and analysis skills (Stevenson, 1999). This category emerged out of the constructivist reform movement of the 1980s and 1990s. Complex problem solving was more likely to be taught in laptop classrooms than in non-laptop classrooms (Weyker, 2002). The author postulates that the current emphasis on NCLB testing of the basic three R?s appears to have curtailed further U.S. research into the relationship between higher order thinking skills and student use of mobile computers. Academic outcomes by subject matter ? exclusive of testing ? were found to have small positive effects for math, reading, writing, science, and communication. These results support earlier findings of researchers examining achievement scores by subject (Christmann, Badgett, & Lucking, 1997). These findings are also consistent with meta analysis results found for technology interventions concerning math, reading, and writing (Penuel et al., 2002). Recommendations The author notes the disparity of small effect sizes for students educational uses of technology compared to the pronounced effectiveness of technology for science, health, banking, and communication. The author argues that this troublesome disconnect between the perceived lower value of technology for education than for the rest of the world is embedded in our concepts regarding basic literacy. In the Agricultural and 115 Industrial Ages, basic literacy was defined as reading, writing, and arithmetic. In the late 1900s basic literacy was defined as follows: ?An individual?s ability to read, write, and speak in English, and compute and solve problems at levels of proficiency necessary to function on the job and in society to achieve one?s goals, and develop one?s knowledge and potential? (National Literacy Act of 1991, Sec. 3). The author proposes that a relevant definition of literacy today should include the proficient use of the tools of technology to calculate, research, organize, evaluate, integrate, transfer, secure, and store data; to engage in collaborative activities; and to communicate verbally and in writing at levels necessary to function in a knowledge society. When schools focus their use of technology on support for NCLB standardized testing type lessons, the old definition of basic literacy limits technology to supporting reading, writing, and math. The diagram below represents an alternate view of the relationship between technology and the old basic skills. Technology is represented as a broad circular area encompassing the old basic literacy skills. The small dark triangle represents the area of reading, writing, and math assessed by NCLB standard tests (see Figure 1). 116 Figure 1. Relationship Between Technology and Area Tested by NCLB Standardized Tests Under the No Child Left Behind Act (NCLB), what researchers perceived as being empirical testing of the use of technology in education has actually been measuring the overlap between reading, writing, math, and technology. This measure is unrepresentative and limits the validity of currently reported effect sizes representing technology. Stated a different way, mobile computers and other technologies appear to have minimal value because technology is being primarily utilized for a limited purpose, the teaching skills necessary to pass the nationally required tests. Until the full potential of technology is utilized for education and the effects are accurately measured, the true value of technology for education will not be known. Inverse Relationships Inverse relationships were found regarding longitudinal studies and methodological quality. In longitudinal studies, duration of study had a hierarchical 117 inverse relationship to magnitude of effect size. Student attitudes became less positive (more negative as time progressed in four separate studies (Gulek, 2005; Newhouse, 1997; Ross, 2002, 2003; Schieber, 1999). Schieber offered several explanations for the deterioration in the laptop computer classrooms. He observed that the novelty of having something new wore off, students and teachers were overwhelmed by both the amount and complexity of the change, and that integrating laptops into the classroom environment is an enormous task (Schieber, 1999). Newhouse observed that the integration of computers into the curriculum could not be accomplished by most teachers. Many teachers were concerned that computer-dependent students would not do well on high-stakes exams. The advanced courses in the second and third years made more complex demands on the students. ?A considerable amount of standard classroom work involved taking notes, reading textbooks, and answering questions. These activities did not lend themselves to using a tool such as a computer. As a result the students tended to give up on trying to use the computers? (Newhouse, 1999, p. 15). An inverse relationship was found for methodological quality of design. Stated another way, the relationship suggests that studies using pre-experimental design are more likely to yield larger effect sizes than quasi-experimental designs. These results were based on limited sample size, and should be interpreted with caution. This meta analysis appears to have found four categories of effect sizes not previously measured in other computer and mobile computer meta-analyses. Effect sizes were obtained regarding mobile computer-using student attitudes concerning Internet- driven communication and multimedia, public versus private acquisition of laptops for 118 experimental groups, higher order thinking skills, and spoken communication and language. Further research is needed in all of these areas. Negative Effect Sizes Two studies produced negative mean effect sizes (Chang, 1998; Newhouse, 1997; Schieber, 1999). In both of these studies, the authors suggest that the teachers had difficulty implementing a one-on-one laptop program. Schieber stated that the added complexity of integrating laptops into the classroom may have led to a deterioration in the classroom environment. Lack of knowledge of and access to useful software and a perception that laptops were non-essential characterized many teachers in the Newhouse laptop program. In the first year of the Newhouse three-year longitudinal study, a group of over 20% of the students appeared to exhibit negative attitudes. By the end of the year these students had overcome their negative attitudes, improved their skill and knowledge, and were making significant use of the computers (Newhouse, 1997). Newhouse investigated this group of students in the second year of the study. While the investigation of anomalies is a commendable approach to experimental research, the mean average effect size was influenced in a negative direction by this approach (Newhouse, 1994). Limitations Two weaknesses related to meta-analyses in general were addressed in this study: Lack of methodological quality of the primary studies and publication bias. The relationship of the methodological quality of the primary studies to the author mean 119 effect sizes was investigated in the independent meta analysis regarding attitudes (Cassil & Shannon, unpublished). A preliminary finding suggests an inverse relationship between design type and effect size magnitude. Publication bias is produced by publication of studies based exclusively on statistically significant results. A search for unpublished studies was conducted via e-mail to address publication bias. Researchers, editors of technical educational research journals, directors of technical education conferences, and leaders of technical educational associations or special interest groups were asked to provide unpublished studies that fulfilled the criteria for this meta analysis. The non-response rate was high. Of the few that did respond, the primary respondents were research authors. No unpublished documents were obtained. In spite of these efforts, the possibility remains that some unpublished studies were not found and that publication bias limits this study. The statistics reported for the independent meta analyses results are based on limited sample sizes. The total meta analysis was based on 31 samples obtained from 25 primary studies. In the case of student attitudes regarding mobile computers, only descriptive statistics were useful due to the limited sample size. With a limited sample of 15 attitude studies, effect size means were easily skewed by a few outlier scores. These results should be interpreted with caution. Simultaneous reform movements tend to confound interpretation of results. Federal, state, district, and school reform movements affecting their programs were frequently discussed by authors of studies. Many of the early studies were embedded in 120 curriculum and teacher training reform movements. The current NCLB program is influencing research designs. The methodological quality of the designs of primary studies in this meta analysis was limited. Only two of the designs were true experimental. The remainder of the designs were divided almost equally between pre-experimental and quasi-experimental designs. The author attempted to compare stationary control groups against control groups using no computers. This effort was thwarted as any authors provided incomplete descriptions regarding the use of stationary computers or the lack thereof by control groups. Control groups were mixed between stationary and non-computer users in at least one study (Ross, 2000, 2001, 2003). Suggested Interventions and Recommendations for Research No primary studies were found that based their empirical research on mobility issues; e.g., the use of mobile computers on biology, art, history, music, or community field trips. Non-empirical studies discussed uses of mobile computers on field trips but the lack of empirical data limits the usefulness of such studies. Current trends in technology toward miniaturization and mobility suggest an increased likelihood of the usefulness of mobile computers for these purposes. Empirical research is needed regarding student use of Internet?related communication devices with an emphasis on determining value for educational purposes. Measurement of student skills in technology should be considered a national priority in 121 education. A national/international assessment of the skills, knowledge, and problem- solving capabilities of students is needed now. Further research is needed regarding the preliminary finding that suggests an inverse relationship between type of experimental design and magnitude of effect size. Comparisons with other meta analyses would be useful in determining whether this finding is an aberration due to small sample size or to some other factor. More generally, further research is needed to determine a reliable and valid measure of methodological quality for primary studies used in meta analyses. Conclusions The meta analysis reported here examined the effectiveness of student use of mobile computers as reported in published and online literature in the period of 1993? 2005. The analysis found a consistent pattern of positive effect size results, indicating that mobile computer use is effective in improving student attitudes and for academic outcomes. The small sample size found for this research synthesis suggests that research regarding mobile computing is in its infancy. Further meta analyses may provide a better understanding of the relationship of mobile computing and learning with technology. Optimism is warranted. 122 REFERENCES *References marked with an asterisk indicate studies included in the meta-analysis (American Psychological Association, 2001, p. 222). 21 st Century Skills. (2005). Learning points associates. Retrieved 5/2/05 at http://www.ncrel.org/enguage/skills/basiclit.htm. American Psychological Association. (2001). Publication manual of the American psychological association (5 th ed). Washington, DC: Author. Anderson-Inman, L., & Horney, M.A. (Ed.). (1998). Transforming text for at-risk readers. Mahwah, NJ: Lawrence Erlbaum Associates. *Anderson-Inman, L., Knox-Quinn, C., & Horney, M. A. (1996). Computer-based study strategies for students with learning disabilities: Individual differences associated with adoption level. Journal of Learning Disabilities, 29, 461?484. Apple Computer. (2003). Apple classrooms of tomorrow (ACOT) Library (1996 to the present). Retrieved April 10, 2003 from http:www.apple.com/education/k12/ leadership/acot/library.html Apple Computer. (1996). Research findings on technology?s impact in the classroom. The impact of technology on student achievement. Retrieved January 10, 2003 from http://www.apple.com/education/research/ 123 Apple Computers. (1995). Changing the conversation about teaching, learning, and technology. A report on 10 years of ACOT research. Retrieved February 11, 2003 from http://images.apple.com/education/k12/leadership/acot/pdf/10yr Ary, D., Jacobs, L. C., & Razavieh, A. (2002). Introduction to research in education (6th ed.). Belmont, CA: Wadsworth/Thompson Learning. Ashmore, B. A. (2001). The relationship between models of student laptop computer use and teacher instructional behavior. Unpublished doctoral dissertation, University of North Texas. Retrieved on May 10, 2004 from http//www.iittl.unt.edu/iittl/ dissertations/ashmore.pdf Bain, A., & Ross, K. (2000). School reengineering and SAT-1 performance: A case study. International Journal of Education Reform, 9, 148?153. Bain, A. & Smith, D. (2000). Technology enabling school reform. T.H.E. (Technological Horizons in Education) Journal, 28, 90. Baker, E. L., Gearhart, M., & Herman, J., Kulik, J. (1994). Evaluating the Apple classrooms of tomorrow. Technology assessment in education and training. Hillsdale, NJ: Lawrence Erlbaum Associates. Baker, E. L. (1989). The ACOT report card: Effects on complex performance and attitude. Los Angeles, CA: California University. Bangert-Drowns, R. L. (1993). The word processor as an instructional tool: A meta- analysis of word processing in writing instruction. Review of Educational Research, 63(1), 69?93. 124 Bangert-Drowns, R. L., & Rudner, L. M. (2003). Meta-analysis in educational research. ERIC Digest. Retrieved September 2, 2003 from http://www.ericae.net/edo/ ED339748 Bangert-Drowns, R. L., & Rudner, L. M. (1991). Meta-analysis in educational research. Practical Assessment, Research & Evaluation, 2(8). Retrieved May 12, 2005 from http://www.pareonline.net/ Baran, G. (1987). Teaching girls science. In M. McNeil (Ed.), Gender and expertise (pp. 87?102). London: Free Association Books. Barker, A. B., & Torgeson, J. K. (1995). An evaluation of computer-assisted instruction in phonological awareness with below average readers. Journal of Educational Computing Research, 13, 89?103. Barrish, T. (2005). ETS Launches ICT Literacy Assessment, an online measure of student information and communication technology proficiency. ETS News & Media Communication & Public Affairs. Retrieved April 18, 2005 from http://www.ets.org.news Bayraktar, S. (2001-2002). A meta-analysis of the effectiveness of computer-assisted instruction in science education. Journal of Research on Technology in Education, 34, 173?188. Beasley, R. E., & Waugh, M. L. (1996). The effects of content-structure focusing on Learner structural knowledge acquisition, retention, and disorientation in a hypermedia environment. Journal of Research on Computing in Education. 28(3), 271?281. 125 Becker, J., & Riel, M. (2000). Teaching professional engagement and constructivist compatible computer use. Center for Research on Information Technology and Organizations. University of California, Irvine, CA. Retrieved October 18, 2004 at http://caret.iste.org/ Becker, J., & Riel, M. (1998). Teaching professional engagement and constructivist compatible computer Use. Irvine, CA: University of California, Irvine, and University of Minnesota, Center for Research on Information Technology and Organizations. Retrieved October 18, 2004 from http://caret.iste.org/ index.cfm?fuseaction=readinglist&studyid=433 Blok, H., Oostdam, R., Otter, M. E., & Overmatt, M. (2002). Computer-assisted instruction in support of beginning reading instruction: A review. Review of Educational Research, 72, 101?130. Blumenthal, P. G. (2003). Evaluating ubiquitous access to technology: Teacher and administrator perceptions of effective classroom practices. Doctoral Dissertation, Virginia Commonwealth University, Richmond, VA. AAT 3081852. Bray, H. (2005) A $100 laptop to change the world. The New York Times Company. Retrieved April 25, 2005 from www.boston.com/ Breakwell, G. M., & Fife-Schaw, C. (1987). Young people?s attitudes toward new technology: Source and structure. In J. H. Lewko (Ed.), How children and adolescents view the world of work (pp. 51?67). San Francisco: Jossey-Bass Inc. Bushweller, K. (Ed.) (2005). Technology counts 2004: Global links: Lessons from the world. Retrieved April 3, 2005 at http://edweek.org 126 Cassil, K. M., & Ross, M. E. (2005). A statistical meta analysis of the effectiveness of laptop computers on student academic outcomes. Unpublished manuscript. Cassil, K. M., & Shannon, D. M. (2005). A statistical meta analysis of attitudes of K?12 student mobile computer users. Unpublished manuscript. Cassil, K. M., & Strom, P. S. (2005). Effects of the distribution of $100 laptops to students in third world countries. Unpublished manuscript. Carrucan, T., Crewe, T., Matthews, E., & Matthews, S. (1998). The internet manual for teachers. London, UK: Macmillan. *Chang, H., Henriquez, A., Honey, M., Light, D., Moeller, B., & Ross, N. (1998). The Union City story: Education reform and technology ? Students? performance on standardized tests. Report produced by the Center for Children & Technology. Retrieved on January 4, 2005 at http://www2.edc.org/ Chen, T. (1994). A meta-analysis of effectiveness of computer-based instruction in mathematics. Unpublished Doctoral Dissertation, University of Oklahoma, Tulsa, OK. Christensen, R., & Knezek, G. (2001). Equity and diversity in K?12 applications of information technology: KIDS project findings for 2000?2001. Denton, TX: University of North Texas. Institute for the Integration of Technology into teaching and learning. ISBN 1-931410-02-x. Retrieved on May 10, 2005 from http://www.iittl.unt.edu/KIDS2001/html/entirebook.htm Christmann, E. (1995). A meta-analysis of the effect of computer-assisted instruction on the academic achievement of students in grades 6 through 12: A comparison of 127 urban, suburban, and rural educational settings. Unpublished Doctoral Dissertation, Old Dominion University, Norfolk, VA. Christmann, E., & Badgett, J. (1999). A comparative analysis of the effects of computer- assisted instruction on student achievement in differing science and demographical areas. Journals of Computers in Mathematics and Science Teaching, 18, 135?143. Christmann, E., Badgett, J., & Lucking, R. (1997a). Microcomputer-based computer- assisted instruction within differing subject areas: A statistical deduction. Journal of Educational Computing Research, 16, 218?296. Clarke, V. (1992). Strategies for involving girls in computer science. In C. D. Martin and E. Murchie-Beyma (Eds), Search of gender free paradigms for computer science education. Eugene, OR: International Society for Technology in Education. Clark, R. E. (1991). When researchers swim upstream: Reflections on an unpopular argument about learning from the media. Educational Technology, 31(2), 34?40. Clements, D. H., Nastasi, B. K., & Swaminathan, S. (1993). Young children and computers: Crossroads and directions from research. Young Children, 48, 56?64. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates. Coley, R. J. (1997). Technology?s impact. Electronic School. Retrieved October 3, 2003 from http://www.electronic-school.com/0997f3.html Cooper, H., & Hedges, L. (Eds.). (1994). The handbook of research synthesis. New York: Russell Sage Foundation. 128 Cramer, S., & Smith, A. (2002). Technology?s impact on student writing at the middle school level. Journal of Instructional Psychology, 29, 3?14. Darwinmag.com. (2003). Retrieved on December 5, 2003 from http://guide.darwinmag. com/techology/personal/pda Dauite, C. (1986). Physical and cognitive factors in revising: Insights from studies with computers. Research in the Teaching of English, 20, 141?59. DeCoster, J. (2002). Meta-analysis notes. Amsterdam, The Netherlands: Author. Retrieved on May 25, 2004 from www.bama.ua.edu/~jamie/meta/metachap.pdf DeGraw, B. C. (1990). A study to determine changes in social interaction among students, teachers, and parents influenced by the placement of microcomputers in the homes and school of fourth-grade Fuqua students during 1988?1989. Dissertation Abstracts International, 51/05-A. Dillman, D. A., & Christian, L. (2002). The influence words, symbols, numbers, and graphics on answers to self-administered questionnaires: Results from 18 experimental comparisons. Retrieved October 19, 2004 from http://survey.sesrc.wsu.edu/dillman/papers.htm. *Dyson, M., & Cairns, L. (2002). An alternative to the traditional educational program for year nine students: A new issue to research in an unchanging system. Paper presented to the Annual Conference of the Australian Association for Research in Education, Brisbane, Australia. Retrieved February 2, 2005 from http://www.aare.edu.au/02pap/dys02138.htm 129 Elais, M. (2005). Electronic world swallows up kids? time, study finds. USA Today, March 10, 1. Embretson, S. E. (2001). The second century of ability testing: Some predictions and speculations (pp. 1?36). Princeton: Educational Testing Service Research and Development Policy Center. Encyclopedia.com. (2004). Retrieved on August 23, 2004 from www.encyclopedia.com Etchinson, C. (1989). Word processing: A helpful tool for basic writers. Computers and Composition, 6, 33?43. Filipczak, B. (1995). Different strokes: Learning styles in the classroom. Training, 32(3), 43?48. Fletcher-Flinn, C., & Gravatt, B. (1995). The efficacy of computer assisted instruction (CM): A meta analysis. Journal of Educational Computing Research, 12, 219? 242. Fouts, J., & Stuen, C. (1997). Copernicus Project: Learning with laptop year 1 evaluation report. (ERIC Document Reproduction Service No. ED 416 847). Funkhouser, C. (1993). The influence of problem solving software in students? attitudes about mathematics. Journal of Research on Computing in Education, 25, 339? 346. Furr, R. M. (2004). Useful effect size formulae. Retrieved on May 10, 2005 from www.wfu.edu/users/furrm/Research.htm Gardner, J., Morrison, H., Jarman, R., Reilly C., & McNally, H. (1994). Learning with portable computers. Computer & Education, 22, 161?171. 130 *Gardner, J., Morrison, H., & Jarman, R. (1993). The impact of high access to computers on learning. Journal of Computer Assisted Learning, 9, 2?16. Gasch, S. (1996). Alan Kay. Retrieved on January 25 at http://ei.sc.vt.edu/history *Gasque, E. M. (2000). Graphing calculators versus graphing calculators and handheld data collection technology: A comparative study on student understanding and interpretation of slope. UMI Number: 3006031. Gess-Newsome, J., Clark, J., & Menasco, J. (2003). Ubiquitous computing: Seeds of a technological or pedagogical revolution? Paper presented at the Annual Meeting of the Association for the Education of Teachers of Science, St. Louis, MO. Glass, G. V. (1976). Primary, secondary, and meta-analysis. Educational Researcher, 5, 3?8. Glass, G. V. (2003). Meta-analysis at 25. Retrieved 9/12/03 from http://glass.ed.asu.edu/ gene/papers/meta25.html Glass, G. V, McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage Publications. Goldberg, A., Russell, M., & Cook, A. (2003). The effect of computers on student writing: A meta-analysis of studies from 1992 to 2002. Journal of Technology, Learning, and Assessment, 2(1). Retrieved February 18, 2003 from http://www.jtla.org *Gulek, J. C., & Demirtas, H. (2005). Learning with technology: The impact of laptop use on student achievement. The Journal of Technology, Learning, and Assessment, 3(2). Found May 22, 2005 at http://www.jtla.org 131 Hannafin, M. J., & Dalton, D. W. (1987). The effects of word processing on written composition. Journal of Educational Research, 80, 338?342. Hattie, J., & Fitzgerald, D. (1987). Sex differences in attitudes, achievement and use of computers. Australian Journal of Education, 31(1), 3?26. *Haynes, C. (1996). The effectiveness of using laptop computers with middle school students identified as being inhibited writers. (UMI Number: 9630228). Higgins, K., & Boone, R. (1991). Hypermedia CAI: A supplement to an elementary school basal reader program. Journal of Special Education Technology, 11, 1?15. Hillel, J., Kieran, C., & Gurtner, J. (1989). Solving structured geometry tasks on the computer: The roll of feedback in generating strategies. Educational Studies in Mathematics, 10, 1?39. History of Laptop Computers ? History. Retrieved July 8, 2003 from http://inventors.about.com/library/inventors/bllapton.htm Hounshell, P. B., & Hill, S. R. (1989). The computer and achievement attitudes in high school biology. Journal of Research in Science Teaching, 6, 543?549. Hunt, M. (1997). How science takes stock: The story of meta-analysis. New York: Russell Sage Foundation. *Jaillet, A. (2004). What is happening with portable computers in schools? Journal of Science Education and Technology, 13, 115?128. Jackson, L., & Yamanaka, E. (1985). Measuring women's attitudes, goals and literacy toward computers and advanced technology. Educational Technology, 25(2), 12? 14. 132 Judge, P. (Ed.) (2000). New Delhi physicist Sugata Mitra has a radical proposal for bringing his country?s next generation into the Info Age. Businessweek Online Daily Briefing. Retrieved May 3, 2005 from http://www.greenstar.org/ butterflies/Hole-in-the-Wall.htm Kay, R. (1992). Understanding gender differences in computer attitudes, aptitude and use: An invitation to build theory. Journal of Research on Computing in Education, 25(2), 159?171. *Kessel, S., & Deschamp, M. (1999). Innovation and best practice project report: Evaluation of the personal laptop program at Penrhos College. Penrhos College, COMO, WA, Australia. Kinzie, B. M., Sullivan, J. H., & Berde, L. R. (1998). Learner control and achievement in science computer-assisted instruction, Journal of Educational Psychology, 80, 299?303. Kinzie, M. B., Sullivan, H. J., & Berdel, R. L. (1992). Motivational and achievement effects of learner control over content review within CAI. Journal of Educational Computing Research, 8(1), 101?114. *Knezek, G., & Christensen, R. (2004). Student home access and the digital divide: An exploratory analysis of laptop computer access with take-home privileges versus restriction to use only in school. The Maine Learning with Laptop Study. Research Brief MLLS0402. Retrieved January 20, 2005 at http://www.mcmel.org Kozma, R. (2003). Technology and classroom practices: An international study. Journal of Research on Technology in Education, 36, 1?14. 133 Kulik, C., & Kulik, J. A. (1991). Effectiveness of computer-based instruction: An updated analysis. Computers in Human Behavior, 71, 75?94. Kulick, C., Kulick, J., Bangert-Drowns, R. L., & Williams, G. (1983). Effects of computer-based teaching on secondary school students. Journal of Educational Psychology, 75, 19?26. Kulik, J. A. (1994). Meta-analytic studies of findings on computer-based instruction. In E. Baker & H. O?Neil (Eds.), Technology assessment in education and training. Hillsdale, NJ: Lawrence Erlbaum Associates. Kurth, R. (1987). Using word processing to enhance revision strategies during student writing activities. Educational Technology, 27, 13?19. Landgraf, K. (2005). The fourth basic literacy: Technology. ETS News & Events. Retrieved May 18, 2005 from http://www.ets.org/ Lavoie, D.R. & Good, R. (1998). The nature and use of prediction skills in a biological computer simulation. Journal of Research in Science Teaching, 2, 335?360. Lazarowitz, R., & Huppert, J. (1993). Science process skills of 10 th -grade biology students in a computer-assisted learning setting. Journal of Research on Computing in Education, 25, 366?382. Leaf, R. C. (1993). Control, volition, and the ?experimental method.? New Ideas in Psychology, 11(1), 3-33. Lee, J. (1999). Effectiveness of computer-based instructional simulation: A meta- analysis. International Journal of Instructional Media, 26, 71?85. 134 LeJeune, J. V. ( 2002). A meta-analysis of outcomes from the use of computer- simulated experiments in science education. Digital Dissertations, (AAT 3058170). Lerew, E. L. (1997). The use of computers to improve writing skills among low-achieving Hispanic students. Unpublished doctoral dissertation, University of La Verne, La Verne, CA. Levin, D., & Arafeh, S. (2002). The digital disconnect: The widening gap between Internet- savvy students and their schools. Washington, DC; Pew Internet & American Life Project. Lavoie, D., & Roth W. M. (Eds.). (1998). Models for science teacher preparation: Bridging the gap between research and practice. New York: Kluwer Academic Publishers. Lewis, N. (2005). The MIT Media Lab is launching a new program to develop a $100 laptop. Retrieved May 3, 2005 from niav@media.mit.edu or http://laptop.media.mit. edu/ *Lewis, S. K. (2004). The relationship of full-time laptop computer access to student achievement and student attitudes in middle school. UMI Number: 3129418. Liao, Y. K. (1992). Effect of computer-assisted instruction on cognitive outcomes: A meta-analysis. Journal of Research on Computing in Education, 24(3), 367?380. Lou, Y. K., Abrami, P. C., & d?Apollonia, S. (2001). Small group and individual learning with technology: A meta-analysis. Review of Educational Research, 71, 449?521. 135 *Lowther, D. L., Ross, S. M., & Morrison, G. R. (2003). The laptop classroom: The effect on instruction and achievement. Educational Technology, Research, and Development, 51, 23?44. Lu, E., Voss, B., & Kleinsmith, L. (1997). The effects of a microcomputer-based biology study center on learning in high school biology students. The American Biology Teacher, 59, 270?278. MacArthur, C. A., Haynes, J. A., & Malof, D. B. (1986). Learning disabled students? engaged time and classroom interaction: The impact of computer assisted instruction. Journal of Educational Computing Research, 2, 198?198. Mann, D., Shakeshaft, C., Becker, J., & Kottkamp, R. (1998). West Virginia story: Achievement gain from a statewide comprehensive instructional technology program. Santa Monica, CA: Milken Exchange on Educational Technology. *Martin, R. A. (2004). The effectiveness of laptop computer use in an elementary school to affect test scores. UMI Number: 1419635. McCoy, L. (1996). Computer-based mathematics learning. Journal of Research on Computing in Education, 28, 438?460. McKenna, M., & Watkins, J. (1996). The effects of computer-mediated trade books on sight word acquisition and the development of phonics ability. Paper presented at the National Reading Conference, Charleston, SC. *McTeer, D. E., Jr. (2004). An evaluation of the South Carolina laptop program to improve SAT scores. UMI Number: 3130471. Medwell, J. (1996). Talking books and reading. Reading, 30, 41?46. 136 Metcalf, S. J., & Tinker, R. (2003, March 23?26). TEEMSS: Technology Enhanced Elementary and Middle School Science. Presentation at the Annual Meeting of the National Association for Research in Science Teaching, Philadelphia, PA. Miniwatts International, Inc. (2005). Internet World Stats Usage and Population Statistics. Retrieved April 4, 2005 from http://www.internetworldstats.com/ stats3.htm Mitchell, M. J., & Fox, B. J. (2001). The effects of computer software for developing phonological awareness in low-progress readers. Reading Research and Instruction, 40, 315?332. Mitra, S. (2000, June 6). Children and the Internet: New paradigms for development in the 21 st Century. Keynote speech: Asian Science and Technology Conference, Tokyo Japan. *Muir, M., Knezek, G. & Christensen, R. (2004). The Maine learning technology initiative: An exploratory study of the impact of ubiquitous technology on student achievement. The Maine Learning with Laptop Study. Research Brief MLLS0401. Retrieved January 20, 2005 from http://www.mcmel.org Mulvaney, B. (1994). Gender differences in communication: an intercultural experience. Retrieved March 3, 2005 from http://www.cpsr.org/cpsr/gender/mulvaney.txt/ National Literacy Act of 1991, Pub L. No.102-73, 105 Stat. 333 (1991). Retrieved February 17, 2003, from http://novel.nifl.gov/public-law.html 137 Newhouse, P. (1999). Portable computers supporting secondary school learning. Proceedings Western Australian Institute for Educational Research Forum 1999. http://education.curtin.edu.au/waier/forums/1999/newhouse.html *Newhouse, P. C. (1997). Teachers? responses and classroom learning environments associated with student access to portable computers. Doctoral Dissertation, Curtin University of Technology, Australia. Newhouse, P. C. (1994). Creating computer supported learning environments: A three- year study. In J. Steele and J. G. Hedberg (Eds.), Learning environment technology: Selected papers from LETA 94. Canberra: AJET Publications. Retrieved on July 27, 2003 from http://cleo.murdoch.edu.au/gen/aset/confs/ edtech94/mp/newhouse.html Niemiec, R., & Walberg, H. J. (1987). Comparative effects of computer-assisted instruction: A synthesis of reviews. Journal of Educational Computing Research, 3, 19?37. EJ 349 632. Niemiec, R. P., & Walberg, H. J. (1985). Computers and achievement in elementary schools. Journal of Educational Computing Research, 1, 440. Nixon, G. (1992). The integration of computer software with printed materials to enhance the reading skills of middle school students. Nova University, Davie, FL. (ED 350 560). North Central Regional Educational Laboratory (NCREL). (2003). Retrieved January 11, 2003 from http://www.ncrel.org 138 North Central Regional Educational Laboratory. (2002). A quantitative synthesis of recent research on the effects of teaching and learning with technology on student outcomes. Learning Point Associates. Retrieved October 20, 2004 from http://www.ncrel.org/tech/effects/intro.htm One-to-one laptops in a high school environment. (2004). Great Maine schools project at the Senator George J. Mitchell Scholarship Research Institute. Piscataquis Community High School Study Final Report. Retrieved on February 4, 2004 from www.greatmaineschools.org Ouyang, R. (1993). A meta-analysis: effectiveness of computer-assisted instruction at the level of elementary education (K?6). Unpublished doctoral dissertation, Indiana University of Pennsylvania, Indiana, PA. *Owston, R. D., & Wideman, H. H. (2001). Computer access and student achievement in the early school years. Journal of Computer Assisted Learning, 17, 433?443. Owston, R. D. (1991). Effects of word processing on student writing in a high-computer- access environment. Technical Report 91-3. North York, Ontario: York University, Centre for the Study of Computers in Education. Penuel, W., Kim, D., Michalchik, V., Lewis, S., Means, B., Murphy, R., Korbak, C., Whaley, A., & Allen, J. (2002). Use of educational technology in home and school: A review of the knowledge base and case studies of best practice. SRI International, SRI Project 11060. Prepared for the Planning and Evaluation Service, U.S. Department of Education DHHS Contract # 282-00-008-Task 1. Menlo Park, CA: SRI International. 139 Persky, S. E. (1992). ?I want to know about ... inquiry-based learning in middle schools.? Schools in the Middle: Theory Into Practice, 2(1), 3?6. Pfeifer, R. S., & Robb, R. (2001). Beaming your school into the 21 st century. Principal Leadership, 1, 1? 4. Retrieved August 23, 2003 from http://www.principals.org/ news/beamingschool.cfm Prensky, M. (2001). Digital natives, digital immigrants. On the Horizon, 9(5), 1?6. Retrieved April 25, 2005 from www.marcprensky.com/writing/default.asp *Priest, N., & May, E. (2001). Laptop computers and children with disabilities: Factors influencing success. Australian Occupational Therapy Journal, 48, 11?24. *Raaflaub, C. A., & Fraser, B. J. (2002). Investigating the learning environment in Canadian mathematics and science classrooms in which laptop computers are used. Paper presented at the Annual Meeting of the American Educational Research Association at New Orleans, LA. (ERIC Document Reproduction Service No. ED 465 521) Reich, R. (1991). The work of nations: Preparing ourselves for 21st century capitalism. New York: A. A. Knopf. Repman, J. (1993). Collaborative computer based learning; Cognitive and affective outcomes. Journal of Educational Computing Research, 9, 149?163. Riel, M. (1990). Computer mediated communication: A tool for reconnecting kids with society. Interactive Learning Environments, 1, 255?263. 140 Riel, M., & Becker, H. (2000). The beliefs, practices, and computer use of teacher leaders. Paper presented at the American Educational Research Association, New Orleans, LA. Riley, R. W. (2000). Remarks as prepared for delivery by U.S. Secretary of Education Richard W. Riley, Web-Based Education Commission, Washington, DC, February 2, 2000. Retrieved January 28, 2003 at http://www.ed.gov/Speeches/02- 2000/20000202.html. Ringstaff, C., Haymore, J., Dwyer, D. (1991). Trading places: When teachers utilize student expertise in technology-intensive classrooms. ACOT Report # 15. Summary of paper presented at the annual meeting of the American Educational Research Association, 1991, Boston. Robertson, E., Ladewig, B., Strickland, M., & Boshung, M. (1987). Enhancement of self- esteem through the use of computer assisted instruction. Journal of Educational Research, 80, 314?316. Rockman, ET AL. (2000). A more complex picture: Laptop use and impact in the context of changing home and school access. A project for Anytime Anywhere Learning by Microsoft Corporation and Notebooks for Schools by Toshiba America Information Systems. San Francisco, CA: Author. Rockman, ET AL. (1998). Powerful tools for schooling: Second year study of the laptop program. A project for Anytime Anywhere Learning by Microsoft Corporation and Notebooks for Schools by Toshiba America Information Systems. San Francisco, CA: Author. 141 Rockman, ET AL. (1997). Report of a laptop program pilot. A project for Anytime Anywhere. Learning by Microsoft Corporation and Notebooks for Schools by Toshiba America Information Systems. San Francisco, CA: Author. *Ross, S. M., Morrison, G. R., Lowther, D. L., & Plants, R. T. (2003). Anytime, anywhere learning: Final Evaluation Report of the Laptop Program. Retrieved February 28, 2005 from www.memphis.edu/crep *Ross, S. M., Lowther, D. L., & Morrison, G. R. (2001). Anytime, anywhere learning: Final Evaluation Report of the Laptop Program. Retrieved February 28, 2005 from www.memphis.edu/crep *Ross, S. M., Morrison, G. R., Lowther, D. L., & Plants, R. T. (2000). Anytime, anywhere learning: Final Evaluation Report of the Laptop Program. Retrieved February 28, 2005 from www.memphis.edu/crep Ryan, B. (1991). Dynabook revisited with Alan Kay. Byte, 16, 43. Sandholtz, J. H., Ringstaff, C., & Dwyer, D. C. (1992). Teaching in high-tech environments: Classroom management revisited. Journal of Educational Computing Research, 8, 479?505. Schacter, J. (2001). The impact of education technology on student achievement: What the most current research has to say. Santa Monica, CA: Milken Exchange on Education Technology. *Schaumburg, H. (2001, July 25?27). Fostering girls? computer literacy through laptop learning: Can mobile computers help to level out the gender difference? Paper presented at the National Educational Computing Conference, ?Building on the 142 Future?, in Chicago, IL. Retrieved December 15, 2004 at http://confreg.uoregon. edu/necc2001/program *Schieber, C. E. (1999). Information technology and education: An evaluation of a school laptop computer program. UMI Number: 9923680. Schofield, J. W., & Verban, D. (1988). Computer usage in teaching mathematics: Issues which need answers. In D. A. Grows & T. J. Cooney (Eds.), Effective mathematics teaching (Vol. I, pp. 169?193). Hillsdale, NJ: Erlbaum. Shade, L. (1993). Gender issues in computer networking. Presented at Community Networking: The International Free-Net Conference. Retrieved March 3, 2005 from http://www.cpsr.org/cpsr/gender/leslie_regan_shade.txt/ Shannon, D. M., & Davenport, M. A. (2000). Using SPSS to solve statistical problems: A self-instruction guide. Upper Saddle River, NJ: Merrill Prentice Hall Publishers. Sherman, D. (2004). Technology and teaching children to read. What does the research say? Northeast and the Islands Regional Technology in Education Consortium (NEIRTEC). Retrieved March 13, 2005 from http://www.neirtec.org/ reading_report/report.htm Siddle, J. (2005). Digital guru floats sub-$100 PC. BBC Go Digital. Retrieved April 4, 2005 from news.bbc.co.uk/go/pr/fr/-/1/hi/technology/4243733.stm *Siegle, D., & Foster, T. (2001). Laptop computers and multimedia and presentation software: Their effects on student achievement in anatomy and physiology. Journal of Research on Technology in Education, 34, 29?37. 143 Silvernail, D. L., & Harris, W. J. (2003). The Maine learning technology initiative: Teacher, student, and school perspectives mid-year evaluation report. Maine Education Policy Research Institute. Retrieved March, 20, 2005 from http://www.usm.maine.edu/cepare/pdf/ts/mlti.pdf Simmons, M., & Cope, P. (1993). Angle and rotation: Effects of different types of feedback on the quality of response. Educational Studies in Mathematics, 24, 163?176. Simmons, M., & Cope, P. (1990). Fragile knowledge of angle in turtle geometry. Educational Studies in Mathematics, 21, 375?382. Sitko, M. C., & Crealock, C. M. (1986). A longitudinal study of the efficacy of computer technology for improving the writing skills of mildly handicapped adolescents. Paper presented at the Invitational Research Symposium on Special Education Technology, Washington, DC. Sivin-Kachala, J. (1998). Report on the effectiveness of technology in schools, 1990? 1997. Washington, DC: Software Publisher?s Association. Sivin-Kachala, J., Bialo, E. (2000). Research report on the effectiveness of technology in schools. Retrieved July 20, 2004 http://www.sunysuffolk.edu/Web/Central/ InstTech/projects/iteffrpt.pdf Smith, P. M. (1995). Reinventing schools: The technology is now. National Academy of Sciences. Retrieved February 12, 2003 from http://www.nap.edu/readingroom/ books/techgap/navigate.cgi 144 Stevenson, K. R. (1999). Evaluation Report ? Year 3 Middle School Laptop Project at Beaufort County School District. Report prepared by Department of Educational Leadership and Policies, University of South Carolina. Retrieved October 20, 2004 from http://www.beaufort.k12.sc.us/district/evalreport3.html. Stevenson, K. R. (1998). Evaluation Report ? Year 2 Schoolbook Laptop Project at Beaufort County School District. Report prepared by Department of Educational Leadership and Policies, University of South Carolina. Retrieved January 15, 2003 from http://www.beaufort.k12.sc.us/district/ltopeval.html *Stolarchuk, E., & Fisher, D. (2001). The effect of using laptop computers on achievement, attitude to science and classroom environment in science. Issues in Educational Research, 11, 1?11. Retrieved January 5, 2005 from http://education.curtin. edu.au/iier11/stolarchuk.html Strom, R., & Bernard, H. (1982). Educational psychology. Belmont, CA: Wadsworth. Strom, R., & Strom, P. (2005). Teaching adolescents and learning from them. Manuscript submitted for publication. Study: PDA?s Enhance Education. (2001). NAVGLOBE: Journal of Global Navigation and Wireless Communication, September 10, 2003. Retrieved September 20, 2003 from http://www.navglobe.com Summary of E-Rate Bill in the 106 th Congress. (1999). Tech Law Journal. Retrieved April 19, 2005 from http://www.techlawjournal.com/cong106/erate/default.htm Tapscott, D. (1997). Growing up digital: The rise of the net generation. New York: McGraw-Hill. 145 Technology?s impact. Electronic school. Retrieved October 3, 2003 from http://www.electronic-school.com/0997f3.html Thomas, J. R., & Nelson, J. K. (2001). Research methods in physical activity. Champaign, IL: Human Kinetics. Thornberg, D. (2002). The New Basics: Education and the Future of Work in the Telematic Age, Association for Supervision and Curriculum Development, Alexandria, VA. Thornton, C. (2003). Laptop era dawns. PC World, October, 2003. Retrieved on January 8, 2005 from http://www.pcworld.com/news/article/0,aid,112213,00.asp Thornberg, D. (2002). The new basics: Education and the future of work in the telematic age. Alexandria, VA: Association for Supervision and Curriculum Development. *Trimmel, M., & Bachmann, J. (2004). Cognitive, social, motivational and health aspects of students in laptop classrooms. Journal of Computer Assisted Learning, 20, 151?158. Topic: Student Learning: Research Evidence. (2005). Center for Applied Research in Educational Technology (CARET). International Society for Technology in Education in partnership with Educational Support Systems. Retrieved on March 13, 2005 from http://caret.iste.org/ Tyler, D. K., & Vasu, E. S. (1995). Locus of control, self-esteem, achievement motivation and problem-solving ability: Logo writer and simulation in the fifth grade classroom. Journal of Research on Technology in Education, 28(1), 98? 120. 146 U. S. Department of Education. (2005). A national education technology plan: The future is now. Retrieved January 6, 2005 from http://nationaledtechplan.org/ U. S. Department of Education. (2001). Public Law 107-110, the No Child Left Behind Act of 2001. Retrieved on March 2005 from http://www.ed.gov/policy/elsec/ leg/esea02/ index.html Vacc, N. N. (1987). Word processor versus handwriting: A comparative study of writing samples produced by mildly mentally handicapped students. Exceptional Children, 54, 156?165. Vahey, P., & Crawford, V. (2002). Palm? education pioneers program: Final evaluation report. Retrieved March 28, 2004 at www.palmgrants.sri.com van?t Hooft, M. (2003). Integrating handhelds into the K?12 curriculum: The KSU PEP research hub. Presented at the NECC 2003 Conference. Waker, M. L. (2001). The predictors of student achievement in computer-integrated learning environments. Doctoral Dissertation, Wayne State University, Detroit, MI. Warning Over Schools? Use of Computers. (1999). British Broadcasting Company (BBC) News, Retrieved January 2002 from http://news.bbc.co.uk/1/hi/education/ 482352.stm Waxman, H. D., Lin, M. & Michko, G. M. (2003). A meta-analysis of the effectiveness of teaching and learning with technology on student outcomes. Learning Point Associates. Retrieved January 20, 2005 from www.learningpt.org. 147 Waxman, H. C., Connell, M. L., & Gray, J. (2002). A quantitative synthesis of recent research on the effects of teaching and learning with technology on student outcomes. North Central Regional Educational Laboratory (NCREL). Retrieved on September 15 2004 from www.ncrel.org Waxman, H. C., & Huang. S. (1996). Classroom instruction differences by level of technology use in middle school mathematics. Journal of Educational Computing Research, 14(2), 157?169. Retrieved February 18 from http://www.ncrel.org/ tech/effects/intro.htm Welkowitz, J., Ewen, R. B., & Cohen, J. (1982). Introductory statistics for the behavioral sciences. San Diego, CA: Harcourt Brace Javanovich, Publishers. Weiser, M. (2004). Retrieved September 21, 2004 from http://www.ubiq.com/ hypertext/weiser/UbiCACM.html Wenglinsky, H. (1998). Does it compute? The relationship between educational technology and student achievement in mathematics. Retrieved on June 15, 2003 from www.ets.org/research/pic/technolog.html Weyker, B. S. (2002). A study of the pedagogical beliefs, practices, and computer use of classroom teachers. Doctoral Dissertation, The University of North Carolina, Chapel Hill, NC. UMI number: 3053340. Windschitl, M., & Sahl, K. (2002). Tracing teachers? use of technology in a laptop computer school: The interplay of teacher beliefs, social dynamics, and institutional culture. American-Educational Research Journal, 39, 165?205. 148 Worthen, B. R., Van Dusen, L. M., & Sailor, P. J. (1994). A comparative study of the impact of integrated learning systems on students? time-on-task. International Journal of Educational Research, 21, 25?37. Zollman, A., Oldham, B., & Wyrick, J. (1989). Effects of computer-assisted instruction on reading and mathematics achievement of Chapter 1 students. (Report No. Ed 313024). Columbus, OH: ERIC Clearinghouse for Science, Mathematics and Environmental Education. (ERIC Document Reproduction Service No. ED 313 024) *Zurita, G., & Nussbaum, M. (2004). A constructivist mobile learning environment supported by a wireless handheld network. Journal of Computer Assisted Learning, 20, 235?243. 149 APPENDICES 150 APPENDIX A EXAMPLE OF E-MAIL REQUEST REGARDING UNPUBLISHED RESEARCH. 151 To: Scholarly Researchers Subject line: Request for assistance regarding a statistical meta analysis Message: I am a doctoral candidate at Auburn University. I am conducting a meta-analysis of the use of mobile computers by K-12 students in the United States and other countries. I have researched databases to locate published research in this field. Because this method requires me to query all experts in the field regarding their knowledge of unpublished studies, I am requesting assistance from you. I hope you will be able to e-mail me the name, location, and/or e-mail address of any researcher who has produced an unpublished article regarding an experimental or quasi-experimental K-12 educational mobile computer study. Thank you for any assistance you can provide. Sincerely, Kathleen Cassil 152 APPENDIX B SAMPLE SURVEY ? ASSESSING THE METHODOLOGICAL QUALITY OF JOURNAL ARTICLES FOR A STATISTICAL META ANALYSIS 153 Assessing the Methodological Quality of Journal Articles for a Statistical Meta analysis 1. Author (Last name first). ________________________________________________ ____________________________________________________________________ 2. Title.________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ 3. Year of Publication. ____________________________________________________ 4. Policy Level (Unspecified = 0; School = 1; District = 2; State = 3; National = 4; Other = 5). ______ 5. Focus (Unspecified = 0; Reducing achievement gaps = 1; Increased use of technology = 2; Increased specific type of use = 3; Academic achievement = 4; Increase effective study strategies = 5; Integration of computer technology across the whole curriculum = 6; Accountability = 7; Psycho Social Factors (Classroom environment) = 8; Improved student attitudes = 9. ______ Demographics of Study 6. Student Sex (Not specified = 0; Males = 1; Females = 2; Mixed = 3). ______ 7. Grade Level (Unspecified = 0; Specified = 1) __________________________________________________________________ __________________________________________________________________ 8. Unit of Analysis. (Unspecified = 0; Individual = 1; Class = 2; Grade level = 3; School = 4; District = 5; State = 6; mixed = 7). _______ 9. Student Sample Size (Report actual sample size; e.g., 4,024). __________________________________________________________________ __________________________________________________________________ 10. Number of classes in study. __________________________________________ 11. School Sample Size (Report number of schools involved in study). __________________________________________________________________ __________________________________________________________________ 154 12. Students? Ethnicity (Unspecified = 0; Black = 1; Hispanic = 2; Asian = 3; White = 4; Mixed = 5; Other = 6; Asian = 7). ______ 13. Students? Socioeconomic Status (Unspecified = 0; Lower = 1; Lower middle = 2; Middle = 3; Upper middle = 4; Upper = 5; Mixed = 6). ______ 14. Country (Unspecified = 0; US = 1; Australia = 2; England = 3; Canada = 4; Germany = 5; New Zealand = 6; Austria = 7; Northern Ireland = 8, France = 9, Chile = 10). ______ 15. US Geographical Regions (Unspecified = 0; Northeast = 1; Southeast = 2; Midwest = 3; South Central = 4; Southwest = 5; Northwest = 6; Northeast = 7; Mixed = 8; Other = 9). ______ 16. School Type (Unspecified = 0; Tax payer supported = 1; Private = 2; Special school = 3; Mixed = 4; Relevance to Special Education = 5; Other = 6). ______ 17. Community Type (Unspecified = 0; Urban = 1; Rural = 2; Suburban = 3; Mixed = 4; Other = 5). ______ 18. Content Area (Content area where technology is used. Unspecified = 0; Reading = 1; Mathematics = 2; Social studies = 3; Science = 4; Writing = 5; Language arts = 6; Foreign language = 7; Technology = 8; Mixed = 9). ______ Technology Characteristics 19. Type of Mobile Computer. (Proprietary software only = 1; Handheld (PDA) = 2; Laptops and notebook computers = 3; Wearable = 4). ______ 20. Software [Unspecified = 0; Tutorial = 1; Drill-and-practice = 2; Exploratory environment (e.g., simulations, microworlds, hypermedia, and hypertext) = 3; word processing = 4; Internet (e-mail, Web, text-messaging, search engines) = 5; Drawing tools = 6]. ______ 21. Tools for other tasks (printing, saving onto materials such as CDs, Internet telephone). (Unspecified = 0, Specified = 1). ______ 22. Technology Resources/Support Available (Unspecified = 0; No resources = 1; Minimal resources = 2; Adequate resources = 3; Ample resources = 4). ______ 23. Role/Focus of Technology (Unspecified = 0; Productivity in school = 1; Use at home = 2; Increasing learning = 3; Gender Equity = 4; Increase Computer Knowledge skills = 5). ______ 155 24. Quantity of Technology (Unspecified = 0; Few [< 3 per classroom] = 1; Average [4?8 per classroom] = 2; one to one = 3). ______ 25. Number of Computer Sessions (Unspecified = 0; Specified = 1). ______ 26. Duration of Computer Sessions (Unspecified = 0; Specified = 1). ______ 27. Task Difficulty (Unspecified = 0; Difficult = 1; Moderately difficult = 2; Not difficult = 3; Mixed levels of difficulty = 4). ______ 28. Type of Learning Task (Unspecified = 0; Basic skills/factual learning = 1; Problem solving = 2; Inquiry/investigation = 3; Project-based = 4; Mixed types = 5). ______ 29. Time of establishment of laptop program. (Unspecified = 0; Initiated at time of research = 1; Established at least 1 year before research = 2; Mixed = 3; Computers delivered late = 4). ______ Student Characteristics 30. Pattern of Student mobile computer Use (Unspecified = 0; Individual = 1; Cooperative = 2). ______ 31. Ownership of computer. (Unspecified =0; School or district = 1; Donated by business = 2; Federal funding = 3; Family leased = 4; Family purchased = 5). ______ 32. Percentage of Students Using mobile computers (Unspecified = 0; > 10% = 1; 10?25% = 2; 26?50% = 3; 51?75% = 4; 76?90% = 5; > 90% = 6). ______ 33. Control ? Percentage of students using stationary computers. (Unspecified = 0; >10% = 1; 10?25% = 2; 26?50% = 3; 51?75% = 4; 76?90% = 5; > 90% = 6). ______ 34. Students? Experience with Technology (Unspecified = 0; None = 1; Minimal experience = 2; Average = 3; Experienced = 4; Very experienced = 5). ______ 35. Special Education (Unspecified = 0; Incidental role of study = 1; Major role of study = 2; Exclusively special education students = 3). ______ 36. Academic Outcomes. [Unspecified = 0; Academic test scores = 1; Higher Order Thinking Skills (HOTS) specified = 2]. ______ 37. Academic Outcomes by subject matter.( Unspecified = 0; reading = 1; math = 2; language arts = 3; writing = 4; science = 5, oral communication = 6, SAT Math = 7; SAT Verbal = 8; The world = 9). ______ 156 38. Applied Outcomes (Not measured = 0; Knowledge of computers = 1, Internet = 2, e-mail = 3, peripherals = 4; CD-ROMS = 5; Hypertext = 6; Multi-media = 7; Word processing = 8.) ______ 39. Attitude outcomes for students when using mobile computers. (Unspecified = 0; toward course content/subject = 1; toward self = 2; toward instructor = 3; toward mobile computers = 4; academic self concept = 5; academic motivation = 6; satisfaction = 7; towards classmates = 8; classroom learning environment = 9). _____ 40. Behavioral Outcomes (Unspecified = 0; Student time-on-task = 1; student perseverance = 2; Tasks attempted = 3; Tasks completed = 4; Success rate = 5; Positive peer interaction = 6; Interactivity with computers = 7; Autonomy = 8; Increased school attendance). _____ Instructional/Teaching Characteristics 41. Teacher Training in Technology (Unspecified = 0; Specified and described = 1). ______ 42. Teachers? Experience with Technology (Unspecified = 0; None = 1; Minimal experience = 2; Average = 3; Experienced = 4; Very experienced = 5). ______ 43. Joint Productive Activity/Collaboration (e.g., Designs instructional activities requiring student collaboration to accomplish a joint product; monitors and supports students collaboration in positive ways. No evidence = 0; Some evidence = 1; Extensive evidence = 2). ______ 44. Language and Literacy Development (e.g., Connects student language with literacy and content-area knowledge through speaking, listening, reading, and writing activities); encourages students to use content vocabulary to express their understanding. No evidence = 0; Some evidence = 1; Extensive evidence = 2). ______ 45. Contextualization/Making Meaning (e.g., Begins activities with what students already know from home, community, and school; encourages students to use content vocabulary to express their understanding. No evidence = 0; Some evidence = 1; Extensive evidence = 2). ______ 46. Challenging Activities (e.g., Designs instructional tasks that advance students? understanding to more complex levels. Assures that students?for each instructional topic?see the whole picture as a basis for understanding the parts. No evidence = 0; Some evidence = 1; Extensive evidence = 2). ______ 47. Instructional Conversation (e,g, Arranges the classroom to accommodate conversational between the teacher and a small group of students on a regular 157 and frequent basis. Guides conversation to include students? views, judgments, and rationales using text evidence and other substantive support). (No evidence = 0; Some evidence = 1; Extensive evidence = 2). ______ 48. Setting (Unspecified = 0; Classroom = 1; Networked lab within class = 2; Computer lab in school = 3; Other = 4). ______ 49. Mode of Instruction (Unspecified = 0; Whole-group instruction = 1; Paired =2; Small-group instruction [3?5 members] = 3; Individualized = 4; Mixed = 5; Other = 6). ______ 50. Role of Teacher (Unspecified = 0; Deliverer of knowledge = 1; Facilitator of groups/student learning = 2; Modeling processes [e.g., problem solving] = 3; Mixed = 4; Other = 5). ______ 51. Learning Responsibility (Unspecified = 0; Student controlled = 1; Teacher directed = 2; System directed = 3; Mixed = 4; Other = 5). ______ 52. Teacher Qualifications (Unspecified = 0; Alternatively certified or provisional certificate = 1; Certified in content area = 2; Not certified in content area = 3; Other = 4). ______ Quality-of-Study Indicators 53. Publication Features (Unspecified = 0; Speeches = 1; Master?s thesis = 2; ERIC = 3; Technology journal = 4; Evaluation Report = 5; Research journal = 6; Ph.D. dissertation ? 7; Peer-reviewed journal = 8). ______ 54. Research hypothesis or question. (Unspecified = 0; Only goal stated = 1; Research questions/hypothesis clearly stated = 2; Research questions/ hypotheses described in detail = 3). ______ 55. Research design. [Unspecified = 0; Type specified (such as experimental, correlation, causal comparative, or survey descriptive) = 1; Rationale provided = 2; Triangulation = 3.]. ______ 56. Sampling. (Unspecified = 0; Target population described = 1; Accessible population described = 2; Sampling procedure (such as random, stratified random, or purposeful) is described = 3; Rationale is provided as to why proposed sampling procedure is appropriate = 4]. ______ 57. Instrumentation. [Unspecified = 0; Type (Such as survey, interview, or observation) specified = 1; Instrumentation clearly described = 2]. ______ 158 58. Method of Observation of Independent Variable (i.e., technology use). Unspecified = 0; Specified = 1 ( Such as systematic observation, informal observation, student survey or interview, teacher survey or interview, administrator survey or interview, computer logs, or multiple methods.); Clearly described = 2; Multiple methods = 3. ______ 59. Reported Reliability of Measures. [Unspecified = 00; Specified =1; Determination of reliability described = 2; On multiple measures, reliabilities were reported for some but not all measures = 3; Actual reliability statistic (e.g., 70 or 83) is reported = 4; Acceptable level of reliability is reported (.90 or above for standardized tests) = 5]. ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ 60. Reported Validity (Unspecified = 0; Specified = 1; Is described = 2.). ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ ____________________________________________________________________ 61. Central tendency and distribution. (Not reported = 0; Means and standard deviation reported = 1; Charts of means and standard deviations are reported = 2). ______ 62. Manner in Which Outcome Scores Are Reported. {Unspecified = 0; Only the number, mean, and standard deviation are reported = 1; Charts containing relevant statistics such as F scores or Effect sizes = 2; Standard journal report format [such as t(17) = 5.14, p<.05] = 3]. ______ 63. Duration of Study (Unspecified = 00; One instrument or set of instruments used at one time = 1; Specified months or years = 2; Longitudinal for 2 years = 3; Longitudinal for 3 years = 4). ______ 64. Feedback and Assessment Practices. Feedback obtained by researcher by means such as of questionnaires or focus groups. (Unspecified = 0; No feedback = 1; Minimal feedback = 2; Moderate feedback = 3; Elaborate feedback = 4). ______ 65. Academic test standardization. (Unspecified = 0; Not standardized = 1; Standardized = 2). ______ 159 66. Number of Comparisons within study (Count comparisons per year in longitudinal studies). Number of reported correlations, t-tests, F-tests, and effect sizes. __________________________________________________________________ __________________________________________________________________ __________________________________________________________________ 67. Effect Size Coefficient. (Actual coefficient is not specified = 0; Coefficient is specified = 1. ______ 68. Statistical Power. [Is the sample size large enough to reject the null hypothesis at a given level of probability, or are the estimate coefficients within reasonably small margins of error? (a sample > 60 for groups such as classes, schools, or districts; a sample >100 for individuals)]. ______ Univariate: Lack of power = 0; Probable threat (< 60 for groups or < 100 for individuals as the unit of analysis) = 1; Threat to statistical power adequately minimized ( > 60 for groups; >100 for individuals) = 2. ______ 69. Statistical Power - Multivariate: Lack of power = 0; Probable threat (<30 for each group or < 100 for individuals as the unit of analysis) = 1; Threat to statistical power adequately minimized (> 30 for each group or >100 for individuals) = 2. ______ 70. Generalizability. (Not discussed = 0; Discussed = 1; Reasonable conclusion = 2). ___ Sources of Invalidity The table, Factors Jeopardizing the Internal Validity of Experimental Designs is useful in answering the following questions (Ary, Jacobs, and Razavieh, 2002, p. 323). 71. Actual type of experimental design as determined by evidence found in the study?s results section. (1 = pre-experimental, 2 = quasi-experimental, 3 = true experimental). ______