AN EXPLORATION OF THE IMPACT OF SPEECH-RECOGNITION TECHNOLOGIES ON GROUP EFFICIENCY AND EFFECTIVENESS DURING AN ELECTRONIC IDEA GENERATION SCENARIO Except where reference is made to the work of others, the work described in this dissertation is my own or was done in collaboration with my advisory committee. This dissertation does not include proprietary or classified information. ___________________________________ Bradley Justin Prince Certificate of Approval: _______________________ _______________________ Terry A. Byrd Casey Cegielski, Chair Professor Associate Professor Management Management _______________________ _______________________ Dianne Hall Stephen L. McFarland Assistant Professor Acting Dean Management Graduate School AN EXPLORATION OF THE IMPACT OF SPEECH RECOGNITION TECHNOLOGIES ON GROUP EFFICIENCY AND EFFECTIVENESS DURING AN ELECTRONIC IDEA GENERATION SCENARIO Bradley Justin Prince A Dissertation Submitted to the Graduate Faculty of Auburn University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Auburn, Alabama May 11, 2006 iii AN EXPLORATION OF THE IMPACT OF SPEECH RECOGNITION TECHNOLOGIES ON GROUP EFFICIENCY AND EFFECTIVENESS DURING AN ELECTRONIC IDEA GENERATION SCENARIO Bradley Justin Prince Permission is granted to Auburn University to make copies of this thesis at its discretion, upon request of individuals or institutions and at their expense. The author reserves all publication rights. _____________________________ Signature of Author _____________________________ Date iv DISSERTATION ABSTRACT AN EXPLORATION OF THE IMPACT OF SPEECH RECOGNITION TECHNOLOGIES ON GROUP EFFICIENCY AND EFFECTIVENESS DURING AN ELECTRONIC IDEA GENERATION SCENARIO Bradley Justin Prince Doctor of Philosophy, May 11, 2006 (M.B.A. Auburn University, 2002) (B.S. Jacksonville State University, 2000) 91 Typed Pages Directed by Casey Cegielski Dynamic, evolving, complex, complicated, fast. All of these words describe today?s business environment. In this atmosphere, making decisions is critical to the success of an organization. To give themselves the best chance for success, many organizations look to groups of people rather than individuals to make decisions. However, especially in the idea generation phase of the decision process, these groups introduce many negative variables such as social loafing, evaluation apprehension, and production blocking. To that end, this research examined the use of Speech-recognition technology in an electronic idea generation scenario as a means of overcoming the negative impacts v the group imposed on the idea generation process. Four treatments were considered: Speech-recognition/Group Brainstorming, Speech-recognition/Nominal Technique, Keyboard/Group Brainstorming, and Keyboard/Nominal Technique. Group performance was compared by measures of efficiency and effectiveness. Results indicate that the speech-recognition technology does not yet offer the benefits expected. Keyboard treatments consistently outperformed Speech-Recognition treatments, while the Nominal Technique treatments consistently outperformed the Group Brainstorming treatments. Limitations and future directions are presented. vi ACKNOWLEDGEMENTS First, I want to thank my wife, Nikole Prince, for her love, support, patience, and understanding as I completed this dissertation. She has been my consolation through the bad and celebration through the good of this dissertation. For that, I am forever grateful. I also want to recognize our children, Hannah and Harrison. Even before we met, the primary motivation for pursuing this degree was to have a flexible career so as to be with my wife and children. Therefore, this dissertation is dedicated to my beautiful family. Also, I want to recognize my parents. I would not be the man I am today, nor would I have accomplished all that I have without their endless love and guidance. Their sacrifices provided opportunities that I did not deserve, but deeply appreciate. My brother, Chad, has been a rock I could always depend on to understand me and help me to understand the world. My extended family of Grandparents, Aunts, Uncles, Cousins, and In-Laws has also been a source of strength throughout my life. I am humbled and thankful to have been blessed with such a wonderful family. Last, I am forever indebted to my advisors. My chair, Dr. Casey Cegielski, has been an excellent mentor. No doubt, he has put many hours into my research that I am unaware of, and I am very appreciative of his time. The comments and direction of Drs. Dianne Hall and Terry Byrd have been a great benefit to me. To research under the direction of such experts was truly a great experience. Thank you for the education. vii Style manual or journal used Publication Manual of the American Psychological Association, 5 th Edn. Computer software used Microsoft Word, Microsoft Excel, Dragon Naturally Speaking Version 6 Preferred, SPSS v.13 viii TABLE OF CONTENTS LIST OF TABLES........................................................................................................... xi LIST OF FIGURES ........................................................................................................ xii CHAPTER I. INTRODUCTION AND BACKGROUND.............................................. 1 Introduction........................................................................................................... 1 Group Decision Support System Environment..................................................... 1 GDSS Research .................................................................................................... 3 GDSS and Electronic Idea Generation ................................................................. 4 Electronic Idea Generation and Speech Recognition Software............................ 6 Problem Statement................................................................................................ 7 Variables............................................................................................................... 8 Research Methodology ......................................................................................... 8 Overview of Dissertation Chapters....................................................................... 9 CHAPTER II. REVIEW OF THE LITERATURE ....................................................... 11 Overview............................................................................................................. 11 Idea Generation?GDSS .................................................................................... 11 Technique Variables ........................................................................................... 12 Individual Variables................................................................................ 14 Group Variables...................................................................................... 14 System Variables .................................................................................... 16 ix Speech Recognition ............................................................................................ 18 Speech Recognition Technology ............................................................ 19 Speech Recognition Applications........................................................... 22 Speech Recognition Research................................................................. 27 Statement of Hypotheses .................................................................................... 28 CHAPTER III. RESEARCH METHODOLOGY ......................................................... 31 Over view............................................................................................................ 31 Design ................................................................................................................ 31 The Subjects............................................................................................ 31 The Sessions ........................................................................................... 32 Statistical Methods.............................................................................................. 33 CHAPTER IV. DATA ANALYSIS.............................................................................. 35 Overview............................................................................................................. 35 Measures............................................................................................................. 35 Subjects............................................................................................................... 37 Treatments .......................................................................................................... 38 Treatment 1: Speech-Recognition?Group Brainstorming.................... 38 Treatment 2: Speech-Recognition?Nominal Brainstorming ................ 41 Treatment 3: Keyboard?Group Brainstorming..................................... 43 Treatment 4: Keyboard?Nominal Brainstorming................................. 45 Analysis of Hypotheses ...................................................................................... 47 Hypothesis 1a.......................................................................................... 47 Hypothesis 1b ......................................................................................... 47 x Hypothesis 1c.......................................................................................... 48 Hypothesis 1d ......................................................................................... 48 Hypothesis 2a.......................................................................................... 49 Hypothesis 2b ......................................................................................... 49 Hypothesis 2c.......................................................................................... 50 Hypothesis 2d ......................................................................................... 50 CHAPTER V. DISCUSSION........................................................................................ 51 Overview............................................................................................................. 51 Efficiency and Effectiveness .............................................................................. 51 Idea Generation................................................................................................... 54 Input Methodologies........................................................................................... 56 Implications For Practitioners............................................................................. 57 CHAPTER VI. CONCLUSIONS.................................................................................. 59 Overview............................................................................................................. 59 Summary of Dissertation .................................................................................... 59 Limitations.......................................................................................................... 63 REFERENCES ............................................................................................................... 65 APPENDIX A. IRB APPROVAL AND INFORMED CONSENT.............................. 75 APPENDIX B. DEMOGRAPHIC SURVEY................................................................ 79 xi LIST OF TABLES Table 1. Advantages and Disadvantages of GDSS Use ................................................... 2 Table 2. Medical Research Within the Speech Recognition Domain ............................ 27 Table 3. Comparisons and Hypotheses........................................................................... 29 Table 4. Treatments ........................................................................................................ 38 Table 5. Treatment 1 Results.......................................................................................... 40 Table 6. Treatment 2 Results.......................................................................................... 42 Table 7. Treatment 3 Results.......................................................................................... 44 Table 8. Treatment 4 Results.......................................................................................... 46 xii LIST OF FIGURES Figure 1. Social GDSS variables ...................................................................................... 3 Figure 2. Technical GDSS variables ................................................................................ 4 Figure 3. Fit-appropriation model for GSS performance ................................................. 5 Figure 4. The study variables............................................................................................ 9 Figure 5. Speech-recognition types: Complexity, training time and accuracy............... 22 Figure 6. Types and Use of speech-recognition software............................................... 24 Figure 7. Progression of idea generation research.......................................................... 56 1 CHAPTER I INTRODUCTION AND BACKGROUND Introduction This dissertation will explore the affect of speech recognition as a data input method on group efficiency and group effectiveness within a simulated business-meeting environment during the idea generation phase of a Group Decision Support System (GDSS) session. Group Decision Support System Environment Today?s dynamically changing business environment has left no room for bad decisions; they are too costly in terms of time and resources. This environment has forced organizations to seek an effective and efficient manner of reaching understanding and resolving conflict in the decision making process (Beeri & Spiegler, 1996; Karacapilidis & Papadias, 1998). Often, many organizations resort to groups for important decisions (Hilmer & Dennis, 2001; Kim, Choi, & Ahn, 1998), specifically, groups of experts (Beeri & Spiegler, 1996; DeSanctis & Gallupe, 1989; Gallupe, DeSanctis, & Dickson, 1988). In theory, groups would have a larger knowledge base than an individual, and be able to come to a more efficient decision (a satisficing, time proven decision) than an 2 individual in a more effective manner (in a shorter amount of time). Logically, groups utilizing tools provided by a GDSS would be even more efficient and effective. However, research has shown neither groups nor the systems to be a complete panacea. Just as groups introduce potential socio-political process losses, Aiken, Vanjami, and Krosp (1995) list the advantages and disadvantages of GDSS, as shown in Table 1, showing that there are losses introduced that could reduce or minimize any process gains. Table 1 Advantages and Disadvantages of GDSS Use Advantages Disadvantages Anonymity Slow communication Parallel communications Resistance to change Automated record keeping Lack of media richness More structure Possible increase of conflict Greater member satisfaction Possible loss of some key participants Greater productivity Misuse of the technology More efficient decision making Costs GDSS systems are designed to overcome some of the problems that hinder groups in traditional face-to-face group meetings (Dennis, George, Jessup, Nunamaker, & Vogel, 1988) and to create an interactive computer-based system that facilitates the solution of ill-structured problems (Kreamer & King, 1988). Specifically, GDSSs alter the way teams interact, endeavoring to increase process gains and reduce process losses (Dennis & Garfield, 2003; DeSanctis & Gallupe, 1987; Nunamaker, Dennis, Valacich, Vogel, & George, 1991). However, research has shown that it may actually impose more losses than gains (Gavish, Gerdes, & Sridhar, 1995). This is in part due to the fact that a computer-mediated group decision process will face the same difficult environment as the individual, though sometimes with more complexity. GDSS Research As with any developing technology, there are numerous research articles proposing theory, presenting hypothesis, and reporting results. Research in the area of GDSS is no exception. As with all technical research, studies can be divided into those that examine the social aspects and those that look at the technical aspects (Bostrom & Heinen, 1977a 1977,b). Here, social variables include the group and the individual, while the technical aspects cover the task and the system. Figures 1 and 2 provide an overview of the variables that can be found in the GDSS literature. Efficiency and Effectiveness Potency Training Emotions Confidence Attitude Objective Structure and Composition Size Leadership Goals Perceived Social Anonymity Cognitive Style Meeting Style Preference Perceived Equal Influence Participation Satisfaction Roles expected vs. roles filled Frustration and Trust Input Method Group Variables Individual Variables Transactional Transformational Judging vs. Perceiving Thinking vs. Feeling Sensing vs. Intuition Extrovert vs. Introvert Face-to-Face vs. Screen-to-Face Computing skills Social GDSS Variables Figure 1. Social GDSS variables 3 Idea Generation Problem Solving Collaborative Learning Design Mode of Communication Problem Modeling Automation Technical Anonymity Databases Temporal Distance Spatial Distance Task Variables System Variables Asynchronous Synchronous Different Time Same Time Different Place Same Place Technical GDSS Variables Nominal Group Technique Brainstorming Mixed-Motive Figure 2. Technical GDSS variables It would be unrealistic for any one research study to be or attempt to be comprehensive. Even if the variables could be completely controlled in a lab, the application of such a study to the field would be greatly challenged. Therefore, a comprehensive study of GDSS technologies is not of interest in this dissertation. This research is designed to investigate the combination of two different areas: electronic idea generation (as a specific task type) and speech recognition software (as an optional input method), so this literature review will focus on these two topics. GDSS and Electronic Idea Generation Most decision processes, group or individual, occur in the following manner: evaluation of current standing, idea generation for future direction, analysis of options, 4 and finally, the decision (Karacapilidis & Pappis, 1997). One of the most important steps of this process is that of idea generation (Jackson & Poole, 2003; Karacapilidis & Papadias, 1998) because if the best course of action is never considered, then the decision process cannot be as efficient as possible. Figure 3 shows the Fit-Appropriation Model constructed for Group Support System (GSS) performance by Dennis, Wixom, and Vandenberg (2001) that provides the theoretical grounding and basis for this study. Using this model, this research investigated whether speech recognition as a data entry method is a better fit for idea generation than the keyboard. Appropriation Support Guidance Facilitation Restrictiveness Appropriation Training GSS Capabilities Communication Support Information Processing Support Task Generation Choice Combination Appropriation Faithful Unfaithful Performance Effectiveness Efficiency Satisfaction Habitual Routines Fit Variables under consideration in this study Figure 3. Fit-Appropriation Model for GSS performance (Dennis et al., 2001) 5 6 Electronic Idea Generation and Speech Recognition Software The level of computing skill required on the part of the group members is a potentially negative variable that has not been studied, particularly with regard to their skills with different input methodologies. GDSS allows (or forces) members to use electronic communication in addition to, or instead of, verbal communication (Ackermann & Eden, 2001; Dennis & Garfield, 2003). For individuals who are less comfortable using computers, this may present a problem. Recent studies have reported that user-driven GDSS (as opposed to facilitator-driven) result in lower individual satisfaction due to technical unfamiliarity (Chun & Park, 1998), while older studies were conducted on the basis that most executives did not like to type (Gray et al., 1981). Aside from typing skills and satisfaction, many individuals simply operate better when they can dictate rather than type because their keyboarding skills interfere with their thought process and/or they are not highly skilled typists (Macarthur & Cavalier, 2004). If similar results are found in an idea generation environment, speech recognition software may provide to be very beneficial to groups engaged in this type of task due to the availability of an easier, more natural means of data entry. Additionally, the use of speech recognition software in the electronic brainstorming environment could indirectly impact two of the three major areas of process losses: production blocking and evaluation apprehension. Production blocking occurs when an individual has an idea but is not able to submit it. This may be because they have to wait for someone else to finish speaking, or because they have to type their idea. In the meantime, they may forget their idea, or miss someone else?s idea that they could have built on. Evaluation apprehension is a socio-political factor that is overcome 7 when group members believe their comments to be anonymous. However, anonymity may open the door to the free riding (a major process loss, also called social loafing) that often occurs when people are not held responsible for their comments, so they can choose not to participate. Problem Statement Idea generation is a very important GDSS task because it serves important functions across many group activities, including problem solving, decision making, negotiation, planning, and innovation (Jackson & Poole, 2003). As stated earlier, GDSSs are specifically aimed at overcoming some of the problems that hinder group idea generation in face-to-face scenarios (Dennis et al., 1988). In theory, groups using GDSS technology to lessen the negative impacts of hindrances should produce more ideas than groups not using the technology. However, Jackson and Poole (2003) and Pinsonneault, Barki, Gallupe, and Hoppen (1999) found that GDSS-supported idea-generation actually produced fewer ideas than non-supported sessions. Furthermore, non-GDSS idea generation has been shown to be more efficient than GDSS-supported brainstorming (Sosik, 1997), indicating that the supposed productivity gains through the use of electronic brainstorming may be an illusion (Pinsonneault et al., 1999). The problem, therefore, is as follows. If GDSS technologies are designed to remove the process losses imposed by groups, then why have teams using non-GDSS supported techniques consistently outperformed or equaled the performance of those making use of electronic brainstorming technologies? Although several studies have 8 investigated this phenomenon, so far none have reported conclusive results for the variables they have studied. This study, therefore, examined the incorporation of speech recognition technology into electronic idea generation sessions in order to investigate whether or not the computer, specifically the keyboard and the required skills that go along with it, impose process losses into the electronic idea generation scenario. Variables As stated above, this research incorporated speech recognition technology into different treatments of electronic idea generation sessions in order to compare outcomes as measured by group efficiency and group effectiveness. The experimental variables consisted of the two idea generation techniques and two input methodologies (speech- recognition and keyboard). The specific idea generation techniques were group brainstorming (generating ideas as a unified group) and the nominal group technique (combined ideas generated as individuals). Figure 4 shows a schematic diagram of the study variables. Research Methodology This research was conducted in the laboratory in order to control as many variables as possible, allowing a closer examination of the desired variables. Student subjects were studied in a group setting as they generated ideas about how to deal with a current issue on campus, either rising tuition costs or campus parking. Each group used speech-based input methods that required them to individually complete a training session on the computer prior to taking part in the group project, enabling the computer to be trained to recognize that subject?s voice. Group Brainstorming Nominal Idea Generation Technique Speech Recognition Input Methodology Comparisons among various combinations such as: Speech Recognition in a group brainstorming idea generation session Versus Speech Recognition nominal idea generation session Group Effectiveness Group Efficiency Measured by: Keyboard Figure 4. The study variables. Prior to the final study, equipment testing was performed to ensure that the lab provided an acceptable level of speech recognition accuracy and isolation for individual users and thus a realistic GDSS environment. Overview of Dissertation Chapters Chapter I of this dissertation has presented a brief introduction to the aspects of GDSS technologies that are pertinent to this study, specifically the idea generation task and speech recognition input methodology. The relevant variables found in the GDSS 9 10 literature were presented, hypotheses were introduced, and the methodology used for the study was discussed. Chapter II will present a detailed review of the GDSS literature, including a discussion of speech recognition technologies and electronic idea generation. Chapter III will develop a complete research methodology, including the research procedures, subjects (and respective groups), and analysis procedures used for the study, and Chapter IV will present the analysis of the collected data. Chapter V will discuss the results and the implications for researchers and practitioners. Finally, Chapter VI will summarize the study and its findings and review its limitations, as well as make suggestions for future research. 11 CHAPTER II REVIEW OF THE LITERATURE Overview Synergy, which is the term used to describe the concept that the sum of the parts is greater than the whole, is the intent of any group project (Pinsonneault et al., 1999). In the context of idea generation, it is the expectation that a group will be able to produce a greater quantity and quality of ideas than an individual working alone. This chapter presents a review of electronic idea generation, speech recognition systems, and the variables that produce process gains (movement toward synergy) and/or process losses (movement away from synergy) in a GDSS environment. Idea Generation?GDSS Idea generation is a very important GDSS task because it serves important functions across many group activities, including problem solving, decision making, negotiation, planning, and innovation (Jackson & Poole, 2003). A more specific definition of GDSS given by Dennis et al. (1988) states that GDSSs are specifically aimed at overcoming some of the problems that hinder group idea generation in face-to- face scenarios; specifically, inhibition, evaluation apprehension, and dominance of a few group members. 12 In theory, groups using GDSS technology that lessens the negative impacts of hindrances (process losses) should produce more ideas than groups not using the technology. However, Jackson and Poole (2003) and Pinsonneault et al. (1999) have reported that GDSS-supported idea-generation actually produced fewer ideas than equivalent non-supported groups. Furthermore, non-GDSS idea generation has been shown to be more efficient than GDSS-supported brainstorming (Sosik, 1997). An obvious deficit in task-technology fit exists, as exemplified by the technology creating process losses. There are many variables that can either help or hinder the group?s objectives. The idea generation technique of choice (nominal group or group brainstorming) determines many of the system variables whose impacts are based on individual preferences. Individuals impose social dynamics such as beliefs about anonymity, preference of meeting style, technology adaptability, and cognitive style that have been shown to impact a group?s overall performance and satisfaction. Similarly, the group itself will exhibit variables impacting idea generation, such as the group?s size and the amount of training its members have with the technology. Overall, there are many variables that must be considered when attempting to achieve synergy in an idea generation scenario. In this section, these variables are discussed in more detail. Technique Variables A possible reason for the discrepancy in outcomes of electronic idea generation research is the difference between idea-generation techniques. These techniques are brainstorming (Osborne, 1957) and the nominal group technique (Delbecq, Van de Ven, 13 & Gustafson, 1975). During a non-GDSS supported brainstorming session, groups are in the same location at the same time. Members state ideas, yet there is no discussion of the idea. A record keeper only takes the idea down and then moves on to the next idea (Osborne). However, there are many socio-political factors that must be considered in this environment. Time pressure, conflicts, hidden agendas, and mistakes may lead a group away from its goal (Jackson & Poole, 2003), while some members may hesitate to contribute their ideas for fear of criticism, or due to the political culture of the organization (Dennis et al., 1988; Valacich & Dennis1994). Additionally, these sessions are limited to asynchronous communications; to propose an idea, one must have the floor. While waiting for permission to speak, a member may forget their idea, or be distracted by another idea that occurs to them. Face-to-face brainstorming has been shown to consistently introduce more process losses than gains (Diehl & Stroebe, 1987; Hill, 1982). The nominal group technique overcomes some of these limitations, yet poses others. Group members are in different places at the same time and they can easily keep up with their own ideas, while never being swayed by others (Delbecq et al., 1975). With anonymity often promised, individuals may feel more secure making their suggestions. Also, synchronous communications (the ability for ideas to be generated at the same time) allows more ideas to be generated as each individual develops his or her own thought process. In an electronic environment, both of these techniques are available. Both require some level of computing skill; the nominal technique requires it on the part of the individual, and the group requires at least one team member with skills. However, the 14 electronic environment obviously does not overcome the problems faced in the non- electronic environment, as shown by past research. As will be discussed later, an electronic environment utilizing speech recognition technology can possibly overcome the common process losses experienced in face-to-face and non-speech recognition based electronic brainstorming sessions. Individual Variables Research has noted that many managers are keyboard averse (Louis, 1996), yet little research was found that tested different input measures, and none of those that were found were within a GDSS environment. By far the most widely distributed form of input is the keyboard. Pen-based and speech recognition interfaces are gaining ground as viable keyboard alternatives with the increased use of PDAs and computers in non- traditional computer settings such as automobiles, stockrooms, and doctor?s offices. Aside from the input method, there are many individual variables that can be associated with the efficiency and effectiveness of an idea generation session. However, it is not the purpose of this dissertation to test each individual variable. Of particular interest for this study, however, is the input methodology used by individuals. Group Variables Group variables are much more dynamic in that the individuals create an environment so unpredictable that it is nearly impossible to trace all variables. However, research has shown that group size has an impact on group performance (measured by efficiency and effectiveness). As mentioned earlier, the purpose of a group is to allow more people to interact, creating synergy for the organization. Many have asked the question; how many people is the optimal size for a group? It is clearly not sufficient to 15 apply the old adage ?the more the merrier?, as numbers can create more losses than gains for a group. For a face-to-face scenario, existing taxonomies consider the optimal group size to be between six and nine members; most state that seven members is most appropriate. It has been suggested that large groups lose interactivity because of human limitations, requiring a facilitator to manage the meeting. However, this heuristic should not be applied to the GDSS environment because it is too restrictive (Gavish et al., 1995). The GDSS itself can manage the meeting through handling the communication and information sharing. Logic dictates that the more people that participate in a meeting the larger the knowledge base will be, which should translate into a better idea generation session. However, depending on the technology and idea generation technique used, this may or not be the case. The total number of unique comments generated by groups using the nominal technique (individual idea generation compiled as a group at the completion of the session) should increase with the group size. Groups using electronic brainstorming, however, may finally produce more unique comments than any other treatment because members will be able to share ideas, while speaking at the same time, in an anonymous environment. The impact of group size in this environment is not known. Dennis and Valacich (1999) suggest that electronic brainstorming is not likely to surpass nominal brainstorming for small groups. However, they propose that for groups of 9 or more people, electronic brainstorming may be superior to the nominal approach. 16 System Variables System variables define the environment of the GDSS. The tools available to group members, the complexity of the tools, the amount of training required, and so on will all impact the ability of members to efficiently adopt and use the tools to their advantage. A difficulty faced by designers is that the same tool that provides a benefit for one individual may create a problem for another. However, despite the complexities faced, these variables must be addressed. Many of the variables mentioned in GDSS research are listed below, along with their definitions. Though these variables were not specifically tested for this study, they are very important to lab design and therefore need to be discussed. Spatial distance (distributed or centralized). Systems can be described by whether they are designed to support centralized or distributed meetings (Gavish et al., 1995; Karacapilidis & Papadias, 1998; Karacapilidis & Pappis, 1997). Centralized meetings are held in the same place in locations that are often referred to as decision rooms (Sambamurthy & Chin, 1994). The technologies applied here are designed more to organize and manage information than to provide communication support. Distributed systems, on the other hand, support meetings held in different locations (screen-to-face rather than face-to-face) (Barkhi, 2002a). When people are in different locations, there must be tools in place to support communication in order to share information. These systems provide many benefits to organizations, including less expense for travel, meeting rooms and individual time in traveling to the meetings. 17 Spatial distance is not as much of a concern for electronic idea generation. In either scenario, group members are connected via the network whether they are in the same room or geographically dispersed. Virtually, a screen-to-face meeting equalizes spatial distance. Temporal distance (synchronous or asynchronous). Synchronous systems support meetings with a set beginning and end, where all members must be present during the meeting. Benefits of synchronous systems are that they require members? attention at a set time, everyone has access to the same information at the same time, and it can be assured that people are available to receive the information (Karacapilidis & Pappis, 1997). Asynchronous meetings, on the other hand, allow users to share information at their own convenience. Here, messages and information may be posted on a bulletin board, or distributed via email, or newsgroups, for example (Karacapilidis and Pappis). Synchronous idea generation describes the electronic brainstorming technique where all members are present in the meeting at the same time. Asynchronous idea generation, on the other hand, is a possible option for the nominal group technique. In this scenario, each member could submit his/her ideas at any time, and later the system could combine all ideas and notify everyone that the idea generation session is complete. Mode of communication. Based on the spatial and temporal distance supported by the GDSS, communication can come in four forms: same time/same place, same time/different place, different time/same place, or different time/different. Each of these has a different set of group, system, and individual 18 variables that must be considered in order for the GDSS to be efficient. Same time/same place technologies would resemble a decision room (Gavish et al., 1995; Sambamurthy & Chin, 1994) where communication would be both face-to- face and screen-to-face. Verbal comments could be expressed and recorded into the system while information was being shared and managed by the GDSS. An example of same time/different place would be a conference call or online Netmeeting. Although different time/same place meetings are another possibility, no reports were found of research using this method. Logically, when people are required to meet in the same place, it would also be at the same time. Different time/different place meetings support communication that takes place over an extended period of time. Individuals are able to enter and review information and comments at their own convenience rather than at a set time and place (Gavish, et al.; Karacapilidis & Pappis, 1997). Speech Recognition Many business managers are keyboard averse (Louis, 1996). Yet with the PC revolution, the growth of the Internet and the increasing need to instantaneously connect with people and information in remote locations, telecommunication and the ability to utilize it for a competitive advantage has become a basic necessity (Carr & Snyder, 2003). Businesses rely on the ability to communicate and share information with employees who may or may not be in the same geographical location. Specifically GDSS technologies utilize the information sharing and communication aspects of the 19 computer. Teleconferences, web meetings, and net meetings are commonplace making human-computer interaction within the business environment almost inevitable. Though speech recognition technology has the potential to allow a user to interact with a computer without training, large-scale applications of ?no-training? are currently only being studied theoretically in labs (Marchewka & Goette, 1992). There are many issues that remain to be resolved in the quest to create this level of usability such as efficiency, or the accuracy of the system; effectiveness, or the completion of a task without the waste of time and energy (Kroemer & Grandjean, 1997); adaptability, in terms of the technology?s adoption by end users; and applications, or which potential applications of speech recognition will actually be the most useful. A discussion of speech recognition as a more natural means of input must be subdivided into separate considerations and therefore this discussion will proceed in the following manner. First, the technology itself and how it works will be considered. Second, the current and potential applications of the technology will be discussed. Third, a review of the research that has been done within the realm of speech recognition will be presented. Speech Recognition Technology Speech recognition technology is a form of human-computer interface that allows the end-user to dictate text or commands to the computer, in natural language. Since the 1970s, research in this area has been conducted leading to the development of today?s current applications (Marchewka & Goette, 1992; William, 1998). On the front end, the user should be able to speak into a microphone and have the computer follow the commands given, or type the corresponding text onto the screen (Hedberg, 1997). There 20 are differences, however, in the ways users can ?speak? to the computer; for example, either discrete speech or continuous speech may be used (Cavalier & Ferretti, 1996; Marchewka & Goette; Rieger, 2003; Schneider & Owsen, 2003; William). Discrete speech recognition was the most widely available in 1990 (Marchewka & Goette, 1992), though it requires the speaker to pause between words (Cavalier & Feretti, 1996; Rieger, 2003; Schneider & Owsen, 2003; William, 1998). Some systems will recognize common phrases or trained phrases (Schneider & Owsen), but almost all the words have to be clearly separated (Rieger; William). This dramatically slows the recognition process because the speaker is unable to speak in a natural manner. Since this is not a natural form of speech, it decreases the syllable per breath ratio (Rieger). The increased workload of the larynx may be detrimental to the health of the voice, similar to the type of repetitive strain injury that leads to carpal tunnel syndrome in the wrist (Rieger). However, this separation allows the system to better recognize the speech and provide a more accurate output (Marchewka & Goette). In contrast, continuous speech allows the user to use a natural voice and speak entire sentences without pausing (Marchewka & Goette, 1992; William, 1998). This greatly increases the text entry speed and the ease of use for the end user, potentially increasing the speed by up to 160 words per minute (Schneider & Owsen, 2003). However, continuous speech requires a much higher degree of training and articulation for the system to be accurate (Rieger, 2003) and the necessary training has proved to be both difficult and time-consuming (Higgins & Raskind, 2004). On the back-end, speech waves (often called tokens) must be broken down such that they can be compared to templates in a look-up table to interpret the meaning of the 21 spoken words (Cavalier & Ferretti, 1996; Kneisel, 1990; Marchewka & Goette, 1992). This look-up process further divides the technology into two separate groups: speaker dependent and speaker independent (Cohen, 2004; Marchewka & Goette; Rebman, Aiken, & Cegielski, 2003; William, 1998). Speaker dependent systems must be trained by the end-user, providing the system with templates specific to that user?s voice, dialect, and word articulation (Schneider & Owsen, 2003). These systems often offer a larger vocabulary, are more accurate, and can be applied to various applications (Marchewka & Goette, 1992; William, 1998). The training process (referred to as enrollment) is sometimes quite rigorous, requiring the user to vocalize 3 to 10 samples of each item in the system?s vocabulary (Cavalier & Ferretti, 1996). This level of training can take hours, but is absolutely crucial to the accuracy and reliability of the system (Cavalier & Ferretti). However, few users are inclined to dedicate this amount of time to the technology (Bajaj, 2004). For this reason, many people will only train a system for about 20 to 30 minutes and set a recognition threshold to allow a more or less exact match between a token and its template (Bristow, 1986). Speaker independent systems, on the other hand, do not need to be trained (Cavalier & Ferretti, 1996). Instead, they operate on the basis of mathematical algorithms such as Hidden Markov models or Neural Networks (Voskuhl, 2004) that allow the computer to analyze any user?s speech and accurately follow commands or type text (Cohen, 2004). Though these systems do not require training by the end user, they do receive extensive training (Voskuhl). Speaker independent systems are trained by building prototypes rather than templates, where a large number of people speak the same word or phrase (Voskuhl; Zagat 2004). This creates the usefulness of independence in that they do not require training from the end-user, however, the number of models and algorithms (prototypes) available limit the possible applications. Figure 5 shows a graphical depiction of the divisions of technology based on speaker dependence, and speech type. Complexity AccuracyTraining Time Dependent/Discrete Dependent/Continuous Independent/ Discrete Independent/Continuous Figure 5. Speech recognition types: Complexity, training time and accuracy Speech Recognition Applications 22 A search of the market for available speech recognition applications reveals that most discrete speech systems are specifically designed to accept commands. Products such as Voice Blaster from Convox, Inc. and Voice Pilot from Microsoft (Gellerman, 1994) allow users to navigate through menu driven applications, while Conversa Web 2.0 23 allows users to surf the web with no enrollment process (William, 1998). This technology benefits many small technologies such as PDAs. Speech-driven commands allow users to use technological devices that are too small for traditional inputs. Aside from scaling down, this type of speech recognition is often used in call-center applications where a customer simply speaks a number to perform a given action (Cohen, 2004). AT&T reportedly plans to replace 3000 to 6000 of its collect call operators with speech recognition technology (Marchewka & Goette, 1992). Continuous speech systems are the most prevalent?both Naturally Speaking, from Dragon Systems and ViaVoice from IBM are finding a growing number of uses on financial and accounting professionals? desks (Schneider & Owsen, 2003), as well as becoming widely used by claims agents (Roth, 2003). These systems, after an enrollment period, allow both dictation and commands to be given to the computer by the user?s voice, though they are primarily used for dictation. A major drawback of these systems, however, is their ability to distinguish between dictated text and spoken commands. Correcting mistakes while dictating has proven to be a difficult task (Hedberg, 1997) and quite often, users develop their own pattern of usage where they combine the microphone, keyboard, and mouse to complete tasks, including correction (Schneider & Owsen, 2003). In conjunction with the Defense Advancement Projects Research Agency (DARPA), the Massachusetts Institute of Technology (MIT) has developed a continuous speech/speaker independent system that can recognize and answer about 66% of presented questions regarding building locations in the Cambridge area (Marchewka & Goette, 1992). Speech recognition systems have endless possibilities, including mobile technologies, small computing technologies such as cell-phones and PDAs, general text input, aiding those with disabilities, and hands busy jobs among others (Figure 6). One of the most prevalent uses of speech recognition is that of general text input. It is used by researchers conducting interviews, secretaries, and even authors (Hedberg, 1997). The benefit of using speech recognition for text input is that most people can dictate up to 160 words per minute (wpm) (Reiger, 2003) while most experienced typist can only type in the range of 50-100 wpm (Hedberg). Corrections, however, are quicker for those who have experience using traditional keyboard/mouse input devices (Schneider & Owsen, 2003). Potential Applications include: These applications only exist in labs at the present time. However, there are endless possibilities as this category would combine all of the benefits and potential applications of the other three. Potential Applications include: Call-center applications, Speaker Independent Dragon Systems Potential Applications include: Rapid text entry, document creation, meeting minutes, e-mail through PDA, Virtual Group Support Systems Potential Applications include: Inventory management, cell-phone applications, police-reports, filling out common documents Speaker Dependent Continuous SpeechDiscrete Speech Figure 6. Types and uses of speech recognition software 24 25 Research has shown that individuals who are new to the computer or who have limited keyboard/mouse skills find speech recognition quite appealing as a form of text input. However, those who are experienced, find that some combination of speech and the traditional inputs works most effectively for them (Schneider & Owsen, 2003). Some doctors are now using speech recognition systems to improve the effectiveness of clinical documentation, an area where hospitals often spend $1,200 or more per month per physician for transcription (Bajaj, 2004). Mobile technologies are exemplified by Honda including a voice activated navigation system as standard equipment on their 2005 Acura RL and as options on both the 2005 Acura MDX and the Honda Odyssey. The system is reported to have the ability to allow drivers to speak any street address represented in the U.S. navigation system database and receive turn-by-turn guidance to their destination. This system actually speaks back to the driver ? responding in a ?text-to-speech? format, specific to the IBM ViaVoice software. Mercedes-Benz has announced that some cars are being equipped with speech recognition technology that will allow hands-free control of cell-phone dialing as well as plans for an AudioNav (audio navigation) similar to the one demonstrated by Honda (Louis, 1996). Other mobile applications would be the use of a voice-activated calling on a cellular phone while driving (Cohen, 2004) or having the ability to ?speak notes? into a recorder to be transcribed later rather than trying to write them down while driving (Schneider & Owsen, 2003). Due to the shrinking size of some computing technologies, speech recognition is a much needed enhancement. Processing power and memory are no longer the limiting factors?rather, the individual?s ability to give input and receive output has become the 26 bottleneck. Speech recognition software has the potential to address this issue and the ability to give commands to a PDA via voice will greatly enhance the usefulness of these technologies, especially with the later models including cell-phone software (Cohen, 2004). Now, mobile applications can include a mobile virtual meeting or a mobile group support system. There are more applications of speech recognition for individuals in ?hand busy? jobs (Cavalier & Ferretti, 1996; Mountford & Farver, 1990; Noyes and Frankish, 1992). Consider inventory managers searching through products, or helping to unload or organize material. They do not have the capability to free their hands to give commands to or input text into a computer. However, speech recognition technologies offer them the freedom to be able to work without constantly needing to touch the computer keyboard or mouse, while still controlling the computer and giving dictation. Research in this area has shown that dramatic improvements in productivity can be derived from the use of speech recognition software. For example, G.H. Bass & Company?s distribution warehouse center reported that an initial investment of $25,000 resulted in annual savings of $60,000 and reduced or eliminated receiving errors and sales cancellations caused by delays (Marchewka & Goette, 1992; DeMars, 1988). Speech recognition has found numerous applications in the medical field. Research has been conducted on topics ranging from aiding students with Learning Disabilities (LD) to take tests or write papers (Macarthur & Cavalier, 2004) to the medical aspects of voice damage vs. carpal tunnel syndrome that is often associated with traditional input methods. Boeing reported that after one of their programmers was severely injured in an accident, with the help of speech recognition technology he was 27 able to continue working for them and become their best programmer in spite of being quadriplegic (Clements, 1990). Overall, medical research has concluded that speech recognition software is a beneficial tool, with less physical stress on an individual?s body. Table 2 summarizes some of the more prevalent medical research done within the speech recognition domain. Table 2 Medical Research Within the Speech Recognition Domain Authors Findings Fried-Oken, 1985 Two individuals with disabilities (quadriplegia and spinal cord injury) were able to utilize commands to control a computer Treviranus, Shein, Haataja, Parnes, & Milner, 1991 Cerebral Palsy patient was able to use speech technology, but it was slower and less accurate than other means of computer interaction Koester, 2004 Experience with speech recognition technology helps those with disabilities by enhancing accuracy. Kristensen, Laurensen, Pilegaard, & Jensen, 2004 No differences in musculoskeletal symptoms were found between traditional and speech based input, though speech is recommended as a supplement Raskind & Higgins, 1999 LD students using discrete speech recognition showed significantly higher scores over a contrast group Speech Recognition Research In addition to these areas of research, many have studied the potential impact speech recognition has on efficiency and effectiveness when completing a task, satisfaction with the completion of the task and the use of the technology, and perceptions of the usefulness of the technology. The results, however, have been mixed. For example, Mountford and Garver (1990) reported superior task performance in busy 28 environments when speech recognition tools were used, while Rieger (2003) reported decreased task effectiveness in the same situation. Though speech recognition promises many advantages, there are several challenges. In any form of communication, noise?a barrier to communication?is a problem (Carr & Snyder, 2003). For speech recognition systems, noise may come in the form of actual background noise, while coughing, fast talkers and sounds such as ?uh,? ?um,? and ?ah? are often difficult for the system to distinguish from the actual text or commands being given (Gellerman, 1994; Louis 1996; William, 1998). Homonyms, namely words that sound the same but have multiple meanings (e.g. two, too, to), are a limiting factor, as are spoken numbers (Louis, 1996). However, despite the challenges facing this technological innovation, businesses will continue to attempt to find practical and cost-efficient applications. Some uses include material handling, sorting, transcription, and aiding the disabled (DeMori, Palakai & Cosi, 1990; Marchewka & Goette, 1990). One potential application that has not been addressed in research is the use of speech recognition systems in a GDSS idea generation environment. Many potential GDSS/speech recognition variables have been studied individually, but no work has yet been done on their combined use in this context. Statement of Hypotheses Table 3 presents the potential comparisons that were used to create the research hypothesis that were constructed to guide this study. The hypotheses will be discussed in the order in which the variables were presented in the literature review. 29 Table 3 Comparisons and Hypotheses Variables Variable under consideration Nominal Technique Group Brainstorming Speech Keyboard Speech * * * Keyboard * * * Technique * * * Technique * * * Input methodology is of particular interest in this study. As mentioned earlier, research has stated that many managers are keyboard averse (Louis, 1996). However, little research has been conducted to compare the (now more available) different forms of input methodology. For this reason, this study compared speech-recognition and the use of a keyboard as alternative input methodologies for an electronic idea generation scenario. H1a: There will be no significant difference in session efficiency between groups using speech-recognition and the keyboard as an input methodology utilizing the nominal technique. H1b: There will be no significant difference in session effectiveness between groups using speech-recognition and the keyboard as an input methodology utilizing the nominal technique. H1c: There will be no significant difference in session efficiency between groups using speech-recognition and the keyboard as an input methodology utilizing the group brainstorming technique. 30 H1d: There will be no significant difference in session effectiveness between groups using speech-recognition and the keyboard as an input methodology utilizing the group brainstorming technique. This research is also interested in the differences (if any) between the Nominal Group Technique and Group Brainstorming. Thus far, most research has shown the superiority of the Nominal Group Technique over that of Group Brainstorming. However, this research proposes that applying speech recognition technologies as an optional input method may remove some of the process losses associated with electronic group brainstorming. Therefore, the following hypotheses are proposed. H2a: There will be no significant difference in session efficiency when comparing the Nominal Group Technique to Group Brainstorming for groups utilizing speech-recognition as the input method of choice. H2b: There will be no significant difference in session effectiveness when comparing the Nominal Group Technique to Group Brainstorming for groups utilizing speech-recognition as the input method of choice. H2c: There will be no significant difference in session efficiency when comparing the Nominal Group Technique to Group Brainstorming for groups utilizing the keyboard as the input method of choice. H2d: There will be no significant difference in session effectiveness when comparing the Nominal Group Technique to Group Brainstorming for groups utilizing the keyboard as the input method of choice. 31 CHAPTER III RESEARCH METHODOLOGY Overview This chapter presents an over of the methodology that was used to statistically examine the hypotheses presented in the previous chapter. First, the subjects are described, followed by a discussion of group make-up. The laboratory setting is explained, along with the training the subjects received on the speech-recognition technology. Design The Subjects The subjects for this study were business students at a large public university in the southeastern United States. They were assigned to groups based on their business major to ensure that the group make-up was similar to that typically found in a multinational corporation. For example, a multinational corporation faced with intense cost pressures from customers and competitors may call a meeting for the generation of ideas on how to reduce costs. Strategic and tactical decisions of this sort would require that each functional area (such as Marketing, Management, Accounting, Finance, Logistics, Management Information Systems, Human Resources, Procurement, etc.) be 32 represented in order to provide a check and balance system for any ideas generated and discussed in connection with different cost reduction options. Accordingly, subjects were grouped by major and provided with an opportunity to generate ideas to help overcome an issue that most students deal with on a regular basis?the rising cost of textbooks. Thus, although the research used students in a lab setting, its findings will be applicable to real-world businesses because the group make- up was diversified according to functional areas and the topic under consideration was of extreme interest to those attending the meeting. Anonymity was promised to the subjects, and they were assured that their responses would be private and only used for research purposes. Copies of the university?s IRB approval and the informed consent forms signed by the participants are included as Appendix A, and the survey used to collect demographic information and assign the volunteers to groups is given in Appendix B. The Sessions Sessions were held three days a week during the day and early evening. Participants did not know prior to the event which idea generation technique they would be using to generate ideas or which type of input they would be using. Those who utilized speech were given instructions on training their computer to recognize their voice. Each participant trained the system using a built-in training procedure that not only trains the computer but also provides the user with information about speech- recognition technologies. 33 Once the training was complete, participants were given instructions on completing the idea generation session. These instructions included the brainstorming rules suggested by the Six Sigma web site. These rules were as follows. 1. There are no dumb ideas. 2. Do not criticize other?s ideas. 3. Build on other?s ideas. 4. Go for quantity over quality. Each group was allowed only ten minutes to complete the session, regardless of input methodology or technique. This time frame was chosen after pilot studies revealed a decrease in idea generation for longer durations. For groups using the nominal technique, the idea generation had almost stopped; for those in the group brainstorming technique, the session turned into a chat/discussion about the ideas. Also, this allows for an easier comparison between efficiency and effectiveness outcomes and levels the playing field between techniques and group sizes. Groups that engaged in idea generation using the keyboard did not have to participate in any training. However, they were provided with the same instructions on brainstorming as the speech-recognition groups. Their sessions also lasted ten minutes. Statistical Methods Since comparison between groups was the analysis of choice for this research, independent sample t-tests were used to test for a significant difference between standardized scores to therefore accept or reject the hypotheses. The measurements utilized for the comparison of session efficiency and effectiveness are defined below. Past research has measured group idea generation outcomes based on the total number of ideas generated during the session, minus repeated comments and flames (sarcastic remarks or comments about others? ideas that are not ideas themselves). This number, divided by the length of the session in minutes, represents the Group Efficiency. This provides a standardized number of original ideas per minute that can easily be compared across studies. For the groups using speech-recognition as the input method, ?indecipherable? comments were also be subtracted from the total number. Group Efficiency: () time flamesrepeatstotal ?? (1) Using a similar approach, Group Effectiveness considers the total number of original ideas as a fraction of the total number of comments. This number reveals how well the method encourages original ideas and discourages repeats and flames. It was expected that there would be repeats and flames during all sessions. As with the formula for Group Efficiency, for the groups using speech-recognition as the input of choice, ?indecipherable? comments were subtracted from the total. Group Effectiveness: ( ) total flamesrepeatstotal ?? (2) 34 35 CHAPTER IV DATA ANALYSIS Overview This chapter provides an overall view of the results obtained and a detailed analysis of the data. In particular, specifics about the measures are discussed and the subjects are presented by treatment. The chapter will conclude by presenting an analysis of the eight hypotheses listed in Chapter II. Measures The two measures under consideration in this study are group efficiency and effectiveness. To provide these numbers, each session had to be broken down into specific comment categories. Specifically, comments were noted as being either an original idea, a repeated idea, a flame (anything sarcastic or non-productive), or an indecipherable comment due to the speech recognition technology. For example, if a participant was the first to suggest ?buy your books online? this would be considered an original idea. However, if someone then specifically mentioned ?buy your books at Amazom.com?, this would be considered a repeated idea since in theory it is the same thing. If someone commented ?I never go to the bookstores? this would be considered a 36 flame because it is neither a new idea nor productive in the discussion, but simply an explanation of how one individual felt about the local bookstores. To provide inter-rater reliability, three colleagues analyzed one group from each treatment. These scores were compared to the author?s rankings and proved to be statistically equivalent. As defined earlier, Group Efficiency is the total number of unique comments presented during an idea generation session. Group Effectiveness is defined as the fraction of total ideas that are unique. Thus, it is entirely possible for a session to be efficient thought not at all effective relative to the other sessions. For example: one particular group who used speech-recognition as the method of input and group brainstorming as the idea generation method produced an efficiency score of 1.4 (1.4 original ideas per minute) and an effectiveness score of 0.61, while a different group using speech-recognition in a nominal brainstorming session produced an efficiency score of 1.3 and an effectiveness score of only 0.27. This means that only 27% of their total comments were productive, as opposed to the 61% demonstrated by the first group. Note that this example was used only to illustrate the difference in efficiency and effectiveness, and does not imply that nominal brainstorming is less effective than group brainstorming. This issue will be discussed later, in Chapter V. Another important measurement was the accuracy of the speech recognition software, Dragon Naturally Speaking Preferred Version 6. The software advertises itself as being 97% accurate but does not specify how much training is required to reach this level of accuracy. Pre-tests revealed that the software was much less accurate than advertised, so to test this assertion, a more detailed test was conducted in which a page of 37 139 words presented in 31 short phrases was read into the computer by each participant. They were instructed not to make any corrections, but to leave them for an accuracy check. Each phrase was compared to the original document for missing, additional, or incorrect words. As was expected from the pretest, the software did not provide the 97% accuracy promised, but rather an overall accuracy of only 83.3%. Of the inaccuracies, 20% were attributed to missing words, 25% to additional words, and 55% to incorrectly recognized words. It should be noted that the participants in this study completed only one training session. This was sufficient for some participants to reach a very acceptable 95% accuracy, while others only attained a 50% accuracy rate. This lack of accuracy was definitely a factor in the analysis of the hypotheses, as discussed later. Subjects As mentioned earlier, the subjects were from a large public university in the southeastern United States and all were registered in an Introduction to Information Technology course that is required by all business majors at the university. Table 4 presents the breakdown of students into their respective treatments. There were 160 participants in all: 87 male participants and 73 female. This provided for 10 groups of 4 for each treatment. The average age was 20. The average response on a scale of 1 (strongly agree) to 5 (strongly disagree) to whether or not they had experience with speech recognition technology was 4.3. Of the 80 participants in the speech recognition treatments, only 3 strongly agreed that they had experience with the technology. 38 Therefore it was assumed that no one participant had experience that would give him/her an unfair advantage in the study. Participants were provided with information about the study by their respective professors. They volunteered to participate and signed up for a time that was convenient for them. Each time slot had four open seats. It took a total of two weeks to collect all of the data. As the subjects entered the lab experiment, they were given instructions and told what would transpire during their session. The session events will be discussed with the respective treatments. Table 4 Treatments Speech-Recognition Keyboard Group Brainstorming Treatment 1 10 Groups (19 male, 21 female) Treatment 3 10 Groups (24 male, 16 female) Nominal Technique Treatment 2 10 Groups (21 male, 19 female) Treatment 4 10 Groups (23 male, 17 female) Treatments Treatment 1: Speech-Recognition?Group Brainstorming Treatment 1 was comprised of 40 students broken into 10 groups of 4, with 19 male participants and 21 female participants. Each of their sessions lasted for approximately 45 minutes. First, they were provided with an explanation of the study and instructions for their session. They were then required to complete a training session 39 on their respective computers allowing the speech-recognition software to ?learn? their voice and dialect. Next they were asked to read a document into the computer for an accuracy check. Before the brainstorming session began, the brainstorming rules from the Six Sigma web site were provided. Also, since anonymity was promised, each participant was seated as far away from other participants as possible and provided with ear plugs to be sure they could only read other?s comments, not hear them. At the completion of the 10 minute session, the results were saved and the participants were allowed to leave. After analyzing the data, the sessions produced an average of 24 total comments, 26% of which were original ideas, 12% repeats, 46% flames, and 15% indecipherable. The results for this treatment are shown in Table 5. On average, the groups in this treatment produced 0.59 original ideas per minute (efficiency). However, they were only 27% effective, meaning that of the 24 total comments, only 27% of those were actually original ideas. This treatment produced the lowest actual score of all treatments. The statistically significance will be discussed in the Analysis of Hypotheses section of this chapter. 40 Table 5 Treatment 1 Results Group Original Repeats Flame Indecipherable Total Efficiency Effectiveness 1 4 (13%) 1 (3%) 21 (66%) 6 (19%) 32 (100%) 0.4 0.13 2 4 (17%) 4 (17%) 8 (33%) 8 (33%) 24 (100%) 0.4 0.17 3 4 (17%) 3 (13%) 12 (50%) 5 (21%) 24 (100%) 0.4 0.17 4 4 (17%) 1 (4%) 17 (74%) 1 (4%) 23 (100%) 0.4 0.17 5 5 (21%) 2 (8%) 8 (33%) 9 (38%) 24 (100%) 0.5 0.21 6 5 (22%) 2 (9%) 14 (61%) 2 (9%) 23 (100%) 0.5 0.22 7 6 (24%) 0 (0%) 17 (68%) 2 (8%) 25 (100%) 0.6 0.24 8 6 (26%) 4 (17%) 12 (52%) 1 (4%) 23 (100%) 0.6 0.26 9 7 (44%) 5 (31%) 1 (6%) 1 (6%) 16 (100%) 0.7 0.56 10 14 (61%) 4 (17%) 3 (13%) 2 (9%) 23 (100%) 1.4 0.61 Average 5.9 (26%) 2.6 (12%) 11.3 (46%) 3.7 (15%) 24 (100%) 0.59 0.27 Minimum 4 0 1 1 16 0.40 0.13 Maximum 14 5 21 9 32 1.40 0.61 41 Treatment 2: Speech-Recognition?Nominal Brainstorming Treatment 2 was comprised of 40 students broken into 10 groups of 4, with 21 male participants and 19 female participants. Their sessions also lasted approximately 45 minutes. These groups were provided with the same instructions and brainstorming rules as those in Treatment 1. They were also spaced as far apart as possible and provided with ear plugs to ensure that their comments were completely their own and that they were not influenced by hearing the others. At the conclusion of their 10 minute session, each participant saved their results and was allowed to leave. Later, their results of each group were combined into one session of four people and analyzed. This treatment generated an average of 52 comments. Of these 52, 38% were original ideas, 27% were repeats, 19% were flames, and 16% were indecipherable. As would be expected when participants are not privy to the ideas of others, the number of repeats was higher. The results for this treatment are given in Table 6. This treatment produced an average efficiency score of 1.91 original comments per minute and a 38% effectiveness rating. This treatment was the most efficient of all the groups, though it was not as effective as the fourth treatment. Again, the statistical significance of the results will be discussed later in this chapter, in the Analysis of Hypotheses section. 42 Table 6 Treatment 2 Results Group Original Repeats Flames Indecipherable Total Efficiency Effectiveness 1 13 (27%) 8 (17%) 17 (35%) 10 (21%) 48 1.30 0.27 2 16 (41%) 9 (23%) 11 (28%) 3 (8%) 39 1.60 0.41 3 18 (30%) 13 (21%) 21 (34%) 9 (15%) 61 1.80 0.30 4 18 (35%) 18 (35%) 1 (2%) 14 (27%) 51 1.80 0.35 5 18 (38%) 19 (40%) 8 (17%) 3 (6%) 48 1.80 0.38 6 17 (44%) 4 (10%) 0 (0%) 17 (44%) 39 1.70 0.46 7 19 (32%) 24 (40%) 5 (8%) 12 (20%) 60 1.90 0.32 8 22 (29%) 18 (24%) 25 (33%) 11 (14%) 76 2.20 0.29 9 22 (50%) 9 (20%) 11 (25%) 2 (5%) 44 2.20 0.50 10 28 (52%) 22 (41%) 3 (6%) 1 (2%) 54 2.80 0.52 Average 19 (38%) 14 (27%) 10 (19%) 8 (16%) 52 (100%) 1.91 0.38 Min 13 4 0 1 39 1.30 0.27 Max 28 24 25 17 76 2.80 0.52 43 Treatment 3: Keyboard?Group Brainstorming Treatment 3 was comprised of 40 students broken into 10 groups of 4, with 24 male participants and 16 female participants. Their sessions lasted only approximately 20 minutes, as this treatment did not have to complete any training on their computer or provide any accuracy data since they were not participating in the speech-recognition sessions, thus making their sessions significantly shorter than the participants in the first two treatments. This treatment was provided with the same brainstorming rules as the others, and were given 10 minutes to complete the session. At the conclusion of their session, the brainstorming session was saved on the server and the participants were allowed to leave. This treatment generated an average of 47 comments. Of these, 30% were original ideas, 20% were repeats, and 50% were flames. Since this was a keyboarding session, there were misspelled words but there were no indecipherable comments. The results for Treatment 3 are given in Table 7. Further analysis shows that this treatment produced an average efficiency score of 1.3 original comments per minute and a 30% effectiveness rating. This treatment was neither the most efficient nor effective. Additional analysis of the results will be discussed later this chapter in the Analysis of Hypotheses section in order to determine if any differences are statistically significant. 44 Table 7 Treatment 3 Results Group Original Repeats Flames Indecipherable Total Efficiency Effectiveness 1 5 (21%) 6 (25%) 12 (50%) 0 (0%) 24 0.50 0.21 2 15 (25%) 8 (13%) 38 (62%) 0 (0%) 61 1.50 0.25 3 16 (24%) 3 (5%) 47 (71%) 0 (0%) 66 1.60 0.24 4 12 (31%) 0 (0%) 27 (69%) 0 (0%) 39 1.20 0.31 5 10 (13%) 7 (9%) 63 (79%) 0 (0%) 80 1.00 0.13 6 13 (76%) 2 (12%) 2 (12%) 0 (0%) 17 1.30 0.76 7 12 (27%) 4 (9%) 28 (64%) 0 (0%) 44 1.20 0.27 8 7 (27%) 8 (31%) 11 (42%) 0 (0%) 26 0.70 0.27 9 14 (19%) 9 (12%) 51 (69%) 0 (0%) 74 1.40 0.19 10 21 (57%) 15 (41%) 1 (3%) 0 (0%) 37 2.10 0.57 Average 12.5 (30%) 6.2 (20%) 28 (50%) 0 (0%) 46.8 1.3 0.3 Min 5 0 1 0 17 0.5 0.1 Max 21 15 63 0 80 2.1 0.8 45 Treatment 4: Keyboard?Nominal Brainstorming Treatment 4 was comprised of 40 students broken into 10 groups of 4, with 23 male participants and 17 female participants. Their sessions were again significantly shorter than the other participants, lasting only approximately 20 minutes, as this treatment also did not have to complete any training on their computer or provide any accuracy data since they were not participating in the speech-recognition sessions. After being provided with the same brainstorming rules as the other treatments, participants were allowed 10 minutes to generate ideas. At the conclusion of the 10 minute session, the results were saved for later analysis. The individual sessions were later combined into groups for analysis. This treatment generated an average of 31 comments. Of these, 56% were original ideas, 39% were repeats, and 6% were flames. Again, there were some misspelled words but there were no indecipherable comments. This treatment generated an average efficiency score of 1.7 original comments per minute and a 56% effectiveness rating. Though this treatment was not the most efficient, it was by far the most effective. No other treatment produced effectiveness scores above 40%. 46 Table 8 Treatment 4 Results Group Original Repeats Flames Indecipherable Total Efficiency Effectiveness 1 20 (63%) 11 (34%) 1 (3%) 0 (0%) 32 2.00 0.63 2 10 (30%) 9 (27%) 14 (42%) 0 (0%) 33 1.00 0.30 3 16 (55%) 13 (45%) 0 (0%) 0 (0%) 29 1.60 0.55 4 16 (70%) 6 (26%) 1 (4%) 0 (0%) 23 1.60 0.70 5 18 (30%) 11 (37%) 1 (3%) 0 (0%) 30 1.80 0.60 6 20 (56%) 16 (44%) 0 (0%) 0 (0%) 36 2.00 0.56 7 23 (50%) 21 (46%) 2 (4%) 0 (0%) 46 2.30 0.50 8 13 (50%) 13 (50%) 0 (0%) 0 (0%) 26 1.30 0.50 9 16 (57%) 12 (43%) 0 (0%) 0 (0%) 28 1.60 0.57 10 17 (65%) 9 (35%) 0 (0%) 0 (0%) 26 1.70 0.65 Average 16.9 (56%) 12.1 (39%) 1.9 (6%) 0 (0%) 30.9 1.69 0.56 Min 10 6 0 0 23 1.00 0.30 Max 23 21 14 0 46 2.30 0.70 The next section will discuss the hypothesis and whether or not the stated differences were statistical or not. 47 Analysis of Hypotheses This section will present an analysis of the hypotheses presented at the end of Chapter II. The first set of hypotheses tested whether or not there was a statistical difference between using speech-recognition software and a keyboard as the input methodology for a given idea generation type, as measured by the resulting group efficiency and effectiveness. The second set of hypotheses tested the difference between idea generation types for a given input method, again as measured by the resulting group efficiency and effectiveness. All hypotheses were tested at the alpha = 0.05 level. Individual results are presented, but they will be discussed in detail in Chapter V. Hypothesis 1a: There will be no significant difference in session efficiency between groups using speech-recognition and the keyboard as an input methodology utilizing the nominal technique. The average efficiency score of the groups generating ideas using the Nominal Technique with speech recognition was 1.91 original ideas per minute, while the efficiency of groups using the keyboard was only 1.69. However, the p-value of the independent sample T-test was 0.223. Therefore, this hypothesis is retained as there is no statistical difference between the average efficiency of these two treatments. Hypothesis 1b: There will be no significant difference in session effectiveness between groups using speech-recognition and the keyboard as an input methodology utilizing the nominal technique. The average effectiveness score of the groups generating ideas using the Nominal Technique with speech recognition was 0.379, while the effectiveness of groups using the keyboard was 0.556. Additionally, the p-value of the independent sample T-test was 48 0.001. Therefore, this hypothesis must be rejected, as there is a statistical difference between the average effectiveness of these two treatments. When using the Nominal Technique, groups using a keyboard to input their ideas were more effective than those using speech-recognition. Hypothesis 1c: There will be no significant difference in session efficiency between groups using speech-recognition and the keyboard as an input methodology utilizing the group brainstorming technique. The average efficiency score of the groups generating ideas using Group Brainstorming with speech recognition was 0.59, while the efficiency of groups using the keyboard was 1.25. Additionally, the p-value of the independent sample T-test was 0.001. Therefore, this hypothesis must be rejected as there is a statistical difference between the average efficiency of these two treatments. Using Group Brainstorming to generate ideas, the keyboard as the method of data entry was more efficient. Hypothesis 1d: There will be no significant difference in session effectiveness between groups using speech-recognition and the keyboard as an input methodology utilizing the group brainstorming technique. The average effectiveness score of the groups generating ideas using Group Brainstorming with speech recognition was 0.273, while the effectiveness of groups using the keyboard was 0.3193. The p-value of the independent sample T-test was 0.579. Therefore, this hypothesis is retained, as there is no statistical difference between the average effectiveness of these two treatments. Even though the previous hypothesis found the keyboard to be a more efficient means of data entry, it is not statistically more effective. 49 Hypothesis 2a: There will be no significant difference in session efficiency when comparing the Nominal Group Technique to Group Brainstorming for groups utilizing speech-recognition as the input method of choice. The average efficiency score of the groups using speech-recognition as the method of data entry while using the Nominal Technique to generate ideas was 0.59, while the efficiency of groups generating ideas through Group Brainstorming was 1.91. The p-value of the independent sample T-test was 0.000. Therefore, this hypothesis must be rejected, as there is a statistical difference between the average efficiency of these two treatments. Groups using the Nominal Technique as the method of idea generation were more efficient (1.91 original ideas vs. 0.59) than those using the Group Brainstorming technique when speech-recognition was used as the method of data entry. Hypothesis 2b: There will be no significant difference in session effectiveness when comparing the Nominal Group Technique to Group Brainstorming for groups utilizing speech-recognition as the input method of choice. The average effectiveness score of the groups using speech-recognition as the method of data entry while using the Nominal Technique to generate ideas was 0.273, while the effectiveness of groups generating ideas through Group Brainstorming was 0.379. The p-value of the independent sample T-test was 0.098. Therefore, this hypothesis is retained, as there is no statistical difference between the average effectiveness of these two treatments even though the Nominal Technique was more efficient than the Group Brainstorming technique when speech-recognition was used as the method of data entry. 50 Hypothesis 2c: There will be no significant difference in session efficiency when comparing the Nominal Group Technique to Group Brainstorming for groups utilizing the keyboard as the input method of choice. The average efficiency score of the groups using keyboard as the method of data entry while using the Nominal Technique to generate ideas was 1.69, while the effectiveness of groups generating ideas through Group Brainstorming was 1.25. The p- value of the independent sample T-test was 0.029. Therefore, this hypothesis must be rejected, as there is a statistical difference between the average efficiencies of these two treatments. Once again, the Nominal Technique was found to be more effective than the Group Brainstorming technique when using the keyboard as the method of data entry. Hypothesis 2d: There will be no significant difference in session effectiveness when comparing the Nominal Group Technique to Group Brainstorming for groups utilizing the keyboard as the input method of choice. The average effectiveness score of the groups using keyboard as the method of data entry while using the Nominal Technique to generate ideas was 0.3193, while the effectiveness of groups generating ideas through Group Brainstorming was 0.5556. The p-value of the independent sample T-test was 0.004. Therefore, this hypothesis must be rejected, as there is a statistical difference between the average effectiveness of these two treatments. The Nominal Technique was more effective than the Group Brainstorming technique when using the keyboard as the method of data entry. 51 CHAPTER V DISCUSSION Overview This chapter will provide a discussion of the results as presented in Chapter IV. It opens by discussing the two measures of efficiency and effectiveness used in the study and their impact on future research. Discussions of the different types of idea generation and input methodologies will follow. Finally, additions to the research field, future research directions, practical applications, and implications for practice will be presented. Efficiency and Effectiveness The measures of Group Efficiency and Group Effectiveness used in this study were developed for this study according to the formula used previously used to measure group idea generation outcomes. Group Efficiency (Equation 1 in Chapter III) was developed to standardize the number of original ideas generated per minute. This measure provides a convenient method for comparing the results obtained by different studies. Group Effectiveness (Equation 2 in Chapter III) makes a similar comparison, but here the results are divided by the total number of comments rather than the time. This gives a measure of how well the idea generation type and the input method promoted ?flame free? idea generation. 52 In this study, four treatments were compared based on their Group Efficiency and Group Effectiveness scores. Post-hoc analysis showed that the most efficient of all treatments were the ones that used the Nominal Group Technique. At the 95% confidence level there was a positive statistical difference between the treatments using the Nominal Group Technique, regardless of input methodology. This suggests that regardless of the input methodology of choice, the nominal method still outperforms the group method. It seems that using technology to overcome the process losses imposed by groups does not achieve its goal. Given that the nominal method is demonstrably more effective than the other method tested, it should be noted that the raw efficiency score for the nominal-speech treatment was higher than the nominal-keyboard treatment and the respective p-value was .223. Even though this is not statistically significant, it suggests that under the assumption that the null hypothesis is actually true (no difference), there is only a 22.3% chance that these same results would be found. In other words, there is a 77.7% chance that there is a difference. Future research should investigate this further to determine the true relationship between these treatments. Also, extraneous variables that could not be controlled may have contributed to this result, which merits further discussion. Observing the sessions, subjects participating in the nominal-keyboard groups appear to have had no outside influences, specifically production blocking, and no evaluation apprehension because their comments were completely anonymous. Also, the method of data entry is fairly natural to them. These subjects, averaging 20 years old, have grown up around computers and have been using the keyboard for data entry for many years. Therefore, they seemed to participate in an idea generation session with 53 ease. This allowed them time to generate ideas, but also show their true feelings about the subject, including lots of flames and inefficiencies. The nominal-speech groups, on the other hand, did have variables introduced that may explain the stated difference. Again, by observation subjects participating in speech-keyboard groups were just as familiar with the keyboard as those using a keyboard. Yet, they were required to use a new and different technology for data entry. This caused quite a bit of frustration, especially considering the relatively low 83% accuracy rate. The biggest process loss was in the indecipherable comments. Had these been transcribed correctly, they may have contained original ideas. However, when the students saw a mistake in transcription, many of them submitted it and tried again, creating a flame. This same phenomenon may have actually increased efficiency by limiting the amount of time the participants had available to generate flames. Their time was so wrapped up in trying to submit a few original ideas that they did not have time to flame, therefore artificially raising the efficiency score. Given the issues raised in the above discussion of Group Efficiency, it is also interesting to look at Group Effectiveness. The most effective treatment was the nominal-keyboard combination, showing that on average this treatment produced the most original ideas as a percentage of the total. In all cases, this treatment statistically outperformed the others with a 55% effectiveness rate compared to the next highest, for nominal-speech, of 38%. These subjects seemed to participate in an idea generation session in its purest form, abiding by all brainstorming rules without any of the process losses normally imposed by a group or a technology. This allowed them time to 54 generate original ideas and to show their true feelings about the subject without outside influence. Of the ineffective groups, were the inefficiencies software related, group related, or both? The results of this study are inconclusive. Statistically, there was no significant difference between these other groups. Future research should delve into this area in order to determine if and how much of a process loss is created by the software and by the groups. Idea Generation Idea generation is a very important step in the decision making process (Jackson & Poole, 2003). Consequently, researchers have long tried to find the most efficient and effective way to accomplish this task. In studying this phenomenon, comparisons have been made between idea generation types (Diehl & Stroebe, 1987; Hill, 1982) and traditional and electronic idea generation (Jackson & Poole, 2003; Pinsonneault et al., 1999; Sosik, 1997). Thus far, research has shown that the Nominal Group Technique has been more efficient and effective than Group Brainstorming (Jackson & Poole; Pinsonneault et al.; Sosik). In an attempt to explain this, later research imposed GDSS technologies hoping to find that electronic brainstorming overcame some of the process losses imposed by the group over the individual. However, in no case was electronic brainstorming more efficient or effective than traditional face-to-face brainstorming. It was assumed that the technology imposed to reduce process losses (production blocking, evaluation apprehension, and social loafing) may actually have added more losses to the equation. This study posed the argument that a technology (if such existed) 55 that allows a completely natural method of interaction with the computer may be the missing link that will lead to the success of electronic brainstorming. By the nature of Hypothesis Testing, all eight of the hypotheses assumed that there would be no difference in treatments. Chapter II showed the reasoning that led to the development of the hypotheses based on the concept that speech-recognition software may be the technology that would allow groups to overcome their self-imposed process losses. However, the results presented in Chapter IV do not support this notion, but quite the opposite. In all cases, the Nominal Technique continued to outperform the Group Brainstorming technique regardless of the variables that were held constant. It seems that researchers keep adding layers to the brainstorming session trying to improve on its efficiency and effectiveness. Figure 7 illustrates the layers and their respective process losses. For the different types of idea generation, the Nominal Technique has again outperformed the group brainstorming technique. Future research should continue to investigate the variables that contribute to the process losses experienced when engaging in electronic brainstorming and its impact on these two techniques. It is entirely possible, however, that all of the research conducted to show that for groups, brainstorming is superior to nominal brainstorming actually shows only one thing, namely that the only way to overcome the issues imposed by using a group (production blocking, social loafing, evaluation apprehension), is to not use a group! It could be that no matter how many input methodologies are tested or how many different versions of electronic brainstorming are utilized, individuals participating in the Nominal Technique will continue to outperform groups. Nominal Superior GDSS tools added to overcome group imposed process losses Group BrainstormingNominal Technique Nominal Superior Group BrainstormingNominal Technique VS VS Speech tools added to overcome technology imposed process losses Nominal Superior Group BrainstormingNominal Technique VS Pr e v io u s R e s e a r c h Cu rre n t Re sea r ch Figure 7. Progression of Idea Generation Research Input Methodologies The foremost discussion here must be about the accuracy of the speech- recognition software. Obviously, it did not perform as well as advertised, with only an average 84% accuracy rate as opposed to the 96% claimed, and therefore the technology cannot yet provide the task-technology needed here. There seem to be some layering issues that cause latency within the software, resulting in the software interacting more slowly with the group support software than it did with the individual software. This latency is another variable that future research should look into. Based on observation alone, participants who generated ideas using the keyboard seemed to feel much more comfortable and natural than those using the speech software. 56 57 By default, they had much more experience with it. Only 3 of the 80 participants using speech-recognition technology reported having had any prior experience with it. Due to the increase in computer competence among college aged people, the studies that concluded that traditional Non-GDSS supported idea generation outperformed GDSS supported should be replicated. It is possible that both the technology involved and the users have evolved into a better task-technology fit that would allow electronic brainstorming to outperform traditional face-to-face idea generation without the need for the speech-recognition software. Currently, speech-recognition software (dependent-continuous) does not provide the functionality needed for this application. However, once it evolves to create a better fit, which would require the accuracy levels to improve dramatically and the latency issue to be addressed, this research should be replicated to retest the stated hypotheses in the new environment. Implications for Practitioners Practitioners wishing to benefit from this research can rest assured that past and current research point to the Nominal Technique as being the most efficient and effective way to generate ideas. This is largely due to the fact that this technique, if anonymity is assumed, does not create an environment conducive for evaluation apprehension or production blocking, the primary process losses associated with Group Brainstorming. However, this and other studies have not taken idea quality into consideration. It may be that the nominal technique generates more ideas, while group brainstorming creates a 58 higher quality of idea. The process losses on one hand may actually be the agents that move an idea generation session toward a steady stream of higher quality ideas. It should also be noted that speech-recognition, though not yet perfected, is improving. There are many applications of speech-recognition tools that may be a better task-technology fit than the one tested in this study. However, there are a few factors at work that may help the popularity of speech-recognition to reach a critical mass where it will be able to compete with the keyboard as the input methodology of choice. Those factors are the shrinking size of computers, faster processors, and smaller, higher capacity memory. As the population of people needing and using these technologies increases demand for these technologies, the availability and effectiveness of speech-recognition and other non-traditional means of input will increase accordingly. The findings of this study have clearly shown that speech-recognition technology is not yet up to the stringent requirements of this task environment, but a repeat of the study in five years may show a completely different result. 59 CHAPTER VI CONCLUSIONS Overview This chapter presents the conclusions of this study. First, a summary of the dissertation will be presented, followed by a description of the limitations of the current study. Suggestions for future research based on addressing these limitations will be included. Summary of Dissertation The current business environment forces organizations to make decisions that are both efficient and effective (Beeri & Spiegler, 1996; Karacapilidis & Papadias, 1998). Often, many organizations resort to using groups for important decisions (Hilmer & Dennis, 2001; Kim et al., 1998), specifically, groups of experts (Beeri & Spiegler; DeSanctis & Gallupe, 1989; Gallupe et al., 1988) because groups are expected to have a larger knowledge base than an individual, and be able to come to a more efficient decision (a satisficing, time proven decision) than an individual in a more effective manner (in a shorter amount of time). Additionally, it was thought groups utilizing tools provided by a GDSS would be even more efficient and effective. 60 GDSS systems are designed to overcome some of the problems that hinder groups in traditional face-to-face group meetings (Dennis et al., 1988). Specifically, GDSSs alter the way teams interact, endeavoring to increase process gains and reduce process losses (Dennis & Garfield, 2003; DeSanctis & Gallupe, 1987; Nunamaker et al., 1991). However, past research and the current study have shown neither groups nor their supporting systems to be a complete panacea. Just as groups introduce potential socio- political process losses, the GDSS technologies used often create additional barriers to group efficiency and effectiveness (Gavish et al., 1995). This is in part due to the fact that a computer-mediated group decision process will face the same difficult environment as the individual, though with more technical complexity. Idea generation itself is a very important GDSS task because it serves important functions across many group activities, including problem solving, decision making, negotiation, planning, and innovation (Jackson & Poole, 2003). In theory, groups using GDSS technology that lessens the negative impacts of hindrances should produce more ideas than groups not using the technology. However, both Jackson and Poole and Pinsonneault et al. (1999) found that GDSS-supported idea-generation produced fewer ideas than non-supported sessions. Furthermore, non-GDSS idea generation has been shown to be more efficient than GDSS-supported brainstorming (Sosik, 1997), revealing the supposed productivity gains of electronic brainstorming to be an illusion (Pinsonneault et al.). A possible reason for the discrepancy in outcomes of electronic idea generation research is the difference between idea-generation techniques; Group Brainstorming (Osborne, 1957) and the Nominal Group Technique (Delbecq et al., 1975). During a 61 Group Brainstorming session, there are many socio-political factors that must be considered, including time pressure, conflicts, hidden agendas, mistakes (Jackson & Poole, 2003), fear of criticism, and the political culture of the organization (Dennis et al., 1988; Valacich & Dennis1994). By nature, these sessions are asynchronous; to propose an idea, one must first gain permission to speak. While waiting to speak, a member could forget their idea, or come up with another idea, forgetting the first. Face-to-face brainstorming has been shown to consistently introduce more process losses than gains (Diehl & Stroebe, 1987; Hill, 1982). While generating ideas using the Nominal Group Technique, group members are in different places at possibly different times, keeping up with their own ideas and never being swayed by others (Delbecq et al., 1975). With anonymity often promised, individuals may feel more secure making their suggestions. However, they may also feel the freedom to engage in social loafing (doing nothing since no one knows their identity). For both of these idea generation types, group member computing skills are a variable that has not previously been studied in the context of idea generation, particularly skills regarding different input methodologies. GDSS allows (or forces) members to use electronic communication in addition to, or instead of, verbal communication (Ackermann & Eden, 2001; Dennis & Garfield, 2003). For less computer savvy people, this is a problem. Recent publications have reported that user- driven GDSS resulted in lower individual satisfaction due to technical unfamiliarity (Chun & Park, 1998) and older studies were conducted primarily on the basis that most executives did not like to type (Gray et al., 1981). 62 Aside from typing skills and satisfaction, many individuals simply operate better when they can dictate rather than type because their keyboarding skills interfere with their thought process and/or they are not highly skilled typists (Macarthur & Cavalier, 2004). Accordingly, the problem statement for this study was stated as follows: if GDSS technologies were designed to remove process losses imposed by groups, then why have teams using non-GDSS supported techniques consistently outperformed or equaled the performance of electronic brainstorming? Specifically, will speech-recognition technology be the variable that allows Group Brainstorming to outperform individuals engaged in the Nominal Technique? This study, therefore, compared speech recognition technology to the use of the keyboard in electronic idea generation sessions for the purpose of investigating whether or not the computer, specifically the keyboard and the required skills that go along with it, impose process losses into the electronic idea generation scenario. Additionally, this dissertation explored two different types of idea generation: the Nominal Group Technique (Delbecq et al., 1975) and Group Brainstorming (Osborne, 1957). The measurements used in this study were group efficiency (the number of original ideas developed) and group effectiveness (the percentage of original ideas compared to the total number of comments generated). Group efficiency was standardized by the amount of time in each session (10 minutes) in order to make future cross-study comparisons easier. Each session was saved in a text file and later analyzed, breaking comments into original, repeats, flames (sarcastic or nonproductive comments), and indecipherable comments (from the speech-recognition software). These numbers were used to produce each group?s respective efficiency and effectiveness score. 63 The results were somewhat disappointing. Even though this study was conducted in a lab setting where many variables could be controlled, there were still uncontrolled variables that impacted the outcome. The speech-recognition software did not perform as advertised, exhibiting an 84% accuracy rate rather than the 96% claimed. Limitations from this problem are discussed in the next section. As a result, the Nominal Group Technique again outperformed the Group Brainstorming technique, measured by group efficiency and effectiveness, in both the speech and the keyboard environment. Additionally, the keyboard groups outperformed the speech groups on all hypotheses except for one. The Group Brainstorming/Speech treatment had a higher efficiency score than Nominal/Speech. This will be explained in the Limitations section. Limitations The most notable limitation of this study was the accuracy demonstrated by the speech-recognition software. The difference between the advertised 96% and the realized 84% is extremely disappointing. This undoubtedly drastically reduced the efficiency and effectiveness of the groups using this technology. Had the ?indecipherable? comments been accurately transcribed, the results may have been very different. Another limitation was that the speech-recognition software seemed to suffer from a layering effect among different applications. There was some latency in that it took longer for the software to transcribe into the group sessions than it did for the individual sessions. This delay definitely added some production blocking and seemed to cause a great deal of frustration on the part of the end user. This latency may have been the cause of the higher efficiency score mentioned above. Paradoxically, these 64 participants were at an apparent disadvantage that may actually have worked to their advantage. Because it was so difficult to put their comments into the system, they did not have time to ?flame? about the topic. They first submitted their own ideas, and then only if they had time flamed about the software. A second limiting feature was the level of experience the population had with the keyboard. Aside from the latency and inaccuracy of the speech software, the fact that this population is much more adept with the keyboard than previous study populations made it a frustrating task to not be able to accurately communicate with the computer. Additional research is needed to address these issues, as well as the others mentioned in this dissertation. 65 REFERENCES Ackermann, F., & Eden, C. (2001). Contrasting single user and networked group decision support systems for strategy making. Group Decision & Negotiation, 10(1), 47-66. Aiken, M., Vanjami, M., & Krosp, J. (1995). Group decision support systems. Review of Business, 16(3) 38-43. Aiken, M., Martin, J., Paolillo, J., & Shirani, A. (1994). A group decision support system for multilingual groups. Information & Management, 26, 155-161. Alavi, M. (1994). Computer-mediated collaborative learning: an empirical investigation. MIS Quarterly, 18(2), 159-185. Bajaj, S. (2004, March). Expanding the reach of speech recognition. Health Management Technology, 16-22. Barkhi, R. (2002a). Cognitive style may mitigate the impact of communication mode. Information & Management, 39, 677-688. Barkhi, R. (2002b). The effects of decision guidance and problem modeling on group decision-making. Journal of Management Information Systems, 18(3), 259-282. Bass, B.M., & Avolio, B.J. (1994) Improving organizational efficiency through transformational leadership. Thousand Oaks, CA: Sage. 66 Beeri, Y., & Spiegler, I. (1996). Synergetic expert systems. Decision Support Systems, 17, 73-82. Bostrom, R. P., & Heinen, J.S. (1977a). MIS problems and failures: A socio-technical perspective, Part 1. MIS Quarterly, 1(3), 17-32. Bostrom, R. P., Heinen, J.S. (1977b). MIS problems and failures: A socio-technical perspective, Part 2. MIS Quarterly, 1(4), 11-28. Bui, T., & Sivasankaran, T.R. (1990). Relation between GDSS use and group task complexity: An experimental study. In Proceedings of the 23 rd Annual Hawaii International Conference on System Sciences, pp. 69-78. Carr, H.H., & Snyder, C.A. (2003). The management of telecommunications: Business solutions to business problems enabled by voice and data communications, 2 nd Ed. New York: McGraw-Hill Higher Education. Cavalier, A.R., & Ferretti, R.P. (1996). Talking instead of typing: Alternate access to computers via speech recognition technology. Focus on Autism & Other Developmental Disabilities, 11(2), 79-85. Chidambaram, L. (1996). Relational development in computer-supported meetings. MIS Quarterly, 20(2), 143-166. Chun, K.J., & Park, H.K. (1998). Examining the conflicting results of GDSS research. Information & Management, 33, 313-325. Cohen, J. (2004, March). Speech recognition comes of age. Wireless Systems Design, 16. 67 Connolly, T., Jessup, L., & Valacich, J. (1991). Effects of anonymity and evaluative tone on idea generation in computer-mediated groups. Management Science, 36(6), 689-703. Daft, R.L., & Weick, K.E. Toward a model of organizations as interpretation systems. Academy of Management Review, 9, 284-295. Daily, B., Whatley, A., Ash, S., & Steiner, R. (1996). The effects of a group decision support system on culturally divers and culturally homogeneous group decision making. Information & Management, 30, 281-289. Delbecq, A., Van De Ven, A.H., & Gustafson, D.D. (1975). Group techniques for program planning: A guide to nominal and Delphi processes. Glenview, IL: Scott, Foresman. Dennis, A., Aronson, J., Heninger, W., & Walker II, E. (1999). Structuring time and task in electronic brainstorming. MIS Quarterly, 23(1), 95. Dennis, A., & Garfield, M.J. (2003). The adoption and use of GSS in project teams: Toward more participative processes and outcomes. MIS Quarterly, 27(2), 289- 323. Dennis, A., George, J., Jessup,L., Nunamaker, J., & Vogel, D. (1988). Information technology to support electronic meetings. MIS Quarterly, 12(4), 591-624. Dennis, A., Wixom, B.H., & Vandenberg, R.J. (2001). Understanding fit and appropriation effects in Group Support Systems via meta-analysis. MIS Quarterly, 25(2), 167-193. DeSanctis, G.L., & Gallupe, R.B. (1987). A foundation for the study of group decision support systems. Management Science, 33(5), 589-600. 68 DeSanctis, G.L., & Gallupe, R.B. (1989, December). Foundations for the study of group decision support systems. Management Science, 315-325. Diehl, M., & Stroebe, W. (1987). Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of Personality & Social Psychology, 53, 497-509. Er, M.C., & Ng, A.C. (1995). The anonymity and proximity factors in group decision support systems. Decision Support Systems, 14, 75-83. Fijol, Y., & Woodbury, M.A. (1987). Group DSS (decision support systems) and decision outcome measures: A comparative study in distributed versus non- distributed settings. DTIC No. AD-A180, 949. Fulk, J., Schmitz, J., & Steinfield, C.W. (1990) A social influence model of technology use. In J. Fulk and C.W. Steinfield (Eds.), Organizations and communication technology. Newbury Park, CA: Sage. Gallupe, R.B. (1986) Experimental research into group decision support systems: Practical issues and problems. In Proceedings of the 19 th Annual Hawaii International Conference in Systems Sciences, pp. 515-523. Gallupe, R.B., DeSanctis, G., & Dickson, G.W. (1988). Computer-based support for group problem finding: Experimental investigation. MIS Quarterly, 12(2), 277- 296. Gass, S.L., Collins, R.W., Meinhardt, C.W., Lemon, D.M., & Gillette, M.D. (1988). The army manpower long-range planning systems. Operations Research, 36(1). Gavish, B., Gerdes Jr., J., & Sridhar, S. (1995). CM3: A distributed group decision support system. IIE Transactions, 27(6), 722-734. 69 Gellerman, E. (1994). Speech-recognition technology: The final frontier. T H E Journal, 21(8), 14-17. Goodhue, D.L., & Thompson, R.L. (1995). Task/technology fit and individual performance. MIS Quarterly, 19, 213-236. Gray, P., Berry, N.W., Aronofsky, J.S., Helmer, O., Kane, G.R., & Perkins T.E. (1981). The SMU decision room project. In Transactions of the 1 st International Conference on Decision Support Systems. Atlanta, GA., pp. 122-129. Guzzo, R., Yost, P., Campbell, R., .& Shea, G. (1993). Potency in groups: Articulating a construct. British Journal of Social Psychology, 32, 87-106. Hatcher, M. (1995). A tool kit for multimedia supported group/organizational decision systems. Decision Support Systems, 15, 211-217. Hayne, C.H., Pollard, C.E., & Rice, R.E. (2003). Identification of comment authorship in anonymous group support systems. Journal of Management Information Systems, 20(1), 301-329. Hedberg, S.R. (1997, Nov/Dec). Dictating this article to my computer: Automatic speech recognition is coming of age. IEEE Expert, 9-11. Hender, J.M., Dean, D.L., Rodgers, T.L., & Nunamaker, Jr., J.F. (2002). An examination of the impact of stimuli type and GSS structure on creativity: Brainstorming versus non-brainstorming techniques in a GDSS environment. Journal of Management Information Systems, 18(4), 59-85. Higgins, E.L., & Raskind, M.H. (2004). Speech recognition-based and automaticity programs to help students with severe reading and spelling problems. Annals of Dyslexia, 54(2), 365-392. 70 Hill, G.W. (1982). Group versus individual performance: Are N+1 heads better than one? Psychological Bulletin, 91, 517-539. Hilmer, K.M., & Dennis, A.R. (2001). Stimulating thinking: Cultivating better decisions with groupware through categorization. Journal of Management Information Systems, 17(3), 93-114. Jackson, M.H., & Poole, M.S. (2003). Idea-generation in naturally occurring contexts. Human Communication Research, 29(4), 560-591. Jarvenpaa, S.L., Rao, V.S., & Huber, G.P. (1988). Computer support for meetings of groups working on unstructured problems: A field experiment. MIS Quarterly, 645-665. Jessup, L.M., Connolly, T., & Tansik, D. (1990). Toward a theory of automated group work: The de-individuating effects of anonymity. Small Group Research, 21(3), 333-348. Jessup, L.M., Tansik, D., & Laase, T.D. (1988). Group problem solving in an automated environment: The effects of anonymity and proximity on group process and outcome with a group decision support system. In Proceedings of the Academy of Management 1988 Annual Meeting. Anaheim, CA. Kahai, S., Avolio, B., & Sosik, J. (1998). Effects of source and participant anonymity and difference in initial opinions in an EMS context. Decision Sciences, 29(2), 427-460. Karacapilidis, N. (2000). Integrating new information and communication technologies in a group decision support system. International Transactions in Operations Research, 7, 487-507. 71 Karacapilidis, N., & Papadias, D. (1998). A computational approach for argumentative discourse in multi-agent decision making environments. AI Communications, 11, 21-33. Karacapilidis, N., & Pappis, C.P. (1997). A framework for group decision support systems: Combining AI tools and OR techniques. European Journal of Operational Research, 103, 373-388. Kiesler, S. (1986). Thinking ahead: The hidden messages in computer networks. Harvard Business Review, 64(1), 4660. Kreamer, K.L., & King, J.L.. (1998). Computer-based systems for cooperative work and group decision making. ACM Computing Surveys, 20(2), 115-146 Juul-Kristensen, B., Laurensen, B., Pilegaard, M., & Jensen, B.R. (2004). Physical workload during use of speech recognition and traditional computer input devices. Ergonomics, 47(2), 119-133. Kim, S.H., Choi, H.C., & Ahn, B.S. (1998). Interactive group decision process with evolutionary database. Decision Support Systems, 23, 333-345. Koester, H.H. (2004). Usage, performance, and satisfaction outcomes for experienced users of automatic speech recognition. Journal of Rehabilitation Research & Development, 41(5), 739-754. Louis, F. (1996). Automatic speech recognition. Information Systems Management, 13(1), 29-38. Macarthur, C.A., & Cavalier, A.R. (2004). Dictation and speech recognition technology as test accommodations. Exceptional Children, 71(1), 43-58. 72 Marchewka, J.T., & Goette, T. (1992, Spring). Will computers have ears? Implications of speech recognition technology. Business Forum, 26-29. Nunamaker, J., Dennis, A., Valacich, J., Vogel, D., & George, J. (1991). Electronic meeting systems to support group work. Communications of the ACM, 34(7), 30- 39. Osborne, A. (1957). Applied imagination. New York: Scribner. Paul, S., Seetharaman, P., Samarah, I., and Mykytyn, P.P. (2004). Impact of heterogeneity and collaborative conflict management style on the performance of synchronous global virtual teams. Information and Management, 41, 303-321. Pinsonneault, A., Barki, H., Gallupe, R.B., & Hoppen, N. (1999). Electronic brainstorming: The illusion of productivity. Information Systems Research, 10(2), 110-133. Rebman, C.M., Aiken, M.W., & Cegielski, C.G. (2003). Speech recognition in the human-computer interface. Information & Management, 40, 509-519. Rieger, J.M. (2003). The effect of automatic speech recognition systems on speaking workload and task effectiveness. Disability & Rehabilitation, 25(4-5), 224-235. Roth, J. (2003, Nov). Speech recognition tech offers low-cost processing. National Underwriter, 15-17. Ruhe, J.A., & Eatman, J. (1977). Effects of racial composition on small work groups. Small Group Behavior, 8, 479-486. Ruhe, J.A., & Allen, W.R. (1977). Differences and similarities between Black and White leaders. In Proceedings of the American Institute of Decision Sciences, Northeast Division, pp. 30-35. 73 Sambamurthy, V., & Chin, W. (1994). The effects of group attitudes toward alternative GDSS designs on the decision-making performance of computer-supported groups. Decision Sciences, 25(2), 215-241. Scott, C.R., Quinn, L., Timmerman, C.E., & Garrett, D.M. (1998). Ironic uses of group communication technology: Evidence from meeting transcripts and interviews with Group Decision Support System users. Communication Quarterly, 46(3), 353-375. Schneider, K.N., & Owsen, D. (2003, Aug). Boosting productivity with speech- recognition systems. The CPA Journal, 12-13. Shea, G.P., & Guzzo, R.A. (1987). Group efficiency: What really matters? Sloan Management Review, 28, 25-31. Sosik, J.J. (1997). Effects of transformational leadership and anonymity on idea generation in computer-mediated groups. Group & Organization Management, 22(4), 460-488. Sosik, J.J., Avolio, B.J., & Kahai, S.S. (1997). Effects of leadership style and anonymity on group potency and efficiency in a Group Decision Support System environment. Journal of Applied Psychology, 82(1), 89-103. Sosik, J.J., Avolio, B.J., Kahai, S.S., & Jung, D.I. (1998). Computer-supported work group potency and efficiency: The role of transformational leadership, anonymity and task interdependence. Computers in Human Behavior, 14(3), 491-511. Valacich, J.S., & Dennis, A.R. (1994). A mathematical model of performance of computer-mediated groups during idea generation. Journal of Management Information Systems, 11(1), 59-72. 74 Voskuhl, A. (2004). Humans, machines, and conversations: An ethnographic study of the making of automatic speech recognition technologies. Social Studies of Science, 34(3), 393-421. Watson, R.T., DeSanctis, G., & Poole, M.S. (1988). Using a GDSS to facilitate group consensus: Some intended and unintended consequences. MIS Quarterly, 462- 478. William, W. (1998). Speech recognition: Instead of typing and clicking, talk and command. T H E Journal, 25(6), 18-21. Zigurs, I., & Kozar, K.A. (1994) An exploratory study of roles in computer-supported groups. MIS Quarterly, 18(3), 277-298. Zhuge, H. (1998). Conflict group decision training: Model and system. Knowledge- Based Systems, 11, 191-196. 75 APPENDIX A IRB APPROVAL AND INFORMED CONSENT 76 77 78 79 APPENDIX B DEMOGRAPHIC SURVEY Thank you for participating in our study. The information contained on this page will be kept completely confidential and will only be used for research purposes. Please provide the following: AU Global ID: Major null Accounting null Finance null Marketing null Management null Logistics null Management Information Systems null Aviation Management null General Business null Other: ____________________________________ Year: null Freshman null Sophomore null Junior null Senior Gender null Male null Female