DOES TARGETED CAMPAIGN MESSAGE IMPACT VOTE INTENTION AND VOTE CHOICE? AN EXPERIMENTAL STUDY OF ALABAMA SENIORS Except where reference is made to the work of others, the work described in this dissertation is my own or was done in collaboration with my advisory committee. This dissertation does not include proprietary or classified information. ___________________________________ Joshua M. Whitman Certificate of Approval: Mitchell Brown Assistant Professor Political Science Cynthia Bowling, Chair Associate Professor Political Science Cal Clark Professor Political Science Brad Moody Professor Political Science George T. Flowers Dean Graduate School DOES TARGETED CAMPAIGN MESSAGE IMPACT VOTE INTENTION AND VOTE CHOICE? AN EXPERIMENTAL STUDY OF ALABAMA SENIORS Joshua Whitman A Dissertation Submitted to the Graduate Faculty of Auburn University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy Auburn, Alabama May 9, 2009 iii DOES TARGETED CAMPAIGN MESSAGE IMPACT VOTE INTENTION AND VOTE CHOICE? AN EXPERIMENTAL STUDY OF ALABAMA SENIORS Joshua M. Whitman Permission is granted to Auburn University to make copies of this dissertation at its discretion, upon request of individuals or institutions and at their expense. The author reserves all publication rights. ______________________________ Signature of Author ______________________________ Date of Graduation iv DISSERTATION ABSTRACT DOES TARGETED CAMPAIGN MESSAGE IMPACT VOTE INTENTION AND VOTE CHOICE? AN EXPERIMENTAL STUDY OF ALABAMA SENIORS Joshua M. Whitman Doctor of Philosophy, May 9, 2009 (M.P.S., Auburn University Montgomery, 2001) (B.S., Troy University, Montgomery, 1996) 182 Typed Pages Directed by Cynthia Bowling Conventional wisdom and the huge amount of money spent by political campaigns in the United States hold that multiple targeted messages influence voter turnout and voter vote choice. Prior studies have focused primarily on voter mobilization with only a few studies testing for a persuasion effect. Additionally, there are very few field experiments that test targeted messages in direct democratic elections such as referendum campaigns. This dissertation expands the literature in those areas by reporting the results of a statewide randomized field experiment conducted during a 2003 nonpartisan special election in Alabama that included 5,000 registered senior voters aged 65 and older. Both a preference effect and a mobilization effect were hypothesized to exist when senior voters received targeted salient biased messages through direct mail v and prerecorded phone calls. There were five treatment groups and a control group. One treatment group received one targeted direct mail message, another received two targeted messages via direct mail, and a third received three targeted messages?two via direct mail and one through a prerecorded call. Another treatment group received one generic message and one targeted message by direct mail and yet another group received only the generic message. The structure of the experiment allows not only for a comparison of each group to the control group, but also a comparison between groups to test the effectiveness of targeted messages over generic messages and to test the impact of multiple messages over a single message. While a voter recall effect was found in that more-educated seniors were more likely to indicate they recalled messages from the campaign, none of the treatments resulted in a detectable mobilization or persuasion effect. However the lack of statistical power may be the most significant limitation in detecting small effects. Additionally, this dissertation provides a discussion regarding variables field researchers should consider when attempting to control for extraneous influences. Similarly, this dissertation advocates for using the terms biased and unbiased related to the types of messages delivered during experiments rather than the possibly confusing partisan and nonpartisan. Implications for both academic researchers and campaign professionals are discussed. vi ACKNOWLEDGMENTS Deep gratitude goes to my current and past committee members for their support, guidance, assistance, motivation, thoughtfulness, and, most importantly, patience during this endeavor. Professors Cynthia Bowling (Chair), Robert Bernstein (former Chair), Cal Clark, Mitchell Brown, Bradley Moody, and Anne Permaloff all played key rolls in my finishing this project and I am forever indebted to them. Dr. Bernstein passed away during this project; his vision and guidance brought it into existence. Further, I would like to thank Professor Paul Starr for energetically serving as my outside reader. I am extremely fortunate to be blessed with an amazingly supportive set of parents, family, and friends. They too have journeyed with me and have experienced many of the same highs and lows that I have gone through. To them I say thanks for your love, support, and friendship. I thank the far-too-numerous-to-list group of teachers, professors, mentors, family, and friends who supported me in my education journey. I thank Dr. Joe Perkins who has been an employer, client, mentor, father figure, and, most importantly, friend to me; I am grateful for his whole extended family?s support. Dr. Jerry Ingram has also been a wellspring of support. I thank my brother Jason who carried an ocean of water for me while I was dissertating and Elaine who gave of her time so that I might dissertate. A special thanks to Dr. Sally Jensen for her guidance in helping me overcome my vii ?gremlins? and stay focused. I thank Dr. James Moore and Sue & John Morris for their assistance. Finally, I thank my wife and son for providing me with the daily support and inspiration I needed to finish this project. For Grayson, I hope I have set a positive example for you relating to perseverance and determination in the same way you have set a positive example for me related to unconditional love and adventure. For my wife, Misha, this dissertation is dedicated to you. We did it! viii Style manual used Publication Manual of the American Psychological Association, 5th edition Software used Bookends 10 & 10.5 Filemaker Pro 9 Advanced JMP 7 Microsoft Excel 2008 Microsoft Word 2008 OmniOutliner Pro 3.7.2 Papers 1.8.6 & 1.9.1 Scrivener 1.11 & 1.5 SPSS 16 ix TABLE OF CONTENTS LIST OF TABLES............................................................................................................xii CHAPTER 1: INTRODUCTION.......................................................................................1 Introduction................................................................................................................1 Background of the Problem.......................................................................................3 Campaign Field Experiments...................................................................................12 The Experiment and Variables.................................................................................14 Amendment One Field Setting.................................................................................16 Study Method...........................................................................................................17 Data Analysis...........................................................................................................18 Organization of Chapters.........................................................................................18 CHAPTER 2: BACKGROUND/LITERATURE REVIEW.............................................20 Introduction..............................................................................................................20 Scope of Review......................................................................................................24 Experimental Methods.............................................................................................26 Referendum..............................................................................................................31 Partisanship Redefined.............................................................................................33 Persuasion................................................................................................................38 Mobilization.............................................................................................................43 Message Synergy.....................................................................................................45 x Message Frequency..................................................................................................50 Message Testing.......................................................................................................55 Message Targeting...................................................................................................60 Seniors......................................................................................................................64 Conclusion...............................................................................................................69 CHAPTER 3: METHODS AND RESEARCH DESIGN.................................................73 Introduction..............................................................................................................73 Literature Summary.................................................................................................73 Research Questions and Hypotheses........................................................................76 Research Design.......................................................................................................77 The Experiment........................................................................................................79 Population and Sample.............................................................................................82 Data Collection........................................................................................................84 Instrumentation........................................................................................................86 Data Analysis...........................................................................................................90 Threats to Validity...................................................................................................91 Internal Validity........................................................................................92 External Validity.......................................................................................92 Summary..................................................................................................................95 CHAPTER 4: RESULTS..................................................................................................96 Study Variables........................................................................................................97 Univariate Analysis..................................................................................................97 Personal Demographics.............................................................................97 xi Political Control Variables......................................................................101 Bivariate Analysis..................................................................................................102 Exploratory Analysis..............................................................................................106 Regression Analysis...............................................................................................110 Summary................................................................................................................116 CHAPTER 5: DISCUSSION..........................................................................................117 Introduction............................................................................................................117 Study Strengths......................................................................................................119 Review of Findings................................................................................................120 Study Limitations...................................................................................................125 Research Implications and the Future....................................................................130 Practical Implications.............................................................................................133 Final Thoughts.......................................................................................................136 REFERENCES................................................................................................................139 APPENDIX A QUESTIONNAIRE................................................................................153 APPENDIX B EXPERIMENTS REVIEWED...............................................................158 APPENDIX C SENIORS I DIRECT MAIL...................................................................162 APPENDIX D SENIORS 2 DIRECT MAIL..................................................................163 APPENDIX E WORKING FAMILIES DIRECT MAIL...............................................165 APPENDIX F SENIORS PHONE..................................................................................166 APPENDIX G STATISTICAL TABLES.......................................................................167 xii LIST OF TABLES Table 1 Primary Variables.................................................................................................15 Table 2 Experimental Groups...........................................................................................15 Table 3 Characteristics of Discussed Referendum Field Experiments.............................72 Table 4 Timeline of Alabama Seniors Experiment...........................................................80 Table 5 Study Variables....................................................................................................98 Table 6 Senior Participants Personal Demographic Control Variables by Group..........100 Table 7 Senior Participants Political Control Variables by Group.................................101 Table 8 Intention to Vote in the September Election by Group (n = 701)......................104 Table 9 Support for Amendment by Group....................................................................106 Table 10 Advertising and How Seniors Would Benefit by Group.................................108 Table 11 Advertising and Impact on Planned Vote by Group........................................109 Table 12 Overall Summary for Logistic Regression on Likelihood to Vote For/Against Amendment One (N = 315).................................................................112 Table 13 Logistic Regression Analysis Predicting Vote On Amendment One (N = 315)..........................................................................................................................113 1 CHAPTER 1: INTRODUCTION Introduction There is a problem in the study of electoral politics and public policy. Currently, there is extensive theoretical literature in the fields of political science and campaign management that suggests targeted messages are important in mobilizing voters and influencing vote choice. Political consultants and the clients that purchase their consulting services base their behavior on such assumptions. However, in the past, it has been difficult for researchers to demonstrate empirically that these assertions are true and that targeted messages do indeed have an impact on the outcome of elections. The efficacy of targeted messages has a direct impact both on professional political consultants as well as academic researchers who strive to better understand the processes by which voters select political leaders. Consultants and researchers traditionally base their work on certain core assumptions: communication and messages to voters influence the outcome of elections, targeted messages are capable of activating voters, and the style of media such as direct mail or radio matters when it comes to achieving certain outcomes. These assumptions remain largely unverified. Because of this gap in knowledge, candidates, campaigns, and academic researchers might not fully understand the campaign dynamics that ultimately lead directly or indirectly to public policy at all electoral levels in our country. 2 Several factors contribute to this gap in knowledge, but two stand out as likely primary causes. First, it is likely that any impact targeted messages might have on elections would be small and therefore difficult to detect. Second, in the past it has been difficult for researchers to control, at once, for all the factors influencing voting behavior. Researchers generally cannot isolate the effects of a message from other effects because of confounding factors, compromising the most basic of scientific principles. This study contributes to the body of knowledge needed to address this problem by attempting to determine the impact of campaign messages on voters? intention to vote and positions of voters during a live campaign. Research that focuses on how campaign message relates to mobilization and vote choice is significant for several reasons. First, this study investigates and expands on the existing research of campaign message on mobilization. Second, it expands limited research associated with campaign message and referendum elections. Third, it goes beyond building on existing research by attempting to fill a gap in the research literature as it relates to the impact campaigns have on vote choice. Fourth, this research answers the call of several recent and prominent authors in campaign literature that encourage researchers to study campaigns using experiments similar to the one that is the focus of this study (Green & Gerber, 2000; Imai, 2005; Johnson-Cartee & Copeland, 1997; Niven, 2004). Finally, this research has practical implications for the campaign consulting profession as well as society at large because it investigates and makes public research that speaks to the impact that campaigns might have on the outcome of elections. 3 Background of the Problem Elections are a core principle and ?centerpiece? in a democratic system (Herrnson, 2004; Trent & Friedenberg, 2000). In fact, having free and fair elections is a core principle used to determine if a country?s political system is truly a democratic one (Huntington, 1993). It is through the outcome of elections that government is legitimized, and democratic theory becomes practice. Through these processes the wishes of the people become the policies of government (Trent & Friedenberg). And it is political campaigns that determine the outcome of elections. Political campaigns underlay democracy?s way of helping to maintain civic order. They are a sublimation of older, more traditional forms of warfare, mechanisms through which various sides and factions can express their preferences without taking up arms against their opponents or the government. Campaigns are a primary way that factions settle disputes on policy issues and a tool through which society helps create long-term public policy. However, little is actually known about the impact campaigns have on the outcome of elections (Goldstein & Ridout, 2004). Specifically, many researchers have argued that campaigns have a ?minimal effect? on the outcome of elections. However, Luks, Miller, and Jacobs argued that ?there is very little research that explicitly examines the direct effects of campaigns? (Luks, Miller, & Jacobs 2003). Moreover, Finkel and Schrott wrote that ?very little research exists that assesses explicitly the amount and type of campaign effects on voters? (1995. p. 350). This has begun to change over the past decade as Gerber and Green (Gerber & Green, 1999; Green & Gerber, 2003) have published and encouraged research into these effects. A thorough review of this 4 experimental research, both by and inspired by Gerber and Green, will be the primary focus of chapter 2 in this dissertation. Very early in the literature on campaign effects, researchers indicated that not only do campaigns themselves have minimal impact on the outcome of elections but campaign messages must have even less impact (Eldersveld & Dodge, 1954; Gosnell, 1927; Kramer, 1970; Niven, 2001). Such research indicates that factors other than message are more likely to explain the outcome of elections. This academic finding is diametrically opposed to the conventional wisdom used by professionals in the field of campaigns and elections. The conventional wisdom holds that message is a crucial input to a campaign?s success. Primarily, academic research indicates that variables such as socioeconomic status, party identification, education, incumbency, name recognition, race, gender, and age have the largest impacts on how and if voters vote in a particular election; the job of a political campaign is to identify its block of voters and figure out how to get those voters to go to the polls in a given election (Herrnson, 2004; Schaffner & Streb, 2000; Schaffner, Streb, & Wright 2000). In other words, the theoretical literature tends to support the idea that, if a campaign?s message has an impact on a given election, that impact is most directly felt through mobilizing predisposed voters to the polls rather than persuading a voter on a given issue (Ansolabehere & Iyengar, 1997; Herrnson). One reason researchers have focused heavily on mobilization is that past research has indicated campaigns have no persuasive impact on vote choice. However, a second reason researchers have focused heavily on mobilization is the idyllic vision that exists of our democratic system. Americans, as citizens of one of the world?s best-known 5 democracies, tend to have an idyllic vision of democracy in action. That vision typically includes all eligible voters actively engaged in seeking perfect information on the positions of candidates or the impact of referendums. Voters in this vision all are able to make very informed decisions in casting their votes during an election. In such a world, election turnout would be high and the public mandate would send a clear message to governmental leaders outlining the policy preferences of society. This ideal is what has likely led researchers to focus on ways to optimize our democratic system and ensure that elections reflect the voices of all voters. Over the years, researchers have studied numerous ways to help increase turnout such as using direct mail, telephone calls, or face-to-face conversations to mobilize voters (Berelson, Lazarsfeld, & McPhee, 1954; Blydenburgh, 1971; Cain & McCue, 1985; Caldeira, Clausen, & Patterson, 1990; Crotty, 1971; Eldersveld, 1956; Gerber & Green, 2000b; Gosnell, 1927; Huckfeldt & Sprague, 1992; Katz & Eldersveld, 1961; Kramer, 1970; Lazarsfeld, Berelson, & Gaudet, 1968; Lupfer & Price, 1972; Niven, 2002; Rosenstone & Hansen, 1993; Wielhouwer & Lockerbie, 1994). Most likely, mobilization has been a primary focus for campaign researchers because, as mentioned earlier, past research indicated that campaigns primarily have a mobilization impact rather than a persuasive impact. In conducting mobilization research, researchers have determined that there are different types of voters based on the costs associated with casting a ballot. These costs include such overt things as having a person take time away from their job to cast a ballot and the ease with which a person learns enough about the candidates to make a decision about whom they should support. ?Low information cost voters? tend to make their vote 6 choice quickly while ?high information cost voters? tend to wait until very late in the campaign (Niven, 2002; Schaffner & Streb, 2000; Schaffner et al., 2000). One might think of a campaign as a series of impressions about the candidates or issues. The Online Model of vote choice asserts that voters maintain a running tally in their minds based on these impressions of a given candidate or referendum. This model asserts that voters make choices that are in line with their ideals and philosophies even when they are not able to articulate why they made such a choice (Lodge, Steenbergen, & Brau, 1995). According to the Online Model, a voter, over the course of a campaign or lifetime, will be exposed to images and messages about a given candidate, issue, or party. That voter will then associate and remember an overall impression of that candidate, issue, or party rather than remembering specifically why they feel a certain way about them. Professional campaigns attempt to control these impressions through selective campaign events and advertising, hoping that a person will add a positive tick to the online tally the voter is subconsciously maintaining. Additionally, by providing these impressions to voters, a campaign lowers the information costs associated with that campaign for a voter. Information costs, however, vary depending on the type and prominence of a given election. Top-of-the-ticket races, during a general election, tend to have the lowest information costs to voters. This is because candidates, campaigns, and special interests, flood the market with ?educational? information, helping to lower the information costs associated with a voter making an electoral decision during a campaign. Additionally, these general election contests are usually held between the two major parties, which provides the easiest and most accessible voting cue that voters use when deciding on who 7 to vote for in those elections?the party of a given candidate. These impressions can come from many different places such as the news media, talk around the water cooler, and even the campaigns themselves. Overall, impressions serve to reduce the information costs associated with a voter making a decision during a given campaign. However, there are other ways that voters simplify the decision-making process during a campaign. Heuristics such as incumbency or a candidate?s party allow voters to make vote choices based on a working knowledge of general policies and principles that the voter perceives a party to represent (Herrnson, 2004). Flanigan and Zingale (2002) indicated that if persuasion does have an impact on vote choice, the influence is probably small. Therefore, during races such as presidential elections and other races heavily associated with party politics, the persuasion effect is likely the smallest. In fact, Goldstein argued that focusing on presidential general elections is the least likely place researchers might find campaign effects (Goldstein & Ridout, 2004). Barker argued that focusing on these general-election campaigns might not allow researchers to see the impact that persuasion has on voters because the impact of partisan identification could overpower whatever small impact persuasion has on vote choice (Barker, 2005). Barker suggested that researchers focus their attention on nominating contests to remove the effect of partisan labels. Thus, researchers would be better able to see any impact persuasion might have on the outcome of an election (Barker). Johnson-Cartee and Copeland (1997) also believed that the effects of campaign advertising would likely be strongest in races for positions lower on the ballot, local races, party primary elections, and nonpartisan elections. This idea was reinforced by research by Luks et al. (2003), which indicated that campaign influence is strongest among third-party candidates who 8 are not part of the existing partisan structure. In other words, campaign influence is strongest for those individuals who do not have a partisan cue to guide their decision making during a campaign. Referendum campaigns are one way researchers can study campaign effects while still reducing or eliminating the significant effects that partisanship and incumbency have on the outcome of elections. Direct democracy, as opposed to representative democracy, is the system whereby citizens have a direct influence over public policy. The referendum electoral process is one of the primary ways voters participate in direct democracy, as a referendum allows voters to directly approve or reject some policy issue by casting their ballot in an election. These elections tend to be high information-cost elections that offer few cues to help voters shorten the decision-making process regarding the policies that are being debated (Bowler & Donovan, 1998; Bowler, Donovan, & Tolbert, 1998). Indeed, the partisanship shortcut used to simplify vote choice is nearly always missing when casting a ballot in a referendum election. And referendums have another important and distinct quality: Incumbency is not a direct factor (although it could be argued that the status quo is the incumbent and, therefore, voting against a policy is easier to do when a voter feels they lack enough information to make a more informed vote choice (Bowler & Donovan, 1994)). Because of qualities such as these, researchers might be better able to determine the impact of campaigns because the extremely powerful outcome predictor of partisanship has been removed from the campaign-effects equation during referendum elections. The huge amount of money spent on political campaigns and referendums in this country offers much anecdotal evidence that professionals and candidates alike believe 9 that campaigns have a significant impact on the outcome of elections. In the 2008 presidential race, for example, the candidates vying for the Republican and Democrat nominations raised nearly $1 billion trying to achieve their goals. The losing primary campaigns as a group raised $650 million (OpenSecrets, 2008) and the winners raised at least $350 million in their efforts to win the right to represent their respective parties in the November General Election. During these and other political campaigns, various strategists, pollsters, and candidates spend huge amounts of money and time trying to develop the messages that will be employed during the campaign. Implicit in this behavior is the assumption that a campaign?s message has some impact on either the mobilization, the decisions, or both for voters and ultimately the outcome of those elections (Herrnson, 2004). Finkel wrote that the central question in the study of electoral behavior is the effect campaign events and campaign stimuli have on vote choice (Finkel & Schrott, 1995). Barker called such attempts at persuasion the ?DNA of our Democratic politics? (Barker, 2005). Clearly, the question of whether persuasive messages have an effect on electoral behavior is a topic worthy of further academic investigation. Fortunately, over the past 20 years, researchers have slowly begun refocusing to better understand the impact campaigns have on the outcome of elections (Ansolabehere & Iyengar, 1997; Ansolabehere, Iyengar, Simon, & Valentino, 1994; Barker; Bartels, 1993; Cobb & Kuklinski, 1997; Gelman & King, 1993; Goldstein & Ridout, 2002; Holbrook, 1996; Kuklinski, 2001; Mutz, Sniderman, & Brody, 1996; Shaw, 1999; Zaller, 1992). To that point, Cobb argued that some of the most influential works in American politics over recent years, such as Zaller?s 1992 text The Nature and Origins of Mass 10 Opinion and Sniderman, Brodey, and Tetlock?s 1991 text (as cited in Cobb & Kuklinski, 1997) Reasoning and Choice, have focused on the idea of political persuasion. However, several pieces have focused somewhat on presidential campaigns or general electoral campaigns (Barker, 2005; Lau, Sigelman, Heldman, & Babbitt, 1999; Shaw, 1999). While there is still no research consensus on the impact of persuasion during a political campaign, some researchers continue to find that campaigns have ?minimal effects? (Barker; Eldersveld, 1956; Eldersveld & Dodge, 1954; Gosnell, 1926, 1927; Kramer, 1970). Perhaps the belief that campaigns can have an impact on vote choice has its origins in the marketing arena (Scammell, 1999). Every Saturday morning, as parents of children will attest, makers of cereal and toys are some of the primary advertisers during morning cartoon shows. Those surfing the Internet, driving, or performing nearly any other human activity will have noticed the steady deluge of marketing that characterizes modern American life. And anyone paying attention to the media discussion preceding the Super Bowl each year would know that mass advertising can be incredibly expensive. If marketing did not have an impact on consumer decisions, the logic goes, the ?free market? would cause those advertisers to stop buying expensive advertising. The idea of improving an advertisement?s efficiency by being able to directly deliver an advertising message to a consumer is a popular one in the marketing world. Arguably, the primary reason for the growth in direct marketing is the availability of massive databases of consumer information that have been built over the years. These databases allow advertisers the ability to target their messages to consumers who are most likely to respond to that advertisement (Bult & Wansbeek, 1995). Direct mail is the 11 primary means of delivering these targeted direct-marketing advertisements (Nash, 1984). In fact, Nash argued that the effectiveness of a direct mail campaign depends on the ?offer, communication elements (such as graphic design), the timing or sequence of these communications, and the list of customers to be targeted? (Bult & Wansbeek; Nash). Further, Bult and Wansbeek wrote that the direct mail message must ?reach the correct list to be effective? (Bult & Wansbeek). This practice of ?targeting,? which amounts to selecting the individuals who will receive a specific message, has become an increasingly common practice during political campaigns with the implied belief that these direct-marketing techniques are working for those who spend the money to implement them in a campaign (Herrnson, 2004). This type of campaign targeting involves identifying political beliefs of different groups and delivering different messages to those groups based on the likelihood that a group will respond in a certain way to those messages (Herrnson). Herrnson articulated that these targeting procedures begin with an initial campaign benchmark poll, which helps a campaign better understand the issue positions, partisanship leaning, and voting preferences of the voters in a given district (Herrnson). Several authors have indicated that if researchers are to determine what effect campaigns and campaign targeting have on the outcome of elections, researchers will need to conduct more field experiments (Goldstein & Ridout, 2004; Green & Gerber, 2000, 2002a; Johnson-Cartee & Copeland, 1997). As an answer to the call by Gerber and Green, political science researchers have begun conducting field experiments, yet those experiments tend to focus on the idea of mobilization (Ansolabehere & Iyengar, 1997; Ansolabehere et al., 1994; Gerber & Green, 2000b, 2001a, 2001b; Gerber, Green, & 12 Shachar, 2003; Green, 2004; Green & Gerber, 2001, 2004; Green, Gerber, & Nickerson, 2003; Green & Shachar, 2000; Niven, 2002, 2004). The research reported in this dissertation is an attempt to extend the study of experimental research of campaign effects such as mobilization into the arena of vote choice. Campaign Field Experiments There are several reasons why field experiments have not typically been a research method used by campaign researchers. First, experiments such as the one addressed in this research are typically expensive to conduct (Green & Gerber, 2002a; Johnson-Cartee & Copeland, 1997). Creating experiments that would allow a researcher to operate in a real-world environment while controlling as many variables as possible are costly to undertake and they normally leave a researcher funding the research personally, finding an academic institution to serve as a sponsor, or finding a campaign or professional organization to fund the research. Second, the need for neutrality required by granting institutions may also have played a role in the lack of research associated with political-campaign messages (Freedman, Franz, & Goldstein, 2004). This need for neutrality might also explain why research has focused on more benign topics such as mobilizing all voters or correlational analyses of demographic variables related to participation or vote choice. Finally, campaign practitioners tend to be private and have little desire to educate the masses on the techniques they use to win elections. In the past, academic research has had a difficult time studying and ultimately finding support for the central belief of campaign managers and politicians?that campaign messages have an impact on the outcome of an election. The research that 13 comprises the primary portion of this dissertation is an attempt to help bridge the gap in knowledge that currently exists between the professional world of politics and academic research. It does so by using a campaign-sanctioned field experiment to determine the extent campaign messages influence the decisions of voters during a live campaign. In fact, this research has the added benefit of helping to make transparent the methods professional campaign consultants use to manipulate the outcome of elections. Some researchers have argued that such manipulation might erode the democratic process by removing public policy from the true underlying preferences of the electorate (Bowler & Donovan, 2002; Broder, 2000; Magleby, 1984; Schrag, 1998; Smith, 1998). Some authors have shown that it has been difficult for researchers to collect data on what consultants do during a campaign (Petracca, 1989; Petracca & Wiercioch, 1998). This research also attempts to explicate an understanding of the campaign consulting world. Additionally, this research attempts to answer the call for more field experiments being conducted during live campaigns (Green & Gerber, 2000; Imai, 2005; Johnson-Cartee & Copeland, 1997; Niven, 2004). Finally, this research adds to the limited research on political persuasion during referendum campaigns. The two primary research questions and subsequent hypotheses that are the focus of the research being conducted for this dissertation are, Q1. Do the type and frequency of targeted-campaign messages result in increasing voters? intention to vote? H1a: The type of targeted-campaign messages increases the likelihood that potential voters indicate their affirmative intention to vote. 14 H1b: The frequency of targeted-campaign messages increases the likelihood that potential voters indicate their affirmative intention to vote. Q2. Do the type and frequency of targeted-campaign messages influence the likelihood that potential voters vote in favor of the targeted-message issue? H2a: The type of targeted-campaign messages increases the likelihood that potential voters vote in favor of the targeted-message issue. H2b: The frequency of targeted-campaign messages increases the likelihood that potential voters vote in favor of the targeted-message issue. As mentioned above, some past research has found that campaign communication has an impact on mobilization, and now researchers are beginning to expand the validity of their research by focusing on different types of messages, different election levels, different types of campaigns, and different subgroups of voters. Additionally, by delivering a message that has been shown, through campaign research, to cause voters to change their vote choice, one would expect to see support for Q2 as this is an assumption most candidates and campaigns make about the impact they have on voters. The Experiment and Variables The two questions and subsequent hypotheses reflect the independent variables and dependent variables. Table 1 summarizes the primary variables used in this study. The independent variables were types of message and frequency of messages. There were two types with one message being designed specifically for the target population of this experiment and the other message being related to the voting issue but generic in nature. The frequency of message contact ranged from one group receiving no contact (control 15 group) to groups receiving up to three combinations of the types of messages. In all, there were five experimental groups and a control group, as listed in Table 2. Table 1 Primary Variables Variable Type Label Description Independent Experimental Group There are six groups each with a diferent treatment or lack of treatment used to determine if mesage targeting and contact frequency impact the Dependent Variables. Dependent Vote Intention Questionaire item Q2 asked, ?As you may know, Alabamians wil go to the pols on September 9th and vote on the tax and acountability package known as Amendment 1. As things now stand, wil you likely be voting in this election or wil you not likely be voting?? Dependent Vote Choice Questionaire item Q3 asked, ?If this election were held today, would you likely vote for or against Amendment 1?? Table 2 Experimental Groups Group Mesage type and frequency A Control group B 1 targeted direct mail mesage C 2 targeted direct mail mesages D 1 targeted direct mail mesage and 1 generic direct mail mesage E 2 targeted direct mail mesages and 1 targeted telephone mesage F 1 generic direct mail mesage The dependent variables were preelection questionnaire items collected through a telephone survey that asks potential voters if they intend to vote, as well as if they favor the issue that is the subject of the messaging. Additional questionnaire items (see Appendix A) provide information on voter demographics as well as several items directly related to the two dependent variables of interest. 16 The survey question that was used to create the Vote Intention dependent variable is Question 2: ?As you may know, Alabamians will go to the polls on September 9th and vote on the tax and accountability package known as Amendment One. As things now stand, will you likely be voting in this election or will you not likely be voting?? The possible responses to this question were: Likely Voting, Not Likely Voting, and Not Sure/Refused The survey question that was used to create the Vote Choice dependent variable was Question 3: ?If this election were held today, would you likely vote for or against Amendment One?? The possible responses for this question were: Definitely Vote For, Probably Vote For, Definitely Vote Against, Probably Vote Against, and Not Sure/Refused. Amendment One Field Setting In May of 2003, less than 4 months after taking office following a contentious election against former Governor Don Siegelman (Democrat), Alabama Governor Bob Riley (Republican) proposed a tax plan that would generate more than $1 billion of new revenue (Bernstein & Seroka, 2004). There were several ideas behind the proposed revenue increase. First, the proposal tried to address expected budget shortfalls for the immediate future in Alabama?s state budget while making Alabama?s tax system fairer. Second, some portion of the revenue would have been used to fund a major college- scholarship program for high school students in Alabama (a similar scholarship proposal failed several years earlier under a Siegelman administration because the funding would have come from a statewide lottery that the voters rejected). Finally, the revenue 17 generated from the Riley Tax Plan would have produced a revenue surplus that would have allowed Alabama to build a strong contingency fund. On June 1, 2003, the Alabama legislature passed legislation that allowed for a vote on Riley?s Tax Plan to take place on September 9, 2003 (Bernstein & Seroka). No other statewide items were considered on the ballot. Immediately after the legislature passed the legislation allowing a vote, the campaigns for and against the amendment began. A public-opinion survey conducted in June 2003 indicated that support for the amendment was ?69% among those that expressed an opinion? (Bernstein & Seroka, 2004, p.1). During the course of the campaign that followed, public opinion steadily turned against the amendment and the final vote tallied more than two-thirds of those voting rejecting the amendment (Bernstein & Seroka). In theory, because the issues that surrounded the campaign were not issues that had specifically been on the public agenda, it was expected that the ability of the campaign to influence vote choice would likely be higher in this campaign than in another campaign in which voters had already determined their vote choice either directly or indirectly in the past (Kriesi, 2002). Study Method This dissertation uses secondary data from a field experiment conducted during the 2003 campaign for the revenue-increasing amendment to Alabama?s Constitution? Amendment One. The experiment was conducted during a live referendum campaign by a local political-consulting firm that allowed the use of their data for this dissertation. Beginning in April 2003, the consulting firm conducted a benchmark survey that showed 18 people aged 65 and older tended not to support Amendment One. However, during that same survey, senior respondents tended to change position from being against the amendment to supporting the amendment when told that seniors would not pay any additional property taxes under the new revenue system. Because the ?no additional property tax? message appeared to change voter preferences, the campaign decided to conduct an experiment to gauge the impact that targeted messages would have on whether or not voters intended to vote and whether or not voters would support the amendment with their vote. Data Analysis The primary statistic employed was chi-square analysis. Chi-square analysis allows for making comparisons among groups on the dependent variables of interest. For these data, two chi-square analyses were conducted. One dealt with whether there were differences among the six groups in their intention to vote and a second analysis with whether they would vote in favor of the issue described in the targeted message. Additionally, several supplemental analyses were conducted with the objective of probing the data further. These included further chi-square analyses of other potential dependent variables from the questionnaire and logistic regression. Where the results of these analyses added a better understanding of the chi-square results, they are reported. The .05 level of statistical significance was used for all of the statistical tests. Organization of Chapters This chapter has provided the foundation for the future chapters. A brief broad discussion of the literature and philosophical ideas that guided this research as well as a 19 brief discussion surrounding the experiment itself should enable a better understanding of the remainder of this text. The next chapter will provide a detailed discussion and review of the existing experimental field research that primarily has been conducted over the past 10 years as it relates to voter mobilization and vote choice during referendum campaigns. Additionally, it will clarify the gaps in the research that this dissertation is attempting to help fill. Chapter 3 provides details of the study?s methodology, including descriptions of the research design, sampling, materials, instrumentation, and data- collection procedures, while chapter 4 presents additional discussion of the analyses that were employed and the results of those analyses performed using the experimental data. Finally, chapter 5 provides a discussion of the findings based on the analyses conducted in chapter 4. Further, chapter 5 discusses the limitations of the research in this dissertation as well as providing guidance for future areas of research. 20 CHAPTER 2: BACKGROUND/LITERATURE REVIEW Introduction America has more elections than any other country on earth. Generally, Americans have the ability, in one decade, to vote in more elections than the Britons, Germans, or Japanese have in their entire lifetimes (Green & Gerber, 2004). Americans can participate at many different electoral levels, from local elections such as mayor and city council to state elections such as governor and U.S. senator to the national level when voting for the president. Voters have the options to vote on candidates that impact public policy for that voter, and voters also have the opportunity to vote on initiatives and referendums to directly impact public policy. Each of these elections typically involves a campaign that precedes the vote by the people; many of those campaigns spend large sums of money in an effort to be victorious on election day. The money is spent with the expectation that campaigns can and do influence an individual?s propensity to vote and to vote a certain way in a given election. However, researchers have yet to fully appreciate and test for the ways in which campaigns can influence turnout and vote choice. That is not to say that the discipline knows nothing, because over the past decade the discipline has made significant progress in determining how and when campaigns can influence voters. Over the past decade, thanks primarily to Gerber and Green?s trumpeting of the randomized-field-experiment methodology for campaign research, researchers have taken 21 up the challenge to extend the understanding of the area of campaign effects. The literature, reviewed for this chapter, represents a total of 57 field experiments related to either vote choice, turnout, or both (see Appendix B). Each section in this chapter will focus on the relevant literature related to a specific variable that has been manipulated in the current study. Several things are clear from the literature that has been reviewed. First, researchers are attempting to improve the efficiency of the campaign industry by working in conjunction with those professionals involved in the business of campaigns and elections. As such, concessions have to be made by researchers to gain the trust and commitment of the sponsoring organizations that are helping to facilitate the research (Green & Gerber, 2003). For example, an opportunity might present itself to study a campaign in the field but the campaign chooses to target specific messages to specific voters, as was the case with the research being analyzed for the present study. In such an instance, the researcher may not be able to fully control the message, mode of delivery, or the population, but there are many other aspects of the design that can be controlled; the key controllable item in experiments is randomization (Green & Gerber, 2002b; 2003). Although the researcher might not be able to manipulate the specific message in working with the campaign, the researcher may be able to manipulate the time the message is delivered or the frequency with which that message is delivered to a given voter. Ultimately, the goal is for the researcher to make the best of the research opportunities that exist (Green & Gerber, 2003). Conducting an experiment in a natural setting provides the researcher with research that is applicable to the real world (Green & Gerber, 2003). However, working 22 in a natural environment typically prevents the researcher from controlling all aspects of the experiment (Green & Gerber, 2003). For example, if a researcher was going to conduct an experiment during a gubernatorial campaign that had two female candidates it would not be possible to study the impact of having a male candidate in that race. Similarly, researchers working with a campaign to conduct field experiments will not have the ability to control all aspects of the experiment, which complicates the isolation of variables to be studied, the review of the research, and the ability to generalize from the results. However, by conducting the research in the field, researchers are better able to generalize the results than had the research been conducted in a laboratory (Green & Gerber, 2003). It is the real-world application that makes the tradeoffs worthwhile. Additionally, it is the process of working with live campaigns that helps researchers begin to extend findings across the spectrum of variables that need to be considered when looking into campaign effects. For example, mobilization or persuasion effects might differ across various groups of people, messages, the levels of election, biased or unbiased messages, partisan or nonpartisan campaigns, positive or negative messages, and geography. Additionally, researchers need to fully consider message frequency, synergy, professionalism of message delivery, the authority of the person or group delivering the message, and many other known and unknown variables that might make a difference in learning about persuasion and mobilization effects. This is no easy task, but the discipline and the industry have and can continue to work together to discover the circumstances under which campaigns can have effects (Green & Gerber, 2002b). 23 In the end, campaign-effects research is an attempt to study human behavior. There is controversy among social scientists about the feasibility of designing models that can predict human behavior. Arguably, researchers cannot uncover the particular causes of a voter voting or voting a particular way. However, it is through the combination and control of the multitude of variables discussed above that the collective search for better understanding of the causes of particular voting behaviors among particular voters might be successful. The referendum research that is the focus of this dissertation adds one small piece of information to aid in understanding voting-behavior motivation. This chapter first provides a discussion related to the scope of the review. Then it focuses on what is generally known or has been studied in the literature about the variables that were controlled during the biased-message-referendum experiment that is the basis of this dissertation. Below, there are discussions about what researchers have studied in the field relative to persuasion and mobilization messages delivered via direct mail or telephone calls as they related to referendums, message frequency, message synergy, targeted messages, and senior voters. Each section largely provides evidence that there are contradictory findings among the reported results and a lack of replication in the body of literature that will need to be honed over time to determine the true effects that campaigns have on the outcomes of elections. Additionally, some of the variables that were controlled in this experiment lack definition; this chapter will offer definitions that may further guide the discipline toward campaign-effects information. 24 Scope of Review The review of literature for this dissertation began in October 2005 and continued through October 2008. Every effort was made to include all available and appropriate literature dealing specifically with randomized field experiments as they related to campaigns and elections in the United States. Additional areas were researched to provide background and support for this and other areas in this dissertation. In preparation for writing this dissertation and specifically writing this chapter, research was done through many different avenues and mediums in an effort to provide the most complete and relevant information that is currently available. This included reviewing appropriate texts such as books, book chapters, journal articles, conference proceedings, and others? dissertations. The appropriate texts where collected through many hours of online research using specialized database collections as well as many hours of research in the libraries provided at Auburn University and Auburn University? Montgomery. The dynamic process of reviewing the relevant literature is somewhat lost in the discussion of the review methods that were used in this dissertation. However, it is worth noting that the terms, databases, authors, and journals discussed below were constructed throughout the research and writing process. As one text was reviewed, other sets of terms, citations, journals, and texts grew from that specific piece of research. For particularly important and relevant citations, a review of the literature that cited those specific items was also conducted. In this respect the research was conducted in two directions: the traditional forward method (terms, journals, databases, authors) and backward (once a source was found to be of significance to this dissertation, a review was 25 completed of other sources that cited that new primary source). Both methods produced significant and relevant documents. The primary search terms both used individually or in combination with other appropriate terms were campaign effects, campaign message, campaigns, candidate choice, canvassing, direct democracy, direct mail, door-to-door, elections, experiment, field experiment, get out the vote (GOTV), leaflet, messaging, mobilization, mobilization effects, nonpartisan, partisan, persuasion, phone bank, political advertising, postelection survey, preelection survey, preference effects, randomized experiment, randomized field experiment, referendum, seniors, targeted marketing, targeted message, targeting, turnout, turnout effects, verified vote, volunteer calls, vote choice, vote intention, and vote preference. The primary databases where these terms were used included not only the Auburn Library System and other libraries around the world, but also the following database collections: Academic OneFile, Academic Search Premier, Expanded Academic ASAP, Journal Citation Reports, JSTOR, LexisNexis Academic, Project Muse, Social Explorer, Web of Science, PsycINFO, Dissertation Abstracts, SAGE Journals Online, and the Social Sciences Citation Index. In an effort to avoid missing relevant literature by simply relying on search terms, the abstracts and titles were reviewed for the past 5 years in the following key journals or conference proceedings: American Demographics, American Journal of Political Science, American Political Science Review, American Politics Research, The Annals of the American Academy of Political and Social Science, Annual Meeting of the American Political Science Association, Annual Meeting of the Southern Political Science Association, British Journal of Political Science, Electoral Studies, 26 Journal of Marketing Research, Journal of Personality and Social Psychology, Journal of Politics, Legislative Studies Quarterly, Midwestern Political Science Association Conference Papers, Personality and Social Psychology Bulletin, Political Analysis, Political Behavior, Political Communication, Political Psychology, Political Research Quarterly, Public Opinion Quarterly, and the Quarterly Journal of Political Science. In the end, the research reported in this literature review represents the major and minor academic research conducted since Gosnell?s 1924 (1927) experiment in voter stimulation. Over the years, research has both waxed and waned. Currently, thanks primarily to the works and guidance of Gerber and Green, experimental research into the study of campaign effects has seen a renaissance. While much has been learned in the past decade in the area of campaign effects, there are still many unanswered questions. The research that is the focus of this dissertation is an attempt to fill the gaps associated with the campaign-effects research. Experimental Methods Campaigns are primarily about persuading voters and turning voters out on election day (Herrnson, Campbell, Ezra, & Medvic, 2005). Political consultants and candidates feel that campaigns are able to do both of these things (Herrnson et al.; Johnson-Cartee & Copeland, 1997; Perlmutter, 1999; Petracca, 1989; Trent & Friedenberg, 2000). Typically, as election day nears, political campaigns focus on turning out the vote (Herrnson et al.). The efforts, resources, and energies that go into a campaign mobilizing voters are vast and it is easy to understand why campaigns would spend such a large amount of resources on mobilization: the person with the most votes wins an 27 election. For example, if the vast majority of the public supports one candidate over another, but they do not vote, the candidate with fewer supporters can win simply by turning out a larger percentage of their supporters than their opposition. In other words, a campaign?s primary purpose is to mobilize their supporters (Shea & Burton, 2006). In leading to the GOTV efforts on election day, campaigns often focus their efforts on increasing the number of people that actually support their candidate or issue (Herrnson et al., 2005). Television advertisements inundate viewers as major election days near and often those advertisements are meant to persuade or change voter?s opinions to support one side of an issue or another (Johnson-Cartee & Copeland, 2004). Celebrity endorsements and telephone calls are piped into potential voters? houses and the direct mail industry begins to produce millions of pieces of mail in an effort to change or sway the opinions of voters (Johnson-Cartee & Copeland, 1997). As with GOTV efforts, those campaigns try to be as efficient as possible with their limited resources: campaigns do not communicate with all citizens in their district, nor do they want to. Typically, campaigns try to focus their resources on those people who are most likely to vote. Political campaigns, in an effort to either mobilize or persuade voters, communicate with those voters. Gerber and Green (2000b) argued that voter turnout rates have declined over the past half century partly because campaigns increased their reliance on mass media and mass-marketing techniques such as radio advertisements, television advertisements, newspaper advertisements, direct mail, and telephone banks to communicate with voters, rather than the older more traditional party mechanisms used in the first half of the 20th century. Gerber and Green (2000b) believed it is the lack of 28 personalization and reduced intimacy that comes with using mass techniques that have become a staple for modern campaigns, which, over time, led to reduced voter turnout. Researchers over the years have used two primary techniques to try to determine the impacts campaigns have on voting behavior. In the early and middle part of the 20th century, researchers focused on field experiments (Eldersveld, 1956; Eldersveld & Dodge, 1954; Gosnell, 1926, 1927). Then, during the later half of the 20th century, as survey research grew in prominence in association with political campaigns, research began to use statistical analysis on survey data to try to ferret out the effects campaigns have on voting behavior (Green & Gerber, 2003). Many of those observational-based survey techniques are prominent throughout the literature; most used some form of regression analysis such as Ordinary Least Squares, Logistic, and Probit regression (Green & Gerber, 2003). While these techniques are sound and have provided countless data sets for researcher to use, and even though survey research-sampling methodologies continue to improve, evolving from random digit dialing to voter-list sampling, these observational data tend to suffer from three significant drawbacks. First, when researchers try to explain voter behavior such as mobilization and persuasion, the survey data generally rely on self-reported independent and dependent variables that are then analyzed using techniques that do not allow a researcher to fully determine and ensure that one variable causes some change in another variable (Vavreck, 2007). Second, the self-reported data relies on the memory of the respondent who may or may not be completely accurate (Vavreck). Third, campaigns tend to focus their efforts on voters who are more likely to vote than not and often more likely to support their side of an issue than not (Arceneaux & Nickerson, 2005). If a 29 researcher looks at campaign communication to determine the impact of that communication on voting behavior, it is possible that the research would falsely conclude that there was an impact simply because campaigns are more likely to communicate with certain types of voters who are more likely to vote than other types of voters. Therefore the relationship that researcher sees would be spurious and not causal (Green & Gerber, 2003). Laboratory experiments offer researchers a methodology that can be used to determine causality, but external validity problems such as generalizability tend to reduce the value of such research as it relates to political campaigns (Green & Gerber, 2002b). Currently, field experiments tend to provide researchers with a strong methodology for determining causal assertions while offering a higher level of generalizability than is offered in laboratory experiments. Field experiments afford a researcher the opportunity to conduct research in a real-world situation while still maintaining control as it relates to the experimental treatments (Green & Gerber, 2003). By maintaining control over who receives what treatments the researcher is able to make a stronger argument that some treatment causes some behavior. By randomizing which participants receive particular treatments, a researcher?s ability to perceive cause and effect becomes stronger and provides the best opportunity to learn how campaigns or organizations influence voter behavior (Green & Gerber, 2003). It is these randomized field experiments that are addressed in the bulk of the discussion in this chapter. In looking at campaign effects, early voter-behavior researchers studied both mobilization effects and persuasion effects (Eldersveld & Dodge, 1954; Blydenburgh, 1971; Miller & Robyn, 1975; Bositis, Baer, & Miller, 1985). That is to say that 30 researchers looked at the impact campaigns can have on turning voters out on election day and whether campaigns can change a voter?s preference from one voting position to another. Those researchers found that there was a strong mobilization effect but no persuasion effect, so research focused on the study of mobilization. As mentioned above, during the later half of the 20th century, researchers left the experimental tradition for survey-based observational techniques (Green & Gerber, 2003) until Gerber and Green began effectively arguing and demonstrating that political scientists needed to return to their voter-behavior research roots. The discipline has seen a resurgence in field experimental research over the past decade. In the past decade field experimental research has focused on mobilization effects and what specific techniques and messages are most efficient at mobilizing voters. These field experiments have looked at both politically biased and nonbiased messages delivered via communication modes such as door-to-door canvassing, telephone calls made by volunteers and paid phone-bank operations, prerecorded calls, leafletting, U.S. postal mail, email, and more recently, text messaging. Collectively, these experiments represent many different election types (candidate, referendum, initiative, etc.), election times (primary election, general election, special election, etc.), and election levels (presidential, gubernatorial, mayoral, etc.) as well as representing a diverse set of geography (local, statewide, national, etc.). (A list of the experiments reviewed for this dissertation is in Appendix B.) As mentioned above, the external validity of field experiments is an issue of primary concern. The primary way of improving external validity of those experiments is replication across different times and places, while simultaneously field-experiment 31 researchers in the field of voting behavior continue to replicate and expand the literature in an effort to improve the discipline?s knowledge of if, how, and why, campaigns impact particular voters in particular ways. They query why those players involved in campaigns and elections continue to spend such huge amounts of money on techniques that may or may not have the desired impact on voter behavior. Referendum America presents more electoral opportunities to voters than any other country in the world (Green & Gerber, 2004). The opportunities range from the local level to the national level and allow voters the opportunity to vote on numerous candidates as well as participate in direct democratic elections (Green & Gerber, 2004). These electoral opportunities also present many opportunities for researchers to investigate campaign effects in the field through experimental methods (Green & Gerber, 2000). However, the large number of candidates, ballot combinations, and election calendars, complicate the researcher?s ability to determine where to look for such effects. This section argues that by focusing on nonpartisan referendums, researchers have an opportunity to control for a number of variables such as voter partisan identification that would be more difficult to control in other elections and thereby increase the likelihood of uncovering a potentially small campaign effect. According to Rational Choice Theory, it is the voter?s responsibility to search out information regarding each electoral choice and then cast a ballot based on consideration of all available information (Green & Shapiro, 1994). Some have argued that this places an unrealistic burden on voters, given the sheer number of electoral opportunities that a 32 voter living in America has during a lifetime (Flanigan & Zingale, 2002). The vast quantity of elections that take place compounds the voter?s difficulty to collect relevant information, review that information, and make voting decisions, and it is no wonder that voters use psychological shortcuts called heuristics to reduce the informational costs associated with making those decisions (Herrnson et al., 2005). In fact, by using these shortcuts, it has been shown that voters are better able to approximate the voting decisions they would make if they did have the resources to seek and analyze more information (Bowler & Donovan, 2002). However, the primary shortcut that voters use is their partisan identification relative to the candidate or issue that is being voted on in a given election (Dalton, 2002). A voter?s partisanship is typically very stable over their lifetime (Dalton). This partisanship stability forms the basis of Converse?s (1966) concept of the Normal Vote. The Normal Vote is the vote that would take place based on partisanship if all other factors were held constant (Converse; Dalton). In a typical electoral contest, campaigns attempt to inject other factors into the voter-decision process in an attempt to affect the outcome of an election (Dalton). Because partisanship is the primary shortcut that voters use, it becomes more difficult for a campaign to overcome the strength of party identification when trying to influence a voter during an election. In fact, the less information a voter has about a given ballot choice, the stronger the role partisanship plays in determining that choice (Flanigan & Zingale, 2002). For example, down-ballot partisan elections would see a higher partisanship-based set of voter decisions than a partisan gubernatorial election (Flanigan & Zingale). 33 Similarly, an election that has no partisan cues such as a primary election or referendum election increases the information costs associated with a voter making relevant decisions about a given election (Magleby, 1984). Voters also change their decisions more during a referendum campaign than they do in a candidate campaign (Cronin, 1999). Because of the reduced influence party identification has on voter decisions, it has been argued that these high-cost elections increase the opportunity of a campaign to influence the decisions a voter makes regarding that election (Magleby). Further, others have argued that elections that lack party identification provide a good opportunity to conduct experiments on campaign effects (Barker, 2005). In reviewing the campaign field-experiment literature, eight field experiments were identified that took place during and focused on referendum campaigns. They took place in Ann Arbor, MI, San Francisco, CA, East Bay, CA, Fresno, CA, Maricopa County, CA, Kansas City, MO, and Los Angeles, CA. Specifically, these eight experiments considered the impacts that campaign communications have on mobilization or vote choice of voters. These eight experiments will become the basis for subsequent sections below. Partisanship Redefined In the experimental research reviewed for this dissertation the terms partisan and nonpartisan have been used to describe both the types of messages used in an experiment as well as the campaign environment in which an experiment takes place. This dual use of the terms partisan and nonpartisan leaves room for confusion that makes reviewing and advancing the literature somewhat more complicated than it might be if the research 34 community used separate terms for message type and election environment. This section advocates for the use of new terms to help better clarify and guide the electoral research of the future. By separating the terms used to describe message type from those used to describe the election environment, perhaps researchers can better advance electoral field experiments. This section supports the idea that message types should be considered either biased or unbiased as it relates to the candidate, campaign, or party that is involved in a specific election. A biased message would be one that supports one side of an election while an unbiased message would be one that does not offer support for one side in an election contest over another. This would then leave the terms partisan and nonpartisan to describe the campaign type that is the focus of the experimental research. For example, a general election for United State Senate would be a partisan election because voters are able to use partisan cues to guide their decisions, but in a primary election partisanship is removed as a cue because all candidates in a given primary election are from the same party. The use of the terms partisan and nonpartisan to describe a specific message are easy to understand in certain instances but not in others. For example, during their seminal experiments in New Haven and West Haven, CT, Gerber and Green distributed ?nonpartisan? messages in an attempt to mobilize voters without supporting or opposing a specific candidate or campaign, however, there were partisan races on the ballot at that time (Gerber & Green, 1999, 2000a, 2000b, 2001a, 2005a; Gerber, Green & Shachar, 2003). Similarly easy to understand is the use of the term partisan to describe the messages sent in support of the two Democrat incumbents state legislators running for reelection in the New Jersey 1999 General Election experiment (Gerber, Green, & Green, 35 2003). However, the use of the terms partisan and nonpartisan become somewhat confusing when used in other situations. It is easy to see that the term partisan message would apply to a message in support of a partisan candidate in a general election. However, consider a hypothetical experiment conducted during a primary election in which the experimental messages supported one candidate over another. This was the case in the Gerber experiment conducted during the 2002 Connecticut Republican congressional primary (Gerber, 2004). In that experiment, messages in support of one Republican candidate over another were distributed as part of the experiment. While the message was in support of a candidate that had chosen to label himself as a Republican, partisanship was less a factor in the election because the other candidate was also a self-labeled Republican. In fact, one of the major reasons to conduct experiments during partisan primaries is that the partisan label typically has little or no value to voters because both or all candidates in that election are from the same party. But, the candidates are in fact members of a party, and they typically distribute messages that support the party line. In these cases, the term biased might better apply than the term partisan to describe these experimental messages. Along those same lines, consider a hypothetical electoral experiment that distributed messages during a referendum campaign that was partisan in nature? That was the case for McNulty?s experiment conducted in the San Francisco area during the 2003 recall election of then Democratic Governor Gray Davis (McNulty, 2005a, 2005b). The messages distributed to experimental subjects were partisan in that they supported the rejection of the recall initiative of a Democratic governor. However, the use of the term becomes clouded if an experiment is conducted during a nonpartisan referendum but the 36 messages distributed are in support of one side or the other as was the case in McNulty?s 2002 San Francisco experiment dealing with the municipal power supply (McNulty, 2005a, 2005b). While it is unclear based on a review of the literature regarding that experiment, it is possible that there was a clear partisan delineation in that election. However, it might still cause confusion in other experiments conducted under less partisan circumstances that are done in support of or against one side of an initiative/referendum or the other. Also, consider a hypothetical electoral experiment that distributed an unbiased message during a general election by a group comprised of members that typically support one party? This was the case during the NAACP experiment conducted by Green during the 2000 general election (Green, 2004). In that experiment, messages were unbiased in nature in that they simply encouraged those contacted to go and vote on election day, yet Green indicated that the messages were nonpartisan in nature. However, they were partisan, for example, in that for a portion of the experiment, they used a recording of former Democratic President Bill Clinton to encourage NAACP members to go and vote on election day. Additionally, the members of the NAACP are traditionally seen as a strong voting block for Democratic Candidates (Green). In this example, one could argue that the message was either partisan or not, but the message was clearly unbiased in that it did not ask voters to support one candidate over another. That is not to say that those contacted wouldn?t likely support Democrats if they voted, however, the message itself was not biased in one direction or the other. Similarly, it would be easy to envision, after having reviewed the relevant literature, an experiment being conducted whereby a party sends out an unbiased GOTV 37 message to their partisan supporters. In that case the sponsor of the message is a party and therefore arguably brings partisanship into the experiment; however, the message would be unbiased because it did not advocate for one side in the given election. The point to the partisan?nonpartisan discussion focuses on whether voters are able to use partisanship as a cue or shortcut when casting their ballot. Using the terms partisan or nonpartisan in such an experiment could cause unnecessary confusion for future researchers. Finally, consider a hypothetical experiment delivering an experimental message in support of or against a candidate for an election that is nonpartisan in nature? Many municipal elections in Alabama, for example, are nonpartisan, but it would be easy to see an experiment being conducted during one of those elections with a message that was biased for one candidate or another. Similarly, some state court candidates have to run for election and they run without a party label, but one could envision a time when biased experiments are conducted during those campaigns. In those instances, using the terms partisan and nonpartisan could cause unnecessary confusion that might be cleared up with the use of the terms biased and unbiased to describe the message type being delivered to the experimental participants. Perhaps the terms biased and unbiased are not the best terms to help describe the message types during campaign experiments. This is particularly true given the somewhat negative image that the term biased conjures when used in relation to academic research. If that is the case, perhaps other researchers will help develop a more appropriate set of terms, but this section should help provide support for the idea of separating the electoral environment from the message type delivered during a campaign. Perhaps in doing this, 38 researchers will be better able to expand and extend the randomized field experiment such that they would better understand when a particular experimental treatment is appropriate during a given election. Ultimately, the goal is to be able to generalize across message types (biased and unbiased), message medium (door-to-door, volunteer calls, prerecorded calls, direct mail, leafletting, etc.), message tone (negative or positive), campaign types (partisan and nonpartisan), election types (candidate, referendum, initiative, etc.), election times (primary, general, special, etc.), election levels (local, state, federal, etc.) and geographic experimental areas (precinct, city, county, state, congressional district, national, etc.). For the remainder of the discussion of the literature the terms biased and unbiased will be used to describe the messages used in a given experiment while the terms partisan and nonpartisan will be used to describe the electoral environment that was the focus of the experiment, when appropriate. Persuasion A political campaign, at its core, has several key purposes: primary among them is thought to be persuasion (Johnson-Cartee & Copeland, 1997; Mutz et al., 1996). Specifically, a political campaign can have an impact on the outcome of an election by manipulating a voter?s electoral vote choice (Johnson-Cartee & Copeland, 1997; Mutz et al.). In fact, Green and Gerber (2004) indicated that winning elections likely has a basis in a campaign?s ability to not only mobilize voters but also persuade some portion of the electorate to support or oppose one side in a given election. Surprisingly, a small percentage of the experiments reviewed for this dissertation considered vote choice as a dependent variable. Among the campaign field experiments reviewed for this 39 dissertation, the primary focus of the majority of the experiments was voter mobilization. This is likely because state voter files are easy to collect and analyze to determine if a voter actually voted in a given election. However, three experiments tested for persuasion effects during referendum campaigns. All three experiments found support for the persuasion effect but only one was conducted after the 1998 experimental resurgence in the campaign literature that inherently indicates further research is necessary to determine under what conditions a persuasion effect might exist. The first referendum field experiment to test for a persuasive effect took place during the 1953 Ann Arbor, MI municipal elections (Eldersveld & Dodge, 1954). During the 1953 Ann Arbor experiment, Eldersveld and Dodge tested the persuasive impact of biased messages sent both by mail and door-to-door canvassing. The messages supported the election of a nine-member commission to study and recommend modifications to the Ann Arbor municipal charter. The authors targeted individuals they knew to be undecided based on a prior survey conducted a year earlier on a similar issue. The delivered messages were crafted to be persuasive to the undecided group of voters, and the dependent variable of vote choice was collected by survey. Ultimately, the authors found that direct mail had a persuasion effect of 20% and that door-to-door canvassing had a persuasion effect of 43% (Eldersveld & Dodge). However, the authors included the door-to-door treatment households that were not contacted in the door-to-door control group, which is known to exaggerate the experimental effects of experiments using door- to-door canvassing or telephone calls as the treatment method (Gerber & Green, 2000b). The second referendum field experiment to test for a persuasion effect took place during the 1980 General Election in Carbondale, IL (Bositis et al., 1985). The 40 researchers, working with a local partisan committeeman, delivered two biased messages in a single mailed letter with the committeeman?s signature to all registered voters in a single precinct. The letter was meant to encourage support for both the Democratic candidate for the Jackson County state?s attorney position and to encourage voters to support a statewide referendum that would modify the multimember district system used to select state legislators. The experimental manipulation was the order of presentation of those two messages in the letter. The dependent variable of vote choice was collected using a survey and the authors found that, among individuals aged 40 and under, the order of the messages had a significant impact on the information levels possessed by voters regarding the referendum. Specifically, the authors found that the message in support of the referendum was most effective when delivered second in the letter. During further investigation, the authors found that the order ?recency? referendum message accounted for a statistically significant 12% of the persuasion effect on vote choice in support of the referendum among voters aged 40 and under while finding no statistically significant persuasion effect for the vote choice related to the state?s attorney candidate (Bositis et al.). However, it should be noted that the authors did not specifically target any one group with a message thought to be more persuasive to that group. Additionally, the experiment did not use a control group so the exact persuasive effects relative to an untreated population could not be determined from the experimental data. The third and most recent campaign field experiment to consider the impact of a treatment on vote choice of a referendum was conducted in Los Angeles, CA (Arceneaux & Nickerson, 2005). During the 2004 general election, California had two statewide propositions on the ballot. The first proposition was to relax the three-strikes criminal law 41 by requiring the third criminal offense be a violent offense to qualify a criminal for a life sentence. The second proposition was to require that large companies pay a minimum of 80% of their employees? health insurance. The experimental treatments consisted of a single biased message altering the tone between positive tone or negative tone that was delivered by door-to-door canvassing to voters who were identified earlier in the experiment to be supporters or undecided on the two propositions. The experimental treatments were conducted in the weeks leading up to the election. The messages were delivered by a nonprofit organization that focused on minority issues. The organization supported the two propositions that were on the ballot at that time. The authors found no statistically significant difference between the two different message tones related to a persuasion effect for either proposition. However, the biased negative message did significantly increase support for the three-strikes proposition by approximately 13% over the control group. It is unclear in the authors? research why the negative message related to the three-strike proposition was found to be significant relative to the control group while the other message findings were insignificant (Arceneaux & Nickerson). All three experiments tested and found some support for a persuasion effect at the local level based on survey research used to determine vote choice. The two earlier experiments indicated that direct mail could have an impact on vote choice (Bositis et al., 1985; Eldersveld & Dodge, 1954); both the 1954 and 2004 experiments considered the impact that door-to-door canvassing could have on vote choice and found support for a persuasion effect. In fact, all three experiments taken together seem to imply that for some groups a persuasion effect of +10% might exist during referendum campaigns. 42 Additionally, the two experiments with a control group to allow for direct comparison developed and delivered messages that were meant for a specific targeted audience. All three of these experiments were conducted at a local level of geography, which leaves open the possibility that a larger level of geography might have a different result on the outcome of the experiment. Also, none of the three experiments focused specifically on senior voters and none of the experiments tested the impact that a telephone call might have on vote choice during a referendum election. Finally, all three experiments were conducted with other items on the ballot at the time of the election, which could confound the impacts that persuasive messages might have on vote choice during referendum campaigns. Two of the experiments considered the impact that mailed messages have on vote choice (Bositis et al., 1985; Eldersveld & Dodge, 1954). When considering the impact that direct mail might have on referendum vote choice, the literature seems to indicate that a persuasion effect might exist. However, the two experiments discussed above that tested messages delivered via mail were conducted prior to 1981, and it is possible that the changing times and increased uses of marketing direct mail have diminished the effectiveness that direct mail has on vote choice during referendum campaigns. Ultimately, while providing support for the idea that a persuasion effect exists during referendum campaigns, these three experiments represent a very small number of experiments that will likely need to be conducted related to the study of persuasion in live campaigns. It is still unclear under what circumstances a group might be persuaded to vote a particular way during a referendum campaign. 43 Mobilization Ultimately, an election is decided by voters casting votes. While a campaign can spend a huge amount of time, energy, and money trying to convince a potential voter to support one candidate or issue, the campaign cannot be successful if voters do not cast ballots in support of the campaign. GOTV represents the part of a campaign that focuses on ensuring that one?s supporters go to the polls and cast votes in favor of the campaign (Green & Gerber, 2004). These efforts typically are made toward the end of the campaign as election day nears and use typical campaign-message delivery systems such as direct mail, phone banks, and door-to-door canvassing (Green & Gerber, 2004). Gerber and Green (2000b) argued that the advent of modern campaign tactics such as direct mail as well as the decline of political power held by political parties and a decline in nonpartisan civic organizations have lead to a reduction in turnout over the last half decade. This led Gerber and Green to each conduct their own field experiments as well as work with other researchers on numerous field experiments in an attempt to learn what campaign methods are effective at mobilizing voters in hopes of providing campaigns and others the guidance to improve overall turnout in America (Green & Gerber, 2004). As discussed above, referendum campaigns offer a different environment in which to test mobilization techniques, primarily because the burden born by a voter to make an informed decision increases in those elections that lack a clear partisan connection (Barker, 2005; Flanigan & Zingale, 2002). The primary method for collecting turnout information in randomized field experiments has been to use public records that indicate if a voter cast a ballot in a given election (Green & Gerber, 2004). However, there have been experiments that use self- 44 reported survey data to indicate if a voter turned out in an election, such as the experiments conducted by Eldersveld and Dodge in 1953 and 1980 by Bositis et al. Both of those experiments used survey-questionnaire items to generate the dependent variable related to turnout. However, survey data should be used with caution in determining turnout because voters are prone to exaggerate self-reported data related to their electoral participation (Bernstein, Chadha, & Montjoy, 2001). Vavreck (2007) found that by using self-reported turnout data rather than verified vote data in the analysis of a randomized field experiment, the estimated effects were reduced by one half rather than overestimating the effect by 100%, if both the dependent variable and independent variable are self-reported. This would imply that finding an effect during the analysis of data from a randomized field experiment using self-reported turnout data against randomized treatment data increases the difficulty of finding statistically significant results. Further, Green and Gerber (2002b) indicated that conducting field experiments during live campaigns inherently means an experiment will be likely to require compromises to ensure the experiment can take place but that researchers should try to work with a campaign to ensure the best possible data can be collected. In reviewing the campaign field-experimental data for this dissertation, seven mobilization experiments were located that were conducted during referendum campaigns. Among those experiments, two experiments tested for but found no support for a mobilization effect through the use of campaign telephone calls (McNulty, 2005a). Similarly, only one experiment tested for but found no support for a mobilization effect through mail (Eldersveld & Dodge, 1954). However, that it not to say that these 45 experiments provide no insight into the expected results from the experimental data to be analyzed in this dissertation. Rather, the seven referendum experiments as well as several of the other nonreferendum-focused experiments provide insight into the outcome of the Alabama Seniors experiment that is the focus of this dissertation. It is generally accepted that campaigns can and do mobilize voters, and experimental research has shown mobilization techniques can be successful at slightly increasing turnout (Green & Gerber, 2004; Gerber & Green, 2005b ). However, it is still unclear as to what techniques are most effective with specific groups of people and what groups of people respond most strongly to what messages. The following sections discuss the experimental results of mobilization experiments as they relate to other variables that are considered in the Alabama Seniors experiment that is the focus of this dissertation. Message Synergy Synergy is the idea that the effectiveness of messages delivered via one medium can be enhanced or improved by messages sent via a subsequently different medium. The belief is that there might be an additive effect or a reinforcing effect when a voter receives messages using multiple mediums (Cardy, 2005). Of the field experiments reviewed for this dissertation, only seven experiments reported results specifically dealing with multimedium treatments on subjects and how those treatments impact turnout as a function of each other (Cardy; Eldersveld, 1956; Friedrichs, 2003; Gerber & Green, 2000b; Green, 2004; Green & Gerber, 2004; Miller, Bositis, & Baer, 1981; Ram?rez, 2005). Only five of those seven were conducted during the renaissance of randomized field experiments into campaign effects that has taken place since 1998 46 (Cardy, 2005; Friedrichs, 2003; Gerber & Green, 2000b; Green, 2004; Ram?rez). Of the seven experiments with synergy, only one dealt with the added dimension of vote choice (Cardy) and none of the seven dealt with the impacts that synergy has in biased experimental treatments during referendum campaigns. Overall, these experiments offer contradictory results and the results that are reported may not generalize across age groups, types of elections, message tone, message type, or election time, which indicates that further research is needed to help test for a synergy effect. In the pre-1998 literature, there are two experiments that test for a synergy effect. During the 1954 Ann Arbor municipal election, Eldersveld conducted a field experiment into the impact that unbiased campaign messages would have on turnout (Eldersveld, 1956). This experiment was the second of two Ann Arbor experiments, the first of which was conducted in the city of Ann Arbor one year earlier during the 1953 municipal referendum campaign (Eldersveld & Dodge, 1954). The 1954 experiment built on Eldersveld?s knowledge and new hypotheses generated during that earlier experiment. During the 1954 experiment, experimental treatments were manipulated in a way that tested for an additive effect from a second subsequent contact through a different medium than the first treatment. In that experiment, there appeared to be no synergy effect as turnout was increased by 18% from door-to-door canvassing alone and turnout was increased by 18% when subjects received a door-to-door treatment as well as a piece of mail (Eldersveld). The second pre-1998 experiment testing for a synergy effect took place during the 1980 Illinois primary election field experiment conducted by Miller et al. (1981) in a Carbondale, Illinois voting precinct. The unbiased messages were delivered from a local 47 committeeman and they encouraged subjects to vote on election day. The messages were delivered through door-to-door canvassing, telephone calls, and mail. In looking at 21?30 year olds, Miller et al. found that the mail treatment significantly increased turnout by 45 percentage points over the control group and that the mailing, when coupled with a door- to-door visit, significantly increased turnout by 56 percentage points over the control group. In looking at participants aged 31?59, Miller et al. found that the combination of the mailing along with a telephone call significantly increased turnout by 67 percentage points over the control group while none of the individual treatments by themselves (telephone, door-to-door visit, and mailing) were statistically significant. However, it is important to note that both the Eldersveld study and the Miller et al. study included uncontacted voters from the treatment groups into the control group, which raises the possibility that their findings are skewed in the direction of a treatment effect. While Eldersveld and Miller did not know that possibility existed at the time, Gerber and Green (2000b) have shown that by moving uncontacted individuals from their respective groups to the control group affords the opportunity for inaccurate analysis to take place. Of those five field experiments conducted since the rebirth of the experimental tradition in campaign-effects research, three were conducted using unbiased messages and none of these three showed a significant difference when looking into the synergy effect. In the 1998 New Haven, CT experiment, Gerber and Green (2000b) looked for but found no synergy effect when direct mail and door-to-door canvassing were coupled together. Then, during the 2000 presidential election, Green analyzed the NAACP experimental data looking for an additive effect when combining unbiased messages delivered via telephone calls with unbiased messages delivered via direct mail. He found 48 nothing statistically significant (Green, 2004). Similarly, Ramirez?s 2002 experiment into Latino mobilization looked into a synergy effect among telephone calls, direct mail, and door-to-door canvassing and found nothing statistically significant (Ram?rez, 2005). All three of these experiments used an unbiased GOTV message to try to mobilize voters to turn out on election day. In looking at the remaining two field experiments that considered a synergy effect, both focused on the impact of a biased message toward a particular candidate or party representing many candidates and the impact those messages would have on voter mobilization. In the 2002 Michigan General Election, Friedrichs worked with the Democratic Party?s Youth Coordinate Campaign to determine the effectiveness of biased messages delivered through phone calls, door-to-door contacts, and door hangers (Friedrichs, 2003). The campaign focused their energies on mobilizing voters aged 18?35 and ultimately found that a telephone call coupled with a door hanger had no synergistic effect. In 2002, Cardy conducted an experiment during an unnamed state?s Democratic primary (Cardy, 2005). Cardy tested the ability of telephone and mail messages to have a synergistic effect on voter mobilization and persuasion. Ultimately, Cardy found no statistically significant synergistic effect, however the turnout point estimate for the phone and mail treatment group (79.3%) was higher than the control group (78.1%), the phone-only group (77.7%), or the mail-only group (77.4%). In looking at the persuasion effect, the results moved in the wrong direction. The point estimates for the sponsor- candidate vote choice among those reporting to have voted in that election were lower in the phone and mail group (54.7%) than for either the phone-only treatment group 49 (59.3%) or the mail-only group (58%). In contrast, the point estimate of the phone- and mail-treatment persuasion effect (54.7%) was higher than the control group of self- reporting voters (53%). In analyzing experiments that consider a synergistic effect, it is important to note that while all experiments since 1998 show no statistically significant mobilization or persuasion effect, none of them focus on a synergistic effect during a referendum campaign, nor did any of them specifically focus on the impacts that might exist in the group of senior voters. Additionally, only one experiment considered the use of a message that was targeted directly to the audience to which the message was delivered (Cardy, 2005). In the case of the Cardy experiment, the point estimates did increase, although not statistically significantly, illustrating there might be a minimal synergistic effect. In further considering the differences between Cardy?s experiment and the others discussed above, one will recall that Cardy conducted the experiment during an election time where partisanship was removed as a heuristic for the voters voting, as the experiment was conducted during the Democratic primary. Perhaps the reason Cardy?s experiment came closer to showing a synergistic effect was because the researcher used messages targeted toward the audience and the experiment was conducted during a time when partisan cues were not available to the voters. This possibility indicates that further research should be conducted testing for a synergy effect and those experiments should be conducted using targeted messages for specific groups of voters. 50 Message Frequency As with synergy, it has been hypothesized that there is an additive or cumulative effect when treating the same experimental participant more than one time through the same medium. But, unlike synergy, the subsequent contact does not necessarily have to be delivered using a different medium from the first contact. In considering the experiments reviewed for this dissertation, eight experiments report results in such a manner that the researcher would be able to distinguish between two different frequencies of contact that were greater than zero (Cardy, 2005; Clinton & Lapinski, 2004; Gerber, 2004; Gerber & Green, 2000b, 2001b; Gerber, Green, & Green, 2003; Gerber, Green, & Larimer, 2008; Green & Gerber, 2004; Niven, 2006; Phillips & Green, 2001). All eight experiments, conducted between 1998 and 2003, measured the cumulative effect of message-delivered frequency. Using the selection criteria described above, any experiment using multiple messages delivered through the same medium and having multiple groups to allow for comparison between different frequencies of treatment are included in the present study. For example, in the 1999 Small Connecticut City experiment conducted during a mayoral race, Green and Gerber compared the treatment group that received nine pieces of mail to the control group that received no treatment (Gerber, 2004). As such, it would not be possible to accurately determine the cumulative effect of each additional piece of mail. In other words, if Green and Gerber had found a 10% mobilization effect, it would not be possible to determine if three pieces of mail generated that effect and any additional mail beyond three pieces had no effect on mobilization. Similarly, Gerber (2004) tested a treatment group of three pieces of mail against the control group during his biased 51 Connecticut State House experiment in 2000, but looking at three pieces of mail compared to no direct mail contact leaves room for uncertainty related to the additive effect of each additional contact. Beginning with the 1998 New Haven, CT direct mail experiment conducted by Gerber and Green (2000b), statistically significant support was found for the additive effect as it relates to mobilization effect. Through their comparison of experimental participants receiving unbiased messages through direct mail, the researchers found a cumulative effect of 0.6% per piece of mail. In other words, an individual receiving one piece of direct mail was 0.6% more likely to turn out on election day than had that person not received the piece of direct mail. Therefore, experimental participants receiving three direct mail messages were 1.8% more likely to turn out on election day than they would have been had they received no direct mail (Gerber & Green, 2000b). Following the New Haven experiment, Gerber, Green, and others began looking for examples of when and under what conditions the additive effect might be found. In their 1999 New Haven experiment looking into biased messages delivered by direct mail during a municipal election cycle, Gerber and Green (2000b) found limited support for the additive effect in mobilization. During this 1999 experiment, the researchers considered the impact that subjects receiving two pieces, four pieces, six pieces, or eight pieces of direct mail would have on turnout. The authors indicated that the effects of direct mail contact appeared to taper off after six contacts (Green & Gerber, 2004). Similarly, the findings of Gerber and Green?s (1999) New Jersey legislative experiment seemed to support a limited impact of up to six pieces of mail (Gerber, Green, & Green, 2003; Gerber et al., 2008). The findings from that experiment indicated that the 52 marginal impact is likely to be between 0.16% and 0.41% per piece of biased direct mail delivered to Republicans and low-turnout Independents. This is borne out in that they found a 0.5% increase in turnout among those receiving four pieces of direct mail and a 1% increase in turnout among those receiving six pieces of direct mail (p < .1; Gerber, Green, & Green, 2003; Gerber et al., 2008). However, different targeted groups or types of elections might yield different effects. During the 2000 presidential campaign, several authors attempted field experiments to dissect the additive effect of other message-delivery mediums such as telephone, e-mail, and television. During their 2000 experiment delivering unbiased messages through telephone calls in an attempt to mobilize youth across six cities in the United States, Gerber and Green (2001b) found no support for the additive effect. Similarly, the Knowledge Network Panel experiment conducted by Clinton and Lapinski (2004) during the 2000 Presidential Election found no support for a mobilization additive effect when experimental participants viewed multiple biased experimental television advertisements. Phillips and Green (2001) examined the impact of multiple unbiased e? mails sent during the 2000 presidential election and found no support for an additive effect in mobilization. Perhaps the differences between the 1998/1999 findings and the 2000 findings were a result of the later experiments being conducted in a presidential election cycle. During the 2002 federal midterm election cycle, Cardy conducted a statewide experiment that used biased messages in support of a gubernatorial candidate, delivered by telephone and direct mail during that state?s Democratic primary (Cardy, 2005). Cardy?s experiment found nothing statistically significant in an additive mobilization 53 effect. In comparing treatment groups, it is possible that some type of additive effect does exist, and although small and not likely significant, it still warrants mentioning. The direct mail-only group received two pieces of mail and turned out at a rate of 77.4%, which was 0.7% less than the control group that received no experimental treatments. However, Cardy?s ?Intensive? group, which received five pieces of direct mail and two phone calls turned out at a rate 1.9% higher than the direct mail-only group. It is important to note that another group that received two pieces of mail and two phone calls also turned out at a rate 1.9% higher than the mail-only group, and therefore it is possible that the effect is actually a synergy effect rather than being related to frequency of treatment. The point is that more research that is carefully designed to separate synergist effects from additive effects is needed to help determine when and if these effects exist during campaigns. The most recently reported experiment that tests for an additive effect was conducted during the nonpartisan 2003 West Palm Beach mayoral election (Niven, 2006). During this experiment, Niven had seven treatment groups receiving between one and three pieces of direct mail that were negative in tone. Niven?s analysis indicated that there was a statistically significant additive effect of 0.6% per piece of direct mail. However, the messages were negative in tone and the campaign was conducted while there were other municipal elections on the ballot, which leaves open the question about elections that exist with no other items on the ballot at the same time. There are a few additional points worth considering about the additive effect and the experiments discussed in this section. First, only two of the experiments discussed in this section report on additive effects related to persuasion. Neither, the New Jersey state 54 legislative experiment nor the Cardy experiment found any statistically significant effects for persuasion (Cardy, 2005; Gerber, Green, & Green, 2003). Second, neither of these experiments, nor any of the others mentioned in this section looked at the impact that an additive effect might have during a referendum campaign. Third, among the eight experiments discussed above, all but two were conducted in a partisan electoral environment. Only the Cardy experiment and the Niven experiment were conducted in environments where partisanship would be reduced or eliminated, but those both contained candidates on the ballot rather than a direct democratic voting option (Cardy; Niven, 2006). Fourth, while the Youth Vote 2000 experiment did consider age, neither it nor any of the other experiments discussed in this section considered the impact of message frequency on senior voters (Gerber & Green, 2001b). The results among the eight have been mixed at best, and there is no clear model that predicts when there should be an additive effect and when there should not be an additive effect. The above literature discussion appears to indicate that a minimal additive effect might exist, at least for direct mail message delivery. However, more research is needed into the additive effect in general and how and when that effect might exist under different election types and appeals across different types of voters, including senior voters. Ultimately, there are numerous conflicting variables that need to be tested and controlled before researchers are able to gain a full understanding of campaign effects during modern campaigns. 55 Message Testing Messages are key to political campaigns (Johnson-Cartee & Copeland, 1997). They are used to articulate the positions and highlight issues surrounding one candidate or issue compared directly on indirectly to another candidate or issue. It is through messages that campaigns communicate the virtues of their candidate or issue and the vices of their opponents. In fact, every campaign communication to voters has some type of message. In this respect given the different types, tones, contents, styles, and mediums to deliver these messages, the variations are limitless, but that has not prevented past researchers from trying to discover what messages might impact a voter?s desire to turn out or vote a particular way on election day. In reviewing the campaign field experiments that focused on persuasion or mobilization, three experiments were identified as having the ability to distinguish between different messages during referendum campaigns. During the 1980 Illinois general election, Bositis et al. (1985) conducted a randomized field experiment in Carbondale that focused on determining the impact that message order would have on voters? choice. The researchers, working with a local committeeman, delivered two biased messages through a single mailed letter with the committeeman?s signature. The letter was meant to encourage support for both the Democratic candidate for the Jackson County state?s attorney position, as well as encourage voters to support a statewide referendum that would modify the multimember district system used to select state legislators. The experimental manipulation was the order of presentation of those two messages in the letter. The dependent variables were collected using a postelection survey and the authors found that, among individuals aged 40 and under, the order of the messages had a significant impact on the information level 56 possessed by voters regarding the referendum. Specifically, the authors found that the message in support of the referendum was most effective when delivered second in the letter. During further investigation, the authors found that the order ?recency? referendum message accounted for a statistically significant 12% of the persuasion effect on vote choice in support of the referendum among voters aged 40 and under. In other words, the authors found a difference between the two messages that were delivered with the recency referendum message having a greater impact on support for the referendum than did the primacy referendum message. The authors found no difference as it related to the candidate message in this experiment. While the findings in the Bositis article were significant, there are several areas where further research might be done to help better determine when and how a persuasion effect might exist depending on the messages that are delivered. First, the authors conducted their experiment in a relatively small geographic area by focusing on a single precinct in Carbondale. Second, the experiment did not have a control group so it was impossible to determine the relative impact each message had compared to the control group. Third, although the authors controlled for respondents? reported partisanship, the messages delivered in the letter were sent by a Democratic committeeman and thereby might possibly have introduced some level of a partisan impact on the vote choice of the respondents. Fourth, the experiment was conducted during the 1980 general election when several other partisan elections were being decided, including the U.S. presidency, which could confound the results of the experiment. Fifth, while the authors found no significant persuasion effect among voters over the age of 40, it is unknown what the connection might have been between the 57 committeeman and the two different age groups (over the age of 40 and aged 40 and under). It is possible there was an undocumented connection between the committeeman and one or both of the age groups. Additionally, the total number of participants in the above 40 group is unknown, but if the number was anything like those in the 40 and under group (n = 65), it is possible that the ability of the experiment to detect an effect among the various treatments was limited in power and perhaps a larger experiment focusing specifically on the higher age group might have a better chance at deducing an effect that is more subtle in that group. Finally, the authors did not consider turnout as a possible dependent variable when looking at their messages. During the 2003 gubernatorial recall election in California, Michelson conducted a randomized field experiment in Fresno to determine the impact that different messages had on voter turnout (Michelson, 2005). Michelson?s experiment considered four different messages (two biased and two unbiased) delivered via door-to-door canvassing. The two biased messages were partisan in nature in that they were sent to either registered Democrats or registered Republicans with a message that was specific to the group that was to receive the message. A message stressing the subject?s civic duty to vote and participate in elections was one of the unbiased messages. The other unbiased message focused on the idea that the recall election was a historic election and that the recipient should participate by casting a vote. The author found no significant treatment effect for any of the messages tested during the experiment. However, in looking at the subgroup of registered Democrats, the author did find the intent-to-treat turnout-effect point estimates to be greatest among those receiving a partisan message (18.3%) compared to the Democrats in the control group (15.7%) or the Democrats who received 58 unbiased messages (15.8%). Again, these are not statistically significant, although they do imply the possibility that a message effect does exist. The author did find a significant effect through regression analysis. The quality of the canvassers was an important factor in the success of the mobilization effort. While all the canvassers were students at Fresno State, Michelson (2005) found that the students in the advanced political science class were more effective at mobilizing voters and that perhaps that was because those students had a higher contact rate than the other canvassers. For example, the biased canvassers in the basic class had a contact rate of 33.8% whereas the advanced class canvassers had a contact rate of 38% for the overall biased-message group. Similarly, the unbiased contact rate was 31.2% among the introductory students and 55.6% among the advanced students. While it is unclear the specific reason why one group was more effective at contacting experimental subjects, it does indicate that there is a difference between the two groups. Further investigation is needed to determine why one group was more effective than the other at contacting experimental subjects. The author does not report the results looking at the various experimental groups split out by the type of canvasser. As with the 1980 Carbondale, IL experiment, the geography covered by the Fresno experiment is small relative to the area of a statewide experiment looking at persuasion or turnout effects during referendum campaigns. Additionally, partisanship was clearly a factor during this experiment in that the candidate up for recall was a Democrat and the two biased-message treatment groups were partisan in nature. Unlike the 1980 experiment, the 2003 experimental treatments were delivered by students going door-to-door in the weeks leading up to the election rather than through the use of mail 59 which would ensure message consistency across all subjects in a given treatment group. Additionally, it is unclear which of the two experiments would have delivered a message most salient to the voters receiving the message. The third campaign experiment conducted specifically during a referendum campaign that allowed for the comparison among groups receiving different messages took place during the 2004 General Election in Los Angeles, CA (Arceneaux & Nickerson, 2005). The experimental treatments consisted of a single biased positive or negative message delivered through door-to-door canvassing in the weeks leading up to the election. The messages were delivered by a nonprofit organization that focused on minority issues. The organization supported two propositions that were on the ballot at that time: one advocating the relaxing of the three-strikes criminal law and the second requiring large companies to pay at least 80% of their employees? health insurance. The authors found no statistically significant difference between the two different message tones related to either a turnout effect or a persuasion effect. However, the biased negative message did significantly increase support for the three-strikes proposition by approximately 13% over the control group. It is unclear in the authors? research why the negative message related to the three-strike proposition was found to be significant relative to the control group while the other message findings were insignificant. Perhaps the message that was delivered regarding the three-strikes proposition was more salient with the largely Latino experimental sample than it would have been with other voters at that time. Unlike the first two experiments discussed above, the 2004 Los Angeles experiment focused on both turnout and vote choice. Like the 1980 experiment, 60 Arceneaux and Nickerson (2005) found a persuasion effect for one message and like the 2003 experiment, the authors found no support for a mobilization effect. As with the other two experiments, the 2004 experiment was conducted at the local level. Additionally, it was conducted during the 1980 presidential general election. In reviewing these experimental findings, it is unclear how and under what circumstances one message might be more effective than another during a referendum campaign. Similarly, it is unclear when one medium might be more effective than another or when a message might be more effective with one group than another. As seems to be a recurring theme in electoral research, there is no clear answer to those questions and further research will be needed to better predict when and where a particular message would have a given impact in an election. Message Targeting Targeting is a strategy that arose in the marketing industry (Johnson-Cartee & Copeland, 1997). Essentially, the aim of targeting is to improve the effectiveness of a marketing communication by delivering that message to a specific group of people. The process used to identify a target market is called market segmentation (Johnson-Cartee & Copeland, 1997). The term segment is used because the potential receiving group is segmented based on some kind of characteristic that allows a marketer to group similar people into a segment. Once segmented, the group then receives a message that is somehow believed to be more salient or specific to that group of individuals than might be the case for the population as a whole. 61 The idea of segmentation and targeting is not new to campaigns and elections. For example, pollsters use segmentation to indicate what groups of voters support or oppose a given candidate during an election (Flanigan & Zingale, 2002). Additionally, campaigns tend to focus their limited resources on voters with a historical pattern of voting so as to improve the efficiency of the money spent during a campaign (Johnson-Cartee & Copeland, 1997). However, there is little research in the field that indicates how effective targeting is when it comes to mobilization and persuasion. In reviewing the campaign and elections field-experiment literature, two field experiments were found to have been conducted using messages that were targeted toward a specific segmented group during a referendum campaign. The two experiments focusing on referendum campaigns and targeting took place during the 2003 gubernatorial recall election in California. The McNulty (2005a) experiment took place in the East Bay area of San Francisco and used biased telephone calls as the experimental treatment. These calls were targeted by the University of California Berkeley Democrats (Cal Dems) student group to newly registered young Democrats who were recently registered on the campus by the Cal Dems. The calls were made by the Cal Dems during the week leading up to the recall election, and the message included asking the recipient to vote no to the recall of Governor Davis, vote against a ballot proposition that was also on the ballot, and to vote in favor of the lone Democrat as the candidate to replace Governor Davis should the Governor be recalled. Being that these were the positions of the Democratic Party, it seems logical to assume that registered Democrats would be more responsive to these messages than would some other segment of the population. 62 Ultimately, McNulty found no support for a mobilization hypothesis when comparing the turnout rates of those in the telephone treatment group to the control group. During the same 2003 recall election, Michelson (2005) conducted a similar experiment using a door-to-door treatment in Fresno, CA. During Michelson?s experiment a portion of the youth voter segment was randomly assigned to receive either a biased partisan message, an unbiased message, or to receive no treatment. The messages were delivered through door-to-door canvassing the week leading to the recall election, and the student volunteers that performed the canvassing were allowed to select which party they wanted to canvass in support of or if they wanted to canvass delivering the unbiased message. The biased partisan message in support of the Republican position on the recall was critical of Governor Davis and encouraged the voter to support a Republican candidate in the recall election. The biased partisan message in support of the Democratic position offered support for Governor Davis and encouraged the voter to vote no on the recall ballot measure. The unbiased message explained the historic nature of the election and reminded voters of their civic duty to participate in the electoral process. Ultimately, the author found no statistically significant support that any one message was more effective at mobilizing youth than another message. However, among the Democratic participants, those that received the biased partisan message had a turnout that was 2.6% higher than the control group?s turnout rate of 15.7% and 2.5% higher than the turnout rate for the nonpartisan message group. Among the Republican participants the turnout for the biased partisan message was 2.7% lower than the control group?s turnout rate of 14.8% and 2% lower than the nonpartisan message group?s turnout rate of 14.2%. This would imply that the Democratic partisan message had some (although not 63 statistically significant) positive effect on mobilization while the Republican partisan message had some (although not statistically significant) negative effect on mobilization among their respective target groups. It is unclear why such an effect might exist but perhaps there is a psychological effect associated with firing someone rather than electing or hiring someone. It is possible that the biased messages that were used in the two Michelson (2005) experiments were not the most salient message that could have been delivered to the respective voters. Neither of the two experiments conducted during the 2003 recall election reported using research to develop a message that was targeted and therefore potentially more salient to the voters receiving the message than any other message that could have been delivered. It seems logical that the partisan messages described above would be more salient than not with their respective partisans, but further research is needed to determine if a more salient message would have more of an impact than a broad-ranging partisan message. Similarly, neither of the experiments discussed here included the testing of a biased targeted message?s effectiveness against a biased nontargeted message?s effectiveness related to the same segmented group of voters. Additionally, neither experiment tested to see if a message delivered by direct mail would have more or less impact when used in a targeted method. It is possible that the less consistent methods of delivering messages in the above experiments (student telephone calls and student canvassing) would make detecting an effect more difficult, whereas a message delivered either by recorded call or by direct mail ensures that all subjects receive nearly the same treatment. 64 The partisan nature of the recall election might further complicate a researcher?s ability to determine the impact of biased messages because partisan cues are often cited as one of the primary predictors of voting behavior (Flanigan & Zingale, 2002). Both of the experiments conducted during the 2003 recall election focused solely on looking at a mobilization effect, leaving open the possibility that a targeted message would be more effective at affecting vote choice rather than mobilization. Similarly, both experiments focused on targeting the youth vote and the possibility exists that the youth segment of the voting population might be different from other segments of the voting population such as seniors or a racial minority. Finally, both of these experiments took place at the local level in California. Further testing is needed to determine if testing in a larger geographic area would have an impact on the outcome of an experiment or if a different area of geography such as a county or different state would make a difference in the outcome of a targeting experiment. Ultimately, given the methods that can be used to target and segment as well as the multitude of possible segments, the limited existing research into political campaign targeting will need to be expanded if researchers are to determine when and how targeting is most effective during political campaigns. Seniors In considering ways to target different groups, a logical next step is to consider the impacts various messages have among the groups that campaigns typically target. Because seniors are among the most reliable group of voters, campaigns tend to focus their resources on them in an effort to affect the outcome of elections (Herrnson, 2004; MacManus & Shields, 2000). Typically, the young voter group is the least reliable 65 voting-age block and as voters age they become consistently more likely to participate in elections (Flanigan & Zingale, 2002). However, in reviewing the campaign field- experiment literature, none of the experiments focused specifically on the senior age group. However 4 field experiments were reviewed that discussed age impacts that included seniors while 14 experiments focused specifically on youth voters (Arceneaux & Nickerson, 2005; Bennion, 2005; Dale & Strauss, 2007; Friedrichs, 2003; Gerber & Green, 2001b; McNulty, 2005a, 2005b; Michelson, 2005; Nickerson, 2005, 2006, 2007a, 2007c; Nickerson, Friedrichs, & King, 2006; Phillips & Green, 2001). This lack of research on senior voters leaves a void in connecting campaign-effect literature with professionals in the campaigns and elections industry. Perhaps the primary focus on youth voters is because proportionately they turn out at a much lower rate than other age groups (Flanigan & Zingale, 2002). This would mean there is a significantly larger portion of that population that can be motivated to participate in elections compared to senior voters. In other words, detecting experimental mobilization effects would likely be easier because there is a large proportion of the youth age group to influence. Additionally, researchers have shown that voting is habit forming in the sense that by voting in one election, a voter?s chances of voting in a subsequent election increases (Gerber, Green, & Shachar, 2003). Researchers have shown that participation rates have declined over the past half century (Gerber & Green, 2000b). Therefore, by focusing on the youth, researchers may be indirectly working to solve the perceived problem of the declining participation over the past half century by mobilizing new voters who will carry that habit forward. 66 In general, those actually casting ballots tend to be more educated, wealthier, and older than the voting-age population as a whole (Flanigan & Zingale, 2002). By focusing on campaign effects for young voters, researchers are missing effects that might exist in other age groups or subgroups of the voting population. Senior voters, because they are so reliable, tend to receive significant attention from campaigns that focus on voters that are more likely to vote than not (Herrnson, 2004; MacManus & Shields, 2000). Additionally, as the baby boomer generation grows into retirement age, senior voters continue to increase in the proportion of votes cast as well as making senior issues more important than ever in political campaigns (MacManus & Shields). Four field experiments were reviewed that discussed the impacts campaigns might have on senior voters. The results conflict and suggest more research is needed before researchers understand what techniques, if any, influence the voting behavior of senior voters. The first experiment reporting results on age effects was Eldersveld?s (1956) Ann Arbor experiment testing an unbiased message delivered through door-to-door canvassing. Eldersveld found that voters aged 50 and older who were contacted with the door-to-door canvassing contact were 12% more likely to turn out than were those in the control group. Additionally, Eldersveld found that voters between the ages of 36 and 50 were 26% more likely to turn out based on the contact. When treating voters with a door- to-door canvassing treatment, it is nearly impossible to treat all subjects that have been selected for treatment. Because voters who are not treated are less likely to vote than a typical voter, it is important to take the contact rate into account when calculating the treatment effect (Gerber & Green, 2000b). Eldersveld did not take the contact rate into account when calculating the treatment effect, which would lead to reporting a lower 67 estimate than actually exists, because unlike direct mail, door-to-door canvassing does not have a 100% contact rate. Recent research has focused on taking that contact rate into account when calculating treatment effects (Gerber & Green, 2000b). Eldersveld also combined uncontacted members of the treatment groups with the control group, which would lead to higher estimates than actually exist in the experiment (Gerber & Green, 2000b). Miller et al. (1981) conducted two experiments during the 1980 election cycle that reported results for different age groups of experimental voters. The first experiment was conducted during the 1980 Democratic primary election in Illinois. The treatment consisted of an unbiased message delivered by the local partisan precinct committeeman that encouraged voters to vote in the upcoming election. The messages were delivered by telephone, mail, and door-to-door canvassing. While this experiment had a small number of participants compared to modern randomized field experiments, the authors did report results related to senior voters. While not reaching the level of statistical significance, the authors indicated that among voters aged 60 and older, telephone contact produced a mobilization effect of 8% while letter contact did not increase turnout compared to the control group. As with the Eldersveld results, the Ann Arbor researchers did not take contact rate into effect when calculating the treatment effects, and they included uncontacted treatment group subjects in the control group. These methods tend to lead to inaccurate estimates regarding treatment effects (Gerber & Green, 2000b). Also during the 1980 election cycle, Miller et al. conducted a persuasion field experiment during the general election (Bositis et al., 1985). During this second experiment the authors delivered a biased message in support of a local candidate for 68 state?s attorney and in support of a statewide referendum measure. The letter was from the Democrat precinct committeeman to the voters in his precinct. They found no persuasion effect among participants 40 years of age or older (Bositis et al.). The only post-1998 randomized field experiment that reported age-related effects among older voters was conducted by Bennion (2005) during the 2002 general election. The experimental-treatment groups received an unbiased message encouraging people to vote on election day delivered through door-to-door canvassing by university students in the South Bend, IN area. Bennion reported that the experimental treatments produced no mobilization effect among voters over the age of 30 while producing a mobilization effect of 18% for voters aged 30 and under. While this experiment clearly supports an age effect among youth related to mobilization, it is possible that the messengers (university students) had an impact on the experiment simply because they were young and possibly had a more personal or salient connection with those youth receiving the messages than those experimental participants in other age groups. As with the other sections discussed in this chapter, more experimental research is needed to improve the community?s abilities to make better predictions about what messages will influence seniors during any given election. Among the four experiments discussed above, only the 1980 general-election experiment in Carbondale, IL focused on persuasion, and it was also the only experiment to consider the impact related to a particular referendum (Bositis et al., 1985). The other three experiments focused on mobilization and none of the four experiments considered a mobilization effect and a persuasion effect in the same experiment. Also, none of the four experiments were conducted without other potentially confounding items on the ballot at the time nor did 69 any of them consider the impact on a level of geography beyond the local level. All three experiments offered support for some type of age effect with two of the experiments specifically showing that an effect might exist among older voters. The Eldersveld (1956) experiment reported that an unbiased door-to-door contact could raise turnout by 12% among voters age 50 and older. The unbiased Miller et al. (1981) experiment reported that a telephone call could increase turnout among voters age 60 and older by 8%. However, both of these experiments have methodological issues that require further experimentation to improve our knowledge in those areas. Finally, none of the experiments reviewed for this dissertation included a message developed and delivered specifically for senior voters, as campaigns are prone to do (MacManus & Shields, 2000). Additionally, research is needed to determine when and under what conditions a campaign can influence the fastest-growing segment of the vote population. Conclusion This chapter has provided a review of the literature leading to the experiment that is the focus of this dissertation. The chapter began with an introduction that laid the basic foundation for the remaining sections as well as discussing the structure of the chapter that followed it. The next section dealt with the scope of the review regarding the literature discussed in this chapter. That section outlined the methods used to locate and review the literature that addresses the focus of this dissertation with the goal of allowing others to duplicate the location of the research discussed. The Experimental Methods section discussed the value and logic behind performing randomized field experiments during live political campaigns. Also discussed were the benefits and issues associated 70 with performing campaign field experiments as well as the analysis techniques used during such experiments. The Referendum section discussed the potential benefits to be gained by studying campaign effects using experimental techniques during direct democratic campaigns. In Partisanship Redefined, the argument was made to separate the term partisanship from describing the type of message a campaign delivers, opting instead to use the terms biased and unbiased. By introducing the biased and unbiased terms, researchers can further allow the term partisan to focus on the campaign environment during a campaign experiment. The Persuasion section offered a discussion regarding the persuasion effect and focused particularly on the persuasion effect during referendum campaigns. Similarly, the Mobilization section reviewed the literature on mobilization with a specific focus on mobilization during referendum campaigns. The Message Synergy section reviewed the literature regarding multiple treatments sent via multiple methods during referendum campaigns and why it is thought that a synergistic campaign effect might exist. The Message Frequency section focused on a review of the literature with specific emphasis on referendum campaigns regarding multiple messages sent via the same medium and why it is though that an additive effect might exist during campaign experiments that focus on mobilization and vote choice. The Message Testing section reviewed the relevant literature that allowed for a comparison between multiple messages in an attempt to illustrate that different messages can have different impacts on turnout and vote choice. Because different messages can impact different voters differently, the Message Targeting section reviewed the relevant literature that tests for campaign effects associated with delivering a targeted message to 71 a specific group of people in an attempt to improve the salience of the message. The last section, Seniors, reviewed the relevant experimental literature regarding campaign effects and seniors. Table 3 reviews the primary characteristics of the eight referendum experiments that were the central focus of chapter 2. The last item in Table 3 illustrates how the Alabama Seniors Experiment relates to the other eight experiments. In the end, this chapter provides the reader with the knowledge and background necessary to understand the hypothesized effects that are expected when the analysis of the Alabama Seniors Referendum Experiment is performed. The next two chapters focus specifically on the Alabama experiment with chapter 3 providing the background and description of the 2003 experiment. Chapter 4 reports the analysis of the data collected during the 2003 experiment. 72 ? Table 3 Characteristics of Discussed Referendum Field Experiments Experiment Year Mesage type Mobilization Persuasion Synergy Frequency Mesage testing Targeting Seniors focus An Arbor, Michigan municipal referendum (Eldersveld & Dodge, 1954) 1953 Biased Yes* Yes* No No No Yes No Carbondale, Ilinois experiment #2 referendum (Bositis et al., 1985) 1980 Biased No Yes* No No Yes No No San Francisco, California proposition (McNulty, 205a, 2005b) 2002 Biased Yes No No No No Yes No East Bay, California recal election (McNulty, 205a, 205b) 2003 Biased Yes No No No No Yes No Fresno, California recal election (Michelson, 205) 2003 Biased and Unbiased Yes No No No Yes Yes No Maricopa County, California local proposition (Michelson, 205) 2003 Biased and Unbiased Yes* No No No No No No Kansas City, Misouri referendum (Arceneaux, 205) 2003 Biased Yes* No No No No No No Los Angeles, California propositions (Arceneaux & Nickerson, 205) 2004 Biased Yes Yes* No No Yes No No Alabama seniors experiment (this disertation experiment) 2003 Biased Yes Yes Yes Yes Yes Yes Yes Note. Apendix B contains a list of al the experiments reviewed for this disertation; * an efect was reported by the author(s) of the experiment regarding mobilization; * an efect was reported by the author(s) of the experiment regarding persuasion 73 CHAPTER 3: METHODS AND RESEARCH DESIGN Introduction This chapter offers details associated with this dissertation?s focal experiment and the experimental design. The chapter begins with a condensed recap of the literature leading to the research questions and hypotheses that are the focus of this dissertation. A discussion of the experimental design choice made by the consulting organization follows along with a discussion of the experimental procedures, the timeline of the experiment, and the campaign surrounding the election. The sections on population, data collection, and instrumentation offer an understanding of the nuances that went into the experiment and data collection. Finally, there are brief discussions of the data analysis and the internal and external validity issues of the research. Literature Summary As discussed in the previous chapter, the current literature regarding persuasion and vote choice is inconclusive and much more research is required before a more complete understanding of campaign effects is reached. However, the prior research does provide some guidance as to the expected outcome of the analysis of the Alabama Seniors Experiment data. This section provides a short summarized review of some of the key literature discussed in chapter 2. 74 As Barker (2005) pointed out, by testing for campaign effects during elections that lack partisan cues, researchers might be in a better position to detect the limited effects associated with political campaigns. In other words, referendum campaigns provide an opportunity to control for the partisanship heuristic that is typically a major factor in voting behavior (Dalton, 2002). Eight campaign field experiments were identified that focused specifically on campaign effects during some type of direct democratic election (see Table 3). Because the data for this dissertation were collected during a referendum campaign, it is expected that these effects might be easier to detect due to the lack of partisanship as a shortcut for the voters that participated in the election. In reviewing the literature for experiments similar to the experiment reported in this dissertation, two experiments stand out as being able to offer insight into the expected outcome of the Alabama Seniors Experiment. In 1953, Eldersveld and Dodge (1954) conducted a field experiment related to an Ann Arbor, Michigan referendum. Their experiment tested for persuasion and mobilization effects among the voters that were thought to be against the referendum. The experiment used biased messages in support of the referendum delivered via mail to a targeted group of voters. The authors found a 20% persuasion effect and a 28% mobilization effect when comparing the treatment groups to the control group. In a similar study nearly 30 years later, Bositis, Baer, and Miller (1985) tested and found support for a persuasion effect based on direct mail that supported a statewide referendum in Illinois. The authors delivered biased direct mail messages in support of the referendum to voters living in precinct located in the city of Carbondale. The purpose of the experiment was to determine if the order of two messages had an impact on voters? 75 support for the referendum. A 12% persuasion effect was found when the referendum message was the last message received of the two messages. Unfortunately, the authors did not have a control group for comparison, which limits the ability to predict the impact that messages might have over no message. Yet, assuming both sequences of messages had a positive impact on persuasion among voters, the 12% persuasion effect could be even higher had there been a control group against which to compare. None of the referendum experiments reviewed for this dissertation tested for a frequency effect, which therefore requires looking outside the referendum experimental literature for guidance related to how frequency would impact the results of the Alabama Seniors Experiment. In reviewing the literature related to frequency, two experiments were identified that were similar to and seemed to offer some insight into the expected outcome of the experiment reported in this dissertation. These two experiments tested the impact of multiple messages on vote choice and mobilization. During the 1999 election cycle, Gerber and Green conducted an experiment testing the impact that multiple biased messages had on both turnout and vote choice during a New Jersey State Assembly election. During that experiment, the authors delivered multiple direct mail messages supporting one candidate over another to voters with the expectation that each additional piece of mail would have an additive effect for both vote choice and mobilization. Ultimately, the authors found limited support for their hypotheses; Gerber (2004) attributed a 0.034% effect per mailing on vote choice and Gerber et al. (2003) attributed a treatment effect of 0.12% per mailing on turnout. During the 2003 election cycle, Niven (2004) conducted an experiment testing the impact of multiple campaign messages delivered via direct mail to voters during the West 76 Palm Beach Mayoral Election. Niven?s experiment was designed to test the additive effect of negative messages on voter mobilization and the data indicates that a single message tended to have a positive effect of around 3%, two negative direct mail messages had an impact of around 6%, and three messages had a turnout effect of 9.4%. In other words, each piece of mail increased the likelihood of voting by approximately 3%. Taken as a whole, these experiments appear to indicate that targeted campaign messages have a positive effect on both turnout and vote choice. Although, given the New Jersey results, the positive effect could be extremely small. However, the power of the New Jersey experiment was such that the persuasion effect was detectable despite its small size. This is due to the large sample size of nearly 20,000 households used in that experiment. The above research coupled with the additional experiments reviewed for this dissertation lead to the following research questions and subsequent hypotheses. Research Questions and Hypotheses 1. Do the type and frequency of targeted campaign messages increase voters? intention to vote? 2. Do the type and frequency of targeted campaign message influence the likelihood that potential voters vote in favor of a targeted-message issue? The research questions were recast into research hypotheses as follows. H1a: The type of targeted-campaign messages increases the likelihood that potential voters will indicate their affirmative intention to vote. 77 H1b: The frequency of targeted-campaign messages increases the likelihood that potential voters will indicate their affirmative intention to vote. H2a: The type of targeted-campaign messages increases the likelihood that potential voters will vote in favor of the targeted-message issue. H2b: The frequency of targeted-campaign messages increases the likelihood that potential voters will vote in favor of the targeted-message issue. Research Design This dissertation uses secondary data collected from a field experiment conducted during a live nonpartisan statewide-referendum campaign for a revenue-increasing measure in Alabama. Because the experiment?s focal campaign was nonpartisan and was the only item on the ballot during the election and because the experiment used targeted persuasive messages, the actual impact of message type and frequency were expected to be more detectible during the analysis. An experimental, 2 x 6 factorial, random-assignment posttest-only control group design is used to test the hypotheses. The independent variables are (a) message type and (b) frequency of message. There were two types of messages. One message was directly targeted with the intent to influence a voter?s vote choice, while the second was related to the voting issue but generic in nature. The frequency of messaging ranged from one group receiving no messaging (control group) to groups receiving up to three combinations of the types of messages. In all, there were five experimental groups and a control group, as discussed below. 78 The dependent variables were preelection questionnaire items collected via telephone survey that asked potential voters if they (a) intended to vote, and (b) if they favored the issue that was the subject of the messaging. Additional questionnaire items (see Appendix A) provided information on voter demographics, as well as additional potential explanatory variables. The experimental design allows a researcher the opportunity to show causality between the independent and dependent variables. The ability to draw a valid causal inference between variables is predicated by a researcher?s ability to ?randomly assign observations to treatment and control groups? (Green & Gerber, 2002b). Most political science research has focused on nonexperimental methods such as survey research and modeling (Green & Gerber, 2002b). However, field experiments provide numerous benefits for political-science research. First, a field experiment provides a significant advantage in determining the causal relationship between the treatment and the effect (Gerber, Green, & Shachar, 2003; Imai, 2005; Johnson-Cartee & Copeland, 1997; Lavine & Snyder, 2000). Second, such research designs allow researchers to make group comparisons with the only theoretical difference among the groups being the treatments themselves (Imai). Third, an experiment allows for less complicated methods of analysis while still allowing the researcher to gain causal knowledge (Green & Gerber, 2002b). Finally, a field experiment is usually more generalizable than a laboratory experiment because it is conducted in a real-world ?naturalistic? setting rather than through a simulation in a laboratory (Green & Gerber, 2002b; Imai). 79 The Experiment As is typical during a political campaign, the professional consulting organization that was running the campaign in support of Amendment One during the course of the campaign produced several pieces of targeted direct mail and targeted telephone calls. Many of these targeted messages were outside the scope of the experiment. For example, the campaign had messages that were targeted toward Black voters, educators, ?do gooders,? and union members. All the campaign messages, including the experimental messages, were the result of professional production either by the campaign consulting firm or by subcontractors who specialize in a given area of the campaign. For example, the consulting organization hired a professional recording studio in Birmingham to acquire talent and produce the phone-banking messages. Additionally, the firm hired professional bulk-mailing houses in both Birmingham and Montgomery to efficiently label and mail hundreds of thousands of pieces of direct mail during the campaign. All direct mail was designed by the consulting firm using their own graphic-design artists and consultants who specialize in campaign messages. As part of the campaign, the consulting firm wanted to test the effectiveness of their campaign messages on senior voters (those voters aged 65 and older) around the state. As discussed in the subsequent Population and Sample section, the firm assigned control and experimental groups and delivered the respective treatments. Below is a detailed discussion of the experimental groups and treatments. Table 4 provides an experimental timeline to help visualize the experiment. On August 11, 2003, the first piece of direct mail (Seniors 1?see Appendix C) reached the post office for distribution to four experimental groups (Groups B?E), as well 80 as all other senior households in the state (except those in the control group). The Seniors 1 piece of direct mail had the targeted message that seniors would have a tax reduction rather than a tax increase should Amendment One pass, and therefore seniors should not be afraid to support the amendment. The Seniors 1 piece of direct mail was a two-sided, four-color, horizontal postcard mailer that was approximately 13" wide by 8.5" tall. Table 4 Timeline of Alabama Seniors Experiment Date Group Treatment Notes August 1, 203 Group A None Control Group August 1, 203 Group B Seniors 1 Targeted Direct Mail August 1, 203 Group C Seniors 1 Targeted Direct Mail August 1, 203 Group D Seniors 1 Targeted Direct Mail August 1, 203 Group E Seniors 1 Targeted Direct Mail August 16, 203 Group F NA Group F created from Group A August 26, 203 Group C Seniors 2 Targeted Direct Mail August 26, 203 Group E Seniors 2 Targeted Direct Mail August 26, 203 Group D Working Families Generic Direct Mail August 26, 203 Group F Working Families Generic Direct Mail September 2, 203 Group E Seniors Phone Bank Targeted Phone Cal September 3?7, 203 NA NA Telephone Survey of Group Participants September 9, 203 NA NA Amendment One Defeated in Election Following the mailing of the Seniors 1 piece of direct mail on August 11, 2003, the campaign decided to add another experimental group to the experiment (Group F). By adding a sixth group to the experiment (Group A was the control group and Groups B?F were the treatment groups), the campaign hoped they would be able to determine if the targeted message for seniors was more effective in changing voter preferences than was a nontargeted piece of direct mail sent to ?working families? around the state. Making such 81 a comparison required the creation of a group that had not received the Seniors 1 mailing. Because all senior voter households in Alabama except those in the control group received the Seniors 1 mailing, the control group was split into two groups. One group would remain the control group (Group A) for the experiment and the other would become an additional experimental group (Group F). On August 26, 2003, the campaign sent out a second batch of direct mail as part of the experiment. Groups C and E, which had already received the Seniors 1 piece, then received a similar piece called Seniors 2 (see Appendix D). The message of Seniors 2 was largely the same as that of Seniors 1, with the Seniors 2 piece focusing on the argument that seniors would see a tax reduction rather than a tax increase should Amendment One pass. Additionally, the Seniors 2 mailer carried the message that seniors needed to protect their prescription-drug program by supporting Amendment One. Like the first piece of Seniors mail (Seniors 1), this piece of mail was a postcard-style, four- color, two-sided mailer. Unlike the first mailer, Seniors 2 was a vertical mailer approximately 6" wide by 8" tall. Also on August 26, 2003, the campaign mailed out the ?Working Families? piece of direct mail to Groups D and F (See Appendix E). Earlier in the experiment, Group D received the Seniors 1 piece of direct mail. Group F had not yet received an experimental treatment, so the Working Families mailing was the first?and ultimately the only? treatment they received. The Working Families mailer had a message that big business would be forced to pay their fair share of taxes and that working families would likely see a tax cut should Amendment One pass. This mailer, like the other two, was a two-sided four-color postcard mailer. This mailer was approximately 17" wide by 11" tall. 82 On September 2, 2003, the campaign initiated a phone bank targeting the voter households in Group E (see Appendix F for script). Those households in Group E received a prerecorded sixty-second telephone message from an elderly woman with the same basic message conveyed in the Seniors 1 and Seniors 2 pieces of direct mail. The campaign hoped that the recorded message would supplement and further expand support for Amendment One among those in Group E. Population and Sample The population for this research was all registered voter households in Alabama with at least one voter aged 65 or older. The Alabama Secretary of State maintains a list of registered voters, including the voter?s date of birth. Typically, every 2 years, various list vendors purchase the list from the state and append additional information such as voter telephone numbers and other geodemographic data to it. The consulting firm contacted such a list vendor and purchased the list of all registered voters aged 65 and older for use during the campaign and in the experiment that is the focus of this dissertation. The purchased list was imported into a computerized database application to yield a list of all unique residential addresses with a telephone number. Each household address then received a random number using the random-number generator function located in the database application. Sorting by random number, the first 5,000 records were selected for the experiment. Each of the 5,000 records was then systematically assigned to one of six equal-sized groups. Group A became the control group and received no experimental 83 treatment and the remaining five groups received some form of an experimental treatment. Below is a short discussion about each of the six groups (one control and five experimental) studied in this research. Appendixes C, D, E, and F contain copies of the direct mail pieces and the telephone script used during the experiment. Group A had 550 senior voter households and was the control group. During the campaign, they received no experimental treatment. There were 147 respondents in the secondary survey data from this group. Group B was comprised of 1,000 senior voter households. The only experimental treatment they received was a single piece of direct mail designed to influence seniors. The message of this piece of direct mail contained the primary message that seniors would not have to pay any additional property taxes should Amendment One pass and that, therefore, seniors should not be afraid to support the Amendment. There were 150 respondents in the secondary survey data from this group. Group C consisted of 1,000 senior voter households. This group received two experimental treatments. First, they received the Seniors 1 piece of direct mail that Group B received. Second, they received an additional piece of direct mail that contained a message similar to the first piece of mail (Seniors 2). There were 151 respondents in the secondary survey data from this group. Group D contained 1,000 senior voter households. Like Group C, this group received two experimental treatments. Just like Groups B and C, Group D received the Seniors 1 mailing. Additionally, they received a piece of mail aimed at influencing working families across the state. The message delivered in the ?Working Families? 84 mailer was that ?big business? would be forced to ?pay their fair share? and that working families would likely see a ?tax cut? should Amendment One pass. There were 150 respondents in the secondary survey data from Group D. Group E was comprised of 1,000 senior voter households from around Alabama. Like Group C, they received both the Seniors 1 and Seniors 2 pieces of direct mail. Additionally, they received a prerecorded sixty-second telephone message of an elderly woman with the same, targeted Seniors message that seniors would not have to pay any additional property taxes should Amendment One pass. There were 150 respondents from Group E in the secondary survey data. Group F was composed of 450 senior voter households. Group F received only the ?Working Families? mailer that Group D received. There were 106 respondents from Group F in the secondary survey data. Data Collection Following the delivery of all experimental treatments, a preelection telephone survey typical of a political campaign was conducted from September 3, 2003, through September 7, 2003. The survey house randomly selected telephone numbers from each group and administered the survey to the individuals who answered the phone. The respondents were first asked if they were ?65 or older?; if so, the interviewer asked a series of questions regarding Amendment One. If the potential respondent was under the age of 65, the call was terminated granted no other respondent aged 65 or older was available. As would be typical in a campaign survey of this type, a quota of 150 respondents per group was instituted to meet the campaign budget requirements. 85 The telephone survey was designed and instituted specifically for the experiment, as the campaign itself had all but shut down in the face of what would ultimately become a sound defeat. The campaign survey is the secondary data source and the focus of this dissertation. The campaign hired a professional marketing and survey-research firm to conduct the telephone interviews that they have allowed be used as the data for this dissertation. That organization was provided a list of telephone numbers and the associated group assignments; however, the polling firm was not provided any information about the experiment and was not aware that this data was being collected for analysis in an experimental setting. The polling firm then set up the questionnaire in their computer-aided telephone- interview system and imported the list of 5,000 unique telephone numbers. The only quota the survey house was given was to try and complete a total of 150 interviews per group. Beginning the evening of September 3, 2003, the computer-aided telephone- interview system randomly selected telephone numbers and the interviews were conducted as if the survey were a typical campaign survey. During the first three nights the questionnaire was being administered, individuals indicating they were not 65 years of age or older would prompt the interviewer to ask to speak to a registered voter that was aged 65 or older. If no one was available, the interviewer asked to schedule a day or time to call back. If a day or time was scheduled, the survey house attempted to conduct the interview again at that time or on that date. However, if the interviewer was unable to schedule a time or day, the number was removed from the sample and no further interview attempts were made to that number. 86 Because the original control group (Group A) was split into two groups (Groups A and F) after the experiment begin, it was impossible for the survey house to complete 150 interviews for those groups. However, the survey house was able to complete 150 interviews for Groups B, C, D, and E. In the end, due to budget restrictions, the campaign terminated the survey after the fourth night with Group A having 147 completed interviews and Group F having 106 completed interviews. The morning of Monday, September 8, 2003, the survey house compiled the data into an SPSS file and provided it to the campaign for their analysis. Following the conclusion of the campaign and the campaign consulting firm?s analysis of the data, the SPSS data file was provided for use in this dissertation as secondary data. Instrumentation In April 2003, the campaign consulting firm hired a political-polling firm to conduct a statewide benchmark survey that would assist in crafting the campaign strategy for Amendment One. Used widely in campaigns, benchmark surveys serve to inform the campaign about issues, voting preferences, and the likely voting population during the election (Herrnson, 2004; Herrnson et al., 2005). The benchmark survey contained over 150 items and took an average of 20 minutes to complete per respondent. There were 903 completed questionnaires included in the benchmark survey. The benchmark survey contained many questions that the campaign believed would give them insight into what messages, issues, facts, and endorsements would influence specific groups of voters on their vote choice. Some questions focused on the favorability and name recognition of public figures. 87 Respondents were asked about the impact endorsements by those public figures, celebrities, and organizations would have on their individual vote choice. A large series of questions sought to educate the voter on particular positive and negative ?facts? associated with Amendment One and the impact those ?facts? would have on the voters? vote choice. These ?fact? questions tested the likely arguments both against and for Amendment One. The benchmark survey, data, analysis, and conclusions are proprietary and not available for public perusal. However, it is through the analysis of the benchmark survey that the campaign and the polling firm concluded that the specific message that ?seniors would pay no additional property taxes on their homes? was a persuasive message with senior voters. Because the campaign was going to target an easily identifiable population of voters, the firm decided to conduct a field experiment during that portion of the campaign. For the campaign to maintain question continuity from the benchmark survey through the end of the campaign, the survey questions used in the experiment questionnaire (as with all their surveys conducted during the Amendment One campaign) were worded the same as they were in the original benchmark survey. The benchmark survey questions were drafted by the professional polling firm that was hired to conduct the benchmark survey. There was no discussion about the validity or reliability of the various measures in the benchmark survey (and by extension the experimental survey). However, the campaign believed the experience and history of the polling firm would produce more likely valid and reliable measures. The experimental questionnaire is attached in Appendix A. 88 The experimental questionnaire was a live telephone interview and contained 16 items that the campaign felt would be valuable in their experiment. The first question was a screening question initially meant to ensure the interviewer was talking with someone aged 65 or older. Questions 2 and 3 dealt with the dependent variables for the experiment. Question 2 asked about the likelihood that the respondent would be voting, with potential responses being likely voting, not likely voting, and not sure/refused. Question 3 asked how the respondent would vote if the Amendment One election were held that day. Possible responses for Question 3 were definitely vote for, probably vote for, definitely vote against, probably vote against, and not sure/refused. It is important to note that for these questions and all other questions in the questionnaire any answers pertaining to not sure/refused or other were not read to the respondents, however, they were answer choices available to the interviewers when the respondent offered them as their answers. Next, the survey asked a series of questions pertaining to the respondent?s recall of advertising associated with the Amendment One campaign. Question 4 asked if the respondent recalled seeing or hearing any Amendment One advertising; available answers were yes, no, and not sure. If the respondent answered Question 4 with a no or not sure, the next two questions (Questions 5 and 6) were skipped because those questions dealt with specific recall of the advertising from the campaign. Question 5 asked a slightly more specific advertising question in that the respondents were asked if they recalled any advertising that explained how seniors would 89 benefit if the tax plan were to pass. The possible responses to that question were yes, no, and not sure. The sixth survey question then asked how much impact the Amendment One campaign advertising had on how the respondent planned to vote. The possible answers for Question 6 included strong impact, moderate impact, very little impact, and not sure. The remaining 10 items on the survey were meant to collect demographic information from the respondent. Question 7 asked about the respondents? partisanship with possible answers choices of strong Democrat, leaning Democrat, strong Republican, leaning Republican, completely independent, and not sure/refused. Question 8 asked if the respondent owned their own home. Yes, no, and not sure were the three possible answer choices for Question 8. Question 9 asked respondents the highest grade of school they completed. The possible answers for Question 9 were grade school or less, some high school, graduated high school, vocational school/technical school, some collage (2 years or less), more than 2 years of college, graduated college, postgraduate degree, and refused. Question 10 asked the respondent about their voting history and how many elections they vote in. Responses for Question 10 included vote in all elections, most elections, about half, less than half, usually do not vote, and not sure/refused. Race was the focus of Question 11, which asked what the respondent?s ethnic background was. Black, White, some other race, and refused were the possible answer choices for this question. Employment was the focus of Question 12, which asked respondents what their employment status was. The available choices for Question 12 included retired, 90 employed full-time, employed part-time, unemployed, laid off temporarily, homemaker, other, and refused. Question 13 asked what the total household income for the prior year (2002) was before taxes and other deductions. The possible answer choices for question 13 were less than 20,000, 20,000 to 30,000, 30,000 to 40,000, 40,000 to 50,000, 50,000 to 75,000, 75,000 to 100,000, 100,000+, and not sure/refused. Question 14 asked the respondent?s age on their last birthday. Question 15 asked if the respondent had a relative or close friend currently in a nursing home. Yes, no, and not sure, were the possible responses to that question. Question 16 was entered by the interviewers listening for verbal cues and recording the gender of the respondent based on the perception of the gender of the person they were speaking with. Male and Female were the two possible answer choices for that item. Question 17 was a number indicating into which experimental group the respondent fell. This was provided to the survey house along with the 5,000 unique telephone numbers for use in conducting the survey. Data Analysis The primary statistic employed was chi-square. Chi-square analysis allows for making comparisons between two discrete variables. For these data, one discrete variable was the six categories of treatment groups. This variable was the independent variable. There were two discrete dependent variables. One was the intention to vote with two categories (yes/no), the second being the likely direction of the vote on the amendment 91 (for/against). Thus, two 2 x 6 chi-square analyses were conducted. One dealt with whether there were differences among the six groups with respect to their intention to vote and a second analysis on whether they would vote for or against the amendment. Similar to the F-ratio in analysis of variance, the chi-square statistic only indicates if there is an overall difference(s) among the groups in relation to the dependent variable. In chi-square, regardless of statistical significance, observation of the standardized residuals provides information about the contribution of each group to the overall chi- square. In this aspect, the standardized residuals are similar to post hoc tests in analysis of variance. Several supplemental analyses were also conducted with the objective of probing the data further. These included additional chi-square tests on other potential dependent variables that were included in the survey questionnaire as well as logistic regression. Where the results of these analyses aided understanding of the results, they are reported. The .05 level of statistical significance was used for all statistical tests. Chapter 4 presents a further discussion of the analyses that were employed and the results of those analyses. Threats to Validity Analyses were conducted to screen the data for differences on the demographic variables among the six groups. Tests of the assumptions underlying the statistical procedures were conducted and revealed no significant divergence from the assumptions. It was thus judged that the randomization of this experiment ensured, to the extent possible, that all groups were equal with the exception of the treatments (or lack of treatment in the case of the control group). The results of these procedures helped to rule 92 out many of the internal and external validity issues described below that might otherwise exist in different types of experiments. Internal Validity The field experiment design (posttest only control-group design) with random treatment assignment helps rule out nearly all threats to internal validity. There are eight threats to internal validity: History, Maturation, Instrumentation, Testing, Statistical Regression, Selection, Experimental Mortality, and Additive and Interactive Effects (Campbell & Stanley, 1966). The randomization of both the group assignments and data collection help to rule out the threats of History, Maturation, Statistical Regression, Selection, and Experimental Mortality. The Testing threat was ruled out because all groups in the experiment were only tested once. The Instrumentation threat was ruled out because the survey instrument used as the posttest did not change during the administration of that posttest. Finally, because there are no overt threats to internal validity, it is not possible to have an additive or interactive effect among those threats and, therefore, the Additive and Interactive Effects threat is also ruled out as a possible rival hypotheses. External Validity One goal in conducting research is to generalize the results to larger populations. Therefore it is important to consider the larger population when making generalizations, because the larger population can vary depending on the perspective of the researcher. Threats to a researcher?s ability to generalize back to the population are known as threats to external validity. 93 In this experiment, the population was all voters aged 65 and older who were registered to vote in Alabama in July 2003. However, it is logical to hope that these experimental results could extend beyond that specific group of seniors at this time and place with these specific treatments. This is where expanding the population complicates the external validity of the research. For example, it is not uncommon for other researchers to conduct further research that crosses different groups of people at different times and places with various similar and different treatments. It is through this ever- expanding set of research that building confidence in external validity is possible. Therefore, extending the external validity of this research to groups other than senior Alabama voters should be done with caution until further research has been conducted on other seniors in other parts of the country in different types of electoral situations. Campbell and Stanley (1966) indicated that there are four different primary threats to external validity: the reactive or interaction effect of testing, interaction effects of selection biases and the experimental variable, reactive effects of experimental arrangements, and multiple treatment interference. In their discussion about these external validity threats, they further indicated that the experimental design used in this experiment (posttest only control-group design) addresses the first threat mentioned above because it does not have a pretest to interact with the ?respondent?s sensitivity or responsiveness to the experimental variable? (Campbell & Stanley, p. 6). They further explained that the last of the four threats discussed above also does not apply to this posttest-only design because each group is not given multiple treatments and measurements over time (Campbell & Stanley). That renders the second and third threats as germane to this experimental design. The second threat to external validity (interaction 94 effects of selection biases on the experimental variable) was addressed through the use of randomization during the selection process. The third threat to external validity discussed by Campbell and Stanley (reactive effects of experimental arrangements) was addressed by the fact that this was a random sample of geographically diverse (statewide) respondents responding to a typical campaign survey. Although Campbell and Stanley?s discussion of (1966) external-validity threats did not delve into specific issues that might surround a researcher?s ability to generalize to other persons, places, and times, or across treatments, the use of randomization and field experiments can help reduce the threats to a researcher?s ability to generalize to the population from which the sample for the experiment was drawn (Shadish, Cook, & Campbell, 2002). In other words, the randomization (within the limits of the sample error) improves a researcher?s ability to generalize to the people in the population (senior registered voters), the population?s place (Alabama), the time of the experiment (2003), and the treatments (targeted direct mail and telephone calls). Beyond that, a researcher must use logic and other research to extend the generalizations to other times, places, people, and treatments. Although generalization might be possible to some degree, Campbell and Stanley asserted that generalization is never fully justified. Ultimately, each researcher or reviewer must personally decide how generalizable the results of certain research are to other people, places, times, and treatments. This researcher believes the results of this experiment are generalizable to the senior voters of Alabama at the time of the election specific to the issues associated with the election. Arguably, the research is generalizable to seniors across the country with similar messages. However, it will take future research to extend the generalizability of 95 this research. In other words, it is future research that will improve the generalizability of this research in the same way that this research has improved the generalizability of past randomized field experiments. Summary This chapter provided a discussion of the experiment and the experimental design chosen and created by the campaign consulting organization. Chapter 3 addressed the research design, the experiment including the treatment groups, the processes and instruments used to collect the experimental data, and a discussion surrounding the various internal and external validity issues that might exist. This background provides a solid foundation for future researchers to duplicate and critique the methods used in this research. The next chapter provides a comprehensive discussion of the processes and analyses performed for this research. 96 CHAPTER 4: RESULTS The purpose of this study was to determine the impact of targeted messages on vote choice and mobilization. The randomized field experiment was conducted during a statewide referendum for a revenue-increasing measure in an Alabama Special Election in 2003. The experiment focused on determining the influence that a targeted message, developed during the course of a live campaign, had on the senior voters subpopulation in terms of mobilization and vote choice. This chapter reports the findings of that experiment. The chapter begins with a description of the variables that were used in the analysis followed by a general discussion of the experimental data set in terms of the personal and political characteristics of each experimental group. Following that, the Univarate Analysis section provides a discussion of the tests done to ensure that randomization created similar groups in relation to those personal and political characteristics of each experimental group. The Bivarate Analysis section that follows focuses on the methods and results used to answer the questions that are the focus of this dissertation. The chi-square statistical test was the primary test used to determine if the treatments had any significant impact among the different experimental groups. Chi- square was limited to the experimental groups and their likelihood to vote, as well as how they planned to vote on Amendment One. Chi-square was followed by a regression analysis that included demographic and political variables in addition to the experimental 97 groups as predictors. Several additional exploratory and regression analyses are also reported. These analyses test if the participants recalled advertisements about Amendment One rather than how they planned to vote. Finally, a short summary is provided at the end of this chapter. Study Variables The dependent and control variables for this analysis were survey-questionnaire items collected in a preelection survey. The independent variables represented the type and frequency of the messages each experimental group received. Table 5 provides a summary of the variables used in the analyses that follows. Univariate Analysis Demographics were collected on respondents? personal and political characteristics. These are provided in Tables 6 and 7. In addition to describing the participant characteristics, the six groups were compared through statistical analysis to assure randomly assigned groups were similar. Personal Demographics Table 6 shows the personal demographics for each of the six experimental groups. It was expected that random group assignments would result in the groups being equal at the beginning of the experiment. As such, any subsequent differences could be attributed to the treatment and not be due to initial group differences that may also have influenced the dependent variables. Initial interest was thus in the percentages across groups to identify variables where the percentage in a group was significantly more or less than the 98 other groups. This was accomplished through visual inspection as well as statistical analysis. Table 5 Study Variables Variable Source Description Independent variables Experimental group Randomized Groups There are six groups each with a diferent treatment or lack of treatment used to determine if mesage targeting and contact frequency impact the Dependent Variables. Dependent variables Vote intention Survey Q2 As you may know, Alabamians wil go to the pols on September 9th and vote on the tax and acountability package known as Amendment 1. As things now stand, wil you likely be voting in this election or wil you not likely be voting? Vote choice Survey Q3 If this election were held today, would you likely vote for or against Amendment 1? Control variables Recal Survey Q4 Do you recal seing or hearing any advertising that suported Amendment 1? Seniors benefit Survey Q5 Do you recal whether any of the advertisements for Amendment 1 explain how seniors would benefit if the tax plan pased? Advertising impact Survey Q6 Would you say this advertising had a strong impact on how you plan to vote, a moderate impact or very litle impact on how you plan to vote? Partisanship Survey Q7 Do you consider yourself to be a Democrat, a Republican or completely independent? Home ownership Survey Q8 Do you own your own home? Education Survey Q9 What is the last grade of schol you completed? Voter participation Survey Q10 Would you say that you vote in al elections, most of them, about half, les than half or do you usualy not vote? Race Survey Q1 What is your ethnic background?are you White, Black or some other race? Employment Survey Q12 What is your curent employment status? Income Survey Q13 What was your total household income in 202 before taxes and other deductions? Age Survey Q14 What was your age on your last birthday? Gender Survey Q15 Survey item Q15 asked the interviewer to make a judgment based on the sound of the respondents voice as to their gender. Note. Apendix A contains a copy of the questionaire and the posible responses. 99 The percentages are shown for each demographic in Table 6. As expected, due to the random assignment, the percentages were similar across the groups. Visual inspection was followed by chi-square analyses that tested for statistical differences in the percentages. This analysis resulted in identifying one variable (Gender) where the group percentage was different from the other groups (!2 (5) = 10.94, p < .05). Observation of the percentages for gender shows that that Group E contributed the most to the difference where there was a larger percentage of men (48%) and smaller percentage of women (52%) than the other groups. These percentages were nearly 10% different from the other groups. Group A also contributed to the significant chi-square where it may be seen that the percentage of men (30%) was smaller than the other groups. No other chi-square analyses approached statistical significance for the remaining personal demographic variables. Table 6 shows that participants were primarily White (over 80% in each group). Most of the participants? highest level of educational attainment was high school or less and as one might expect, the majority of the participants were retired, as is shown by each group having more than 80% in the retired category. Over 90% of each group indicated they owned their homes and more than half of the participants in each group indicated their income was less than $30,000 a year. Additionally, about 20% of each group indicated they had a friend or relative who was in a nursing home. 100 Table 6 Senior Participants Personal Demographic Control Variables by Group Group A(123) B(136) C(140) D(126) E(129) F(96) Personal demographics % % % % % % Gender Men 30 39 32 38 48 37 Women 70 61 68 62 52 64 Race White 88 86 82 83 82 85 Black 8 12 11 13 12 8 Other 4 2 7 5 6 6 Education Grade schol 7 5 9 6 10 14 Some HS 12 13 15 14 12 14 High schol 37 40 34 32 27 28 Vocational 3 2 2 6 1 3 Colege/some 10 13 23 19 12 18 Colege/> 2 years 6 6 3 5 10 3 Colege/graduated 16 13 9 10 18 14 Colege/post grad 10 9 7 10 12 6 Employment Retired 81 85 84 90 88 94 Ful time 7 5 7 5 6 2 Part time 3 3 4 2 2 1 Other 8 7 4 2 3 3 Income (Thousands) < $20 47 40 35 36 24 40 $20?$30 19 15 18 21 33 29 $30?$40 11 13 20 16 14 11 $40?$50 7 9 9 9 11 6 $50?$75 9 12 10 15 14 6 $75?$100 5 6 6 3 3 4 > $10 3 6 2 1 3 6 Home Ownership Yes 93 91 91 90 93 94 No 7 9 9 10 7 6 Nursing Home Friend/Relative Yes 19 24 24 22 22 20 No 81 76 76 78 78 80 Note. Group Ns are in parentheses. A = No mesage; B = One mail targeted mesage; C = Two mail targeted mesages; D = One mail targeted and one mail generic mesage; E = Two mail targeted and one telephone targeted mesage; F = One mail generic mesage. 101 Political Control Variables Two political demographics were collected and are shown in Table 7. Observation of the rows for partisanship indicates that the group percentages were similar for the three categories, although the Group E percentage for Democrats was somewhat lower (26%) than the other groups. The subsequent chi-square analysis was not statistically significant. The second political demographic question asked participants if they voted in previous elections. Nearly 95% of the seniors in each group reported that they voted in all elections with very few reporting that they voted in half, less than half of the elections, or not at all. Table 7 Senior Participants Political Control Variables by Group Group A (109) B (121) C (125) D (14) E(16) F(90) Political control variables % % % % % % Partisanship Democrat 39 42 31 38 26 32 Republican 31 28 33 37 43 30 Independent 30 30 36 25 31 38 Vote in elections Al or most 93 93 95 95 97 95 Half or les 6 4 4 4 2 4 Do not vote 2 4 1 1 2 1 Note. Group Ns are in parentheses. A = No mesage; B = One mail targeted mesage; C = Two mail targeted mesages; D = One mail targeted and one mail generic mesage; E = Two mail targeted and one telephone targeted mesage; F = One mail generic mesage. In summary, the demographics provide a profile of the participants on a number of variables of interest as well as adding assurance that the random assignment to groups resulted in equalizing the groups on variables that may have influenced the dependent 102 variables. The next section provides the results of the analyses related to the research questions. Bivariate Analysis This study addresses two research questions. Q1. Did the type and frequency of targeted campaign messages result in increasing voters? intention to vote? Q2. Did the type and frequency of targeted campaign messages influence the likelihood that potential voters would vote in favor of a targeted-message issue? The dependent variable for Q1 is the survey-questionnaire item asking respondents to indicate their likelihood of voting (see Table 4). The dependent variable for Q2 above is the survey-questionnaire item asking respondents to indicate their probability of voting for Amendment One. The independent variable for both questions above was the treatment that each experimental group received. The data related to each of the two questions were categorical. Participants in one of the experimental groups (A?E) were placed into one of two categories for Q1 depending on their intentions to vote?likely to vote or not likely to vote. Likewise for Q2 they were placed according to whether they were likely to vote for or against Amendment One. This resulted in two 2 x 6 contingency tables: one for their intention to vote in the election and one for whether they would vote for the amendment. Two 2 x 6 chi-square analyses were employed, one for each research question. Chi-square is the most common statistical method used to analyze categorical data and examines the association between two variables (Babbie, 2007). The null hypotheses 103 tested were that there would be no association between the independent variable and the dependent variables. The .05 level was the criterion used to reject the hypotheses. Table 8 provides the results for Q1 (Vote Intention). It was expected that if the targeted messages were effective, the participants in the groups who received the targeted messages (Groups B, C, D, and E) would report that they would be more likely to vote than those participants who did not receive a targeted message (Groups A and F). Further, if frequency had an impact on mobilization there would be differences among Groups B, C, D, and E in their intention to vote dependent on the contact frequency. Table 8 provides the actual (or observed) responses by group. For example, in Group A (the control group), 104 participants indicated that they intended to vote while 9 responded that they were unlikely to vote. It can be seen that intentions are similar across each of the groups. The expected value is of most interest and yields the most information. It indicates what the expected number would be if the null hypothesis is true. Thus, the greater the difference between the actual and expected values, the more likely the null hypothesis will be rejected and the research hypothesis supported. The residual shows the contribution of each of the 12 cells to the chi-square. It has been standardized, which allows for direct comparisons among the cells. The summary chi-square is shown at the bottom of the table and is not statistically significant at the .05 level. Thus, the null hypothesis was not rejected and from a statistical-significance perspective there was no support for the expectation that the type and frequency of messages affected a respondent?s reported vote intention. It should be noted that it was not known at the time the experiment was conducted that nearly 100% of the participants would report that they had voted in all previous elections (Table 7). 104 Assuming this was true, there was little opportunity to increase their likelihood to vote since these respondents would have reported having voted regardless of the type and frequency of targeted messages. From a statistical standpoint, variation is required to detect an effect. In considering Question 1, over 95% of the participants reported that they had voted in all previous elections. Consequently, Research Question 1 was no longer considered because this population of voters were mobilized to begin with and there was little room for variation in the likelihood to vote regardless of messaging. Table 8 Intention to Vote in the September Election by Group (n = 701) Group A (13) B (129) C(126) D (19) E (124) F (90) September election n % n % n % n % n % n % Likely to vote Actual 104 92% 123 95% 120 95% 114 96% 115 93% 80 89% Expected 106 121 118 111 116 84 Residual -0.2 0.2 0.2 0.3 0 -0.5 Not likely to vote Actual 9 8% 6 5% 6 5% 5 4% 9 7% 10 11% Expected 7 8 8 8 8 6 Residual 0.6 -0.8 -0.7 -1 0.4 1.8 ! 2 (5)= 6.1 ns Note. A = No mesage; B = One mail targeted mesage; C = Two mail targeted mesages; D = One mail targeted and one mail generic mesage; E = Two mail targeted and one telephone targeted mesage; F = One mail generic mesage. Group Ns are in parentheses; ns = Not statisticaly significant at the .05 level. Table 9 shows the results for Q2 (Vote Choice). For this question it was expected that if the targeted messages were effective, the participants in the groups who received the targeted messages (Groups B, C, D, and E) would report that they would be more likely to support Amendment One than those participants who did not receive a targeted message (Groups A and F). If frequency of messages was effective at changing vote 105 choice, there would also be differences among Groups B, C, D, and E in their support, depending on the number of messages received. It can be seen in Table 9 that the chi-square was not statistically significant and the null hypothesis of no association was not rejected. This indicates that the likelihood of supporting Amendment One was not affected by the various types and frequencies of messages presented to the six groups. In looking at those who indicated they would support the amendment, both Group B and Group E had the largest differences from the expected values. Group B had 7 fewer supporters (28) than expected (35) and 7 more against (71) than expected (64). Group E had 5 more supporters (37) than expected (32) and 5 fewer against (55) than expected (60). However, these differences were not large enough to indicate a significant difference among groups using the chi-square statistic. The standardized residuals show the contribution of each group to the small differences that did occur. In summary, the statistical results showed no support for the effect of type or frequency of message associated with group membership and the reported likelihood to vote or to vote for Amendment One. In respect to Q1 there was little opportunity to show an effect because this sample of seniors, as with seniors in general, were likely to vote anyway. Three supplemental analyses are presented in the next section that further explores the data beyond the two primary research questions. Two of the analyses deal with the influence of overall campaign advertising from all sources on Amendment One and not just the messaging used in the experiment. The third analysis is a logistic regression that extends the chi-square findings by including personal and political 106 demographic variables with the experimental groups in relation to support for Amendment One. Table 9 Support for Amendment by Group Group A (81) B (9) C (94) D (8) E (92) F (75) September election n % n % n % n % n % n % For Actual 28 35% 28 28% 34 36% 34 39% 37 40% 24 32% Expected 28 35 33 31 32 26 Residual 0 -1.1 0.2 0.6 0.9 -0.4 Against Actual 53 65% 71 72% 60 64% 54 61% 55 60% 51 68% Expected 53 64 61 57 60 49 Residual 0 0.8 -0.1 -0.4 -0.6 0.3 ! 2 (5)= 3.94 ns Note. A = No mesage; B = One mail targeted mesage; C = Two mail targeted mesages; D = One mail targeted and one mail generic mesage; E = Two mail targeted and one telephone targeted mesage; F = One mail generic mesage; Group Ns are in parentheses; ns = Not statisticaly significant at the .05 level. Exploratory Analysis During the course of the campaign leading up to the election there were numerous television and radio advertisements by and between the various groups supporting and opposing Amendment One. Thus, regardless of whether participants participated in this study, all voters had many opportunities to become aware of the Amendment One issues. However, due to the randomized nature of this experiment, these unknown extraneous influences should be similar across all groups, as are the known demographics discussed above. When interviewing this study?s participants, one of the questions asked if the person recalled seeing or hearing any advertisements that supported Amendment One. Of 107 the original 854 participants, 520 recalled advertisements on the issue. They were then asked if they also remembered if any of the advertisements explained how seniors would benefit if Amendment One passed. This raised the following question: Was group membership associated with recalling campaign advertising about how Amendment One would benefit seniors? Table 10 gives the chi-square analysis resulting from the above question. It can be seen that members of each of the six groups recalled advertising. For the analysis they were categorized concerning whether they recalled if the advertising explained how seniors would benefit should Amendment One pass. It can be seen that members of Groups A and F recalled advertising on the amendment issue, although they did not receive any targeted messages in this study. The chi-square was statistically significant (!2 (5) = 11.65, p < .05) indicating an association between group membership and the recall of advertising associated with seniors. Observation of the residuals shows that Group A contributed the most to the association. Further, the largest association (residual = 2.3) is with the Group A participants that recalled the advertising but did not recall if it explained how seniors would benefit. This finding is of interest in a subtle way because Group A was the control group and received no messages in this experiment. The data in the previous analysis on recalling advertising was broken down further with the focus being on those who not only recalled advertising about the amendment but also recalled that it explained how seniors would benefit. A follow-up item in the survey asked participants if the advertising had an impact on how they planned to vote on Amendment One. This raised the following question: For those who 108 recalled advertising about how Amendment One would benefit seniors, was there an association between group membership and the impact of the advertisements on how they might vote? Table 10 Advertising and How Seniors Would Benefit by Group Group A (80) B (8) C (104) D (91) E (98) F (59) Recaled explanation n % n % n % n % n % n % Yes Actual 42 53% 58 66% 75 72% 68 75% 68 69% 39 66% Expected 54 59 70 61 66 40 Residual -1.6 -0.2 0.6 0.9 0.3 -0.1 No Actual 38 48% 30 34% 29 28% 23 25% 30 31% 20 34% Expected 26 29 34 30 32 19 Residual 2.3 0.2 -0.9 -1.2 -0.4 0.2 ! 2 (5)= 1.65* Note. A = No mesage; B = One mail targeted mesage; C = Two mail targeted mesages; D = One mail targeted and one mail generic mesage; E = Two mail targeted and one telephone targeted mesage; F = One mail generic mesage; Group Ns are in parentheses; * = Statisticaly significant at the .05 level. The results for this analysis are provided in Table 11, composed of those participants (in the top half of Table 10) who recalled advertising that explained the benefits of Amendment One. The chi-square was statistically significant (!2 (5) = 21.71, p < .05) showing that there was a relationship between group membership and recalling advertising on how seniors would benefit from passage of the amendment. Because the chi-square is an overall test representing all the categories a significant ! 2 does not identify which groups are the main contributors to statistical significance. Similar to post hoc tests in analysis of variance, the standardized residual in chi-square may be used to determine which groups are the major contributors. If a standardized residual is 2.00 or 109 greater it can be considered to be a major contributor and analogous to a z-score for a continuous variable where a z-score of 2 or greater is considerably different from the other scores in the distribution (Field, 2005). As may be seen, Group B/Strong contributed the most to the association (residual = 3.2). No other residuals approached 2.00 but each contributed a small amount that resulted in the chi-square being statistically significant. The two analyses on advertising and its impact on planning to vote both tend to suggest that advertising has some influence on voters. Table 11 Advertising and Impact on Planned Vote by Group Group A (7) B(94) C (104) D (93) E (90) F (5) Impact n % n % n % n % n % n % Strong Actual 9 12% 27 29% 10 10% 11 12% 14 16% 9 16% Expected 12 15 16 15 14 9 Residual -0.9 3.2 -1.5 -0.9 0 0.1 Moderate Actual 17 22% 14 15% 23 22% 18 19% 26 29% 10 18% Expected 16 20 22 20 19 12 Residual 0.2 -1.3 0.2 -0.4 1.6 -0.5 Litle Actual 51 66% 53 56% 71 68% 64 69% 50 56% 36 65% Expected 49 59 66 58 57 34 Residual 0.3 -0.8 0.6 0.7 -0.9 0.2 !2(5)= 21.71* Note. A = No mesage; B = One mail targeted mesage; C = Two mail targeted mesages; D = One mail targeted and one mail generic mesage; E = Two mail targeted and one telephone targeted mesage; F = One mail generic mesage; Group Ns are in parentheses; * = Statisticaly significant at the .05 level. 110 Regression Analysis The primary objective was to determine if the experimental treatments had an effect on the likelihood to vote and if so, whether the participant was likely to vote for or against Amendment One. Concerning the likelihood to vote, preliminary analysis showed that nearly all of the seniors (+/- 95%) reported that they had voted in most or all of the previous elections. Thus, it was no surprise to find that the same percentage reported that they were likely to vote in the September Amendment One special election. Given this finding, the likelihood to vote was no longer considered to be a dependent variable. Although the chi-square results reported above have established that the frequency and type of targeted messages had little effect on likelihood to support the amendment, it was decided to explore further using logistic regression analysis. Logistic regression permits the overall prediction of group membership. Predictors may be continuous, categorical, or a mixture of the two. For these data group membership was the likelihood to vote for or against the amendment and thus designated as the criterion or dependent variable. It was felt that one or more of the treatment groups, when combined with demographic variables, might contribute to the relationship with the criterion. Thus, the predictors or independent variables were the experimental groups: party identification, gender, age, education, and income. Also, included as an independent variable was whether advertising about the amendment was recalled. Experimental group, party identification, recall, and gender were categorical variables while age, education, and income were continuous. Even though logistic regression accepts categorical variables, they are converted to continuous variables for statistical analysis. The conversion may be done prior to the analysis by the researcher or 111 at the time of the analysis by designating which variables are categorical when setting up the analysis. In either case, categorical variables are converted to continuous variables through dummy coding. This procedure results in placing a participant in, or not in, a particular group. A ?1? designates in and a ?0? designates not in. This results in a ?score? of 0 through 1 and thus satisfies the assumption for a continuous variable. Where there are three or more categories, one is identified as the comparison group and is omitted from the analysis. However, the resulting regression coefficients for the dummy coded variables represent deviations from the comparison group on the criterion and may be interpreted in that context. In an experiment, the control group is the logical comparison group, as with these data. For partisanship, Republican was designated as the comparison group. This designation was because the amendment was proposed and supported by the Republican administration that was in office at the time. Table 12 presents the overall results of the logistic regression analysis. First, the actual numbers voting on the amendment were 127 in favor and 188 against for the 315 participants for which there are complete data. Hypothetically, if one wished to predict future results under similar circumstances and had no predictors, the ?best? prediction would be that everyone would vote against the issue. Everything being equal, one would be correct for 60% of the voters (188/315 = .60). By using potential predictors, the logistic regression determines if successful predictions will be better than 60%. The likelihood value (Table 12) is the result of the process that determines the best fitting model for the data. Chi-square is used to test the model. It may be seen that the chi-square was statistically significant, ! 2 (12, N =315) = 35.99, p < .05. This indicates that the set of predictors did reliably distinguish between those likely to vote for 112 and against the amendment using the .05 level of probability. The Nagelkerke R 2 is somewhat similar to the squared correlation in standard regression in that the greater its magnitude the stronger the relationship. The lower half of Table 10 shows the predictions when applying the model to the data. Predictions were correct for only 45% of those who had indicated being in favor of the amendment and better than 82% for those who indicated that they were against the amendment. The total percent correct was 67. Thus, while not impressive, the prediction equation improved the overall correct prediction by seven percentage points over just ?guessing? as described in the hypothetical situation above. Table 12 Overall Summary for Logistic Regression on Likelihood to Vote For/Against Amendment One (N = 315) -2 Log likelihod !2 Nagelkerke R 2 38.8 35.9* 0.15 Actual Predicted for Predicted against percent corect For 127 57 70 45 Against 188 34 154 82 Overal percent corect: 67 * p < .05. The overall test for the set of predictors was followed by determining the contribution of each predictor to the model. The Wald statistic (Table 13) is used for this purpose and is analogous to the t-ratio used in multiple regression. Gender, age, education, and income contributed the most to the regression model. Their probabilities were each less than .05 and are shown with asterisks in Table 13. None of the experimental or political groups or advertisement recall approached statistical significance and contributed only small amounts to the overall model. 113 Table 13 Logistic Regression Analysis Predicting Vote On Amendment One (N = 315) Variable B SE Wald !2 p Ods Ratio Group A B -0.31 0.42 0.5 0.46 0.73 C -0.24 0.43 0.31 0.58 0.79 D 0.06 0.43 0.02 0.90 1.06 E 0.13 0.43 0.10 0.75 1.14 F 0.04 0.47 0.01 0.93 1.04 Party Republican Independent 0.16 0.30 0.28 0.60 1.17 Democrat -0.26 0.31 0.69 0.41 0.7 Recal 0.29 0.39 0.58 0.45 0.75 Gender 0.60 0.27 5.03 0.03* 1.82 Age 0.04 0.02 4.95 0.03* 1.04 Education 0.21 0.07 9.62 0.01* 1.23 Income 0.19 0.08 4.94 0.03* 1.21 Note. A = No mesage; B = One mail targeted mesage; C = Two mail targeted mesages; D = One mail targeted and one mail generic mesage; E = Two mail targeted and one telephone targeted mesage; F = One mail generic mesage; * these items contributed most to the regresion model. Although the relationships were small between the predictors and the criterion for any of the variables, the odds ratios help to explain the findings further. The nearer to one the ratio is the less the relationship with the criterion variable. For the groups in the experiment, with the control group designated as the comparison group, the odds ratios for groups B (.73) and C (.79) indicate that those two groups were less likely to vote for the amendment than was the control group. Groups D (1.06), E (1.14), and F (1.04) were more likely to vote for the amendment than was the control group. However, these tendencies were very slight because the ratios were close to one. For party identification, with Republican as the comparison group, the odds for independents (1.17) were that 114 they were slightly more likely to vote for the amendment than were the Republicans. As might be expected, considering that the amendment was proposed by the sitting Republican administration, the Democrats were less likely to vote for the amendment than were the Republicans (Odds ratio = .77). Observation of the remaining odds ratios shows that all but gender (1.82) were close to 1 showing very little relationship with the criterion. Based on the coding, the odds ratio indicates that females were more likely to vote for the amendment than were males. Likewise, the greater the age, education, and income, the greater was the tendency to plan to vote for the amendment. On further investigation using additional regression models, the income effect was largely attributed to males and the education effect was largely attributed to females. The odds for recall show that those who recalled advertisements were slightly less likely to vote for the amendment. Several additional logistic regression analyses were also run and are summarized below. Appendix G provides the statistical tables associated with each of the analyses. One analysis included the same variables as reported above but replaced the overall recall variable with the variable that asked whether they recalled that the advertisements explained how seniors would benefit from the amendment if it passed. The results of this analysis were the same for the experimental groups, which showed no relationships. Only education and age contributed statistically to the model and indicated that the greater the education and age, the more likely to vote for the amendment. However, although statistically significant, the relationships were weak (See Tables G1 and G2, Appendix G). 115 Two logistic regression analyses were done using the same independent variables, except for the recall variables. In these analyses the overall advertising-recall variable was the dependent variable in one analysis while the recall variable about explaining how seniors would benefit was the dependent variable in the other analysis. The results of the analysis on recalling advertisements overall showed no relationships for the experimental groups. The only variable that was statistically significant was education, indicating that the greater the education, the more likely the recall of advertising (See Tables G3 and G4, Appendix G). The analysis of recalling if senior benefits were explained as the dependent variable resulted in two statistically significant predictors. One was income and indicated that the greater the income, the more likely to remember advertisements that explained benefits. The other was experimental group D. This group received one targeted message and one generic message (See Tables G5 and G6, Appendix G). Regression analyses were performed in anticipation that the other variables in conjunction with the experimental groups might lead to one or more of the groups emerging as having an effect on how the participants planned to vote. While several of the predictors were related (gender, age, education, and income) the results of the logistic regression analyses for the experimental groups coincided with the chi-square analysis, indicating that the frequency and type of target messaging had little relationship with the likelihood of how one planned to vote on Amendment One. 116 Summary In conclusion, both the chi-square and logistic regression analysis indicate that the experimental treatments had no significant impact on either reported vote intention or reported vote choice. The chi-square analysis indicated no significant relationship among the experimental groups while the regression analysis considered other variables to determine if there were subgroups for which the treatments might have an impact. The overall significance or impact of these findings will be discussed in the next chapter. There were significant findings, as shown in the supplemental analysis, associated with those who recalled advertising explaining how Amendment One would benefit seniors and the experimental groups. Similarly, there were significant differences between the groups and the responses related to the impact that the Amendment One advertising would have on a respondent?s vote choice. However, as will be discussed in the next chapter, these findings are not easy to interpret and leave room for future research. 117 CHAPTER 5: DISCUSSION Introduction This dissertation used secondary data collected during a live 2003 political campaign in Alabama. The special election was for a revenue-increasing referendum item, an attempt to address a perceived significant budget shortfall that would take place in the coming years? budgets. One of the campaign-management teams that was involved in the campaign in support of Amendment One conducted a randomized field experiment to determine the impact of targeted messages on both vote choice and vote intention of senior voters across Alabama. Ultimately, Amendment One was soundly defeated, but the results of the experiment, along with relevant literature, have been reported in this dissertation. The first chapter of this dissertation began with a background discussion of the relevant theory and hypotheses associated with this field experiment. The second chapter focused on the extant literature on campaign field experiments and their impact on vote choice and mobilization with a primary focus on referendum-based campaign experiments. The third chapter discussed in detail the experimental methods used during the Alabama Seniors Experiment, while the fourth chapter provided a detailed discussion of the results associated with that experiment. This, the fifth chapter in the dissertation, provides a short summary of the experiment, a discussion regarding the strengths of this research, a discussion of the limitations of the findings, how those findings fit into the 118 academic literature and the professional world of campaign consulting, as well as suggestions for future researchers. Early in the life of the campaign for Amendment One, a benchmark survey was conducted by the pro-Amendment One campaign that indicated senior voters responded in a positive manner to the message that seniors would pay no additional taxes should Amendment One pass. Based on the analysis of the benchmark survey, the campaign decided to distribute the ?no additional taxes? message to all registered senior voters across the state. A list of senior voters was purchased from a voter-list vendor and 5,000 senior-voter households were randomly assigned to one of six experimental groups. One group (Group A) was assigned to be the control group while the remaining five groups (Groups B?F) became the treatment groups. Each of these groups received at least one treatment with either a targeted message or a generic message that was delivered through direct mail postcards that are typical of political campaigns today. Group B received only one targeted direct mail message (Seniors 1) while Group C received two targeted direct mail messages (Seniors 1 & 2). Group D received a targeted direct mail message (Seniors 1) and a generic biased direct mail message (Working Families). Group E received three targeted messages, two of which were direct mail pieces (Seniors 1 & 2) and one targeted prerecorded call with a similar message to that of the targeted direct mail pieces. Group F received only the generic direct mail piece (Working Families). The experiment was designed to compare the differences that message targeting and message frequency had, if any, on senior voters? vote choice and vote intention. The dependent variables were questionnaire items on a preelection survey that was conducted during the last few days of the campaign. The survey participants were randomly selected 119 from the treatment sample for each experimental group. Univariate, bivariate, and multivariate analyses were used to analyze the data with a specific focus toward testing the hypotheses that the type and frequency of targeted messages would have a positive persuasion effect and a positive mobilization effect. The analysis produced statistically insignificant results, indicating that there was no statistical support for either of the hypotheses. Study Strengths This study has several strengths worth mentioning. First, this experiment was the first of its kind to focus on senior voters during a live campaign. This is important because the senior voter population is expected to continue growing as the baby boomer generation continues to age in the coming years. Second, this dissertation expands the limited research associated with referendums and message targeting, message frequency, and message synergy as they relate to vote choice and vote intention. In fact, as it relates to vote choice, this experiment expands a very limited subset of field experimental research that currently exists in that area. Third, this dissertation clarifies and outlines a list of variables that researchers might consider in future randomized campaign field experiments. Specifically, special attention was given to the terms partisan and nonpartisan as they have been confusingly used in the past. Instead, this study uses the terms biased and unbiased to describe the type of experimental message, while using the terms partisan and nonpartisan to describe the environment surrounding a given campaign with respect to whether a voter can use partisan cues to make their vote choice. Fourth, this dissertation illustrates that campaign professionals have conducted and will 120 continue to conduct randomized field experiments and that academics can work in conjunction with those campaign professionals to expand the campaign-effects literature and help to bridge the divide that exists between them. Finally, this dissertation answers the calls of previous authors to conduct and publish the results of campaign field experiments, regardless of the findings, for it is through continued publication of experiments that the community will answer the question of when, how, and to what extent campaigns might influence the outcome of elections. Review of Findings The experimental literature on campaigns in the past has used the terms partisan and nonpartisan to describe both the type of campaign and the type of message that was delivered to an experimental participant. This dissertation advocates for a separation of the terms used to describe messages from the terms used to describe the campaign environment. This would allow researchers to use the terms partisan and nonpartisan when partisanship is a factor in the minds of the voters at the time they cast their ballot. This leaves open the terms used to describe the message type. This dissertation argues that the terms biased and unbiased should be used to describe the messages that are delivered to the experimental participants. This allows researchers to better track two of the variables that might play key roles in the effectiveness of any given experiment and gives future researchers a standard structure by which future comparisons can be easily made. Additionally, this dissertation provided the beginnings of a larger structure of terms that can be used to compare and contrast future experiments. This list of 121 terms/variables allows researchers to better articulate and compare one type of research to another while allowing future researchers to consider what variables they might be able to control in their experiments. In the research reviewed for this dissertation, it is clear that researchers are considering these variables, but none of the literature specifically outlined a list of variables that might act as a starting place for both academics and professionals who are conducting campaign-effects field experiments on mobilization and vote choice. These variables include, but are not limited to, election type (candidate, referendum, initiative, etc.), election time (primary, general, municipal, special, etc.), election levels (president, mayor, governor, etc.), geographic level (local, statewide, national, etc.), voter party identification, campaign environment (partisan, nonpartisan), number of items on the ballot, ballot position, message type (biased or unbiased), message delivery medium (door-to-door, leafleting, direct mail, prerecorded call, etc.), message tone (positive or negative), salience of message to voter, personalization/intimacy of message, professionalism, authority of those delivering the message, actual message, message targeting and target population, message frequency, message synergy, and time of delivery of message. The experiment that was the focus of this dissertation controlled for partisanship variables by testing for campaign effects during a referendum campaign that lacked a clear partisan shortcut for voters to guide their vote intention or vote choice. Additionally, the Amendment One referendum was the only statewide item on the ballot at the time of the Alabama special election. The messages were targeted and biased in that they supported and encouraged the voters to vote for the adoption of the referendum. The messages were delivered via direct mail and prerecorded calls to registered senior- 122 voter households around Alabama. The experiment tested the impact of both biased positive targeted messages that were thought to be salient to senior voters and biased generic messages. Additionally, the experiment tested for a synergistic effect and a frequency effect for the targeted biased messages. All messages were developed and distributed by a professional campaign consulting organization with a history of working on Alabama political campaigns. The messages were all delivered in the last month of the campaign. As it relates to vote intention and vote choice, none of the experimental groups were found to be statistically significantly different from any of the other groups. There are a number of possible explanations for these results, but a few are worth discussing. The first and most obvious reason is that there is no true difference between the groups and that campaigns cannot influence vote choice or intention among seniors during referendum campaigns in Alabama. Second, there could be a difference, but the statistical power of the experiment was not sufficient to detect the differences among the experimental groups. To confirm the lack of impact, future research is necessary. However, as discussed in chapter 4, when looking at vote intention, four of the five treatment groups did have point estimates higher than the control group for likelihood to vote, with the largest positive difference between Group A and Group D (+3.7%). Group D was the treatment group that received one targeted message and one generic message via direct mail. Perhaps the reason for the increase between Group D and the control group is that voters in Group D received two different kinds of messages related to Amendment One rather than simply receiving two very similar messages related only to seniors. In other words, perhaps the impact of a second similar message is less than the 123 impact of a second message that is new to the voter. Similarly, with regard to vote choice, three of the five treatment groups had point estimates higher than the control group with the largest difference between Group A and Group E (+5.7%). Group E received only one generic message delivered by direct mail. In this case, perhaps the generic message had a more significant connection with the voters than did the targeted message. This is in line with the results reported by Gerber that indicate targeted messages are most effective with non-targeted groups (Gerber, 2004). Although the analyses did not produce statistically significant results related to vote intention or vote choice, they do raise the possibility that a mobilization effect and persuasion effect might exist among senior voters. In considering the 57 campaign experiments reviewed for this dissertation, only 8 focused in some fashion on detecting campaign effects during referendum campaigns (See Table 3 for a list of the eight referendum experiments). In fact, when one begins to look for similar experiments considering the combination of variables that can be addressed in an experiment, it becomes difficult to find clear examples of replication. It is through the continued replication of experiments that true effects might be learned. To that end, none of the eight referendum campaign experiments is exactly like the experiment reported in this dissertation with regard to the message type, message targeting, partisanship, message frequency, vote intention, and vote choice. In other words, more experimental research is needed and this experiment adds to the growing body of work. Three of the 57 known and reported randomized campaign field experiments considered campaign persuasion effects during referendum campaigns. Taken together, 124 those three seem to indicate that a 10% persuasion effect might exist among voters. In that regard, this experiment runs counter to the idea that a persuasion effect exists at all during referendum campaigns or that, if an effect does exist, it is likely far smaller than the 10% reported in those experiments. The most recent of the three was conducted by Arceneaux and Nickerson (2005) during the 2004 election cycle. They considered a door- to-door treatment, whereas the other two older experiments considered direct mail as a treatment variable. Perhaps the voters? receptiveness has changed since the 1953 experiment by Eldersveld and Dodge (1954) or the 1980 experiment by Bositis et al. (1985). Perhaps the expanded use of direct mail through other areas in society has reduced the effectiveness of direct mail during political campaigns. Regardless of whether modern voters are less influenced by direct mail than mid-20th-century voters, the experiment reported in this dissertation is the first experiment since 1980 to consider the impact of direct mail on vote choice during a referendum campaign. Additionally, this experiment is the first to consider a larger level of geography than the local level as it relates to referendum campaigns and persuasion in an experiment that was conducted on a statewide scale. It has generally been accepted that campaigns can mobilize voters and increase turnout during any given election depending on the treatment (Green & Gerber, 2004). Statistically, this experiment does run counter to that claim in that no statistical support was found supporting a mobilization hypothesis. In considering the seven referendum experiments that investigated a mobilization effect, the Alabama Seniors Experiment lends support to the findings of four of those experiments that found no support for the 125 mobilization hypothesis (Arceneaux & Nickerson, 2005; McNulty, 2005a, 2005b; Michelson, 2005). Gerber and Green (2004) argued that the more personal the contact method, the more effective that method is at voter mobilization. In this experiment, the messages were thought to be more salient and, therefore, more personal than a generic campaign communication. Specifically, in this experiment the campaign developed and delivered targeted messages that used senior imagery in direct mail and audio in a prerecorded call with hopes of improving the salience and personal nature of the messages. Perhaps targeting, salient messages, images, and audio on a prerecorded call do not reach the level of personalization that is needed to influence the outcome of a campaign. Or, if they do have an impact, perhaps it would prove to be a far less efficient mechanism than the door-to-door personal contact that Gerber and Green advocate (2004). Regardless, the findings of this experiment indicate that, among seniors, these personalization techniques did not have a statistically significant impact on mobilization. Study Limitations This was a study that attempted to improve the accuracy and generalizability of the research by using a field-experiment design. In that respect it was successful in that it provided an additional set of campaign field-experiment data for others to grow and use to generalize in the future. However, it is important to consider this or any other research in light of the limitations that may have an impact on how the data are interpreted. This research has several limitations worth mentioning. 126 First, this was a field experiment that was performed during a special election in a southern state that lacked partisanship as a cue for voters. Inherently, the Alabama Seniors Experiment was different from many other elections around the country. The ability to generalize to seniors in other settings, locations, and campaigns is, therefore, a limitation. Similarly, the makeup of the voting population is likely different in a special referendum election than the voter makeup in a partisan general election, which leads to a further limitation in its generalizability. However, as the research into nonpartisan elections and seniors expands, the true generalizability of this research will be better understood. Second, the research was performed in 2003 by one of the key campaign consulting organizations working in support of Amendment One. Due to the time that elapsed between the experiment being performed and the data being studied and analyzed for this dissertation, several questions have not been answered with regard to the data. For example, the voter file that was used at the time of the experiment has changed over the years, which complicates the researcher?s ability to ensure the final treatment groups are representative of the senior population as a whole. Similarly, it is not possible to ensure that the survey groups exactly represent the treatment groups. While the experimental groups were similar with respect to most of the demographic variables, the lack of ability to compare the groups to the larger universe of senior Alabama voters is still one worth considering. Along those lines, there were some differences among the group demographics that were discussed in chapter 4. Group E was statistically different from the other groups in that there was a larger percentage of men and smaller percentage of women than the 127 other groups. Similarly, Group E had statistically fewer Democrats than the other groups. Group E was the group that received three targeted treatments (two direct mailings and one prerecorded call). Given that the data were collected randomly and at the same time as the other groups, it seems that these differences exist simply by chance. However, knowing that the groups were different prior to the analysis of the data is a limitation of the data. Low statistical power is the fourth limitation of this research. The analysis found no statistical support for either the mobilization or the persuasion hypotheses. Although the Chi-square results for the vote choice and vote intention analyses were not statistically significant there were small effects. For this kind of research, which studies small electoral margins, even small effects may be important. Based on the data from this experiment, a post hoc power analysis showed that the statistical power of the Alabama Seniors Experiment was quite weak with little chance to show statistically significant results given the sample size. For the effects to be statistically significant at the .05 level with a power of .80 the number of respondents would need to be approximately 1,600. This experiment used 701 respondents when considering the mobilization hypothesis and 529 respondents when considering the persuasion hypothesis. Given that, this research provides a benchmark for future researchers in the study of senior voters regarding the sample size needed to test for these hypotheses. Fifth, the sample was drawn from all senior voter households on the state voter rolls in Alabama at the time of the experiment. The treatments were then distributed to the households, but it is possible that individuals other than those who were ultimately surveyed received the treatments. While it has been shown that there is a significant 128 diffusion effect in other experiments (Nickerson, 2008), it is possible that the diffusion effect is different among senior voters. One possible way to address this in future research would be to randomly select senior voters from single-voter households. Another way would be to use verified votes based on the publicly available vote records to test for a mobilization effect rather than relying on a survey question. This leads to the sixth limitation of this field experiment?self-reported data. This research used self-reported data for both the mobilization and vote choice variables. As discussed in chapter 2, the vast majority of past research into campaign effects tested only for a mobilization effect, and the majority of that research used publicly available vote-history data to determine if an experimental subject participated in a given election. The experimental data provided by the campaign consulting organization for analysis in this dissertation did not include any identifiable information about the survey subject, thereby preventing a review of the voting history records. However, as the 2002 experiment of Vavreck (2007) showed, using self-reported data rather than verified vote data leads to rejecting the mobilization hypotheses when it should be accepted rather than the other way around. In other words, an effect is underestimated rather than being overestimated. Regardless, given the expanded tracking and ease of access to a voter?s vote history, future research should use verified vote data to test for mobilization effects. However, there is no verifiable data when it comes to determining who or what an experimental participant voted for or plans to vote for. This raises the possibility that future research can somehow compare voters? self-reported vote data against verified vote data to better estimate the accuracy of the voters? self-reported vote choice data. In other words, future research that looks at persuasion effects using self-reported survey 129 data might also collect the vote intention data and compare it to the actual publicly available vote-history data in an attempt to model which voters are more likely to provide inaccurate information for the vote choice question. Vavreck provided a very solid foundation for this idea that other researchers might be able to expand in the future (2007). The seventh limitation is concerned with the manner in which political consultants conduct their research outside of the public view and behind a cloak of secrecy. As mentioned earlier in the dissertation, the campaign consultants conducted a benchmark survey and analyzed that survey to determine what messages should be delivered to what groups including the senior voter group. That survey and the subsequent analysis were not released for public consumption, which means it is possible that the methods used to determine what message to send to senior voters or the messages and delivery modes that were chosen were not the most efficient ones that could have been used. This comes back to the idea that campaigns and campaign professionals tend to be private with their techniques and academics have to make concessions when working in that environment, as mentioned by Green and Gerber (2002b). Future researchers will no doubt have to make similar concessions to execute field experiments testing for campaign effects. Nonetheless, perhaps future research can delve into the techniques used for targeting and better test for the various methods that are used in the professional campaign arena. Similarly, given the limitless number of options related to the design and message of campaign communications, future research should consider the impacts of different message types, designs, and compositions to determine their effects on the outcome of campaigns. Because these are areas that marketing researchers have 130 focused on for many years, perhaps expanding the marketing research into campaign research would help the campaign-effects research progress at a faster rate. Finally, the ability of the treatment recipient to interpret and comprehend the messages that campaigns distribute could be a limitation of this research. Given that the United States has a literacy rate of around 80% (Kirsch, Jungeblut, Jenkins, & Kolstad, 2002), it is possible that the effectiveness of direct mail to influence a voter?s behavior is reduced given the likely inability of illiterate and functionally illiterate voters to comprehend the messages being delivered to them. Future researchers should try to determine the impact that literacy has on the ability to influence the outcome of an election. Research Implications and the Future While the prior research reviewed for this dissertation represents a tremendous expansion in the knowledge base for campaign effects since they were first studied in 1928, significantly more research is needed before the campaign-effects research community has a true understanding of when, how, where, and why campaigns have an impact on the outcome of elections. The lack of statistically significant findings in this experiment would suggest that no campaign effect exists related to senior voters and a campaign?s ability to mobilize and persuade during referendum campaigns. However, there are several implications that can be found in this research that might help the advancement of the discipline. First, this dissertation provides an example of how academic researchers can interact with a campaign consulting organization to publish results of in-house 131 experiments that consulting organizations conduct but are not likely to publish. In that regard, it opens the opportunity for campaign experiments to be analyzed and published by individuals that lack the means necessary to perform their own experiments. Private campaign consulting organizations can and do conduct experiments of their own to determine how to improve their techniques to win campaigns. By publicizing those results, this research, and hopefully future research, provides a peek behind the curtain of privacy of campaign consultants. Through publication of this data, academic researchers will improve public knowledge while at the same time improving the effectiveness of political campaigns. Second, it is possible that the window of time in which a campaign has an impact on senior voters is very small and the potential effect is even smaller. This implies that studying this window of opportunity in the future might not be a worthwhile academic study. However, because campaigns and campaign professionals continue to target senior voters and the senior-voter population will continue to grow as the baby boomer generation ages, it is imperative that researchers expand senior voter research. The research that was the focus of this dissertation is the first to specifically focus on and target senior voters. Third, as with the item above, this research raises the issue of statistical power when studying populations that likely have little variance in their behavior. As with senior voters, perhaps there are other groups of voters that also turn out in high numbers, which leaves limited room for researchers to detect campaign-mobilization effects among those groups. Thus a larger sample size is necessary to increase the power of those 132 experimental studies to detect what could be small effects among highly active or extremely partisan voters. Fourth, this experiment, by reporting statistically insignificant results, answers the call for authors to report their insignificant findings. The implications of authors reporting valid yet statistically insignificant results is important in that the discipline can grow through the knowledge that other researchers gained, regardless of the results. By reporting insignificant results, the entire academic community can grow in a more efficient way because the entire community has access to all the research. This should not only prevent unnecessary duplication, but should also reduce the opacity by which authors see the world when looking through glasses that only show the significant results. Finally, the experimental data showed that the messages delivered to the experimental participants did have an effect among better-educated senior voters in terms of the voters? recall of campaign messages. This does not seem too surprising in that one would expect more educated voters to pay more attention to politics because they are more active in the political system (Flanigan & Zingale, 2002). The interesting part of this finding is that the recall effect did not lead to a change in voting behavior. This raises the possibility that campaigns and their methods of developing and delivering messages do not lead to changed voter behavior. The possibility also exists, as Bernstein pointed out, that socially invested voters have more of a reason to lead the interviewer to believe that the voter has or will cast a vote even when the voter has not or will not cast a vote (Bernstein 2001). Regardless, the opportunities to further investigate the decision-making processes of voters and how those relate to campaign communications seem abundant. 133 Similarly, in thinking about the transition that likely must exist between campaign/issues awareness and voting behavior, perhaps the timing in which the messages were distributed in the Alabama Seniors Experiment was not as effective as it needed to be. In thinking about senior voters, perhaps they make up their minds earlier in the campaign and, therefore, the campaign should have targeted them earlier than 1 month before the election. Or, maybe the frequency or synergy of the messages needed to be different or more dramatic than was the case in this experiment. Future researchers can now consider these questions as they relate to senior voters. Practical Implications While all research should have implications for future research, not all research will necessarily have practical implications as well. This dissertation does have several practical implications for campaign professionals. The most glaring implication seems to be that campaigns that attempt to persuade or mobilize senior voters are not spending their campaign dollars as wisely as they might had other groups been targeted?youth for example. As the baby boomer generation continues to move into retirement, the senior vote is likely to play an even more significant roll in future political campaigns. This implies that campaigns will continue to spend larger amounts of time and money attempting to influence that block of voters. The research reported in this dissertation makes an argument that senior voters should be a group that is given a lower priority in terms of campaign focus. The second implication deals with the targeting processes that are currently used in the campaign environment. Candidate and campaign consultants routinely use survey 134 research to develop and target campaign messages with the expectation that they can influence a voter?s probability of voting or influence a voter?s vote choice in a given election. Senior voters tend to be among the most reliable voters, and thus campaigns tend to focus on them using these methods. The campaign consultants that conducted this experiment used a benchmark poll to determine that senior voters were responsive to the message that was thought to be more salient to that group of voters. The campaign distributed that message to the senior-voter bloc and tested the impact of those messages. This raises the issue that the methods used by campaign professionals to develop and target messages are lacking in some fundamental area and should be rethought or at least tested further. Third, as discussed in chapter 4, a portion of the voters in this experiment did appear to recall the information from the messages that were delivered; yet when looking only at those voters, the experimental groups were not statistically significantly different for voters? intention to vote or vote choice. In other words, perhaps campaigns can educate senior voters, but that education does not translate into changes in voting behavior and, thereby, reduces a campaign?s efficiency and effectiveness when the campaign targets senior voters. At the very least, this research should raise the issue in the minds of campaign professionals that the use of experimental methods coupled with statistical analysis can help improve the efficiency of their campaigns and perhaps professionals can determine the mechanisms by which groups of voters translate campaign messages into voting behavior. This raises the issue of the methods used to mobilize senior voters. Perhaps methods such as direct mail and telephone calls are simply not able to mobilize senior 135 voters. Given that a larger percentage of those registered voters who are seniors are likely to have something that inhibits their ability to vote such as lack of transportation or illness, other methods might prove more effective at getting seniors to cast a ballot. The use of absentee ballots and the techniques associated with distributing them might be more effective at increasing turnout among the senior voter subgroup than other methods or other groups. In fact, one of the earliest campaign experiments reported results by age groups by modes of message delivery and found that different age groups responded differently to different modes of delivery (Eldersveld, 1956). Perhaps the modes used in this experiment were not the most efficient modes for this group of voters. Fifth, while the effects were not statistically significant in this experiment, the point estimates imply that perhaps a very small mobilization or persuasion effect does exist. Future experiments should be conducted with stronger statistical power in an effort to detect what might be very small effects. Finally, this research represents an expansion of the literature through the joint work of both the professional and the academic world and can provide a model for how the two groups can work effectively to generate and improve the collective knowledge of campaign effects. In this example, the research was originally performed and destined to remain unpublished. However, the consulting organization provided the data set for analysis and publication in this dissertation. While the results were not statistically significant, the knowledge gained by all parties is practically significant. The implication is that both academics and consultants can grow by simply working together on future experiments in the field of campaign effects. 136 Final Thoughts Finkel wrote that the central question in the study of electoral behavior is the effect campaign events and campaign stimuli have on vote choice (Finkel & Schrott, 1995). Similarly, voter mobilization is also believed to be a critical aspect of political campaigns (Herrnson et al., 2005). Currently, there appears to be a difference of opinion between researchers and campaign professionals regarding the actual impact that campaigns can have on the outcomes of elections: professionals believe campaigns have a larger impact than researchers have been able to detect. This dissertation, by using secondary data collected during a live referendum campaign by one of the consulting arms of that campaign helps to bridge that gap. It does so by answering the call of several prior researchers to conduct and publish the results of other campaign field experiments (Green & Gerber, 2000; Imai, 2005; Johnson-Cartee & Copeland, 1997; Niven, 2004). The central hypotheses of this dissertation focused on the extent to which targeted campaign messages and the frequency with which those messages are delivered actually matter in relation to the vote choice and mobilization of voters. Ultimately, no statistical support was found for either the mobilization hypothesis or the persuasion hypothesis. As discussed above, perhaps focusing on seniors limited the ability to detect a mobilization or persuasion effect. Or, perhaps the power of the experiment was the reason a small effect was not detected. Or, perhaps there is no effect to detect. While there were 57 randomized campaign field experiments reviewed for and considered in this dissertation, 57 is not enough to answer the myriad of questions that come to mind related to campaigns and the effects they may have on voting behavior. Many more experiments are 137 needed to improve the overall generalizability of the existing research into campaign effects. As such, larger experiments focusing on additional voter subgroups are in order. The sheer number of unanswered questions provides a ripe opportunity for those with an interest in discovering when and how campaigns influence a voter?s behavior. However, regardless of whether academic researchers study these effects in the future, if history is any indication, campaigns and the professionals behind them will continue to spend large and ever-increasing sums of money with the goal of improving their candidate or issues? chances of winning an electoral contest. Similarly, as has been a central motivator in publicizing this dissertation, campaign professionals are conducting campaign-effects research, yet they are not bound to or necessarily have a desire to make public the knowledge that they gain through their experiments. It does, therefore, fall on the shoulders of the academic community that has been built on the idea of open-source knowledge, to perform and make public the research associated with these effects. In that respect, the academic community has an opportunity to further democratize the electoral process by making open and public the information, techniques, and strategies that are used by a select and privileged few. Further, an overarching theme of this dissertation, working with campaigns and campaign professionals, can help the researcher improve the knowledge base for everyone. This dissertation has shown that at least some campaign professionals are open to the idea of working with and having their experiments published by the academic community. As Green and Gerber (2002b; 2003) discussed, by relaxing some of the controls a researcher would normally have by conducting their own field experiments, independent of other political entities, the opportunities for knowledge to advance and 138 grow increase simply by virtue of the large number of democratic elections that take place in the United States and around the world. Given the facts that the senior-voter population will continue to grow as the baby boomer generation ages, senior voters tend to be the most politically active subgroup of voters, and political campaigns regularly target them, further studies concerning seniors are also in order (Herrnson, 2004; MacManus & Shields, 2000). This dissertation has contributed to the knowledge researchers have about senior voters. However, many more campaign field experiments are necessary to provide support or counter the research presented in this dissertation. As with other subgroups and field experiments, the process will not be easy or fast, but the knowledge learned today will help future generations of researchers and professionals. 139 REFERENCES Adams, W. C., & Smith, D. J. (1980). Effects of telephone canvassing on turnout and preferences: A field experiment. Public Opinion Quarterly, 44, 389?395. Ansolabehere, S. D., & Iyengar, S. (1997). Going negative: How political advertisements shrink and polarize the electorate. New York: Free Press. Ansolabehere, S. D., Iyengar, S., Simon, A., & Valentino, N. (1994). Does attack advertising demobilize the electorate? American Political Science Review, 88, 829?838. Arceneaux, K. (2005). Using cluster randomized field experiments to study voting behavior. Annals of the American Academy, 601, 169?179. Arceneaux, K. (2007). I?m asking for your support: The effects of personally delivered campaign messages on voting decisions and opinion formation. Quarterly Journal of Political Science, 2, 43?65. Arceneaux, K., Gerber, A. S., & Green, D. P. (2006). Comparing experimental and matching methods using a large-scale voter mobilization experiment. Political Analysis, 14, 37?62. Arceneaux, K., & Nickerson, D. (2005, September). Two field experiments testing negative campaign tactics. Paper presented at the Annual Meeting of the Midwestern Political Science Association, Washington, DC. 140 Babbie, E. (2007). The practice of social research (11th ed.). Belmont, CA: Wadsworth Thomson Learning. Barker, D. C. (2005). Values, frames, and persuasion in presidential nomination campaigns. Political Behavior, 27, 375?394. Bartels, L. (1993). Messages received: The political impact of media exposure. American Political Science Review, 87, 267?285. Bennion, E. A. (2005). Caught in the ground wars: Mobilizing voters during a competitive congressional campaign. Annals of the American Academy of Political and Social Science, 601, 123?141. Berelson, B., Lazarsfeld, P., & McPhee, W. (1954). Voting: A study of opinion formation in a presidential campaign. Chicago: University of Chicago Press. Bernstein, R. A., Chadha, A., & Montjoy, R. (2001). Overreporting voting. Public Opinion Quarterly, 65, 22?44. Bernstein, R. A., & Seroka, J. (2004, September). Alabama?s tax reform: What went wrong and why? Paper presented at the Citadel Conference on Southern Politics and Policy, Charleston, SC. Blydenburgh, J. C. (1971). A controlled experiment to measure the effects of personal contact campaigning. Midwest Journal of Political Science, 15, 365?381. Bositis, D. A., Baer, D. L., & Miller, R. E. (1985). Cognitive information levels, voter preferences, and local partisan political activity: A field experimental study on the effects of timing and order of message presentation. Political Behavior, 7, 266? 284. 141 Bowler, S., & Donovan, T. (1994). Information and opinion change on ballot propositions. Political Behavior, 16, 411?435. Bowler, S., & Donovan, T. (1998). Demanding choices: Opinion, voting, and direct democracy. Ann Arbor: University of Michigan Press. Bowler, S., & Donovan, T. (2002). Do voters have a cue? Television advertisements as a source of information in citizen-initiated referendum campaigns. European Journal of Political Research, 41, 777?793. Bowler, S., Donovan, T., & Tolbert, C. J. (1998). Citizens as legislators: Direct democracy in the United States. Columbus: Ohio State University Press. Broder, D. S. (2000). Democracy derailed: Initiative campaigns and the power of money. New York: Harcourt. Bult, J. R., & Wansbeek, T. (1995). Optimal selection for direct mail. Marketing Science, 14, 378?394. Cain, B., & McCue, K. (1985). The efficacy of registration drives. Journal of Politics, 47, 1221?1230. Caldeira, G. A., Clausen, A. R., & Patterson, S. C. (1990). Partisan mobilization and electoral participation. Electoral Studies, 9, 191?204. Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Chicago: Rand McNally. Cardy, E. A. (2005). An experimental field study of the GOTV and persuasion effects of partisan direct mail and phone calls. Annals of the American Academy of Political and Social Science, 601, 28?40. 142 Clinton, J., & Lapinski, J. (2004). ?Targeted? advertising and voter turnout: An experimental study of the 2000 presidential election. Journal of Politics, 66, 69? 96. Cobb, M. D., & Kuklinski, J. H. (1997). Changing minds: Political arguments and political persuasion. American Journal of Political Science, 41(1), 88?121. Converse, P. E. (1966). The concept of a normal vote. In A. Campbell, P. E. Converse, W. E. Miller, & D. Stokes (Eds.), Elections and the political order (pp. 9?39). New York: Wiley. Cronin, T. E. (1999). Direct democracy: The politics of initiative, referendum, and recall. Cambridge, MA: Harvard University Press. Crotty, W. (1971). Party effort and its impact on the vote. American Political Science Review, 65, 439?450. Dale, A., & Strauss, A. (2007). Mobilizing the mobiles: How text messaging can boost youth voter turnout. Retrieved July 15, 2008, from http://www.mindless philosopher.net/princeton/Youth%20Vote%20and%20Text%20Messaging_9.6.07 .pdf Dalton, R. J. (2002). Citizen politics (3rd ed.). New York: Chatham House. Eldersveld, S. J. (1956). Experimental propaganda techniques and voting behavior. American Political Science Review, 50, 154?165. Eldersveld, S. J., & Dodge, R. W. (1954). Personal contact or mail propaganda? An experiment in voting and attitude change. In D. Katz, D. Cartwright, S. J. Eldersveld, & A. M. Lee (Eds.), Public opinion and propaganda (pp. 534?542). New York: Dryden. 143 Field, A. (2005). Discovering statistics using SPSS. Thousand Oaks, CA: Sage. Finkel, S. E., & Schrott, P. R. (1995). Campaign effects on voter choice in the German election of 1990. British Journal of Political Science, 25, 349?377. Flanigan, W. H., & Zingale, N. H. (2002). Political behavior of the American electorate (10th ed.). Washington, DC: Congressional Quarterly. Freedman, P., Franz, M., & Goldstein, K. (2004). Campaign advertising and democratic citizenship. American Journal of Political Science, 48, 723?741. Friedrichs, R. D. (2003). Mobilizing 18?35 year old voters: An analysis of the Michigan Democratic Party?s 2002 youth coordinated campaign. Unpublished master?s thesis, Harvard University, Cambridge, MA. Gelman, A., & King, G. (1993). Why are American presidential election campaign polls so variable when votes are so predictable? British Journal of Political Science, 23, 409?451. Gerber, A. S. (2004). Does campaign spending work? Field experiments provide evidence and suggest new theory. American Behavioral Scientist, 47, 541?574. Gerber, A. S., & Green, D. P. (1999). Does canvassing increase voter turnout? A field experiment. Proceedings of the National Academy of Sciences of the United States of America, 96, 10939?10942. Gerber, A. S., & Green, D. P. (2000a). The effect of a nonpartisan get-out-the-vote drive: An experimental study of leafletting. Journal of Politics, 62, 846?857. Gerber, A. S., & Green, D. P. (2000b). The effects of canvassing, telephone calls, and direct mail on voter turnout: A field experiment. American Political Science Review, 94, 653?663. 144 Gerber, A. S., & Green, D. P. (2001a). Do phone calls increase voter turnout? A field experiment. Public Opinion Quarterly, 65(1), 75?85. Gerber, A. S., & Green, D. P. (2001b). Getting out the youth vote: Results from randomized field experiments. Unpublished report to the Pew Charitable Trusts and Yale University?s Institute for Social and Policy Studies. Gerber, A. S., & Green, D. P. (2005a). Do phone calls increase voter turnout? An update. Annals of the American Academy of Political and Social Science, 601, 142?154. Gerber, A. S., & Green, D. P. (2005b). Recent advances in the science of voter mobilization. Annals of the American Academy of Political and Social Science, 601, 6?9. Gerber, A. S., Green, D. P., & Green, M. (2003). Partisan mail and voter turnout: Results from randomized field experiments. Electoral Studies, 22, 563?579. Gerber, A. S., Green, D. P., & Larimer, C. W. (2008). Social pressure and voter turnout: Evidence from a large-scale field experiment. American Political Science Review, 102, 33?48. Gerber, A. S., Green, D. P., & Shachar, R. (2003). Voting may be habit-forming: Evidence from a randomized field experiment. American Journal of Political Science, 47, 540?550. Gerber, A. S., & Rogers, T. (2007). The effects of descriptive social norms on voter turnout: The importance of accentuating the positive. Unpublished Manuscript, Yale University. Goldstein, K. M., & Ridout, T. N. (2002). The politics of participation: Mobilization and turnout over time. Political Behavior, 24, 3?29. 145 Goldstein, K. M., & Ridout, T. N. (2004). Measuring the effects of televised political advertising in the United States. Annual Review of Political Science, 7, 205?206. Gosnell, H. F. (1926). An experiment in the stimulation of voting. American Political Science Review, 20, 869?874. Gosnell, H. F. (1927). Getting-out-the-vote: An experiment in the stimulation of voting. Chicago: University of Chicago Press. Green, D. P. (2004). Mobilizing African-American voters using direct mail and commercial phone banks: A field experiment. Political Research Quarterly, 57, 245?255. Green, D. P., & Gerber, A. S. (2000, December). Reclaiming the experimental tradition in political science. Paper presented at the Annual Meeting of the American Political Science Association, Washington, DC. Green, D. P., & Gerber, A. S. (2001). Getting out the youth vote: Results from randomized field experiments. Retrieved June 27, 2006, from http://www.yale.edu/isps/publications/Youthvote.pdf Green, D. P., & Gerber, A. S. (2002a). The downstream benefits of experimentation. Political Analysis, 10, 394?402. Green, D. P., & Gerber, A. S. (2002b). Reclaiming the experimental tradition in political science. In I. Katznelson & H. V. Milner (Eds.), Political science: State of the discipline (pp. 805?832). Washington, DC: American Political Science Association. 146 Green, D. P., & Gerber, A. S. (2003). The underprovision of experiments in political science. Annals of the American Academy of Political and Social Science, 589, 94?112. DOI: 10.1177/0002716203254763 Green, D. P., & Gerber, A. S. (2004). Get out the vote: How to increase voter turnout. Washington, DC: Brookings Institution. Green, D. P., Gerber, A. S., & Nickerson, D. (2003). Getting out the vote in local elections: Results from six door-to-door canvassing experiments. Journal of Politics, 65, 1083?1096. Green, D. P., & Shachar, R. (2000). Habit formation and political behavior: Evidence of consuetude in voter turnout. British Journal of Political Science, 30, 561?573. Green, D. P., & Shapiro, I. (1994). Pathologies of rational choice theory. New Haven CT: Yale University Press. Herrnson, P. S. (2004). Congressional elections: Campaigning at home and in Washington. Washington, DC: CQ Press. Herrnson, P. S., Campbell, C., Ezra, M., & Medvic, K. (2005). Guide to political campaigns in America. Washington, DC: CQ Press. Holbrook, T. M. (1996). Do campaigns matter? Thousand Oaks: Sage. Huckfeldt, R., & Sprague, J. (1992). Political parties and electoral mobilization: Political structure, social structure, and the party canvass. American Political Science Review, 86, 70?86. Huntington, S. P. (1993). The third wave: Democratization in the late twentieth century. Norman: University of Oklahoma Press. 147 Imai, K. (2005). Do get-out-the-vote calls reduce turnout? The importance of statistical methods for field experiments. American Political Science Review, 99, 283?300. Johnson-Cartee, K. S., & Copeland, G. (1997). Inside political campaigns: Theory and practice. Westport, CT: Praeger. Johnson-Cartee, K. S., & Copeland, G. (2004). Strategic political communication: Rethinking social influence, persuasion, and propaganda. Lanham, MD: Rowman and Littlefield. Katz, D., & Eldersveld, S. J. (1961). The impact of local party activity upon the electorate. Public Opinion Quarterly, 25, 1?24. Kirsch, I. S., Jungeblut, A., Jenkins, L., & Kolstad, A. (2002). Adult Literacy in America (3rd ed.). Retrieved 12/11/2007. 2007, from http://nces.ed.gov/pubs93/93275.pdf Kramer, G. H. (1970). The effects of precinct-level canvassing on voting behavior. Public Opinion Quarterly, 34, 560?572. Kriesi, H. (2002). Individual opinion formation in a direct democratic campaign. British Journal of Political Science, 32, 171?185. Kuklinski, J. H. (2001). Thinking about political psychology. New York: Cambridge University Press. Lau, R. R., Sigelman, L., Heldman, C., & Babbitt, P. (1999). The effects of negative political advertisements: A meta-analytic assessment. American Political Science Review, 93, 851?876. 148 Lavine, H., & Snyder, M. (2000). Cognitive processes and the functional matching effect in persuasion: Studies of personality and political behavior. In G. Maio & J. Olson (Eds.), Why we evaluate: Functions of attitudes (pp. 97?131). Mahwah, NJ: Lawrence Erlbaum. Lazarsfeld, P. F., Berelson, B., & Gaudet, H. (1968). The people?s choice (3rd ed.). New York: Columbia University Press. Lodge, M., Steenbergen, M. R., & Brau, S. (1995). The responsive voter: Campaign information and the dynamics of candidate evaluation. American Political Science Review, 89, 309?326. Luks, S., Miller, J. M., & Jacobs, L. R. (2003). Who wins? Campaigns and the third party vote. Presidential Studies Quarterly, 33, 9?29. Lupfer, M., & Price, D. (1972). On the merits of face to face campaigning. Social Science Quarterly, 53, 534?543. MacManus, S. A., & Shields, K. H. (2000). Targeting senior voters: Campaign outreach to elders and others with special needs. Lanham, MD: Rowman and Littlefield. Magleby, D. B. (1984). Direct legislation: Voting on ballot propositions in the United States. Baltimore: Johns Hopkins University Press. Mann, C. B. (2005). Unintentional voter mobilization: Does participation in preelection surveys increase voter turnout? Annals of the American Academy of Political and Social Science, 601, 155?168. McNulty, J. E. (2005a). Phone-based GOTV: What?s on the line? Field experiments with varied partisan components, 2002?2003. Annals of the American Academy of Political and Social Science, 601, 41?65. 149 McNulty, J. E. (2005b). Sensitivities of voter turnout: Field experiments exploring effects of a variety of treatments. University of California, Berkeley. Michelson, M. R. (2003). Getting out the Latino vote: How door-to-door canvassing influences voter turnout in rural central California. Political Behavior, 25, 247? 263. Michelson, M. R. (2005). Meeting the challenge of Latino voter mobilization. Annals of the American Academy of Political and Social Science, 601, 85?101. Miller, R. E., Bositis, D. A., & Baer, D. L. (1981). Stimulating voter turnout in a primary: Field experiment with a precinct committeeman. International Political Science Review, 2, 445?460. Miller, R. E., & Robyn, D. (1975). A field experimental study of direct mail in a congressional primary campaign: What effects last until election day? Experimental Study of Politics, 4, 1?36. Mutz, D. C., Sniderman, P. M., & Brody, R. A. (1996). Political persuasion and attitude change. Ann Arbor: University of Michigan Press. Nash, E. L. (1984). The direct marketing handbook. New York: McGraw-Hill. Nickerson, D. W. (2005). Partisan mobilization using volunteer phone banks and door hangers. Annals of the American Academy of Political and Social Science, 601, 10?27. Nickerson, D. W. (2006). Volunteer phone calls can increase turnout. American Politics Research, 34, 271?292. Nickerson, D. W. (2007a). Does email boost turnout? Quarterly Journal of Political Science, 2, 369?379. 150 Nickerson, D. W. (2007b). The ineffectiveness of e-vites to democracy field experiments testing the role of e-mail on voter turnout. Social Science Computer Review, 25, 494?503. Nickerson, D. W. (2007c). Quality is job one: Professional and volunteer voter mobilization calls. American Journal of Political Science, 51, 269?282. Nickerson, D. W. (2008). Is voting contagious? Evidence from two field experiments. American Political Science Review, 102, 49?57. Nickerson, D. W., Friedrichs, R. D., & King, D. C. (2006). Partisan mobilization campaigns in the field: Results from a statewide turnout experiment in Michigan. Political Research Quarterly, 59, 85?97. Niven, D. (2001). The limits of mobilization: Turnout evidence from state house primaries. Political Behavior, 23, 335?350. Niven, D. (2002). The mobilization calendar: The time dependent effects of personal contact on turnout. American Politics Research, 30, 308?323. Niven, D. (2004). The mobilization solution? Face-to-face contact and voter turnout in a municipal election. Journal of Politics, 66, 868?884. Niven, D. (2006). A field experiment on the effects of negative campaign mail on voter turnout in a municipal election. Political Research Quarterly, 59, 203?210. OpenSecrets. (2008). Banking on becoming president. Retrieved June 13, 2008, from http://www.opensecrets.org/pres08/ Perlmutter, D. D. (1999). The Manship School guide to political communication. Baton Rouge: Louisiana State University Press. 151 Petracca, M. (1989). Political consultants and democratic governance. Political Science & Politics, 22, 11?14. Petracca, M., & Wiercioch, C. (1998, April). Consultant democracy: The activities and attitudes of American political consultants. Paper presented at the Annual Meeting of the Midwest Political Science Association, Chicago. Phillips, J. A., & Green, D. P. (2001, November). Using e-mail to mobilize young voters: A field experiment. Paper presented at the Annual Meeting of the Southern Political Science Association, Atlanta, GA. Ram?rez, R. (2005). Giving voice to Latino voters: A field experiment on the effectiveness of a national nonpartisan mobilization effort. Annals of the American Academy of Political and Social Science, 601, 66?84. Rosenstone, S. J., & Hansen, M. (1993). Mobilization, participation, and democracy in America. New York: Macmillan. Scammell, M. (1999). Political marketing: Lessons for political science. Political Studies, xlvii, 718?739. Schaffner, B., & Streb, M. (2000, September). Voters without cheat sheets: The education bias of the nonpartisan ballot. Paper presented at the Annual Meeting of the American Political Science Association, Washington, DC. Schaffner, B., Streb, M., & Wright, G. (2000). Teams without uniforms: The nonpartisan ballot in state and local elections. Political Research Quarterly, 53, 7?30. Schrag, P. (1998). Paradise lost: California?s experience, America?s future. New York: New Press. 152 Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi- experimental designs for generalized causal inference. Boston: Houghton Mifflin. Shaw, D. R. (1999). The effect of TV ads and candidate appearances on statewide presidential votes, 1988?96. American Political Science Review, 93, 345?361. Shea, D. M., & Burton, M. J. (2006). Campaign craft: The strategies, tactics, and art of political campaign management (3rd ed.). Westport, CT: Praeger. Smith, D. A. (1998). Tax crusaders and the politics of direct democracy. New York: Routledge. Trent, J. S., & Friedenberg, R. V. (2000). Political campaign communication: Principles and practices (4th ed.). Westport, CT: Praeger. Trivedi, N. (2005). The effect of identity-based GOTV direct mail: Appeals on the turnout of Indian Americans. Annals of the American Academy of Political and Social Science, 601, 115?122. Vavreck, L. (2007). The exaggerated effects of advertising on turnout: The dangers of self-reports. Quarterly Journal of Political Science, 2, 325?343. Wielhouwer, P. W., & Lockerbie, B. (1994). Party contacting and political participation, 1952?1990. American Journal of Political Science, 38, 211?229. Wong, J. S. (2005). Mobilizing Asian American voters: A field experiment. Annals of the American Academy of Political and Social Science, 601, 102?114. Zaller, J. (1992). The nature and origin of mass opinion. New York: Cambridge University Press. 153 APPENDIX A QUESTIONNAIRE INTRODUCTION: Hello, this is _____ calling from Southeast Research. We?re conducting a public opinion survey about issues affecting people in your area. I?m not selling anything. May I speak to a male/female 65 years of age or older living in this household. 1. Are you 65 years of age or older? -1 ( ) Yes [CONTINUE] -2 ( ) No [TERMINATE] 2. As you may know, Alabamians will go to the polls on September 9th and vote on the tax and accountability package known as Amendment 1. As things now stand, will you likely be voting in this election or will you not likely be voting? -1 ( ) Likely voting -2 ( ) Not likely voting -3 ( ) Not Sure/ Refused 3. If this election were held today, would you likely vote for or against Amendment 1? (PROBE ? DEFINITELY OR PROBABLY) -1 ( ) Definitely vote for -2 ( ) Probably vote for -3 ( ) Definitely vote against 154 -4 ( ) Probably vote against -5 ( ) Not Sure/ Refused 4. Do you recall seeing or hearing any advertising that supported Amendment 1? -1 ( ) Yes -2 ( ) No [SKIP TO Q. 7] -3 ( ) Not Sure [SKIP TO Q. 7] 5. And, do you recall whether any of the advertisements for Amendment 1 explain how seniors would benefit if the tax plan passed? -1 ( ) Yes -2 ( ) No -3 ( ) Not Sure 6. Would you say this advertising had a strong impact on how you plan to vote, a moderate impact or very little impact on how you plan to vote? -1 ( ) Strong impact -2 ( ) Moderate impact -3 ( ) Very little impact -4 ( ) Not Sure 7. These last few questions are just for classification purposes. First, do you consider yourself to be a Democrat, a Republican or completely independent? (PROBE ? STRONG OR LEANING) -1 ( ) Strong Democrat -2 ( ) Leaning Democrat 155 -3 ( ) Strong Republican -4 ( ) Leaning Republican -5 ( ) Completely Independent -6 ( ) Not Sure/Refused 8. Do you own your own home? -1 ( ) Yes -2 ( ) No -3 ( ) Not Sure 9. What is the last grade of school you completed? -1 ( ) Grade school or less -2 ( ) Some high school -3 ( ) Graduated high school -4 ( ) Vocational school/Technical school -5 ( ) Some college (2 years or less) -6 ( ) More than 2 years of college -7 ( ) Graduated college -8 ( ) Post graduate degree -9 ( ) Refused 10. Would you say that you vote in all elections, most of them, about half, less than half or do you usually not vote? -1 ( ) Vote in all elections -2 ( ) Most elections -3 ( ) About half 156 -4 ( ) Less than half -5 ( ) Usually do not vote -6 ( ) Not Sure/Refused 11. What is your ethnic background ? are you white, black or some other race? -1 ( ) White -2 ( ) Black -3 ( ) Some other race? -4 ( ) Refused 12. What is your current employment status ? are you? -1 ( ) Retired -2 ( ) Employed full-time -3 ( ) Employed part-time -4 ( ) Unemployed -5 ( ) Laid off temporarily or -6 ( ) Homemaker? -7 ( ) Other -8 ( ) Refused 13. What was your total household income in 2002 before taxes and other deductions? Would that be? -1 ( ) Less than 20,000 -2 ( ) 20,000 to 30,000 -3 ( ) 30,000 to 40,000 157 -4 ( ) 40,000 to 50,000 -5 ( ) 50,000 to 75,000 -6 ( ) 75,000 to 100,000 or -7 ( ) 100,000 and above? -8 ( ) Not Sure/Refused 14. What was your age on your last birthday? ____________ 15. Do you currently have a relative or a close friend in a nursing home? -1 ( ) Yes -2 ( ) No -3 ( ) Not Sure 16. OBSERVE AND RECORD: -1 ( ) Male -2 ( ) Female 158 APPENDIX B EXPERIMENTS REVIEWED Pre 1998 1924 & 1925 Chicago, Illinois (Gosnell, 1927) 1953 Ann Arbor, Michigan (Eldersveld & Dodge, 1954) 1954 Ann Arbor, Michigan (Eldersveld, 1956) 1967 Monroe County, New York (Blydenburgh, 1971) 1972 Unknown congressional campaign (Miller & Robyn, 1975) 1979 Washington, DC (Adams & Smith, 1980) 1980 Carbondale, Illinois experiment #1 (Miller et al., 1981) 1980 Carbondale, Illinois experiment #2 (Bositis et al., 1985) Federal Midterm Elections 1998 New Haven, Connecticut (Gerber & Green, 1999, 2000a, 2000b; Gerber et al., 2008; Gerber, Green, & Shachar, 2003; Green & Gerber, 2004) West Haven, Connecticut (Gerber & Green, 2001a, 2005a; Green & Gerber, 2004) State & Municipal Elections 1999 New Haven, Connecticut (Gerber et al., 2008; Green & Gerber, 2004) Small city in Connecticut (Gerber, 2004; Gerber, Green, & Green, 2003) New Jersey (Gerber, 2004; Gerber, Green, & Green, 2003; Gerber et al., 2008) 159 Presidential Elections 2000 Connecticut (Gerber, Green, & Green, 2003) Nationwide NAACP National Voter Fund experiment (Green, 2004) Albany, Stoneybrook, Eugene, and Bolder experiments (Green & Gerber, 2001; Nickerson, 2006) Nationwide Knowledge Network panel (Clinton & Lapinski, 2004) Nationwide e-mail experiment (Phillips & Green, 2001) Municipal Elections 2001 Multi-city experiments by coalition of community groups (Green & Gerber, 2004; Green et al., 2003) Dos Palos, California (Michelson, 2003, 2005) Boston, Massachusetts (Nickerson, 2006) Seattle, Washington (Nickerson, 2006) Federal Midterm Elections?2002 Connecticut congressional experiment #1 (Gerber, 2004) Connecticut congressional experiment #2 (Gerber, 2004) Los Angeles, California (Wong, 2005) Multi-state NALEO?s experiment (Ram?rez, 2005) Unknown state gubernatorial election (Cardy, 2005) San Francisco, California (McNulty, 2005a, 2005b) Fresno, California (Michelson, 2005) South Bend, Indiana (Bennion, 2005) Iowa & Michigan (Arceneaux, Gerber, & Green, 2006; Gerber & Green, 2005a) 160 Michigan Youth Voter (Friedrichs, 2003; Green & Gerber, 2004; Nickerson, 2005, 2006) East Bay, California experiment #1 (McNulty, 2005a, 2005b) East Bay, California experiment #2 (McNulty, 2005a, 2005b) New York/Maryland/Pennsylvania survey experiment (Mann, 2005) Nationwide youth vote experiments (Nickerson, 2007c) Denver, Colorado (Nickerson, 2008) Minneapolis, Minnesota (Nickerson, 2008) Five university e-mail experiments (Nickerson, 2007a) Nationwide Knowledge Network panel (Vavreck, 2007) Pennsylvania (Gerber et al., 2008; Green & Gerber, 2004) State & Municipal Elections?2003 East Bay, California (McNulty, 2005a, 2005b) West Palm Beach, Florida (Niven, 2006) Fresno, California (Michelson, 2005) Maricopa County, California (Michelson, 2005) Kansas City, Missouri (Arceneaux, 2005) Houston, Texas (Nickerson, 2007a) Presidential Elections?2004 Minnesota (Arceneaux & Nickerson, 2005) Los Angeles, California (Arceneaux & Nickerson, 2005) Bernalillo County, New Mexico (Arceneaux, 2007) Queens, New York (Trivedi, 2005) 161 Seven state experiments (Nickerson, 2007a, 2007b) State & Municipal Elections?2005 New Jersey (Gerber & Rogers, 2007) Federal Midterm Elections?2006 California (Gerber & Rogers, 2007) Michigan (Gerber et al., 2008) Nationwide text messaging experiment (Dale & Strauss, 2007) 162 APPENDIX C SENIORS I DIRECT MAIL 163 APPENDIX D SENIORS 2 DIRECT MAIL 164 165 APPENDIX E WORKING FAMILIES DIRECT MAIL 166 APPENDIX F SENIORS PHONE 60 Second Telephone Script for Seniors (read by a senior female professional talent that was hired by the production studio) Hello. I am not selling anything today but I am calling to talk to you about the future of Alabama. On September 9th, we will have the opportunity to change Alabama forever by voting to support Amendment One. In fact, under Amendment One, seniors over 65 will pay no state property tax on their homes and their retirement fund income will be exempt from taxes up to forty thousand dollars. Maybe that?s why both the Alabama Silver Haired Legislature and the AARP have endorsed Amendment One as the way to address Alabama?s budget shortfall. In fact, the AARP?s own analysis indicates that most seniors will receive a tax cut of between twelve hundred and fifteen hundred dollars per year. On Tuesday September 9th, I will vote YES for Amendment One and I want to encourage you to vote YES for Amendment One also. 167 APPENDIX G STATISTICAL TABLES Table G1 Overall Summary for Logistic Regression on Likelihood to Vote For/Against Amendment One (N = 250) -2 Log likelihod !2 Nagelkerke R 2 309.1 28.9* 0.14 Actual Predicted For Predicted Against Percent Corect For 101 58 43 43 Against 149 118 31 79 Overal percent corect: 64 * p < .05. 168 Table G2 Logistic Regression Analysis Predicting Vote On Amendment One (N = 250) Variable B SE Wald !2 p Ods Ratio Group A B -0.70 0.48 2.13 0.14 0.50 C -0.3 0.46 0.49 0.48 0.72 D -0.30 0.47 0.39 0.53 0.74 E 0.20 0.48 0.17 0.68 1.2 F -0.18 0.54 0.1 0.74 0.84 Party Republican Independent -0.06 0.34 0.03 0.86 0.94 Democrat -0.32 0.35 0.82 0.37 0.73 Recal Seniors -0.52 0.32 2.76 0.10 1.69 Gender 0.40 0.30 1.74 0.19 1.48 Age 0.05 0.02 5.71 0.02 1.06 Education 0.16 0.08 4.31 0.04 1.18 Income 0.14 0.10 2.23 0.14 1.16 Note. A = No mesage; B = One mail targeted mesage; C = Two mail targeted mesages; D = One mail targeted and one mail generic mesage; E = Two mail targeted and one telephone targeted mesage; F = One mail generic mesage. Table G3 Overall Summary for Logistic Regression Recalling Advertising (N = 315) -2 Log likelihod !2 Nagelkerke R 2 214.12 21.78* 0.13 Actual Recaled Predicted Yes Predicted No Percent Corect Yes 276 276 0 100 No 39 39 0 0 Overal percent corect: 88 *p < .05. 169 Table G4 Logistic Regression Analysis Predicting Recall of Advertising (N = 315) Variable B SE Wald !2 p Ods Ratio Group A B 0.63 0.60 1.1 0.29 1.8 C 0.53 0.61 0.76 0.38 1.67 D 0.56 0.61 0.83 0.36 1.74 E -0.06 0.60 0.01 0.92 0.95 F 0.02 0.62 0.0 0.97 1.03 Party Republican Independent -0.53 0.51 1.07 0.30 0.59 Democrat -0.76 0.48 2.49 0.12 0.46 Gender -0.64 0.41 2.47 0.12 0.53 Age -0.02 0.03 0.65 0.42 0.98 Education 0.21 0.1 3.72 0.05 1.23 Income 0.1 0.14 0.65 0.42 1.12 Note. A = No mesage; B = One mail targeted mesage; C = Two mail targeted mesages; D = One mail targeted and one mail generic mesage; E = Two mail targeted and one telephone targeted mesage; F = One mail generic mesage. Table G5 Overall Summary for Logistic Regression on Recall of Seniors Benefits (N = 215) -2 Log likelihod !2 Nagelkerke R 2 28.3 23.61* 0.13 Actual Recaled Benefits Predicted Yes Predicted No Percent Corect Yes 171 159 12 93 No 79 62 17 22 Overal percent corect: 70 *p < .05. 170 Table G6 Logistic Regression Analysis Predicting Recall of Senior Benefits (N = 215) Variable B SE Wald !2 p Ods Ratio Group A B -0.12 0.46 0.07 0.79 0.8 C 0.47 0.48 0.96 0.3 1.59 D 1.05 0.51 4.3 0.04 2.86 E 0.65 0.5 1.7 0.19 1.91 F 0.38 0.5 0.49 0.49 1.46 Party Republican Independent 0.49 0.37 1.68 0.2 1.62 Democrat 0.69 0.37 3.46 0.06 2 Gender 0.42 0.32 1.75 0.19 1.52 Age -0.03 0.03 1.36 0.24 0.97 Education -0.05 0.08 0.4 0.53 0.95 Income 0.37 0.1 1.1 0.01 1.4 Note. A = No mesage; B = One mail targeted mesage; C = Two mail targeted mesages; D = One mail targeted and one mail generic mesage; E = Two mail targeted and one telephone targeted mesage; F = One mail generic mesage.