Philippine Election Polls: Accurate Surveys or Bad Forecasts – Remember Truman!
Another survey result, another wide lead over whatever. Then there’s this presumptiveness and arrogance that if Noynoy Aquino wins, there automatically is massive cheating. Assuming a smooth, quick, and peaceful election – if the Bradley effect is not enough, there’s also the US 1948 Presidential Elections.
Philippine Election Polls: Accurate Surveys or Bad Forecasts – Remember Truman?
It was predicted that Thomas Dewey would defeat Harry S. Truman in the 1948 US presidential elections (just like Noynoy defeating Gordon or Villar). Well, I don’t want to spoil the story if you don’t know it yet. For those who do, just read along anyway.
The site About.com provides a good narrative on what happened during the 1948 US Presidential elections. Here goes:
While completing Roosevelt’s term, Truman was responsible for making the fateful decision to end the war with Japan by dropping atomic bombs on Hiroshima and Nagasaki; creating the Truman Doctrine to give economic aid to Turkey and Greece as part of a containment policy; helping the U.S. make a transition to a peace-time economy; blocking Stalin’s attempts to conquer Europe, by instigating the Berlin airlift; helping create the state of Israel for Holocaust survivors; and fighting for strong changes toward equal rights for all citizens.
Yet the public and newspapers were against Truman. They called him a “little man” and often claimed he was inept. Perhaps the main reason for the dislike for President Truman was because he was very much unlike their beloved Franklin D. Roosevelt. Thus, when Truman was up for election in 1948, many people did not want to the “little man” to run.
Political campaigns are largely ritualistic…. All the evidence we have accumulated since 1936 tends to indicate that the man in the lead at the beginning of the campaign is the man who is the winner at the end of it…. The winner, it appears, clinches his victory early in the race and before he has uttered a word of campaign oratory.1
— Elmo Roper
For four terms, the Democrats had won the presidency with a “sure thing” – Franklin D. Roosevelt. They wanted another “sure thing” for the presidential election of 1948, especially since the Republicans were going to choose Thomas E. Dewey as their candidate. Dewey was relatively young, seemed well-liked, and had come very close to Roosevelt for the popular vote in the 1944 election.
And though incumbent presidents usually have a strong chance to be re-elected, many Democrats didn’t think Truman could win against Dewey. Though there were serious efforts to get famed General Dwight D. Eisenhower to run, Eisenhower refused. Though many Democrats were not happy, Truman became the official Democratic candidate at the convention.
Give ‘Em Hell Harry vs. The Polls
The polls, reporters, political writers – they all believed Dewey was going to win by a landslide. On September 9, 1948, Elmo Roper was so confident of a Dewey win that he announced there would be no further Roper Polls on this election. Roper said, “My whole inclination is to predict the election of Thomas E. Dewey by a heavy margin and devote my time and efforts to other things.”
Truman was undaunted. He believed that with a lot of hard work, he could get the votes. Though it is usually the contender and not the incumbent that works hard to win the race, Dewey and the Republicans were so confident they were going to win – barring any major faux pas – that they decided to make an extremely low-key campaign.
Truman’s campaign was based on getting out to the people. While Dewey was aloof and stuffy, Truman was open, friendly, and seemed one with the people. In order to talk to the people, Truman got in his special Pullman car, the Ferdinand Magellan, and traveled the country. In six weeks, Truman traveled approximately 32,000 miles and gave 355 speeches.3
On this “Whistle-Stop Campaign,” Truman would stop at town after town and give a speech, have people ask questions, introduce his family, and shake hands. From his dedication and strong will to fight as an underdog against the Republicans, Harry Truman acquired the slogan, “Give ’em hell, Harry!”
But even with perseverance, hard work, and large crowds, the media still didn’t believe Truman had a fighting chance. While President Truman was still on the road campaigning, Newsweek polled 50 key political journalists to determine which candidate they thought would win. Appearing in the October 11 issue, Newsweek stated the results: all 50 believed Dewey would win.
By election day, the polls showed that Truman had managed to cut Dewey’s lead, but all media sources still believed Dewey would win by a landslide.
As the reports filtered in that night, Truman was ahead in the popular votes, but the newscasters still believed Truman didn’t have a chance.
By four the next morning, Truman’s success seemed undeniable. At 10:14 a.m., Dewey conceded the election to Truman.
Since the election results were a complete shock to the media, the Chicago Daily Tribune got caught with the headline “DEWEY DEFEATS TRUMAN.” The photograph with Truman holding aloft the paper has become one of the most famous newspaper photos of the century.
1. Elmo Roper as quoted in David McCullough, Truman (New York: Simon & Schuster, 1992) 657.
2. Ibid 657.
3. John Hollister Hedley, Harry S. Truman: The ‘Little’ Man from Missouri (New York: Barron’s Educational Series, 1979) 183.
Can the Philippines 2010 Presidential Elections be a Repeat of the 1948 US Presidential Elections?
There is still a possibility that the Gordon campaign can build up like a steamroller in the next 10 days.
Thus, if Aquino loses due to a Gordon victory – I seriously disagree that there is massive cheating. The calculus just doesn’t add up. For one, Gordon does not have the money, machinery, and wherewithal to cheat. For instance, all the political infomercials aired prior to the campaign period were blatant campaign violations – cheating. Who benefitted from the airing of all these ads? Exactamente – ABS-CBN😆 Cmon guys, maglolokohan pa ba ba tayo?
The winds changed overnight in the 1948 US Presidential elections – I don’t see any reason why it can’t happen in the Philippines. Truman won – not because of massive cheating.
It is also a stretch to compare Aquino to Truman because of the use of the word “little man” or the reference to being inept, given that Truman had a lenthy list of achievements unlike Noynoy.
Also, Aquino is more likely to say “let me get a consensus” instead of “the buck stops here” – the phrase is more consistent with Gordon’s persona.
Noynoy Aquino and the LP, like Dewey and the Republicans, are so confident they are going to win – barring any major faux pas – that they are make an extremely low-key campaign and avoiding debates – just like Erap Estrada in a previous presidential election.
10 days till election day (as of writing)- anything can still happen. Pollster.com points out the existence of divergent polls as well:
If a single survey with a sampling error of 3% (based on a 95% confidence level) shows Bush at 49%, we know with 95% confidence that if every voter in the country were interviewed for that survey, Bush’ support would lie somewhere between 46% and 52%. If the same survey has Kerry at 42%43%, his support could range from 39%40% to 46%. Thus, the 49% to 42% survey tells us with 95% confidence that the race could be anywhere from a dead heat to a 1412-point Bush lead.
There is the 95% – and there’s the 5%. Truman’s narrative belongs to the 5% – and it can be the narrative of the best man for the job, too – Gordon.
Understand The Polls Some More
What are the chances that SWS and Pulse Asia can be wrong, or for that matter, how can the SWS, Pulse Asia, and other polling agencies be wrong? Here are some possibilities:
Potential for inaccuracy
Polls based on samples of populations are subject to sampling error which reflects the effects of chance and uncertainty in the sampling process. The uncertainty is often expressed as a margin of error. The margin of error is usually defined as the radius of a confidence interval for a particular statistic from a survey. One example is the percent of people who prefer product A versus product B. When a single, global margin of error is reported for a survey, it refers to the maximum margin of error for all reported percentages using the full sample from the survey. If the statistic is a percentage, this maximum margin of error can be calculated as the radius of the confidence interval for a reported percentage of 50%. Others suggest that a poll with a random sample of 1,000 people has margin of sampling error of 3% for the estimated percentage of the whole population.
A 3% margin of error means that if the same procedure is used a large number of times, 95% of the time the true population average will be within the 95% confidence interval of the sample estimate plus or minus 3%. The margin of error can be reduced by using a larger sample, however if a pollster wishes to reduce the margin of error to 1% they would need a sample of around 10,000 people. In practice pollsters need to balance the cost of a large sample against the reduction in sampling error and a sample size of around 500–1,000 is a typical compromise for political polls. (Note that to get complete responses it may be necessary to include thousands of additional participators.)
Another way to reduce the margin of error is to rely on poll averages. This makes the assumption that the procedure is similar enough between many different polls and uses the sample size of each poll to create a polling average. An example of a polling average can be found here: 2008 Presidential Election polling average. Another source of error stems from faulty demographic models by pollsters who weigh their samples by particular variables such as party identification in an election. For example, if you assume that the breakdown of the US population by party identification has not changed since the previous presidential election, you may underestimate a victory or a defeat of a particular party candidate that saw a surge or decline in its party registration relative to the previous presidential election cycle.
Over time, a number of theories and mechanisms have been offered to explain erroneous polling results. Some of these reflect errors on the part of the pollsters; many of them are statistical in nature. Others blame the respondents for not giving candid answers (e.g., the Bradley effect, the Shy Tory Factor); these can be more controversial.
Since some people do not answer calls from strangers, or refuse to answer the poll, poll samples may not be representative samples from a population. Because of this selection bias, the characteristics of those who agree to be interviewed may be markedly different from those who decline. That is, the actual sample is a biased version of the universe the pollster wants to analyze. In these cases, bias introduces new errors, one way or the other, that are in addition to errors caused by sample size. Error due to bias does not become smaller with larger sample sizes, because taking a larger sample size simply repeats the same mistake on a larger scale. If the people who refuse to answer, or are never reached, have the same characteristics as the people who do answer, then the final results should be unbiased. If the people who do not answer have different opinions then there is bias in the results. In terms of election polls, studies suggest that bias effects are small, but each polling firm has its own formulas on how to adjust weights to minimize selection bias.
Survey results may be affected by response bias, where the answers given by respondents do not reflect their true beliefs. This may be deliberately engineered by unscrupulous pollsters in order to generate a certain result or please their clients, but more often is a result of the detailed wording or ordering of questions (see below). Respondents may deliberately try to manipulate the outcome of a poll by e.g. advocating a more extreme position than they actually hold in order to boost their side of the argument or give rapid and ill-considered answers in order to hasten the end of their questioning. Respondents may also feel under social pressure not to give an unpopular answer. For example, respondents might be unwilling to admit to unpopular attitudes like racism or sexism, and thus polls might not reflect the true incidence of these attitudes in the population. In American political parlance, this a phenomenon is often referred to as the Bradley Effect. If the results of surveys are widely publicized this effect may be magnified – a phenomenon commonly referred to as the spiral of silence.
Wording of questions
It is well established that the wording of the questions, the order in which they are asked and the number and form of alternative answers offered can influence results of polls. For instance, the public is more likely to indicate support for a person who is described by the operator as one of the “leading candidates”. This support itself overrides subtle bias for one candidate, as is lumping some candidates in an “other” category or vice versa. 21st century Polling arms variate in complexity due to these circumstances. Thus comparisons between polls often boil down to the wording of the question. On some issues, question wording can result in quite pronounced differences between surveys. This can also, however, be a result of legitimately conflicted feelings or evolving attitudes, rather than a poorly constructed survey.
A common technique to control for this bias is to rotate the order in which questions are asked. Many pollsters also split-sample. This involves having two different versions of a question, with each version presented to half the respondents.
The most effective controls, used by attitude researchers, are:
* asking enough questions to allow all aspects of an issue to be covered and to control effects due to the form of the question (such as positive or negative wording), the adequacy of the number being established quantitatively with psychometric measures such as reliability coefficients, and
* analyzing the results with psychometric techniques which synthesize the answers into a few reliable scores and detect ineffective questions.
These controls are not widely used in the polling industry.
Another source of error is the use of samples that are not representative of the population as a consequence of the methodology used, as was the experience of the Literary Digest in 1936. For example, telephone sampling has a built-in error because in many times and places, those with telephones have generally been richer than those without.
In some places many people have only mobile telephones. Because pollsters cannot call mobile phones (it is unlawful in the United States to make unsolicited calls to phones where the phone’s owner may be charged simply for taking a call), these individuals will never be included in the polling sample. If the subset of the population without cell phones differs markedly from the rest of the population, these differences can skew the results of the poll. Polling organizations have developed many weighting techniques to help overcome these deficiencies, to varying degrees of success. Studies of mobile phone users by the Pew Research Center in the US concluded that “cell-only respondents are different from landline respondents in important ways, (but) they were neither numerous enough nor different enough on the questions we examined to produce a significant change in overall general population survey estimates when included with the landline samples and weighted according to US Census parameters on basic demographic characteristics.”
This issue was first identified in 2004, but came to prominence only during the 2008 US presidential election. In previous elections, the proportion of the general population using cell phones was small, but as this proportion has increased, the worry is that polling only landlines is no longer representative of the general population. In 2003, a 2.9% of households were wireless (cellphones only) compared to 12.8 in 2006. This results in “coverage error”. Many polling organisations select their sample by dialling random telephone numbers; however, there is a clear tendency for polls which included mobile phones in their sample to show a much larger lead for Obama than polls that did not.
The potential sources of bias are:
1. Some households use cellphones only and have no landline. This tends to include minorities and younger voters; and occurs more frequently in metropolitan areas. Men are more likely to be cellphone-only compared to women.
2. Some people may not be contactable by landline from Monday to Friday and may be contactable only by cellphone.
3. Some people use their landlines only to access the Internet, and answer calls only to their cellphones.
Some polling companies have attempted to get around that problem by including a “cellphone supplement”. There are a number of problems with including cellphones in a telephone poll:
1. It is difficult to get co-operation from cellphone users, because in many parts of the US, users are charged for both outgoing and incoming calls. That means that pollsters have had to offer financial compensation to gain co-operation.
2. US federal law prohibits the use of automated dialling devices to call cellphones (Telephone Consumer Protection Act of 1991). Numbers therefore have to be dialled by hand, which is more time-consuming and expensive for pollsters.
An oft-quoted example of opinion polls succumbing to errors was the UK General Election of 1992. Despite the polling organizations using different methodologies virtually all the polls in the lead up to the vote, and to a lesser extent exit polls taken on voting day, showed a lead for the opposition Labour party but the actual vote gave a clear victory to the ruling Conservative party.
In their deliberations after this embarrassment the pollsters advanced several ideas to account for their errors, including:
Voters who changed their minds shortly before voting tended to favour the Conservatives, so the error was not as great as it first appeared.
Conservative voters were less likely to participate in surveys than in the past and were thus under-represented.
The Shy Tory Factor
The Conservatives had suffered a sustained period of unpopularity as a result of economic difficulties and a series of minor scandals, leading to a spiral of silence in which some Conservative supporters were reluctant to disclose their sincere intentions to pollsters.
The relative importance of these factors was, and remains, a matter of controversy, but since then the polling organizations have adjusted their methodologies and have achieved more accurate results in subsequent elections.
For those who want to read up some more on polls, here are some links recommended by Harvard U:
- The American Association for Public Opinion Research is the main professional organization for public opinion researchers. They have put together a page of helpful information on pre-election polls. http://www.aapor.org/electionpollingresources2
- If you really want to learn how to interpret polls in the media, take the on-line course journalists take: “Understanding and Interpereting Polls.” This course by the Poynter Institute is targeted to journalists, but is free and available to anyone. http://www.newsu.org/courses/course_detail.aspx?id=aapor_polling07
- Mark Blumenthal of Pollster.com has a great set of FAQ’s.
- Rutgers Professor Cliff Zukin has a useful resource talking about pre-election polls: “Sources of Variation in Election Polls: A Primer” (Cliff Zukin). http://www.aapor.org/uploads/zukin_election_primer.pdf