|If you received a paper copy of (the first page of)
this handout, check out the online version (on the course web page) to
follow the embedded links and to view the supporting materials. The direct
URL for this document is:
All seven surveys were conducted immediately after the conclusion of the first Presidential debate between George W. Bush and Al Gore on Tuesday, October 3, 2000, in Boston. The question was who won the debate. Given the simplicity of the question, the effect of specific question wording (usually a very important aspect to be considered) is negligible. Hence, the differences in results are solely due to sampling method (determining the target sample) and "field administration" (realization of an actual sample). These seven surveys (polls) can be divided into two groups
With this caveat: A "margin of error" (better: sampling error) of 4 percentage points means that the "true" percentage in the whole population (here: among all registered voters of which there some 150 Mill.) is plus/minus 4 percentage points starting with the sample percentage. E.g., based on the ABCNews poll, we could say that we are 95% confident that 42 +/- 4 percent -- or between 38 and 46 percent -- thought that Gore won the debate. (95% is a commonly used level, a sheer convention; similar mathematical calculations can be performed at any given "confidence level".)
While all four of the serious polls have Gore ahead, the difference between Gore and Bush ranges from +3 (ABC) to +14 (CBS). Taking the margin of error into account, and based on the ABC and CNN polls, we cannot be 95% confident that Gore is ahead of Bush in the population of all registered voters. (In the media, this situation is often labeled as a "dead heat" -- a very unfortunate and rather misleading term.) But based on the CBS and MSNBC polls, we can be 95% confident that Gore is truly ahead of Bush. So, the four polls lead to different conclusions -- and there is no way to tell which of them is closest to the "true" situation, the percentages among all 150 Mill. registered voters. It is theoretically possible, but highly unlikely, that all four are off the mark and that among all registered voters Bush is ahead -- as the junk polls indicate. An article in the Washington Post ("A quick take on the quick polls") provides additional background on the four serious polls.
Note how much the results of the Web "junk polls" differ from the results
obtained with much more rigorous sampling. Of course, methods textbooks
use the term "availability" or "convenience" sampling (for which there
are also legitimate uses). I use the term "junk poll" to indicate the use
of convenience sampling without any justifying scholarly reason. And, in
general, junk polls are not restricted to the web; PIPs (phone-in polls)
are another genre of junk polls. But I could not find one for this particular
|Instant Polls after Oct 3 Bush-Gore Debate||Sample size||Sampling/admin method||Percentage
|491 registered voters||Telephone -- RDD||42||39|
|812 registered voters||Web -- random||56||42|
|CNN-USA Today (Gallup)||435 registered voters||Telephone -- RDD||48||41|
|MSNBC-Reuters (Zogby)||536 registered voters||Telephone -- RDD||46||36|
|CNN||40,882 page visitors
61,170 page visitors
61,197 page visitor
|Web -- junk (~8:30am)
|vote.com||34,568 page visitors
106,411 page visitors
120,236 page visitors
|Web -- junk (~8am)
(still open; result page)
|Harris/Excite||20,291 page visitors
124,268 page visitors
132,280 page visitors
|Web -- junk (~8am)
1. Results for junk polls are snap shots at specific times, first taken on Oct 4, 2000.
2. The sample for the CBS poll was constructed in several stages. First, a conventional RDD sample; people willing to participate in the study were given WebTV so that they could participate at the designated time via the Web. Check Knowledge Network web site for additional details.
3. Percentages do not add to 100 as there are additional response categories: "neither" and/or "don't know"
4. Links to the instant polls may become invalid as news organizations often display documents for a limited period of time only.
5. The commercial survey organizations actually conducting the polls are listed in parentheses after the sponsoring news organization.
What follows below are "screen shots" taken mostly on the morning of
Oct 4, 2000, to document some of the figures shown in the table and to
provide illustrations of how web junk poll are conducted. But there is
also some information about other serious web based surveys.
I have experienced problems with displaying the results page when using Netscape (4.75), MS Internet Explorer (5.5) seems to work better with this site.
|Vote.com sends how I voted directly to both candidates
Date: Wed, 4 Oct 2000 07:49:23 -0400 (EDT)
Dear Vote.com voter:
Your vote has been mailed on behalf of firstname.lastname@example.org to:
Gov. George W. Bush (email@example.com) presidential candidate, Bush
for President, Inc.
The email reads:
On the question Who Won the October 3, 2000 Presidental Debate?, I voted (erased to protect privacy)
Please consider my vote when making your decision on the issue.
If you would prefer not to receive this type of mailing, please click
the link below to unsubscribe:
Note that Harris also does serious web based surveys. Harris Interactive claims to be the "Global Leader in Internet-based Market Research". They have assembled a pool of over 7 Million respondents connected to the Internet and supposedly willing to participate in Harris online surveys. As home Internet users are distinctly different form the general population (in particular, they are more affluent and have higher formal education), stratified and weighted random samples are drawn from this pool to approximate a random sample of the general population. In contrast to Knowledge Networks (CBS Poll above), pool members are not given WebTV or any other assistance in order to connect to the Internet. Both methodologies are still in an experimental stage and there are different views among survey research professionals about the current level of achievement and the prospects for the future.
Personally, I strongly believe that telephone surveys will be replaced by web-based surveys as the main mode of administration, but only the future will tell for sure. However, keep in mind that not all web polls are junk polls.
|Excerpt from ABCNews web site at
|Excerpt from CBSNews web site at
Presidential debate watchers came away with the impression that Vice President Al Gore won the debate against Texas Governor George W. Bush. Over half - 56 percent - of debate watchers feel that Gore won the debate, while 42 percent think Bush did the better job.
Both candidates improved their image somewhat with voters through the debate. Thirty-five percent of those voters who watched the debate say that their image of Bush has changed for the better as a result. Gore saw a similar gain in post-debate popularity, with 32 percent saying their image of Gore had changed for the better as a result of the debate. Roughly one in five voters who saw the debate say they have lowered their opinions of each candidate as a result.
Bush made only marginal progress, however, in another key area - the perception that he is not prepared to be president.
In the post-debate poll, 54 percent said Bush was well prepared for the job of president. This shows only slight improvement from a CBS News/New York Times poll earlier this week in which 49 percent of voters said they thought Bush was well prepared for the job. In contrast, seven in ten voters said both now and earlier this week that Gore is well prepared for the job.
Vote preferences have changed little as a result of the debate. In a pre-debate survey, Gore had a four-point edge over Bush, 46-42 percent. Among debate watchers, Gore gained a few points, but Bush held his own. Fifty percent of debate watchers support Gore, 42 percent support Bush.
This CBS News poll was conducted online by Knowledge Networks among a nationwide random sample of 812 registered voters. Knowledge Networks polled a sample of registered voters in its household panel, which is a nationally representative sample of households given access to the Internet via Web TV. The margin of sampling error is plus or minus 4 percentage points for results based on the entire sample, and 4 points for results among those who watched the debates.