It Just Doesn't Add Up
As the country enters the last month of the fall political campaigns, newspapers and television stations all over the country are sponsoring an increasing number of polls, trying to assess voter opinions on candidates and issues.
Polls are the currency of campaigns, and voters can be excused for being a bit lost when it comes to decoding what a poll is saying.
Media outlets often try to interpret them for you, but they're getting better at giving you the details so you can interpret the data for yourself.
Take two of the Lexington Herald-Leader's recent front-page headlines: Monday, September 25th: "Bush has slim edge in 6th, poll reports," and Tuesday, September 26th: "Forgy has slight lead on Keller, poll shows."
If you'd only scanned the headlines, you'd be forgiven for believing that these candidates each held the lead in their campaigns.
However, the stories themselves do a reasonable job of explaining that, in actuality, each race being reported on is statistically too close to call, but the misleading headlines (written by a different staffer) give a totally different spin on the results.
The problem with the headlines in both instances is a failure to accurately represent the nitty-gritty details of the poll, especially when it comes to what the "margin of error" means.
The September 25th story reports results of a poll of 1000 likely voters in the 6th congressional district, of which Lexington is the largest community. The results show that 46 percent would vote for George W. Bush, 44 percent would vote for Al Gore, 2 percent support Ralph Nader, and 1 percent would cast a ballot for Pat Buchanan.
But since the poll has a margin of error of 3.1 percent, it's entirely possible that a different group of 1000 voters would show, say, 47 percent for Gore and 43 percent for Bush.
The story explains that the race is "neck-and-neck," that Bush may not have any lead at all, because the 2 point lead is within the 3.1 point margin of error.
The same problem is found in the September 26th piece on the race for the Central Kentucky state Supreme Court seat. The headline reports a slight lead for challenger Larry Forgy, when in fact his lead over incumbent James Keller is also within the margin of error.
The margin of error is often given short shrift by journalists, maybe because it muddies the waters of poll interpretation. (Simply stated, the larger the margin of error, the less confident you can be that the results closely mimic what you would have gotten if you'd polled the entire population.)
It would be much easier for readers if they could just look at the percentages and determine with clarity who was the leader.
But polling is not an exact science, for many reasons. For one, the average person is overwhelmed by annoying and intrusive telemarketing phone calls, and the tendency to hang up on a pollster is greater now than ever before. This particular poll doesn't report its non-response rate (the number of folks called who refused to participate at all), but does say that 3 percent of their respondents refused to tell the interviewers for whom they were going to vote.
"Non-response" is a huge problem in surveys of all kinds, because the opinions of the folks who hang up are just as important to know as those of the people who are willing to talk to an interviewer.
Another problem with the analysis of polling data is the parsing out of smaller subsamples of the data, without providing equivalent margins of error. For example, the September 25th poll was presented as indicating a "clear" gender gap, with more men supporting Bush and women supporting Gore. But when you break down a poll into smaller subgroups, the margin of error increases, and the gender gap may not be as clear-cut as their numbers would indicate.
It's hard to understand this tendency to declare a leader, when calling the race a dead heat is just as interesting a story.
This political season, the daily paper has taken the usual flak about its so-called "liberal" bias; Editor Pam Luecke recently wrote an editorial explaining the Herald-Leader's strategy for covering the candidates for president, and their efforts to produce unbiased coverage. Perhaps the paper was worried about appearing pro-Democrat if it described the races as even, when the Republican candidates in each case polled a bit ahead of the Democrats.
So it's a positive step that most media outlets now give audiences the raw numbers to interpret for themselves.
If they would just do a better job of interpreting and conveying the margin of error, audiences would have an easier time deciphering what they were saying.
HOME | THIS ISSUE | ACE ARCHIVES