While I am certainly addicted to digital news and information, I also love to watch the national evening news the old fashioned way -- on television. I prefer NBC Nightly News with Brian Williams, even though the idea of a news organization having direct financial ties to a major defense contractor is pretty ridiculous (General Electric owns 49% of NBC Universal).
The other night, while watching NBC Nightly News, a story really jumped out at me. After a feature on the horrendous News Corporation phone hacking scandal, Williams segued into a segment on the national debt that included a new NBC News/Wall Street Journal poll. Excuse me? Isn’t the Wall Street Journal owned by Rupert Murdoch’s News Corporation? And didn’t the publisher of the Wall Street Journal just resign because of the phone hacking scandal? Williams never brought up those inconvenient facts.
That extreme example of modern newsroom incestuousness made me laugh, and in my mind it made the results of that poll if not tainted, at least strangely suspect. That incident got me thinking about the world of online polls, one of the true staples of digital information culture. Can you trust them? Are you supposed to trust them? What good are they?
The first place I turned to investigate the phenomenon was the site for Gallup Polls. I was absolutely shocked at what I found – there were no online polls! That’s right, the premiere polling organization does not use the Internet to conduct its polls, and there is no way for a regular citizen to make his voice known at Gallup.com. Not only that, according to a 2010 news release from the company, they are considering going back to shoe leather in this high-tech age:
Another important change was becoming obvious by the mid- to late 2000s: the increasing use of cell phones by Americans. As a result, by 2008, Gallup had shifted its interviewing to include traditional landline and cell phone sampling. Today, with increasing shifts in communication that may eventually move some Americans beyond any type of phone toward texting and written communication, there has been increased attention on the part of survey professionals to the possible benefits of a move back to “old-fashioned” residential, address-based sampling.
For now, however, the vast majority of Gallup surveys intended to represent the national population are based on interviews conducted by landline and cell telephones. This method builds off of the central assumption that most Americans still either live in a residence with a telephone or own a personal cell phone. So, reaching people on their telephones is the starting place for current national surveys.
That logical rationale actually made me feel really good about Gallup polls. At a time when they could easily begin to rely on faster, simpler, cheaper digital technology, the company is focusing on how best to maintain the accuracy of their information. Bravo.
Next, I surfed over to the other polling giant, Harris. It was a much different experience. Harrispollonline.com is essentially a simple page that encourages people to sign up and participate in their online polls. And if you do sign up, there’s a world of wonderful treats in store for you! According to the site, “To show our thanks for your support and participation, you'll be enrolled in our appreciation program, Harris Poll Online Rewards, where you can redeem points for rewards that include a variety of merchandise and gift certificates. You can also participate in our sweepstakes for cash prizes!”
Hmmm. Wouldn’t incenting people to take polls skew the results? If you are being given something in return for taking a poll, wouldn’t you want to please the pollster, and wouldn’t you be more willing to participate in a poll you had no strong opinions about in order to score some juicy cash prizes?
But what about polls on the big news sites? One of the major features on the front page of CNN is “Quick vote.” Recently, the question of the moment was, Do you expect Congress to raise the debt ceiling by August 2? I voted that I did, and after I clicked my response the results popped up. More than 57,000 votes had been cast, but at the bottom of the results was this strange phrase: “This is not a scientific poll.” OK. But what does that even mean? To me it means that “The Most Trusted Name in News” is publishing data that it doesn’t even trust.
Over at nytimes.com, there are no editorial online opinion polls on the home page. But the site does have an entire section called Poll Watch. That section, however, does not conduct online polls, it merely reports and analyses recent polls in the news. It also features the results of polls the paper conducts with CBS News, however those polls are “nationwide telephone polls” a la Harris, not digital surveys. The New York Times’ lack of interest in online opinion polling isn’t surprising, because it would be hard to imagine the paper publishing a phrase like “This is not a scientific poll” accompanying its reporting.
But while the Times may not embrace online polling as an institution, they do permit their advertisers to conduct these polls, sometimes in shocking ways. I recently saw a large promotion on the home page of the Times asking users to give their opinion on how much the richest 2% of Americans should pay in taxes. It was NOT labeled as an advertisement, and when I clicked on the ad it took me to defeatthedebt.com. I voted, was thanked for my vote, but there was no information on the site about the poll’s methodology.
I looked into that site and discovered that it is an arm of the Employment Policies Institute. According to SourceWatch, “The Employment Policies Institute (EPI) is one of several front groups created by Berman & Co., a Washington, DC public affairs firm owned by Rick Berman, who lobbies for the restaurant, hotel, alcoholic beverage and tobacco industries.” Does the Times endorse the results of those polls? I am sure they don’t, but I’m not so sure Times users understand that.
On the Times Poll Watch page, they include a Polls Navigator, which they describe as “A list of polling resources from around the Web selected by editors of The New York Times.” One of the links in the list goes to the well-respected Pew Research Center. On that site, they actually delve into the debate surrounding the usefulness of online polls. In their Ask the Expert column, someone submitted the following question: How statistically accurate is an online poll in which participants sign on to contribute their opinions? Scott Keeter, the Director of Survey Research at the Pew Research Center, responded thusly:
The accuracy of a poll depends on how it was conducted. Most of Pew Research's polling is done by telephone. By contrast, most online polls that use participants who volunteer to take part do not have a proven record of accuracy. There are at least two reasons for this. One is that not everyone in the U.S. uses the Internet, and those who do not are demographically different from the rest of the public. Another reason is that people who volunteer for polls may be different from other people in ways that could make the poll unrepresentative. At worst, online polls can be seriously biased if people who hold a particular point of view are more motivated to participate than those with a different point of view.
Keeter goes on to say that, “there is nothing inherently wrong with conducting surveys online,” and that Pew actually conducts some of its own polls online. The issue is not necessarily how the poll is conducted – online, in person, on the phone – but how the sample audience taking the poll is chosen. If the sample chooses itself, as with the polls on most mainstream news and entertainment sites, then the results are basically nothing more than a stab in the digital dark.
John D. Thomas is the former editor of Playboy.com.