An Accurate Online News Poll? Maybe

By: Steve Outing In a recent column, I wrote about the dangers of online polls presented by many news Web sites. The biggest problem with them is that they often get results that are not even close to the results of traditional scientific public opinion polls on the same topic. I argue that such online spot polls -- even when presented with disclaimer language -- can do a grave disservice to the public.

Short of limiting news site online polls to trivial topics where inaccurate results won't do much harm ("Do you like or dislike Monica Lewinsky's hairstyle?"), there may be other ways to conduct online polls on weighty subjects and get results that are reasonably similar to general public opinion. One Australian entrepreneur is among those who think this is possible.

Leni Mayo is CEO of Melbourne-based Moniker Pty. Ltd., a company that specializes in scalable object-oriented middleware for connecting databases and transaction processing software to the Internet. Earlier this year, his company started development of an online polling, survey and voting service called Votebot, which is designed to assist distributed communities in forming group opinions on issues.

The Votebot Web site, which is currently in beta mode, offers a free service where anyone can run an online poll. These "public votebots" have pretty much the same problem as the online polls I mentioned in my previous column -- self-selecting electors and some electors using multiple e-mail addresses to stuff the ballot (ergo, the results aren't likely to be statistically valid). But Moniker also is offering "private votebots" which can be used to conduct online polls in a more controlled way in order to attain legitimate results.

How to do it right

The key, says Mayo, is that in an online poll the participants must be randomly chosen from a carefully chosen sample subset of the total audience. The process should be similar to a telephone poll, where pollsters select a sample set from which to select random phone numbers. It's the reliable mapping from demographics of the sample set back to the general population that give the results statistical validity, Mayo says.

Mayo is currently looking for a media company (or market research firm) to help prove out his concept for valid online polling. He's looking for a large news site that has a large set of e-mail addresses and some good demographic information on those users -- that is, the site needs to require user registration. Mayo hopes to prove the concept by running a controlled online poll alongside a telephone poll on the same topic, then comparing the results. Selected e-mail addresses chosen from the news site's registered users will be invited to participate in the online poll.

The most important point is "the ability to create a statistical correlation between the online electors and the real world," Mayo says. The polls would be conducted matching the demographics of the electorates, probably by choosing a particular Zip code in a U.S. city and focusing on online registrants who live there. The results of phone vs. online polls can be compared, as well as the cost of running each poll.

Even though all the controlled online poll participants will be Internet users, "we think this won't matter -- at least in developed countries -- because enough people have e-mail to allow measurement of the bias. ... The results could be presented in exactly the same way as telephone polls, with a disclaimer explaining why they hold water in a statistical sense. ... Our line of argument is that the significant difference between the telephone surveys and the (existing) online polls is the way the electorate is chosen, not the fact that the poll is taken electronically."

Mayo is bullish on online polls because "the incremental cost per voter is almost zero, so the economics allow very large electorates. This means that the results could potentially be more accurate than telephone polls."

The Yahoo! poll?

Moniker, obviously, would like to be the technology provider behind media companies' online polls. Mayo envisions a future where media organizations from the New York Times on the Web to Yahoo! conduct carefully controlled online polls that are taken as seriously as today's polls by big-name media organizations. Moniker's focus has been on scaling the early Votebot technology so that it works with hundreds of thousands of voters. The company did some marketing around the recent Australian federal elections, and now hopes to get noticed as the U.S. election season heats up.

The market research industry is an obvious potential user of such technology, but Mayo complains that its practitioners tend to be conservative in adopting new technology. Thus, he hopes media companies will be the first to try out and help prove the concept.

At this point, with the technology and concept of statistically valid online polls as yet untried in a real-world environment, Mayo's interest is in finding a partner willing to underwrite a telephone poll to go along with the online poll for comparison purposes. Moniker also would seek to have its costs covered. "To run a quality study would require time and effort from the news organization, including the costs of the telephone polling," he says.

The Votebot technology can be run off Moniker's servers, or the company licenses the software.

Contact: Leni Mayo, leni@moniker.net

Get Stop The Presses! by e-mail

If you would like to get e-mail delivery of the Stop The Presses! column, there are now two options:

1) Text e-mail. I send out a text e-mail message containing a brief description of the current column, along with a URL link to the actual column on the E&P Web site. To receive these regular reminders, sign up here.

2) HTML e-mail If you have a mail reader that can handle HTML messages, have the entire column delivered to you whenever a new one is published. Sign up here.

Steve

Got a tip? Let me know about it

If you have a newsworthy item about the newspaper new media business, please send me a note.

This column is written by Steve Outing for Editor & Publisher Interactive. Tips, letters and feedback can be sent to Steve at steve@planetarynews.com