The Web's 'Unique' Problem

By: Carl Sullivan More than a few online publishers have a hard time believing the "unique user" line in their own log files. How could a print newspaper with a circulation of 50,000 have 200,000 unique users?

Out of market visitors? Former readers who moved away? Double counting from the same user who logs on both at work and at home? One-hit visitors who follow a link to one story and then leave and never come back?

All of these scenarios are valid, but could these factors alone account for such high "uniques," as they're called in the online biz.

For the last year or so, Belden Associates of Dallas has been trying to get a handle on this "unique" problem. This week, the group is releasing a preliminary report on the "real story" about newspaper site traffic. "Few people believe their own unique user number, but many are reporting the numbers anyway" to advertisers and other interested parties, says Greg Harmon, director of interactive services. And quite a few newspapers don't report their unique users, preferring instead to emphasize page views -- since they're more confident in the accuracy of the reporting on those numbers.

The lack of confidence about uniques leaves newspaper Web sites uncertain about their reach, frequency and true audience composition, Harmon says. As a result, "newspapers have been unable to settle on rate and pricing models for their sites that they confidently believe reflect the real value of their site audience to advertisers," he adds.

In frequency, Belden's review of newspaper Web sites finds that nearly every site reports only two or three visits per month by each user. "This has been very frustrating for Web managers," Harmon says. "Seemingly no matter what they do, they can't get above that two to three visits per month" -- at least according to the log files.

Belden's online surveys of newspaper Web site users reveal much higher frequency, with a significant proportion saying they visit once a week or more. Similar surveys by other companies via telephone and other means find similar results.

So what gives? Cookies, Harmon says.

After conducting over 70,000 interviews with users at 40 different newspaper Web sites during the last three years, Belden has come to the conclusion that regular users are loyal, experienced and competent. "They know how to use their browsers," Harmon says, pointing out that commonly over 60% of a typical newspaper site's visitors have been coming to the same site for over a year.

Based on that profile, Belden felt comfortable asking more complicated questions about user behavior, such as "How often do you clear the cookies on your machine?"

The surprising result at one site: 45% said they clear cookies at least once a month. The average was once a week. "For work and home, this behavior was consistent," Harmon explains. Many may not be using browser tools to remove cookies, but rather "spyware" and "computer hygiene" tools that have become more popular as the public worries about unauthorized snooping on their online activities.

The implications are obvious. "If a user is clearing cookies once a week from home, the Web sites he or she visits are counting many people at home as four different unique users over the course of a month," Harmon says. "The unique user numbers are vastly inflated, but at the same time, frequency of visiting is undercounted. What this says is that frequency on newspaper Web sites is actually much higher. Plus Web sites are nowhere near the threat to print that people are thinking, because Web sites don't have the gigantic audiences that circulation executives are worried about."

Belden cautions that it currently only has this data for three sites in smaller markets, and will be following up in more surveys at other newspapers later this year. But the preliminary data suggests that regular users (which Belden estimates comprise 40% of total users at most newspaper sites) may be counted up to eight times each on the uniques log. "Many visitors may be counted as many as eight times. If this holds true, then visitors may average between eight to 18 visits per month, possibly more," Harmon says. "This is very good news for publishers and advertisers" because it shows that online newspapers are competitive with local TV and radio in terms of audience reach and frequency.

Steve Yelvington, vice president for strategy and content at Morris Digital Works in Augusta, Ga., thinks Belden is on to something. "Not just on the cookie-clearing issue, but on the importance of analyzing the audience in segments," he says. "We've all been swayed by averages that don't accurately reflect the behavior of our loyal core segment. We're stronger than we had believed in some key audience retention and habituation measures."

It should be noted that Belden has a vested interest in promoting their theory. Their report says "The single best method to discover how visitors use sites and to determine reach and frequency is to ask visitors directly and actively through survey research and develop models based on respondent data" -- which is actually the type of service that Belden offers.

Meanwhile, the panel-based research firms point out that their research doesn't rely on cookies or log files. "Cookies wouldn't have any impact on our user panels," says Graham Mudd, Chicago-based senior analyst for comScore Media Metrix of Reston, Va. "One of the often stated benefits of a panel-based approach is that it allows you to look at unique users regardless of cookies and duplication [individual users using more than one computer]."

Nielsen//NetRatings also reports that cookie-clearing has no impact on their research because they're monitoring consumer behavior with their own technology, which doesn't rely on cookies.

Ironically, Belden's new research may actually benefit the panel-based researchers. For years, some online newspaper executives have blasted Media Metrix and Nielsen//NetRatings for undercounting newspaper Web audiences. "Our log files show far more users than the panels," was the common refrain. Well, Belden's research may prove that the panel researchers were closer to the mark all along.

It may be some time before there's a "winner" in the accurate audience measurement contest, but as the researchers perfect their methodology, online publishers have the most to gain. The next steps for Belden? A team of academics, including Rich Gordon of Northwestern University; Clyde Bentley at the University of Missouri, Columbia; and Alan Hubbard at the University of California, Berkeley; will review the data and publish an analysis. Stay tuned.

For more information, contact Harmon at gharmon@beldenassociates.com.