It's How You Count, that Counts Most

By: Jennifer Saba The disparities are startling. Check out the monthly Web audience numbers for The News & Observer in Raleigh, N.C.: In September 2006, ABCi tallied 1.3 million unique users. ComScore Media Metrix tracked 358,000 million unique visitors. And Nielsen//NetRatings counted 990,000 uniques for the same period. But why?

Of course, the methodology used by each company bears different results. But in this case, and in several other instances, the data swings so much it's hard to tell just how many people are truly visiting newsobserver.com. It helps at the very least to understand how each firm arrived at the number, and the various ways the counting can be done.

The Audit Bureau of Circulations, through its division ABCi, audits Web sites by surveying internal server logs. "We have access to the Web server log files that record all the transactions the Web server performed during a month," says Scott Hanson, ABC's senior vice president of electronic and centralized audit services.

He adds, "When you start with a foundation that is census-based, you have access to all the information. You are not reliant on sampling or panels that may not represent that particular market very well." Every page and every file served is recorded on the internal log. It reveals how many people are coming to view a Web site ? but not necessarily who.

From there, ABCi says it scrubs the data of any software that could artificially inflate the audience.

Sounds fairly airtight, but it's far from perfect. One of the drawbacks, Hanson says, is that internal server logs could count a user multiple times. For example, one person can visit the same site using different Internet browsers (say, Firefox and Internet Explorer). Or if a Web site user clears his cookies (segments of data stored on a user's computer every time a Web surfer visits a particular site), they could be counted multiple times in a monthly period.

Nielsen//NetRatings and comScore both use panels to determine the size of a Web site's audience. As Mainak Mazumdar, vice president of product marketing and measurement science at Nielsen//NetRatings, explains it, "Every household in the U.S. has a chance of being selected" through random-digit dialing. If a person accepts, he downloads software at work and at home. Currently, Nielsen//NetRatings has a sample pool of some 30,000 people.

ComScore uses methodology similar to that of Nielsen//NetRatings, with a larger panel (some 120,000 people). Executive Vice President Jack Flanagan says by using comScore, a client can get a more complete view of the competitive landscape and user demographics.

There are disadvantages to panel-based measurement firms ? they don't count public computers, such as in libraries or cafes. And recruiting people to download software on their computers at work is also problematic, since many companies have policies against it.

Flanagan sums up different measurement services and methodologies this way: "They have very unique selling points, and should be used accordingly."