MEASURING WEB AUDIENCES STILL A CHALLENGE

By: Karim Mostafa and Carl Sullivan Newspapers Devise Their Own Counting Techniques


When it comes to reaching their local audiences, newspaper Web
sites in some of America's largest markets get mixed reviews. In
Los Angeles, Atlanta, Boston, and Washington, D.C., newspaper
sites are the No. 1 source for local news and information. But in
other markets, competitors such as CNN.com and CitySearch-
TicketMaster Online get far more visitors than online newspapers.

On the national level, USAToday.com and washingtonpost.com are
the highest ranking newspaper sites, each drawing about 3.5% of
the Internet audience in the U.S. That pales in comparison to the
top news site, MSNBC.com, which drew 12% of all online users in
the first quarter.

Those were the results of a Jupiter Media Metrix (JMM) analysis
of the nation's largest markets, comparing the first quarter of
this year with the same period for 2000. The news channels of
popular services such as America Online and Yahoo! are not
included because New York-based JMM doesn't measure those
services in its syndicated reports. Evidence suggests that those
services, which cull news and information from a variety of
sources, may be the top news providers in many markets. (To see
the top news and information sites for Atlanta, Boston, Chicago,
Dallas/Fort Worth, Los Angeles, New York City, Philadelphia,
Seattle, and Washington, D.C., click here.)

Some newspapers continue to criticize these numbers, claiming
that they don't adequately measure local markets. Publishers say
their own server logs show far more visitors than the ratings
services do. Many papers are now developing their own measurement
systems, combing their internal statistics with syndicated
research from JMM, its competitor Nielsen//NetRatings, and any of
a number of other Internet measurers.

Flawed measurement?

A year ago, the Newspaper Association of America formed a task
force to study the issue of online audience measurement. "We
initially thought a lot of the measurement was flawed," said
Randy Bennett, NAA vice president/electronic media and industry
development. The task force was created to get a handle on Web
measurement and methodology, he said.

Mike Donatello, former director of research at
Washingtonpost.Newsweek Interactive and now an industry
consultant, said that the task force's first goal was to
publicize the newspaper industry's view that syndicated services
such as JMM and Nielsen//NetRatings don't accurately reflect
online newspapers' penetration of their local markets. On this,
the task force made some headway, Donatello said.

The second goal was to try to effect some change. Not much was
accomplished here, Donatello said, although the task force did
provide an analysis of the various audience measurement services.

Bennett, along with others, said that the newspaper industry
alone is not going to change how these firms measure online
audiences. The theory is that newspapers are not big enough
customers of these firms to have that kind of influence.

Tim Stehle, senior director/research at KnightRidder.com in San
Jose, Calif., said that the difficulty of measuring local
audiences isn't just a newspaper problem. "It is a local-
audience-on-the-Internet problem," he said. Television and radio
Web sites are also trying to get a better handle on local
audiences, he said.

Lincoln Millstein, executive vice president at New York Times
Digital (NYTD), said that JMM and Nielsen//NetRatings are good
for a glimpse at sites' national reach. The combined network of
NYTD sites (including boston.com) just recently broke into Media
Metrix's Top 50. But the panels of Web users are too small to
accurately measure local audiences, Millstein said.

The newspaper industry's concerns about extracting local market
data from a panel that was originally developed for national
measurements is understandable, said Doug McFarland, president of
JMM. "I think the data are extraordinarily accurate within the
limitations," said McFarland, who said his company has worked
with the newspaper industry to refine the data.

"We have a number of initiatives underway right now to improve
the geographic distribution [in the top 30 markets], but it's not
going to answer all of their concerns," he said. If JMM surveys a
panel of 600 in any given market, the newspaper inevitably thinks
the company should increase the panel size to 1,200, McFarland
said. But that's not economically feasible.

"We talked with a lot of the large newspaper groups about putting
together projects [to better measure local audiences] for them,
but so far there hasn't been a major group to step forward to pay
for this," he said.

Slicing and dicing

For now, many publishers seem content to combine syndicated
research (often from more than one vendor) with their own in-
house statistics. Newspapers are slicing and dicing information
from this cornucopia of sources both for their own marketing
purposes and for their advertisers.

From a Scarborough Research report, NYTD discovered that 1
million users of NYTimes.com were not print subscribers. The site
began marketing the print product to those users, and so far this
year, more than 50,000 print subscriptions have been sold through
the site.

Meanwhile, NYTD uses @Plan to provide demographic data for
advertisers, according to Millstein. And there's a new "loyalty
index" that NYTimes.com has built in-house to measure which users
are coming back repeatedly. The "loyalty index" formula takes the
average number of daily visitors and divides it by the total
number of monthly unique visitors (a JMM figure) to get a
percentage of highly loyal users.

The loyalty index is used predominantly in-house as a retention
indicator, but advertisers are especially interested in the
average time spent by users during a single visit, according to
Millstein. For the month of May, NYTimes.com users were averaging
34 minutes and 9 seconds on the site, according to NYTD. The
"time spent" figure has been increasing as NYTD continues to
fine-tune the site's design. Millstein said that something as
simple as adding a navigation bar to every story page increased
the amount of time visitors spent on the site considerably.

JMM's McFarland has no problem with these creative uses of his
research. "I don't blame any company for using multiple research
sources," he said. That's particularly true because the different
researchers measure different things and cover different
geographic areas. For example, Nielsen//NetRatings measures
designated market areas (DMAs) while JMM measures metro areas.
"It's all about telling the best possible story," McFarland said.

Telling the best possible story often means explaining to
advertisers what all these numbers mean. While at The Washington
Post Co.'s new-media unit, Donatello labored hard to make sure
everyone understood that Nielsen//NetRatings only measures at-
home usage. That's important because approximately 50% of
washingtonpost.com's traffic comes during business hours -- most
of it presumably from office users. "People want black and white
research. The fact is that research is about shades of gray,"
Donatello said.

Dave Morgan, founder of the online advertising firm Real Media,
argues that the Internet is capable of providing black and white
numbers. "There's no reason to extrapolate audiences," he said.
"Audience measurement should be real and actual because the
Internet provides the technology to do that." Morgan recently
left Real Media to start a new company that will help publishers
collect and manage customer data.

Morgan thinks newspapers should strive for actual counts of their
online audiences, just as they have with their print editions.
And the Audit Bureau of Circulations is ready to audit newspaper
Web site numbers. So far, between 10 and 15 newspapers are
letting ABC Interactive audit their audience numbers, said Dick
Bennett, senior vice president and chief technology officer at
ABCi.

But, as KnightRidder.com's Stehle points out, ABCi is a different
kettle of fish. The Rolling Meadows, Ill.-based company doesn't
count visitors itself. All they're doing is verifying the process
that sites use to count visitors. Stehle suggests that we need
better counting that reflects an audience's true size before we
worry about auditing.

But with the current economic downturn, publishers don't have
unlimited resources to spend on online research. Research budgets
are being cut just as companies need more information to attract
potential advertisers.

"The good news is that we've come a long way in five years,"
Stehle said. "The bad news is we're not where we need to be to
support the businesses."



Karim Mostafa (kmostafa@editorandpublisher.com) is associate editor and
Carl Sullivan (csullivan@editorandpublisher.com) is editor of E&P Online.



Related stories:
NEWSPAPERS STRUGGLE WITH ONLINE AUDIENCE MEASUREMENT (05/26/00)
TOP AUDIENCE TRACKERS PROMISE BETTER LOCAL STATS (07/14/00)
NAA TASK FORCE RELEASES FIRST REPORT (06/09/00)
WASHINGTONPOST.com WINS ANOTHER AUDIENCE PRIZE (06/09/00)
GANNETT'S NEWFOUND WEB POWER (04/18/01)



Copyright 2001, Editor & Publisher.