Paid Content Trend Is Dangerous

By: Steve Outing The trend of news Web sites charging for content is clear -- and it's dangerous.

No, the trend is not that news sites en masse will start locking down all their content from those not willing to pay -- there's little likelihood that more than a handful of news organizations will copy the model of The Wall Street Journal's online edition and charge subscription fees to one and all. (As Carl Sullivan reported last week for E&P, only 21 daily newspapers currently restrict their Web sites to paying customers.)

What is likely is that an increasing number of news sites -- including the most prominent ones -- will create more and more "premium" content, for which they will charge fees to view. That's additional online material that sites offer along with the free stuff (which I expect will continue to account for the majority of most news sites' overall content).

This is a burgeoning problem, because all that "good stuff" that publishers think is worth charging for will, for the most part, be visible only to users of those individual publishers' Web sites. News search engines, which send plenty of user traffic to news sites, won't see or refer their users to such material. And that has potentially dire implications for both hybrid-revenue-model news sites and for the search engines.

Search industry expert Gary Price worries that as more and more content goes paid, effective Internet search becomes more and more difficult.

Bad news for news publishers

Let's start with why this is bad for news sites; later, I'll discuss the outlook for news search engines, such as Google News, the hottest player in that space right now.

Right now, if you go to Google News, you'll see headlines from a wide range of news sites -- some 4,000, with content crawled and indexed every 15 minutes or so. This is the global digital newsstand, and it's free. It's quite an achievement to have most of the world's news organizations' fresh content amassed in one place, and all free. What a wonderful consumer convenience, albeit one with a dubious economic model.

But while you'll find links to articles from The New York Times, Los Angeles Times, The Guardian, etc. on a visit to Google News, you won't find:

* Stories from The Wall Street Journal.
* Articles from the paid areas of FT.com, the Financial Times' site.
* Content from the paid services of TheStreet.com.
* Subscription-only articles from the Salon Premium area of Salon.com.

Following this trend, you don't have to drive far to see that there's a problem that has not yet been addressed. If a growing number of Internet users are finding news stories by using news search engines, then they have no way of knowing about this wonderful crop of content that's so good that it carries a price tag.

The boom in premium content is enlarging the "Invisible Web" -- the term that Gary Price and co-author Chris Sherman use in their recent book by the same name. The Invisible Web is the huge amount of information that's on the Internet but is not visible to the major Internet search services (Google, Yahoo!, AltaVista, Lycos, HotBot, and so on). For Invisible Web content, you have to know that it's there or know the "secret" to accessing it.

In the case of fresh premium news content, most often the only way to access it is via the news site that publishes it. Ergo, it's mostly invisible. That's a pretty lousy marketing proposition. Many publishers are now thinking through the process of creating online content that people will pay for, but they're missing a key ingredient to the formula -- utilizing the massive traffic of Web search engines to let Net users know about this great stuff.

Bad news for search engines

Google News is proud of the scope and breadth of its news selection. But a growing wave of premium content could potentially degrade the public perception that this is a world-class, comprehensive news tool.

Google executives don't think this is likely to happen -- but I'm not so sure. Google senior research scientist Krishna Bharat, who was the lead developer of the Google News service, says "this is more a problem for them [news publishers] than for us," because there's so much quality free news content available throughout the Web that Google News users will always have a cornucopia of sources. News-site publishers have a challenge in figuring out how to entice consumers to pay for premium online content when there's so much still available for free, he says.

Google spokesman Nate Tyler, while sticking to that line of thinking, does admit that having fewer free news sources available on the Web "would be bad for everyone," but he's not ready to predict how the premium-content trend will play out.

The Google juggernaut

Web news search engines have been around for quite a while, but while I don't mean to discount its competitors (Moreover, AltaVista News, AlltheWeb, Yashoo! News, DayPop, etc.), Google News I think will have the greatest impact in this space. Google doesn't publicly release figures on the traffic it generates, nor estimates of how much a particular part of its service generates in relation to the whole, but third-party studies have suggested that Google has grown to account for about 75% of all Internet search queries. You can imagine, then, the impact that Google News -- which is new and still in beta -- will have on referring Web users to news organizations' content.

It's incredibly important for news publishers who produce premium content to have it be part of Google News and other news search engines. Those placements could greatly boost the content's profit potential.

Is that likely to happen? Bharat and Tyler won't tip their hands about future strategy for Google News (as is routine company policy at the company), but they did leave the door open to someday having paid content be part of Google News search results. "Are we willing to crawl paid content? That's a policy decision we'd have to make," says Bharat, though it's certainly technically feasible. One thing he emphasizes, though, is that if Google were to add paid news content to its crawlers, "we would not take money" to place that content higher on the search-results pages. That ethical approach to search results is part of why Google is so popular with consumers (along with world-class technology and a great brand name that's becoming as widely used as "Kleenex" and "Xerox").

Tyler sends this message to news publishers: Google is open to having discussions with the news industry about paid content or any other issue that concerns both parties.

They'll do it, eventually

Search-industry expert Danny Sullivan, editor of SearchEngineWatch.com, thinks that eventually the search industry will begin to crawl and index paid content -- though he's not aware of any currently operating initiatives. What he envisions and expects to happen is that a news search engine will begin to offer both free and paid Web news content. Most likely, the consumer will experience it as a paid subscription service -- say, a Google News Premium -- that gives paying users access to paid content from a list of sites offering it. The news-search engine and participating news publishers would share the revenues. (Alternatively, of course, paid-content links would rely on individual news sites' payment mechanisms.)

There are a couple of ways this could be executed by a willing news-search engine. First, paid content from news sites is crawled by the search engine's spiders and indexed along with freely accessible articles. Results are presented such that paid-content articles are tagged as such, so that users know what to expect before they click, and paid articles are ranked according to identical criteria as free-access articles.

The second possibility is that the news-search engine creates a secondary element on the search-results page that features all the paid-content links separately. Search for "Bush tax cut" and you get a list of free news stories in a main window, plus a secondary box containing links to paid stories on that topic. (Google.com -- the main search engine, not the news-search one -- already has precedent for this, with its paid contextual text ads in the right column.) Obviously, this is a big improvement over a news results page that omits coverage from, say, The Wall Street Journal on the Bush tax cut. Under this scenario, news consumers willing to pay can have it all -- and news-search engines and paid-content Web publishers both benefit.

Sullivan points out that this model is not new. The Northern Light search engine has been offering a mix of free and paid news search results for several years, but it never really took off. He's not sure if that was because of inadequate marketing (outside of the librarian/researcher community, Northern Light isn't a well-known brand) or because Northern Light's mix-of-paid-and-free-content model came along too soon to grab a hold in the market.

Sullivan thinks (and I agree) that news publishers offering premium paid content should start thinking about how to let search engines know about it. His SearchEngineWatch.com offers a mix of free and paid content, so it's an issue dear to his heart. He points to Inceptor's eLuminator "content optimization service," which automatically extracts key words and phrases from hidden content (paid-access or hidden behind firewalls) and creates optimized pages that can be crawled by search engines. Paid-content publishers should be employing such solutions to let the rest of the world know what they've got.

What else is the news-publisher community to do about this issue? The ideal thing would be for the biggest news publishers to form a consortium for the purpose of negotiating with Google and other news-search services -- striking up a revenue-sharing arrangement for search-engine referrals to paid content and hashing out technical details for allowing the search engines to crawl for-fee articles. Sullivan's not holding his breath on that happening. (The news industry doesn't have a great track record of new-media cooperation -- i.e., New Century Network.)

Fly in the ointment

The prospect of search engines with massive user bases referring online users to paid content of course sounds great. But if it does happen, there's still the important issue of payment. While Sullivan's idea of paid premium-content subscriptions that share revenues between news-search engines and publishers is a possibility, that won't be easy to set up -- with so many players involved.

The simpler scenario is that the news industry convinces Google News or another news-search provider to crawl and link to paid premium content, but payments (and referral fees) are handled by individual news sites. The trouble with that scenario, suggests Nick Denton, former CEO of news aggregator Moreover and now a Weblog entrepreneur, is that it would be a consumer nightmare -- with online users asked to pull out their credit cards for each different site that owns a piece of premium content.

Denton is pessimistic about the idea of news-search engines linking to paid content -- until an Internet-wide payment system is devised that can be used to purchase content at any and all sites. Till that day comes, he suggests, a paid-content component of Google News or other news-search engine is a pipe dream.

While there remain many obstacles, the idea still has merit. This should be an industry priority through 2003.

---

Letters

Is it worth the money?

Great resolutions. I would add one small recommendation related to your first resolution, on "fixing archives." The link to the paid archives should offer some indication of the depth of coverage that's available, either by way of volume (i.e., 15k) or, better yet, a word count and any indication of photo and graphics coverage. I've actually paid to get an archived story after reading the lead, only to find there's just two more paragraphs. If I've got to pony up money, let me know if it's worth the time it takes to pull out a credit card, as well as my dinero.

Bill Sutley

---

Times archive solution

Regarding the "archive mess," a very nice solution to this problem is actually offered by The New York Times. As far as I've been able to work out, the URL to any Times story really works forever. But if you use the NYTimes.com archive to find that same story, you have to pay. I don't know if they lose some revenue this way, but it certainly is a great service to the user. It shows that the Times people have understood and care for the Web, despite their registration-required and paid-archive policy (the optimal policy for the user is of course when everything's free, but we all know news Web sites' need more revenue).

Olav A. Ovrebo

---

Salon listens

In regards to your New Year's resolution article: We don't offer an option to pay to read a single Salon article because we've surveyed our audience several times on the matter and overwhelmingly they respond that they aren't interested in consuming our premium content in that way. They were much more interested in paying to access our content for a short duration like a month (or a few months) which is why we offer that option.

Just thought you should know that we -- and I expect other sites -- are not blind to the wishes of our readers. We're giving them the options they've indicated that they are most interested in.

Patrick Hurley
Senior Vice President of Business Operations
Salon Media Group Inc.

---

Pie in the sky

I usually hang on your every word, and buy into about 90% of them.

But, alas, your resolutions for the new year betray you as an ivory-tower egghead. (It's OK; I'm still a fan.)

The fact remains: For all but the largest regional operations, news content delivery and production remain a cottage industry, understaffed and poorly understood. So your suggestions are brilliant but essentially pie in the sky.

Take, for example, the item about expired stories and pointers to the archives. Maintaining that would cost 4 to 6 man-hours daily, 365 days a year. Does the archive budget cover that? Highly doubtful. That stone must yield as much blood as possible. And so on.

The reality is: You should think profit, not prophetic.

Bill Zapcic
Online Editor
Home News Tribune

---

Links and redesigns

Just a quick response to your very good point about rotten links and pages being moved. Webmasters everywhere seems to be extraordinarily ignorant on this issue. Even the Norwegian government didn't manage to keep old addresses alive when they changed layout 18 months or so ago.

The solution, when publishing with database-based content, is extraordinarily easy (we have done it at www.kommunal-rapport.no, though sadly only at our most recent remake).

In the new database, make a table that refers to both the new ID/URL of the article and the old URL. When someone accesses the old URL, the site finds this in the database and returns the new URL, preferably with a 3-second message that the page has moved, and that you should update your links accordingly.

The next time we change systems (if we ever do...), we will keep two generations of old links alive by adding another set of old URLs in the database. This way, theoretically, we will never experience another rotten link at our site.

Database information also makes it quite unnecessary to move pages in order to be put into a paid archive. We have solved this by making every page of a certain template a paid article the moment it passes 14 days of age. URL stays the same and everyone who links to us (some news pages may be linked to by thousands of web sites in Norway) will contain a healthy link.

Ole P. Pedersen
Internet editor
http://www.kommunal-rapport.no

---

Other recent columns

New Year Resolutions: Fix Archives, Ban Pop-ups, Wednesday, Jan. 8
2003 Predictions for Online News Biz, Wednesday, Dec. 18
Walking the Convergence Talk, Wednesday, Dec. 11
Convergence For the Common Good In Rochester, Wednesday, Nov. 27
Have Newspapers Lost Help-Wanted Ads For Good?, Wednesday, Nov. 13
News Sites Need To Go On Diets, Wednesday, Oct. 30
Previous columns

---

Stop the Presses! by e-mail

To be notified by e-mail when a new "Stop the Presses!" column has been published, sign up on our home page.