News-Only Search Sites Proliferate

By: Steve Outing

As a current-awareness tool, news-only search Web sites are great. It wasn't long ago that having access to stories freshly published from publications around the world was just a dream, or available only from expensive clipping services. Now, we can search many of the world's newspapers for current stories -- for free. As the current advertising slogan for MCI says, "Is this a great time, or what!"

The idea behind the search sites is simple enough. These services regularly index current content from dozens or sometimes hundreds of news Web sites, so a search for "earthquake" will turn up stories from around the world about recent quakes. Search for "Denver Broncos" and you'll get stories from various publications, allowing you to compare coverage of Monday night's football game against the New England Patriots.

The sites are not designed to be archives, but do provide a valuable service for those looking for current news from more than one media source. The news search sites are especially useful for journalists, to find out what others have written about a particular topic, or to quickly get up to speed on an issue by reviewing recent coverage.

Existing current-news search services include:
New Century Network's NewsWorks, which lets you search current and recent news content from 130 U.S. newspaper Web sites. NewsBot from Wired Digital, which indexes dozens of news sites several times a day. Excite's NewsTracker, a clipping service with a search function that covers several hundred news sources. InfoSeek, which allows you to search selected news sources, such as wire services, major newspaper Web sites, and online news Web sites. WiseWire, a current awareness site that covers a broad range of Web sites, including news sites. Yahoo! includes a news search engine, but it only covers seven days of news from selected wires services. Two new news search services have also entered the field. Both take the approach of indexing many Web news sites frequently, without having a formal relationship with the sites they index. The model, at this point, is simply to gain advertising revenues to support the services.

TotalNEWS' entry

Introduced this week is Paradigm NEWS Search from TotalNEWS, the small Scottsdale, Arizona, company that made headlines when several major news organizations threatened to sue it for inserting their Web sites into frames of TotalNEWS' "mega-news" site (which had the effect of inserting TotalNEWS' advertising onto individual news sites' Web presentations). TotalNEWS is still kicking after its settlement with the publishers.

TotalNEWS president Roman Godzich explains that the search engine is based on an index of dozens of news sites updated every 3-4 hours. Paradigm currently indexes 200,000 unique URLs, with 30,000 to 50,000 being added per day -- ultimately up to a number of URLs in the low millions.

A problem with some news search sites is that news URLs expire quickly, because some news sites remove news content and put it into paid archives after a short period. Users conducting a search find an item of interest, but click on the resulting link only to find that the story no longer exists. Godzich says the solution to that problem is that during each reindexing by Paradigm, the system checks the validity of every URL it already has in its index, and kills those that are no longer functional. Also, URLs older than two weeks are killed out.

The Paradigm search engine also delivers what Godzich calls "keyword centered abstracts," which for each search hit present the paragraph of a story that centers around the searched keywords -- as opposed to delivering only the headline or the first paragraph of a story, as some other news search engines do.

Paradigm is currently a version 1.0 release, and future enhancements will include the ability to define the date range (freshness) of searched news. Competing news search engines like Wired's NewsBot already do that.

Godzich says that the TotalNEWS site itself seems to attract a user base of business executives, government officials, journalists, and others who have a high degree of interest in comparing news from various sources. "At first, we thought (TotalNEWS) would attract the Joe Sixpack crowd," he says. Instead, the site is pulling in a higher demographic, with 30% of users coming from outside the U.S.

Godzich envisions the Paradigm search engine as making the TotalNEWS site the logical place for news consumers to come when a major story breaks to more quickly find all the stories written about a particular news event. The site will give readers a chance to compare how different media cover a single story.

News Index

Another upstart in the news search engine space also believes that some consumers will find value in the ability to compare news sources' treatment of stories. News Index is a similar search engine focusing exclusively on worldwide news Web sites, produced by Sean Peck, a lone entrepreneur trying to get some notice for the new service. His service tracks 100-200 news sites, which are reindexed every 1-2 hours, making it useful for keeping track of breaking news on the Web.

In Peck's Web page describing the service, he writes, "Despite what every news outlet wants you to believe, no single outlet delivers the truth; they all deliver their version of the story, and only through reading different versions of a story can you finally derive some semblance of what actually happened."

Like TotalNEWS, News Index's model initially is to support the service with advertising and provide the service free to consumers. There is no formal relationship with any of the "content providers" -- the news organizations that actually produce the stories that the site links to.

Both TotalNEWS and News Index hope to license their technology for other uses, and each wants to get search links onto news sites themselves in an as yet unshaped business arrangement.

Publishers should not worry about these services "borrowing" their intellectual property. (They're not. They are merely linking to your content, not copying it for presentation to their own viewers.) They are providing a valuable service to publishers by bringing new traffic to news Web sites. Indeed, as I pointed out in a recent series of articles about permanent Web links for news stories, news publishers can help the news search engines do a better job of bringing them traffic by not allowing current news stories' URLs to expire too quickly.

Contacts: Roman Godzich, rgodzich@grouper.com
Sean Peck, speck@westol.com

Can't please everyone department

Stan Jones of the Anchorage Daily News comments on my column of last Friday, in which I lobbied for higher ethical standards for newspapers covering cyber industries in which they now find themselves as participants:

"If you want to tell us a story, tell it with names, dates and places. Otherwise fageddaboudit. A story that relies on an anonymous account (especially, in this case, a sole-source anonymous account!) to make its point isn't journalism, IMHO. At best, it's journalism on that slippery slope into ... television-land!"

Steve

Previous day's column | Next day's column | Archive of columns
This column is written by Steve Outing exclusively for Editor & Publisher Interactive three days a week. News, tips, and other communications may be sent to Mr. Outing at steve@planetarynews.com

The views expressed in the above column do not necessarily represent the views of the Editor & Publisher company