By: Mark Fitzgerald Tribune Co. is blaming the circulation of a six-year-old article that briefly triggered a Wall Street panic squarely on Google's search technology.
Google's automated search agent, Googlebot, is simply unable "to differentiate between breaking news and frequently viewed stories on the websites of its newspapers," Tribune said.
In a statement issued late Wednesday, Tribune issued its third, and most detailed yet, explanation of how an old Chicago Tribune story archived in the Web site of the sibling South Florida Sun Sentinel managed to get picked up as new news, and trigger a frantic sell-off of United Airlines stock that sent shares plunged by more than 70% in the first half-hour of trading. Trading was halted in United, and the stock returned to normal levels after the market realized the Tribune article had been reporting not on any current plans to file bankruptcy, but the Airline's Chapter 11 reorganization six years ago.
This time around, Tribune said Google's automated news collection not only confused old news for new -- but did so despite warnings "months ago" by Tribune to stop crawling its newspaper Web sites.
Specifically, Tribune asserts the Dec. 2002 article that caused all the trouble for United Airlines stock last Monday was accessed from the archives of the South Florida Sun-Sentinel by a single user looking up stories about airline cancellation policies late on Saturday night. The user accessed the bankruptcy story.
At such a time of light use of the newspaper site, that single hit made the six-year-old article one of the "most viewed" items by default.
That would have been that, Tribune said, if Googlebot had not swooped in 52 seconds later, and crawled the story. "Shortly thereafter," Tribune said, "Google provided a link to the old story on Google News and dated it September 6, 2008. Google's dating the story on Google News made it appear current to Google News users."
Tribune added it had "identified problems with Googlebot months ago and asked Google to stop using Googlebot to crawl newspaper websites, including the Sun Sentinel for inclusion in Google News," but the search agent continued to crawl the Fort Lauderdale paper.
"Despite the company's earlier request and the confusion caused by Googlebot and Google News earlier this week, we believe that Googlebot continues to misclassify stories," Tribune said.
A Google spokesperson in a statement said, "The claim that the Tribune Company asked Google to stop crawling its newspaper Web sites is untrue." A Google blog also asserted the United story did not have a standard timestamp, so the bot accessed the current, Sept. 7, date on the top of the page.
Following is Tribune's summary of what happened with the United story:
"The article, headlined United Airlines Files for Bankruptcy,' was originally published in the Chicago Tribune in 2002, and appeared on the newspaper's website.
"It then became part of the online database of Tribune's newspapers. Our records indicate that the Googlebot crawled this story as recently as September 2 and September 3 and apparently treated it as old news.
"On September 7, 2008 at 1:00:34 ET, (Sept. 6, 2008, 10:00:34 PT) our records indicate that the article received a single visit.
"Given the fact that it was the middle of the night, traffic to the business section of the Sun Sentinel site was very low at the time.
"We believe that this single visit resulted in a link to the old article being created on a dynamic portion of the Sun Sentinel's business section under a tab called 'Popular Stories Business: Most Viewed.'
"Again, no new story was published and the old story was not re-published-a link to the old story was merely created. The URL for the old story did not change when the link appeared. On September 7, at 1:36:03 ET (Sept. 6, 10:36:03) a user of the Sun Sentinel's website viewing a story about airline policies regarding cancelled flights, clicked on the link to the old story under the 'Popular Stories Business: Most Viewed' tab.
"Fifty-two seconds later, at 1:36:57 ET (10:36:57 PT), Googlebot visited the Sun Sentinel's website again and crawled the story. This time, despite the fact that the URL to the old story hadn't changed, despite the fact that Googlebot had seen this story previously, it was apparently treated as though it was breaking news. Shortly thereafter, Google provided a link to the old story on Google News and dated it September 6, 2008. Google's dating the story on Google News made it appear current to Google News users.
"The first referral to the story from the link provided by Google News came just three minutes later, at 1:39:59 ET (10:39:59 PT). Traffic to the old story increased during the course of the day, Sunday, September 7, with the bulk of it being referrals from Google. On Monday, September 8, traffic increased even more after a summary of the Google News story was made available to subscribers of Bloomberg News."
Comments
No comments on this item Please log in to comment by clicking here