“Never send a human to do a machine’s job.”—Agent Smith, “The Matrix.”
Since we were children, movies and television have done a good job preparing us for an inevitable future where the intelligent machines we build to make our lives easier end up taking over and enslaving mankind.
Unfortunately, some of us carry these science fiction fantasies a little close to the chest. So when it was announced that the Associated Press was partnering with a company called Automated Insights to automate the process of writing quarterly financial results, media writers shifted into hyperbole quicker than Schwarzenegger’s “Terminator” ripped through time.
“Should We Be Afraid or Excited About Robot Journalism?” asked the Huffington Post.
“Computers Could Soon Replace Australian Journalists,” reported the Australian.
“AI Will Replace Human Journalists This Month,” warned the Examiner.
Even Marshall Brain, founder of How Stuff Works and author of “Robotic Nation,” put journalists on notice as being one of nine professions that will ultimately be replaced by computers.
Unfortunately, those viewing the news of automated reporting through the prism of the Syfy channel might be disappointed that not only is this new technology not going to replace reporters, it may actually free them up to do more of what they love to do: tell interesting stories.
How Wordsmith, the name of the platform created by Automated Insights, works is by amassing massive amounts of quantifiable data in verticals dominated by detailed numerical information, like financial reports, real estate and sports. It then uses specific algorithms to compare that data to itself to find interesting aspects to uncover, compiles them into a story, and uses parameters pre-set by producers to place adjectives and punctuation in a way that makes the report as readable as your traditional newspaper brief.
Lou Ferrara, the AP managing editor who oversees business news, notes that not only will this new partnership not replace a single reporter, “instead of providing 300 stories manually, we can provide up to 4,400 automatically for companies throughout the United States each quarter.”
In the case of the AP, Automated Insights takes its automation technology and pairs it with data from Zacks Investment Research, which is already being used by AP business reporters. By automating the process of doing earnings reports for all U.S. companies, Ferrara notes it will free up his journalists to focus on stories about what the numbers mean.
“This is about using technology to free journalists to do more journalism and less data processing, not about eliminating jobs,” Ferrara notes. “In fact, most of the staff has been receptive to the effort and involved for the past few months of discussion.”
Take Automated Insights’ partnership with Yahoo! for an example. If you play fantasy football on the Yahoo! platform, before and after every game you’ll be treated with a specific story about your matchups, players and opponent, as if it were written by a human being. That story is really only relevant to two people—you and the friend you faced off against this week—and would be infeasible to produce by hand. But those type of hyper-interested stories focused on an extremely small readership are now possible, as long as the data is available to mine.
“Instead of publishing one story they hope 10 million people will read, we’ve given them the capability to publish 10 million stories one or two people will read,” said Adam Smith, the vice president of business development and marketing at Automated Insights.
Smith knows what he’s talking about. In 2013, Automated Insights produced a staggering 300 million stories in verticals ranging from personal fitness to business intelligence. That’s nearly 10 stories a second.
Smith is downright giddy about the applications of the technology in journalism, and notes as it is refined and become more widespread, there are a number of avenues available to both niche publishers and smaller-sized newspapers and web sites.
“As long as you have the data, the application of the technology is almost limitless,” Smith says. “Stories about politics, budgets, crime, travel, weather, there are all kinds of things happening on a very-local level that have data that you could tell a story about.”
Over at the Los Angeles Times, they’ve taken automated reporting into action for a very relevant local issue people care a great deal about: earthquakes. Quakebot, an algorithm created by database producer Ken Schwencke, takes data from the U.S. Geological Survey about any earthquake greater than a 3.0 magnetite and converts it into a text story. The algorithm even adds a map and creates a headline, all within minutes of an earthquake first being detected.
According to Schwencke, Quakebot eliminates most of the grunt work in reporting on earthquakes. Schwencke gets the byline, but all he really has to do these days is respond to a ping from the algorithm, proofread and fact check the post, and set it to go live on the site. Think about it like this: the algorithm is looking at the data the same way a reporter would during the normal course of reporting a story.
“Algorithms can help you find and report the news,” L.A. Times database producer Ben Welsh told Journalism.co. “You can write code that will ask and answer the common questions that a reporter would ask when looking at that same dataset.”
The L.A. Times actually has several bots like this that help its reporters get a leg up on stories. One algorithm creates the first sentence in the paper’s Homicide Report, which produces a small story about every homicide in the Los Angeles area. Another algorithm sends reporters a daily email highlighting arrests made by the LAPD, paying special attention to cases that may be more worthy of more reporting, such as an abnormally high bail. Another practical place media companies could look to automated reporting is real estate. Real estate ads are not only a lucrative source of revenue for many media companies, the data-rich nature of the marketplace lends itself to the creation of bots and algorithms to create timely, unique content.
One company that is on the forefront of this type of reporting is Homesnap, a real estate search platform that uses data to create engaging content that’s hyper-relevant to local readers interested in purchasing properties. In fact, one of their best-known features is the Homesnap app, where you can snap a photo of any home using your smartphone and be treated to a wealth of information about the property.
Thank algorithms and those pesky robot journalists.
“The wizard behind the curtain is our immense database that houses all that information,” says Lou Mintzer, the senior VP of product development at Homesnap. Cobbled together over the course of a couple of years, Homesnap’s database cross-references traditional sources of real estate data, like MLS listings and county tax information, with things like Foursquare data and census results to compare, contrast and ultimately paint a picture of what someone interested in a house is truly going to get.
“It’s not feasible to have a beat reporter to cover an individual zip code. You’d need an army of thousands of bloggers to cover that beat,” says Mintzer, who is interested in partnering with local publishers to help them tell better stories about their local real estate marketplace.
So the core message to publishers, both large and small, is that automated reporting can help not only better manage the time of their reporting staffs, but lead to more relevant stories and even minimize mistakes.
Rob Tornoe is a cartoonist and columnist for Editor and Publisher. Reach him at firstname.lastname@example.org.