As AI Technology Advances, So Does its Ability to Assist Journalists

Illustration by Tony O. Champagne

Artificial intelligence typically evokes a mixed bag of emotions from people across all industries. The pair of words can easily conjure a dystopian image where workers are replaced by machines more effective than their human counterparts. While this fear was once relegated to the confines of science fiction, advancements in technology have only further exacerbated this fear.

In reality, artificial intelligence has already entrenched itself into our daily routines. Thanks to AI, Netflix is able to offer suggestions for the next series you’ll find binge-worthy. Meanwhile, popular apps such as Google Maps, rely on complex algorithms to suggest the most convenient and efficient routes for you and your destination.

For journalists, the question is not a matter of if but when AI becomes a permanent fixture in their own newsroom. And as some news organizations are already discovering, AI can often offer a number of benefits toward enhancing—instead of replacing—the quality of journalism it produces.

Journalists meet in the newsroom at the AP’s new global headquarters in New York City. (Photo courtesy of TPG Architecture)

Setting the Standard

A few years ago, the Associated Press recognized that competing in a digital, oversupplied news market required a fresh approach to the way they had been doing things. With more than half the world’s population viewing its content every day, the news agency not only had to do more, but do better.

“We needed more volume to enable more choice and satisfy additional needs from news organizations we serve,” said Francesco Marconi, strategy manager and AI co-lead. “We also needed to build differentiation, so we can distinguish AP from all others. AI enables us to accomplish both.”

The AP’s first foray in the field of AI focused on automation of repetitive tasks such as writing news articles that follow a template structure. This automated model has primarily been used to create larger numbers of financial stories and sports recaps.

Prior to its partnership with the artificial intelligence company Automated Insights, an AP staff of 65 business reporters could write about 6 percent of earnings reports possible of America’s 5,300 publicly held companies. Just two years later, the AP’s AI system had the ability to generate 3,700 quarterly earnings stories.

“In addition to increasing news coverage and extracting hidden insights from data, AI can improve processes such as automatically tag photos, generate captions for videos and even deploy AI powered cameras to capture angles not easily available for journalists,” Marconi said.

Last year, the AP began using the same software to cover more than 10,000 minor league baseball games across the country. By using data from box scores, stories can go on the wire within minutes of the final pitch.

Additionally, the AP has experimented with Raspberry Pis, a low cost, easily programmable computer, to construct a prototype concept capable of calculating the ground vibration at construction sites and noise level from entertainment and political venues. The prototype could indirectly determine important factors such as the most beloved songs at a concert, critical plays of a game or compelling moments in a campaign rally.

“The successful implementation of AI in the newsroom should focus on freeing journalists from time intensive and repetitive tasks,” Marconi said. “It’s not only imperative to save time and money in an era of shifting economies, but also find ways to keep pace with the growing scale and scope of the news itself.”

Instead of spending valuable work hours transcribing interviews and manually poring over datasets, a reporter’s daily duties could be focused on making calls and pursuing leads derived from an AI analysis.

Of course, that isn’t to say that the introduction of AI into journalism comes without its own set of challenges. According to Marconi, maximizing AI’s potential in the newsrooms requires new training and skills.

“Artificial intelligence is complicated, and there are many ways it can be implemented in an editorial setting,” he said. “Just like any other technology, the more you know about a tool, the more effectively you can use it.”

The Washington Post has developed a number of in-house AI tools to help enhance its journalism. (Photo provided)

An Open Mind

As one of the leaders of innovation in the newspaper industry, the Washington Post has witnessed firsthand how AI can benefit the newsroom. Any new technologies that can help reporters more efficiently and effectively perform are welcomed, said Jeremy Gilbert, director of strategic initiatives.

When it comes to AI, Gilbert said he believes it can offer journalists a comprehensive set of tools and techniques to speed up production, identify stories and eliminate tedious tasks.

“Artificial intelligence will be as valuable to journalists as the introduction of personal computers and smartphones,” he explained. “AI is not about replacing reporting or editing. It will make journalists better at all the things they already do and serve readers with the news they want, in more personal ways and on a far vaster number of topics.”

For more than a year, the Post has experimented with Heliograf, its in-house automation tool built specifically for speed, scale and personalization of storytelling. The program’s software recognizes relevant data, pairs it with corresponding phrases in a template and publishes various versions of the story across different platforms. The Post first utilized the tool to produce several hundred short reports and alerts for its readers throughout the 2016 Rio Olympics.

“Human editors and reporters are critical for shaping what stories are worth telling and how these stories should be told,” Gilbert said. “Heliograf tells far more of those stories.”

After the Olympics, the paper managed to replicate its initial success with Heliograf during the presidential election, where it updated more than 500 stories every 90 seconds.

“That cadence is impossible even for a newsroom as talented as the Post,” Gilbert said. “In addition, it freed up human reporters to do more than report the publicly available statistics. They were free to dig for the deeper stories behind the numbers—to get more and better interviews and to analyze the news as it happened.”

Since then, the Post has expanded use of Heliograf to cover all Washington, D.C. area high school football games every week. They also produced Heliograf-written articles about individual players and teams as the season progressed.

The positive impact of AI has been felt by the Post’s comment moderators as well. Due to the high volume of online traffic the paper’s stories attract, it’s not uncommon to see articles draw several hundred comments within a matter of minutes.

In order to address this problem, the Post’s engineering team developed Modbot, which uses machine learning to filter through comments that require moderating, and approve or delete comments based on the paper’s discussion policy.

Director of digital news projects Greg Barber noted that their comment moderators previously spent at least a quarter of their time approving comments there were “generally anodyne.” A main source of comments in violation of the rules stem from newly created accounts.

“That took up a tremendous amount of our resourcing,” he said. “Thanks to Modbot, we now have a service that checks and scores each comment, allowing our human moderators to focus on comments that are the best, the most problematic, or ones Modbot has trouble parsing—tasks more focused than serving as a human checkpoint.”

Meanwhile, the Post has continued to explore the possibilities of using Amazon’s own AI technology to deliver the news in non-traditional ways. Earlier this year, the paper launched a summer experiment where it used Amazon Polly to translate written articles into lifelike speech. Over the course of a month, mobile readers had the option to listen to content, instead of read it. Users could select up to four stories a day as part of the trial.

According to Joe Price, senior product manager, his team converted more than 150 articles to audio in select categories such as business, lifestyle, technology and entertainment.

“While some readers engaged heavily with the audio, we found that it was more manually intensive than anticipated to scale for optimal text-to-speech conversion,” he said. “We’re watching how text-to-speech technologies like Polly evolve and continue to explore how we can serve loyal readers who want to listen to the Post.”

Quartz built a workshop for its team last year at its new headquarters in New York City. Pictured is Sam Williams, director of the workshop. (Photo courtesy of Mark Craemer)

A New Conversation

Exploring and experimenting with the possibilities of bots in journalism has been a substantial theme at the digital news site Quartz. Their app, which was released last year, presents its content in a conversational interface unlike any other typical media outlet.

Bot developer and product manager John Keefe alluded to an experiment currently underway with a Facebook Messenger chatbot about the Netflix show “Stranger Things.”

“Fans can use the bot no matter when they decide to dive into the show, and no matter how many episodes they may have seen,” Keefe said. “We keep track of where they are and carefully avoid any spoilers.”

Given the success of the app and recent experiments, Keefe said he believes the chat-like interfaces are here to stay.

“How news and information providers fit into that world is still being worked out, and that’s what we’re exploring in the Quartz Bot Studio,” he said.

Launched in November 2016, the bot studio focuses on experimenting with applications of bots, AI and related technologies for journalism on new platforms. Most recently, the studio unveiled a suite of Slack-based tools designed to help journalists do their jobs every day called Quackbot. At the moment, it does very simple things such as look up reliable data sources or take a screenshot of a website for you. Eventually, Keefe said it will extract text from image PDFs, make simple charts and watch websites for changes.

For example, a local reporter could use the tool to create a bot that oversees a city’s police department’s website and be notified any time there’s an update.

Keefe acknowledged that malicious bots which act in the guise of humans are real, and as with viruses and phishing scams, there remains a need for newsrooms to understand how they work and inform people about them.

“It’s also pretty clear that all software reflects the biases of its creators and any data used to teach it,” Keefe said. “As people trained to consider bias in our reporting, writing and editing, it’s important to consider such biases in the machine-learning software and models we create too.”

Getting the Facts Right

Among the most basic, yet essential, functions AI can positively impact is the fact-checking process. As a result of several grants totaling more than $1 million, The Duke Tech & Check Cooperative is currently developing new ways to automate fact-checking.

During the two-year project, journalism faculty and computer scientists from Duke and a pair of other universities will build a variety of new tools and apps to assist reporters verify claims.

“Fact-checkers can use AI to do the most tedious and repetitive tasks,” said Bill Adair, the Knight Professor of the Practice of Journalism and Public Policy at Duke and leader of the Tech & Check Cooperative.

For example, it’s not uncommon for reporters and editors to spend lots of time looking through transcripts of cable news shows and legislative proceedings for claims to check. By working with a team of computer scientists at the University of Texas at Arlington, Adair said they are automating that work so that their bot, ClaimBuster, scours the transcripts and recommends the most promising checkable claims.

“That will help local papers that don’t have the resources to spend hours going over transcripts but still want to do an occasional fact-check,” he said.

Though the prospect of automated fact-checking may no longer be an illusion, Adair emphasized that the technology is still not quite capable of completely replacing human journalists.

“The robot overlords aren’t taking over fact-checking,” he said. “They’re just helping journalists discover the best claims to a fact-check and identifying previously published fact-checks when politicians repeat a claim. These are good first steps.”

Stephen Masiclat, director of the New Media Management program at The S.I. Newhouse School of Public Communications, said the ability for AI to report like a human journalists is a skill that will be “well beyond machines for many, many years.”

“People wouldn’t be forthcoming if they were being interviewed by chatbots,” he said. “To get sources to confide or open up, you need human reporters.”

According to Masiclat, it’s actually television journalists that should be worried. “As soon as there is a system to animate a synthetic news anchor from a script, the glamour jobs in television news will be taken over by machines. Count on it.”

The AI system Masiclat said he’d try to design for a newsroom would aim to help journalists better connect with audiences by tracking the time people spend with various articles and maintaining a record of the most read stories.

“We would call these ‘high engagement’ articles, and they would be unique to each reader,” he said. “The AI then analyzes these articles to discern the characteristics of high engagement articles such as article structure, word choice tendencies and sentence complexity.”   As a result, every reader would have a unique set of preferences for high engagement content, and the system would understand the mechanical characteristics of each reader’s preferred stories. Once journalists report, interview, and gather primary information and write a bare-bones, just-the-facts story, the AI engine may customize it according to the discerned preferences of individual readers.

“Each reader gets a version of the story ‘written’ in the voice and style they engage with best,” Masiclat said. “The net effect would be that a single journalist could reach and engage with larger and more diverse audiences.”

Additionally, readers would receive articles that they find more engaging, according to Masiclat, which in turn will help raise the perceived value of the publication.

“Journalists could spend more time reporting and cultivating sources knowing that the system would help them produce the final stories,” he said. “Advertisers would welcome higher engagement and would likely want to use the same data to customize ads more likely to engage with readers.”

However, the system Masiclat envisions also admittedly comes with some potential ethical challenges.

“First and foremost, who owns the writing style? If the publisher creates and owns the AI that derives the writing style, then do they own it?” he said. “What if a reader’s preferred style is predominantly derived from the style of one or two writers, do they have any claim to the output? These kinds of second-order effects questions will likely inform whether we consider AI a net benefit or problem.”

How Newsrooms Today are Using AI

The Guardian

Last year, the paper launched its Chatbot via Facebook messenger. The chatbot allows users to pick from U.S., U.K. and Australian versions of Guardian news and select the time it will deliver news stories to them every day.

New York Times

In 2016, the New York Times announced its partnership with Jigsaw to improve moderation of its comment sections. By using the Jigsaw software, the Times’ community desk is able to increase the speed at which comments are reviewed. Prior to the partnership, the paper’s team of 14 moderators manually reviewed about 11,000 comments each day.

USA Today

The newspaper uses Wibbitz, an AI-driven production software, to create short videos. The tool can condense news articles into a script, string together a selection of images or video footage, and even add narration with a synthesized newscaster voice.

DB Corp

DB Corp, India’s largest newspaper group, publishes 62 editions in four languages and sells roughly 6 million newspaper copies per day. In order to address the growing number of customers and diverse readership, reporters use Google Cloud Translation to document interviews and source material for articles.

Reuters

Reuters uses News Tracer, a proprietary algorithm that relies on more than 700 signals to determine whether trending topics on Twitter are newsworthy and trustful. Reuters journalists taught the tool to ask key questions, consult historical data, weigh relevant information and make a decision within 40 milliseconds.