Chapter 2: AI in the newsroom
As more and more AI-powered tools become available, it’s difficult to know where to start if you want to start using AI in your newsroom. I’ll walk through a few options, as well as some things to think about when reporting on AI. The homework section of this chapter also contains resources for getting started with covering and using AI.
Generated articles?
Let's start by talking about articles generated by ChatGPT and other commercially available LLMs.
This may be the most obvious use case for generative AI in the newsroom. Generative articles have already been used to quickly write earnings reports and sports recaps for quite some time. But for more complex stories with multiple human sources and complicated topics, it’s not as easy as it sounds.
These models can’t be “taught” news judgment in the same way people can because there is no real process of “judgment” by which the model arrives at its output, only the statistical logic of prediction.
At this point, models like GPT-3.5 (which powers ChatGPT) do not work very well as information access systems. Their strength is their ability to act as “word calculators,” predicting the most likely next word in a sentence based on the input they are given. They are unable to discern between fact and that which merely looks like a fact. This is a main danger of publishing synthetic content: it’s all style and no substance.
However attached to it we are, the journalistic style is not the primary element that makes our work valuable. As John Herrman writes in “How Will Artificial Intelligence Change the News Business?” for Intelligencer: “The news is made novel, and valuable, by what it reveals, not the particular style of its revelation or distribution.”
There is also very little transparency around the data that models are trained on. Not only does this make synthetic content extremely difficult to verify without careful editorial oversight, it means that it’s hard to know who to hold accountable when a machine gets something wrong.
As Hamilton Nolan argues in “Writing the AI Rulebook” for Columbia Journalism Review, “AI lacks the accountability of a human journalist—and it can never be accountable, no matter how refined its algorithms get. An editor cannot have a difficult conversation with AI about why it made certain choices in a story; AI will always be presenting the mere appearance of transparency, never a true exploration of its decisions and motivations. It has no soul, it has no mind. If AI is neither accountable nor transparent, its work can never be ethically published as journalism.”
So if not for churning out articles, what is it good for?
Using AI
There are many uses for AI and machine learning in journalism that do not involve the creation of synthetic content. It’s very likely that your newsroom already uses one of these methods already. However, in all of these methods, it’s important to adopt a human-in-the-loop approach. This means creating an organized way to closely monitor the output these systems produce, and creating opportunities for thorough human oversight at every crucial step. The goal should not be to let the AI tools make decisions for you, but to let them assist the process.
Here are some, but not all of the ways AI is already being used in newsrooms. More examples can be found in the homework section:
Automated audio transcription: AI tools like otter.ai are very popular among journalists for their quick ability to process audio recordings of conversations, like interviews, and transcribe them into text. Journalists can check these transcriptions with the original recording in order to ensure their accuracy.
Summarization and analysis: Generative AI models can process large documents like public records and legal documents, and summarize them to speed up reporting. Andrew Calderon at The Marshall Project documented how he used ChatGPT to summarize different states’ rules around what books are banned in prisons and jails. He set up an organized human-in-the-loop approach, specifically in establishing how exactly ChatGPT would be used, how its input would be given, and its output would be verified.
Detecting trends: AI can be used to detect trends and outliers in time-series data about online activity. The Financial Times built a machine learning model to detect anomalies in user activity on their site to try and predict trending topics in the news based on previous activity. They documented the process in a detailed blog post here.
Tagging/categorization: Machine learning can be used to help keep consistent tagging systems for articles, or suggest new tags that editors may not have thought of. Another project at the Financial Times used unsupervised machine learning to dig through the site’s archive and suggest new clusters of articles based on patterns it detects in their language.
Fact checking: multiple newsrooms have used AI to automate their fact checking processes. One project, Claimbuster, matches claims made by politicians in debates and on social media to previously recorded fact checks. Journalists can then review these matches to more efficiently parse through what politicians are saying.
When using AI systems, not only is it important to establish guidelines for where human judgment will be applied in the process, it’s critical to understand how your newsroom’s data will be used in the greater AI environment. As generative AI systems are integrated into search engines, the information they use to churn out responses to users will be built on the vast archive of content on the web, which includes news articles.
Models like Chat GPT have already shown evidence of having memorized New York Times articles. Because of the opacity of these systems, it’s very hard for journalists to know if their work has been used to train an AI model. This creates a copyright issue that is not even close to being solved, and stretches far beyond the walls of the journalism industry.
Some publishers have taken legal action against companies like OpenAI, while others, like the Associated Press, have come to licensing agreements about how content will be used in training data. The development of more powerful AI tech, specifically AI powered news distribution systems, would result in a major shift in the technological power structure that journalists must navigate.
Reporting on AI
In addition to fostering a critical literacy of AI systems among their colleagues, journalists must do the same for their audiences. When reporting on AI, journalists should emphasize that AI systems are created by humans and for humans. They should understand the narratives that contribute to AI hype (which I will cover in the next chapter), and take care to emphasize that AI technology is the product of many human decisions. As the Associated Press’ guidelines to reporting on AI puts it:
“Stories about AI technologies should be explicit that these systems are developed, built and deployed by humans, even if the most advanced models can later train themselves without our intervention. As such, they reflect human choices and risk reflecting human biases, which could be present in the data that powers the model as well as the rules that guide it. They are also dependent on human labor, from engineers working in corporate labs to independent wage laborers hired to label the data powering certain models.”
Good journalism about AI should leave audiences with a better understanding of how AI tech works, in whose interests it is being built, who makes money off it, who is being exploited in its creation, and what it could mean for their own lives. Reporting on new AI tools should seek to test the limits of these systems, showing where they break and why.
When training data is available, reporters should rigorously interrogate the data, looking into its origins, relevance, and whether or not it was gathered and annotated in an ethical way. When training data isn’t available, journalists should be skeptical about why, and hold AI companies accountable for any lack of transparency.
Chapter 2: Homework!
Here are some resources for getting started with AI projects, strategy, and reporting on AI. This is a big one! If you’re short on time, I’ve put stars next to the most essential resources.
If you want a foundation for doing deeper research into journalism and AI:
- ⭐ Start with “Actually, it’s about Ethics, AI, and Journalism: Reporting on and with Computation and Data,” a Columbia Tow Center report by Mark Hansen and Bernat Ivancsics. This report details the history of computation in journalism, examining the idea of “the news as data” and the implications this has had on the industry.
- Also read “Artificial Intelligence: Practice and Implications for Journalism,” a report from Columbia’s Brown Institute for Media Innovation and the Tow Center.
If you want to read about practical examples of AI being used in the newsroom:
- ⭐ Start with the “AI Journalism Starter Pack,” a vast compilation of real-life AI journalism projects. This was created by JournalismAI at Polis, the journalism thinktank at the London School of Economics and Political Science.
- Next, read “AI and News: What’s Next,” by David Caswell for Generative AI in the Newsroom, a project edited by Nick Diakopoulos. This article gives a very detailed account of the possibilities for AI in the newsroom in terms of:
- Strategy
- Newsroom AI readiness and infrastructure
- Organizational structure for AI teams
- ⭐ The other blog posts in Generative AI in the Newsroom are also fantastic resources for detailed documentation of generative AI journalism experiments. Some of these experiments involve attempting to use AI to:
- Rate the newsworthiness of scientific articles
- Summarize policy documents
- Cover events
- Generate illustrations
- Fact check
- Find story angles
- Search engine optimize
- Assist local newsrooms
- Other resources in Generative AI in the Newsroom address pressing issues in AI and journalism.
- Here’s an article about writing newsroom AI guidelines.
- Here’s an article about the copyright implications of using synthetic content.
- Here’s a Google CoLab notebook which walks you through six tasks that generative AI can be used for.
- Also, check out the Financial Times’ Product & Technology Medium page, where they post detailed documentation for applications of machine learning in their newsroom. They have posts about using ML to predict trending topics, how they test different algorithms for vectorizing their articles, and how they use AI to cluster content to find new ways of categorizing their archive.
If you want to read about what other journalists think about AI being used for journalism:
- ⭐ Start with the Paris Charter on AI and Journalism. This charter, written by a Reporters Without Borders commission chaired by Maria Ressa, lays out ten key principles for the responsible use of AI in journalism. These principles offer a very useful ethical framework for newsrooms crafting their own AI policies:
- Journalism ethics guide the way media outlets and journalists use technology.
- Media outlets prioritize human agency.
- AI systems used in journalism undergo prior, independent evaluation.
- Media outlets are always accountable for the content they publish.
- Media outlets maintain transparency in their use of AI systems.
- Media outlets ensure content origin and traceability.
- Journalism draws a clear line between authentic and synthetic content.
- AI-driven content personalisation and recommendation upholds diversity and the integrity of information.
- Journalists, media outlets and journalism support groups engage in the governance of AI.
- Journalism upholds its ethical and economic foundation in engagements with AI organizations.
- Read the report “Generating Change: A global survey of what news organizations are doing with AI” from JournalismAI at LSE. This report gives a comprehensive survey of how journalists from over 100 newsrooms around the world are feeling about AI readiness, newsroom strategy, and AI literacy among their colleagues.
- There’s also a short video summary, and Columbia Journalism Review published an interview with one of the co-authors about the report.
- Columbia Journalism Review, NiemanLab, and Poynter Institute all have useful articles about pressing topics in journalism and AI. Start with these ones about:
If you want to learn more about reporting on AI:
- ⭐ Start with the Associated Press’ guidelines for covering AI, which provides key tips for covering AI in a useful and responsible way. There are also clear definitions of common AI terms like training data, and the distinction between artificial intelligence and artificial general intelligence. The guidelines emphasize the importance of explaining AI tech to readers in simple terms, and reporting on the social effects of AI technology. A few of their key points are to use reporting to show:
- How these systems work.
- Where they are deployed.
- How well they perform.
- Whether they are regulated.
- Who benefits and makes money as a result.
- Which communities may be negatively impacted by such tools.
- Read “How to report better on artificial intelligence” by Sayash Kapoor, Hilke Schellmann, and Ari Sen, published in Columbia Journalism Review. This guide gives extremely valuable advice from experts on how to see through AI hype, evaluate data and models, and consider the harms that certain AI systems can create.
- ⭐ Read “Eighteen pitfalls to be aware of in AI journalism” by Sayash Kapoor and Arvind Narayanan, from their Substack, AI Snake Oil. This post outlines some of the ways journalists tend to inadvertently contribute to AI hype through tropes like:
- Comparing AI systems to humans
- Perpetuating hyperbolic claims about AI
- Platforming AI companies without proper context or critique
- Failing to address the limitations of AI
- For a critical breakdown of one hype-filled piece of AI reporting, check out “On NYT Magazine on AI: Resist the Urge to be Impressed” by computational linguist and AI researcher Emily Bender. Bender goes through a New York Times Magazine article about GPT-3 (the LLM behind ChatGPT) from 2022 and takes it apart paragraph by paragraph, pointing out the particular ways the story contributes to AI hype. It’s also available in audio form.
- Check out “Better Images of AI,” a project which aims to help news outlets and marketers use better stock imagery of AI to more accurately cover the topic. Their guide takes a stance against the sci-fi inspired images of humanoid robots and glowing cyborg brains. They also have a useful curated library of more appropriate and interesting AI images to draw inspiration from.
If you want to read about the potential effects of AI on journalism business models
- ⭐ Read “The news media and AI: a new front in copyright law,” by Yona TR Golding for Columbia Journalism Review. This article gives an overview of the copyright implications created by Large Language Models trained on digital news.
- Read about the licensing deal reached between the Associated Press and OpenAI, which is the first of its kind.
- Read John Herrman’s Intelligencer piece “How Will Artificial Intelligence Change the News Business?” which outlines three theories (of varying degrees of optimism) for how AI could impact the business of journalism.