How Norway’s public broadcaster uses AI-generated summaries to reach younger audiences

1. The project in a nutshell

The Norwegian Broadcasting Corporation (NRK) recently began adding AI-generated summaries to many articles published on its website to appeal to younger audiences.

These summaries are made up of a handful of bullet points, which appear on published article pages in a pale blue box section just below the headline and feature image, with only the top of the summary visible unless the reader chooses to expand the box. Below is a brief message reading: “The summary is made by an AI service from OpenAI. The content is reviewed by NRK's ​​journalists before publication.” I talked to NRK’s Knut Sætre and Thomas Nikolai Blekeli, who developed the tool, to find out more.

2. The problem they wanted to solve

Norwegian audiences are moving away from traditional news sources. This is especially true for young people. 

Norwegian statistics bureau Statistics Norway published the latest update in the Norwegian Media Barometer on 14 May. The Barometer collects data on media use (not necessarily for news) from a representative sample of the Norwegian population. The latest figures show a continued decline in the use of traditional forms of media, such as print newspapers, television and radio, as audio, video and use of the Internet rise. This pattern is particularly marked among young adults, a much sought-after and often elusive audience segment for traditional news media outlets. 

Print newspaper consumption is very low in Norway among 16-24-year-olds, at only 3%, compared to 17% of the total population. TV and radio use is also in decline, while video and audio media such as podcasts are very popular.

Across all age groups, the use of typical news sources such as newspapers, radio and TV is decreasing both in their traditional form and their internet counterpart. Accessing news articles via the internet is also seeing a decrease from 2022 after a couple of years of slow rise. 

This decline worries NRK. “We are losing young readers, but the whole media industry is losing young readers. And it's really hard to attract young readers to our stories, to our ordinary news sites,” Sætre said.

“About five years ago, we developed some user principles after qualitative insights with a target group, aged 19 to 29. These principles contain several important guidelines for better reaching a younger audience. One of those principles was to break content into manageable chunks,” Blekeli said. 

In particular, the team wanted to allow readers to get a quick overview of the content so they could decide if it was worth their time. However, this extra task would be tough to fit into the journalists’ busy workflow. “I thought, why not use AI as a tool to accomplish the tasks we can't get done ourselves?” Blekeli explained. 

“We also discussed the difference between a lead text and a summary. When I started as a reporter, a lead would be kind of a summary, but more recently we use it more as a teaser … a bullet point summary seems to work better,” Sætre said. “Maybe it’ll be a small tool to make the stories a little more accessible to young readers,” he said.

3. The problems they encountered

The size of NRK as an organisation means changes can be slow to occur. “It takes time for lots of bureaucracy,” Sætre said. He and Blekeli had to present the idea to several different teams. “Usually one of the standard challenges is to anchor our new product across different editorial newsrooms and departments,” Blekeli added. Working on the tool also had to go hand in hand with developing an ethical framework for the editorial use of AI.

Scepticism from journalists less willing to adopt AI tools was less of a problem. “That's probably because we were around them, had presentations and took questions. Of course, some were sceptical, but all in all, it's been quite painless,” Blekeli said.

But what about hallucinations? These falsehoods keep popping up on output from large language models such as ChatGPT or Google's Gemini. "We are aware that there is a small possibility that hallucinations may happen. That is why we have strict guidelines for the use of these models, highlighting that the summaries must be reviewed carefully. Line by line. We also emphasise to journalists that the summarised text is generated by AI, but is not a fact-checker," Blekeli said.

4. How the tool works in practice

Journalists can choose whether or not to use the AI system NRK set up to create a summary, and some still opt to write it themselves without AI input, Sætre said. Those who want to use the AI-supported option copy the article URL into the system before publishing. This analyses the text using OpenAI technology and returns the summary along with three suggestions for the article headline: an ordinary headline, an SEO-optimised headline and a teaser headline for the front page, Sætre explained. The summary is ready to be copied and pasted into the NRK's Content Management System (CMS).

The system is currently not integrated into NRK’s CMS, which the broadcaster is in the process of replacing, but it will be integrated in the coming version, Sætre said. 

According to NRK data for 9-16 May 2024, 89 articles published during that time featured summary boxes. Out of 1.2 million total pageviews of those articles, there were 359,000 clicks to expand the summary box, a 19% rate. 

Looking at the list of articles with the most summary box clicks, Sætre notes that “users normally expand very complicated articles.” Offering a recent example, he described an article about the outcome of the agreement for a new city government for Oslo, reached in October 2023. While the main body of the article covered the agreement itself, the summary box listed the consequences of the deal for Oslo residents in terms of changes to school policies, toll fees, taxes and other key day-to-day impacts. In this story, the summary box expansion rate rose to almost 60%, Sætre said.

Readers who clicked to expand the summary box tended to spend more time on the page than those who didn’t, according to the same data for 9-16 May. The median time spent on an article with a summary box by users who expanded the box on 9-16 May was 49 seconds, almost twice the median time spent by users who did not (25 seconds). 

“We have to do more research on this. But the little we have suggests that when [users] read the summary, then they understand this is an article that [they are] interested in so they choose to read more,” said Sætre. 

In terms of age, readers younger than 50 were more likely than not to expand the summary box on 9-16 May. Readers aged 50 and above were more likely not to expand it. Although this seems to line up with NRK’s aim of appealing to younger audience members, these figures only represent users logged into the website, not everyone who visits the article pages and only cover a very short time. 

5. How the tool can be improved

Aside from integrating the text assistant into their CMS, Sætre and Blekeli were satisfied with how the tool works. However, they’re looking ahead to explore other similar initiatives. Sætre and Blekeli are part of a newly established team of developers, reporters and editors working together to develop new AI-powered tools. “We're in the early stages and exploring the different possibilities, and looking for ways to automate some of the repetitive tasks that take quite some time, but we shouldn't do manually in the long run,” Blekeli said.

They’re testing a tool to help with the live transcription of press conferences and court cases, which would allow for live reporting, a translation tool to translate their content into Sámi, the language of this Indigenous minority, and looking for ways they can use generative AI systems to analyse troves of data for patterns to aid investigative work.

6. How this can help other newsrooms 

The team shared the project, including the source code, with the Norwegian Institute of Journalism, which can use it for a version of the tool. NRK’s team regularly collaborates with the Norwegian Institute, a nonprofit foundation, as well as with their counterparts at media company Schibsted, the largest private media group in the country. “We will compete on the best stories, but not on the tools necessarily,” said Sætre.