Alice Weidel, co-leader of the Alternative for Germany party (AfD), gestures as SpaceX CEO Elon Musk appears on screen during a central election campaign event of the AfD in Halle (Saale), Germany, January 25, 2025. REUTERS/Karina Hessland
ChatGPT was the first AI chatbot to capture audiences’ imagination and it may still be the most used. But it’s been more than two years since ChatGPT’s public release in November 2022 and competitors such as Grok, Perplexity.ai and Google Gemini have entered the scene.
These chatbots’ relationship with the news industry has been complicated. At the time of this writing, OpenAI, ChatGPT’s developer, had struck content licensing deals with at least three dozen news publishers. But it was still fighting a lawsuit brought about by the New York Times in a case that will have implications for copyright legislation in the United States.
However, OpenAI’s core product is not explicitly geared toward news. ChatGPT’s main interface prompts users to create and analyse images, code, and even ask for advice, but doesn’t encourage people to ask about the latest news. Google Gemini has taken a similar approach and is especially cautious when asked about sensitive topics such as elections. Asked about those topics, Gemini rarely produces a response.
Using chatbots as a news source is still relatively rare. According to this report we published in May 2024 based on survey data from six countries, only 5% of those who said they’d used an AI chatbot had done so to get the latest news.
However, two chatbots appear to be gearing their products toward bringing users news-related output: Perplexity.ai and Grok. The data we collected in early 2024 suggests these tools were much less popular than ChatGPT. But this space is evolving fast and both chatbots have certainly gained some traction in the last year. Both Grok and Perplexity.ai provide interesting examples of how AI chatbots may begin to market themselves more widely as potential sources of news. I explored how both tools do so right now and experimented with asking them about an important news topic: the upcoming federal elections in Germany, which are taking place on Sunday 23 February 2025.
Grok was developed by xAI, an artificial intelligence company Musk founded, and is closely intertwined with X, the platform he bought in 2022. The chatbot has access to X posts and it sources its answers both from those posts and from broader internet access. Recently, Grok question buttons have been appearing under some X posts, prompting users to ask the chatbot about the topics mentioned in the post. Grok was initially only available to paid X subscribers, but since December 2024 has been accessible for all users with usage limits.
Grok may be best known for its image-generation capacity, which is significantly less restricted than that of other AI image-generation tools. Notably, Grok allows users to generate images depicting real people, something several other tools refuse to do.
In April 2024, X’s ‘Explore’ page started featuring summaries of trending topics, often news stories, generated by Grok and based on popular X posts.
Grok explicitly prompts its users to ask about news, with its landing page advertising its use to “get up-to-date info on the latest news and trending topics, exclusively from X.” Musk has repeatedly stated that X is the top source of news on Earth, something that is not reflected in the data we collect about how audiences worldwide access news. The Digital News Report 2024 found that X is used for news by 10% of our global sample each week, compared to 37% who use Facebook, 31% for YouTube, 21% for WhatsApp and 18% for Instagram.
Perplexity.ai is a conversational search engine that works using external large language models, such as OpenAI’s GPT-3.5, and always presents several sources for its responses. As well as its chat interface, Perplexity.ai has a Discover tab with a feed of articles split into topics such as tech and science, finance, and arts and culture, strongly resembling the home page of a news site. The username of the account responsible for curating the article appears like a byline under a header image. It is not a clickable link and there’s no further information available about these accounts. The bulk of the article is split into blocks of text under subheadings and images. The sources for the information are listed at the top and under relevant subsections.
Some news organisations see Perplexity.ai’s use of news websites’ content in these AI-generated summaries as problematic. Rupert Murdoch’s media group is suing the company for copyright infringement, alleging Perplexity.ai engaged in “massive illegal copying” and using their content without permission.
More than a year ago, I experimented with asking ChatGPT about the news soon after it was connected to the Internet in September 2023. As Grok and Perplexity.ai leant into news, I wanted to test their output in the context of an important news event. So I asked them both a range of questions about the upcoming German election.
My goal was to understand how each chatbot could work as a tool to access the news. I picked this election because it’s a prominent news topic with a longer shelf life than many breaking news items.
It’s important to stress that this is not a piece of academic research but just an exploration of a possible use case for this technology at this particular time. The responses' accuracy could be different in the future and in the context of elections in other countries. I accessed Grok2 from Oxford through the Reuters Institute’s Premium X account and used the Auto setting on a free Perplexity.ai account.
I decided to ask the chatbots questions regarding basic election information, some about more complex policy issues and some specifically about Elon Musk’s interventions in the run-up to the vote. The questions on basic election information were geared towards checking for accuracy, while with the latter questions, I aimed to see if the chatbots could produce nuanced discussions of key issues in a similar way to journalists reporting on them. One of my goals was to see if the responses provided by a tool on a platform Musk owns differed from those provided by Perplexity.ai. I asked the questions in English and German to see if there were significant differences in how the tools responded.
To evaluate the accuracy of the responses, I used official government sources for basic election information (mostly the Federal Agency for Civic Education) and reporting and polling by established and reliable polling companies and news organisations for the remaining questions, particularly public broadcasters DW and ARD, and international poller YouGov. Here are the questions I asked both in English and German.
The responses I received from both chatbots seemed accurate in almost all cases. However, I did notice some factual errors in the answers to the basic election information questions.
When answering the question of who is eligible to vote, Grok and Perplexity.ai's English-language responses included an inaccuracy regarding the right to vote for people ‘legally declared incapacitated’.
Grok:
Perplexity.ai:
The disqualifications cited above were in place until a 2019 ruling by the German Constitutional Court, which found that ‘the exclusion from voting rights of persons placed under full guardianship and of criminal offenders confined in a psychiatric hospital [was] unconstitutional.’
The source referenced in the Perplexity.ai response is the English-language Wikipedia page ‘Electoral system of Germany’, an article flagged as needing to be updated in a notification at the top of the page. Under the ‘Right to vote’ section of the article is the same incorrect information about Germans excluded from voting, referencing the law found to be unconstitutional in the 2019 ruling.
Grok didn’t provide any source for this question or for the question on how the 5% threshold works, perhaps suggesting it had not surfed the web in search of a response.
Grok’s response also included a disclaimer stressing these were “the general criteria” and that “there might be specific circumstances or changes in legislation that could affect voter eligibility.” The chatbot also advised people to check official sources for the most current information.
The German-language responses to the same question by both Grok and Perplexity.ai were entirely accurate. Grok’s response was relatively brief and didn’t mention the exceptions inaccurately covered in the English responses. The response from Perplexity.ai correctly referenced the exceptions currently allowed for under the law.
Similar inaccuracies were also found in both the Grok and Perplexity.ai responses to the third question concerning the 5% threshold, the requirement under German election law for parties to obtain a minimum of 5% of the party votes to be represented in parliament.
The information given by the response in German accurately describes the way the system has worked until now but misses out on important recent changes in the rules limiting the number of parliamentarians in the upcoming election, which will affect the processes described.
It also included a vague and inaccurate statement regarding exceptions to the 5% rule: it references an exception for former East German states where parties win at least three direct mandates. In fact, this exception applies equally to the whole country.
Only in the 1990 federal election was the 5% rule applied separately for West and East German states. The phrasing in the Grok response makes it sound as if a similar policy is still in place when it is not. These inaccuracies were not found in Perplexity.ai.
These factual inaccuracies are subtle, and the complicated nature of the subject matter makes them difficult to spot and clarify. While most of the information provided was accurate, precise information when it comes to voting is crucial, and may mislead citizens at a very sensitive time.
The overwhelming majority of sources cited both by Grok and Perplexity.ai were either official websites or news organisations.
Institutional sources were cited especially often when it came to the questions on basic election information. On the two questions asking about voters’ opinions (those on Germany’s most pressing problems and on what Germans think about Musk’s interventions in their country’s politics) both chatbots repeatedly cited polling by ARD-Deutschlandtrend and YouGov.
Mainstream, non-partisan news sources in both English and German were referenced in almost every response by either chatbot. The most mentioned included Reuters, Tagesschau, Politico Europe, and public broadcasters DW, ZDF and BBC.
On some occasions, niche media outlets were also included. In Grok’s English-language response to the question on immigration policy platforms, four out of six X posts and three of the 15 web pages linked belonged to InfoMigrants, a news service catering to migrants travelling to Europe. Wikipedia, a website Musk has criticised repeatedly, was also cited several times.
Some of the sources listed by both chatbots were unusual choices for the questions asked. For example, this page detailing how to fill out a ballot paper from the website for the Australian Capital Territory was cited by Grok on the question of filling out the ballot paper in the Bundestag election. However, it doesn’t seem to have affected the accuracy of the response.
Other sources cited were less outlandish, but probably wouldn’t have been the ones informing a journalist writing an article on the topic. On the question of Germany’s most pressing problems according to voters, Grok referenced data on this topic from 2021, as well as the Wikipedia page for the 2021 German federal election. While this might be useful context, its inclusion might also be confusing for users.
Other sources mentioned by both chatbots just stood out as unexpected, such as the inclusion of web pages only tangentially related to the topic or news organisations specialising in other subject matters or regions. For the German-language answer to a question on the price of energy, Perplexity.ai referenced campaign groups supporting an energy transition towards renewables, a countryside-focused news website, a report commissioned by Greenpeace and the website of an energy provider. While these are all related to the topic, they don’t seem like the most useful sources to address the question.
In response to a question on whether Elon Musk is promoting the AfD, the English-language response by Perplexity.ai includes seven links to several mainstream international news organisations as well as a Facebook link to a video posted by digital video publisher Brut India. The content of the video was relevant to the question, but the reason why this particular source was chosen is unclear.
Other strange source choices by Perplexity.ai are seen in responses to ‘Did Elon Musk call on Germany to move on from its “culture of remembrance”?’. In English, it references articles on this topic by UniLad and the South China Morning Post. In its German-language response to the same question, it cites the Ukrainian newswire UNN.
On occasion, both Grok and Perplexity.ai cited one or two political parties’ websites or agendas in a response that otherwise appeared to be fairly balanced and not favouring a particular party or point of view.
Moreover, Grok cites X posts as sources as well as web pages. It is unclear how and why these ‘relevant posts’ are chosen. Some are very popular and others have very little engagement. Some are by users sporting blue ticks, usually indicative of a paid subscription, and others aren’t. They also don’t appear to favour any particular point of view or have any political leaning. Some X posts cited just appeared to be stating an individual user’s opinion. In answer to one question, asked in German, some of the X posts cited were in Spanish.
The best example I found of an X post being a relevant source for one of the questions I asked is in the English-language response to ‘Is Elon Musk promoting the AfD?’. The single X post cited was Musk’s tweeting in favour of the AfD.
In general, the responses both by Perplexity.ai and Grok appeared to be balanced and not slanted in a particular direction. This is despite an overarching trend, led by Elon Musk, towards right-wing content on X, including boosting his own posts.
Some of the X posts cited as sources for Grok’s responses did reflect right-wing points of view about some of the topics I asked about, including immigration and energy sources, but others did not. Several were very critical of Musk personally, particularly in the responses to the questions asking about his involvement in the run-up to the election.
Asked about controversial topics such as whether Musk called Germany to move on from its “culture of remembrance” – a reference to Nazi crimes and the Holocaust – and Germans’ opinions about his interventions, Grok’s response appeared not to be biased in his favour.
Grok:
Perplexity.ai:
Assistance with German translation by Felix Simon.
In every email we send you'll find original reporting, evidence-based insights, online seminars and readings curated from 100s of sources - all in 5 minutes.
In every email we send you'll find original reporting, evidence-based insights, online seminars and readings curated from 100s of sources - all in 5 minutes.
