Are AI chatbots like ChatGPT politically biased? We tested them.

President Donald Trump and other conservatives have accused artificial intelligence chatbots of being politically biased against them — and an executive order he signed that said they must be “neutral, nonpartisan tools” triggered fears from Democrats that AI could start tilting to the right.

So, are chatbots politically biased? The Washington Post tested the AI models behind OpenAI’s ChatGPT, Google’s Gemini and others using political questions designed by researchers to gauge how chatbots respond to hot-button political issues. The results suggest that chatbots have clear political leanings that can conflict with promises made by the companies behind them.

The model that powers ChatGPT answered nearly every question exclusively with left-leaning arguments and presented only right-leaning positions just once. Google’s Gemini mostly took a both-sides approach, offering both left and right positions in more than 90 percent of its answers.

And even AI models marketed as having conservative views, including Elon Musk’s Grok, offered by his company SpaceX, cited left-leaning arguments more often, on average. (The Post has a content partnership with OpenAI.)

The Post’s results come after several previous academic studies found that AI models powering chatbots tend to favor left-leaning positions.

Sean Westwood, director of the Polarization Research Lab at Dartmouth College, said understanding the positions that AI tools amplify is important because they are becoming increasingly influential as more people use them to understand the world or news events. “These AI tools are not presenting a truly neutral representation of really nuanced policy debates, on average,” he said.

The Post modeled its tests on research published last year by Westwood’s lab in collaboration with researchers at Stanford University, which developed more than two dozen political questions designed to reflect things people might ask a chatbot.

AI models were asked answer each of the questions in 30 words, without personalization settings turned on. A reporter reviewed the responses to score whether they included a left-leaning position, a right-leaning position or both. Political topics rarely break down neatly along partisan lines, but the questions covered a wide range of topics, and The Post checked that the AI models were consistent in their answers.

In response to this question about Citizens United, the 2010 Supreme Court case that loosened restrictions on corporate spending in elections, OpenAI’s model said the decision should be overturned. The answers from Google and Anthropic, which offers the Claude chatbot, presented opposing perspectives on the issue.

OpenAI’s model gave the most skewed answers overall, with 80 percent presenting only left-leaning arguments. It endorsed abolishing the electoral college in favor of picking the president by popular vote; raising taxes on the wealthy; and adopting single-payer health care.

Chinese company DeepSeek’s AI model was close behind and also leaned left in its answers. Both models argued against the death penalty, which a majority of Americans have consistently supported for decades, according to Gallup.

Google spokesperson Lauren Fine said that “Gemini is designed to provide balanced responses that don’t favor any political ideology.” The company was unable to reproduce one-sided responses that occurred in The Post’s tests, she said.

Anthropic spokesperson Michael Aciman said “We train Claude to treat different political viewpoints equally and test extensively for bias before every model launch.” The Post’s tests do not reflect how most people use the company’s products, he said, and Claude generally has more space to include context when discussing politics.

OpenAI, SpaceX, DeepSeek and Gab did not respond to a request for comment.

OpenAI CEO Sam Altman said of ChatGPT in 2023 that the company “will try to get the default version to be as neutral as possible,” but that the solution is giving users personalization because “neutral” means different things to different people.

Chatbots can pick up political perspectives in different ways. Most are trained on large collections of text scraped from the internet, but companies can choose what data to include. AI firms also hire workers to refine what their models say by scoring which responses are considered better, and companies write system instructions that guide their chatbots’ responses.

The decisions tech companies make as they build their AI tools can bake in biases that go beyond partisan politics, said Ceren Budak, a professor at the University of Michigan who has studied how social media and other technologies interact with political polarization. The data that shapes AI models, she said, tends to reflect the values of Western, educated, industrialized, rich and democratic people.

With AI tools like chatbots, tech firms are taking a more actively political role, she said, because the products can generate political speech directly, instead of just curating speech by humans as social media platforms do. Even people who don’t talk to chatbots about politics are increasingly exposed to AI-generated text in online content and via other channels, Budak said.

“It would be helpful for us to have some clarity on what are [companies’] current value systems so that when we are using them we know what we are using,” Budak said.

The original Dartmouth and Stanford study that tested older AI systems with political questions asked a nationally representative sample of 10,000 Americans whether the AI responses appeared to be politically slanted.

People preferred neutral answers, even over answers that matched their party preference, said Andrew Hall, a Stanford researcher on the study.

“People really like when the model puts in the effort to describe all of the different arguments that people have,” Hall said.

The Post’s testing suggests that most chatbots do not provide that. Google’s Gemini model was the exception, even giving “both sides” answers to whether the United States should use its military to conquer new territories for resources. No other model offered an argument for conquest.

Many scholars argue political neutrality is impossible. Even “neutral” or middle-ground positions are positions themselves, and these tend to benefit the stronger side. By that thinking, Google’s both-sides approach is itself political.

“Neutrality is only one of the values that we actually care about,” Budak said. She is more concerned with the potential for what AI tools say to be harmful, especially to populations that are already oppressed.

AI companies must contend with different categories of questions, Hall said. Some are objective and factual, like “What is the speed of light?,” which chatbots can answer simply.

“Most political questions don’t have that feature, where we know what’s true,” Hall said. “You have to take the facts, and then you have to add your values on top of them.”

He said he was surprised that all the leading chatbots did not respond more neutrally to political questions, as Gemini did. “I would have thought the other models had caught up,” he said.

Some companies design their AI products to promote specific political values. Grok, which Musk has touted as a “truth-seeking” and anti-“woke” AI chatbot, gave more right-leaning responses than any other in The Post’s testing, but more often it provided a wholly left-leaning position.

Gab, a right-wing social media site, offers an AI model called Arya that it says was “built with Christian values and conservative principles.” But in The Post’s testing, it responded with a left-leaning argument 12 times more often than a right-leaning argument.

Few Americans use AI to help them understand politics directly, but nearly half occasionally use AI for news, according to a survey in March by the Polarization Research Lab.

“Both Democrats and Republicans don’t trust AI to be neutral, and they’re keeping it at arm’s length from their votes,” said Westwood of Dartmouth. “It’s one of the few places in our modern political landscape where we can agree.”