By OpenAI's own testing,tradeshow sex video its newest reasoning models, o3 and o4-mini, hallucinate significantly higher than o1.
First reported by TechCrunch, OpenAI's system card detailed the PersonQA evaluation results, designed to test for hallucinations. From the results of this evaluation, o3's hallucination rate is 33 percent, and o4-mini's hallucination rate is 48 percent — almost half of the time. By comparison, o1's hallucination rate is 16 percent, meaning o3 hallucinated about twice as often.
SEE ALSO: All the AI news of the week: ChatGPT debuts o3 and o4-mini, Gemini talks to dolphinsThe system card noted how o3 "tends to make more claims overall, leading to more accurate claims as well as more inaccurate/hallucinated claims." But OpenAI doesn't know the underlying cause, simply saying, "More research is needed to understand the cause of this result."
OpenAI's reasoning models are billed as more accurate than its non-reasoning models like GPT-4o and GPT-4.5 because they use more computation to "spend more time thinking before they respond," as described in the o1 announcement. Rather than largely relying on stochastic methods to provide an answer, the o-series models are trained to "refine their thinking process, try different strategies, and recognize their mistakes."
However, the system card for GPT-4.5, which was released in February, shows a 19 percent hallucination rate on the PersonQA evaluation. The same card also compares it to GPT-4o, which had a 30 percent hallucination rate.
In a statement to Mashable, an OpenAI spokesperson said, “Addressing hallucinations across all our models is an ongoing area of research, and we’re continually working to improve their accuracy and reliability.”
Evaluation benchmarks are tricky. They can be subjective, especially if developed in-house, and research has found flaws in their datasets and even how they evaluate models.
Plus, some rely on different benchmarks and methods to test accuracy and hallucinations. HuggingFace's hallucination benchmark evaluates models on the "occurrence of hallucinations in generated summaries" from around 1,000 public documents and found much lower hallucination rates across the board for major models on the market than OpenAI's evaluations. GPT-4o scored 1.5 percent, GPT-4.5 preview 1.2 percent, and o3-mini-high with reasoning scored 0.8 percent. It's worth noting o3 and o4-mini weren't included in the current leaderboard.
That's all to say; even industry standard benchmarks make it difficult to assess hallucination rates.
Then there's the added complexity that models tend to be more accurate when tapping into web search to source their answers. But in order to use ChatGPT search, OpenAI shares data with third-party search providers, and Enterprise customers using OpenAI models internally might not be willing to expose their prompts to that.
Regardless, if OpenAI is saying their brand-new o3 and o4-mini models hallucinate higher than their non-reasoning models, that might be a problem for its users.
UPDATE: Apr. 21, 2025, 1:16 p.m. EDT This story has been updated with a statement from OpenAI.
Hiroshima Kenjinkai Launches Disaster Relief Drive
APAFT Hosts Theater Party for Scholarship Winners
PancakeSwap Unveils Gamified Quest Platform
Istanbul Blockchain Week Welcomes Arkham as Title Sponsor
Results for APA Candidates in County, Municipal Races
Recipients of Kimi Sugiyama Human Service Award Named
Nisei Post’s In
Nanka Wakayama Kenjinkai Offers Scholarships
CAPAC Leaders Celebrate APA Heritage Month
L.A. Asian Pacific Film Fest to Celebrate 33 Years (Part 1)
Mineta Joins with Korematsu Center in Census Citizenship Case
In the Family Business
Asian Americans Advancing Justice Celebrates Allen v. Milligan Decision
55th Anniversary Screening of ‘Yojimbo’ at Laemmle Theaters
Higa, Perron Receive Fall Decorations
Merge Madrid: Connecting Europe & Latin America via WEB3
Greater LA JACL to Discuss Upcoming Events
BeInCrypto Partners Restaking and Emerging Markets Online Summit
Celebrating Asian American Historic and Cultural Contributions in L.A. at Community Symposium
Time to Start Talking and Stop the Hate
Commanders vs. Steelers 2024 livestream: How to watch NFL onlineBest 50th birthday gifts: Celebrate half a century with the perfect presentNYT mini crossword answers for November 9Twitter's blue couch saga is dividing the timelineChiefs vs. Broncos 2024 livestream: How to watch NFL onlineJaguars vs. Vikings 2024 livestream: How to watch NFL onlineBest Fitbit deal: Save $80 on the Fitbit Versa 4The most badass Latin phrasesBest free book deal: Target Circle members can buy two books, get one freeTwitters adds exclamation point to its 'what's happening' prompt Romney dining with Trump is the ultimate real Young Chinese Aussies say they face highest levels of discrimination Technology improvements aid Instagram's drone photographers Justin Bieber covers Taylor Swift again and could it spell T Mysterious bread sponge is desperately trying to be 'the dress' Australian startups need more diversity to thrive, not closed borders Hire someone to make your wedding hashtag because #itshardtobeclever Cute animals, musical numbers and tearjerkers abound in 2016's top viral ads Puppy finds adorable way to clean his pee puddle Meet your new hero, an ascot
0.1773s , 14317.5078125 kb
Copyright © 2025 Powered by 【tradeshow sex video】Enter to watch online.OpenAI's o3 and o4,Global Perspective Monitoring
