Stanford Tested ChatGPT, Gemini and Claude as Research Tools. Here's When AI Wins

Stanford University researchers put ChatGPT, Gemini, and Claude through multiple-choice questions about recent events, and the results now anchor a growing guide on when AI should replace the search box. The team was specifically testing AI's effectiveness as a news source, because more and more people use it that way. But the strengths and weaknesses they uncovered, according to Minda Zetlin, author of "Career Self-Care," apply to almost any type of query.

The Stanford findings reached a wider audience through Wired, where tech journalist David Nield combined the study with other recent research into a brief guide on when to search with AI and when not to. Zetlin, writing an expert opinion column, merged Nield's advice, the Stanford research, and her own long experience with online research into a practical decision framework.

When should you use AI instead of search?

Zetlin's first rule: use AI when you want a general sense of something. She points to her own question about why so many people consider sourdough bread healthier than traditional yeast bread. AI handled it well, explaining that sourdough's long fermentation process makes it easier to digest, causes less of a glucose spike, and allows better absorption of iron and other minerals. She also credits AI with useful general how-to information, including gardening and cooking advice.

The second case is information that's widely known. Nield's example in Wired is the height of the Empire State Building. Because that figure is accurately reported in many different places, the odds are high that AI will return the correct answer. Widely replicated facts give models plenty of reliable training material.

The third case is complexity that needs plain English. Zetlin recounts a friend's prostate cancer diagnosis, which came with a flood of information about test results and antigen levels. "It was all a little overwhelming," she writes. The friend entered some of the medical information into Google Gemini, which produced a straightforward description of what it all meant. He had never used AI before, but he was an instant convert.

The fourth case may matter most for business readers: questions with multiple parameters. Nield's example is a family of five looking for dinner in San Francisco for under $200. Zetlin extends the logic to lead generation. She knows professional speakers who ask AI to list the associations in their area that cover their topic and are holding events between six months and a year out. Search engines, she notes, can't handle several elements at once in this way.

What does the Stanford test actually measure?

The Stanford study deliberately focused on questions about recent events, a category where AI tools face their hardest test. The researchers chose that framing because AI is increasingly serving as a de facto news source for ordinary users. Zetlin cautions that the weaknesses the study exposed in news-related queries are worth weighing before you trust an AI answer on anything fast-moving or contested.

The practical takeaway is a split decision. For stable, widely documented facts, multi-parameter requests, and translating dense material into plain language, AI tools outperform conventional search. For recent and rapidly changing information, the Stanford results suggest users should still reach for traditional sources — or at least verify what the model tells them.

That division of labor is likely to sharpen as more professionals fold AI into everyday research tasks like the speaker lead-generation queries Zetlin describes.