Key Takeaways
- Leading AI chatbots are surprisingly bad at giving financial advice, often providing incorrect information.
- Researchers tested ChatGPT-4o, DeepSeek-V2, Grok 3 Beta, and Gemini 2 with 12 finance questions.
- None of the chatbots scored higher than 5 out of 12 points for accuracy and analysis.
- Chatbots made basic math errors and lacked critical thinking, even on simple financial concepts.
- Despite errors, the AI presented information confidently, which could mislead users seeking reliable advice.
Despite optimistic predictions about artificial intelligence, the world’s most advanced chatbots still struggle significantly with financial advice.
Researchers Gary Smith, Valentina Liberman, and Isaac Warshaw put four popular AI models to the test: OpenAI’s ChatGPT-4o, DeepSeek-V2, Elon Musk’s Grok 3 Beta, and Google’s Gemini 2. They posed 12 common finance questions to gauge their abilities.
The results were underwhelming. In a study highlighted by the Walter Bradley Center for Natural and Artificial Intelligence, the experts found each chatbot was “consistently verbose but often incorrect.”
This isn’t the first time such issues have surfaced. A similar study last year involving earlier AI versions yielded nearly identical findings: grammatically sound and authoritative-sounding responses filled with calculation and thinking errors.
Using a simple scoring system (0 for wrong, 0.5 for correct analysis with math errors, 1 for fully correct), none of the AIs excelled. ChatGPT-4o scored the best with just 5 out of 12 points. DeepSeek received 4 points, Grok got 3, and Gemini trailed with a mere 1.5 points.
Some mistakes were glaring. For instance, when asked to sum monthly expenses of $3,700 rent and $200 utilities for a rental property, Grok incorrectly calculated the total as $4,900.
According to the report detailed by Futurism, the chatbots also produced strange typos and failed to offer insightful analysis for relatively basic financial questions.
Even when the AIs provided decent answers, like explaining how Roth IRAs work, the information seemed pulled directly from online sources rather than demonstrating genuine understanding.
A significant concern is the chatbots’ delivery. The researchers noted they all project a “reassuring illusion of human-like intelligence” with a friendly, conversational style, often using exclamation points. This could easily be mistaken for confidence and accuracy by an average user.
The study concludes with a warning: the real danger isn’t that AI is smarter than humans, but rather that we might mistakenly believe it is, leading us to trust it with decisions it’s not equipped to handle.
