Trusting AI with your money is subject to market risks — SkimNews
Get the Finance newsletter
Daily finance — markets, central banks, M&A, the prints that move money. Free.
- Saturn, a UK fintech firm, commissioned a July 2026 study in which an experienced paraplanner tested 18 widely used AI models against 121 real-world money questions (asked five times each, producing over 10,000 responses), judging each answer against standards a qualified adviser would deem essential.
- Overall accuracy across the 17 current models tested was just 43% — wrong or incomplete more often than right — and failure climbed to 88% on harder, multi-step questions, with even simple ones (definitions for first-time investors) scoring only 54%.
- Free chatbot advice was accurate only 37% of the time, compared with roughly 51% for paid models, undercutting the claim that AI widens access to reliable financial guidance.
- Hallucinated rules produced concrete damage in the study examples: one free model pushed debt prioritisation by interest rate without flagging that unpaid rent or missed council tax carry far more serious UK consequences, while a pension answer could have exposed an investor to an unauthorised-payment tax charge of up to £17,500 (roughly ₹22-23 lakh).
- The danger is fluency, not uncertainty — the study found models rarely flagged when a question was beyond their reliable scope, so confident, well-structured wrong answers give users little reason to verify elsewhere.
- India-specific data is missing — there is no equivalent Indian study, and India-focused rules (capital gains tax, NPS, EPF, mutual fund taxation) likely make up a smaller share of model training data than UK or US rules, potentially raising error risk for Indian investors.
- Sebi's own investor survey found 62% of investors said they were influenced by finfluencers offering no accountability or verified track record — a vulnerability a general-purpose AI chatbot could deepen without ever being labelled a finfluencer.
Why it matters: For millions of investors who turn to ChatGPT, Gemini or Claude for free money guidance, the 43% overall accuracy and 88% failure rate on complex queries means confident wrong answers on taxes, debt prioritisation or retirement withdrawals. With no Indian benchmark yet to gauge local risk, the only safe stance is to treat AI financial advice as a first draft until India-specific testing catches up.
Ask SkimNews
