New benchmarks from Vals AI and NextLadder Ventures, advised by the Financial Health Network, tested 28 AI models across 256 simulated financial counseling scenarios involving eviction, medical debt and other challenges facing low-income families.
SAN FRANCISCO, Nov. 13, 2026 /PRNewswire/ -- Today, Vals AI and NextLadder Ventures released a new benchmark analysis examining general-purpose AI models' performance in providing financial advice in high-stakes scenarios often faced by low-income Americans, compared against best practices identified by financial counseling experts. The evaluation tested 28 models across 256 simulated financial counseling scenarios developed with practicing financial counselors.
The benchmarks are part of a broader effort spearheaded by NextLadder Ventures to accelerate the development of AI tools tailored to the unique needs and challenges facing low-income Americans navigating fork-in-the-road moments with disproportionate impact on their economic mobility.
"Every day, millions of Americans are struggling to make ends meet on tight budgets without much guidance or support," said Ryan Rippel, CEO of NextLadder Ventures. "Effective financial guidance that helps families make sense of their unique circumstances with reliable, tailored recommendations can make a world of difference. But too little of that guidance is available to low-income families, and the counselors who do work with these families are wildly over-stretched. AI-powered financial health tools have monumental potential to make a real dollars-in-pockets difference for low-income families, but far too many gaps remain in the general-purpose models that most people turn to today. We should not accept that the true measure of success for leading AI models means shattering the frontiers of mathematics, but not whether they can do the kitchen table math that faces tens of millions of families. It's our hope that these benchmarks will provide unbiased and actionable insights on how to make these tools better equipped to help low-income Americans move up the economic ladder."
"I'm optimistic that AI can make useful financial guidance available to more people," said Rayan Krishnan, Vals AI's cofounder and CEO. "Our evaluation shows there is still work to do. Models sometimes gave advice without understanding someone's circumstances, missed important deadlines or failed to connect them with available help. Independent evaluation makes those gaps visible and gives developers a way to measure whether their systems are becoming more reliable."
Vals AI worked with practicing financial counselors to develop 256 test scenarios drawn from the questions people bring to financial counselors. Twenty-eight AI models were tested in the role of financial counselor between September 14 and October 12, 2026. A second AI played the client and disclosed key facts, including income, debts, deadlines, family size, only when the counselor asked about them. Each of the 7,167 completed conversations (one per model per scenario) was scored against a checklist developed by financial counseling practitioners. An AI model did the grading; on practitioner-graded checklist items, it agreed 85% of the time.
The benchmark tested models from 11 developers: OpenAI's GPT-6.1 Sol, GPT-6 Sol, GPT-6 Astra, GPT-6 Luna, GPT-5.6 Sol, GPT-5.6 Luna and GPT-5.6 Terra; Google's Gemini 4 Argon, Gemini 3.1 Pro Preview, Gemini 3.8 Flash, Gemini 3.6 Flash, Gemini 3.5 Flash and Gemini 3.5 Flash-Lite; Meta's Muse Spark 1.2, Muse Spark 1.3, Muse Spark 1.3 Max, Wonder and Wonder Full; Anthropic's Claude Opus 5.5, Claude Sonnet 5.5 and Claude Fable 5.1; xAI's Grok 4.7; DeepSeek's V4.1 Flash; Alibaba's Qwen 3.8 Max; Moonshot AI's Kimi K3; Zhipu AI's GLM 5.3; Xiaomi's MiMo V2.6 Pro; and MiniMax's MiniMax-M3. Each model was tested directly at its standard settings, outside the consumer apps that run on it. The results measure model performance in simulated conversations, rather than the complete experience offered by consumer apps.
Full results are posted at vals.ai/benchmarks/financial-health. Key takeaways include:
- Models gave advice before gathering essential facts. Every model tested performed better at giving advice than at gathering information about the client's circumstances. Even the models that gathered the most information left about a fifth of the client's key facts, including their household size, income, or debts, undiscussed. In about one conversation in ten, the advice relied on an incorrect assumption about the client's circumstances. Such assumptions could cause a model to overlook available assistance or recommend steps that do not fit the household's needs. For example, an AI that assumes someone is a single adult without confirming if they have children would not refer them to thousands of dollars they may be eligible for via programs like the Child Tax Credit and Earned Income Tax Credit.
- Models sometimes substituted general rules for advice tailored to the client. For instance, when faced with severe credit card interest payments, AIs still recommended against making early withdrawals from retirement accounts to pay off the balance even when the interest payments significantly exceeded penalties. The stronger models in the test worked from the client's actual income, debts, and bills; weaker ones fell back on general rules that did not account for the client's circumstances.
- The gaps grew when the stakes were highest. In a crisis such as an eviction or a bill in collections, free tools regularly left out key details that more capable models included. Free Gemini (Gemini 3.5 Flash-Lite) left out at least one point most top chatbots made in about 6 in 10 crisis conversations. Free ChatGPT (GPT-6 Luna) did in about 5 in 10, and the top-ranked chatbot (Claude Opus 5.5) in about 1 in 10. High-impact gaps included failing to warn that forgiven debt can be taxed as income, and failing to warn about impending deadlines to respond in eviction cases.
- Free models were less likely to point people to accessible help. Where a counselor would have referred someone to a hospital charity-care program, a nonprofit credit counselor or legal aid, the 12 top-scoring models left it out up to 25% of the time, against about 30% for free ChatGPT (GPT-6 Luna) and 40% for free Gemini (Gemini 3.5 Flash-Lite), giving incomplete information and leaving clients to navigate their next steps on their own.
Vals AI and NextLadder Ventures are collaborating on an additional brief to help frontline practitioners understand the gaps that remain to be closed with these tools.
Examples from simulated financial counseling conversations
- "Denise", on disability with a $9,000 medical bill in collections, was told by nearly every chatbot to ask the hospital to forgive the bill. Free Gemini didn't mention it and said she could throw the collection letters away, which risks missing a court summons.
- "Maria", a widow paying her late husband's credit cards, was told by every chatbot that his debts generally aren't hers. Only 16 of 28 added that her Social Security is generally protected.
- "Tom", weighing a loan against his house to pay off $30,000 in credit cards, was warned by every chatbot about the risk. About half suggested free nonprofit credit counseling. Neither free ChatGPT (GPT-6 Luna) nor free Gemini (Gemini 3.5 Flash-Lite) did.
- "Pat", a recent retiree three payments behind on an FHA mortgage, was pointed by every chatbot to a free HUD-approved housing counselor. Only 17 of 28 warned about foreclosure-rescue companies that charge upfront fees. Neither free ChatGPT (GPT-6 Luna) nor free Gemini (Gemini 3.5 Flash-Lite) did.
About NextLadder Ventures: NextLadder Ventures is a time-bound venture with one goal: empower millions of Americans to reach their potential by 2040. Backed by over $1 billion in capital, the organization invests in breakthrough technologies that remove barriers to economic success and put people in control of their futures. NextLadder Ventures is trailblazing a new market for tech-enabled Navigation Technology tools that help people access the resources they need to navigate pivotal moments — offering flexible, risk-tolerant capital to entrepreneurs building these transformative tools today, while creating a pipeline of tech, talent, and capital for the long run. To learn more, visit nextladder.com.
About Vals AI: Vals AI is an independent, third-party AI benchmarking and evaluation company, and the standard used by every frontier AI lab and cited in their model releases. Vals designs and operates rigorous benchmarks that measure how AI models perform on real-world professional tasks across domains including cyber, military, legal, finance, healthcare, coding, public benefits, and youth safety. Its work has been covered by outlets including the New York Times and Washington Post. Vals is a Series A company backed by a16z and headquartered in San Francisco.
View original content:https://www.prnewswire.com/news-releases/leading-ai-models-still-missing-critical-details-when-giving-financial-advice-new-research-finds-302938452.html
SOURCE NextLadder Ventures
