JPMorgan Chase published research last week showing that custom-built AI agents outperformed the standard 60-40 stock/bond portfolio allocation in every backtest over the past two decades. In the best run, the AI agents beat the benchmark by 0.7 percentage points per year with lower volatility. They also outperformed JPMorgan’s own rules-based market regime model, which categorizes conditions as Goldilocks, reflation, stagflation, or risk-off before allocating accordingly.

Then the same analysts who ran the tests published a warning: do not trust these results.

“We strongly caution against uncritically accepting what amounts to in-sample, overly confident answers of AI,” wrote the research team led by strategist Thomas Salopek, according to American Banker. “Agentic AI needs to be grounded in a well thought-out asset allocation process, rather than naively assuming the agent can be the source of the domain knowledge.”

The specific concern is data leakage. JPMorgan’s backtests used date-anonymized prompts and lagged data to prevent the AI from simply looking up what happened next. But the underlying LLMs were trained on text published after the test periods. “The LLM models are still trained on data after the cut-off point and may implicitly recall the outcome of recognizable historical episodes (e.g., 2008, COVID, etc.),” the report noted. An AI agent asked to allocate capital during a period it can recognize from its training data is not forecasting. It is remembering.

The Trust Deficit

The timing of the JPMorgan report matters because it arrived alongside survey data showing investors don’t want what the bank is testing. PwC surveyed more than 1,000 respondents about financial decision-making during market volatility, according to the same American Banker report. Only 24% said they would rely on AI-powered tools or assistants. That trailed online research and financial news (50%) and human financial advisors (48%) by wide margins.

This is not a niche audience skepticism problem. PwC’s finding reflects a structural gap: financial services firms are building AI agent capabilities faster than their customers are willing to adopt them. Most firms have deployed AI for research summaries, meeting notes, and back-office automation, according to PwC. Very few have given agents authority over investment decisions. JPMorgan’s backtesting is a step toward that boundary. The investor survey suggests the boundary is further away than the technology implies.

Bryan Byrer, founder of Millennial Financial Planning in Indianapolis, told American Banker he has yet to see a client use AI to second-guess his financial advice. “Intelligence is different from emotions, and people do emotional things with money but not always intelligent things with money,” Byrer said. “That’s an understanding that we won’t ever get, or at least for a very long time, from AI.”

Backtesting Theater and the Agent Deployment Gap

JPMorgan’s report is a case study in a pattern that keeps repeating across agent deployments. The lab results look good. The caveats are severe enough that the researchers themselves flag them. And the market is not asking for the capability being tested.

The 0.7 percentage point annual outperformance is real in the backtest but contextually thin. Over a 20-year horizon, that margin compounds meaningfully. But it assumes the AI can replicate that edge forward, in live markets, without data leakage, under conditions it has never trained on. JPMorgan’s own researchers flagged a second risk beyond leakage: if AI-guided allocations become widespread, the resulting herd behavior could distort the very market conditions the agents are trained to exploit. The alpha disappears the moment everyone chases it.

This is the same tension visible across the financial agent space. Robinhood opened autonomous crypto trading to AI agents in July after 70,000 users signed up for its equity and options agentic trading beta within two months. Kraken rebuilt its app around AI agents that monitor markets and provide portfolio guidance based on risk preferences. Visa integrated with ChatGPT for autonomous AI agent purchases. The infrastructure for financial AI agents is being built at scale. The evidence that they work in production, with real capital, in forward-looking markets, remains close to zero.

JPMorgan deserves credit for publishing the caveats alongside the headline results. Most agent vendors don’t. But the pattern of “impressive backtest plus disqualifying caveat” is becoming the default output of financial AI research. The question for builders deploying these systems is not whether agents can outperform benchmarks in controlled conditions. It is whether the controlled conditions have any relationship to the uncontrolled ones.

The PwC survey suggests that investors, at least, already know the answer.