EVIDENT AI SYMPOSIUM

Welcome

The State of AI in Financial Services

Evident co-founders Alexandra Mousavizadeh and Annabel Ayles opened this year’s Symposium by examining the latest AI trends within the financial service sector. Their central message: the industry has reached an inflection point, with leading banks turning four years of investment into more use cases, more sophisticated applications , and more visible financial impact.

Key Takeaways:

1. Industrialize what works. Leading banks identify successful AI applications and scale them across the business.
2. Optimize and reinvent in parallel. Leaders improve existing processes while rethinking how work gets done.
3. Measure value and control spend. Disciplined value measurement and automated cost controls distinguish the leaders.
4. Build reusable infrastructure. A common agent harness enables banks to apply controls and capabilities consistently across deployments.

These four themes frame the wider agenda for Evident’s fifth AI Symposium.

Spotlight

In conversation with… Stephanie Ferris, CEO & President of FIS

FIS CEO Stephanie Ferris argued that AI delivers value in regulated industries when organization's redesign workflows around trusted, accountable systems. In financial services, the model matters less than the controls, accountability, and operational design surrounding it.

Key Takeaways:

1. Agents expand financial crime capacity. FIS’s anti-money-laundering agent, built with Anthropic, reduces first-level scanning work by 90% and second-level review work by 98%, allowing teams to handle more cases without removing human oversight.
2. Accountability must be explicit. Agents need named owners, auditability and traceability. As Ferris put it: “If you can’t name the person responsible for the outcome, you can’t run that agent…”
3. Productivity must justify rising costs. As enterprise technology costs increase, FIS is shifting from broad access to targeted pilots with defined outcomes, including faster customer service and legacy modernization.

Spotlight

In conversation with… Lori Beer, Global Chief Information Officer at JPMorganChase 

JPMorganChase is scaling AI through shared platforms, business-owned initiatives, and controls designed for enterprise deployment. Global CIO Lori Beer emphasized that adoption cannot be driven centrally alone: business teams must own outcomes, with success measured against build costs, running costs and tangible value.

Key Takeaways:

1. Scale through shared platforms. LLM Suite has 200,000 active users and more than 1,000 production applications. In asset and wealth management, one idea moved from demo to launch in four months.
2. Engineering offers significant productivity gains. Coding assistance for 70,000 technologists is the bank’s largest agent use case and biggest token expense, delivering reported productivity gains of 20–30%. Routing routine tasks to cheaper models helps manage costs.
3. Controls make scale possible. Beer said non-deterministic agents require 15 controls, covering areas such as discoverability, guardrails, and evaluations. Alongside agent-specific identity and access management. JPMorganChase has standardized on a core agent-building platform to make those controls reusable.

Spotlight

In conversation with… Mojgan Lefebvre, Executive Vice President and Chief Technology & Operations Officer at Travelers 

Travelers’ Mojgan Lefebvre argued that safe AI deployment starts with proprietary data, domain expertise, and governance built in from Day One. The insurer’s approach combines specialized models with continuous monitoring, enabling narrowly scoped applications to scale nationally.

Key Takeaways:

1. Monitor actions, not just answers. Travelers’ Claim Assist voice agent uses human evaluators, LLM judges, and live observability – each with the ability to intervene, reroute, or shut it down. Lefebvre called continuous observability essential to scaling with confidence.
2. Specialized models can outperform on cost. TravelersLLM, fine-tuned on decades of claims and underwriting data, ran 5x faster and at 80% lower cost than frontier models across thousands of insurance tasks.
3. Start narrow and scale. Claim Assist expanded nationwide in six months, initially focusing on auto property damage. Of customers who opt in, 80–90% remain engaged through filing.

Spotlight

In conversation with… Drew Cukor, Co-Global Head, Vergence.ai and Tyler Jewell, President & CEO, Akka

Drew Cukor and Tyler Jewell argued that AI transformation is an organizational challenge, not simply a technical one. Without CEO ownership and a focus on business outcomes, companies risk accumulating disconnected experiments instead of changing how work gets done.

Key Takeaways:

1. AI transformation starts with the CEO. Cukor argued that AI cannot be delegated solely to a CTO, CIO, or CAIO. He went further, predicting that the Chief AI Officer (CAIO) role could have less than two years left.
2. Falling costs demand reusable controls. Jewell said the cost of intelligence is falling by roughly 90% annually, potentially driving greater demand. Organizations will need infrastructure that continuously tests new models and applies controls consistently across deployments.
3. Measure business outcomes, not use cases. Cukor’s guiding principle: “The unit of measurement is not the use case; it’s the business objective.”

Spotlight

In conversation with… Marco Argenti, Chief Information Officer at Goldman Sachs

Goldman Sachs CIO Marco Argenti argued that enterprise AI is moving from answering questions to deploying agents across the workforce. As employees learn to direct and supervise these systems, organizations must pair human judgment with disciplined cost management and security controls.

Key Takeaways:

1. Every employee becomes a supervisor. Goldman is moving beyond Q&A tools toward agents coordinated by orchestrators. Employees need to learn delegation and verification, including the principle that even AI-native hires must “trust and verify.”
2. Build capability, then move on. Forward-deployed engineers help teams navigate major technology shifts, but their goal should be to transfer expertise and leave. Argenti said a “forever FDE” signals a problem.
3. Contain agents rather than trust them. Because agents are stochastic, Goldman uses defense in depth and gates access to tools before payments or transactions. Safety must be enforced by the surrounding environment, not left to the agent itself.

Spotlight

In conversation with… Daniele Tonella, Chief Technology Officer, ING 

ING CTO Daniele Tonella argued that banks need a deterministic control framework around AI agents before deploying them in live production. ING combines fixed decision gates with probabilistic AI analysis, prioritizing trust and consistency over speed for its own sake.

Key Takeaways:

1. Combine deterministic controls with AI analysis. ING uses explicit “yes -or- no” gateways alongside probabilistic models. Tonella warned that skipping this framework risks fragmenting the technology landscape and undermining trust.
2. Keep experimentation outside production pipelines. ING supports vibe coding before the formal software development lifecycle, but engineers must own the pipeline and bring resulting applications under enterprise controls.
3. Tie token spend to business value. Tonella warned that tokens could account for around 30% of bank spending within five years, on top of existing costs. Coding productivity only counts when it translates into lower staffing costs or greater revenue-generating capacity.

ROUNDTABLE

How is AI usage evolving?
#efficiency, tokens, role transformation.

The panel argued that the era of maximizing AI access and token consumption is giving way to a more disciplined approach. Financial institutions must combine workforce training and experimentation with fit-for-purpose models, transparent cost measurement, and shared infrastructure.

Key Takeaways:

1. Build fluency before tightening controls. Liberty Mutual employees have created more than 30,000 assistants on Liberty GPT, while Scotia Intelligence reaches around 70,000 people. Both organizations pair broad access with training and structured support for business-led adoption.
2. Measure the full cost of AI. Banks are adopting gateways, caching, process-owner budgets, and license reclamation. EY’s Sameer Gupta said meaningful ROI calculations must include six or seven cost categories, including governance, change management, and model maintenance.
3. Collaborate on shared infrastructure. Scotiabank, Telus, and Sun Life jointly built ACP – a control plane for agent permissions, cost tracking, and kill switches. Each member owns and operates its own codebase, and the consortium is developing a token exchange for bulk purchasing and allocation.

SPOTLIGHT

In conversation with… Yann LeCun, Executive Chairman at AMI Labs

Yann LeCun argued that large language models remain limited in their ability to reason and plan in the physical world. The AI pioneer, whose Paris-based AMI Labs has raised approximately $1 billion, sees world models as a potential next paradigm – particularly for systems that must understand complex environments rather than simply generate language.

Key Takeaways:

1. Capability is not the same as intelligence. LeCun challenged the assumption that machines that outperform humans on difficult tasks are necessarily more intelligent. Despite progress in exams and coding, he noted: “We still don’t have domestic robots.”
2. World models offer an alternative to LLMs. LeCun argued that LLMs struggle with real-world planning and criticized the industry’s concentration on language models while other approaches advance.
3. Predict what matters, not every detail. AMI Labs’ JEPA architecture predicts in an abstract representation space, ignoring unpredictable details. LeCun linked this principle to finance, where data is “...mostly noise with a little bit of information in it.”

ROUNDTABLE

How do we harness agents?
#agentic orchestration, architecture, complexity.

The panel argued that agent autonomy should expand only as quickly as organizations can verify performance and control risk. The critical work lies in the infrastructure around agents: identity, data access, telemetry, testing and clear intervention mechanisms.

Key Takeaways:

1. Improve the harness, not just the model. Nubank’s Rohan Ramanath cited coding benchmarks in which changing the harness alone nearly doubled frontier-model performance. Nubank also simplified its APIs, so agents can complete tasks in one call rather than five — and test them against 16,000 simulated customers before launch.
2. Give agents identities and oversight. Allianz requires agents to share telemetry before entering production. Each receives its own identity and is managed throughout its lifecycle, with security teams able to shut it down. Cross-cloud governance remains a challenge.
3. Promote agents on measured performance. Nubank’s card-delivery agent was expanded to all card management after outperforming human quality benchmarks; AI now handles 60% of support tickets. BMO’s Kristin Milchanowski stressed the importance of first measuring how often human employees make mistakes.

SPOTLIGHT

In conversation with… Prem Natarajan, EVP, Chief Enterprise AI Officer and Chief Scientist at Capital One 

Capital One’s Prem Natarajan argued that banks can turn proprietary data into a competitive AI advantage by deeply customizing models rather than relying on retrieval alone. Smaller, fine-tuned models, continuous measurement, and shared learning can improve performance while also reducing latency and cost.

Key Takeaways:

1. Turn proprietary data into proprietary AI. Natarajan said retrieval captures only part of the potential value. Capital One benchmarks open-weight models to determine where deeper customization offers the greatest advantage.
2. Use smaller models for targeted tasks. For Chat Concierge, Capital One distilled fine-tuned models into smaller versions, cutting latency by more than fivefold and helping increase dealership engagement by over 50%.
3. Learn from every interaction. Natarajan advocated collecting telemetry across applications and reinstrumenting older systems to create a continuous learning loop. He also encouraged banks to try Vulnhunter, an open-source scanning harness, and contribute improvements back to the community.

SPOTLIGHT

In conversation with… Hari Gopalkrishnan, Chief Technology & Information Officer at Bank of America

Bank of America is shifting from AI demonstrations to enterprise-wide deployment, with tens of thousands of employees already using AI and spending expected to double next year. Gopalkrishnan described how the bank’s Orchestra platform balances adoption with model selection, cost controls and shared safety measures.

Key Takeaways:

1. Move from demos to diffusion. Erica Assist supports 18,000 call centre agents, 40,000 financial advisors use an AI-enabled CRM, and 4,500 investment bankers use AI to build pitchbooks. Next year’s priorities will combine central strategy with 2,000 ideas gathered from employees.
2. Match models to the task. A simple request, such as initiating a Zelle payment, does not require a massive LLM. Orchestra evaluates the cost of on-premises open-weight and cloud models for each use case, then enforces model and token quotas at run-time.
3. Make guardrails reusable. The challenge is not building a demo but knowing whether a non-deterministic system is correct. Orchestra provides shared evaluations for toxicity, prompt injection and MCP safety, while keeping agent autonomy tightly bounded.

ROUNDTABLE

How to get an edge from evals?
#evaluation, scale, durable advantage.

Evals are becoming essential infrastructure for scaling AI responsibly, measuring value, and monitoring performance. Banks are using them to establish baselines, optimize token spend, and accelerate model reviews. But reliable success measures – and the expertise to maintain them – remain key challenges.

Key Takeaways:

1. Establish baselines. Clear measures of success are essential to demonstrate productivity gains, particularly in processes such as underwriting and KYC.
2. Measure engineer-level productivity. CommBank is developing individual evaluations based on merged pull requests and defect rates to improve token allocation across its 8,000 engineers. 
3. Track task costs, while accelerating model review. Anthropic reported a 10x reduction in the cost of a benchmarked financial task over 12-months. At the same time, the frontier lab is also helping banks shorten model risk reviews from around two months to a few weeks, as new models arrive roughly monthly.
4. Make approved platforms the easiest route. RBC’s agentic control plane streamlines risk reviews for agents built within its platform. BBVA’s Antonio Bravo also highlighted strong evaluations and reliable ground truths as potential sources of lasting competitive advantage.

SPOTLIGHT

In conversation with… Jodie Wallis, Global Chief AI Officer at Manulife and Tyler Jewell, CEO & President at Akka

Manulife’s ambition to generate $1B in AI value depends on scaling production – not simply building models. Wallis and Jewell emphasized shared infrastructure, embedded controls, and automated release processes to help distributed teams move quickly while maintaining governance.

Key Takeaways:

1. Embed controls from the start. Integrating evaluations, risk management, and validation into delivery avoids costly delays. Automated “shipping factories” can accelerate releases while preserving human oversight.
2. Centralize the platform, distribute development. Manulife’s 10-person platform team supports teams across 14 countries, providing infrastructure for business-critical applications.
3. Demonstrate measurable gains. Migrating Singapore’s AI sales capabilities to Manulife’s platform improved latency by 93% and reduced technical errors from 3.1% to 0.02%, without changing prompts.