Sign up to our newsletter

Data-driven insights and news
on how banks are adopting AI

Market rewards bank AI leaders

18 April 2024

TODAY’S BRIEF

Hello again. Welcome to The Brief, the publication that serves up the latest data-driven insights and news on how banks are adopting AI.

Today, the Evident team parses quarterly results and asks, do markets reward banks for leaning into AI? Check out the new Evident Poll. Plus, our Use Case Corner and why the EU and Canada are second-class LLM citizens. The Brief is 2,230 words, a 9 minute read.

Before you dive in, we’re excited to announce that we’re hosting our first London Evident AI Symposium on Wednesday 19 June. We’re bringing together senior AI leaders from across the banking sector to discuss how banks are generating value from AI now. Join us.

If this newsletter was forwarded to you, subscribe here. Have a news tip? Letter to the editor? Write us: [email protected].

Alexandra Mousavizadeh & Annabel Ayles

TOP OF THE NEWS

SIX AI-TAKEAWAYS FROM BANK RESULTS

AI muted. It’s earnings season, and while U.S. banks beat expectations, AI commentary was down. Of the 10 largest banks whose earnings calls we reviewed, only Goldman Sachs, State Street, BNY Mellon and Charles Schwab went into any depth on AI, down from six of the 10 that discussed AI in Q4 calls.

Here are five other takeaways from the quarterly calls.

  • Dimon leans in. In his shareholder letter – in case you missed the blanket coverage… – JPMorgan’s CEO sounded downbeat on America’s economy and world peace. But Jamie Dimon went all in on AI (again), invoking comparisons to the advent of the printing press and electricity.
    • JPMorgan revealed over 400 use cases in production, up from 300 or so the bank mentioned last year. Contrast that with other banks that don’t expect to see returns on AI investments for years. (Our ask: More details next time, please.)
    • Also notably, for those with an eye on his political future, Dimon pledged to save banking jobs from automation and “aggressively retrain and redeploy our talent to make sure we are taking care of our employees”. That’s not just very un-Elon – it goes against the grain in banking too. Job cuts, resource shifts to tech and possible culls to incoming analyst classes (per the New York Times) are the talk of the street.
  • Solomonic conversion. In a first, the Goldman Sachs CEO invoked AI in both his shareholder letter and the earnings call. It might be partly because David Solomon didn’t have to dwell on problems with Goldman’s retail division (so 2023, when calls for his head were loud) and could bask in an eye-catching turnaround.
    • In contrast to Dimon, Goldman’s chief struck a measured tone on AI: “Adoption rates will lag, the most fascinating use cases are in their early stages, and a lot of work still needs to be done in data security, regulatory frameworks and ethical considerations.”
    • Solomon highlighted opportunities he saw to advise clients on AI or finance AI-driven companies … but stayed mostly mum on Goldman’s own use cases. This is unsurprising: as we noted in our AI Leadership Report last year, Goldman regularly publishes thought leadership on AI and its impact on society - and consequently generates a fifth of all AI media coverage related to the banks - but rarely comments on the bank's own AI initiatives.
  • ROI realism at BNY Mellon. In contrast to Solomon, BNY Mellon CEO Robin Vince was more specific about the bank’s AI activities, describing how its centralized AI hub - now at 100 employees - is critical to scaling the bank’s use case roll-out: “It’s particularly important because problem statements that sound different can in fact have very common root causes.” The call was also notable for highlighting more sober time frames when it comes to ROI in AI: it’s “a '26 and out benefit”.

  • BoA ❤️ Erica. In his annual shareholder letter, Bank of America CEO Brian Moynihan said its Erica chatbot has fielded over two billion requests since launch in 2018, a billion alone in the last year-and-a-half. (By contrast, Wells Fargo’s chatbot handled 20 million interactions in the past year.)
    • Erica resolved 43% of corporate customer inquiries, reducing the strain on customer service and bank branches. That’s impressive, though Moynihan didn’t offer specifics on savings.
    • After 10 years spent developing the chatbot, one concern for BoA must be the emergence of large language models that’ll let competitors easily catch up. ING recently revealed that their new LLM-driven chatbot was built in seven weeks (more below in the Use Case Corner).
  • Bringing the work home. State Street’s CEO suggested AI will let it offshore fewer operational jobs. “The technology is at a cost level where we can simply eliminate the labor,” said Ron O’Hanley.
    • Fine for ops and customer service, but U.S. banks are certainly looking to India for software and data engineering talent, according to Evident’s AI Talent Report.

EVIDENT POLL

NOW A QUESTION FOR YOU

When do you expect to see significant return on AI investment in banks?

A. This year
B. 2025
C. 2026
D. By 2050 for sure
E. Never

We’ll share the results next time.

AI-LPHA

MARKET REWARDS EVIDENT AI LEADERS

Bank stocks were driven down this week by concerns over inflation, interest rates, and geopolitics. Over the longer term, some banks have fared a lot better than others. What’s the differentiator? Well, we’ve always believed that banks that lean into AI and innovation do right by shareholders … and ran the numbers to check.

No question: The chart toppers on the Evident AI Index (a proxy for AI maturity) outperformed their peers. As an unweighted basket of stocks, the top-5 banks in the Index (JPMorgan, Capital One, Royal Bank of Canada, Wells Fargo and UBS) were up 33.55% the past year, double the performance of their top-25 Index peers and over three times the sector as a whole, according to Seeking Alpha. We don’t pick stocks here, but the chart speaks for itself.

12-Month Stock Price Performance of Banks Topping Evident AI Index

Updated: April 15, 2024

12-Month Stock Price Performance of Banks Topping Evident AI Index

Source: The Evident AI Index, November 2023 | Seeking Alpha

SPEED READS

THREE ITEMS THAT CAUGHT OUR ATTENTION

1

Main Street could benefit as much from AI as Wall Street, reports CNBC. "The teller line, as we see it today, will eventually die," said Christopher Naghibi, CEO of First Foundation Bank, predicting branches will become “a wall of screens.” One big challenge for smaller banks: data. It’s king when it comes to building AI tools and they have a lot less of it than the giants.

2

Stanford’s annual AI Index Report highlights America’s dominant position over rivals. The skyrocketing cost of training these models – Google spent $191 million on compute to train Gemini Ultra, for example – also explains why industry dominates academia in frontier research, Stanford says.

3

Crédit Mutuel’s Arkéa division quietly released a selection of large language models on the open-source AI platform Hugging Face. In an interview, Arkéa CDO Maxime Havez said the bank’s embrace of open source reflects a commitment to “transparency.” Sure. It’s also good PR – their top model has over 25,000 downloads on the platform – and good for attracting research talent.

LONDON CALLING

ANNOUNCING THE EVIDENT AI SYMPOSIUM

Image showing advert for the London Evident AI Symposium on 19 June 2024

Following our inaugural Evident AI Symposium in New York last November, come join us for the inaugural symposium in Europe this June. We’ll gather senior leaders from across the banking sector to cut through the hype to understand the realities of AI adoption in the banking sector. The theme of our London event will be “Accelerating Outcomes: How are banks delivering value from AI now?”

LATEST FROM THE EVIDENT AI INDEX

AMERICA KEEPS INNOVATION LEAD

Per Isaac Newton’s first law of motion, an object in motion will remain in motion at a constant velocity unless acted on by a net external force. Physicists call this property inertia. Our AI Innovation Report shows Newton’s law in, umm, action.

We found plenty of activity: 284 new AI research papers, 449 new AI patents, and 97 new AI investments registered by banks in the six months since our last update. And nothing changed in the rankings. As we were saying, inertia.

The top-3 banks (JPMC, Capital One, RBC) in the Innovation pillar lead the overall ranking. Nine of the top 10 are in North America, early movers in building AI research teams, patenting and pushing their venture arms toward AI. Also no change. The winners keep winning. For more, listen to Alexandra (a.k.a. “Indexing Boss”) at the Brainstorm AI conference in London on Monday.

Innovation Pillar Leaders, by Geography

Innovation Pillar Leaders, by Geography

Source: The Evident AI Innovation Report, April 2024

NOTABLY QUOTABLE

BEST OF EVIDENT ROUNDTABLE

We used our first virtual roundtable of the year to explore key findings from the Evident AI Innovation Report with Chintan Mehta, Group CIO of Wells Fargo, and EJ Achtner, Head of Applied AI at HSBC. Watch the entire session on our YouTube channel. A sampling here:

How banks define innovation

"The principal outcome here is how to better serve the customer in a way that aligns to your organization’s risk appetite and ultimately delivers the expected results. And while Evident’s five [Innovation] sub-pillars are critical ingredients—across our bank’s 60+ market footprint, how we do what we do and where we do it varies greatly." - EJ Achtner

Balancing “blue sky” innovation against measurable ROI

"The approach is a combination of sourcing ideas from anywhere and everywhere, and filtering them and taking them through an experimentation and research process which is cross-disciplined, but orchestrated centrally. We use the DVF framework (Desirability, Viability, Feasibility) and repeat this filtering process across all stages of production . . . and it has been working reasonably well so far." - Chintan Mehta

How an increasingly open-source ecosystem might challenge conventional strategy on patents

"My present perspective is different now because of what’s going on in open-source and what’s happened over the last few years in terms of the barrier to entry for production. So patents and IP protection are absolutely an important part of the ecosystem, but at the end of the day, I really question the importance in this particular area—because of the pace of innovation securing customer, shareholder, and regulatory outcomes." - EJ Achtner

USE CASE CORNER

SHOW ME THE RO(GEN)AI

Banks are increasingly sharing how they deploy AI and hinting at the ROI of their investments — we call it ROAI, return on AI. We scour and reality check the announcements. These three generative AI use cases stood out in the past two weeks.


#1: Generative AI-enabled software development

Bank: ANZ
Use Case:
Ensayo AI
Vendor:
HCL & AWS

Why it’s interesting: Rico Zhang, Platform Engineering and SRE Capability Area Lead at ANZ, says the tool streamlines every stage of development. That helps business analysts write better requirements from the beginning as well as analyze code changes, identify issues and give developers recommendations.

Potential ROI → reduced time to production, increased software quality
Reported ROI
→ API testing time reduced by 72%, integration testing time reduced by 56%


#2: Internal chatbot for financial advisors

Bank: Morgan Stanley
Use Case:
AI @ Morgan Stanley Assistant (AIMS Assistant)
Vendor:
OpenAI

Why it’s interesting: Morgan Stanley offered a rosy picture of the uptake of its new tool, but didn’t provide any fresh information on efficiency gains. Business Insider dug deeper, finding a financial advisor who showed how the tool improved his workflow.

Potential ROI → increased employee efficiency, increased accuracy, increased customer satisfaction
Reported ROI
→ 98.5% of teams had at least one member of the team use the product once a week or more


#3: Generative AI chatbot for customer support

Bank: ING
Use Case:
Customer-facing chatbot
Vendor:
McKinsey

Why it’s interesting: In partnership with McKinsey, ING released the tool in September 2023. McKinsey has begun to publicly share data on increased use, but not other quality metrics, like time for query resolution.

Potential ROI → increased NPS (net promoter) scores, decreased time to resolve customer query
Reported ROI
→ 20% more customers assisted in first seven weeks of use

TALENT MATTERS

MASTER-RESHUFFLE

Mastercard appointed Greg Ulrich its Chief Data & AI Officer. Ulrich will be heading up a newly centralized Data and AI division. The payments provider recently hired Director of Global Product Data Strategy Phani Kaligotla from AWS.

Citi is looking for a Director of Generative AI in London.

BNP Paribas is looking for an AI Research Head in Montreal.

BNY Mellon has an opening for an AI Research SVP in Tel Aviv.

AI GEOPOLITICS

WORLD AI-N’T FLAT

The planet can be divided into AI have and have nots – when it comes to access to the most state of the art large language models. The countries in blue can make full use of the latest LLMs, including from Anthropic and Google. Red means you can only get ChatGPT-4 from OpenAI.

The EU and Canada are frozen out of non-ChatGPT models

The EU and Canada are frozen out of non-ChatGPT models

Source: Evident Analysis

To create this map, we looked at the regional availability of APIs that let developers use models for their own code. OpenAI is freely available in Europe and Canada. Anthropic only offers limited access to its smartest model Claude Opus via Amazon, and Google doesn’t offer API access to Gemini Ultra at all in Europe and Canada, four months after its U.S. release.

What this also shows: Worries about the EU’s and Canada’s regulatory regimes stifling innovation, from the point of view of the model makers. At a recent hearing about Canadian legislation, largely modeled on the EU’s AI Act, Meta complained that “compliance costs are incredibly high” and could limit access to its products.

Why this matters for banks: It could limit experimentation with generative AI. Models have different strengths and weaknesses (eg. Anthropic’s model outperforms on coding), so a lack of options could hit performance or raise costs. That’s potentially a problem for European banks, already behind their North American counterparts. (In the most recent Evident AI Index, only two of the top 10 banks are European.)

THE BRIEF TEAM

Alexandra Mousavizadeh | Co-founder & CEO | [email protected]

Annabel Ayles | Co-founder & co-CEO | [email protected]

Colin Gilbert | VP, Intelligence | [email protected]

Andrew Haynes | Head of Data Science | [email protected]

Alex Inch | Data Scientist | [email protected]

Sam Meeson | AI Research Analyst | [email protected]

Matthew Kaminski | Senior Advisor | [email protected]