In-House Roundtable

Mike Silverman

Normally, I'd start by asking what surprised you about the new Index, but I’m curious what didn’t surprise you here?

Annabel Ayles

What’s not surprising to me is that North American banks are still ahead. Last year, we called out that they had made early AI investments in 2018 and 2019, as well as being early to set top-down AI strategies. Everyone has accelerated over the last year, but banks starting behind can’t just catch up overnight. Think about what it means to hire the right talent: making the decision to hire, attracting good people, and recruiting them... It’s not something you fix in a couple months, it takes years.

Colin Gilbert

I would say that one thing which didn’t surprise us was the acceleration in the Leadership pillar. In terms of the number of people discussing AI on earnings calls, or the number of announcements being made, or the number of organizational changes that are being driven by AI strategies, everyone could see this coming. To me, this year’s ranking feels like the space in between heartbeats where everybody coalesces on the same playing field. On the next heartbeat, we’ll see who accelerates and who stays fixed.

The other thing that we really felt in last year’s Index was that a lot of the performance was due to the conscious choice of whether or not to be an active or passive participant in the conversation. That’s no longer a choice.

Annabel

We definitely saw these jumps in Leadership and Transparency scores coming. We published the first Index ranking in January 2023, just after ChatGPT’s release, and that was really a starting line. Before the advent of Gen AI, few banks had any kind of AI strategy, and we're only now seeing some get signed off. The big banks, the ones that are leading and driving this, have taken a solid 12 months or so to define their vision.

What we’ve seen play out in the ranking this year is testing and deploying use cases and getting senior leaders’ heads around AI. What we haven’t seen are decisions around Talent and Innovation pillars, but I think the big moves are going to come next year. If you haven’t spent the last year thinking about your Talent and Innovation strategies, you won’t see improvements come through in next year’s ranking.

Colin

That’s not to say no one is improving on Talent or Innovation. When we look at the big movers in the index, including CIBC, BBVA and HSBC, they’re investing in Talent development, in multiple areas of Innovation, and in greater Transparency.

Mike

And what was surprising in this year’s update?

Alexandra Mousavizadeh

We've seen an acceleration, with scores rising across many banks. Maybe that’s not surprising, but it is interesting to see that play through the data. Our hypothesis last year was that the European banks were one or two years behind. But when we look at some of the major improvements in the ranking, they’re happening at European banks: big improvements at BBVA and HSBC, as well as continued high performance for UBS and BNP.

Another hypothesis we had was that the gap between the three leaders and the rest of the top 10 was going to grow, but it’s actually contracted slightly. On the other hand, when we look at the lowest ranked banks, we can see they’re falling. For us that means that if you stand still, you’re really at the risk of falling behind. Your business is really at risk.

I also think that the Transparency pillar is very interesting: We went from a very sparsely populated pillar last year to something where everyone’s playing to show more visibility. The same can be said for Leadership. That said, banks who are only ratcheting up their performance on the Leadership and Transparency pillars won’t have much more low hanging fruit to reach for. Next comes the really hard graft of hiring talent and putting an innovation strategy in place. We're probably going to see that there’s a limit to how much banks can rise unless they really enact a transformative AI agenda.

Mike

On those surprises, what if we look more deeply into the different pillars of the index? Talent, for starters.

Colin

The fact that the number of AI jobs in the 50 Index banks is growing by 17% – that’s a massive number. Talent pools don’t grow by double digits very often, and this is just a sustained ramp. Usually that growth begins to plateau once you hit critical mass, but massive organizations like JPMorganChase are still growing 16%. The fact that the leaders are not taking their foot off the gas gives no one else room to maneuver or catch up.

Another really important point on talent: If you look at the top-10, banks score well in Talent Capability or Talent Development, but they rarely score well in both. That makes sense, because there’s a difference between hiring for a role and hiring for a capability – we hear that constantly in interviews with the banks. One thing we’re intrigued by is that there are a handful of banks that do both well, and that includes JPMC and Capital One, which is one of the reasons they’re so high on the list. It’s also true of BBVA, which is part of why they climbed so far this year. These banks are fairly unique in that they appear to be prioritizing both the immediate talent needs with the intermediate needs for training and career development specific to AI. They’re fighting the battle, but they’re also thinking about the war.

Mike

Do we want to talk through some of the other pillars real quick?

Colin

The storyline for Innovation is it’s the most stable. In the top 10, there’s some shuffling, but nine of the 10 banks remained identical, and that’s because we have different leaders in each element of innovation – whether it’s research and patents, investments and acquisitions, or partnerships. In some ways it’s the hardest pillar to analyze, because it’s not one story, it’s multiple individual stories.

Mike

How about Leadership. Who are our big influencers?

Sam Meeson

The most interesting aspect of Leadership is that, similar to the overall index scores, on average everyone has gone up by two or three points. So everyone is doing more. There’s a lot more talk going on, driven by banks like CIBC and BBVA, but we’re increasingly seeing banks walk the walk as well, demonstrated by some putting monetary value on specific use cases. That said I don’t think we’re at a run quite yet. For us that would look like more banks pinning a total ROI on AI. Only two have done so this year.

Mike

Let’s talk through what we’re seeing on responsible AI (RAI) in the Transparency pillar? What’s really interesting here?

Sam

I was a little surprised at the scale of the improvement. Last year, Transparency was something that most banks just didn’t score well on. With factors like Gen AI and regulation leading to exec buy-in I think an improvement makes sense, but we didn’t predict just how many of the banks would be able to turn it around in 12 months. Take RAI principles, which take a lot of work to draft, establish and publish. The number of banks publishing RAI Principles grew 3x, usually in key strategic documents, which I think demonstrates the synergy between RAI strategy and overall AI strategy.

On the regional side, the Transparency pillar is dominated by British, Canadian and Australian banks. The EU is notably absent, and there’s only two US banks in the top 10. For the U.K., we've seen banks doubling down on RAI-specific talent, and that’s a big driver of the score. More broadly, the more open regions generally have strong regulatory environments, with close relationships between regulators and financial institutions in a way we don’t anecdotally see in the US.

Colin

And that regional point is significant because these are countries where the regulatory environment is pushing them to move faster on RAI. The open question is whether other territories will follow suit.

Mike

Let's talk about the future. What does the 2024 Index show us about what’s going to happen in 2025? What does next year look like?

Alexandra

I think one thing worth mentioning is the speed at which AI is developing. As a voyeur of technological innovation, I sit back and I cannot believe it. Like everyday, a new model. We’ve suddenly got agentic AI on the scene, which we weren’t expecting for another two years. And so, I think it’s not just us surprised by that, the banks are also surprised. Many banks are saying: “We’re just getting our head around putting something into ChatGPT and getting something out,” and now they have to figure out agentic AI with multiple functions at a time running without a human in the loop.

Colin

And what occurs to us, and also what we’ve heard a lot at our events, is the barriers to entry and the costs of actually using Gen AI are going to zero. That is an exciting and terrifying time for lots of industries, including financial services, because it means that people can catch up very quickly to capital investments that took years, if not decades. That leveling of the playing field is a big open point in terms of how this plays out in the next three to five years. It’s a very exciting time, but it’s a very scary time for a lot of people as well.

Annabel

If we bring it back to the banks, I would say three things:

The first is that I think we’ll start seeing AI actually impact cost-income ratios. At the moment, we’ve seen improvements in cost-income ratios year-on-year across the banks, but it doesn’t really correlate to investments into AI. With more AI use cases in production delivering value, we’re expecting to see that start playing out in cost-income ratios in 2025 or 2026. I think we might actually start seeing ROI from AI show up on the bottom line.

Secondly, I think we’re going to hear more communication around ROI, whether it’s at the use case level or across the bank. What the banks have done this year on comms around ROI still leaves a lot to be desired if we’re being honest. Next year, I think we’ll see even better reporting, so we’ll have to actually start hearing about the ROI.

And lastly I think this is going to be a year of capability building, workforce planning, talent changes and innovation strategies playing out.

Colin

One thing that I would tack on is that it’s one thing to build something, it’s another thing to implement it, and then it’s another thing to get it adopted at scale. That’s where the Index is really going to split into different camps: The speed at which people move through those stages will directly translate to tangible financial KPIs.

Alexandra

These questions are so important: When is that impact coming through? When will we see more robust announcements on ROI? Will it be next year or the following year? There is a risk that boards and exec teams will lose the appetite for investment unless they actually see this coming through. So, I think the pressure is on for good. We’re heading into probably the most critical year for AI adoption. It wasn’t last year, wasn’t the year before, wasn’t really up until now. This is the turning point.