New model? Whatever

The new models keep getting rolled out. The reaction from businesses that are supposed to be a core audience for them: 🤷‍♀️.

When OpenAI launched GPT-6 Astra last week, its president, Greg Brockman, declared it the beginning of the AGI (artificial general intelligence) era. Nvidia CEO Jensen Huang went further: “AGI has arrived.”

The people who use this stuff for work don’t seem to see it that way. “After living with [Astra] for the last few days, I’m not even convinced it’s the step change people want it to be,” wrote Simon Villani, lead AI engineer at ANZ. “Sometimes it does something where you think, holy shit, this is getting serious. Then 20 minutes later it’s confidently disappearing down some dumb rabbit hole that should have been killed three experiments ago.”

The model is no doubt book smart: It announced itself by solving a number of long-standing math problems. But it’s not all that (Wall) street smart, it seems: On benchmarks that look more like the work banks might give it – reading piles of documents or handling multi-step office tasks – the results are mixed. Astra gained points on one widely-used test of workplace skills like presentation building and memo writing, but it lost roughly the same amount on another similar test, independent evaluator Artificial Analysis found. The cost of completing those tests with Astra, meanwhile, is 60% higher than its predecessor, GPT-5.6 Sol.

Banks – which care more about their bills than mathematical proofs – have been voting with their feet on these models that push the so-called frontier. Astra is a big improvement over GPT-5, the model released around this time last year. In these past 12 months though, OpenAI's market share among the 50 banks we track has fallen another 15%, according to Evident’s Use Case Tracker, our database of all the use cases lenders have publicly rolled out.

That doesn’t mean Astra won’t be useful for banks in time: Other tests measuring the model’s ability to run a hypothetical business show clear gains over peer models like Anthropic’s Fable 5.1. But the problems banks need AI to solve today aren’t on the Astra(l) plane; they’re much more practical. “A lot of the work in finance isn’t technically complex, it’s just very fragmented or very time intensive,” Katy Barker-Cook, director of future capability at Nationwide Building Society, said this week. “It’s not the thinking that takes the time. It’s all the stitching together of that work.”

The latest models are getting much better at the thinking part, but until they make the stitching work cheaper, more reliable and easier to govern, each leap in the frontier – and each new claim of AGI – feels disconnected from what’s actually happening in banks. For now, these feel like two completely different races. And bankers are skeptical that world-beating advancements on the model frontier actually translate to real-world prowess.

“Today I would not fly a plane where the operating system is developed by AI,” ING CTO Daniele Tonella said this week. “As simple as that.”

Banks have been leaning on “skills” – bite-size instructions that teach AI models how to do certain jobs – so LLMs don’t start from scratch each time they’re asked to do a similar job (see: “Skills pay the bills,” The Brief, Feb. 12). It lets good ideas – and good ways of doing a job – travel faster around a bank. It has the potential to do the same for bad ones, too.

In this week’s “Corner,” we look at a new tool from Nubank (#1 in the Evident AI Index for Banks - LATAM) that screens these homegrown AI skills for potential risks before they end up in an internal marketplace where other developers can borrow them.

Use case: Skill Vetter
Vendor: N/A
Bank: Nubank

Why it’s interesting: Nubank built a security tool that stops the spread of bad AI around the bank. If an engineer writes a sloppy skill and publishes it to the bank’s internal marketplace, it can get passed along to any other developers at the bank, Lucas Palma, a security engineering senior manager, said at an event this summer. That means small errors can spiral. Say a developer writes a skill that gives an agent more access than it actually needs. It may be harmless in that developer’s project. But if that skill is reused somewhere else, it could cause havoc. “Depending on how the skill was configured, it might have excessive permissions much more than what was needed,” Palma said. “Even a typo can make some dangerous stuff [happen] depending on who is using that skill.”

How it works: Skill Vetter starts by running through a list of the obvious issues: instructions to pull sensitive files or commands to change or delete them. But it goes further than checking skills against a rubric: Nubank’s tool gets an LLM to “interpret relationships between instructions, tools and intended behavior,” Palma and fellow security engineer Paulo Martins wrote. Say a skill tells an agent to ask for confirmation before taking a riskier action. Skill Vetter wouldn’t just look for the word confirmation; it would read the surrounding instructions to determine who’s actually allowed to give it that permission. Without that kind of check, an agent could end up asking another AI – or even itself – for permission before carrying on, Palma said.

How they did it: Nubank designed the tool to work like a bouncer for its internal skills marketplace: Unless it passes the check, it doesn’t get in. Developers have the option to run the same kind of checks themselves as they’re building, but Nubank runs them again automatically before they go wide. That makes it the “boundary where we are trying to make it safer,” Palma said.

By the numbers: The bank has now run the tool across more than 2,000 skills in its marketplace and has found 1,600 potential risks as a result. Roughly 1,000 have already been fixed, the bank said.

Bigger picture: Banks are getting less comfortable relying on existing guardrails to keep AI agents in line. Models have shown they’re getting better at finding the gaps between what they’re told to do and what the system actually prevents them from doing. Just ask OpenAI and Anthropic (see: “Rogue AI,” The Brief, Aug. 6). That’s pushing banks towards security tools like Nubank’s and tighter guardrails altogether: DBS is rewriting its rules around system access for agents. Santander is hard-coding checks and balances. And Capital One is using an AI model dubbed a “guardian” to monitor what other models around the bank are getting up to.

India is the hottest market for AI banking jobs, new Evident analysis of hiring trends shows.

In the last six months, the 50 lenders in the Evident AI Index for Banks have brought on 3,000 new people in India. Bengaluru hiring alone equaled that of New York, London and Toronto – the three cities with the most AI banking talent overall – combined.

Banks have been hiring in India for decades, largely to cut costs by filling data, tech and support roles with cheaper talent. That’s still the case. But the unique view workers at these Global Capability Centers (GCCs) have into where data lives and how it all joins together is making them useful to banks’ broader AI buildout, too.

“GCCs are no longer just ‘support hubs.’ They are the architects of the Enterprise AI engine,” wrote Shrikrishna Shanbhag, a Bengaluru-based senior director of data, AI and engineering at NatWest. “If the reference data is wrong in Bengaluru, the AI hallucination is felt in London, New York, and beyond.”

Ramkumar Narayanan, FIS’s executive vice president of enterprise and AI platforms as well as its leader of the India GCC, has a foot in both places. He’s seeing the Indian teams add more value as financial firms look to have AI agents handle processes from start to finish. “These are the people who have very good visibility into how you service your customers,” he said on a podcast. “There’s a big opportunity for those people to get together and redefine what those workflows could look like.”

The hiring doesn’t appear to be slowing: Charles Schwab opened a new office in Hyderabad last month that it plans to grow to 2,000 employees next year. JPMorganChase plans to hire another 1,000 people in its GCC to work on AI data pipelines. In 2029, the bank will also open a Mumbai campus that’s roughly 80% of the size of its new Manhattan headquarters by square footage.

Banks are still figuring out how to fill those spaces in ways that’ll advantage their AI efforts most, and the question of which roles to outsource to cheaper markets always remains open. For now, the push seems to be bringing offices thousands of miles away closer together.

“The distinction between ‘onshore’ and ‘offshore’ is gradually becoming less important,” wrote Keshav Saraf, an investment banker at JPMC. “GCC professionals are working alongside bankers.”

Filippo Scognamiglio joined JPMorganChase as head of global technology strategy. Previously, he was a managing director and partner at BCG, where he led the firm’s cloud transformation practice. The bank also brought on another consulting vet, Gary Ryan, to be executive director of data strategy and architecture for applied AI. Ryan spent the last 14 years at Accenture.

TD Bank is shaking up its tech leadership: Vlad Shpilsky is now group head of the Global Technology and Solutions group at the bank. In the role, he’ll lead technology strategy for the Canadian bank, including”enabling the bank’s use of artificial intelligence,” the release said. He’d served as the bank’s U.S. CIO since 2024. Paul Whitehead, meanwhile, was named senior executive vice president of the Global Corporate Services group, where he’ll “assume responsibility” of enterprise AI. Whitehead has worked at the bank for 38 years. With the changes, COO Taylan Turan is leaving the bank. Turan had been with TD for roughly a year after joining from HSBC.

Michael Demissie joined Apollo Global Management as chief AI officer. Demissie was previously head of applied AI and practice at BNY, where he’d spent seven years.

Goldman Sachs hired Dinesh Keswani as CTO of a unit called “The Core,” a group of more than 2,000 engineers delivering analytics, data and engineering work to six of the bank’s core businesses. Keswani joins from Nomura, where he was global CTO.

Jonathan Lofthouse, longtime CIO at Citi, will join LSEG as CIO of the firm’s data and analytics division. In the role, he’ll “shape how we use AI, modern engineering and our unique data assets to build the next generation of products,” he wrote on LinkedIn.

Wells Fargo hired Gyan Prakash as executive director and principal engineer within the cognitive AI solutions team. He’ll be “focused on agentic AI and graph intelligence” in the new post, he wrote. He was previously senior principal software engineer at PNC.

Truist hired Christopher Liszewski as senior AI security engineer. He’ll focus on “shadow AI detection and enforcement, AI security governance review, and building the frameworks and processes that allow the organization to identify, assess, and manage AI-related risk across the enterprise,” he wrote. He previously worked at Nutanix.

“We should not stop hiring juniors for a couple of reasons. First of all, because if we stop hiring juniors, we are assuming that AI is going to really take away the whole chain and we don't know yet. So we might be making a stupid mistake. Second, if I look at graduates, they have been growing with a way different AI attitude than the people we have in the organization. So we need to have them to really smell what works and what doesn't.”

–Daniele Tonella, CTO of ING, on a podcast, Sept. 8

TD Bank launched an agentic tool that helps the bank with collections, CEO Raymond Chun told investors Wednesday. When the bank was using humans to try to connect with the person who owed money, they were getting through 6% or 7% of the time, Chun said. The bank is now using agents to figure out when the best time to contact them is and has upped that rate to between 20 and 25%. “It's not just a cost reduction.” Chun said, “There's significant productivity benefits.” It’s not the only lender using agents to handle parts of the collections process: BBVA is piloting an agent from Sierra, OpenAI Chairman Bret Taylor’s firm, “with the objective of proactively engaging customers, understanding their individual situation, and helping them identify the most appropriate path forward,” a spokesperson from the bank told The Banking Brief. So far, BBVA is trialing the agent with 100,000 customers in Argentina and 5,500 in Spain. It was able to set that pilot up in 30 days.

The top 10% of AI users on OpenAI’s research team are now on pace to spend $1.7 million per year each on coding agents, the firm said this week. That’s up from their pace of roughly $500,000 per person annually just a month prior. The spike comes from growth in a metric the firm is dubbing “agent workdays.” OpenAI’s agents, the firm says, are now doing three days worth of work for every one human day – meaning each person becomes capable of doing the work of four. But as their spending spree shows, it’s not work that’s coming cheaper: The firm is effectively tripling its spend to triple its productivity. And missing from the release is a qualification of whether those three days of work are actually up to snuff. OpenAI will likely find a way to bring the cost of the extra productivity down, but for now it’s getting more for more rather than more for less.

UBS is requiring new banking hires to demonstrate they are proficient with AI, adding interview questions to assess whether a candidate knows how to use the tech to get jobs done. It “complements, rather than replaces” the regular academic credentials, the bank said. AI proficiency is getting baked into hiring across the financial sector, even if it’s not quite so explicit everywhere: Coinbase this summer detailed how it changed up engineering hires now that AI is writing almost all the firm’s code. 

Demand for DeepSeek’s products has the firm’s systems scrambling to keep up. The Chinese model-maker is going on what it’s calling an “unprecedented” hiring spree, looking for 150 senior engineers who can overhaul the firm’s back-end systems. The push for new talent comes as DeepSeek beefs up its enterprise push: The firm gained traction with models that cost a fraction of U.S. alternatives. Now it’s pushing to build the infrastructure to make those gains stick in business.