How Banking Execs Are Navigating the Pitfalls of Artificial Intelligence

How AI Drift Impacts Results and Creates Risks

The Federal Reserve Bank of New York is among those that have run into challenges with the specialized language models underlying its AI systems.

“As you can imagine, the financial language that’s being used changes over time,” said Harry Mendell, data architect specializing in artificial intelligence at the New York Fed. The terms “omnichannel”and, for example, don’t mean exactly what they did five years ago, and even now, humans don’t always agree on what they mean.

“We had to keep retraining our models,” said Mendell.

It became clear that continuous training made more sense, so the New York Fed now strives for weekly updating.

Suresh Ande, director of global markets risk analytics at Bank of America Merrill Lynch, and Ercan Ucak, a vice president at Cerberus Capital Management, agreed that frequent retraining of artificial intelligence technology is necessary. All three executives participated in an event that Re•Work hosted in the spring to allow AI practitioners in financial services to share their experiences — and lessons learned with each other.

The structure of AI models can reflect the way a particular process was first designed, Ucak said. Sometimes processes can change sufficiently enough that retraining in the basics may be necessary.

Another aspect of drift which calls to mind HAL from the 1968 film “2001: A Space Odyssey” happens inside the technology. Sophisticated AI models can migrate forward from their original programming, according to Techopedia. One notable result, the website recounted, was when two Facebook chatbots began to “talk” to each other several years ago in a secret code not envisioned by their builders.

The idea of drift comes up in other contexts in financial services as well, such as concerns that, over time, AI used for credit evaluation can pick up biases that can lead to discrimination.

How Do You Measure Return on Investment from Artificial Intelligence?

The banking industry has seen a virtual explosion of use cases for artificial intelligence, a trend hastened by the belief in its potential to boost revenue, increase production, improve efficiency, and otherwise generate benefits that go far beyond what can be achieved with traditional technology.

However, the three panelists at the AI in Finance Summit which focused on exploring the challenges of adopting this technology in financial services made it clear that there are detours between the adoption of AI and the eventual benefits.

Even determining the return on investment for an AI implementation is not a straightforward matter. Cerberus Capital’s Ucak told listeners that establishing key performance indicators up front is essential to understanding what effect an AI application might be having. To make that happen, a governance structure has to be put in place to ensure consistent measurement and evaluation.

Ande of BofA Merrill Lynch said that some types of ROI can be more easily measured than others. For example, let’s say a bank decides to use AI-based computer code generation tools to speed up programming time. If the process usually takes 10 days and AI cuts it to five days, a clear increase in productivity has resulted, he said.

Mendell said that the New York Fed’s supervisory division has used AI to assist with bank examinations. “We were able to have examiners complete their work in a matter of days instead of weeks,” he said. That’s a clear gain in productivity.

At a traditional commercial bank, AI might deliver similar benefits when used for an internal loan review or a compliance audit.

But assessing the return on investment is trickier in cases where artificial intelligence is used for customer interactions, Ande noted. A lot would depend on the metric being used.

For example, both retention and satisfaction affect the bottom line. Measuring customer retention is straightforward people stay or they leave. But assessing customer satisfaction would require qualitative research to see how happy or unhappy people were.

In addition, if an AI-driven process alienates some customers and they complain on social media, how should that reputational erosion be factored into ROI calculations?

“It’s very difficult,” Ande said, “to measure things in terms of negativity.”

What Are ‘Digital Twins’ in the Banking Space?

One of the many interesting topics that came up in the panel discussion is the concept of digital twins.

A digital replica of retired basketball star Carmelo Anthony is a fun application of the concept, said Ande.

Soul Machines, a company that designs avatars of people for use in the metaverse, created the of Anthony. It makes “Digital Melo” available for influencer appearances on social media and even for appearances with the real Anthony himself.

Business-oriented digital twinning is not as fun in spirit yet is meaningful in results

“You basically look at your business and then develop a digital counterpart of it a clone,” said Ucak. These twins can range from very elaborate to cursory. At the far end, internet-of-things devices and other monitors can be tied in for added realism. “You can go all out, having a lot of devices everywhere,” said Ucak.

Mendell said the New York Fed has been using a digital twin approach to better manage its currency distribution function. Using AI technology for that has made it more efficient.

Source: THE FINANCIAL BRAND

Synthesized by the author group DTSVN - Digital transformation solutions for the Finance - Banking industry.

-------------

DTSVN is a pioneering Digital Transformation Company providing the latest digitization solutions exclusively for businesses in the Finance - Banking industry in Vietnam; helping banks and financial institutions quickly perfect the technology system for Digital Transformation.

Contact us now for advice and experience the solution here.