There is no shortage of conversation about what AI might do for investment firms. At a roundtable we hosted this week at TDFM New York, we were more interested in what has to happen before firms can put it to work at scale.
We brought together senior technology, data and operations leaders from across investment management, banking and asset servicing and started with a fairly simple question: if you were handed a capable AI agent today, what would stop you from putting it into production?
"Data" came up almost immediately.
That was perhaps unsurprising. Financial institutions have been trying to solve data quality, ownership and integration challenges for decades. AI has not made any of those problems disappear. If anything, it is exposing how consequential they become when systems are expected to consume information from across an organization and then reason or act on it.
One participant described the problem in terms most people in the industry will recognize: different functions can have different ways of dealing with the same missing data point. Those approaches may each make sense within the context of a particular risk or business process, but feed all of them into an AI system without that context and the problem becomes considerably harder.
The conversation was therefore less about achieving some ideal state of perfectly clean data and focused more on understanding the context of the data. Where did it come from? Which source is authoritative? Who owns the definition? What happens when something is missing or stale? What assumptions have already been applied before the data reaches the model?
That distinction is important because nobody around the table seemed to believe firms could wait until every data problem had been solved before moving ahead with AI.
There was also a useful counterpoint. AI may itself become part of the answer to some of the operational data problems firms have struggled with for years. Participants talked about using agents to monitor pipelines, identify anomalies, investigate root causes and help teams deal with exceptions without adding large numbers of people to data engineering and operations functions.
So the relationship runs both ways. AI needs better data infrastructure, but firms may increasingly use AI to help maintain that infrastructure.
Data Governance
This is where the discussion became much less theoretical.
Most financial institutions already have well-developed frameworks around data access, information security, model risk and accountability. But an AI agent does not necessarily behave like a human employee using an application. It can move between systems, invoke tools, retrieve information and perform a series of actions in pursuit of an assigned objective.
This raises some fairly practical questions about whether existing access models are sufficient. For example, one person may be assigned a role and receive a corresponding set of permissions, while an agent may need access to a particular system only long enough to complete one task. Several participants suggested that task-based or just-in-time access could become more appropriate as firms introduce greater levels of autonomy.
There was also considerable discussion about traceability. If an agent takes an action, firms need to be able to understand what it did, what information it used and why. The logging requirements alone could become significant as the number of agents and interactions grows.
When to Stop the Agents
The idea of a kill switch came up more than once, not because anyone expects AI systems to routinely run out of control, but because the consequences of an unusual failure in a regulated financial institution can be significant. A system that performs correctly the overwhelming majority of the time can still require very strong controls around the small number of circumstances in which it does not.
This led to a particularly good discussion around the familiar concept of keeping a human in the loop, which may sound reassuring, but several participants questioned how well that works once systems begin operating at machine speed. Can any one person or team meaningfully supervise every decision an autonomous agent makes in real time?
Perhaps the better question is, "where is human judgment genuinely required?"
There will be decisions that need approval before anything happens. There will be others that can operate within agreed thresholds. Some exceptions should stop a process and be escalated. Other activity may be better suited to retrospective review.
The firms that work this out will need something more sophisticated than adding an approval box to an automated process.
For many use cases, the group saw value in beginning with AI in an advisory capacity, e.g., let the system produce an answer, then compare it with an existing process, understand how often it is right, where it struggles and what happens on the periphery. Only then does it make sense to consider giving it greater authority.
One participant made the point that trust does not appear overnight. In institutional environments, it has to be earned through repeated evidence.
There was another example from the discussion that illustrated the difference between an AI tool working and an AI tool being ready for production: An agent was asked to retrieve information that was not available where it expected to find it. It successfully went elsewhere, found the right answer and returned it.
From the user's perspective, this was success. But from an operational perspective, there were more questions. The path the agent took was substantially more expensive than the normal request. It also raised questions about where it had gone, which sources it had used and whether that behavior should be allowed to repeat.
The answer was right. The process still needed work, and that distinction will become increasingly important as firms move beyond pilots. Accuracy is one measure, but production environments also require consistency, cost control, permissions, monitoring and a clear understanding of how a system behaves when the expected path breaks down.
It was striking, too, how different the adoption challenge feels compared with previous generations of enterprise technology. One participant described having users who needed to be pulled back rather than persuaded to try the technology. They were prepared to trust what the model produced because it was fast, useful and appeared convincing.
With that scenario comes a different change-management problem: Firms may spend less time convincing people to adopt AI and more time teaching them when to question it.
By the end of the session, we had spent, perhaps unsurprisingly little time discussing models themselves.
The conversation was about data definitions, controls, entitlements, cost, accountability, operating procedures and organizational behavior. In other words, many of the subjects investment firms have been dealing with for a long time.
What AI changes is the speed and reach with which those old problems can travel through an organization.
More to Discuss
We look forward to continuing the conversation and are considering both a short survey, so participants can compare approaches and priorities with their peers, and another working session, virtually or in person in New York.
If you were around the table, we would also like to know what you want to ask the group next. The best follow-up may be to take some of the questions raised this week and find out how firms are actually answering them inside their own organizations. Please get in touch if you'd like to participate in the next round of discussions, contribute a question to our survey and/or receive the results of our post-panel survey: erika.alter@fundguard.com. In the meantime, please review our recent AI white paper and related thought leadership.
