The warning didn't sound like sirens; however perhaps it should have been. It seems that in some of the more closed policy rooms and the urgent internal reports of the European banking and financial supervisory bodies, the authorities are beginning to suspect something alarming. The coming financial apocalypse may not be caused by any of our fellow homo sapiens.
First of all, an AI model that can test the system, find its vulnerabilities, and in some cases, use it. According to several anonymous industry sources, the ECB has started contacting banks to ask them how they feel about this new risk exposure. It has already happened, and there are details in this article you can read more about: ECB officials are looking at the risk that financial institutions may be exploited by the agency's AI models.
Yes, we always talk about the risk of cyberattack from cyber attacks, and of course we always had that. But this is no ordinary guy wearing a hoodie underneath. This is a code with steps that I can think of, actions that I can combine and to test some complex attacks. This is what worries them.
And now, we come to the strangest part of this story. Some of those executives have publicly stated that they are “very aware” of the risks; i.e. they don't sleep well at night. AI systems such as Anthropic's Mythos are reportedly already capable of conducting autonomous cyberattack simulation exercises without human intervention. If this doesn't sound too scary.
But that's not the whole picture. There are also reports that Mythos is the first version of AI agents. Overall, the evolution of AI goes from a chat-bot that can generate text, to one that can perform planning and reasoning steps, to AI agents that can actually execute the programs they generate. Regulators are increasingly concerned about AI agents and are debating what kind of regulations might be needed.
So where does that leave us? In a kind of “surprised,” half-nervous kind of place. On the other hand, imagine AI that can find security holes and fix them before bad actors do. On the other hand… imagine bad players getting them first.
Then there's the issue of trust, which doesn't get much paper. If banks, some of the most risk-averse organizations in the world, are in turmoil, what should the rest of us think? Would you trust an AI with your money if it has the power to steal from someone else?
Some people believe that the “AI race” is already underway, with nations and companies trying to develop defenses as quickly as they see offensive uses.
Ultimately, this is not just another technical issue; it is one of the many signs of the future.
