Seven Theses on the Current AI Safety Debate
How can trust in artificial intelligence be turned into verifiable safety? Michael Weberberger, Managing Director of AI service provider TRUSTIFAI, the joint venture of TÜV AUSTRIA and the Software Competence Center Hagenberg, formulates seven theses on the current AI safety debate and argues for the next step beyond governance and compliance: measurable, scientifically grounded AI assurance.
AI safety has entered the mainstream. What only a few years ago was considered a specialist topic for research teams has become a strategic issue for companies, technology providers, customers, investors, and regulators.
The debates surrounding the major AI labs show that the more capable AI systems become, the more important it is to understand how their risks can be systematically identified, tested, and controlled.
The responses vary widely. Technology providers are calling for greater oversight, regulators are introducing new control mechanisms, and companies are increasingly focusing on AI governance and assurance. At the same time, the debate risks becoming trapped between two opposing poles: warnings about ever-growing risks on the one hand, and concerns that regulation could slow innovation on the other.
It is time to frame the debate differently.
1. AI Safety Does Not Begin at the Technological Frontier
The most high-profile examples in today’s safety debate come from the major AI labs. But treating AI safety as a specialist problem of frontier research misses the point.
AI safety already matters to companies developing AI, optimizing models for specific tasks, deploying RAG systems, connecting agents to tools, or integrating AI into operational processes.
Neither AGI nor superintelligence is required for this to become relevant. An AI agent with access to customer data, production systems, ordering processes, or internal software can already have significant real-world impact today.
AI safety therefore does not begin at the technological frontier. It begins wherever AI influences decisions or triggers actions.
2. The Risk Is Not the Model Alone, but the System
The safety debate often focuses on the AI model itself. In real-world enterprise environments, however, the actual risk profile emerges only from the interaction of the entire system: models, system prompts, data sources, RAG components, APIs, tools, permissions, policies, software logic, and the specific application context.
This means that the model can remain unchanged while the risk profile of the overall system changes significantly.
Anyone who takes AI safety seriously therefore cannot limit evaluation to models alone. What matters is evaluating the entire AI system.
3. More Oversight Does Not Automatically Mean More Safety
Powerful AI requires independent oversight. But one equation is too simplistic: more control equals more safety.
If oversight mainly consists of documentation, checklists, and formal approvals, a system may appear perfectly safeguarded on paper while still containing significant technical weaknesses.
The key question is therefore not only whether oversight exists, but how it is carried out and what evidence it produces.
AI safety must not become safety theatre. It requires statistical methods, technical testing, and scientifically robust evidence.
4. Compliance Matters. Assurance Goes One Crucial Step Further.
Regulation creates rules, roles, responsibilities, and minimum requirements. That is necessary. But compliance primarily answers one question: Have the prescribed requirements been met?
Assurance asks a more demanding question: Can it be demonstrated with sufficient confidence that a system operates reliably, safely, and controllably under real-world conditions?
Answering that question requires testing, measurement, technical evaluation, and traceable evidence.
Trustworthy AI is not created by promises. It is created by scientifically robust evidence.
5. AI Cannot Be Tested Once and Then Checked Off
Traditional assessment models are often designed for relatively stable systems. AI systems are dynamic: models change, prompts are adapted, data sources grow, APIs are added, agents receive new tools, and business processes evolve.
An assessment is therefore always a snapshot of a specific system state.
The consequence is clear: AI assurance must increasingly be designed as a continuous process.
This may include re-testing, automated monitoring, change detection, and more deeply embedded forms of independent evaluation. Not every minor change requires a new certification. What matters is the ability to detect and assess when a change materially alters the risk profile.
6. AI Will Monitor AI. But Who Monitors the Monitor?
As AI scales, purely human oversight quickly reaches its limits. Millions of automated decisions, agent actions, or model interactions cannot be reviewed manually. Technical monitoring systems and AI itself will therefore take on a significant share of the oversight function.
This creates a second layer of trust: How reliable is the system that monitors the AI?
A monitoring system whose performance cannot be understood statistically merely introduces another layer of uncertainty.
We therefore need measurable error rates and robust evidence showing which risks are detected, which may be missed, and with what level of reliability. Otherwise, one black box is simply replaced by another.
7. Safety Must Not Become a Privilege of Large Companies
One legitimate concern in the current debate is this: What happens if increasingly complex safety and compliance requirements themselves become a barrier to market entry?
If only large companies can afford the structures required to meet them, little would be gained for either safe or innovative AI.
Good AI assurance must therefore be proportionate: risk-based, capability-based, and closely aligned with the actual application.
An internal assistance system requires a different level of scrutiny than an autonomous agent with access to critical systems. The goal therefore cannot be maximum bureaucracy. The goal must be maximum insight into real-world risks.
What Follows From This?
The current AI safety debate is important, but it should not be reduced to the question of whether we need more or less regulation.
The decisive question is: What makes safety verifiable?
That is where the next stage of development lies: from governance to evidence, from compliance to assurance, from point-in-time assessments to continuous safeguards, and from abstract trust to robust proof.
TRUSTIFAI supports organizations in developing, deploying, and operating AI in a safe, trustworthy, and demonstrable way — from AI governance and readiness to the assessment of specific AI systems and preparation for testing and certification.