Agent-based testing: How AI agents are revolutionising quality assurance in the banking and insurance sectors

The financial sector is under immense pressure: tighter regulation, an accelerating digital transformation and, at the same time, a growing shortage of skilled workers. Whilst AI has already found its way into many areas of software development – for example, as part of adesso’s AI-driven process model ‘adSCAILE’ – a new approach is now emerging that has the potential to revolutionise quality assurance in particular: agentic testing with AI agents.

These AI-based test agents promise to make the traditional testing process – from test case creation through test execution to defect handling – significantly more efficient and to noticeably reduce the human testing workload. Initial estimates suggest savings of up to 80 per cent of the manual testing effort. The aim is explicitly not to replace people, but to relieve them of specific tasks and move them into a new role: from test executor to quality decision-maker.

Why testing effort is skyrocketing in the financial sector

Factor 1: Regulation is driving up testing requirements

Banks and insurers in particular have been facing significantly stricter compliance requirements for several years now. Regulatory frameworks such as DORA and supplementary BaFin guidelines, which are due to come into force from the start of 2025, aim to achieve greater cyber resilience – with direct implications for software quality assurance.

The number and complexity of cyberattacks have risen considerably. Multi-vector attacks and AI-enabled attacks are forcing institutions to regularly assess the resilience and security of their applications through additional, specialised tests. Added to this are greatly expanded verification and documentation requirements:

• seamless traceability from requirements through test cases to test results,
• audit-proof documentation of every release,
• transparent assessment of findings and defects according to criticality and risk.

The result: more and more frequent regression tests – even for seemingly minor changes such as configuration adjustments or patches.

Additional effort arises from services outsourced to third-party ICT providers such as cloud providers or SaaS providers. These must be integrated into the organisation’s own testing strategy, regularly assessed on a risk-based basis, and the joint tests planned, coordinated and documented. The testing workload is increasing noticeably across all areas.

Factor 2: Digital transformation and the cloud increase complexity

Traditional banks and insurers – unlike many fintechs – have been operating highly complex legacy landscapes that have evolved over decades. Host systems and COBOL or PL/1 applications often form the backbone of their core business. At the same time, modern technologies such as the cloud, AI and microservices are to be seamlessly integrated.

This has resulted in numerous migration and modernisation projects (“Change the Bank”), in which:

• legacy systems are gradually modernised,
• on-premises applications are migrated to the cloud,
• and new platforms and partners are integrated.

The example of Commerzbank illustrates this trend: strategic partnerships with Google Cloud and Microsoft, the migration of numerous applications to cloud platforms, and a gradual modernisation of the IT infrastructure.

With the move to the cloud, the operational framework (“Run the Bank”) is also changing:

Cloud providers are regarded as third-party ICT service providers under DORA and must be listed in information registers, monitored and tested.

In-house developments in the cloud and purchased cloud software (SaaS) are subject to an evergreen model: updates and new features are frequently rolled out in short cycles – often without the customer having any say in the matter or receiving detailed information at an early stage.

For banks and insurers, this means: more frequent, comprehensive regression tests to ensure that processes and workflows continue to function smoothly with every new version. The testing workload is also increasing significantly in this area.

Factor 3: Shortage of skilled workers amid rising demands

At the same time, another trend is exacerbating the situation: the ongoing shortage of skilled workers in IT and specialist departments. Banks and insurers are competing with fintechs, tech conglomerates and other sectors, whilst at the same time being confronted with cost-cutting programmes.

• More and more testing tasks have to be carried out by fewer and fewer staff.
• There is a shortage of IT experts with methodological testing expertise.
• There is a lack of subject-matter experts who have a deep understanding of business logic and can evaluate new applications from a business perspective.

It is precisely these experts who should be deployed as effectively as possible – not for time-consuming, repetitive tasks such as the manual documentation of regression tests. Otherwise, there is a risk of overwork, frustration and, ultimately, a loss of motivation.

Why traditional approaches are not enough

False solution 1: Test automation

For many years, organisations have been trying to mitigate the rising testing workload through test automation. The idea is that test cases are automated once and can then be executed at any time ‘at the touch of a button’. In practice, however, this promise has rarely been fulfilled.

Admittedly, there are a multitude of established tools – such as Tosca (Tricentis), Playwright (Microsoft) or Selenium as an open-source solution. But reality shows that:

• The introduction of test automation is often time-consuming and complex.
• Despite modern features such as capture and replay, modifications to scripts are necessary, which require programming knowledge.
• Even minor changes to the application cause automated tests to fail and necessitate time-consuming adjustments.

It is not uncommon for the outcome to be sobering: the effort involved in setting up and maintaining test automation exceeds the actual benefits and the time savings achieved. Consequently, many banks and insurers have either had negative experiences or now use automation only very selectively for clearly defined scenarios.

False solution 2: Outsourcing

Another frequently used option is the outsourcing of testing activities to near-shore or offshore centres. The aim: to reduce costs and expand capacity in the short term. However, here too, practical experience reveals limitations:

• Alignment and coordination across time zones are time-consuming.
• Language and cultural barriers make close, technically precise collaboration difficult.
• For complex banking-specific issues, the need for in-house experts remains high.

In short: neither traditional test automation nor pure outsourcing provides a sustainable solution to the structural conflict of objectives arising from growing testing workloads, regulatory pressure and limited resources.

AI agents as the next stage of evolution in testing

What sets AI agents apart

With the rapid development of Large Language Models (LLMs) and generative AI, a new generation of systems is coming to the fore: AI agents. They go far beyond traditional AI applications:

• They act like virtual employees,
• are proactive, can collaborate in teams,
• learn new skills and continuously improve,
• and can be trained to use other software in a targeted manner.

Many banks and insurers are already using AI assistance systems modelled on ChatGPT, which are operated internally and protect confidential data. The next step is now to deploy these capabilities specifically in quality assurance.

‘adesso test agents’: Agent-based testing in practice

adesso has responded to these developments and developed its own solution for agent-based testing: ‘adesso test agents’. Behind this lies an agent-based framework that can be deployed in any cloud environment – such as Google Cloud, Microsoft Azure or AWS – and uses it to provide AI agents for testing tasks.

How the test agents work

The basic principle: a team of virtual testers (AI agents) is controlled by a higher-level orchestrator agent – the “team lead”. Together, they support human testers throughout the entire testing process:

• deriving test cases from requirements and acceptance criteria,
• executing the test cases,
• recording and describing defects.

The key point here is: AI does not replace humans, but works hand in hand with them. The greatest potential for cost savings lies in particular in:

• integration and end-to-end tests,
• regression tests,
• business acceptance tests,
• migration and data tests.

These areas traditionally involve the greatest workload – and it is precisely here that AI agents can provide significant relief through automation, standardisation and scaling.

The role of the human tester is evolving: away from manual execution towards meta-level quality assurance – acting as a decision-maker and supervisor of the AI results.

Human-in-the-loop and regulation

Full automation of the testing process by AI agents is currently neither technically feasible nor desired from a regulatory perspective. The EU AI Act explicitly requires a “human-in-the-loop” approach for high-risk systems – which generally include banks and insurance companies:

• Humans must review the AI’s results,
• be able to intervene in AI-assisted workflows,
• and be able to correct or halt decisions.

Agent-based testing takes this framework into account: AI agents take on the time-consuming, repetitive tasks, whilst humans retain responsibility and control.

Specific advantages of agent-based testing

Compared to purely human testing, but also to traditional test automation, agent-based testing offers a number of tangible benefits:

• Consistent, high-quality test cases
  AI agents generate well-structured, consistent test cases in popular test management suites in a very short time. Differences in style and quality, which are common when using multiple human testers, are significantly reduced.
• Scalability and 24/7 availability
  In theory, agents work round the clock, without breaks or fatigue. The number of agents working in parallel can be quickly increased as required – ideal for mass testing, tight release windows or extensive regression testing.
• Use of external knowledge sources
  AI agents can analyse large amounts of contextual information from sources such as Confluence, SharePoint or other document repositories in a fraction of a second and incorporate it into test case design – a task that would be extremely time-consuming for humans.
• Resilience to changes
  Unlike traditional automation scripts (Playwright, Selenium, Tosca), which can fail at the slightest change to the application, AI agents respond much more resiliently to UI or process adjustments. This eliminates the need for the time-consuming maintenance of rigid scripts.

Architecture and integration of “adesso test agents”

“adesso test agents” is designed as a flexible framework:

• Cloud-neutral: Can be deployed in various cloud environments (e.g. Google Cloud, Azure, AWS).
• LLM-agnostic: Depending on the complexity of the task, data protection requirements and costs, different LLMs are used – such as models from the Claude series, Gemini or OpenAI.
• Seamless integration: Connection to existing test management tools and workflows, e.g. via Jira/Xray. The agents can learn to work with different tools.

adesso has deliberately opted for its own agent-based testing solution rather than relying on existing products. The aim was to create a solution that meets both its own requirements and the specific needs of customers in regulated industries.

Unlike many agent-based testing tools available to date, “adesso test agents” focuses on high-performance agent-based test execution, which:

• operates entirely without coded scripts,
• simulates real user behaviour using mouse and keyboard controls (“computer-based approach”),
• can be flexibly integrated into a wide variety of test toolchains.

Initial results from real-world projects

‘adesso test agents’ is already being used in initial client projects – for example, at a major German insurer as part of a proof of concept. The results there show that:

• time-consuming test case creation and execution are significantly reduced,
• human testers are relieved of repetitive tasks,
• test coverage increases because significantly more test runs can be carried out in a shorter time.

adesso is also deploying the test agents in internal projects and gathering further experience to continuously optimise the framework.

Conclusion: Agent-based testing as the next stage of evolution

The combination of growing regulatory requirements, complex digital transformation and an increasing shortage of skilled workers is forcing banks and insurers to explore new approaches to quality assurance. Traditional test automation and outsourcing are increasingly reaching their limits.

Agent-based testing using AI agents marks the beginning of a new stage of evolution in software quality assurance:

• repetitive, documentation-intensive tasks are automated,
• human experts focus on technical evaluation and oversight,
• and high test coverage and traceability can be achieved even under strict regulatory requirements.

With “adesso test agents”, banks and insurers already have access to a tried-and-tested framework that addresses precisely these challenges – and paves the way for a more efficient, resilient and future-proof testing organisation.

More information:

adSCAILE