Munich

27 October 2026

Next-Gen TestingBetween Real Browsers and AI

A prominent violet AI figure behind a real browser, with code and a blue verification checkmark in front

Rainer Hahnekamp Soverius AI

1

Soverius AI — We build production-ready AI systems

Why Browser Tests?

Part 1

Why
Browser Tests?

3

Agenda

01

Why Browser Tests?

02

Testing and AI

4

Testing Strategies

Spotify's original Testing Honeycomb: Integrated, Integration and Implementation Detail

Kent C. Dodds's original Testing Trophy: End to End, Integration, Unit and Static

Original images: Spotify Engineering - Testing of Microservices; Kent C. Dodds - Write Tests

5

Recall the Testing Honeycomb and Testing Trophy. Both emphasize integration testing, in different contexts: Spotify discusses microservices, while Kent C. Dodds discusses JavaScript applications. Their terminology and boundaries are not identical. These are strategic illustrations rather than fixed test ratios. Honeycomb source: https://engineering.atspotify.com/2018/1/testing-of-microservices Original image: https://images.ctfassets.net/p762jor363g1/attachment_ef0d406f3bcfc883f8c1d784879943d3/18c9abd8dccd9223ca4db4d5b3cb6c3a/attachment_ef0d406f3bcfc883f8c1d784879943d3.png Trophy source: Kent C. Dodds, Write tests. Not too many. Mostly integration. https://slides.com/kentcdodds/write-tests/fullscreen Original image: https://s3.amazonaws.com/media-p.slid.es/uploads/55780/images/4579002/testing-trophy.png The original published images replace the earlier redraws at Rainer's request on 26 October 2026.

Angular 21: The Shift Towards Browser Tests

6

Angular 21: The Shift Towards Browser Tests

7

Angular 21: The Shift Towards Browser Tests

8

Why Vitest?

Angular CLI pull request description: Vitest becomes the default unit testing runner for new projects and libraries

Browser Mode is Stable: the Vitest 4 announcement and provider setup

9

Connect Angular CLI choosing Vitest by default with stable Browser Mode in Vitest 4. The three images are fresh browser screenshots captured directly from the original websites on October 27, 2026. They show the Vitest 4 release header, Angular CLI pull request #31578 (merged October 23, 2025), and the Browser Mode stability section of the release article. https://vitest.dev/blog/vitest-4 https://github.com/angular/angular-cli/pull/31578 https://vitest.dev/blog/vitest-4#browser-mode-is-stable Original talk: DevFest Karlsruhe 2025, slide 8. https://speakerdeck.com/rainerhahnekamp/zuruck-in-den-browser-das-comeback-der-frontend-tests Reveal order: start with the title only; first advance shows the Vitest 4 release screenshot, second the Angular CLI pull request, third the stable Browser Mode announcement. Previous reverses each reveal. Audience readability: screenshots fill the slide and overlap in reveal order. Each new screenshot sits above the earlier ones. The Angular PR is framed around its description and main schematic changes; Browser Mode is framed around its heading, explanatory paragraphs and initial import changes. The original screenshot assets are unchanged.

A Little Bit of History

Browser

Emulated Browser

jsdom

Browser

10

Why the Browser?

Real Browser Behavior
- Rendering
- User Events
- Timings (Signals)
- New Technologies (Shadow DOM, Hydration)
Switch to "UI Is the API"
Performant Headless Browsers
Visualization (DX)

11

Browser Mode

Vitest

Common API

Delegates
DOM Interaction

Playwright

WebdriverIO

12

Browser Mode in a Nutshell

Testing Library

Advantages

Less Manual SynchronizationwhenStable() · detectChanges()
User Actions Check Elements
Queries via A11y Selectors

Playwright

Power of E2E

Real Asynchronous Waiting
Actionability Checks and Auto-Wait
Powerful Locators

13

Vite-Powered Build With Vitest

Vite

Build

Solid

Vue

React

Framework-Specific Vite Plugins

Vitest

Execution

14

Angular Build With Vitest

Angular

Build

Vitest

Execution

15

Vite-Powered Build With Vitest

Vite

Build

Analog Angular

Angular

Angular as a Vite Plugin

Vitest

Execution

16

“All-In” Features

Configuration as DocumentedTimeouts, etc.

VS Code Support

Trace Viewer Support

Debugging

Vitest Extensions

17

Coding Time

18

Testing in the Age of AI

Part 2

Testing in
the Age of AI

19

How Do We Make Sure We Have the Right Tests?

Skills

Existing Quality Tests

Include Tests in the Prompt / Plan

Review the Tests

Write Test Titles

20

Models Generate. Tools Verify.

Non-Deterministic

Models

Same Model + Same Prompt

Run 1

Run 2

Run 3

Different Answers Are Possible

Deterministic

Tools

Tests: Behavior
ESLint: Code Rules
TypeScript: Types
Architecture Rules: Dependencies

21

Repeated runs of the same model with the same prompt can produce different answers. The three run cards show that variation; they do not represent different models. Variability depends on sampling, context, model version and serving conditions; this is not a claim that every repeated request always differs. A model-based review can vary as well. Use deterministic tools to evaluate each candidate against criteria that do not depend on which coding model produced it: tests check specified behavior, ESLint applies configured code rules, TypeScript checks types, and architecture tools enforce dependency boundaries. Stable checks provide comparable feedback and help detect regressions when a coding model changes. The criteria remain usable across model changes; future code is not guaranteed to pass. Repeatability assumes the same code, input data, tool versions, configuration and controlled environment. Tests can be flaky if they depend on timing, randomness or external services, and custom lint rules may also depend on uncontrolled inputs. A passing result covers only the behavior and rules encoded in the checks. Developers still own intended behavior, the rules and the acceptance decision. A deterministic check does not prove that the specification is complete or correct. Sources: https://www.anthropic.com/research/demystifying-evals-for-ai-agents https://eslint.org/docs/latest/use/core-concepts/

Three Ways to Run the Same Tests

Approach	Who Initiates?	Main Benefit	Main Tradeoff
Hooks	Coding agent (lifecycle event)	Predictable execution	Redundant runs
Review Agent	Separate reviewer	Focused repair feedback	Coordination and repeated checks
Automatic	The model	It just works	Reliance on the model

Anthropic - When to Fold Subagents Into Your Main Agent Loop ↗

22

Introduce three execution arrangements rather than three kinds of tests. The coding agent initiates hooks through its lifecycle events, and the harness dispatches the configured commands; a review agent has its own role/context; the automatic approach leaves the decision to run tests to the model itself. These arrangements can be combined. Predictable execution assumes a configured command hook fires on the relevant event and runs the required command. A reviewer only runs tests if its workflow/instructions require it. Repeated tests can be appropriate after code changes; redundant runs mean rerunning an equivalent check against unchanged relevant state. Rainer's argument for the automatic approach: newer, capable models can plan and manage their own workflow and decide whether and when to run tests. A single loop retains implementation and failure context and avoids extra handoffs. The tradeoff is reliance on model judgment. The Anthropic article supports trying simpler single-agent orchestration with skills and tools; leaving tests optional is Rainer's application of that argument. Subagents can still help with independent parallel work, context limits or conflicting instructions. Agent count and mandatory test enforcement are separate choices: a single agent can still have required validation gates. Compare quality, latency and cost on the actual tasks. Diane Sarkis, Building agents: When to fold subagents into your main agent loop, October 22, 2026. https://claude.com/resources/articles/building-agents-when-to-fold-subagents-into-your-main-agent-loop

Test Code - The Last Code We Read?

Test Code -
The Last Code
We Read?

23

Charted Coding With Younes Jaaidi

Video frame showing Younes Jaaidi alongside his Charted Coding human and AI participation diagram. — Younes Jaaidi - Original Video at 4:24

Younes Jaaidi - Original Video - June 20, 2025

24

Charted Coding: Human Participation

Younes Jaaidi's Charted Coding diagram. Human and AI participation: design document 80/20; agent rules 80/20; writing failing tests 50/50; making tests pass 5/95; refactoring 50/50. — Younes Jaaidi - Original Video at 4:24

Younes Jaaidi - Charted Coding - June 20, 2025

25

Younes Jaaidi's Charted Coding approach illustrates a different balance of human and AI participation across development activities. His diagram assigns 80% human / 20% AI to the design document and to agent rules and instructions, 50% / 50% to writing failing tests, 5% / 95% to making tests pass, and 50% / 50% to refactoring. These are illustrative participation ratios in his proposed workflow, not empirical measurements or a universal prescription. The graphic labels test creation, not a separate verification stage. Connect it to the preceding correctness-ownership question: humans remain involved in specifying intended behavior and reviewing the tests, while AI can carry out much of the implementation. A passing suite supplies evidence against those expectations; humans still decide whether the expectations capture the requirement. Original video: https://www.youtube.com/watch?v=8z9tUsSoros&t=264s Published June 20, 2025. Screenshot captured from the original video at 4:24 on October 26, 2026. The original screenshot bytes are preserved locally; CSS frames the diagram region without redrawing it.

Mostly Integration Still Applies

Guillermo Rauch - December 10, 2016

26

Guillermo Rauch published this advice on December 10, 2016, almost ten years before this talk. The screenshot comes from Rainer and links to the original tweet. Connect the principle to AI-assisted development: generating tests is cheaper, while review, execution and maintenance still cost effort. Prefer meaningful behavioral coverage across components over a large volume of tests coupled to implementation details. Broader tests refers to their behavioral scope, not longer test functions. This application to AI is the talk's argument. Original tweet: https://x.com/rauchg/status/807626710350839808 Date and wording also preserved at https://kentcdodds.com/blog/write-tests Animation: first show the original tweet. On the next advance, reveal the banner: Today we see the complete opposite with AI. A lot of tests, mainly unit tests. This is Rainer's observation for discussion, not a quantified industry-wide finding. Previous hides the banner; leaving and returning resets it.

Testing Solved!?

Screenshot of Anthropic’s October 22, 2026 article: LLMs are making formal software verification feasible at a much larger scale.

Screenshot of Anthropic’s Figure 2 in its completed state: source code and fuzz test, compiled RISC-V executable, specification, Claude-generated proof and the Rocq proof checker.

anthropic.com/research/formal-verification

27

Use the provocative title as a question. Anthropic and Theorem start with existing fuzz targets and their assertions, mechanically build specifications for compiled RISC-V executables, and have Claude construct proofs checked by the Rocq kernel. Instead of repeatedly sampling inputs looking for a failure, a successful proof rules out the covered failures throughout the stated domain. Existing property-based assertions can be covered when encoded in the verified target. This is not a claim that all testing is solved. The main experiment reports 239 entry points across 123 libraries, inputs below 64 MiB and assumptions including a single-threaded static RISC-V/Linux executable and fixed memory budget. Proofs apply to the verified binary and modeled behavior, not every build, entry point, platform or unspecified business requirement. Re-running that same fuzz target cannot find another covered failure within the proven scope. Tests remain useful as executable specifications; the proof checker supplies deterministic validation of the generated proof. Article: https://www.anthropic.com/research/formal-verification Published October 22, 2026. Screenshots captured October 26, 2026. Left: original article headline and cover. Right: original Figure 2 animation, captured at 44 seconds as a full-resolution frame; no redraw. Figure 2 source: https://cdn.sanity.io/files/4zrzovbb/website/ce66333a831ef2f37e91c2f0ecfdaca27cdfa2c9.mp4 Technical appendix: https://www-cdn.anthropic.com/files/4zrzovbb/website/6867eadf760258fa8b42a504600cd3c036d16358.pdf

Tests Fulfill Many Purposes

Tests Fulfill
Many Purposes.

Documentation and a “Guaranteed Abstraction”
for Knowing That Our Code Works.

28

Thank You!

Questions and Discussion

soverius.ai/feedback/ijs-munich-2026-testing-ai

29

Next-Gen TestingBetween Real Browsers and AI

WhyBrowser Tests?

Why Browser Tests?

Testing and AI

Spotify Testing Honeycomb

Kent C. Dodds' Testing Trophy

Browser

Emulated Browser

Browser

Real Browser Behavior

Switch to "UI Is the API"

Performant Headless Browsers

Visualization (DX)

Advantages

Power of E2E

Coding Time

Testing inthe Age of AI

Skills

Existing Quality Tests

Include Tests in the Prompt / Plan

Review the Tests

Write Test Titles

Models

Tools

Test Code -The Last CodeWe Read?

Tests FulfillMany Purposes.

Thank You!

Why
Browser Tests?

Testing in
the Age of AI

Test Code -
The Last Code
We Read?

Tests Fulfill
Many Purposes.