Trained as a software tester (2023). Two years of hands-on AI practice. I test AI agents and document where they cross the line.
Estonian, based in Finland.
Most AI work is judged on output quality. I test what happens on the way there: which source the agent used, what it was allowed to touch, where it stopped, who actually made the decision, and whether a third party can reconstruct the run afterwards.
Six documented control tests against live agents (Claude Code, Codex, NotebookLM), a control framework built from them, and a case study on validating an automated quality gate. The report is in Estonian, the tables and results read across languages.
Three findings from that work:
- An agent refused to write a report on a false premise, then created a file anyway without asking. The output was reasonable. The control boundary broke.
- An agent stopped safely but misread its own permission state, reporting that no approval token existed while one had been granted. Files stayed intact, the state bug did not.
- In a workflow where AI writes and AI judges, the automated judge agreed with the human reviewer on 7 texts out of 20, and approved the single text the human rejected.
| Boundary | The question | Evidence I keep |
|---|---|---|
| Source | Did it use the right, authoritative input? | source list, origin check, contradiction log |
| Permission | Did it hold only the access the task needed? | allowed tools, permission mode, forbidden actions |
| Mutation | Did risk or contradiction stop the action before impact? | hashes, file listing, action trace |
| Decision | Did a human actually decide, independently? | neutral decision packet, exact approval token |
| Evidence | Can a third party reconstruct the run? | prompt, raw output, timestamp, stated gaps |
The last row is the one most reports skip. A finding without a reconstructable trail is an anecdote.
Every run is a folder, not a screenshot: plan.json, run.json, an activity log, evidence files, validation.json and a residual-risks note written before anyone asks for it.
Integrity is decided by a separate validator. It either stamps a run VALID or names the reason it cannot, and a run that fails that check says so in public.
So far: 29 standalone QA runs plus one 49-run controlled experiment across three reasoning-effort levels with repeats. 69 runs carry a VALID stamp.
- LinkedIn: Allan Noor
- Detailing business I run and write for: alandetailing.com