APPLIED EVALUATION / OCTOBER 2026
When “Passed”
wasn't proof.
What evidence is enough to call something finished? In one AI-assisted software project, a validation process could report success without actually completing the check. I tightened the standard for accepting the result.
CONTROLLED COMPARISON / SAME FIXTURES
A pass had to
earn its own proof.
Eleven scenarios ran three times per version, producing 66 controlled executions. The comparison measured acceptance decisions & process cleanup, not production customer outcomes.
Correct decisions
9 false approvals · 3 fixture child processes left runningCorrect decisions
0 false approvals · 0 fixture child processes left runningA controlled rerun, not an estimate. 11 scenarios × 3 repetitions × 2 versions = 66 executions.
01 / THE FAILURE
Success was being inferred from silence.
A code-checking wrapper could time out without printing compiler errors & still return a success result. The process hadn't finished, but the check could still get a pass.
Simply increasing the timeout wouldn't fix the faulty decision rule. Acceptance needed positive evidence that the process completed successfully.
Silent timeout
No diagnostics appeared before the time limit expired.
Ignored termination
The timed-out process did not stop when initially requested.
Late success code
A zero exit code arrived only during termination, not completed validation.
02 / THE CORRECTION
Block failure.
Block uncertainty.
Under the revised decision rule, a pass requires completed success. A failed, interrupted or inconclusive check must block acceptance rather than inherit the appearance of success.
The stricter gate subsequently exposed accumulated type errors that needed correction before a later change could be accepted.
03 / A REUSABLE METHOD
Prove the job.
Not just the component.
My contribution was defining product requirements, expected behavior, the QA judgment & acceptance decisions, then directing AI-assisted implementation & fresh verification.
Define the outcome
Who acts, what changes & what should happen when an action fails.
Match evidence to the claim
Use the right kind of test for structure, logic, workflow & completion.
Test failure too
Include interruptions, retries & denied paths. Missing evidence is inconclusive.
Record the decision
Connect expected & actual results to a version, evidence & a next step.
View the test record
In the October 5 controlled comparison, source & test-file hashes were verified. The harness ran process fixtures for success, failure, timeout & termination in separate local trial directories.
- 66 controlled process runs
- 22 unchanged repository regression tests passed
- 10 new request-helper probes passed
- Process cleanup checked before fixture cleanup
These results establish outcomes for this specific controlled evaluation. They do not represent a general production reliability rate.