Case Study - Automated Retesting (Prove the Fix Fixed It)

“Sudah diperbaiki, Pak.” Retest 2 menit kemudian: masih bisa dieksploitasi. Envoy header beda, bug sama.

ILLUSTRATION — RETEST LOOP

🔴 VULN OPEN→ 🔧 FIX CLAIMED→ 🔁 AUTO RETEST→ 🟢 VERIFIED CLOSED

Same exploit, same evidence format — before/after diff archived

Client Context

Insurance + fintech — 90 findings from pentest, audit deadline 30 days, developers claiming fixes in chat. Auditor (Big 4) rejected spreadsheet self-attestation.

Challenge

After remediation, automatically retest to prove vulnerabilities are truly resolved — same technique, fresh evidence, no “trust me” — producing retest evidence auditors accept.

Scope - Snipeyes Automated Retesting

Fix-tied triggers: ticket “done” or commit tag auto-queues identical retest case
Before/after pair: same payload, same tool, same environment tier; screenshot/log diff
Verdicts: Closed / Partially fixed / Reopened (regression) / Compensated (WAF-only flagged honestly)
Output: retest evidence pack per finding + closure certificate summary

Key Findings (redacted)

First retest: only 41% truly closed — 33% partially fixed (WAF block but backend still vulnerable), 26% reopened (fix reverted by next deploy)
Classic: XSS “fixed” with case-sensitive filter (<ScRiPt> bypassed); IDOR “fixed” on web but not on mobile API twin
Environment skew: staging fixed, prod still vulnerable on 7 findings

Outcome

Second cycle: 100% critical/high verified closed with paired evidence — auditor accepted without sampling re-perform
Regression guard: retest cases kept as permanent checks; 2 regressions caught post-audit automatically
Developer behavior changed: fewer premature “done” once retest was automatic

Relevance for you: If audit hinges on “sudah fix” — demand proof, not promises.