Automated penetration testing case study: a payroll platform that ships every week
This software company released new code every week but paid for a penetration test (an authorized, controlled attack that finds weaknesses before criminals do) once a year. With a SOC 2 audit eight weeks away, it needed proof that each customer’s data stayed separate in the months between those tests.
DISCOVER
SCAN
EXPLOIT
EVIDENCE
Safe by default: agreed scope, limited speed, staging first, instant stop, full audit log
An audit on the calendar and a gap in testing
The client runs a cloud HR and payroll platform for corporate customers. The platform is multi-tenant, which means every customer (a “tenant”) shares the same system and the software must keep each tenant’s data apart. If one company’s salary and staff data showed up in another company’s account, the client could lose customers and contracts.
The yearly manual test covered one version of the platform, so most of the year’s releases were never tested. A scanner ran regularly, but it could only report known flaws. It could not tell whether a weakness was actually exploitable, and it raised far more alerts than engineers could review.
SOC 2 is an independent audit of how a service company protects customer data. The auditor would want evidence that testing kept up with the releases.
What we set up
We built an automated cycle that finds the parts of the platform a tenant can reach, scans them, tries to exploit what it finds and records the evidence. It runs on every change as well as on a schedule.
Safety came first. Tests used dedicated test tenants, ran at limited speed within an agreed scope, went to staging before production and could be stopped at once. Where possible, proof was read-only. Every result sent to the client came with the request and response, screenshots, logs and a severity score. Anything we had not confirmed stayed in our own triage queue and out of the management report.
The test cases follow the OWASP Web Security Testing Guide and the access-control and session requirements of OWASP ASVS.
The chain the scanner missed
The scanner had reported no critical findings. Our testing confirmed 28 real findings from more than 400 raw alerts, and two of them were serious.
The first was a chain of small flaws. Invitation links were predictable enough to guess, and a guessed link let our test user join another customer’s workspace. Once inside, a flaw in how invoices were fetched (an insecure direct object reference, or IDOR) let us export that company’s staff personal data and salaries. We proved it using two test tenants and kept screenshots of every step.
The second was in password reset. Reset tokens were not tied to a specific user. For 15 minutes after a reset request, an attacker could use one to take over the account.
The scanner missed both because they depend on business logic and on combining steps, which scanners are not built to do.
After the fix
The client rebuilt tenant isolation, made invitation links impossible to guess and tied each reset token to the user who requested it. Our retest confirmed the chain was blocked, and the evidence was archived for the auditor. The auditor accepted it as SOC 2 evidence for the system monitoring controls (CC7).
Releases kept their weekly pace. Tests now run on every deployment and stop a release only for a confirmed critical finding. The annual manual test still happens, but the testers now spend their time on deep business logic while the automated cycle catches regressions.
A company that releases every week but tests once a year has the same blind spot this payroll provider had.