Questions Your Board Should Ask About a Penetration Test
Picture the meeting. The CISO has a penetration test report that runs long and a one-page summary on top. You look at the summary and ask, “So are we okay?” It’s a fair question, and the honest answer depends on things a one-page summary can’t show: what was tested, how anyone knows the findings are real, and whether the serious ones were fixed and checked.
You don’t need to read an exploit to ask about any of that. Six questions will tell you most of what the report can’t. Under each one we’ve written what a good answer tends to sound like, and where it helps, a weak one. A weak answer isn’t an accusation. It’s a gap you can close before the next meeting.
Start with what was left out
Every test covers a defined set of systems. Everything outside that set is untested, and the report won’t call it safe or unsafe. It simply says nothing.
So ask for the scope, then ask what isn’t in it. A good answer might sound like this, for example: “We tested the mobile app, the customer API and the payment module, because that’s where money moves and customer data sits. The old intranet was out, and here’s why.” The scope follows what would hurt most if it failed, and the exclusions have names.
If you hear a list of hostnames with no reason, or “everything important,” ask who decided what counts as important.
Ask how they know a finding is real
A scanner’s list says what might be wrong. A penetration test is different: each finding is something a tester actually did, and the report records the proof.
Here is what that looked like on an open-banking API for a bank. We were asked to test its 27 endpoints. We found a flaw that allowed funds to be diverted, and signed requests that could be replayed. Both were confirmed and the proof was recorded, because a finding we can’t show is real doesn’t go in the report. The bank fixed the critical chain in four days.
That is the difference between a long list of issues and a short list of things that mattered. Ask whoever presents to pick one serious finding and explain it in words you’d use at dinner. If you get a vulnerability identifier and a severity score, ask again.
Who tested, and who do they answer to?
People are poor judges of their own work, which is why PCI DSS asks for some distance between the tester and the thing tested. For payment card environments it calls for a qualified tester who is organizationally independent of the systems and teams being tested. It doesn’t require that person to be a QSA. It also asks for a documented method based on industry-accepted approaches, so the method should exist on paper.
You can ask for the name of the provider, how they sit apart from engineering, and the method they followed. Published methods are easy to name: the OWASP Web Security Testing Guide for web applications, NIST SP 800-115 for security testing in general. Ask also what accreditation the provider holds as a company. That is a question about the firm, and a good provider answers it without pointing to individual certificates.
“Our own security team ran it” isn’t wrong in every case, but it leaves you unable to say whether an auditor would accept the result.
Which serious findings are closed, and who checked?
This is the one we’d most like boards to ask, because the report can’t answer it on the day it arrives. A fix is a claim until someone tests it again.
Four days to fix the critical chain in the open-banking case is a good number, but only if someone then tested the fix. What matters is the order: find, fix, test the fix. You want to know the last step happened, not hear that engineering says so.
For card environments PCI DSS 11.4.4 expects it, so be ready to show it: exploitable weaknesses are corrected, then testing is repeated. For the wider file that sits behind a test, see what penetration testing evidence auditors expect.
“Engineering has closed them” should always prompt a second question. Closed by whom, and shown how?
Know what you chose not to fix
Some findings will stay open. A system may be due for retirement, or a fix may break something customers rely on. That’s a normal business decision, and it only becomes a problem when it’s made quietly.
Ask for the list of accepted risks. Each one needs a reason, a named owner, a date to look at it again and someone with enough authority to accept it for the organization. If you’re a director, that someone may be you. When the reply is a pause and “we’ll get back to you,” the list doesn’t exist yet, and that’s easy to fix before the next meeting.
Ask when it happens again
The last question is about rhythm. The answers you hear will differ, so here is what each one tells you.
| If someone says | What it tells you |
|---|---|
| “ISO 27001 requires an annual test.” | Not quite. ISO/IEC 27001 sets no test type or cadence. Your own risk assessment does, so ask to see the one that produced the schedule. |
| “PCI DSS covers us.” | For card environments, testing is due at least every 12 months and after a significant change. Ask what changed since the last test and whether it triggered one. |
| “We test every year, like the regulator wants.” | That may be true for your sector. Ask which regulator, which requirement, and which of your systems it covers, and ask to see it. |
| “We’ll test before the audit.” | The audit sets the deadline. The risk sets the need. Ask what has changed in the system since the last test. |
Questions readers ask
Should the board read the full penetration test report? Not as a rule. You need a one-page summary and the answers to the questions in this article. The full report is for engineers and auditors, and you should know who keeps it.
Does a clean penetration test result mean the organization is secure? It means the testers did not find a way in within the scope, the time and the rules they were given. Whether that reassures you depends on the scope, which is why the first question is about what was left out.
Take two questions if you only have time for two
Take the second and the fourth: how do we know the findings are real, and which serious ones were fixed and checked. Those two get past the cover page quickly.
Then ask for a summary you could read aloud: what was tested, what was found in business terms, what is fixed and verified, what is accepted and by whom, and when the next test is. The summary, the engineers’ detail and the auditor’s mapping should come from the same pages, so nobody is handed a different version of the story. Our penetration testing page explains how an engagement runs.
Snipeyes is a CREST Member and certified to ISO/IEC 27001:2022, and our retest is included, so the question about fixes has an answer from us. If you’d like to talk through scope with a specialist before your next board cycle, you can book a free 90-minute scoping session.