Security 13 min read

Three questions hiding inside one

Anyone asking “what kind of test do you do?” is really asking three things at once. First: how much does the tester know beforehand? That is called the box, and it is where black box, grey box and white box come from. Second: how deep does it go? That is the difference between an automated scan, a pen test and a red team, and it mostly sets the price. Third: which disciplines are in scope? Web, integrations, network, source code, mobile apps, AI features. Those three axes are independent. You can run a shallow white box and a deep black box. As long as they get mixed up, quotes talk past each other.

Black box: here is the address, good luck

You hand over the URL and nothing else. No login, no documentation, no code. Exactly what any outsider has. I work out for myself which pages exist, which integrations sit underneath, which ports are open and where the weak spots are. Upside: it is the most realistic picture of an attack from outside. Downside: time goes into finding things, and what you do not find you do not test. Everything behind the login stays invisible. This is a good first measurement or a yearly check on a public site, but it is not the way to buy depth.

Grey box: the address and a pass

You add a test account, the kind your customers have, and where possible a description of the integrations. Now everything behind the login can be tested too: can a user reach someone else's file, stretch their permissions, skip a step in a process, change an amount that should be fixed. This returns by far the most value per hour, because no time is lost searching. And it is no less honest than black box: a real attacker simply signs up. If your application publishes a technical description of its own integrations, it gets picked up automatically and the scan knows which calls exist without guessing. If it does not, the reach is smaller — and that belongs in the quote up front, not in the debrief afterwards.

White box: the vault and the blueprints

You hand over the source code, or the container the application runs in. Then nothing has to be guessed. I read where a database query is assembled, see which passwords or keys were accidentally left behind in the code, and which bundled components are outdated and carry known holes. This finds things that are invisible from outside. The downside: you also see code that never runs, so it has to be prioritised, and the code says nothing about how the production server is configured. That is why white box never stands alone here: it runs on top of testing the live application, not instead of it.

One correction: white hat is not white box

These two get confused constantly. Hat is about the hacker's intent: black hat is criminal, white hat works with permission, grey hat acts uninvited but not maliciously. Box is about what the tester may see beforehand. We are always white hat, and it is precisely as a white hat that we can run black, grey and white box tests. One word is about who you are, the other about what is on the table.

Scan, pen test or red team

A vulnerability scan is a machine ticking off known weaknesses. Fast, broad and cheap, but it also produces false alarms and it does not understand your business logic. A pen test is that scan plus a human who verifies, chains separate findings into an attack route and judges what it means that somebody can reach that data. A red team is something else entirely: targeted, stealthy, running for weeks, including physical access and tricking staff. That last one I do not do, so I do not promise it either. What I deliver is the first two: a broad systematic scan with verification behind it, plus human judgement on what comes out.

What exactly gets tested

The application itself and its integrations, with the classic OWASP categories as the baseline: injection, broken access control, misconfiguration, broken sessions. Integrations separately, because REST, GraphQL and the older variants each have their own failure modes. Modern web applications that assemble everything in the browser are walked through with a real browser rather than a plain text crawler, otherwise you find only a fraction of the surface. Beyond that: the login and the permission model per role, whether anyone can reach another customer's data, the server and its open ports, the settings of the lock on your connection, third-party components with known holes, and AI features in your software — can a user make the assistant ignore its instructions or give up data. Mobile apps for Android and iOS are a separate track, as is a phishing simulation, which requires its own permission.

Three packages, rising with what you hand over

Basic is black box: a full port inventory plus the standard checks on everything findable from outside, with a report and one retest. Meant to establish where you stand. Standard is grey box: everything in Basic, plus authenticated testing behind the login, browser-based discovery for modern web applications, the full check set instead of the standard set, and an evidence appendix with network captures and screenshots. Premium is white box: everything in Standard, plus source code analysis, hunting down leftover keys and passwords, an inventory of bundled components with their known holes, and mapping the findings to standards. In Premium the credential is redacted in the report, so the report itself does not become a new leak.

What goes in the report

Written to be read by someone who is not a hacker. Four things per finding: what it is, where it sits, how serious it is and what it means for your business — so not “configuration issue detected” but what somebody can do with it. Alongside that, per finding, how to reproduce it safely, with screenshots and network captures as evidence, and a retest criterion: the concrete check that later proves it is closed. Up front sits a top five of where to start, because a list of two hundred findings without ranking is not help. If you want it for an audit file, findings are mapped to the requirements they touch under OWASP, ISO 27001, NIS2, PCI-DSS, SOC 2 and the GDPR.

Testing once is a snapshot

Software changes weekly, and every month new vulnerabilities surface in components you have used for years. So last year's report says little. That is why a rhythm can sit underneath: a light scan monthly and a deep one quarterly, with each round compared against the previous one. Then you see not only what is there, but what has appeared since last time and what you actually fixed. Every fix comes with a retest, because without one “closed” is a claim rather than a fact.

What I don't promise

Nobody finds everything, and anyone promising that is selling you something. So the report also states what was not tested and where I stopped: which parts fell outside scope, where the time limit sat and which parts turned out to be unreachable. Automated testing does not replace an experienced specialist spending days on one application; it delivers the systematic half, provably and repeatably. Weeks of quiet intrusion including tricking your staff is not something I do. And “mapped to ISO 27001” means I show which requirement a finding touches — it is not an audit and not a certificate, and those two must never blur together in a quote.

What belongs in the engagement

Four things, always. First, written permission before the first scan, with the scope in it: which domains and addresses, in which time window and who the contact is if something falls over. Without that permission I do not test; that is not a formality but criminal law. Second, explicitly what falls outside the test. Third, a processor agreement if I can reach personal data, which in a test behind the login is almost always the case. Fourth, the agreement that one retest after the fix is included. If you get a quote from anyone that lacks these four, ask about them before you sign.

The short version

Box is about what the tester sees up front: only your address, a test account as well, or the source code as well. Seeing more means testing more, and a real attacker simply signs up — which is why grey box returns the most value per hour. Depth is about who is looking: a scan ticks off, a pen test verifies and interprets. You get a report with the evidence, the reproduction and the retest criterion per finding, plus an honest account of what was not tested. Written permission always comes first.

Want to know what your software shows to the outside world?

Book your free intro call →