The ETH Zurich CCS 2024 follow-up pinned whatever effect simulations have to the nudge itself, the periodic reminder that phishing is real, not to the training content. Most employees don't read the training.

Mature vendor platforms automate templating, scheduling, reporting, and even role-based targeting. Marginal cost per campaign is small. Capital outlay falls on the licence.

PCI-DSS, ISO 27001 Annex A, NIST CSF, HIPAA. All reference awareness training and the demonstration of such. A click-rate dashboard answers the question with a number, even if the number doesn't predict anything real.

"Click rate fell from 14% to 6% this year" reads as progress to a non-technical audience. The number is concrete, comparable across quarters, easy to chart. Whether it predicts a real-world outcome is a separate question. Boards rarely ask it.

A live simulation is a recurring touchpoint between the security team and the workforce. Run well, it normalises security comms. Run badly, it builds resentment.

The 2025 large-scale empirical assessment grounded in the NIST Phish Scale, with 12,511 participants, found no statistically significant impact of training modality on either click rates or reporting behaviour across the conditions tested.

The 14,000-employee, 15-month ETH Zurich study at IEEE S&P 2022 reported that embedded training as commonly deployed in industry does not make employees more resilient and, in places, made them more susceptible. This is now corroborated across multiple field studies.

Harder lures produce higher click rates, easier lures lower. No industry-standard methodology exists for calculating click rates. A team that wants a better number can simply pick easier lures next quarter and report improvement.

The UK NCSC has stated in its public guidance for over half a decade that punishment-oriented programmes suppress reporting: users who fear reprisals will not report mistakes promptly, if at all. The behaviour the security team needs most is the one the programme discourages.

A hospital case study found employee workload was the dominant predictor of phishing vulnerability. Staff intended to detect attacks but could not under load. Any programme that frames clicking as a knowledge or competence failure is testing the wrong thing.

Real operators use lures that work against the specific organisation: invoice fraud against finance, CV attachments against HR, internal-looking calendar invites against engineering. Vendor template libraries test against a threat that no longer exists in current operations.

"Sarah clicked again" is not a work order. It identifies no control gap, no configuration drift, no policy correction. The hours spent on the simulation programme are hours not spent on DMARC enforcement, attachment policy, FIDO2 rollout, or conditional access, which would.

For regulated scenarios where the threat model explicitly includes external initial access, end-to-end testing answers a specific question. TIBER-EU's standard variant covers this. The trade-off is that the answer is rarely surprising.

The gateway, sandboxing, and impersonation controls get an end-to-end workout that a configuration review cannot fully replicate. The catch is that this is also achievable with a much narrower mail security assessment for a fraction of the cost.

CBEST and DORA both reference threat-intelligence-led testing that includes initial access scenarios. For financial entities subject to those regimes, full-chain testing is sometimes expected. The expectation is itself softening: TIBER variants now explicitly allow assume-breach.

Every credible threat report from Mandiant, CrowdStrike, and Verizon over the last decade shows that determined adversaries achieve initial access. Spending a quarter of a multi-week engagement re-establishing this point surfaces no actionable new finding. The defensible question is what happens after.

If the SOC catches the phish, which is the entire point of the awareness and email security programme, the red team has nowhere to go. The client paid for a multi-week engagement and got one finding about their mail filter.

Registered domains, SSL certificates, mail relay configuration, payload development, anti-detection tuning. None of it carries to the next engagement. One to two weeks of senior engineer time goes on building the entry vehicle rather than on testing the client's controls.

"We phished a finance contractor" tells you very little about whether the same approach would work against a different role next quarter. "Lateral movement from a low-priv user succeeded via SeImpersonatePrivilege abuse" tells you everything, because the technique applies to every user with that privilege.

Whether a user clicks depends on their workload, fatigue, pretext quality, and the time of day the lure lands. The same engagement run twice produces different results. Assumed-breach starts from the same position every time, which makes year-over-year improvement measurable.

A red team phish lands in the same inboxes that the awareness team is trying to measure separately. The signals contaminate each other. Real phishing reports become indistinguishable from red team simulations, and trend data gets noisy.

In a six-week engagement, post-compromise work might get two to three weeks. The same two to three weeks against the same internal estate, with a foothold provided on day one, is the assumed-breach engagement. The client pays roughly a third more for the privilege of seeing the email security stack tested in the same engagement, which a much cheaper assessment could have covered.