Ten questions to ask a human risk vendor before signing

The conclusion: if the vendor cannot prove behavior change, you are buying activity

Of the ten questions on this list, the one that carries the most weight in the decision is the first: how does the vendor prove that someone who fell for a simulated attack actually changed their behavior. If the answer is a course completion percentage, an attendance certificate, or a click rate that drops month over month, the conversation is about activity, not risk. The other nine questions exist to check whether that first answer holds up under any pressure at all.

It is worth remembering why this purchase deserves rigor. Cisco's 90-5-5 framework, which estimates that around 90 percent of breaches involve a human factor, puts people at the center of the problem. And the category has grown enough to attract offers of every kind: Mordor Intelligence values the security awareness training market at 6.74 billion dollars in 2026 (up from 5.77 billion in 2025) and projects it to reach 14.66 billion by 2031, at a compound annual growth rate close to 16.82 percent, with small and midsize businesses as the fastest growing segment. Translated to the buying table: there is a lot of supply chasing midsize companies, and not all of it measures the same thing.

These are the buyer's questions to the vendor, not the general doubts about the category (we answer those separately in our human risk management FAQ). The goal here is different: to keep the demo from being run by the salesperson. We suggest asking them in this order and writing down the literal answer, because what gets promised on the call rarely survives the contract.

On validation: how they prove someone actually learned

1. How do you prove that a person who failed learned, rather than simply completed the module?

This is the question that organizes all the others. The reassuring answer describes a concrete mechanism: testing that person again weeks later with an attack of the same category and difficulty, but with a different template and a different context. That is a retest, and it is not the same as resending the original email, which only confirms the person remembered one specific message. The worrying answer changes the subject to the content library or the completion rate.

This is not a difficult buyer being difficult. Peer-reviewed evidence (Ho et al., IEEE S&P 2025; Lain et al., IEEE S&P 2022) shows that completing training does not by itself predict a reduction in real failures. What demonstrates change is observing the behavior again under similar conditions. A serious vendor knows that limitation and admits it before you bring it up.

2. What happens to a person's risk score between the failure and the next test?

This reveals whether the system has memory or whether every simulation lives alone. The useful answer explains that a failure leaves the person in an elevated risk state that training does not clear, and that only several passed tests afterward do. The weak answer is that the score recalculates with the latest simulation, because that means someone can fail in March, pass by luck in April, and show up green in the May report.

On personalization: what data they use and where it comes from

3. Is the simulation chosen per person, or is the same one sent to everyone?

Sending one campaign to the whole company creates two problems at once: half the staff spots it immediately because it has nothing to do with their work, and the other half gets used to the style. Ask them to show you live how the system decides what goes to someone in finance versus someone in operations. The answer should mention role, department, prior behavior, and the platforms your company actually uses, not a segmentation by country or tenure.

4. Where does the data behind the lure come from, and how is the email generated?

Two questions in one, and both matter. On the data, a vendor should be able to state precisely what information it collects, from where, and on what legal basis, especially when handling employee personal data in the region. On generation, there are two legitimate approaches: programmatic field substitution over curated templates, which favors consistency and controlled realism, or generation with AI, which favors variety. Neither is wrong on its own. What is a red flag is a vendor who cannot explain which one they use or what controls prevent a simulated email from saying something inappropriate to a real employee.

It is also worth asking about business email compromise (BEC), where the lure impersonates an executive or a known supplier. It is the costliest scenario and the hardest to simulate well, which makes it a useful stress test: if the catalog does not distinguish a generic phishing email from an executive authority pretext, the personalization is more brochure than mechanism.

On credentials: what they monitor and what they do with the finding

5. What sources do you check, how often, and what will you show me in week one?

This question has an advantage: you can verify it fast. Monitoring exposed credentials does not need months of behavioral data to deliver something useful, so a vendor with this capability should be able to show you an initial exposure report almost immediately. If they tell you the value arrives next quarter, that layer either does not exist or is thin.

6. What happens automatically when an employee's credential shows up?

This is the difference between an alert and a response. A platform that only notifies is handing one more task to a security team that is already stretched. The answer you want describes a chain: the person is notified, given the action with a deadline, assigned training if it applies, and the event is logged. Ask what happens if the employee ignores the deadline, because that is where you see whether the system follows up or quietly files the matter away.

On measurement: what metric they report to the board

7. What exact number do you put in front of the board, and what does it mean?

Ask to see a real report, anonymized if needed, not a slide from the sales deck. Then ask the uncomfortable question: does this metric measure that someone did something, or that someone changed. Plenty of dashboards look thorough and only answer the first. The distinction between click rate, report rate, and retest, and the order in which they should be read, is something we develop in what to measure in 2026 and in what order; walking into the demo with that distinction clear changes the entire tone of the meeting.

8. How does the report show a person who failed, got trained, and failed again?

Almost everyone fumbles this one. A good system has the answer ready, because repeat failure is the single most informative event in the whole program: it means the remediation did not work and the risk is still live. If the vendor has to improvise, their reporting was probably designed to show progress, not reality.

On operations: how much time the program consumes each month

9. How many hours a month will this cost my team?

Nobody is going to say "a lot," so break it down: who builds the campaign, who decides what each person receives, who chases the ones who did not complete, who assembles the report for the committee. If the answer to each is "your team, with our tools," the license was the cheap part. A program that depends on someone remembering to push it every month lasts exactly as long as that person's availability.

10. What do you need to get started, and when do I get the first data point?

Onboarding reveals the architecture. An OAuth connection to the corporate mail directory and an inventory of people and roles should be enough to begin; if they ask for an integration project, installed agents, or a three-month steering committee, the program will die before the first retest. And pin down a concrete date for the first deliverable, not a vague promise of partnership.

We close with the question that sums up the exercise, best asked at the end once the salesperson has relaxed: six months from now, when I ask you what changed in my people's behavior, what will you show me? The answer is already contained in the ten questions above. All that is left is to listen to it.

Fensivo was built to answer these questions with mechanisms rather than adjectives. It continuously monitors whether employee credentials appear in breaches and triggers the action with a deadline, sends phishing simulations personalized by role, prior behavior, and the platforms the company actually uses, delivers microlearning within minutes when someone falls, and validates with a retest weeks later that the conduct changed. It is human risk management (HRM) running as a closed loop, focused on companies of 25 to 500 employees and starting through OAuth. You can see how it applies to each scenario in use cases.

If you asked your current vendor the first question on this list today, would they answer with a mechanism or with a course completion percentage?