6

I'm looking at an organization that requires that all employees undergo an annual one-hour online cybersecurity training (watch a video and take a quiz, apparently built using SANS's end-user security awareness training).

Is there any evidence on whether this is effective and how effective it is? For instance, is there any measure or estimate of how much this kind of online cybersecurity awareness training reduces security incidents, or improves outcomes in some other measurable way? Should one expect to see a 5% reduction in security compromises? 50% reduction? 0% reduction?

I did check on SANS's Securing the Human site, figuring that if there was any evidence or quantitative data they would show it prominently, but they don't seem to say anything about it.

Improve this question
4
  • Could you disambiguate "awareness presentation" from "training" when you use the term "awareness training" which looks at the frontier between the two?
    – dan
    Commented Nov 3, 2015 at 22:14
  • Related but not identical: security.stackexchange.com/questions/63430/…
    – Scott C Wilson
    Commented Nov 9, 2015 at 19:40
  • Can we deduce the efficacy of training from the fact that bad guys don't just attach .exe files to spam mail that much anymore? (They use infected Office docs/PDFs or use Exploits to deliver malware.)
    – Scott C Wilson
    Commented Nov 9, 2015 at 19:48
  • @ScottWilson, no, we can't. (An alternative hypothesis is that spam filters have gotten better at detecting those kind of attacks and have made those attacks less effective.)
    – D.W.
    Commented Nov 9, 2015 at 19:50

1 Answer 1

Reset to default
4

The efficacy of a 1-hour annual online series of videos to affect user behaviour is very low (industry stats are 0-5% change in behaviour - perhaps statistically insignificant). Compliance is higher within the days after training, but then trails off very quickly. This type of training needs to be coupled with other supports in order to see results, but it is possible to see positive results up to 70% (consistent adoption of targeted behaviours) with certain supplements to this kind of training. Repetition of training and support and follow-up are perhaps more important than the knowledge transfer itself.

As for supplements, the most effective methods include regular prompting of behaviours and providing immediate feedback to the user as to the correctness of the presented behaviour. The most common form of this is simulated phishing, but it can include any behaviour the organization wishes to see.

In some phishing simulation programmes, studies have shown a decrease in users clicking links in emails of up to 70%. This usually requires regular testing and users slowly learn what to do over time. The key is regularity and immediate feedback.

This same approach can be done for password policies, tailgating, locking computers, incident reporting, USB device handling, etc.

Awareness is awareness. Knowledge transfer is knowledge transfer. But behavioural change is a different ballgame. It starts with knowledge (sometimes), but then it needs to transition to action. And that can't be done with a 5 minute video.

(I am writing a book on this very topic)

Improve this answer
10
  • Agreed. Awareness is defined by NIST Special Publication 800-16 as follows: “Awareness is not training. The purpose of awareness presentations is simply to focus attention on security. Awareness presentations are intended to allow individuals to recognize IT security concerns and respond accordingly." Thus, while awareness is important to disseminate IT security needs of the organisation, the effectiveness of the training is dependant on a lot of other variables as well. sbs.ox.ac.uk/cybersecurity-capacity/system/files/…
    – Joe
    Commented Nov 3, 2015 at 6:27
  • "If you are hoping that awareness will result in lowered security incidents or lowered occurrences of non-compliant behaviour" -- yes, I sure am! Or rather, whether or not it does have that kind of result is exactly the question I'm asking. Training would be pointless if it didn't lead to improvement in security outcomes, wouldn't it? I mean, we don't do security awareness training because we think security awareness is good in its own right, but rather because it is a means to an end: we think it will improve security and help the organization resist attacks more effectively.
    – D.W.
    Commented Nov 3, 2015 at 6:56
  • @Joe, I'm a bit confused by your comment. I wonder if perhaps my use of the word "awareness" has caused some miscommunication? See the Securing the Human web site, where they say "our goal is to not only ensure you are compliant but provide training that changes user behavior and helps your organization manage risk" - in other words, it is explicitly about influencing behavior, and it is training. As far as other variables, I'm fine with an answer that is caveated in that way, but I still wonder if there are any measurements at all.
    – D.W.
    Commented Nov 3, 2015 at 7:00
  • @schroeder, wow, that sounds extremely useful and exactly the sort of thing I was asking about! If you might feel inspired to supplement your answer with that information and some examples or characterization of the supplements, that'd be awesome and exactly what I was wondering about.
    – D.W.
    Commented Nov 3, 2015 at 21:37
  • 1
    @Schroeder Yeah, that's the reality that everyone's start to come up against nowadays. It's funny, but you even get that from pentesters & tool writers who specialize in other penetration vectors. I was watching 2 or 3 conference presentations on YouTube last week about using web app vulnerabilities to get past the perimeter and I'd swear they all started with the same disclaimer: "Now, of course, uh, you know the best way to make your initial breach is usually going to be malware delivered via phishing. But in case you want to do it another way... [begin presentation topic]"
    – mostlyinformed
    Commented Nov 4, 2015 at 1:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .