SlideShare a Scribd company logo

Threat Modeling In 2021

•Download as PPTX, PDF•
•
Adam Shostack
Adam Shostack

Threat modeling is a way of viewing the world, and so what's changing in threat modeling reflects that. There's a global pandemic. The ways we build software are changing. The threats are evolving, and attacks through systems are growing in importance.

1 of 74
Threat Modeling in 2021 Adam Shostack • adam@shostack.org
About Adam Shostack
What’s changing in the world? What’s Changing in Threat Modeling? 4
What’s changing in the world? What’s Changing in Threat Modeling? 5
Development in a COVID World
Key Drivers of Change • Everyone is scared, stressed and sick Including your customers, prospects and employees Many are grieving for losses, caring for children or elderly • Not “working from home” Sheltering in place during a global catastrophe Getting a work done as we can • Teams are distributed Implicit and informal communication is, at best, changed
Responding to Changes • Development teams need More communication tools Specific frameworks Assurance and reassurance • Security features becoming more important This trend overlaps with COVID
Four Question Framework – still works! 11
Agenda • What are we working on? How are we working on it? The fast moving world of cyber • What can go wrong? Threats evolve! STRIDE Machine Learning [Conflict Modeling] 12
The fast moving world of cyber 13
Everything’s Changing So Fast!…? • Models help us see similarities & understand change • Example: Morris worm (1988) Stack smashing (~1970-now*) Common passwords (epoch – end of days) Mis-configured daemons (1988-200?) 14
Fast Changing World: IoT • More sensors and actuators Look like cars and door-opening dogs • Run Linux like it’s 1999 • Cost: lightbulbs to jet engines • Impact: water sensors to medical devices • New attackers 15
The Ways To Threat Model Are … Evolving and Responding • Many building blocks Tools: MS TM (IDE), Tutamantic (discrete), IriusRisk (enterprise)* Approaches: STRIDE, Kill Chain Deliverables: bugs, backlogs, documents… • Building block frame helps contextualize change 16 * Disclosure: I’m on the advisory board of IriusRisk
Agile 17
Fast Moving World of Development • Threat modeling is no more inherently waterfall than Ruby • Threat modeling in agile, CI/CD • Waterfall vs agile Skills, tasks, frameworks are similar Deliverables and scoping are very different • Benefits of fast cycles Controls, quality to address threats in the backlog 18
Waterfall: “Threat Model Documents” Agile: “Bugs and conversations” System Model • Big complex scope • System diagrams & essays • Gates, dependencies • Scope tiny: this sprint’s change • Big picture as security debt Finding Threats • Brainstorm • STRIDE • Kill Chain • Same, aim at in-sprint code Fixes • Controls • Mitigations • Test cases • Spikes to understand • Sec-focused stories in sprint, backlog, or epic • Sec. acceptance criteria Quality • Test plans • Test automation 19
Starting Threat Modeling When Agile • Start agile: work the features being built Develop skills Demonstrate value Get buy-in: security properties and assurance • Then worry about the security debt “What can go wrong” analysis exposes debt All up dataflows (borrow from GDPR) 20
Dialogue before Discussion Dialogue •Explore ideas and consequences “What if?” “How about” •Prototypes & experiments •Fluid not fixed Discussion •Commit to one idea •Production code •Fixed not fluid Borrowing from John Allspaw (Etsy, kitchensoap.com)
Review Use Discussion Dialog System Models Serve Different Goals 27 Pictures Words Whiteboard Spec Plan of record “Visio” Slack / email Time Photoshop
Different Goals • Different goals, different deliverables Dialogue: whiteboard Inform: fancy documents • Implicit goals generate conflict If you want dialogue, don’t ask team to bring a diagram “Oh, you want a review and sign off, not new choices!” • Implicit goals generate work Who needs a fancy document and why? 28
29
Cloud and Serverless • Cloud provider takes over platform issues Platform-level threats are theirs • Business level threats remain Spoofing an employee of your company to your cloud admin • Threat model your build, deploy pipelines 30
What Can Go Wrong? 31
“What Can Go Wrong” Agenda Details • Supply Chain • STRIDE • Adversarial Machine Learning • Operations: Kill Chain/Threat Genomics/Att&CK • Conflict 32
Supply Chain • You don’t need threat modeling to pay attention to Vulnerabilities Compilation, delivery and installation of updates Trade policy 34
STRIDE • Turned 21 last year! • Still helpful mnemonic Spoofing, Tampering, Repudiation, Info Disclosure, DoS, Elevation of Privilege Wide range of system types New details for various threats 35
Spoofing • Spoofing package names, content • GPS Spoofing • Phone authentication • Markets for selfies • Audio/video spoofing 36
Spoofing Package Names, content • Create a package in a public repository Name matches a private repo Build calls pip install package_name Alex Birsan made $130k in bug bounties, Feb 2021 • Use Unicode RLO & other tricks https://trojansource.codes November 2021, Ross Anderson & team https://threatpost.com/supply-chain-hack-paypal-microsoft-apple/163814/
Spoofing GPS • Now a commercial reality 38 https://www.technologyreview.com/s/614689/ghost-ships-crop-circles-and-soft-gold-a-gps-mystery-in-shanghai/
Spoofing and Phone Authentication • SMS or calls SMS specifically deprecated by US Gov regulators • Phone porting & SIM porting attacks • Scamicry: Callers demand authentication from callee 39
Spoofing Facial Recognition • Markets for Selfies April 2016: MasterCard announces Identity Check (“Pay with a selfie!”) March 2018: Sixgill reports selfies in darkweb fullz • Impersonation tools LED Baseball cap allows impersonation 40
Spoofing Audio • Voice cloning as a service! Startups, open source: CandyVoice, Festvox, Vivotext, Lyrebird… • Formal or background authentication • Google Duplex voice interaction as a service lets you scale BEC 2.0: “This is the CEO, need you to pay …” https://thenextweb.com/security/2019/09/02/fraudsters-deepfake-ceos-voice-to-trick-manager- into-transferring-243000/* Phishing 3.0: “Hi honey, just real quick, what’s the Netflix pw?” 41
Spoofing Video • “Deepfake” video democratizes, improves video fakery • Machine learning to imitate a victim • Create new video • Overlay new faces onto existing • Warning: lots of disturbing examples • https://geminiadvisory.io/deepfakes-id-verification/ 42
Deepfake Example (SFW) 43
Tampering • Physical access “AirBNB attacker” can tamper with each device (Thanks to Roy D’Souza for the evocative term) Cars are accessed by the owner*, their spouses and children, their mechanic *The “owner” is probably a bank Odometers, “black boxes”, OBD • Tapplock vs screwdriver 45
Repudiation 46
Repudiation 2
Information Disclosure • Location DOD Ban • Contact tracing • Other sensors 48 https://www.bellingcat.com/resources/articles/2018/07/08/strava-polar-revealing-homes-soldiers-spies/
Information Disclosure & Location 49 https://electrek.co/2020/08/27/tesla-hack-control-over-entire-fleet/ https://www.bloomberg.com/news/articles/2020-09-01/amazon-drivers- are-hanging-smartphones-in-trees-to-get-more-work
Info Disclose & Fast Moving World of Sensors • Phones drive sensor tech: quality, cost • Sensors in everything that exceed our intuition Barometers measure altitude Accelerometers measure typing Microphones + ultrasound disclose location • Examples: Bus stop signs reflected in pupil Fingerprints in photos Offscreen typing in zoom 50
Denial of service • Classically absorb compute, storage or bandwidth Compute transforms into crypto currency • Battery • Money 51
Denial of service (2)
Elevation of Privilege • Many isolation breaks Spectre/Meltdown EoP from cloud, browser Rowhammer and RAMPage EoP from app We’ll see more, and responses are mostly at the platform • Disentangling device control can be impossible “Depression of Privilege” 53
Threats Evolve: STRIDE Is One Of Many Machine Learning Kill Chains Conflict modeling 55
Kill Chain as Alternative to STRIDE • Kill Chain & variants for operational threat models Especially attack.mitre.org • Unifiedkillchain.com for analysis & comparison Doesn’t yet include threat genomics Date: December 7, 2017 Supervisor: Dr. ir. Pieter Burghouwt Second Reader: Prof. dr. ir. Jan van den Berg Institution: Cyber Security Academy (CSA) Initial Foothold: Compromised System • Reconnaissance • Weaponization • Delivery • Social Engineering • Exploitation • Persistence • Defense Evasion • Command&Control Pivoting Network Propagation: Internal Network • Discovery • Privilege Escalation • Execution • Credential Access • Lateral Movement Access Action on Objectives: Critical Asset Access • Collection • Exfiltration • Target Manipulation • Objectives 56
Adversarial Machine Learning • To violate goals of your ML • To bend your ML to attacker’s goals • (Also, training data) • Machine learning is code Code has bugs More complex code has more bugs 57
Adversarial Machine Learning Resources • Microsoft has released several processes https://docs.microsoft.com/en-us/security/threat-modeling-aiml https://docs.microsoft.com/en-us/security/securing-artificial- intelligence-machine-learning My analysis https://shostack.org/blog/tmt-machine-learning/ • Berryville Institute of Machine Learning https://berryvilleiml.com/results/ My analysis https://shostack.org/blog/tmt-biml-machine-learning- risk-framework/ 58
Conflict & Threat Modeling What goes wrong isn’t just sploits 59
Threat Impacts Beyond Tech • Threat modeling will help you find threats to systems • Can also look for threats through systems • Examples touch on politics Let’s focus on the evocative examples, not the politics
Threat Modeling In 2021
Threat Modeling In 2021
Red Hen on Yelp 63
Attackers Adjust 64
Four Question Frame Works for Conflict What are we working on? A system with social aspects or UGC (user generated content) What can go wrong? Conflict as well as exploit What are we going to do? Intuitive measures often fail, we should catalog & study defenses Did we do a good job? 65
What Goes Wrong: Inter-personal Conflict • Explicitly adapting threat modeling to conflict • Shireen Mitchell & Jon Pincus diversity approach • Amanda Levendowski’s SCULPT (in progress) Safety, comfort, usability, legal, privacy, and transparency Focus on mitigation techniques • Used by nation states! 67
What to do? Obvious Fixes Fail or Exacerbate 69
What to Do? Learn from Success • Nextdoor “private social network for your neighborhood” • Had a problem with racial profiling in posts • A/B tested 6 ways to add detail when post mentions race • Says new forms have “reduced posts containing racial profiling by 75%...” 70
What to do about conflict? • Fixes for conflict are less obvious • Need expertise in human behavior to design • Need a catalog of effective design patterns • Github.com/adamshostack/conflictmodeling 71
Summary: Threats • STRIDE instances evolve • Kill chains have emerged as a useful technique • Conflict looms 73
Key Takeaways • Fundamental skills of threat modeling remain important • Details of what we’re working on, how we work and threats are all changing • Importance of conflict modeling 74
Threat Modeling Resources • Threat Modeling: Designing for Security Wherever fine books are sold • Shostack.org/resources/ • shostack.org/blog 75
Questions? adam@shostack.org • shostack.org
Thank you!
Backup
TM Resources (Automotive) • Safety First for Automated Driving paper (2019) • “UN Regulations on Cybersecurity and Software Updates to pave the way for mass roll out of connected vehicles” (2020) • UL 4600 (2020) • SAE J3061(2016) • Evita Project (2011) 79
What’s changing in the world? What’s Changing in Threat Modeling? 80
What’s changing in the world? What’s Changing in Threat Modeling? 81
What’s changing in the world? What’s Changing in Threat Modeling? 82
What’s changing in the world? What’s Changing in Threat Modeling? 83
What’s changing in the world? What’s Changing in Threat Modeling? 84
What’s changing in the world? What’s Changing in Threat Modeling? 85
What’s changing in the world? What’s Changing in Threat Modeling? 86
What’s changing in the world? What’s Changing in Threat Modeling? 87
88

More Related Content

Threat Modeling In 2021

  • 1. Threat Modeling in 2021 Adam Shostack • adam@shostack.org
  • 3. What’s changing in the world? What’s Changing in Threat Modeling? 4
  • 4. What’s changing in the world? What’s Changing in Threat Modeling? 5
  • 5. Development in a COVID World
  • 6. Key Drivers of Change • Everyone is scared, stressed and sick Including your customers, prospects and employees Many are grieving for losses, caring for children or elderly • Not “working from home” Sheltering in place during a global catastrophe Getting a work done as we can • Teams are distributed Implicit and informal communication is, at best, changed
  • 7. Responding to Changes • Development teams need More communication tools Specific frameworks Assurance and reassurance • Security features becoming more important This trend overlaps with COVID
  • 8. Four Question Framework – still works! 11
  • 9. Agenda • What are we working on? How are we working on it? The fast moving world of cyber • What can go wrong? Threats evolve! STRIDE Machine Learning [Conflict Modeling] 12
  • 10. The fast moving world of cyber 13
  • 11. Everything’s Changing So Fast!…? • Models help us see similarities & understand change • Example: Morris worm (1988) Stack smashing (~1970-now*) Common passwords (epoch – end of days) Mis-configured daemons (1988-200?) 14
  • 12. Fast Changing World: IoT • More sensors and actuators Look like cars and door-opening dogs • Run Linux like it’s 1999 • Cost: lightbulbs to jet engines • Impact: water sensors to medical devices • New attackers 15
  • 13. The Ways To Threat Model Are … Evolving and Responding • Many building blocks Tools: MS TM (IDE), Tutamantic (discrete), IriusRisk (enterprise)* Approaches: STRIDE, Kill Chain Deliverables: bugs, backlogs, documents… • Building block frame helps contextualize change 16 * Disclosure: I’m on the advisory board of IriusRisk
  • 15. Fast Moving World of Development • Threat modeling is no more inherently waterfall than Ruby • Threat modeling in agile, CI/CD • Waterfall vs agile Skills, tasks, frameworks are similar Deliverables and scoping are very different • Benefits of fast cycles Controls, quality to address threats in the backlog 18
  • 16. Waterfall: “Threat Model Documents” Agile: “Bugs and conversations” System Model • Big complex scope • System diagrams & essays • Gates, dependencies • Scope tiny: this sprint’s change • Big picture as security debt Finding Threats • Brainstorm • STRIDE • Kill Chain • Same, aim at in-sprint code Fixes • Controls • Mitigations • Test cases • Spikes to understand • Sec-focused stories in sprint, backlog, or epic • Sec. acceptance criteria Quality • Test plans • Test automation 19
  • 17. Starting Threat Modeling When Agile • Start agile: work the features being built Develop skills Demonstrate value Get buy-in: security properties and assurance • Then worry about the security debt “What can go wrong” analysis exposes debt All up dataflows (borrow from GDPR) 20
  • 18. Dialogue before Discussion Dialogue •Explore ideas and consequences “What if?” “How about” •Prototypes & experiments •Fluid not fixed Discussion •Commit to one idea •Production code •Fixed not fluid Borrowing from John Allspaw (Etsy, kitchensoap.com)
  • 19. Review Use Discussion Dialog System Models Serve Different Goals 27 Pictures Words Whiteboard Spec Plan of record “Visio” Slack / email Time Photoshop
  • 20. Different Goals • Different goals, different deliverables Dialogue: whiteboard Inform: fancy documents • Implicit goals generate conflict If you want dialogue, don’t ask team to bring a diagram “Oh, you want a review and sign off, not new choices!” • Implicit goals generate work Who needs a fancy document and why? 28
  • 21. 29
  • 22. Cloud and Serverless • Cloud provider takes over platform issues Platform-level threats are theirs • Business level threats remain Spoofing an employee of your company to your cloud admin • Threat model your build, deploy pipelines 30
  • 23. What Can Go Wrong? 31
  • 24. “What Can Go Wrong” Agenda Details • Supply Chain • STRIDE • Adversarial Machine Learning • Operations: Kill Chain/Threat Genomics/Att&CK • Conflict 32
  • 25. Supply Chain • You don’t need threat modeling to pay attention to Vulnerabilities Compilation, delivery and installation of updates Trade policy 34
  • 26. STRIDE • Turned 21 last year! • Still helpful mnemonic Spoofing, Tampering, Repudiation, Info Disclosure, DoS, Elevation of Privilege Wide range of system types New details for various threats 35
  • 27. Spoofing • Spoofing package names, content • GPS Spoofing • Phone authentication • Markets for selfies • Audio/video spoofing 36
  • 28. Spoofing Package Names, content • Create a package in a public repository Name matches a private repo Build calls pip install package_name Alex Birsan made $130k in bug bounties, Feb 2021 • Use Unicode RLO & other tricks https://trojansource.codes November 2021, Ross Anderson & team https://threatpost.com/supply-chain-hack-paypal-microsoft-apple/163814/
  • 29. Spoofing GPS • Now a commercial reality 38 https://www.technologyreview.com/s/614689/ghost-ships-crop-circles-and-soft-gold-a-gps-mystery-in-shanghai/
  • 30. Spoofing and Phone Authentication • SMS or calls SMS specifically deprecated by US Gov regulators • Phone porting & SIM porting attacks • Scamicry: Callers demand authentication from callee 39
  • 31. Spoofing Facial Recognition • Markets for Selfies April 2016: MasterCard announces Identity Check (“Pay with a selfie!”) March 2018: Sixgill reports selfies in darkweb fullz • Impersonation tools LED Baseball cap allows impersonation 40
  • 32. Spoofing Audio • Voice cloning as a service! Startups, open source: CandyVoice, Festvox, Vivotext, Lyrebird… • Formal or background authentication • Google Duplex voice interaction as a service lets you scale BEC 2.0: “This is the CEO, need you to pay …” https://thenextweb.com/security/2019/09/02/fraudsters-deepfake-ceos-voice-to-trick-manager- into-transferring-243000/* Phishing 3.0: “Hi honey, just real quick, what’s the Netflix pw?” 41
  • 33. Spoofing Video • “Deepfake” video democratizes, improves video fakery • Machine learning to imitate a victim • Create new video • Overlay new faces onto existing • Warning: lots of disturbing examples • https://geminiadvisory.io/deepfakes-id-verification/ 42
  • 35. Tampering • Physical access “AirBNB attacker” can tamper with each device (Thanks to Roy D’Souza for the evocative term) Cars are accessed by the owner*, their spouses and children, their mechanic *The “owner” is probably a bank Odometers, “black boxes”, OBD • Tapplock vs screwdriver 45
  • 38. Information Disclosure • Location DOD Ban • Contact tracing • Other sensors 48 https://www.bellingcat.com/resources/articles/2018/07/08/strava-polar-revealing-homes-soldiers-spies/
  • 39. Information Disclosure & Location 49 https://electrek.co/2020/08/27/tesla-hack-control-over-entire-fleet/ https://www.bloomberg.com/news/articles/2020-09-01/amazon-drivers- are-hanging-smartphones-in-trees-to-get-more-work
  • 40. Info Disclose & Fast Moving World of Sensors • Phones drive sensor tech: quality, cost • Sensors in everything that exceed our intuition Barometers measure altitude Accelerometers measure typing Microphones + ultrasound disclose location • Examples: Bus stop signs reflected in pupil Fingerprints in photos Offscreen typing in zoom 50
  • 41. Denial of service • Classically absorb compute, storage or bandwidth Compute transforms into crypto currency • Battery • Money 51
  • 43. Elevation of Privilege • Many isolation breaks Spectre/Meltdown EoP from cloud, browser Rowhammer and RAMPage EoP from app We’ll see more, and responses are mostly at the platform • Disentangling device control can be impossible “Depression of Privilege” 53
  • 44. Threats Evolve: STRIDE Is One Of Many Machine Learning Kill Chains Conflict modeling 55
  • 45. Kill Chain as Alternative to STRIDE • Kill Chain & variants for operational threat models Especially attack.mitre.org • Unifiedkillchain.com for analysis & comparison Doesn’t yet include threat genomics Date: December 7, 2017 Supervisor: Dr. ir. Pieter Burghouwt Second Reader: Prof. dr. ir. Jan van den Berg Institution: Cyber Security Academy (CSA) Initial Foothold: Compromised System • Reconnaissance • Weaponization • Delivery • Social Engineering • Exploitation • Persistence • Defense Evasion • Command&Control Pivoting Network Propagation: Internal Network • Discovery • Privilege Escalation • Execution • Credential Access • Lateral Movement Access Action on Objectives: Critical Asset Access • Collection • Exfiltration • Target Manipulation • Objectives 56
  • 46. Adversarial Machine Learning • To violate goals of your ML • To bend your ML to attacker’s goals • (Also, training data) • Machine learning is code Code has bugs More complex code has more bugs 57
  • 47. Adversarial Machine Learning Resources • Microsoft has released several processes https://docs.microsoft.com/en-us/security/threat-modeling-aiml https://docs.microsoft.com/en-us/security/securing-artificial- intelligence-machine-learning My analysis https://shostack.org/blog/tmt-machine-learning/ • Berryville Institute of Machine Learning https://berryvilleiml.com/results/ My analysis https://shostack.org/blog/tmt-biml-machine-learning- risk-framework/ 58
  • 48. Conflict & Threat Modeling What goes wrong isn’t just sploits 59
  • 49. Threat Impacts Beyond Tech • Threat modeling will help you find threats to systems • Can also look for threats through systems • Examples touch on politics Let’s focus on the evocative examples, not the politics
  • 54. Four Question Frame Works for Conflict What are we working on? A system with social aspects or UGC (user generated content) What can go wrong? Conflict as well as exploit What are we going to do? Intuitive measures often fail, we should catalog & study defenses Did we do a good job? 65
  • 55. What Goes Wrong: Inter-personal Conflict • Explicitly adapting threat modeling to conflict • Shireen Mitchell & Jon Pincus diversity approach • Amanda Levendowski’s SCULPT (in progress) Safety, comfort, usability, legal, privacy, and transparency Focus on mitigation techniques • Used by nation states! 67
  • 56. What to do? Obvious Fixes Fail or Exacerbate 69
  • 57. What to Do? Learn from Success • Nextdoor “private social network for your neighborhood” • Had a problem with racial profiling in posts • A/B tested 6 ways to add detail when post mentions race • Says new forms have “reduced posts containing racial profiling by 75%...” 70
  • 58. What to do about conflict? • Fixes for conflict are less obvious • Need expertise in human behavior to design • Need a catalog of effective design patterns • Github.com/adamshostack/conflictmodeling 71
  • 59. Summary: Threats • STRIDE instances evolve • Kill chains have emerged as a useful technique • Conflict looms 73
  • 60. Key Takeaways • Fundamental skills of threat modeling remain important • Details of what we’re working on, how we work and threats are all changing • Importance of conflict modeling 74
  • 61. Threat Modeling Resources • Threat Modeling: Designing for Security Wherever fine books are sold • Shostack.org/resources/ • shostack.org/blog 75
  • 65. TM Resources (Automotive) • Safety First for Automated Driving paper (2019) • “UN Regulations on Cybersecurity and Software Updates to pave the way for mass roll out of connected vehicles” (2020) • UL 4600 (2020) • SAE J3061(2016) • Evita Project (2011) 79
  • 66. What’s changing in the world? What’s Changing in Threat Modeling? 80
  • 67. What’s changing in the world? What’s Changing in Threat Modeling? 81
  • 68. What’s changing in the world? What’s Changing in Threat Modeling? 82
  • 69. What’s changing in the world? What’s Changing in Threat Modeling? 83
  • 70. What’s changing in the world? What’s Changing in Threat Modeling? 84
  • 71. What’s changing in the world? What’s Changing in Threat Modeling? 85
  • 72. What’s changing in the world? What’s Changing in Threat Modeling? 86
  • 73. What’s changing in the world? What’s Changing in Threat Modeling? 87
  • 74. 88

Editor's Notes

  1. 20 years of threat modeling From startups to Microsoft
  2. Kim Yong Chol, former NK military intel chief, FBI has publicly attributed break in to NK https://www.fbi.gov/news/pressrel/press-releases/update-on-sony-investigation
  3. Just joking.
  4. https://www.youtube.com/watch?v=EmBneh0oy7E
  5. * Now if you include IoT which fails to compile with modern defenses.
  6. New attackers are covered in Tampering, EoP and Conflict
  7. Explicitly not “the way to threat model is”
  8. https://twitter.com/evacide/status/878695077085708288
  9. Discussion and dialog, Allspaw’s Kitchen Soap blog
  10. https://www.washingtonpost.com/news/dr-gridlock/wp/2018/07/01/theft-and-destruction-of-dockless-bikes-a-growing-problem/ http://keranews.org/post/dallas-gives-5-bike-share-companies-deadline-tidy https://uk.reuters.com/article/uk-france-paris-bicycles/dockless-bike-share-pioneer-gobee-quits-paris-idUKKCN1G7247
  11. Not taxonomy
  12. https://techcrunch.com/2016/10/04/mastercard-launches-its-selfie-pay-biometric-authentication-app-in-europe/ https://darkwebnews.com/dark-web/selfie-darknet-sale/ https://arxiv.org/pdf/1803.04683.pdf
  13. https://www.theverge.com/tldr/2018/4/17/17247334/ai-fake-news-video-barack-obama-jordan-peele-buzzfeed * There are good questions about this report; https://www.linkedin.com/feed/update/urn:li:activity:6575424961991766016/
  14. https://www.theverge.com/tldr/2018/4/17/17247334/ai-fake-news-video-barack-obama-jordan-peele-buzzfeed
  15. https://www.theverge.com/tldr/2018/4/17/17247334/ai-fake-news-video-barack-obama-jordan-peele-buzzfeed
  16. https://www.theregister.co.uk/2018/06/15/taplock_broken_screwdriver/
  17. http://www.iflscience.com/plants-and-animals/migrating-stork-racks-up-2700-on-researchers-cell-phone-bill/ https://www.npr.org/2021/08/02/1023801277/your-facebook-account-was-hacked-getting-help-may-take-weeks-or-299
  18. https://www.theregister.com/2021/10/15/amazon_textbook_rental/ https://www.npr.org/2021/08/02/1023801277/your-facebook-account-was-hacked-getting-help-may-take-weeks-or-299
  19. https://www.defense.gov/News/Article/Article/1594486/new-policy-prohibits-gps-tracking-in-deployed-settings/ https://www.theguardian.com/world/2018/jan/28/fitness-tracking-app-gives-away-location-of-secret-us-army-bases https://www.bellingcat.com/resources/articles/2018/07/08/strava-polar-revealing-homes-soldiers-spies/
  20. https://www.defense.gov/News/Article/Article/1594486/new-policy-prohibits-gps-tracking-in-deployed-settings/ https://www.theguardian.com/world/2018/jan/28/fitness-tracking-app-gives-away-location-of-secret-us-army-bases https://www.bellingcat.com/resources/articles/2018/07/08/strava-polar-revealing-homes-soldiers-spies/
  21. https://www.newsweek.com/stalker-finds-idol-reflection-pupils-1464373 https://techxplore.com/news/2020-11-hack-reveals-text-contents-viewing.html https://www.merseyside.police.uk/news/merseyside/news/2021/may/liverpool-man-latest-to-be-jailed-as-part-of-national-operation-venetic/
  22. https://www.eetimes.com/document.asp?doc_id=1333308
  23. Stress how usability again becomes a security property, and how hard configuration can be to understand. https://www.nytimes.com/2018/06/23/technology/smart-home-devices-domestic-abuse.html https://threatpost.com/rowhammer-variant-rampage-targets-android-devices-all-over-again/133198/
  24. SSH auth forwarding still rocks by default 
  25. Paul Pols
  26. More complex code, more bugs goes back to the intro to the 1st ed of firewalls & Internet security by Cheswick & Bellovin
  27. Conflict Countries & “Non-state actors” with geopolitical goals Between groups Between people ”non-state actors” like ISIS
  28. Note the technical choices: create an interstitial; review (rather than delay) reviews; explain what a good review is https://www.yelp.com/biz/the-red-hen-lexington-3 https://www.nbcwashington.com/news/local/Wrong-Red-Hen-DC-Restaurant-Getting-Death-Threats-After-Spot-With-Same-Name-Booted-Sarah-Huckabee-Sanders-486500061.html
  29. Note how this person has 3 reviews, is from California, and just happened to eat in Virginia the day after the story broke! Also, she paid attention to the instructions in the interstitial
  30. https://www.facebook.com/FacebookforDevelopers/videos/10155607404583553/ https://www.bloomberg.com/features/2018-government-sponsored-cyber-militia-cookbook/
  31. https://www.levendowski.net/conflict-modeling http://achangeiscoming.net/2017/04/15/transforming-tech-diversity-friendly-software/
  32. Here’s a more structured example. What are some of the ways an harasser could attack somebody? Original: https://docs.google.com/presentation/d/1JB3bTbJvjEypKlPu1JKV20Oz9YlF5zRCl3vLIPdDTrA/edit#slide=id.g2073602466_0_0
  33. https://splinternews.com/how-nextdoor-reduced-racist-posts-by-75-1793861389 https://blog.nextdoor.com/2016/08/24/reducing-racial-profiling-on-nextdoor/