Deepfakes - How they work and what it means for the future
•
6 likes•4,449 views
Deepfakes originally started as cheap costing but believable video effects and have expanded into AI-generated content of every format. This session dove into the state of deepfakes and how the technology highlights an exciting but dangerous future.
Report
Share
More Related Content
What's hot
What's hot (20)
Similar to Deepfakes - How they work and what it means for the future
Similar to Deepfakes - How they work and what it means for the future (20)
More from Jarrod Overson
More from Jarrod Overson (20)
Recently uploaded
Recently uploaded (20)
Deepfakes - How they work and what it means for the future
- 1. Jarrod Overson Director of Engineering at Shape Security How do they work and what does it mean for the future? DEEPFAKES
- 5. The state of deepfakes1 Beyond faceswaps2 How to detect the current generation3 The future of disinformation4 Agenda
- 7. Deepfakes, then: GAN-assisted celebrity faceswaps in adult videos. Deepfakes, now: Believable, AI-assisted content in any format.
- 8. Faceswap The project formed in response to the rise of deepfakes on reddit.
- 9. DeepFaceLab #2 in open source deepfake projects
- 10. FakeApp Once popular, now defunct
- 11. Zao Chinese mobile app that went from release to #1 in two days.
- 12. Step 1 Collect source and destination material Source ● Images with the target's face. Destination ● The video you are looking to modify
- 13. Step 2 Extract faces Histogram of Ordered Gradients ● Reduce every image to greyscale ● Calculate direction of gradient for every pixel
- 14. Step 3 Detect features ● Identify features that anchor a face ● Look for videos and sources with minimally obstructed faces.
- 15. Step 4 Train your model
- 16. Generative Adversarial Networks (GANs) Introduced by Ian J. Goodfellow in 2014 Training Data Output Generative Model Discriminative Model ? ?
- 17. Generative Adversarial Networks (GANs) Let's say we want to make butterflies
- 18. How good did we do? 0% good. Generative Model
- 19. Generative Model How good did we do? 10% good.
- 20. Generative Model How good did we do? 20% good.
- 21. Step 5 Convert
- 22. The state of deepfakes Beyond faceswaps How to detect the current generation The future of disinformation Agenda 1 2 3 4
- 23. Faceswaps are what go viral but machine-generated media comes in many forms.
- 25. Generative Artwork Estimated worth: $7,000 - $10,000. Actual sale price: $432,500
- 27. Generating arbitrary images https://github.com/NVlabs/styleganNone of these people exist.
- 29. Generating images from text • the flower shown has yellow anther red pistil and bright red petals. • this flower has petals that are yellow, white and purple and has dark lines • the petals on this flower are white with a yellow center • this flower has a lot of small round pink petals. • this flower is orange in color, and has petals that are ruffled and rounded. • the flower has yellow petals and the center of it is brown • this flower has petals that are blue and white. • these white flowers have petals that start white in color and end in a white towards the tips. https://github.com/zsdonghao/text-to-image
- 30. Voice recognition and transcription Mozilla's Common Voice https://voice.mozilla.org/en
- 31. Text to speech Beyond voice fonts: TacoTron2 & Waveglow https://devblogs.nvidia.com/generate-natural-sounding-speech-from-text-in-real-time/?linkId=100000007949356
- 33. Marionetting existing video Using facial features from a source to manipulate destination video https://www.youtube.com/watch?v=ohmajJTcpNk
- 34. Marionetting existing video Using facial features from a source to manipulate destination video
- 35. The state of deepfakes Beyond faceswaps How to detect the current generation The future of disinformation Agenda 1 2 3 4
- 36. Resolution differences between suspect portions and rest of video.
- 37. Pay attention to shots where the face is angled or obscured.
- 38. Pay attention to shots where the face is angled or obscured.
- 39. Look at how facial features scale with the rest of the head Ears, jawline, & hairline alignments may change in different scenes.
- 40. Look at how facial features scale with the rest of the head Ears, jawline, & hairline alignments may change in different scenes.
- 41. Look for inconsistent skin tones/shimmering Particularly around the borders of suspect areas.
- 42. This is all temporary though. Detection is an arms race we will lose.
- 43. The state of deepfakes Beyond faceswaps How to detect the current generation The future of disinformation Agenda 1 2 3 4
- 45. How do we identify disinformation? Does it look suspicious? Do we trust the source?
- 48. Fake Article that lasted on Wikipedia for over 1 year.
- 49. 0 million 550 million 1,100 million 1,650 million 2,200 million Q4 17 Q1 18 Q2 18 Q3 18 Q4 18 Q1 19 https://fbnewsroomus.files.wordpress.com/2019/05/cser-data-snapshot-052219-final-hires.png Facebook banned 2.2 Billion fake accounts in Q1 of 2019 alone
- 50. How many accounts were caught automatically?
- 51. How many accounts were caught automatically? 99.8%
- 52. 99.8% of all fake Facebook accounts were caught automatically. 43.8 million accounts were not caught until reported. 1 user is all it takes to spread disinformation. An internet scale problem.
- 56. Not a real quote.
- 57. We already have a "fake" problem.
- 58. What normal traffic looks like
- 59. Traffic at a top US retailer.
- 60. Fake logins were 92% of the traffic.
- 61. Top 3 web retailer 91% fake Top EU airline 79.5% fake Major hotel chain 47% fake
- 63. BADASS is a nonprofit organization dedicated to providing support to victims of revenge porn/image abuse. If you find yourself victim of unauthorized photo sharing, deepfakes or otherwise, contact https://badassarmy.org/
- 64. THANK YOU Jarrod Overson @jsoverson on twitter, github, and medium.