Ai powered images-geneva
•Download as PPTX, PDF•
0 likes•111 views
Doug Sillars discusses using AI and machine learning to simplify image preparation for the web. He describes how object detection can be used for automatic cropping, blurring, object removal, and generating alt text. Sillars also demonstrates training a model to add sunglasses to images with minimal manual editing. Throughout the document, Sillars provides examples of using various libraries and services like ImageAI, TensorFlow, Google Cloud, and Cloudinary to perform these image processing tasks with machine learning.
More Related Content
Similar to Ai powered images-geneva
Similar to Ai powered images-geneva (20)
More from Doug Sillars
More from Doug Sillars (20)
Recently uploaded
Recently uploaded (20)
Ai powered images-geneva
- 1. Using AI to Power Your Images Doug Sillars Geneva August 26, 2019
- 2. Contact Me: @DougSillars Doug.Sillars@gmail.com www.dougsillars.com Doug Sillars Freelance Developer Relations Performance Audits: Web/Native Workshops: Performance/Images/Video/AR/ML http://bit.ly/HighPerformanceAndroidApps
- 3. Images Dominate the Web HTTPArchive mobile websites 04/2019
- 4. Image Prep for the Web
- 5. Image Prep for the Web
- 6. Image Prep for the Web
- 7. Image Prep for the Web
- 8. Image Prep for the Web
- 9. Shortcuts for Image Prep?
- 10. Shortcuts for Image Prep? Ensure You are delivering the optimal image
- 11. Shortcuts: Taking screenshots of Photoshop
- 12. Shortcuts for Image Prep?
- 13. Images on the web named screen*shot 426,000 screenshots live on the web April 2019
- 14. Images on the web named screen*shot 106,000 sites with Screen Shots April 2019
- 15. Images on the web named screen*shot 5M sites tested 106,000 sites with Screen Shots
- 16. Images on the web named screen*shot 5M sites tested 106,000 sites with Screen Shots 2.1% of the web! =
- 17. Image Formats – Screenshots -> JPG https://dougsillars.com/2019/02/10/using-screenshots-in-production/ Sometimes our Shortcuts have lasting effects
- 18. Shortcuts for Image Prep?
- 19. Shortcuts for Image Prep?
- 20. Shortcuts for Image Prep? https://res.cloudinary.com/dougsillars/image/upload/c_crop,h_1000,w_1000,q_auto,f_auto/v1558503538/doug_headshot_alps _tfzado.jpg
- 21. Shortcuts for Image Prep? https://res.cloudinary.com/dougsillars/image/upload/c_crop,h_1000,w_1000,q_auto,f_auto/v1558503538/doug_headshot_alps _tfzado.jpg 120 KB
- 22. Shortcuts for Image Prep?
- 23. Shortcuts for Image Prep? https://res.cloudinary.com/dougsillars/image/upload/c_crop,h_1080,w_1000,q_auto,f_auto/ v1558504239/IMG_20160527_131042692_HDR_c21a3k.jpg
- 24. Shortcuts for Image Prep Manual crop
- 25. Shortcuts for Image Prep Manual crops are not a shortcut
- 26. Shortcuts for Image Prep: Detection of Objects https://res.cloudinary.com/dougsillars/image/upload/g_auto,c_crop,w_1000,h_1000/ v1558504239/IMG_20160527_131042692_HDR_c21a3k.jpg
- 27. Image Cropping in Production: Twitter
- 28. Image Cropping in Production: Twitter https://blog.twitter.com/engineering/en_us/topics/infrastructure/2018/Smart-Auto-Cropping-of-Images.html
- 29. Object Detection Using Machine Learning • ImageAI library • Tensorflow based • Need Image Model Library detector = ObjectDetection() detector.setModelTypeAsRetinaNet() detector.setModelPath( os.path.join(execution_path , "resnet50_coco_best_v2.0.1.h5")) detector.loadModel() detections = detector.detectObjectsFromImage(input_image=os.path.join(execution_path , "berwickontweed.jpg"), output_image_path=os.path.join(execution_path , "resnet.jpg")) resnet=plt.imread('resnet.jpg')
- 30. Object Detection Using Machine Learning
- 31. Object Detection Using Machine Learning
- 32. Object Detection with Machine Learning
- 34. Object Detection • Cropping • Blurring objects
- 35. Object Detection • Cropping • Blurring objects • Object Removal http://bit.ly/bootsphoto
- 36. Object Detection • Cropping • Blurring objects • Object Removal • Alt Text Creation
- 37. Object Detection • Cropping • Blurring objects • Object Removal • Alt Text Creation Alt = “A Person and 2 dogs”
- 38. If the Object is not Modeled…
- 39. If the Object is not Modeled… •person : 99.72 •dog : 88.67 •dog : 96.92
- 40. …Try a new library • https://llama.vision
- 41. …Try a new library • https://llama.vision
- 42. ML ALT text in production
- 43. ML ALT text in production alt="Image may contain: sky, cloud, ocean, outdoor and nature"
- 44. …Or you can train a model
- 46. Google Object Detection: Model Creation
- 47. Google Object Detection: Model Creation
- 48. Google Object Detection: Model Creation
- 49. Google Object Detection: Model Creation
- 50. Train Your Model Precision: How often Model’s predictions were correct. In this case 1/6 is a false positive Recall: How often model found known objects. 1/6 is a missed prediction.
- 51. Deploy Model and Use
- 52. Deploy Model and Use
- 53. Model Results!!
- 54. Model Results!!
- 55. Image Fix 1: Cropping
- 56. Image Fix 1: Cropping
- 57. Image Fix 1: Cropping
- 58. Image Fix 2: Inpainting
- 59. Image Fix 2: Inpainting Create a mask of the area
- 60. Image Fix 2: Inpainting Create a mask of the area
- 61. Image Fix 2: Inpainting
- 62. Image Fix 2: Inpainting
- 63. InPaint: Works well on single color areas
- 64. InPaint: Works Not So well with lots of color
- 65. Tricking InPaint Remove “outlier” colors from the border Use the inside as inpainting area
- 66. Tricking InPaint Convert to Grayscale colors are 0 – 255 (1D) Kmeans to bucket the colors
- 67. Tricking InPaint
- 68. Tricking InPaint 10 most common gray colors found
- 69. Tricking InPaint 10 most common gray colors found Frequency
- 70. Tricking InPaint 1. Loop through Color list
- 71. Tricking InPaint 2. Ignore the White Center
- 72. Tricking InPaint 3. If Color is an outlier (lower than 25th percentile Higher than 75th) Mask and replace
- 73. Tricking InPaint
- 74. Tricking InPaint
- 75. Tricking InPaint
- 79. Manually edit 1,000s of glasses?
- 83. Sunglasses Project 1. Remove Background
- 84. Sunglasses Project 1. Remove Background
- 85. Sunglasses Project 2. Make (slightly) transparent https://res.cloudinary.com/dougsillars/image /upload/e_make_transparent,o_70/v1558760 129/6658.png
- 86. Train a model!
- 87. Want to minimize frame capture
- 88. Train a model!
- 89. Beginning
- 90. Find the arms
- 91. Find the arms Finding an Odd number: Fail
- 92. Modify the border around the arms
- 94. Putting it all together https://res.cloudinary.com/dougsillars/image/upload/v1558855691/IMG_20181207_163533_ibfm7a.jpg
- 95. Putting it all together https://res.cloudinary.com/dougsillars/image/upload/c_crop,g_face,w_1500,h_1500/v1558855691/IMG_2018120 7_163533_ibfm7a.jpg
- 96. Putting it all together https://res.cloudinary.com/dougsillars/image/upload/c_crop,g_face,w_1500,h_1500/v1558855691/IMG_2018120 7_163533_ibfm7a.jpg l_6558_f,o_80,e_make_transparent/
- 97. Putting it all together https://res.cloudinary.com/dougsillars/image/upload/c_crop,g_face,w_1500,h_1500/l_6558_f,o_80,e_make_transparent/ g_adv_eyes,fl_layer_apply/v1558855691/IMG_20181207_163533_ibfm7a.jpg
- 99. Conclusion • Image processing with AI and ML can simplify image prep for the web • Cropping • Blurring • Object removal • Alt text
- 100. Love Building with Video and Images? Media Developer Expert mde-comm@cloudinary.com Become a