SlideShare a Scribd company logo

Generative AI, WiDS 2023.pptx

•Download as PPTX, PDF•
•
Colleen Farrelly
Colleen Farrelly

This talk overviews my background as a female data scientist, introduces many types of generative AI, discusses potential use cases, highlights the need for representation in generative AI, and showcases a few tools that currently exist.

1 of 16
GENERATIVE AI: AN INTRODUCTION Colleen M. Farrelly
WHO AM I? • Data science lead/advisor • Author of The Shape of Data • AI artist • Miami creative
MY PATH • Told I was dumb and bad at school • Athlete/writer/inventor/ engineer outside of school • Tried to drop out 1st semester of university • Dropped out of MD/PhD program to enter AI • 10+ years in R&D and AI for social good • Possibly PhD student in the future?
WHAT IS GENERATIVE AI? • Set of algorithms that generate: • Images • Text samples • Videos • Audio content • Guided by: • Training sample • User specifications
USE CASE 1: PUBLIC HEALTH CAMPAIGNS South Africa AIDS education example (2006- 2007) COVID public health messaging campaigns
USE CASE 2: CHATBOTS Educational chatbot example Built while working at Jenzabar Automates student support Ethical considerations of chatbots
U SE C ASE 3: BETA TESTIN G C ON TEN T • New cartoon targeting males ages 4-9 who watch a lot of science fiction: • Generate lots of potential content • Choose promising characters/scenes • Do a pilot • Create the show
OTHER USE CASES • Video game character design (Tomb Raider-type example) • Public policy campaign content generation • Blog content • Educational video generation • Virtual therapist apps • Personal assistant bots • Many, many, many more!
HOW DO THESE ALGORITHMS WORK?
GPT • Generative Pre-trained Transformer 3 • Decoder-only transformer network • Gives sequence-to-sequence decoder with long-range memory • Already blurring lines between human composition and AI
DALL-E 2 (OPEN AI) • Capabilities: • Can generate images from text • Can insert new features or styles into that image to modify it • System pieces: • Contrastive Language-Image Pre-training (CLIP) • Prior model (build off existing repository) • Decoder Diffusion model (inverse step)
STABLE DIFFUSION • Text-to-image technology • Based on latent diffusion models coupled to text input • Translation of text cues • Markov chains wandering around a latent space • Denoise and renoise images • Leverages some inpainting techniques to fill in gaps
PROMPT ENGINEERING • New field of engineering unique to generative AI • Leverage • Knowledge of technical architecture • Good combinations of elements to coax system to desired output • Growing need for prompt engineers in industry • Many new career opportunities • Gaming systems • Other content generation
REPRESENTATION • Languages with no or bad models • Lingala • Hausa • Patwa • Cultural contexts missing • Burqa • Subgroups • Lack of worldwide access to some tools • OpenAI and dozens of countries
OTHER RESOURCES • https://www.linkedin.com/in/c olleenmfarrelly/ • https://stablediffusionweb.co m/ • https://openai.com/ • https://hourone.ai/generative- ai-video-des/ • https://midjourney.com/home/ • https://www.lesswrong.com/p osts/fRzkDWewwyS6fHp4w/ palm-api-and-makersuite • https://mathgpt.streamlit.app/
TOOLS THAT SHOULD WORK IN ANYWHERE Speech generation: • https://play.ht/text-to-speech-voices/egyptian- arabic/ Text generation (OpenAI alternative, GPT- 2): • https://huggingface.co/tasks/text-generation Image generation: • https://creator.nightcafe.studio/create Hopefully OpenAI (and Google’s PaLM API): • DALL-E, ChatGPT, GPT-4, Whisper

More Related Content

Generative AI, WiDS 2023.pptx

  • 1. GENERATIVE AI: AN INTRODUCTION Colleen M. Farrelly
  • 2. WHO AM I? • Data science lead/advisor • Author of The Shape of Data • AI artist • Miami creative
  • 3. MY PATH • Told I was dumb and bad at school • Athlete/writer/inventor/ engineer outside of school • Tried to drop out 1st semester of university • Dropped out of MD/PhD program to enter AI • 10+ years in R&D and AI for social good • Possibly PhD student in the future?
  • 4. WHAT IS GENERATIVE AI? • Set of algorithms that generate: • Images • Text samples • Videos • Audio content • Guided by: • Training sample • User specifications
  • 5. USE CASE 1: PUBLIC HEALTH CAMPAIGNS South Africa AIDS education example (2006- 2007) COVID public health messaging campaigns
  • 6. USE CASE 2: CHATBOTS Educational chatbot example Built while working at Jenzabar Automates student support Ethical considerations of chatbots
  • 7. U SE C ASE 3: BETA TESTIN G C ON TEN T • New cartoon targeting males ages 4-9 who watch a lot of science fiction: • Generate lots of potential content • Choose promising characters/scenes • Do a pilot • Create the show
  • 8. OTHER USE CASES • Video game character design (Tomb Raider-type example) • Public policy campaign content generation • Blog content • Educational video generation • Virtual therapist apps • Personal assistant bots • Many, many, many more!
  • 9. HOW DO THESE ALGORITHMS WORK?
  • 10. GPT • Generative Pre-trained Transformer 3 • Decoder-only transformer network • Gives sequence-to-sequence decoder with long-range memory • Already blurring lines between human composition and AI
  • 11. DALL-E 2 (OPEN AI) • Capabilities: • Can generate images from text • Can insert new features or styles into that image to modify it • System pieces: • Contrastive Language-Image Pre-training (CLIP) • Prior model (build off existing repository) • Decoder Diffusion model (inverse step)
  • 12. STABLE DIFFUSION • Text-to-image technology • Based on latent diffusion models coupled to text input • Translation of text cues • Markov chains wandering around a latent space • Denoise and renoise images • Leverages some inpainting techniques to fill in gaps
  • 13. PROMPT ENGINEERING • New field of engineering unique to generative AI • Leverage • Knowledge of technical architecture • Good combinations of elements to coax system to desired output • Growing need for prompt engineers in industry • Many new career opportunities • Gaming systems • Other content generation
  • 14. REPRESENTATION • Languages with no or bad models • Lingala • Hausa • Patwa • Cultural contexts missing • Burqa • Subgroups • Lack of worldwide access to some tools • OpenAI and dozens of countries
  • 15. OTHER RESOURCES • https://www.linkedin.com/in/c olleenmfarrelly/ • https://stablediffusionweb.co m/ • https://openai.com/ • https://hourone.ai/generative- ai-video-des/ • https://midjourney.com/home/ • https://www.lesswrong.com/p osts/fRzkDWewwyS6fHp4w/ palm-api-and-makersuite • https://mathgpt.streamlit.app/
  • 16. TOOLS THAT SHOULD WORK IN ANYWHERE Speech generation: • https://play.ht/text-to-speech-voices/egyptian- arabic/ Text generation (OpenAI alternative, GPT- 2): • https://huggingface.co/tasks/text-generation Image generation: • https://creator.nightcafe.studio/create Hopefully OpenAI (and Google’s PaLM API): • DALL-E, ChatGPT, GPT-4, Whisper