SlideShare a Scribd company logo

AI and Interactive Narrative

•
•
Mirjam Eladhari
Mirjam Eladhari

An overview on developments in artificial intelligence and discuss their impact on interactive narrative design: State in September 2019.

Related slideshows

Mind reading computer
Mind reading computerMind reading computer
Mind reading computer
 
Artificial Intelligence Overview PowerPoint Presentation Slides
Artificial Intelligence Overview PowerPoint Presentation Slides Artificial Intelligence Overview PowerPoint Presentation Slides
Artificial Intelligence Overview PowerPoint Presentation Slides
 
Mind reading computer
Mind reading computerMind reading computer
Mind reading computer
 
1 of 53
Download to read offline
AI AND INTERACTIVE NARRATIVE Mirjam Palosaari Eladhari, PhD
 Otter Play Independent studio and Södertörn University, Media technology, Game Research Group Interactive Narrative Design Think Tank, Nederlands Film Festival September 29, 2019
Colossuses of Confusion
MIND MODULE INFO
Mind Module
 Spreading Activation Network
AI and Interactive Narrative
AI and Interactive Narrative
A WORLD, SO PERFECTLY DESIGNED, THAT DRAMAS AND STORIES ARE MEANINGFUL FOR ALL PLAYERS A WORLD AFFORDING EMERGENCE OF GREAT DRAMA.
OVERVIEW ➤ AI for Games/Interactive Narrative ➤ Developments, past decade ➤ Tech at our finger tips: ➤ Procedural Content Generation ➤ Machine learning ➤ Opportunities, Challenges and wish lists
ABOUT ME ➤ Developed narrative games and applications ➤ Lead game programmer & story logics 1999-2001 (Liquid Media, Stockholm) ➤ MMORPG Systems and location based games- MindModule (five game prototypes exploring character driven narrative)2002-2011 (Tech Lead at Zero Game Studio, Interactive Institute, then PhD Student Gotland Uni, Tokyo Tech, Georgia Tech, USCS, Teesside Uni) ➤ Co-creative systems, and story making games (suite of games for C2Learn project, using computational creativity tech, Lead designer) ➤ Board game design (minimalist CI and story making games) ➤ Narrative generation (tarot based) ➤ Written papers on the above, and on AI Based Game Design ➤ Teaching Game Design, tech innovation (TOG approach), research method (lecturer Gotland Uni, Malta Uni, now at Södertörn Uni) ➤ Organising workshops on Social Believable Agents, NPC Technologies and INT for past decades, ambulating to different conferences (AIIDE, AAAI, FDG, DiGRA)
ON AI FOR GAMES AND NARRATIVE
“INTELLIGENCE” IN THE EYE OF THE BEHOLDER TURING TEST
ARTIFICIAL INTELLIGENCE -> COMPUTATIONAL INTELLIGENCE -> “SMART” SYSTEMS
AI PERCEPTION VS REALITY HUMAN-LIKE INTELLIGENCE VS. SMART SYSTEMS NO ARTIFICIAL BRAIN, BUT WE HAVE SIRI, ALEXA AND OTHERS, EG. IN CURRENT CARS EXAMPLES: - PATH FINDING - THE ENEMIES IN A VIDEO GAME FOLLOW YOU - MAP GENERATION - MANY MOBILE GAMES CREATE NEW LEVELS ON THE FLY - NPC BEHAVIOR - BRANCHING STORIES - COMPUTERS PLAYING CHESS, GO, STARCRAFT
IN GAMES SMART SYSTEMS, BIG WORLDS Sandboxes, Rulesets that enable play and emergence that doesn’t break the system. SMOKE AND MIRRORS. Simple AI looks smarter than it is when designed well BELIEVABILITY VS REALISM Expressive agents that are believable in *their* fictional worlds
BUT I’M NOT DOING GAMES, FILM(VIDEO) IS DIFFERENT YES, BUT THE LINES BETWEEN GAMES AND FILM/VIDEO IS BLURRING (E.G. GAME ENGINES RE USED FOR REAL- TIME CGI WORKFLOWS)
MAIN TYPES OF AI FOR NARRATIVE NARRATIVE GENERATION From old approach of fully automated authoring process (BRUTUS, tale spin ) to assisted authoring, co-creative systems. CHARACTER AI From state machines more sophisticated NPCs to synthetic humans IVA style. Also Chatbots, twitter bots,
 AUTHORING TOOLS Inform7, Twine, and about 300 more SYSTEM FRAMEWORKS FOR STORY TELLING Story telling reasoning, story logics, ‘drama management’ AI game masters etc. 

DEVELOPMENTS LAST 5 - 10 YEARS
IN GAMES: IMPROVED, MORE SOPHISTICATED METHODS ➤ More sophisticated ways of solving the same problems. 
 Game AI is way past A* Pathfinding and state machines. ➤ But the technology has not changed much. Same problems are tackled but with more sophistication: ➤ deeper searches with for example MTCSes (AI winning against human GO player) ➤ behaviour trees and utility systems rather than state machines. ➤ Planners -> reactive planners and forward chaining. (eg, new plan for agent if situation changes)
NEW DIRECTIONS IN FIELD OF GAME AI ➤ General Game AI: defining a language for AI that can be used across games. (still new, quite promising endeavour) ➤ AI Based games has become a topic of study and experimentation.
AI and Interactive Narrative
PAST 5 - 10 YEARS: DEVELOPMENTS IN FIELD OF GAME AI (2) ➤ The holy grail of the open world system with character driven drama has been modified ➤ Adding systems for drama management -> to give player reader direction and “story beats” in a vast world with backstory that can be explored. ➤ Great games/interactive narratives are built (many mentioned here this evening) but their greatness is most often due of human artistry, skill, sensitivity and hard work. (rather than any groundbreaking AI causing it) ➤ The breakthrough moment for more ambitious interactive narrative might be just around the corner, several projects existed and have been stopped for business reasons (Versu, Story Bricks, Telltale titles etc). The tech is ready.
AI and Interactive Narrative
GAME AI HAS NOT YET UTILISED MUCH OF THE OPPORTUNITIES GIVEN BY MACHINE LEARNING.
THERE ARE REASONS! (for not immediately jumping into using data mining and Machine learning when making games/interactive narratives)
GROUND BREAKING TECHNOLOGIES ALREADY AT OUR FINGERTIPS ➤ Procedural generation
 
 
 
 
 
 
 ➤ Machine Learning
Dwarf fortress (2006) Akalabeth (1980) 2008 2016
GROUND BREAKING TECHNOLOGIES ALREADY AT OUR FINGERTIPS ➤ Procedural generation
 
 PCG is a family of techniques, algorithms and procedures used for generating contents in an automatic way rather than manually.” ➤ Machine Learning
➤ Challenge: Curation of the content. What is “good”? What is useful? GROUND BREAKING TECHNOLOGIES ALREADY AT OUR FINGERTIPS ➤ Procedural generation
 
 PCG is a family of techniques, algorithms and procedures used for generating contents in an automatic way rather than manually.” ➤ Machine Learning
SOLUTIONS FOR PCG CHALLENGE:
 USEFUL AND GOOD CAN BE RATED
AI and Interactive Narrative
AI and Interactive Narrative
SOLUTIONS FOR PCG CHALLENGE: 
 CURATED SEEDS AND PATTERNS FOR WHAT SEEDS THE GENERATION OF CONTENT
http://alumni.media.mit.edu/~hugo/conceptnet/
AI and Interactive Narrative
TAROT BASED NARRATIVE GENERATION
➤ http://www.asdesigned.com/tarot/
AI and Interactive Narrative
AI and Interactive Narrative
TAROT BASED NARRATIVE GENERATION RESOURCES ➤ Application: http://www.asdesigned.com/tarot/ ➤ Paper: Tarot-Based Narrative Generation – Sullivan, A.  Eladhari M.P., and  Cook M, PCG 2018 Workshop, FDG2018, Malmö, Sweden, August 2018. ➤ Code: https://github.com/anneandkita/tarot
➤ Challenge: Curation of the content. What is “good”? GROUND BREAKING TECHNOLOGIES ALREADY AT OUR FINGERTIPS ➤ Procedural generation
 
 PCG is a family of techniques, algorithms and procedures used for generating contents in an automatic way rather than manually.” ➤ Machine Learning
 ML provides systems the ability to automatically learn and improve from experience without being explicitly programmed. ML focuses on the development of computer programs that can access data and use it learn for themselves.
A Style-Based Generator Architecture for Generative Adversarial Networks Tero Karras, Samuli Laine, Timo Aila, 2018 Generative Adversarial Networks
AI and Interactive Narrative
8 YEARS AGO, TWO CLEVERBOTS TALKING (2011) By Igor Labutov, Jason Yosinski, and Hod Lipson of the Cornell Creative Machines Lab (http://creativemachines.cornell.edu/)
MICROSOFT TWITTERBOT TAY PICKED UP RACISM - ONLY UP ONE DAY (2016) Machine Learning - what the machine learns, depends on what knowledge is given to to learn from
➤ Challenge: Curation of the content. What is “good”? 
 
 
 
 ➤ Challenge: Knowledge-base design. What data to give the neural nets to learn from? GROUND BREAKING TECHNOLOGIES ALREADY AT OUR FINGERTIPS ➤ Procedural generation
 
 PCG is a family of techniques, algorithms and procedures used for generating contents in an automatic way rather than manually.” ➤ Machine Learning
 ML provides systems the ability to automatically learn and improve from experience without being explicitly programmed. ML focuses on the development of computer programs that can access data and use it learn for themselves.
REASONS (for not immediately jumping into using data mining and machine learning when making games/interactive narratives) Unpredictability: ➤ Quality of the content ➤ Nature of the content In order for a machine to learn - you give it a data set. The nature of the data set defines what the machine can learn. That works when you take materials that exists in the world. But when you create a world -> you don’t have data until it exits. Q1: What real world data could be used when building worlds? (for example, create richness in backstories?) If Q1, then Q2: How can big sets of data be curated to be useful in narrative systems? (starting from where, and how to scrape the data?)
WHAT’S AROUND THE CORNER ➤ Unity! Build me an 18th century sitting room! ➤ Manipulate video - edit out people and objects (tool out soon) ➤ Deep fake videos - high fidelity speech synthesis + image
WHAT’S AROUND THE CORNER? ➤ Unity! Build me an 18th century sitting room! ➤ Manipulate video - edit out people and objects (tool out soon) ➤ Deep fake videos - high fidelity speech synthesis + image actress Amy Adams in the original (left) is modified to have the face of actor Nicolas Cage (right).
WHAT’S AROUND THE CORNER? ➤ Unity! Build me an 18th century sitting room! ➤ Manipulate video - edit out people and objects (tool out soon) ➤ Deep fake videos - high fidelity speech synthesis + image ➤ The holy grails of the open-world emergent drama - the technology is coming along! actress Amy Adams in the original (left) is modified to have the face of actor Nicolas Cage (right).
LET’S MAKE OUR WISH LISTS!
A DOWN TO EARTH WISH LIST OF A NARRATIVE DESIGNER ➤ Story logics - AI making sure sprawling story lines don’t have dead ends and logical fallacies (correcting my human errors) ➤ Usable interface for creating rich, round, NPCs in the world for players to get to know ➤ Curation of player created content: Allow players to add own content, but not have to curate it manually, or building systems for player community ratings - instead have an AI doing the labour (detecting too provocative entries, curating out repetitive themes etc.) ➤ Q: What more?
A LOFTY WISH LIST OF AN INT/AI RESEARCHER ➤ Using ML and GANs to enable players to create their own symbolic languages - taking dream symbolics into the digital space ➤ Be able to raise the dead: make a true to life representation of long lost relatives - I know their behaviour, I’d author them, and then they would grow in the now - eg my grandmother at age 30 - what dance club would she go to? What would we talk about? (eg Synthetic Human AI + Big Data + Machine Learning) ➤ …What would you put on this list?
FIN http://www.mimmi.net http://www.otter-play.com

More Related Content

AI and Interactive Narrative

  • 1. AI AND INTERACTIVE NARRATIVE Mirjam Palosaari Eladhari, PhD
 Otter Play Independent studio and Södertörn University, Media technology, Game Research Group Interactive Narrative Design Think Tank, Nederlands Film Festival September 29, 2019
  • 7. A WORLD, SO PERFECTLY DESIGNED, THAT DRAMAS AND STORIES ARE MEANINGFUL FOR ALL PLAYERS A WORLD AFFORDING EMERGENCE OF GREAT DRAMA.
  • 8. OVERVIEW ➤ AI for Games/Interactive Narrative ➤ Developments, past decade ➤ Tech at our finger tips: ➤ Procedural Content Generation ➤ Machine learning ➤ Opportunities, Challenges and wish lists
  • 9. ABOUT ME ➤ Developed narrative games and applications ➤ Lead game programmer & story logics 1999-2001 (Liquid Media, Stockholm) ➤ MMORPG Systems and location based games- MindModule (five game prototypes exploring character driven narrative)2002-2011 (Tech Lead at Zero Game Studio, Interactive Institute, then PhD Student Gotland Uni, Tokyo Tech, Georgia Tech, USCS, Teesside Uni) ➤ Co-creative systems, and story making games (suite of games for C2Learn project, using computational creativity tech, Lead designer) ➤ Board game design (minimalist CI and story making games) ➤ Narrative generation (tarot based) ➤ Written papers on the above, and on AI Based Game Design ➤ Teaching Game Design, tech innovation (TOG approach), research method (lecturer Gotland Uni, Malta Uni, now at Södertörn Uni) ➤ Organising workshops on Social Believable Agents, NPC Technologies and INT for past decades, ambulating to different conferences (AIIDE, AAAI, FDG, DiGRA)
  • 10. ON AI FOR GAMES AND NARRATIVE
  • 11. “INTELLIGENCE” IN THE EYE OF THE BEHOLDER TURING TEST
  • 13. AI PERCEPTION VS REALITY HUMAN-LIKE INTELLIGENCE VS. SMART SYSTEMS NO ARTIFICIAL BRAIN, BUT WE HAVE SIRI, ALEXA AND OTHERS, EG. IN CURRENT CARS EXAMPLES: - PATH FINDING - THE ENEMIES IN A VIDEO GAME FOLLOW YOU - MAP GENERATION - MANY MOBILE GAMES CREATE NEW LEVELS ON THE FLY - NPC BEHAVIOR - BRANCHING STORIES - COMPUTERS PLAYING CHESS, GO, STARCRAFT
  • 14. IN GAMES SMART SYSTEMS, BIG WORLDS Sandboxes, Rulesets that enable play and emergence that doesn’t break the system. SMOKE AND MIRRORS. Simple AI looks smarter than it is when designed well BELIEVABILITY VS REALISM Expressive agents that are believable in *their* fictional worlds
  • 15. BUT I’M NOT DOING GAMES, FILM(VIDEO) IS DIFFERENT YES, BUT THE LINES BETWEEN GAMES AND FILM/VIDEO IS BLURRING (E.G. GAME ENGINES RE USED FOR REAL- TIME CGI WORKFLOWS)
  • 16. MAIN TYPES OF AI FOR NARRATIVE NARRATIVE GENERATION From old approach of fully automated authoring process (BRUTUS, tale spin ) to assisted authoring, co-creative systems. CHARACTER AI From state machines more sophisticated NPCs to synthetic humans IVA style. Also Chatbots, twitter bots,
 AUTHORING TOOLS Inform7, Twine, and about 300 more SYSTEM FRAMEWORKS FOR STORY TELLING Story telling reasoning, story logics, ‘drama management’ AI game masters etc. 

  • 18. IN GAMES: IMPROVED, MORE SOPHISTICATED METHODS ➤ More sophisticated ways of solving the same problems. 
 Game AI is way past A* Pathfinding and state machines. ➤ But the technology has not changed much. Same problems are tackled but with more sophistication: ➤ deeper searches with for example MTCSes (AI winning against human GO player) ➤ behaviour trees and utility systems rather than state machines. ➤ Planners -> reactive planners and forward chaining. (eg, new plan for agent if situation changes)
  • 19. NEW DIRECTIONS IN FIELD OF GAME AI ➤ General Game AI: defining a language for AI that can be used across games. (still new, quite promising endeavour) ➤ AI Based games has become a topic of study and experimentation.
  • 21. PAST 5 - 10 YEARS: DEVELOPMENTS IN FIELD OF GAME AI (2) ➤ The holy grail of the open world system with character driven drama has been modified ➤ Adding systems for drama management -> to give player reader direction and “story beats” in a vast world with backstory that can be explored. ➤ Great games/interactive narratives are built (many mentioned here this evening) but their greatness is most often due of human artistry, skill, sensitivity and hard work. (rather than any groundbreaking AI causing it) ➤ The breakthrough moment for more ambitious interactive narrative might be just around the corner, several projects existed and have been stopped for business reasons (Versu, Story Bricks, Telltale titles etc). The tech is ready.
  • 23. GAME AI HAS NOT YET UTILISED MUCH OF THE OPPORTUNITIES GIVEN BY MACHINE LEARNING.
  • 24. THERE ARE REASONS! (for not immediately jumping into using data mining and Machine learning when making games/interactive narratives)
  • 25. GROUND BREAKING TECHNOLOGIES ALREADY AT OUR FINGERTIPS ➤ Procedural generation
 
 
 
 
 
 
 ➤ Machine Learning
  • 27. GROUND BREAKING TECHNOLOGIES ALREADY AT OUR FINGERTIPS ➤ Procedural generation
 
 PCG is a family of techniques, algorithms and procedures used for generating contents in an automatic way rather than manually.” ➤ Machine Learning
  • 28. ➤ Challenge: Curation of the content. What is “good”? What is useful? GROUND BREAKING TECHNOLOGIES ALREADY AT OUR FINGERTIPS ➤ Procedural generation
 
 PCG is a family of techniques, algorithms and procedures used for generating contents in an automatic way rather than manually.” ➤ Machine Learning
  • 29. SOLUTIONS FOR PCG CHALLENGE:
 USEFUL AND GOOD CAN BE RATED
  • 32. SOLUTIONS FOR PCG CHALLENGE: 
 CURATED SEEDS AND PATTERNS FOR WHAT SEEDS THE GENERATION OF CONTENT
  • 39. TAROT BASED NARRATIVE GENERATION RESOURCES ➤ Application: http://www.asdesigned.com/tarot/ ➤ Paper: Tarot-Based Narrative Generation – Sullivan, A.  Eladhari M.P., and  Cook M, PCG 2018 Workshop, FDG2018, Malmö, Sweden, August 2018. ➤ Code: https://github.com/anneandkita/tarot
  • 40. ➤ Challenge: Curation of the content. What is “good”? GROUND BREAKING TECHNOLOGIES ALREADY AT OUR FINGERTIPS ➤ Procedural generation
 
 PCG is a family of techniques, algorithms and procedures used for generating contents in an automatic way rather than manually.” ➤ Machine Learning
 ML provides systems the ability to automatically learn and improve from experience without being explicitly programmed. ML focuses on the development of computer programs that can access data and use it learn for themselves.
  • 41. A Style-Based Generator Architecture for Generative Adversarial Networks Tero Karras, Samuli Laine, Timo Aila, 2018 Generative Adversarial Networks
  • 43. 8 YEARS AGO, TWO CLEVERBOTS TALKING (2011) By Igor Labutov, Jason Yosinski, and Hod Lipson of the Cornell Creative Machines Lab (http://creativemachines.cornell.edu/)
  • 44. MICROSOFT TWITTERBOT TAY PICKED UP RACISM - ONLY UP ONE DAY (2016) Machine Learning - what the machine learns, depends on what knowledge is given to to learn from
  • 45. ➤ Challenge: Curation of the content. What is “good”? 
 
 
 
 ➤ Challenge: Knowledge-base design. What data to give the neural nets to learn from? GROUND BREAKING TECHNOLOGIES ALREADY AT OUR FINGERTIPS ➤ Procedural generation
 
 PCG is a family of techniques, algorithms and procedures used for generating contents in an automatic way rather than manually.” ➤ Machine Learning
 ML provides systems the ability to automatically learn and improve from experience without being explicitly programmed. ML focuses on the development of computer programs that can access data and use it learn for themselves.
  • 46. REASONS (for not immediately jumping into using data mining and machine learning when making games/interactive narratives) Unpredictability: ➤ Quality of the content ➤ Nature of the content In order for a machine to learn - you give it a data set. The nature of the data set defines what the machine can learn. That works when you take materials that exists in the world. But when you create a world -> you don’t have data until it exits. Q1: What real world data could be used when building worlds? (for example, create richness in backstories?) If Q1, then Q2: How can big sets of data be curated to be useful in narrative systems? (starting from where, and how to scrape the data?)
  • 47. WHAT’S AROUND THE CORNER ➤ Unity! Build me an 18th century sitting room! ➤ Manipulate video - edit out people and objects (tool out soon) ➤ Deep fake videos - high fidelity speech synthesis + image
  • 48. WHAT’S AROUND THE CORNER? ➤ Unity! Build me an 18th century sitting room! ➤ Manipulate video - edit out people and objects (tool out soon) ➤ Deep fake videos - high fidelity speech synthesis + image actress Amy Adams in the original (left) is modified to have the face of actor Nicolas Cage (right).
  • 49. WHAT’S AROUND THE CORNER? ➤ Unity! Build me an 18th century sitting room! ➤ Manipulate video - edit out people and objects (tool out soon) ➤ Deep fake videos - high fidelity speech synthesis + image ➤ The holy grails of the open-world emergent drama - the technology is coming along! actress Amy Adams in the original (left) is modified to have the face of actor Nicolas Cage (right).
  • 50. LET’S MAKE OUR WISH LISTS!
  • 51. A DOWN TO EARTH WISH LIST OF A NARRATIVE DESIGNER ➤ Story logics - AI making sure sprawling story lines don’t have dead ends and logical fallacies (correcting my human errors) ➤ Usable interface for creating rich, round, NPCs in the world for players to get to know ➤ Curation of player created content: Allow players to add own content, but not have to curate it manually, or building systems for player community ratings - instead have an AI doing the labour (detecting too provocative entries, curating out repetitive themes etc.) ➤ Q: What more?
  • 52. A LOFTY WISH LIST OF AN INT/AI RESEARCHER ➤ Using ML and GANs to enable players to create their own symbolic languages - taking dream symbolics into the digital space ➤ Be able to raise the dead: make a true to life representation of long lost relatives - I know their behaviour, I’d author them, and then they would grow in the now - eg my grandmother at age 30 - what dance club would she go to? What would we talk about? (eg Synthetic Human AI + Big Data + Machine Learning) ➤ …What would you put on this list?