Exploring the Deep Dream Generator (an Art-Making Generative AI)
•
3 likes•716 views
The Deep Dream Generator was created by Google engineer Alexander Mordvintsev in 2014. It has a public facing instance at https://deepdreamgenerator.com/, which enables people to use text prompts and image prompts (individually or in combination) to inspire the art-generating generative AI to output images. This work highlights some process-based walk-throughs of the tool, some practical uses, some lightweight art learning, some aspects of the online social community on this platform, and other insights. Some works by the AI prompted by the presenter may be seen here: https://deepdreamgenerator.com/u/sjjalinn. (This is the first draft of a slideshow that will be used in a conference later in the year.)
Report
Share
Exploring the Deep Dream Generator (an Art-Making Generative AI)
- 1. Exploring the Deep Dream Generator (an Art-Making Generative AI) Shalin Hai-Jew Kansas State University
- 2. Presentation Overview • The Deep Dream Generator was created by Google engineer Alexander Mordvintsev in 2014. It has a public facing instance at https://deepdreamgenerator.com/, which enables people to use text prompts and image prompts (individually or in combination) to inspire the art-generating generative AI to output images. This work highlights some process-based walk-throughs of the tool, some practical uses, some lightweight art learning, some aspects of the online social community on this platform, and other insights. Some works by the AI prompted by the presenter may be seen here: https://deepdreamgenerator.com/u/sjjalinn. 2
- 3. 3
- 4. 4
- 5. 5
- 6. 6
- 7. About the Deep Dream Generator https://deepdreamgenerator.com/ 7
- 8. Origins • The Deep Dream Generator was created by Alexander Mordvintsev, a Google engineer (“DeepDream,” Mar. 19, 2023) in 2014. The computationally-enabled visual effects may be seen in the prior article. • On the Deep Dream site, more information may also be acquired: https://deepdreamgenerator.com/about. • Some of the code linked to Deep Dream is available on GitHub. 8
- 9. Deep + Dream Deep • Profound • Meaningful • Mysterious • Back-of-the-mind • Hidden • Not shallow • Inaccessible / hard to access • Convoluted Dream • Fantasy • Imaginary, fictional, unreal • Nightmare • Fragmented • Expression of desire • Subliminal • Magical, fantastical • Over-the-top 9
- 10. About Generative AI Models (generally speaking) 1. Curation of underlying data (images, video, audio, text, or some combination) 2. Labeling of data to represent contents 3. Training of a data program using artificial intelligence (AI), such as artificial neural networks 4. The capture of micro-features linked to particular text prompts, styles, artist styles, and others (through so-called “deep learning”) 5. Algorithms for delimiting the types of acceptable prompts and outputs 10
- 11. About Generative AI Models (generally speaking)(cont.) 6. Algorithms for layout, compositing 7. The auto-generation of visuals on a two-dimensional plane (x and y axes) based on the following: • Text to dream = AI-generated image • Deep style: (uploaded or prior-generated) base image + visual style (transfer) • Deep dream: (uploaded or prior-generated) base image + settings (including number of “inception depths” (1 – 25) • Post-spawn editing and on- and off-platform versioning 11
- 12. 12
- 13. 13
- 14. A Deep Dream Generator Niche? 14
- 15. The Deep Dream Generator Niche? • The Deep Dream Generator can take a non-substantive text prompt and weave out a whole scene from it. • It generates people with clarity, but the people it generates seem informed by fashion magazines and heavy air-brushing. • It sometimes has the same issues of generated too many limbs for animals, too many fingers for humans, and so on. • The artists it features (style-wise…for style transfer) are not of equal caliber. They do all have some representation online, though. • The works often have a recognizable look-and-feel. • The tool feels like a purveyor of a particular brand of fantasies: a world where everything is proportional, perfumed, sweetened, and somewhat fluffy. 15
- 16. 16
- 17. 17
- 18. 18
- 19. A Genteel Layer? • As with other generative AIs, this one lacks “implicit knowledge” even as it has been trained on big data. It shows a lack of understanding of physics…and common experiential sense. • The tool applies a genteel layer in its depictions. • The genteel layer may be counteracted with “photorealistic” applied to the visual. • There is also a preference for glamor. • There is a preference to go “epic” rather than “mundane.” • In some cases, stereotypes may emerge…as a result of text and modifier and other such prompts (or prompt combinations). • The gentility can be from the company (its peoples) behind DDG. • I reached out to them to make a suggestion about enabling the users to designate proportionality of, say, two animals in a scene, so that their sizes are more correct…and they actually responded quickly and politely and positively to the idea (which they were already working on). 19
- 20. 20
- 21. 21
- 22. Some Walk-throughs of the Deep Dream Generator 22
- 23. A Free- to Cost- Model 23 • Newbie (20 credits, 3 energy credits per hour and up to 5 GB of memory, confirming email) • Member (20 credits, 3 energy credits per hour of recharging, up to 5 GB of memory, 5 dreams in account in “1+ days”) • Dreamer (35 credits, 5 energy credits per hour of recharging, 7 GB of memory; publishing 15 dreams in account over 7+ days, receive 200 likes) • Deep Dreamer (70 credits, 8 energy credits per hour of recharging, 10 GB of memory; publish 40 dreams in account over 21+ days, receive 1000 likes)
- 24. Energy Usage 24 • The type of processing + the export megapixels • 5 credits: Text 2 Dream, Deep Style (default levels) • 2: Deep Dream (default level) • 10: Text 2 Dream at 1 MP • 32: Deep Style at 2 MP • 80: 5 MP of Deep Style • Seems to align with processing amount
- 25. Monthly Plans 25
- 27. Prompting Images via text, via visuals, via modifiers, and others… 27
- 28. Parameters in the Setup • Can select the type of AI model (Fusion, Artistic, Fantasy, Photoreal, Stable (old); Cyberspace (v2), Quantum (v2), Stable (v2) • Aspect ratio: square, landscape, portrait • Quality • Negative prompt (what not to show) • Face Enhance • Upscale & Enhance 28
- 29. Modifiers in the Setup • Generic (such as material type) • Artists (various potential artists) • Quality (amount of detailing) • Effects • Photography 29
- 30. 30
- 31. “Try it” Feature • Public “dreams” have “Try it” buttons to enable the use of the exact same prompts (but not seeding images) of a work in order to see what the user would acquire visual-wise. • One recent one read: “Retrofuturistic biomechanical clockwork woman- cyborg next by the window in the factory, surrealism, by David Mink, highly detailed sharp focus fantasy intricate oil on canvas very attractive poster imperial colors fantastic view high definition colourful John James Audubon, 8k photorealistic hyperrealistic 4K 3D NightCafé perfect facial features Midjouney 3D highly detailed wombo Perfect closed lips Perfect hyper realistic eyes perfect ears, Nikon D850 intricate 8k 4k very attractive beautiful hyperrealistic ultra detailed hdr” 31
- 32. Transparency • The platform is quite transparent, from its code to its origins. • Each output dream comes with the various (multilingual) text prompts and uploaded images that were used to create it. 32
- 33. 33
- 34. “Random prompt” Feature • There is a “random prompt” feature as well, which enables a user to have the prompt generated by Deep Dream Generator. • One can cycle through the various prompts and select and even edit the random prompt. 34
- 35. 35
- 36. 36
- 37. Bottom-Up Builds • The visuals outputs are not necessarily predictable, given wildcard factors, given complexity. • Sometimes, very vague text prompts are mitigated by the generative AI selecting a topic and filling out a whole surface. • The only way I could get the AI to not fill out the whole surface was to upload a line art piece with a blank transparency channel (alpha channel). • Sparsity seems difficult to enforce. 37
- 38. 38
- 39. Seeding Images • Seeding images seem to sometimes have an outsized influence in the returned visual. Other times, they are almost wholly overwritten by the AI’s “vision”. • I assume image upload and other protections. • Can copyrighted images be uploaded to seed visuals? • Can NSFW images be uploaded to seed visuals? 39
- 40. About Negative Prompts • This, not that… (think about this as a Boolean) • [AUTO_NEGATIVES] Signature, signed, watermark, copyright, logo, text, writing, written script, printed language, pictures of paintings, frames, borders, hands, extra fingers, fingers, extra lips, deformed mouth, more than 2 lips, extra eyes] 40
- 41. About Stories • Deep Dream Generator knows stories. When I put in “Jack and Jill” and “Hansel and Gretel,” it made scenes from the respective stories. • In other words, it is culturally trained. 41
- 42. Flexibility • Deep Dream Generator is very fast. • Its outputs tend to be refined and fairly finished. • Some of its works have been used in published articles. Many appear on the open Web. Some have been submitted to open image-sharing websites. • The works are not visually (or otherwise) watermarked. • There are sometimes faked signatures (scribbles) for paintings. 42
- 43. A Few Notes about Composition Compositional Smarts • Using one or two spaces of focal visual interest • Sometimes “the rule of 3s” Compositional Unsmarts • Some challenges with relative sizing of multiple entities in a scene 43
- 44. Handling the Outputs • Does help to iterate over time • Can take exported digital visual objects into other generative AIs • Can do digital image editing and revision in software tools • Can meld photos and images 44
- 45. 45
- 46. 46
- 47. (Un)Charm Charm • Part of the charm of the tool comes from its AI-ness, and its mediated references to the world that is not fully understood as a set of complex rules. Un-charm • Repeated patterns become un- charming in short-order. 47
- 48. Naïve Art-Making Generative AI Effects • Some of my more memorable visuals include the following: • Bubble gum balls with yolks • A bok choy stir fry, but the whole plant is lying sideways in a fry pan • I found some objects that seem commonplace but did not communicate to the output visual. • I tried for a basic “seesaw” or “teeter totter” but could not get to a usable image even after a number of tries. • I tried for “animal crackers” and didn’t get anything usable either. • (I know that I can try to find a seeding image to serve as an underlying infrastructure, but was still surprised that the text alone couldn’t do it even after multiple iterations.) 48
- 49. 49
- 50. Trying to Confound the Generative AI • I have experimented with trying to confound the generative Ai. • I’ve put in prompts and then ticked all the boxes for all the artists… In this case, the visual seems to have been dominated by the work of one of the sculptors…and the other influences were ignored. • I have uploaded non-representational line art or other visuals and asked the AI to come up with a wholly unrelated image. • Deep Dream Generator avoids some of the obvious potential gaffes. 50
- 51. 51
- 52. 52
- 53. 53
- 54. Going with Open-Source Art or Generative AI Creations? 54
- 55. Considerations about Sourcing Visuals Upstream • Legal liabilities (a showstopper) • Provenance (where did it come from, how has it been edited) • Usage by others (and reputational effects) Downstream • Speed (efficiencies) to achieve a usable digital image • Fit of the image to the local need • Versioning to 3d 4d • Generic vs. recognizable style • Fit with other images in use for the project • Effort for cleanup 55
- 56. About “Self-Generated Visuals” (as Alternatives to Generative AI Art) • Legal, clearly sourced, provenanced • Accessible • Technical (in type, spatial resolution, editability, versioning) • Aesthetics • No prior usage by others • Ability to set rights releases in terms of others’ usage • High local investment to create • Direct costs, media releases from talent 56
- 57. About “Stock Imagery” (as Alternatives to Generative AI Art) • Legal • Accessible • Technical (in type, spatial resolution, editability, versioning) • Aesthetics • Direct costs, strings attached, contractual releases (micropayment to the generator of the visual) • Public reputation and prior usage by others? (not wanting long legacy) • Reverse image search • Effort cost to re-edit • Large numbers of stock, but fit to the local context may be inappropriate 57
- 58. About “Open-Source Images” (as Alternatives to Generative AI Art) • Legal? Unclear provenance (in final image and in contributory layers)? • Accessible • Technical (in type, spatial resolution, editability, versioning) • Aesthetics • Reputation and prior usage by others? (not wanting long legacy) • Reverse image search • Effort cost to re-edit • Other available resources 58
- 59. Going with Generative AI Art: Pros and Cons • Feels human generated psychologically (psychological ownership) even though the heavy lifting is by the machine • Involves a learning curve • Outputs as original visuals • No-cost to low-cost • Efficient creation • Looks derivative • Requires reverting from a refined and finished look back to something more raw (more rough) to be useful • Saves on analog materials (paper, pens, inks, paints, etc.) 59
- 60. Going with Generative AI Art: Crediting • How to keep records • How to prevent additional copies in the world (so no publish, no public access) • Striving for novelty • Defining the visual for the text prompts • Crediting the Generative AI • Notifying learners of the visual’s origins and how to understand the visual information being communicated 60
- 61. 61
- 62. 62
- 63. 63
- 64. 64
- 65. Some Practical Uses of Deep Dream Generator 65
- 66. Image Usage in my Academic Instructional Design Work • From most common to less common • Data visualization • Validation of facts (such as in the use of photos) • Relief from gray text, “eye candy” • Directing attention • Entertainment • Illustration, exemplification (examples), description [of people, history, time periods, places, themes, biology, and others] • Explanation, elaboration (of phenomena, of sequences, of processes) • Advance learning, emphasize important learning ideas 66
- 67. Image Usage in my Academic Instructional Design Work(cont.) • Course, module, learning object • Storytelling • Prompts (assignments) • Event branding, logos • Brainstorming, doodling, • And others… 67
- 68. Some Dead Ends • DDG is not great for mathematical presentations. • It is poor at maps. • It is poor at texts. • (These are challenges separate from my text prompts, which may well be lacking, too.) 68
- 69. So Far… • I have used some of the Deep Dream Generator (DDG) visuals to illustrate slideshows (like this one). • I have used some of the DDG visuals to illustrate articles in the C2C Digital Magazine and one published chapter. • The uses have been illustrative vs. anything representationally factual. 69
- 70. 70
- 71. 71
- 72. 72
- 73. Lightweight Art Learning via Deep Dream Generator 73
- 74. A Designed and Aesthetic World • People use DDG in various ways. • Some people have pages full of visual wit. • Some are edgy. • Some have expressions of risqué visuals. • Some are highly humorous. • Some go to particular art styles (with steampunk very popular). • Some convey socio-political messaging, such as related to various demographic diversities. 74
- 75. Learning about Art Terminology • There is a value in looking at the text prompts and seeing what visual outcomes come from those prompts. • Recently, I have explored terms on the site: • Bodyscape • Patchwork • Biomechanical • Biomorphic • Alien 75
- 76. 76
- 77. 77
- 78. Then, An Accidental Mass Deletion Analog and digital 78
- 79. An Accidental Mass Deletion • So I went into my folders and identified visuals I no longer wanted and did an en masse deletion. • I thought the folder was for seeding images only if I wanted to use these to seed future iterated visuals. • Then, I realized that I had also deleted the various public “dreams” related to the visuals (and the related likes, comments, and such). 79
- 80. My Response • My response was telling. The works were not really from my own direct vision and hard work…and I actually did not sweat the losses. • My focus was (and usually is) on the learning. I am exploring the generative AI…and testing its capabilities against others I’ve tried (to varying degrees)…including CrAIyon, Midjourney, and others… • My takeaways are experiential vs. digital-material. 80
- 81. My Response(cont.) • The visuals seem like the mundane all dressed up in the glowy, in details, in the shiny. • I did salvage a few visuals. • I realized how unoriginal my works were. • For example, I had the AI make a chocolate fountain with berries and then saw then someone else had made a delectable chocolate layered cake with cream filling and berries…that looked even more delectable! 81
- 82. What to Keep Anyway? • What makes an image worth keeping (that cost me so little to make)? • Something that surprises (but the surprises diminish with more exposure to the visuals) • A visual with a relevant idea • Something charming and funny 82
- 83. 83
- 84. 84
- 85. Online Social Community on Deep Dream Generator Platform 85
- 86. Actual Apparent Names or Apparent Handles • The Deep Dream Generator platform seems to have people who use actual names and those who use handles, with more of the latter than the former. 86
- 87. Preferences in Likes and Comments • The community seems to like steampunk, cats, architecture, people in cosplay, people of various races, children, ships, moons, forests, and so on. • The community tends to reward what is “eye-popping,” particularly those with novel visuals. • They tend to like anomalies. • Some vote for prosocial messaging. 87
- 88. 88
- 89. 89
- 90. Unintended Implications of Art- Making Generative AI? 90
- 91. Changing up Preferences? • Acclimating to digital art—with the high reflectance, the artifice— results in manual art made with analog materials seeming drab. • The ease with which generative AIs create usable image references…can be demotivating to those who would create something manually. • There may be risks to downskilling, deskilling, and demotivation in terms of art-making vs. upskilling, skills maintenance, and motivation. • The generative AI saves on human time but to what end? 91
- 92. Some Extant Questions • Do people own their own likenesses anymore, even for the living? • Are there images where there are so few examples in the training set that the AI can somehow contravene others’ copyright? • Can a generative AI be said to have an “imagination”? 92
- 93. Telltale Machine Heart • What are ways to tell if a visual work is made by machine? Or is a “deep fake”? • Look for human “flaws”. • Look for slants in human manipulation of a pen (digital or analog). • Look for selective highlights to emphasize and lowlights to de-emphasize. • Look for irregularities and quirks. • Look for analog art materials and different rates of drying. • Look for the absence of a human mind in the messaging. • Look for the algorithmic compositing. • Look for machine sheen. 93
- 94. Telltale Machine Heart(cont.) • Look at the materiality in the visual. Consider how accessible such materials are and how true to the light they are. • In photos: Look for the physics of real-world multi-directional light vs. artificial light. • Is there locational metadata? Does the metadata jive with an actual location? • Run reverse image searches online (against records of 50+ billion prior profiled images). • Use computational testing methods for differentiating between human- and machine- origins. 94
- 95. Telltale Machine Heart(cont.) • Check for the provenance of a work based on content attribution validation systems. • Apply digital forensics. • Look as aspect ratios. • And more. 95
- 96. Zeitgeist of the Age • Now is the hot moment for art-making generative AIs. • The typical trajectory is a sharp rise in excitement, a realization of the limits of the innovation, and then a sunsetting…as the next new thing arises. • The value of the art-making generative AI is still being explored. 96
- 97. 97
- 98. 98
- 99. 99
- 100. Inferences about the Training Imageset • There are ways to make inferences about the training imageset. • In a recent interview, the bestselling movie director and environmentalist James Cameron suggests an analogy of the generative AI programs (and the underlying training imagesets) as part of a vast Jungian human subconscious (https://www.youtube.com/watch?v=WmOQ16PHgDA), what I think of as massmind. 100
- 101. Some Wants 101
- 102. Some Wants • There is much to appreciate with the Deep Dream Generator. • It would be cool if one could better control the inputs towards a final image and iterate and adjust. • It would be cool if there were 3d in actual 3d file type outputs. • It would be cool if there were 4d in actual 4d file type outputs. • It would be cool if there were polygon-based shapes for augmented reality (AR) animations. • It would be cool if there were polygon-based shapes for AR and 4d (motion, changes over time). 102
- 103. Some Wants (cont.) • It would be cool if there were a wider range of artists, including other hemispheres. 103
- 104. 104
- 105. 105
- 107. 107
- 108. Conclusion and Contact • Dr. Shalin Hai-Jew • ITS • Kansas State University • 785-532-5262 • shalin@ksu.edu • All the visuals in this slideshow were made using the Deep Dream Generator. These have not been “published” elsewhere. • The page for the presenter is https://deepdreamgenerator.com/u/sjjalinn. 108