Multisensory and multimedia
•Download as PPTX, PDF•
2 likes•2,503 views
This document summarizes key points from Bill Moggridge's book about designing multisensory and multimedia interactions. It discusses the work of several interaction designers including Hiroshi Ishii who developed tangible user interfaces that blend the physical and digital worlds. Durrell Bishop emphasized making digital objects self-evident like physical objects. Joy Mountford helped develop QuickTime and add video and sound to computers. Bill Gaver studied sound perception and developed auditory icons. The document also describes several prototypes like Ishii's Music Bottles, Bishop's Marble Answering Machine, and Gaver's History Tablecloth that sensor weight over time.
Report
Share
Multisensory and multimedia
- 1. Multisensory and MultimediaBill Moggridge – Designing InteractionsID 501 – Kıvılcım Çınar
- 2. Content:5 senses: Vision, hearing, touch, smell & tasteConnection with computers “sensory deprived and physically limited”explores the opportunities for interaction design to become multisensory and to take advantage of multimedia.Interviews:Hiroshi Ishii (Associate professor of Media Arts and Sciences @ MIT)Durrell Bishop (Teacher of Product Design @ Royal College of Art in London)Joy Mountford (Member of Human Interface Group @ Apple)Bill Gaver (Professor of Interaction Research @ Royal College of Art in London)
- 3. Vision« Visions of Japan » by Arata IsozakiRealms ofCliché(presented various entertainments(such arts as the tea ceremony, flower arrangement and popular and classical performing arts)Kitsch(examined the nature of competitiveness in games and sports)Simulation(demonstrated a dramatic expansion of the potential for visual display )The images produced by these systems are displayed on screensbut they are completely separated from the real things themselves—processed, edited, and otherwise qualitatively changed, often intosomething completely different.“blurring the line between real and simulated”
- 4. Hiroshi IshiiPhysical & Digital world Interaction requires 2 key components:Controls ( manipulate access to digital informations and computations)Representation(people can perceive the results of computations)
- 5. Hiroshi IshiiPhysical & Digital world Interaction requires 2 key components:Controls ( manipulate access to digital informations and computations)Representation(people can perceive the results of computations)Intangible pixels- windows,menus,foldersTo control representations- remote controls,mouseTangible representation (control mechanism) allow user to directly grap and manipulate. Dynamic output or feedback(video, sound)
- 6. Tangible User InterfacesPing Pong Plus(http://www.youtube.com/watch?v=AZO8sfmpKIQ&playnext=1&list=PL38930FA142C5C68F)a ping pong table that could sense ball hits, which it used to control various visualizations projected on the table.Music Bottleshttp://www.youtube.com/watch?v=U4IYyNL4ld8&feature=relatedbottles as containers and controls for digital informationbottles enable each one to be wirelessly identifiedsimple affordance of opening and closing
- 7. Durrell BishopThings should be themselves. (self-evident - to describe whatthey are actually doing)Monopoly versus Video Tape RecorderMonopoly makes the functions of the game self-evident.There is a physical representation of the different elements of the software of the game. For VTR, you might want to know whether it is recording or not, how much space there is left, or where things are.
- 8. Physical object represent meaning..We remember the things,experience them repeatedly Unusual or surprising to our memoryExample: coin (has value – has country, ownership)«...make virtual items, both space andobjects, seem just as real as physical items by designing themtosay more about themselves.»physical object pointer to a window or folder,thephysical environment is just as real as the screen environment
- 9. Marble Answering Machineprototype telephone answering machine. Incoming voice messages are represented by marbles,the user can grasp and then drop to play the message dial the caller automatically. It shows that computing doesn’t have to take place at a desk, but it can be integrated into everyday objectsThe Marble Answering Machine demonstrates the great potential of making digital information graspable.http://ccdc11.wordpress.com/2011/02/15/durrell-bishops-marble-answering-machine/He suggests that physical tagscan be designed to represent any abstract item, as long as we can remember what they mean by recognizing their form and behavior.The marbles are chosen tophysical represent incoming phone messages which demonstrated the great potential of making digital information.
- 10. Joy MountfordQuick TimeMountford & Mike Mills developed designs to include images and videos in graphical user interfaces, integrating QuickTime and moving the personal computer towards multimedia.Personal computers started to grow.Bringing video to personal computer turn the PC into multimedia device.Paper animation techniques was replaced by Quick Time.
- 11. People started to be familiar with film and video editing tools and techniques.( edit clips of their children)Hand controllerNavigable movies (became QuickTime VR ) – Golden Gate Bridgehad a video camera pointing at an object, seemingly doing thesame thing day after day.You could move around the camera’s viewof the room just like you could manipulate a 3-D view of an object with the virtual sphere.http://www.panoramas.dk/US/golden-gate.html
- 12. The Bead Boxsound should be used morein interface design, integrating music, voice, and more sophisticated sound effects.70 percent of the population think they’re amateurmusicians, which is pretty high.age – type of music listenedGives opportunities to people who want to make music but have notmastered a conventional instrumentnotation or the meaning of a score.
- 13. Bill Gaverstudied sound perceptionhis work is about auditory icons meet Joy Mountford – started to work as an intern at Appledeveloped SonicFinder – tried to add sound to the software of Finderwhen you clicked on a file, you would hear a tapping noise
- 14. History TableclothThe History Tablecloth uses weight sensing, measuring how longthings have been left on the surface.Provide us to see history of objects in the homeload sensors calculate product’s position, and cells are lit to form a halo that grows slowly over a period of hours.
- 15. Key Table and Picture Frameprovide a place for people to leave the things they carry with them when out of the house—keys, mobile phonecontains a load sensor mounted under the top surfaceto measure the duration of oscillation caused by a new load, a good indication of impact force.When objects are put on the Key Table, this measurement is wirelessly transmitted to the picture frame, which tilts accordingly.
Editor's Notes
- The entire floor and walls of a high-ceilinged516 | Chapter 8Room 3, Dreams—”The Realm of Simulation” •Arata Isozaki •rectangular hall were completely covered with moving images.Some of the time they were separated into individual imagesmaking a collage of screens, and some of the time they were hugepanoramas, stitched together to form a single image.“blurring the line between real and simulated”was demonstrated very convincingly by the exhibition, but theaudience were passive receivers of the experience, like cinemagoers;there were no interactive enhancements to their lives.
- He tries to add the dimensions of physical feeling to the interface.
- He tries to add the dimensions of physical feeling to the interface.
- The opening and closing of a bottle is also detected. When a bottle is placed onto the stage area of the table and the cork is removed, the corresponding instrument becomes audible. A pattern of colored light is rear-projected onto the table's translucent surface to reflect changes in pitch and volume. The interface allows users to structure the experience of the musical composition by physically manipulating the different sound tracks
- Durrell Bishop wants to design objects that are self-evident,whether they are physical or virtual. He looks at electronics andnotices that the shape of the objects does little to describe whatthey are actually doing; it is hard to see the difference between aradio and calculator by their design.
- It has value; yes,it’s written on it what the value is, and its size and shape denotethat to some extent, but the real idea of value is not a physicalthing. It also has country; it belongs to a nation. But most of all,and strangest of all, it has ownership; the ownership isn’t representedon it at all, but we totally accept it, and it’s mostly defined by thedistance it is from you. If I’m holding it, it’s mine; if it’s in mypocket, it’s mine. If I place it far enough away, we start questioningwhose it is, but we don’t think about this as a pointer to money; it’snot a tag for money—it is money.
- He started his ownproject without explaining to her what he was trying to do. Hehad a video camera pointing at an object, seemingly doing thesame thing day after day.
- What really intrigued mewas how much variation of the audio or music mix you could actuallycreate, even with the simplest samples and changes. It was reallyquite addictive, and it became almost an art to see how wonderfulyour music could be with just three beads.
- When objects are left on the table, load sensors under the table legs are used to calculate their position, and cells are lit to form a halo that grows slowly over a period of hours. When objects are removed, the corresponding halo is designed to disappear over a period of about 30 sec. The History Tablecloth makes visible the movement (or lack thereof) of objects in the home but leaves interpretation of that situation to the people who encounter the tablecloth.