SlideShare a Scribd company logo

From context aware to socially awareness computing - IEEE Pervasive Computing, 11 (1), 32–41. (2012

•Download as PPTX, PDF•
•
F
Fread Mzee

Paul Lukowicz, DFKI Kaiserslautern, Germany Alex Pentland, MIT Mdeia Lab Alois Ferscha, Johannes Kepler Univ. of Linz, Austria

Related slideshows

Bridging Sensor Data Streams and Human Knowledge
Bridging Sensor Data Streams and Human KnowledgeBridging Sensor Data Streams and Human Knowledge
Bridging Sensor Data Streams and Human Knowledge
 
RoutineMaker: Towards End-User Automation of Daily Routines Using Smartphones
RoutineMaker: Towards End-User Automation of Daily Routines Using SmartphonesRoutineMaker: Towards End-User Automation of Daily Routines Using Smartphones
RoutineMaker: Towards End-User Automation of Daily Routines Using Smartphones
 
Context Awareness in Mobile Computing
Context Awareness in Mobile ComputingContext Awareness in Mobile Computing
Context Awareness in Mobile Computing
 
1 of 17
From Context Awareness to Socially Aware Computing Paul Lukowicz DFKI Kaiserslautern, Germany Fredrick Awuor Alex Pentland MIT Mdeia Lab Alois Ferscha Johannes Kepler Univ. of Linz, Austria IEEE Pervasive Computing, 11 (1), 32–41. (2012)
Outline  Introduction  Beyond locomotion and location  Challenges  Towards Solution  Realizing opportunistic recognitions  Socially Aware Computing  Sensing Social Interactions  Sensing Social Phenomena  Discussion  Comment 31-Mar-15 2
Introduction  Context Awareness – Notion of systems adapting their functionality to user’s activities and situation in the environment  Marc Weiser’s dream: mobile phones featuring acceleration and light sensors and offering ability to recognize simple context such as device position (on the table or in the pocket), mode of location, and user location – and adapt their functionality accordingly  Key directions that are driving recognition research:  Moving beyond locomotion and location to incorporate complex activity recognition into real-world applications  Realizing opportunistic recognitions; and  Moving from ”single-system, single user” perspective toward large-scale ensembles of networked systems interacting with communities of users 31-Mar-15 3
Beyond Modes of Locomotion and Location  Context is more than simply recognizing motion state and location.  Why less impact of complex activity recognition to real world applications? Challenges: #1. Recognizing compound activities  Recognizing modes of locomotion is well-defined constrained problem (there’s limited number of nonoverlapping, discriminant states (labels) with labels such as standing, sitting, driving, cycling, walking, running  Assumption: People walk, stand or sit but not two at the same time (exceptions such as sitting while driving exit)  Activities of daily living (ADL –based approach) is motivated by contextual activities and include things such as housekeeping, meal preparation, watching TV.  Recognition labels here represent complex compound activities, some of which can be composed of numerous, diverse lower-level activities  Many of the activities can be executed in parallel (housekeeping and watching TV) or interleaved 31-Mar-15 4
Beyond Modes of Locomotion and Location Challenges: #2. Accommodating high variability  Modes of Locomotion have a distinct signature in an easily deployable sensor modality (acceleration or motion sensors in general).  Activates like housekeeping, there is simply too much variability in the way they can be performed.  We can rely on highly instrumented environment  Putting RFID tags on every relevant object in a household and RFID reader on a user’s wrist to provide recognition  Smart apartments with binary sensors indicating the opening of every cabinate or removal of every object  Problems:  How practical is such extensive instrumentation  The transfer of results from one environment to another isn’t straightforward. One person might use an electric kettle to warm water for tea, while another gets hot water from coffee machine 31-Mar-15 5
Beyond Modes of Locomotion and Location Challenges: #3. User-specific training  Modes of locomotion can be trained reasonably well in a user-independent way. That is, smartphone app can be pretrained and delivered ready to use.  For complex activities, good results are mostly achieved only with user (and environment) specific training. 31-Mar-15 6
Beyond Modes of Locomotion and Location Towards a Solution: #1. Activity models and high-level reasoning  Complex structure of human activity: Activity descriptions (and range of variations) can be mined from the web or contributed by users  Also, models of high-level activities can contribute to increasing the accuracy of the low-level actions on which they build by providing additional boundary conditions and prior probabilities  Finding a set of activities that covers a significant portion of relevant applications and models that optimally support the recognition process. #2. Need for standard datasets for training and evaluation #3. Experimental procedures and description standards  Much of activity-recognition work published to date is inherently difficult to replicate. Observing a set of generally accepted “best practice guidelines” could significantly reduce the problem #4 Better Evaluation Metrics: Example: Evaluation on sensor-frame basis, even basis etc Bottom line: Need to put more focus on incremental improvements of algorithms 31-Mar-15 7
The Need for Opportunistic Systems  Assuming we can reliably recognize complex activities in real-life environments, will that automatically lead to wide-scale adoption of context aware systems?  State-of-the-art approaches to context recognition mostly assume fixed, narrowly defines system configurations dedicated to often equally narrowly defined tasks.  Thus, for each application, the user must deploy and place specific sensors at certain well-defined locations in the environment or his body (purposefully deployed)  For universal context awareness, this approach is not realistic  We need systems that can exploit devices that just “happen” to be in the environment  Sensors are invisibly becoming omnipresent in our environment eg miniaturized motion sensors unobtrusively integrated into clothing  Note: widespread sensor availability doesn’t necessarily mean that an application can assume that required sensors will continuously be available in the same configuration. 31-Mar-15 8
The Need for Opportunistic Systems  Assumption:  As user moves around, s/he is at times in highly instrumented environments where a lot of information sources are available.  At other times, s/he might stay in places with little or no intelligent infrastructure.  Methodological approach for context recognition reverses:  Sensors no longer purposefully deployed to satisfy data needs for recognition algorithms and applications.  Instead, sensors are there and recognition algorithms and applications can make appropriate use of them as needed  This is called opportunistic recognition system (everything is a sensor)  Consider every source of data as an abstract sensor  Miniaturized unit of hardware  Online sensors such as (Twitter feed, facebook updates)  Harvest sensors (which can collect and locally store data)  Playback sensors (which can generate data streams form repositories) 31-Mar-15 9
The Need for Opportunistic Systems  Abstracting all kinds of information sources into generic sensor categories providing standardized access interface is the key to opportunistic sensing  Example: goal-oriented sensor assemblies that configure spontaneously to achieve a common activity (context recognition goal), without requiring a predefined sensor infrastructure or fixed recognition goal to be defined at design time  Sensor interface abstraction need to be preserved (spontaneous sensor ensemble configuration)  Another Approach: Have sensors and systems accumulate and exchange experiences about successful sensing missions for the recognized activities in nonsupervised learning procedure.  “Transfer learning” (then next, “informed” sensing) 31-Mar-15 10
Socially Aware Computing  Implication so far:  The environment will be full of users equipped with devices that can reliably perform complex activity recognition.  Every device will be “willing” and able to identify and exploit potential information sources in its neighborhood  What happens if, instead of considering each device for itself, we look at them as a collaborating collective? Social aware computing  Novel, more powerful methods for monitoring and analyzing social interactions  Subsuming the information form many individual interactions into models of aggregate human behavior and social dynamics  Developing collaborative, “social” algorithms for the recognition process itself  Social aware computing is related to recent trend or leveraging people’s mobile phones for large-scale, mobile sensing 31-Mar-15 11
Socially Aware Computing  Social aware computing is more than scale of sensing. It allows fundamentally new types of applications, extending the notion of context aware into a different domain.  #1: Sensing Social Interactions:  Early work focused on looking at signal from a system of a single user to establish the type of social activity in which s/he could be engaged in  User location, motion patterns, and vocal behavior would be used to determine the type of social context in which the user was engaged (eg at a meeting, party or alone)  Leveraging collaborative analysis to “reality mine” information from many users collected continuously during everyday interactions can reveal a much more detailed and subtle picture of social interactions 31-Mar-15 12
Socially Aware Computing  Example: Consider reality mining mobile phone – phone GPS data, calls, and email records to better understand “traffic” within an organization.  Analyzing these digital traces creates a detailed picture of face-to-face, voice, and digital communication patterns. 31-Mar-15 13  A German bank achieved significant improvements in performance just by relocating a previously orphaned “customer service” group (labeled “CService,” mostly communicating by email in the original setup) to facilitate more face to face communication. This simple, inexpensive change made sure that that everyone was “in the loop.”  Red: Face-to-Face Communication  Blue: Email Communication
Socially Aware Computing  #2. Sensing Social Phenomena:  Further exploration leads to more than social interactions of individuals, but second-by- second models of group dynamics and reactions over extended periods of time, providing dynamic, structural, and content information.  Idea: Harness these streams of personal data and use them to create and drive dynamic models of aggregate human behavior.  Example:  Public health beyond static demographics (patterns of movement between the places a person lives, eats, works, hang out ie mobile behavior pattern)  Using mobile traces, we can also cluster together people with similar behavior patterns to discover largely independent subgroups in a population (subpopulation)  Knowing mobility behavior of different subpopulations provides a far more accurate picture of their preferences and risks than does standard demographic information 31-Mar-15 14  Pervasive sensor systems collaborating in complex, dynamic way could help us discover complex social dynamics
Discussion  Envisions moving towards reliable, practicable recognition of complex contexts and activities and performing such recognition using dynamically changing, a priori, unknown system configurations in virtually new real-life setting  Then we can recognize social contexts, community level situations, and collective human behaviors  This will facilitate socially aware and adaptive computing as the logical next step from context awareness 31-Mar-15 15
Context to Socially Aware Computing: Challenge  Challenge? Privacy and Data Ownership  Insightful approach: You own your data  You have the right to possess your data  You control the use of your data  You have the right to dispose or distribute your data Reality Mining 31-Mar-15 16
Comments 31-Mar-15 17

More Related Content

From context aware to socially awareness computing - IEEE Pervasive Computing, 11 (1), 32–41. (2012

  • 1. From Context Awareness to Socially Aware Computing Paul Lukowicz DFKI Kaiserslautern, Germany Fredrick Awuor Alex Pentland MIT Mdeia Lab Alois Ferscha Johannes Kepler Univ. of Linz, Austria IEEE Pervasive Computing, 11 (1), 32–41. (2012)
  • 2. Outline  Introduction  Beyond locomotion and location  Challenges  Towards Solution  Realizing opportunistic recognitions  Socially Aware Computing  Sensing Social Interactions  Sensing Social Phenomena  Discussion  Comment 31-Mar-15 2
  • 3. Introduction  Context Awareness – Notion of systems adapting their functionality to user’s activities and situation in the environment  Marc Weiser’s dream: mobile phones featuring acceleration and light sensors and offering ability to recognize simple context such as device position (on the table or in the pocket), mode of location, and user location – and adapt their functionality accordingly  Key directions that are driving recognition research:  Moving beyond locomotion and location to incorporate complex activity recognition into real-world applications  Realizing opportunistic recognitions; and  Moving from ”single-system, single user” perspective toward large-scale ensembles of networked systems interacting with communities of users 31-Mar-15 3
  • 4. Beyond Modes of Locomotion and Location  Context is more than simply recognizing motion state and location.  Why less impact of complex activity recognition to real world applications? Challenges: #1. Recognizing compound activities  Recognizing modes of locomotion is well-defined constrained problem (there’s limited number of nonoverlapping, discriminant states (labels) with labels such as standing, sitting, driving, cycling, walking, running  Assumption: People walk, stand or sit but not two at the same time (exceptions such as sitting while driving exit)  Activities of daily living (ADL –based approach) is motivated by contextual activities and include things such as housekeeping, meal preparation, watching TV.  Recognition labels here represent complex compound activities, some of which can be composed of numerous, diverse lower-level activities  Many of the activities can be executed in parallel (housekeeping and watching TV) or interleaved 31-Mar-15 4
  • 5. Beyond Modes of Locomotion and Location Challenges: #2. Accommodating high variability  Modes of Locomotion have a distinct signature in an easily deployable sensor modality (acceleration or motion sensors in general).  Activates like housekeeping, there is simply too much variability in the way they can be performed.  We can rely on highly instrumented environment  Putting RFID tags on every relevant object in a household and RFID reader on a user’s wrist to provide recognition  Smart apartments with binary sensors indicating the opening of every cabinate or removal of every object  Problems:  How practical is such extensive instrumentation  The transfer of results from one environment to another isn’t straightforward. One person might use an electric kettle to warm water for tea, while another gets hot water from coffee machine 31-Mar-15 5
  • 6. Beyond Modes of Locomotion and Location Challenges: #3. User-specific training  Modes of locomotion can be trained reasonably well in a user-independent way. That is, smartphone app can be pretrained and delivered ready to use.  For complex activities, good results are mostly achieved only with user (and environment) specific training. 31-Mar-15 6
  • 7. Beyond Modes of Locomotion and Location Towards a Solution: #1. Activity models and high-level reasoning  Complex structure of human activity: Activity descriptions (and range of variations) can be mined from the web or contributed by users  Also, models of high-level activities can contribute to increasing the accuracy of the low-level actions on which they build by providing additional boundary conditions and prior probabilities  Finding a set of activities that covers a significant portion of relevant applications and models that optimally support the recognition process. #2. Need for standard datasets for training and evaluation #3. Experimental procedures and description standards  Much of activity-recognition work published to date is inherently difficult to replicate. Observing a set of generally accepted “best practice guidelines” could significantly reduce the problem #4 Better Evaluation Metrics: Example: Evaluation on sensor-frame basis, even basis etc Bottom line: Need to put more focus on incremental improvements of algorithms 31-Mar-15 7
  • 8. The Need for Opportunistic Systems  Assuming we can reliably recognize complex activities in real-life environments, will that automatically lead to wide-scale adoption of context aware systems?  State-of-the-art approaches to context recognition mostly assume fixed, narrowly defines system configurations dedicated to often equally narrowly defined tasks.  Thus, for each application, the user must deploy and place specific sensors at certain well-defined locations in the environment or his body (purposefully deployed)  For universal context awareness, this approach is not realistic  We need systems that can exploit devices that just “happen” to be in the environment  Sensors are invisibly becoming omnipresent in our environment eg miniaturized motion sensors unobtrusively integrated into clothing  Note: widespread sensor availability doesn’t necessarily mean that an application can assume that required sensors will continuously be available in the same configuration. 31-Mar-15 8
  • 9. The Need for Opportunistic Systems  Assumption:  As user moves around, s/he is at times in highly instrumented environments where a lot of information sources are available.  At other times, s/he might stay in places with little or no intelligent infrastructure.  Methodological approach for context recognition reverses:  Sensors no longer purposefully deployed to satisfy data needs for recognition algorithms and applications.  Instead, sensors are there and recognition algorithms and applications can make appropriate use of them as needed  This is called opportunistic recognition system (everything is a sensor)  Consider every source of data as an abstract sensor  Miniaturized unit of hardware  Online sensors such as (Twitter feed, facebook updates)  Harvest sensors (which can collect and locally store data)  Playback sensors (which can generate data streams form repositories) 31-Mar-15 9
  • 10. The Need for Opportunistic Systems  Abstracting all kinds of information sources into generic sensor categories providing standardized access interface is the key to opportunistic sensing  Example: goal-oriented sensor assemblies that configure spontaneously to achieve a common activity (context recognition goal), without requiring a predefined sensor infrastructure or fixed recognition goal to be defined at design time  Sensor interface abstraction need to be preserved (spontaneous sensor ensemble configuration)  Another Approach: Have sensors and systems accumulate and exchange experiences about successful sensing missions for the recognized activities in nonsupervised learning procedure.  “Transfer learning” (then next, “informed” sensing) 31-Mar-15 10
  • 11. Socially Aware Computing  Implication so far:  The environment will be full of users equipped with devices that can reliably perform complex activity recognition.  Every device will be “willing” and able to identify and exploit potential information sources in its neighborhood  What happens if, instead of considering each device for itself, we look at them as a collaborating collective? Social aware computing  Novel, more powerful methods for monitoring and analyzing social interactions  Subsuming the information form many individual interactions into models of aggregate human behavior and social dynamics  Developing collaborative, “social” algorithms for the recognition process itself  Social aware computing is related to recent trend or leveraging people’s mobile phones for large-scale, mobile sensing 31-Mar-15 11
  • 12. Socially Aware Computing  Social aware computing is more than scale of sensing. It allows fundamentally new types of applications, extending the notion of context aware into a different domain.  #1: Sensing Social Interactions:  Early work focused on looking at signal from a system of a single user to establish the type of social activity in which s/he could be engaged in  User location, motion patterns, and vocal behavior would be used to determine the type of social context in which the user was engaged (eg at a meeting, party or alone)  Leveraging collaborative analysis to “reality mine” information from many users collected continuously during everyday interactions can reveal a much more detailed and subtle picture of social interactions 31-Mar-15 12
  • 13. Socially Aware Computing  Example: Consider reality mining mobile phone – phone GPS data, calls, and email records to better understand “traffic” within an organization.  Analyzing these digital traces creates a detailed picture of face-to-face, voice, and digital communication patterns. 31-Mar-15 13  A German bank achieved significant improvements in performance just by relocating a previously orphaned “customer service” group (labeled “CService,” mostly communicating by email in the original setup) to facilitate more face to face communication. This simple, inexpensive change made sure that that everyone was “in the loop.”  Red: Face-to-Face Communication  Blue: Email Communication
  • 14. Socially Aware Computing  #2. Sensing Social Phenomena:  Further exploration leads to more than social interactions of individuals, but second-by- second models of group dynamics and reactions over extended periods of time, providing dynamic, structural, and content information.  Idea: Harness these streams of personal data and use them to create and drive dynamic models of aggregate human behavior.  Example:  Public health beyond static demographics (patterns of movement between the places a person lives, eats, works, hang out ie mobile behavior pattern)  Using mobile traces, we can also cluster together people with similar behavior patterns to discover largely independent subgroups in a population (subpopulation)  Knowing mobility behavior of different subpopulations provides a far more accurate picture of their preferences and risks than does standard demographic information 31-Mar-15 14  Pervasive sensor systems collaborating in complex, dynamic way could help us discover complex social dynamics
  • 15. Discussion  Envisions moving towards reliable, practicable recognition of complex contexts and activities and performing such recognition using dynamically changing, a priori, unknown system configurations in virtually new real-life setting  Then we can recognize social contexts, community level situations, and collective human behaviors  This will facilitate socially aware and adaptive computing as the logical next step from context awareness 31-Mar-15 15
  • 16. Context to Socially Aware Computing: Challenge  Challenge? Privacy and Data Ownership  Insightful approach: You own your data  You have the right to possess your data  You control the use of your data  You have the right to dispose or distribute your data Reality Mining 31-Mar-15 16

Editor's Notes

  1. Late Marc Weiser (1952-1999)is considered the father f ubiquitous computing, chief scientist at Xerox PARC, US
  2. To understand this, consider diseases of behavior, such as diabetes. It makes sense tht the types of restaurants you go to, the frequency of waling, and dimilar patterns of behaviou strongly influences your risk for diabetes.
  3. To understand this, consider diseases of behavior, such as diabetes. It makes sense tht the types of restaurants you go to, the frequency of waling, and similar patterns of behaviou strongly influences your risk for diabetes.
  4. To understand this, consider diseases of behavior, such as diabetes. It makes sense tht the types of restaurants you go to, the frequency of waling, and similar patterns of behaviou strongly influences your risk for diabetes.