SlideShare a Scribd company logo

COBWEB - infrastructure and platform for Environmental Crowd Sensing and Big Data

•Download as PPTX, PDF•
•
EDINA, University of Edinburgh
EDINA, University of Edinburgh

The document introduces COBWEB, a European Commission-funded project that develops a crowdsourcing infrastructure for collecting and analyzing environmental data. It summarizes the goals of the project, its partners which include UNESCO biosphere reserves, methods for co-designing use cases, and the development of quality assurance processes and mobile/web apps. Key components under development include workflows, services, sensor networks, and tools for customizing data collection and ensuring data quality.

1 of 33
COBWEB EnviroInfo & ICT4S 2015, Workshop - Infrastructures and Platforms for Environmental Crowd Sensing and Big Data, Copenhagen, 9th Sept, 2015 Chris Higgins chris.higgins@ed.ac.uk Sta
Introduction to COBWEB • Research Project: Funded under the European Commission’s Framework Programme 7 • Started Nov 2012 for 4 years (Month 35 of 48) • First demonstrator completed and tested during the 2015 field season • Why? – GPS enabled, internet connected mobile devices now ubiquitous – Lots of potential, eg, can citizen sourced environmental data be useful for decision making?
Citizen Observatory Web • Generic crowdsourcing infrastructure – A toolkit which can be downloaded and used in multiple scenarios • Data which supports policy • Address data quality issues • Open standards
Project Partners
UNESCO World Network of Biosphere Reserves Sites of excellence to foster harmonious integration of people and nature for sustainable development through participation, knowledge sharing, poverty reduction and human well-being improvements, cultural values and society's ability to cope with change, thus contributing to the Millennium Development Goals
COBWEB Biosphere Reserves Biosffer Dyfi Biosphere Mount Olympus Gorge of Samaria Wadden See & Hallig islands
Dyfi Biosphere Reserve #1 www.biosfferdyfi.org.uk
Dyfi Biosphere Reserve #2 Contains Ordnance Survey data © Crown copyright and database right (2013) ©CountrysideCouncilforWales.Allrightsreserved.
Co-design
Co-design – Snowdonia National Park • Japanese Knotweed - Fallopia japonica • Giant Knotweed – Fallopia sachalinensis • Hybrid knotweed – Fallopia x bohemica
Co-design – RSPB • Domenlas Saltmarsh – Survey vegetation – Use quadrats – Soil salinity levels – Weather conditions – Look at reversion process • Covert Coch Peatbog – Survey vegetation – Use quadrats – Soil moisture – Grazing history Janetbaxter.co.uk
Co-Design - Penparcau Community Forum
Co-Design - Penparcau Community Forum • Temperature • Humidity • Wind speed • Wind direction • Pressure • Precipitation In situ sensors
Usability Testing
Not just apps… A number of demonstrator mobile phone applications – Exactly what, deliberately left open and subject to discussion with community 3 pilot case study areas: 1. Validating earth observation products 2. Biological monitoring 3. Flooding
COBWEB Framework
Key components at different TRL’s • Conflation • QA workflow editor • QA WPS/services • Sensor networks • GeoNetwork/Portal • Middleware • Authoring tool/Survey designer • Apps • User management and privacy • Access control • Authentication
The app itself Key features • Capture information – Images – Audio – Text – Location • High quality background maps • Saved maps for use “offline” • Custom data collection forms • Manual location correction
View COBWEB Portal
Join COBWEB Portal
Customise your own app for your survey
Customise your own app for your survey
Customise your own app for your survey
Citizen captures data on their phone
Approach to QA • Quality Assurance: complex problem and often use case specific: – Internal quality (metrics) – External quality (fit for purpose) • A generic system that can be easily customised to fit new use cases, based on standards • Understanding data quality is likely to require a combination of approaches and tests, the system is designed to enable this • As quality requirements are use case specific, the quality control tests are configurable to utilise a wide variety of datasets and parameters
Classifying quality: Seven pillars Pillar Example Test Notes Pillar 1 – Location Based services Assessment of spatial accuracy – estimate from a mobile device and number of satellites Tests often carried out on the mobile device Pillar 2 – Cleaning Removal of junk data via an attribute text check Very lightweight, can flag or remove malicious entries Pillar 3 – Automatic validation Analysis whether an image is blurry Higher level testing, often used to assess ranges Pillar 4 – Comparison with authoritative data Use of a set of boundary polygons to check whether an observation is in or out Wide variety of tests that involve comparison with what it known Pillar 5 – Model based validation Running a flood model Can be complex, and may also include question based modeling Pillar 6 – Big/Linked data Querying Twitter via a hashtag for similar phenomena Tapping into large databases such as sensor records and social media Pillar 7 – Semantic harmonisation Rationalisation of entries via an ontology Attempts to recognise multiple entries of the same observation
Web Processing Service • OGC Standard • Web facing • Public • Holds spatial and non-spatial processes • Processes are suited to chaining • BPEL is the traditional method of chaining – However, removed from WPS 2.0
BPMN Workflow Engine • JBOSS JBPM – A Java based BPM workflow engine • Orchestrates processes in a given order with defined inputs • Chains results from one process to the next • Can be executed remotely via REST • Graphical interface utilises BPMN2.0 – a recently ratified OMG standard • Unlike BPEL this interface is standardised
Current questions • What is the Post Quality database. How to manage different schemas and different data sources? • RDF to handle the fuzzy view of crowdsourced data? • How is client side QA being managed? • GeoPackage + watching service? • Client side vs server side • Security • Can we protect certain datasets/processes and make them available to those who have the correct credentials? • Many others!!
For more information on QA – see: • Relevant sessions at the OGC Technical Committee in Nottingham next week • Architecture Implementation Pilot – 8 deep dive videos: http://www.ogcnetwork.net/node/1969
Service Provider (SP) Identity Provider (IdP)Discovery Service (DS) “GEOSS user” Single- Sign-On Trust Gateway (TG) to OpenID Google OpenId COBWEB/GEOSS AIP-6 Federation NASA Ames Secure Dimensions CUAHSI* Catapult University of Edinburgh Kst. GDI.DE *: Consortium of Universities for the Advancement of Hydrologic Science EarthServer (FP7) project MEEO
Remainder of project…emphasis shifting • Rolling out software • Getting greater buy-in • Sustainability • More focus on research agenda • Greater focus on Greece, Germany • Additional Biosphere Reserves
Thank you • Website: http://cobwebproject.eu/ • Mailing list: http://eepurl.com/Ita05 or use the QR code • Follow us: @cobwebfp7 • And look out for members of the COBWEB team at events. chris.higgins@ed.ac.uk

More Related Content

COBWEB - infrastructure and platform for Environmental Crowd Sensing and Big Data

  • 1. COBWEB EnviroInfo & ICT4S 2015, Workshop - Infrastructures and Platforms for Environmental Crowd Sensing and Big Data, Copenhagen, 9th Sept, 2015 Chris Higgins chris.higgins@ed.ac.uk Sta
  • 2. Introduction to COBWEB • Research Project: Funded under the European Commission’s Framework Programme 7 • Started Nov 2012 for 4 years (Month 35 of 48) • First demonstrator completed and tested during the 2015 field season • Why? – GPS enabled, internet connected mobile devices now ubiquitous – Lots of potential, eg, can citizen sourced environmental data be useful for decision making?
  • 3. Citizen Observatory Web • Generic crowdsourcing infrastructure – A toolkit which can be downloaded and used in multiple scenarios • Data which supports policy • Address data quality issues • Open standards
  • 5. UNESCO World Network of Biosphere Reserves Sites of excellence to foster harmonious integration of people and nature for sustainable development through participation, knowledge sharing, poverty reduction and human well-being improvements, cultural values and society's ability to cope with change, thus contributing to the Millennium Development Goals
  • 6. COBWEB Biosphere Reserves Biosffer Dyfi Biosphere Mount Olympus Gorge of Samaria Wadden See & Hallig islands
  • 7. Dyfi Biosphere Reserve #1 www.biosfferdyfi.org.uk
  • 8. Dyfi Biosphere Reserve #2 Contains Ordnance Survey data © Crown copyright and database right (2013) ©CountrysideCouncilforWales.Allrightsreserved.
  • 10. Co-design – Snowdonia National Park • Japanese Knotweed - Fallopia japonica • Giant Knotweed – Fallopia sachalinensis • Hybrid knotweed – Fallopia x bohemica
  • 11. Co-design – RSPB • Domenlas Saltmarsh – Survey vegetation – Use quadrats – Soil salinity levels – Weather conditions – Look at reversion process • Covert Coch Peatbog – Survey vegetation – Use quadrats – Soil moisture – Grazing history Janetbaxter.co.uk
  • 12. Co-Design - Penparcau Community Forum
  • 13. Co-Design - Penparcau Community Forum • Temperature • Humidity • Wind speed • Wind direction • Pressure • Precipitation In situ sensors
  • 15. Not just apps… A number of demonstrator mobile phone applications – Exactly what, deliberately left open and subject to discussion with community 3 pilot case study areas: 1. Validating earth observation products 2. Biological monitoring 3. Flooding
  • 17. Key components at different TRL’s • Conflation • QA workflow editor • QA WPS/services • Sensor networks • GeoNetwork/Portal • Middleware • Authoring tool/Survey designer • Apps • User management and privacy • Access control • Authentication
  • 18. The app itself Key features • Capture information – Images – Audio – Text – Location • High quality background maps • Saved maps for use “offline” • Custom data collection forms • Manual location correction
  • 21. Customise your own app for your survey
  • 22. Customise your own app for your survey
  • 23. Customise your own app for your survey
  • 24. Citizen captures data on their phone
  • 25. Approach to QA • Quality Assurance: complex problem and often use case specific: – Internal quality (metrics) – External quality (fit for purpose) • A generic system that can be easily customised to fit new use cases, based on standards • Understanding data quality is likely to require a combination of approaches and tests, the system is designed to enable this • As quality requirements are use case specific, the quality control tests are configurable to utilise a wide variety of datasets and parameters
  • 26. Classifying quality: Seven pillars Pillar Example Test Notes Pillar 1 – Location Based services Assessment of spatial accuracy – estimate from a mobile device and number of satellites Tests often carried out on the mobile device Pillar 2 – Cleaning Removal of junk data via an attribute text check Very lightweight, can flag or remove malicious entries Pillar 3 – Automatic validation Analysis whether an image is blurry Higher level testing, often used to assess ranges Pillar 4 – Comparison with authoritative data Use of a set of boundary polygons to check whether an observation is in or out Wide variety of tests that involve comparison with what it known Pillar 5 – Model based validation Running a flood model Can be complex, and may also include question based modeling Pillar 6 – Big/Linked data Querying Twitter via a hashtag for similar phenomena Tapping into large databases such as sensor records and social media Pillar 7 – Semantic harmonisation Rationalisation of entries via an ontology Attempts to recognise multiple entries of the same observation
  • 27. Web Processing Service • OGC Standard • Web facing • Public • Holds spatial and non-spatial processes • Processes are suited to chaining • BPEL is the traditional method of chaining – However, removed from WPS 2.0
  • 28. BPMN Workflow Engine • JBOSS JBPM – A Java based BPM workflow engine • Orchestrates processes in a given order with defined inputs • Chains results from one process to the next • Can be executed remotely via REST • Graphical interface utilises BPMN2.0 – a recently ratified OMG standard • Unlike BPEL this interface is standardised
  • 29. Current questions • What is the Post Quality database. How to manage different schemas and different data sources? • RDF to handle the fuzzy view of crowdsourced data? • How is client side QA being managed? • GeoPackage + watching service? • Client side vs server side • Security • Can we protect certain datasets/processes and make them available to those who have the correct credentials? • Many others!!
  • 30. For more information on QA – see: • Relevant sessions at the OGC Technical Committee in Nottingham next week • Architecture Implementation Pilot – 8 deep dive videos: http://www.ogcnetwork.net/node/1969
  • 31. Service Provider (SP) Identity Provider (IdP)Discovery Service (DS) “GEOSS user” Single- Sign-On Trust Gateway (TG) to OpenID Google OpenId COBWEB/GEOSS AIP-6 Federation NASA Ames Secure Dimensions CUAHSI* Catapult University of Edinburgh Kst. GDI.DE *: Consortium of Universities for the Advancement of Hydrologic Science EarthServer (FP7) project MEEO
  • 32. Remainder of project…emphasis shifting • Rolling out software • Getting greater buy-in • Sustainability • More focus on research agenda • Greater focus on Greece, Germany • Additional Biosphere Reserves
  • 33. Thank you • Website: http://cobwebproject.eu/ • Mailing list: http://eepurl.com/Ita05 or use the QR code • Follow us: @cobwebfp7 • And look out for members of the COBWEB team at events. chris.higgins@ed.ac.uk