SlideShare a Scribd company logo

What is Crowdsourcing - Nicola Osborne

•Download as PPTX, PDF•
•
EDINA, University of Edinburgh
EDINA, University of Edinburgh

Crowdsourcing involves breaking large tasks into smaller pieces that can be completed by members of the general public, often online. It provides benefits like lower costs, greater scale and access to citizen data. However, it also faces challenges like ensuring data quality and sustained participation. Well-designed projects and engagement are needed to motivate crowds and address issues. Quality assurance processes are important to build trust in citizen science.

Related slideshows

The value of engagement
The value of engagementThe value of engagement
The value of engagement
 
Perspectives from the African Open Science Platform/Susan Veldsman
Perspectives from the African Open Science Platform/Susan VeldsmanPerspectives from the African Open Science Platform/Susan Veldsman
Perspectives from the African Open Science Platform/Susan Veldsman
 
Digital Inclusion strategies & Seattle's Community Technology Program
Digital Inclusion strategies & Seattle's Community Technology ProgramDigital Inclusion strategies & Seattle's Community Technology Program
Digital Inclusion strategies & Seattle's Community Technology Program
 
1 of 12
What is Crowdsourcing? Nicola Osborne nicola.osborne@ed.ac.uk Digital Education & Service Manager, EDINA University of Edinburgh Crowd Sourcing & Citizen Science Network
“collaborative production of knowledge and change” (CSCS 2016) citizen science n. scientific work undertaken by members of the general public, often in collaboration with or under the direction of professional scientists and scientific institutions. (OED 2014) • Generally consists of large specialist tasks broken into smaller non- specialist tasks that anyone can do – especially in “Citizen Science”. • Usually managed via online and/or mobile platforms but… • Crowd sourcing is not a new concept… • The first edition of the Oxford English Dictionary was crowdsourced (Winchester 1998). • RSPB Big Garden Bird Watch was conducted on paper for years before online crowdsourcing emerged. Crowdsourcing is…
• Citizen-led initiatives to gather data not otherwise available/collected – e.g. OpenStreetMap, FixMyStreet. • Research or analysis at scale – such as facilitated annotation, analysis, pattern spotting etc. of large data sets, e.g. Old Weather. • Research-led initiatives where crowd sourced data and/or analysis has a clear research purpose, e.g. GalaxyZoo, UK Lamprey Watch. • Collaborative creative projects, e.g. Wikipedia. • Paid participation on bit-work basis, e.g. CrowdFlower, Amazon Mechanical Turk • Mining and analysis of available crowd sourced data sets– Twitter, Flickr etc. e.g. WeSenseIt, Reading the London Riots • Games where research tasks are abstracted, e.g. FoldIt • On the ground reporting of in-the-moment events, e.g. COBWEB Flooding, Guardian Witness Crowdsourcing & Citizen Science can take many forms
Why Crowdsource? • Cost – enables collection of data at scale and (usually) without payment. • Access – citizens on the ground are well placed for incidents, unexpected events, or to regularly access data about their location. • Engagement – makes your presence visible; enables development of relationships with communities; route to engaging citizen in research and policy making. • Scale – some tasks are not realistic at large scale for expert contributors. • Serendipity – the crowd bring a fresh eye and can find anomalies and make unexpected connections
Challenges of crowdsourcing • Time/Cost – to engage with a community takes authentic commitment, substantial communication channels, significant time, appropriate resourcing. • Data Quality – isn’t guaranteed, addressed through training/guidance in platform, simple interfaces, QA processes, comparison to authoritative data, sound research design. • Expectation Management – the crowd need to be motivated and rewarded for participation, how will you do that but manage expectations of change resulting from their participation? • Participation Levels and Locations – to work crowd sourcing projects have to attract a significantly large group of participants, in the right places, and sustain their attention. • Lack of Control – the crowd will not always work as you expect and may develop its own priorities and etiquette. • Legal Risk/Liability, Safety – especially in emergency situations. T&Cs and warnings may be required. • Data Visibility and Unexpected Implications – such as changes in insurance premiums due to better understanding of flood area extent/risk; exposure of location of at-risk species, etc.
Project Design and Engagement Matter • Solid research design matters - hard to change approach part-way through. • Identifying audience(s)/community(ies) and level of engagement desired helps you to tailor engagement. • Communication is critical: feedback and reward mechanisms are essential to ensure citizens remain motived. • Technical considerations need to include; what is possible; what can citizens access/use. • Balance priorities: question/problem and data capture/analysis needs to be useful for project team, but accessible and realistic for non- specialists. • Communities/participants require support and guidance. • Quality assurance processes need to be established, whether through metadata capture, comparison, manual checking, etc. • Contingency planning – in case of poor participation, less successful outcomes, disapointment/expectation management. • Plan/mechanism to conclude, exit or handover project.
Technology and Tool Selection • Impacts accuracy of data collected (e.g. volunteered vs. GPS vs. IP based location). • Interface design, usability, familiarity and language - can ease or increase complexity of participation. • Home computer based tasks can exclude participation by e.g. lower income, older, less able, or more remotely located communities (see RSE 2014). • Mobile devices limit to those with appropriate devices, often also limited to those with iOS/Android/etc. Also require wifi or 3G signals, or apps/interfaces which can function offline. • App and data upload size may mean volunteers incur costs when participating.
Quality Assurance Trust in Citizen Science by researchers and policy makers varies, quality impacts re/use. • Quality Assurance method/process : – Manual/moderation (e.g. Conker Tree Science); – Repetition and redundancy (e.g. Galaxy Zoo); – Targeted comparison with trusted data (e.g. sensors); – Technical measures (e.g. location of submission); – Trust/expertise level (e.g. based on previous submissions). • Raises social challenges of managing potential rejection of volunteered data/effort. • See Chapman 2015.
Questions/Comments?
References & Resources • Chapman, C. 2015. COBWEB D.2.2 Value Adding to Crowdsourced Data for Decision Making. COBWEB Project. Available from: https://cobwebproject.eu/news/publications/deliverables • Dunn, S. and Hedges, M., 2012. Connected Communities: Crowdsourcing in the humanities, a scoping study. AHRC and Kings College London. Available from: http://www.ahrc.ac.uk/documents/project-reports-and-reviews/connected- communities/crowd-sourcing-in-the-humanities/ • Graham, G.G., Cox, J., Simmons, B., Lintott, C., Masters, K., Greenhill, A. and Holmes, K, 2015. How is success defined and measured in online citizen science: a case study of Zooniverse projects. Available from: http://eprints.whiterose.ac.uk/86535/ • Roy, H. et al, 2012. Understanding Citizen Science for Environmental Monitoring. CEH. Available from: http://www.ceh.ac.uk/news/news_archive/documents/understandingcitizenscienceen vironmentalmonitoring_report_final.pdf • Royal Society of Edinburgh, 2014. Spreading the Benefits of Digital Participation in Scotland Final Report. Royal Society of Edinburgh. Available from: http://www.royalsoced.org.uk/1058_SpreadingtheBenefitsofDigitalParticipation.html • Winchester, S., 1998. The Professor and the Madman: A tale of murder, insanity and the making of the Oxford English Dictionary. HarperCollins Publishers.
Crowd Sourcing & Citizen Science Resources • European Citizen Science Association: http://ecsa.biodiv.naturkundemuseum-berlin.de/ • UK-EOF’s Understanding Citizen Science and Environmental Monitoring: http://www.ceh.ac.uk/products/publications/understanding-citizen- science.html • SWE4Citizen Science – OGC Standards for Citizen Science Projects: https://cobwebproject.eu/about/swe4citizenscience-github • University of Edinburgh Crowd Sourcing & Citizen Science Network: http://citsci.ed.ac.uk/ • COBWEB: Citizen Observatory Web Project – an EU-standards compliant citizen science infrastructure, including flooding demonstrators and QA work: https://cobwebproject.eu/ • Crowd Flower: https://www.crowdflower.com/ • Edinburgh CityScope – infrastructure for open creation and sharing of data, including crowd sourcing projects: http://www.edinburghcityscope.org/
Citizen Science Projects and Tools Mentioned Here • Amazon Mechanical Turk: https://www.mturk.com/mturk/welcome • Click to Cure: http://www.clicktocure.net/ • FieldTrip GB – mobile app for small scale crowdsourcing: http://fieldtripgb.blogs.edina.ac.uk/ • FoldIt: https://fold.it/portal/ • GalaxyZoo: http://www.galaxyzoo.org/ • Guardian Witness: http://www.theguardian.com/community/series/guardianwitness • Leafwatch: Conker Tree Science: http://naturelocator.org/leafwatch.html • Old Weather: https://www.oldweather.org/#/ • OPAL: http://www.opalexplorenature.org/ • OpenStreetMap: http://www.openstreetmap.org/ • RSPB Big Garden Bird Watch: http://www.rspb.org.uk/birdwatch/ • UK Lamprey Watch: https://envscot-csportal.org.uk/lampreywatch/ • WeSenseIt: http://wesenseit.eu/ • Wikipedia: https://en.wikipedia.org/wiki/Main_Page

More Related Content

What is Crowdsourcing - Nicola Osborne

  • 1. What is Crowdsourcing? Nicola Osborne nicola.osborne@ed.ac.uk Digital Education & Service Manager, EDINA University of Edinburgh Crowd Sourcing & Citizen Science Network
  • 2. “collaborative production of knowledge and change” (CSCS 2016) citizen science n. scientific work undertaken by members of the general public, often in collaboration with or under the direction of professional scientists and scientific institutions. (OED 2014) • Generally consists of large specialist tasks broken into smaller non- specialist tasks that anyone can do – especially in “Citizen Science”. • Usually managed via online and/or mobile platforms but… • Crowd sourcing is not a new concept… • The first edition of the Oxford English Dictionary was crowdsourced (Winchester 1998). • RSPB Big Garden Bird Watch was conducted on paper for years before online crowdsourcing emerged. Crowdsourcing is…
  • 3. • Citizen-led initiatives to gather data not otherwise available/collected – e.g. OpenStreetMap, FixMyStreet. • Research or analysis at scale – such as facilitated annotation, analysis, pattern spotting etc. of large data sets, e.g. Old Weather. • Research-led initiatives where crowd sourced data and/or analysis has a clear research purpose, e.g. GalaxyZoo, UK Lamprey Watch. • Collaborative creative projects, e.g. Wikipedia. • Paid participation on bit-work basis, e.g. CrowdFlower, Amazon Mechanical Turk • Mining and analysis of available crowd sourced data sets– Twitter, Flickr etc. e.g. WeSenseIt, Reading the London Riots • Games where research tasks are abstracted, e.g. FoldIt • On the ground reporting of in-the-moment events, e.g. COBWEB Flooding, Guardian Witness Crowdsourcing & Citizen Science can take many forms
  • 4. Why Crowdsource? • Cost – enables collection of data at scale and (usually) without payment. • Access – citizens on the ground are well placed for incidents, unexpected events, or to regularly access data about their location. • Engagement – makes your presence visible; enables development of relationships with communities; route to engaging citizen in research and policy making. • Scale – some tasks are not realistic at large scale for expert contributors. • Serendipity – the crowd bring a fresh eye and can find anomalies and make unexpected connections
  • 5. Challenges of crowdsourcing • Time/Cost – to engage with a community takes authentic commitment, substantial communication channels, significant time, appropriate resourcing. • Data Quality – isn’t guaranteed, addressed through training/guidance in platform, simple interfaces, QA processes, comparison to authoritative data, sound research design. • Expectation Management – the crowd need to be motivated and rewarded for participation, how will you do that but manage expectations of change resulting from their participation? • Participation Levels and Locations – to work crowd sourcing projects have to attract a significantly large group of participants, in the right places, and sustain their attention. • Lack of Control – the crowd will not always work as you expect and may develop its own priorities and etiquette. • Legal Risk/Liability, Safety – especially in emergency situations. T&Cs and warnings may be required. • Data Visibility and Unexpected Implications – such as changes in insurance premiums due to better understanding of flood area extent/risk; exposure of location of at-risk species, etc.
  • 6. Project Design and Engagement Matter • Solid research design matters - hard to change approach part-way through. • Identifying audience(s)/community(ies) and level of engagement desired helps you to tailor engagement. • Communication is critical: feedback and reward mechanisms are essential to ensure citizens remain motived. • Technical considerations need to include; what is possible; what can citizens access/use. • Balance priorities: question/problem and data capture/analysis needs to be useful for project team, but accessible and realistic for non- specialists. • Communities/participants require support and guidance. • Quality assurance processes need to be established, whether through metadata capture, comparison, manual checking, etc. • Contingency planning – in case of poor participation, less successful outcomes, disapointment/expectation management. • Plan/mechanism to conclude, exit or handover project.
  • 7. Technology and Tool Selection • Impacts accuracy of data collected (e.g. volunteered vs. GPS vs. IP based location). • Interface design, usability, familiarity and language - can ease or increase complexity of participation. • Home computer based tasks can exclude participation by e.g. lower income, older, less able, or more remotely located communities (see RSE 2014). • Mobile devices limit to those with appropriate devices, often also limited to those with iOS/Android/etc. Also require wifi or 3G signals, or apps/interfaces which can function offline. • App and data upload size may mean volunteers incur costs when participating.
  • 8. Quality Assurance Trust in Citizen Science by researchers and policy makers varies, quality impacts re/use. • Quality Assurance method/process : – Manual/moderation (e.g. Conker Tree Science); – Repetition and redundancy (e.g. Galaxy Zoo); – Targeted comparison with trusted data (e.g. sensors); – Technical measures (e.g. location of submission); – Trust/expertise level (e.g. based on previous submissions). • Raises social challenges of managing potential rejection of volunteered data/effort. • See Chapman 2015.
  • 10. References & Resources • Chapman, C. 2015. COBWEB D.2.2 Value Adding to Crowdsourced Data for Decision Making. COBWEB Project. Available from: https://cobwebproject.eu/news/publications/deliverables • Dunn, S. and Hedges, M., 2012. Connected Communities: Crowdsourcing in the humanities, a scoping study. AHRC and Kings College London. Available from: http://www.ahrc.ac.uk/documents/project-reports-and-reviews/connected- communities/crowd-sourcing-in-the-humanities/ • Graham, G.G., Cox, J., Simmons, B., Lintott, C., Masters, K., Greenhill, A. and Holmes, K, 2015. How is success defined and measured in online citizen science: a case study of Zooniverse projects. Available from: http://eprints.whiterose.ac.uk/86535/ • Roy, H. et al, 2012. Understanding Citizen Science for Environmental Monitoring. CEH. Available from: http://www.ceh.ac.uk/news/news_archive/documents/understandingcitizenscienceen vironmentalmonitoring_report_final.pdf • Royal Society of Edinburgh, 2014. Spreading the Benefits of Digital Participation in Scotland Final Report. Royal Society of Edinburgh. Available from: http://www.royalsoced.org.uk/1058_SpreadingtheBenefitsofDigitalParticipation.html • Winchester, S., 1998. The Professor and the Madman: A tale of murder, insanity and the making of the Oxford English Dictionary. HarperCollins Publishers.
  • 11. Crowd Sourcing & Citizen Science Resources • European Citizen Science Association: http://ecsa.biodiv.naturkundemuseum-berlin.de/ • UK-EOF’s Understanding Citizen Science and Environmental Monitoring: http://www.ceh.ac.uk/products/publications/understanding-citizen- science.html • SWE4Citizen Science – OGC Standards for Citizen Science Projects: https://cobwebproject.eu/about/swe4citizenscience-github • University of Edinburgh Crowd Sourcing & Citizen Science Network: http://citsci.ed.ac.uk/ • COBWEB: Citizen Observatory Web Project – an EU-standards compliant citizen science infrastructure, including flooding demonstrators and QA work: https://cobwebproject.eu/ • Crowd Flower: https://www.crowdflower.com/ • Edinburgh CityScope – infrastructure for open creation and sharing of data, including crowd sourcing projects: http://www.edinburghcityscope.org/
  • 12. Citizen Science Projects and Tools Mentioned Here • Amazon Mechanical Turk: https://www.mturk.com/mturk/welcome • Click to Cure: http://www.clicktocure.net/ • FieldTrip GB – mobile app for small scale crowdsourcing: http://fieldtripgb.blogs.edina.ac.uk/ • FoldIt: https://fold.it/portal/ • GalaxyZoo: http://www.galaxyzoo.org/ • Guardian Witness: http://www.theguardian.com/community/series/guardianwitness • Leafwatch: Conker Tree Science: http://naturelocator.org/leafwatch.html • Old Weather: https://www.oldweather.org/#/ • OPAL: http://www.opalexplorenature.org/ • OpenStreetMap: http://www.openstreetmap.org/ • RSPB Big Garden Bird Watch: http://www.rspb.org.uk/birdwatch/ • UK Lamprey Watch: https://envscot-csportal.org.uk/lampreywatch/ • WeSenseIt: http://wesenseit.eu/ • Wikipedia: https://en.wikipedia.org/wiki/Main_Page