An Exploration of Crowdwork, Machine Learning and Experts for Extracting Information from Data

By Fabion Kauker, Kayan Hau and John Iannello

3-GIS Product Architect Fabian Kauker coauthored a paper on setting the expectations for building systems with machine learning to impact GIS projects. Some of the topics covered include:

  • types of systems and their benefits and performance
  • which approach is the best fit
  • key factors that impact the productivity and quality of results

Fabion's work was awarded best paper in the Human Interface and the Management of Information thematic area by HCI International for 2018.

Fill out the form to download the paper


The growing use of data to derive insights and information presents many challenges and opportunities. Further, the increased awareness of the potential of crowdworking and machine learning technologies has created a need to understand the benefits and caveats of these approaches. By reviewing current research and then comparing a novice based crowdworking approach against experts and machine learning benchmarks we seek to assess the trade-offs. The task specifically requires users to interpret satellite imagery and determine the location of residences or businesses. We are able to demonstrate that a novice approach can provide value where the data collected meets an accuracy tolerance that closely matches the expert users. Further, the potential for equivalent results is shown to be possible based on potential improvements to the system and user familiarity with the task.