Human-Machine Collaboration for Fast Land Cover Mapping (Papers Track)
Caleb Robinson (Georgia Institute of Technology); Anthony Ortiz (University of Texas at El Paso); Nikolay Malkin (Yale University); Blake Elias (Microsoft); Andi Peng (Microsoft); Dan Morris (Microsoft); Bistra Dilkina (University of Southern California); Nebojsa Jojic (Microsoft Research)
Abstract
We propose incorporating human labelers in a model fine-tuning system that provides immediate user feedback. In our framework, human labelers can interactively query model predictions on unlabeled data, choose which data to label, and see the resulting effect on the model's predictions. This bi-directional feedback loop allows humans to learn how the model responds to new data. Our hypothesis is that this rich feedback allows human labelers to create mental models that enable them to better choose which biases to introduce to the model. We implement this framework for fine-tuning high-resolution land cover segmentation models and evaluate it against traditional active learning based approaches. More specifically, we fine-tune a deep neural network -- trained to segment high-resolution aerial imagery into different land cover classes in Maryland, USA -- to a new spatial area in New York, USA. We find that the tight loop turns the algorithm and the human operator into a hybrid system that can produce land cover maps of large areas more efficiently than the traditional workflows.