Interpreting Black-Box Semantic Segmentation Models in Remote Sensing Applications

Janik, Adrianna; Sankaran, Kris; Ortiz, Anthony

Interpreting Black-Box Semantic Segmentation Models in Remote Sensing Applications

Files

007-011.pdf (2.2 MB)

Date

2019

Authors

Janik, Adrianna
Sankaran, Kris
Ortiz, Anthony

Publisher

The Eurographics Association

Abstract

In the interpretability literature, attention is focused on understanding black-box classifiers, but many problems ranging from medicine through agriculture and crisis response in humanitarian aid are tackled by semantic segmentation models. The absence of interpretability for these canonical problems in computer vision motivates this study. In this study we present a usercentric approach that blends techniques from interpretability, representation learning, and interactive visualization. It allows to visualize and link latent representation to real data instances as well as qualitatively assess strength of predictions. We have applied our method to a deep learning model for semantic segmentation, U-Net, in a remote sensing application of building detection. This application is of high interest for humanitarian crisis response teams that rely on satellite images analysis. Preliminary results shows utility in understanding semantic segmentation models, demo presenting the idea is available online.

        @inproceedings{10.2312:mlvis.20191158
,
booktitle = {Machine Learning Methods in Visualisation for Big Data
},
editor = {Archambault, Daniel and Nabney, Ian and Peltonen, Jaakko
},
title = {{Interpreting Black-Box Semantic Segmentation Models in Remote Sensing Applications
}},
author = {Janik, Adrianna and 
Sankaran, Kris and 
Ortiz, Anthony
},
year = {2019
},
publisher = {The Eurographics Association
},
ISBN = {978-3-03868-089-5
},
DOI = {10.2312/mlvis.20191158
}
}