The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Download
Publications Copernicus
Download
Citation
Articles | Volume XLIII-B3-2021
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B3-2021, 887–894, 2021
https://doi.org/10.5194/isprs-archives-XLIII-B3-2021-887-2021
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B3-2021, 887–894, 2021
https://doi.org/10.5194/isprs-archives-XLIII-B3-2021-887-2021

  29 Jun 2021

29 Jun 2021

EVALUATING VISUAL IMPRESSIONS BASED ON GAZE ANALYSIS AND DEEP LEARNING: A CASE STUDY OF ATTRACTIVENESS EVALUATION OF STREETS IN DENSELY BUILT-UP WOODEN RESIDENTIAL AREA

T. Oki and S. Kizawa T. Oki and S. Kizawa
  • Tokyo Institute of Technology, 2-12-1-M1-27 Ookayama, Meguro-ku, Tokyo 152-8550, Japan

Keywords: Gaze Analysis, Convolutional Neural Network, Grad-CAM, Densely Built-up Wooden Residential Area, Attractiveness, Google Street View, Questionnaire, Semantic Segmentation

Abstract. This paper examines the possibility of impression evaluation based on gaze analysis of subjects and deep learning, using an example of evaluating street attractiveness in densely built-up wooden residential areas. Firstly, the relationship between the subjects' gazing tendency and their evaluation of street image attractiveness is analysed by measuring the subjects' gaze with an eye tracker. Next, we construct a model that can estimate an attractiveness evaluation result using convolutional neural networks (CNNs), combined with the method of gradient-weighted class activation mapping (Grad-CAM) - these in in visualizing which street components can contribute to evaluating attractiveness. Finally, we discuss the similarity between the subjects' gaze tendencies and activation heatmaps created by Grad-CAM.