Ethan Brewer, Giovani Valdrighi, Parikshit Solunke, Joao Rulff, Yurii Piadyk, Zhonghui Lv, Jorge Poco, Claudio T. Silva
Overall steps of the semi-supervised methodology. First, an unsupervised clustering algorithm is used to cluster small patches of neighborhoods from aerial imagery. The clustering uses ResNet50 as a feature extractor and an autoencoder. The second step is supervised regression on the target variables using the distribution of clusters composed of neighborhood patches.
Many areas of the world are without basic information on the socioeconomic well-being of the residing population due to limitations in existing data collection methods. Overhead images obtained remotely, such as from satellite or aircraft, can help serve as windows into the state of life on the ground and help 'fill in the gaps' where community information is sparse, with estimates at smaller geographic scales requiring higher resolution sensors. Concurrent with improved sensor resolutions, recent advancements in machine learning and computer vision have made it possible to quickly extract features from and detect patterns in image data, in the process correlating these features with other information. In this work, we explore how well two approaches—a supervised convolutional neural network and semi-supervised clustering based on bag-of-visual-words—estimate population density, median household income, and educational attainment of individual neighborhoods from publicly available high-resolution imagery of cities throughout the United States. Results and analyses indicate that features extracted from the imagery can accurately estimate the density (R^2 up to 0.81) of neighborhoods, with the supervised approach able to explain about half the variation in a population's income and education. In addition to the presented approaches serving as a basis for further geographic generalization, the novel semi-supervised approach provides a foundation for future work seeking to estimate fine-scale information from aerial imagery without the need for label data.
 title = {Granularity at Scale: Estimating Neighborhood Socioeconomic Indicators from High-Resolution Orthographic Imagery and Hybrid Learning},
 author = {Ethan Brewer AND Giovani Valdrighi AND Parikshit Solunke AND Joao Rulff AND Yurii Piadyk AND Zhonghui Lv AND Jorge Poco AND Claudio T. Silva},
 journal = {IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
 year = {2024},
 volume = {17},
 pages = {5668--5679},
 url = {},