Natural Disaster Relief Using Machine Learning
Natural disasters affect approximately 160 million people worldwide each year.
With over 400 natural disasters in 2019, relief organisations and governments are struggling to turn these massive amounts of data into actionable insights.
Fortunately, advances in machine learning and artificial intelligence have made it possible for researchers, engineers, and scientists to explore and analyse various data sources more efficiently than ever before.
So, how can machine learning help with disaster relief?
Appsilon Data Science Devised A Novel Solution
The company created machine learning models to assess structural damage by analysing before-and-after satellite images of natural disasters as an entry to the xView2 competition organised by the Defense Innovation Unit at the United States Department of Defense. They created their models with PyTorch and their critical parts with fast.ai.
How Does Appsilon’s Machine Learning For Natural Disaster Relief Work?
Using the xBD dataset, the Appsilon Data Science Machine Learning team created ML models. The dataset includes information from eight different types of disasters, 15 different countries, and thousands of square kilometres of imagery. The goal of the ML models was to assess infrastructure damage in order to reduce human labour and shorten the time required to plan an adequate response.
The models not only achieve high accuracy, but also have an easy-to-use user interface that allows anyone to take advantage of their capabilities. Appsilon’s own shiny.semantic open source package was used to develop and implement the interface in Shiny.
What Are The Technical Specifications Of Appsilon’s Machine Learning Models For Natural Disaster Relief?
The App for Damage Assessment
Appsilon Data Science implemented their models in a Shiny app that allows users to investigate the impact of four real-life natural disasters by running their model on pre-built scenarios: Hurricane Florence, the Santa Rosa Wildfire, the Midwest Flooding, and the Palu Tsunami.
The latter was caused by the September 2018 earthquake in Indonesia and caused significant property damage in the Palu area.
The Information
The xBD dataset is made up of satellite imagery data from several regions that have recently been impacted by natural disasters. The dataset is quite diverse, encompassing a wide range of affected locations, ranging from remote forests to industrial areas with large buildings to dense urban landscapes.
The team’s main challenges were the various locations and building sizes, as well as the variety of disasters.
The model had to assess structural damage in addition to being able to localise the buildings. That meant taking one approach for areas destroyed by fire and another for those destroyed by flood. Volcanic eruptions, hurricanes, devastating floods, tsunamis, raging wildfires, tornado damage, and bushfires were all included in the dataset. Saving response planners hours of time spent searching through thousands of images or conducting face-to-face surveys allows them to focus their limited resources on taking appropriate action and, as a result, saving more lives. In terms of imagery, having ‘before’ and ‘after’ images of affected areas proved critical for expediting humanitarian response.
The Pipeline For Machine Learning
Building localization and damage assessment are two components of developing a high-accuracy ML model for satellite imagery analysis. Although a single model could perform both tasks, Appsilon Data Science chose to create a separate model for each task.
This proved difficult because ‘each model requires a separate training dataset with the same preprocessing schedule.’ Preprocessing entails altering the colour histograms of images, as well as cropping, padding, and flipping, among other data augmentation techniques. Naturally, the team struggled to keep track of so many moving parts. Furthermore, performing inference on parameters that are slightly different from the parameters used for model training would result in a very low score.
So, How Did Appsilon Manage To Avoid This Risk?
Using an appropriate ML pipeline. As a result, the training process remained completely reproducible and efficient. They did this by basing the pipeline on their internal framework. Because it remembered the outcomes of all the steps, they were able to reuse them. Furthermore, it ran the computation automatically if any hyperparameters or dependencies changed. All of this contributed to a much faster and error-free development process.
The specific steps are as follows:
Appsilon Data Science Machine Learning Pipeline for Natural Disaster Relief
However, it was not only the ML pipeline that allowed the team to deliver an accurate model. Two other techniques, transfer learning for localization and 7-channel input for classification, also contributed to their success in a short period of time.
Transferring knowledge for localization
Appsilon selected one of the best SpaceNet competition models. They then used transfer learning to assess building damage in natural disaster response.
The accuracy of the model is heavily reliant on localization. After all, the model can’t classify building damage unless it finds it first. According to one team member, “the better the localization model, the higher the potential score for damage assessment.”
The team relied on neural networks based on the UNet architecture to solve such segmentation problems.
UNet architectures encode the image to a lower resolution representation before decoding it to generate a segmentation mask. Still, developing appropriate weights for the segmentation exercise can take a significant amount of time and effort.
Fortunately, building localization on images is a well-studied topic. As a result, the Appsilon team used existing, cutting-edge ML localization solutions developed through a series of SpaceNet challenges to build their model. They used XDXD pre-trained weights created for one of the top SpaceNet4 solutions, which was built with a VGG-16 encoder.
Mehrdad Yazdani’s Unet Architecture
Mehrdad Yazdani’s UNet architecture example – Own work, CC BY-SA 4.0, Creative Commons
They continued training the model with pre-existing weights on the xView2 data once it was ready. As a result, they improved accuracy while spending much less time on computation when compared to starting from scratch.
7-Channel Classification Input
Appsilon accelerated the training of the damage assessment model even further by using a 7th channel to identify the location of a building in an image.
They were able to identify the locations of individual buildings thanks to the localization model. How?
They cut each building and a small portion of the surrounding area out of the larger image.
This was necessary because a building’s damage score was sometimes affected by its surroundings. A building with an undamaged roof, for example, was still scored as having sustained significant damage because it was completely surrounded by floodwater. This meant that Appsilon Data Science had two cutouts for each building, one before and one after the disaster (each had 3 channels: RGB).
They also used a 7th channel – a mask that highlighted the location of the building within the cutout. The 7th channel enables the classification model to quickly identify the image’s most important component – the building itself. True, such an addition increases the size of the model and may slow the inference process slightly. Nonetheless, it speeds up the model’s training because it learns where to focus its attention more quickly. And, in the end, this leads to greater precision.
How Can You Help With Natural Disaster Relief Using Machine Learning?
We believe that Appsilon Data Science’s machine learning solution for disaster relief is an excellent example of how the data science community can assist in resolving one of today’s most pressing issues. As part of their AI for Good initiative, the company reaches out to the global tech community to see if they can use their expertise to help those working on the front lines of natural disaster management and response by providing them with cutting-edge solutions.
