This story is divided in two parts: Methodology and the waterdetect package. In the methodology, the main concepts of the algorithm are given, in order to provide the reader a better understanding of the package and how to tune it. The second part is a tutorial on the waterdetect package with sample codes to run it.

Methodology

Introduction

The use of deep learning techniques for remote sensing applications has been increasing in recent years. The recently published review paper “Object Detection and Image Segmentation with Deep Learning on Earth Observation Data: A Review-Part I: Evolution and Recent Trends” (Hoeser and Kuenzer 2020)[1] presents the evolution of Convolutional Neural Networks (CNN) in satellite applications, notably Object Detection and Image Segmentation, where it achieves state-of-the-art results. …

We know that image augmentation is a key factor for computer vision tasks. It helps the algorithm to avoid overfitting, as well as it limits the need for huge training datasets [1]. Most deep learning frameworks have a vision module that implements augmentation “out-of-the-box”, as it is the case of Keras, PyTorch and Fastai libraries. The problem starts to rise when we need to feed the model with images that don’t match the 3-channel standard (RGB). That’s the case of most remote sensing applications (ex. Figure 1) and a number of other areas.

To overcome this limitation, I will show how to implement a multispectral augmentation using the Albumentations library [2] and plug it into a Fastai v2 DataBlock for further training. The principles about how to create a DataBlock for satellite images can be found in my previous story “How to create a DataBlock for Multispectral Satellite Image Segmentation with the Fastai-v2”. …

Fastai is an open source deep learning library that adds higher level functionalities to PyTorch and makes it easier to achieve state-of-the-art results with little coding.

The vision module is really handy when we need to quickly create an image dataset, apply data augmentations, resize, crop or even overlay a segmentation mask (Figure 1). However, all this simplification comes with a cost. Most of these higher level APIs for computer vision are optimized to RGB images, and these image libraries don’t support multispectral or multichannel images.

Introduction

In some previous stories (here, here and here) we’ve used PyTorch and Fast.ai library to segment clouds in satellite images, using as reference a public dataset (Kaggle’s 38-Cloud: Cloud Segmentation in Satellite Images). However, there are cases when we need to prepare our own dataset from the beginning, and that can be time-consuming without the proper tools.

As it is not my objective here to explain GEE in depth, I will cover just the basics needed to accomplish our final goal, that is to obtain training patches ready to be consumed by any deep learning framework. …

Introduction

The clustering analysis is one type of unsupervised technique used to identify (i.e. group) similar samples in a multi-dimensional data space. In the context of remote sensing data, it is largely used for pixel classification, where the multi-dimensional space represents the pixel reflectances in n different wavelengths (sensor bands) or any other combination.

During my first year of PhD, at the Paul Sabatier University (Toulouse), more specifically working at the GET laboratory, we have been working with satellite images from Sentinel-2 and Landsat-8 missions to assess inland water surface. As the water reflectance can vary a lot depending on the region, atmospheric conditions, and constituents it is not viable to employ a single threshold, or a set of fixed rules to separate water from land. …

In my previous story (here), I showed how to create a multi-channel dataset for satellite images from scratch, without using the torchvision module.

Now, we will move on to create a simple deep learning model, for semantic segmentation of satellite images and check how it performs using the 38-Cloud: Cloud Segmentation in Satellite Image, from Kaggle.

To make things easier, this code is available in Kaggle notebook 38-Cloud-Simple_Unet, available here.

The Model

My different model architectures can be used for a pixel-level segmentation of images. The 2019 Guide to Semantic Segmentation is a good guide for many of them, showing the main differences in their concepts. …

In my last Medium story (here) I proposed an approach using the high level API Fast.ai to detect cloud contours in satellite images. Detecting object contours (i.e. all the pixels belonging to the same object) is called semantic segmentation. The dataset that was used for the task is the 38-Cloud: Cloud Segmentation in Satellite Image, from Kaggle.

Although we could achieve a relatively good accuracy of 96% with a few lines of code, the model was not able to consider all the input channels, Red, Green, Blue and NIR(Near Infrared) provided in the dataset. The problem is that most of the semantic segmentation models found in deep learning frameworks like Keras, Fast.ai and even PyTorch are designed to, and come with pre-trained weights, to work with RGB images. Besides that, the vision module of these libraries are also stuck to RGB files. …

Neural nets can be both, threatening and "passionate" at the same time. The more you learn and the more you go deep into it, you unleashes its power and a whole bunch of new ideas and applications come to our minds. Environmental applications are amongst those subjects that attracts more attention daily and remote sensing is one powerful tool for researchers, students and policy makers to understand environmental processes.

But, how could we bridge between neural nets and remote sensing into something useful? A quick research on the internet will show you up many ideas. The purpose of my first Medium publication, is to provide a simple introductory notebook on using Convolutional Neural Networks (CNNs) to segment clouds from satellite images. …