Cloud detection in satellite images using multi-objective social spider optimization

Most machine learning algorithms assume stationary environments, require a large number of training examples in advance, and begin the learning from scratch. In contrast, humans learn in changing environments with sequential training examples and leverage prior knowledge in new situations. To deal with real-world problems in changing environments, the ability to make human-like quick responses must be developed in machines. Many researchers have presented learning systems that assume the presence of hidden context and concept drift. In particular, several systems have been proposed that use ensembles of classifiers on sequential chunks of training examples. These systems can respond to gradual changes in large-scale data streams but have problems responding to sudden changes and leveraging prior knowledge of recurring contexts. Moreover, these are not pure online learning systems. We propose an online learning system that uses an ensemble of classifiers suited to recent training examples. We use experiments to show that this system can leverage prior knowledge of recurring contexts and is robust against various noise levels and types of drift.

We present a new method of learning control policies that successfully operate under unknown dynamic models. We create such policies by leveraging a large number of training examples that are generated using a physical simulator. Our system is made of two components: a Universal Policy (UP) and a function for Online System Identification (OSI). We describe our control policy as universal because it is trained over a wide array of dynamic models. These variations in the dynamic model may include differences in mass and inertia of the robots' components, variable friction coefficients, or unknown mass of an object to be manipulated. By training the Universal Policy with this variation, the control policy is prepared for a wider array of possible conditions when executed in an unknown environment. The second part of our system uses the recent state and action history of the system to predict the dynamics model parameters mu. The value of mu from the Online System Identification is then provided as input to the control policy (along with the system state). Together, UP-OSI is a robust control policy that can be used across a wide range of dynamic models, and that is also responsive to sudden changes in the environment. We have evaluated the performance of this system on a variety of tasks, including the problem of cart-pole swing-up, the double inverted pendulum, locomotion of a hopper, and block-throwing of a manipulator. UP-OSI is effective at these tasks across a wide range of dynamic models. Moreover, when tested with dynamic models outside of the training range, UP-OSI outperforms the Universal Policy alone, even when UP is given the actual value of the model dynamics. In addition to the benefits of creating more robust controllers, UP-OSI also holds out promise of narrowing the Reality Gap between simulated and real physical systems.

Humans are natural face recognition experts, far out-performing current automated face recognition algorithms, especially in naturalistic, “in the wild” settings. However, a striking feature of human face recognition is that we are dramatically better at recognizing highly familiar faces, presumably because we can leverage large amounts of past experience with the appearance of an individual to aid future recognition. Meanwhile, the analogous situation in automated face recognition, where a large number of training examples of an individual are available, has been largely underexplored, in spite of the increasing relevance of this setting in the age of social media. Inspired by these observations, we propose to explicitly learn enhanced face representations on a per-individual basis, and we present two methods enabling this approach. By learning and operating within person-specific representations, we are able to significantly outperform the previous state-of-the-art on PubFig83, a challenging benchmark for familiar face recognition in the wild, using a novel method for learning representations in deep visual hierarchies. We suggest that such person-specific representations aid recognition by introducing an intermediate form of regularization to the problem.

Lifelong Machine Learning (or Lifelong Learning) is an advanced machine learning paradigm that learns continuously, accumulates the knowledge learned in previous tasks, and uses it to help future learning. In the process, the learner becomes more and more knowledgeable and effective at learning. This learning ability is one of the hallmarks of human intelligence. However, the current dominant machine learning paradigm learns in isolation: given a training dataset, it runs a machine learning algorithm on the dataset to produce a model. It makes no attempt to retain the learned knowledge and use it in future learning. Although this isolated learning paradigm has been very successful, it requires a large number of training examples, and is only suitable for well-defined and narrow tasks. In comparison, we humans can learn effectively with a few examples because we have accumulated so much knowledge in the past which enables us to learn with little data or effort. Lifelong learning aims to achieve this capability. As statistical machine learning matures, it is time to make a major effort to break the isolated learning tradition and to study lifelong learning to bring machine learning to new heights. Applications such as intelligent assistants, chatbots, and physical robots that interact with humans and systems in real-life environments are also calling for such lifelong learning capabilities. Without the ability to accumulate the learned knowledge and use it to learn more knowledge incrementally, a system will probably never be truly intelligent. This book serves as an introductory text and survey to lifelong learning.

In this paper, we address the task of scene image classification in sparse framework. Recent scene image datasets consist of thousands of different size images with size of the order of 106 pixels. Motivated by the fact that every image has a different size, we propose a dynamic kernel1 which works over set of feature maps obtained for an image from last convolutional pooling layer of a pre-trained CNN. The size of feature maps depends on the input image size leading to the requirement of a dynamic kernel to compute similarity score between feature maps of different images. The kernel matrix obtained by using a dynamic kernel is large in size owing to the large number of training examples. To handle this we propose to use the concept of reduced virtual features (RVFs) obtained by diagonalizing the kernel matrix. RVF is a fixed length representation of a scene image irrespective of its true size. Classification is done in sparse framework by applying block sparsity constraint over sparse coefficients using dictionary built from RVFs. The proposed approach tested over standard datasets like Vogel-Schiele, MIT-8, MIT-67 and SUN-397 yields good results.

Few-shot Learning (FSL) methods are being adopted in settings where data is not abundantly available. This is especially seen in medical domains where the annotations are expensive to obtain. Deep Neural Networks have been shown to be vulnerable to adversarial attacks. This is even more severe in the case of FSL due to the lack of a large number of training examples. In this paper, we provide a framework to make few-shot segmentation models adversarially robust in the medical domain where such attacks can severely impact the decisions made by clinicians who use them. We propose a novel robust few-shot segmentation framework, Prototypical Neural Ordinary Differential Equation (PNODE), that provides defense against gradient-based adversarial attacks. We show that our framework is more robust compared to traditional adversarial defense mechanisms such as adversarial training. Adversarial training involves increased training time and shows robustness to limited types of attacks depending on the type of adversarial examples seen during training. Our proposed framework generalises well to common adversarial attacks like FGSM, PGD and SMIA while having the model parameters comparable to the existing few-shot segmentation models. We show the effectiveness of our proposed approach on three publicly available multi-organ segmentation datasets in both in-domain and cross-domain settings by attacking the support and query sets without the need for ad-hoc adversarial training.KeywordsFew-shot segmentationNeural-ODEAdversarial robustness

Reviewing video from wireless capsule endoscopy is a tedious work that requires concentration for extended hours. Computer algorithms are sought to reduce the review time and large number of training examples, to reduce the generalization error of machine learning algorithms. Therefore efficient and scalable approaches are needed which can modify knowledge structures in an incremental fashion without having to revisit all previously processed data. This paper proposes a novel incremental learning method for SVMs. The intuition is that the samples within the skin are retained alone in training. When additional samples are provided, they will be used together with the skin of the convex hull constructed from previous dataset. This result in much less number of instances and are used in incremental steps of the training process. Experiments were performed to segment frames of capsule endoscopy videos into four sections of gastrointestinal tract.

We introduce a Convolutional Neural Network (CNN) to post-process recordings obtained by means of Background Oriented Schlieren (BOS), a popular technique to visualize compressible/convective flows. To reconstruct BOS image deformation, we devised a lightweight network (LIMA) that has comparatively fewer parameters to train, allowing the deployment of the network on embedded GPU hardware. To train the CNN, we introduce a novel strategy based on the generation of synthetic images with random, irrotational displacement field that mimic those provided by real BOS recording. This allows us to generate a large number of training examples at minimal computational cost. To assess the accuracy of the reconstructed displacement, we consider test cases consisting of synthetic images with sinusoidal displacement as well as images obtained in a real experimental study of flow past and inside a heated hollow hemisphere. By comparing the prediction of the CNN with conventional post-processing techniques such as Direct Image Correlation (DIC) and Particle Image Velocimetry (PIV) cross-correlation, we show that LIMA gives more accurate and robust results for the synthetic example. When applied to the recordings from the real experiment, all methods provide consistent deformation fields. As they offer similar or better accuracy at a fraction of the computational costs, properly designed CNNs offer a valuable alternative to conventional post-processing techniques for BOS experiments.

An important problem in pattern recognition is that of pattern classification. The objective of classification is to determine a discriminant function which is consistent with the given training examples and performs reasonably well on an unlabeled test set of examples. The degree of performance of the classifier on the test examples, known as its generalization performance, is an important issue in the design of the classifier. It has been established that a good generalization performance can be achieved by providing the learner with a sufficiently large number of discriminative training examples. However, in many domains, it is infeasible or expensive to obtain a sufficiently large training set. Various mechanisms have been proposed in literature to combat this problem. Active Learning techniques (Angluin, 1998; Seung, Opper, & Sompolinsky, 1992) reduce the number of training examples required by carefully choosing discriminative training examples. Bootstrapping (Efron, 1979; Hamamoto, Uchimura & Tomita, 1997) and other pattern synthesis techniques generate a synthetic training set from the given training set. We present some of these techniques and propose some general mechanisms for pattern synthesis. Request access from your librarian to read this chapter's full text.

Abstract. We trained two Random Forest (RF) machine learning models for cloud mask and cloud thermodynamic-phase detection using spectral observations from Visible Infrared Imaging Radiometer Suite (VIIRS) on board Suomi National Polar-orbiting Partnership (SNPP). Observations from Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) were carefully selected to provide reference labels. The two RF models were trained for all-day and daytime-only conditions using a 4-year collocated VIIRS and CALIOP dataset from 2013 to 2016. Due to the orbit difference, the collocated CALIOP and SNPP VIIRS training samples cover a broad-viewing zenith angle range, which is a great benefit to overall model performance. The all-day model uses three VIIRS infrared (IR) bands (8.6, 11, and 12 µm), and the daytime model uses five Near-IR (NIR) and Shortwave-IR (SWIR) bands (0.86, 1.24, 1.38, 1.64, and 2.25 µm) together with the three IR bands to detect clear, liquid water, and ice cloud pixels. Up to seven surface types, i.e., ocean water, forest, cropland, grassland, snow and ice, barren desert, and shrubland, were considered separately to enhance performance for both models. Detection of cloudy pixels and thermodynamic phase with the two RF models was compared against collocated CALIOP products from 2017. It is shown that, when using a conservative screening process that excludes the most challenging cloudy pixels for passive remote sensing, the two RF models have high accuracy rates in comparison to the CALIOP reference for both cloud detection and thermodynamic phase. Other existing SNPP VIIRS and Aqua MODIS cloud mask and phase products are also evaluated, with results showing that the two RF models and the MODIS MYD06 optical property phase product are the top three algorithms with respect to lidar observations during the daytime. During the nighttime, the RF all-day model works best for both cloud detection and phase, particularly for pixels over snow and ice surfaces. The present RF models can be extended to other similar passive instruments if training samples can be collected from CALIOP or other lidars. However, the quality of reference labels and potential sampling issues that may impact model performance would need further attention.

Cloud contamination is a critical source of errors in the data assimilation of hyperspectral infrared radiance (IR). Therefore, it is necessary to filter out cloudy observations. In this study, we review and summarize the principles and research progress of cloud detection methods for the hyperspectral IR in the past two decades. Based on the impact of IR data utilization on cloud detection results, cloud detection methods are categorized into five types, namely clear field-of-view (FOV) detection, clear channel detection, three-dimensional cloud detection, cloud-clearing and deep learning methods. Clear FOV methods and clear channel methods aim to identify the purely clear FOVs and spectral channels that are not affected by clouds, respectively. Cloud-clearing methods are used to reconstruct clear-column radiance for cloudy observations. Deep learning cloud detection methods can quickly learn the mapping relationship between infrared hyperspectral radiation characteristics and FOV cloud distribution from a large amount of infrared radiative information with known FOV cloud labels. In this paper, we discuss and provide an outlook on the key issues in current hyperspectral IR cloud detection. Specifically, we analyze and summarize the factors affecting cloud detection, such as surface background information, vertical cloud distribution, hyperspectral IR channel selection, improvements in cloud detection algorithms and model applicability. The results indicate the use of deep learning methods offer advantages in detection accuracy and algorithm efficiency of hyperspectral IR cloud detection.

On-board image classification has the ability to significantly impact the design of future exploration missions. Classification algorithms on Earth observing satellites could be used to point the satellite at dynamic natural phenomena (such as erupting volcanoes), tag high priority data for expedited analysis on the ground, or trigger increased compression in lower priority scenes. For high data volume Earth-observing missions utilizing a spectrometer in the visible, short-wave and infrared wavelengths, it may be acceptable to lossily compress pixels containing clouds to reduce the data downlink volume. This study will evaluate the feasibility and advantage of using an on-board cloud detection and compression algorithm. The accuracy of an algorithm will be discussed, the resulting data volume savings will be calculated and the performance of a sample algorithm on an FPGA will be characterized. The detection algorithm will be tested on sample data from a similar airborne spectrometer. It is desired that the detection algorithm minimizes the incidence of false positive cloud detection. The suggested compression involves reducing the radiometric and spectral resolution of the cloudy pixels. Providing the capability for autonomous image classification on an Earth-observing mission opens the door for more extensive classification in later mission stages and flexibility to changing mission requirements.

Clouds are an important component of weather and climate change and have a significant impact on ecology, weather forecasting and aviation safety. Cloud detection is a preprocessing step in many satellite image processing and remote sensing inversion. Clouds not only cover the subsurface, but also absorb and scatter solar radiation, which affects the remote sensing analysis and study of the surface and reduces the utilization of data. Accurate detection of cloud-covered areas in satellite images can effectively reduce the interference of cloud cover on remote sensing data application. The Fengyun-4B Satellite is the latest generation of geostationary meteorological satellites in China, with significant performance advantages. In this paper, a land cloud detection algorithm for AGRI sensors is constructed based on the existing cloud detection algorithm using the AGRI full disk data of Fengyun 4B Satellite. Through the analysis of cloud detection product quality, the algorithm threshold is optimized to significantly improve the accuracy of cloud detection. The experimental results show that the algorithm has good overall cloud detection performance in full disk imagery, especially in the thick cloud region, but there are some misclassifications in the high brightness surface region such as the Tibetan Plateau. This study expands the application scenarios of domestic geostationary meteorological satellites, and provides new technical support for meteorological monitoring and remote sensing applications.

Clouds play a major role in controlling Earth's climate, and cloud detection is a crucial step in the numerical weather prediction, and global climate models. Multi-angle imaging spectroradiometer (MLSR) and moderate resolution imaging spectroradiometer (MODIS) were launched in 1999 by clouds. However, cloud detection algorithms using either MISR or MODIS data separately do not take full advantage of the data collected by both sensors. In this paper, we propose and test two schemes to combine MISR and MODIS data for cloud detection in polar regions, Both schemes are follow-ups of a two-step polar cloud detection algorithm using MISR data: enhanced linear correlation matching classification followed by quadratic discriminate analysis (ELMS-QDA) (T, Shi et al.). The first scheme is mapping the MODIS cloud detection results to the MISR grid based on a nearest neighbor method, then only reporting the agreed pixels of the ELCMC-QDA results (from MISR) and MODIS operational results. This scheme improves the classification accuracy, but reduces the coverage of the results. Instead of combining the MISR and MODIS results directly, the second scheme uses the agreed pixels of ELCMC results and MODIS operational results as the training data for the QDA on MISR features, and output the results from the QDA. Both schemes are tested over a region where expert labels show that both MISR and MODIS operational algorithms do not work well (according to expert labels, 53% and 12.72% misclassification rates for MISR and MODIS operational algorithms respectively). The first scheme only makes 0.72% of errors, but leaves 68.72% of pixels unclassified. The second scheme reaches a 2.93% of misclassification rate, which is smaller than a 4.09% rate from ELCMC-QDA, and it provides a full coverage. Hence we propose using QDA on ELCMC and MODIS agreed pixels as an algorithm to fuse the MISR and MODIS information for the polar cloud detection.

A cloud detection algorithm for satellite radiance from the microwave temperature sounder (MWTS) on board the first satellite of the Chinese polar-orbiting Fengyun-three series (FY-3A) is proposed based on the measurements at the frequencies of 50.3 and 53.6 GHz. The cloud liquid water path index (LWP index) is calculated using the brightness temperature at these two channels. Analysis of one case carried out in January2010shows the great consistency be-tween this new algorithm result and the available liquid water path product from the Meteorological Operational satellite A (MetOp-A).In general, about 60% of the global MWTS data are considered to be contaminated by cloud by virtue of the new cloud detection algorithm. A quality control (QC) procedure is applied to MWTS measurements with emphasis on the cloud detection. The QC steps are composed of (i) channel 2 over sea ice, land and coastal field of views (FOVs); (ii) channels 2 and 3 over cloudy FOVs; and (iii) outliers with large differences between observations and model simulations. After QC, MWTS measurements of channels 2–4 agree very well with the model simulations using the National Centres for Environmental Prediction (NCEP) forecast data as radiative transfer model input; the scan biases are reduced significantly, especially at the edges of the swath; and the frequency distributions of the differences between observations and model simulations become more Gaussian-like.