BasicTAD: An astounding RGB-Only baseline for temporal action detection

Regional clustering-based spatial preprocessing for hyperspectral unmixing

Cropland classification can be used to monitor cropland distribution and its change over time. In this letter, a new superpixel-based cropland classification method is proposed for synthetic aperture radar (SAR) imagery through the integration of statistical texture, polarization, and spatial information. First, the method combines random forest algorithm and superpixels, which are generated using simple linear iterative clustering algorithm with polarization features of Pauli decomposition and spatial information. Superpixel-based spatial context information is used to reduce the influence of coherent speckle and misclassification in cropland blocks. Second, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$G^{0}$ </tex-math></inline-formula> statistical texture feature is used to reduce the interference of background targets such as woodland in cropland classification. Comparison experiments of different methods using C-band airborne SAR (AIRSAR) polarimetric data acquired in early July show that the proposed method has better classification performance, with an overall accuracy of 88.62%. The classification accuracy of corn and soybean is above 95% and 91%, respectively. The <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$G^{0}$ </tex-math></inline-formula> statistical texture feature is helpful to eliminate woodland that may cause crop misclassification using single-date SAR image.

In the context of automotive driving assistance, this paper describes a generic (i.e. applicable to both vehicle interior and exterior scenes) vision based approach for scene content analysis. It makes use of temporal and spatial information from a stereoscopic sequence of images to localize objects and estimate their position and motion. The proposed method is divided into three steps. First, image features are selected, tracked and reconstructed in the 3D world space. Second, a clustering step is processed in the 5D space made of the positions and 2D motions parameters. The last step is devoted to clusters interpretation: it is out of the scope of the paper, however orientations are given to illustrate the capabilities of the proposed approach. The paper is organized as follows: first, the use of temporal and spatial information from a stereoscopic sequence is investigated. A state of the art of existing methods is presented. Then, a generic approach for object segmentation is proposed. Lastly, experimental results are presented.

Pansharpening is an important way of integrating spatial and spectral information in the field of remote sensing. This field uses the complementary and redundant information between multispectral (MS) images and panchromatic (PAN) images to obtain high spectral and high spatial resolution images. Various pansharpening methods have been introduced so far, each one attempting to provide a pansharpened image with the least distortion and maximum preservation of spectral and spatial information. Due to the importance of this issue, there should be methods and indices to evaluate the performance of different pansharpening algorithms and assess the quality of pansharpened images. In this paper, a segmentation-based method for assessing the quality of fused images is proposed. The advantage of this approach over pixel-based methods is that the pixel-based methods consider the fused images as a set of separate pixels while segmentation can take into account useful spatial information such as neighbourhoods, textures, etc. In the proposed method, by using k-means clustering algorithm, the reference and pansharpened images are segmented into areas with similar spectral and spatial features and the corresponding segments of the images are compared. This method is tested on three real data sets acquired by Pleiades, GeoEye-1, and QuickBird sensors. Experimental results demonstrate the effectiveness of the proposed method in evaluation of the quality of fused images.

We propose a new method for learning ensembles of process-based models for predictive modeling of dynamic systems from data and knowledge. Previous work has shown that ensembles based on sampling data (i.e., bagging), significantly improve predictive performance of process-based models. However, this improvement comes at the cost of a substantial computational overhead needed for learning. On the other hand, methods for constructing ensembles based on sampling knowledge (i.e., random library samples, RLS) allow for efficient learning ensembles of process-based models, while maintaining their long-term predictive performance. This paper aims at checking the conjecture whether the combination of these methods has a potential for further performance improvements. The proposed method, bagging of random library samples for learning ensembles of process-based models combines the afore-mentioned approaches in terms of sampling both data and knowledge. We apply the method to and evaluate its performance on a set of automated predictive modeling tasks in two lake ecosystems from data and library of knowledge for modeling population dynamics. The experimental results serve both to identify the optimal design decisions regarding the proposed method as well as to asses its predictive ability. The results show that such ensembles outperform single process-based model, but also outperform each of the two methods for learning ensembles from data samples (bagging) and knowledge samples (RLS).

In this study, we consider fully automated action recognition based on deep learning in the industrial environment. In contrast to most existing methods, which rely on professional knowledge to construct complex hand-crafted features, or only use basic deep-learning methods, such as convolutional neural networks (CNNs), to extract information from images in the production process, we exploit a novel and effective method, which integrates multiple deep-learning networks including CNNs, spatial transformer networks (STNs), and graph convolutional networks (GCNs) to process video data in industrial workflows. The proposed method extracts both spatial and temporal information from video data. The spatial information is extracted by estimating the human pose of each frame, and the skeleton image of the human body in each frame is obtained. Furthermore, multi-frame skeleton images are processed by GCN to obtain temporal information, meaning the action recognition results are predicted automatically. By training on a large human action dataset, Kinetics, we apply the proposed method to the real-world industrial environment and achieve superior performance compared with the existing methods.

Light field image becomes one of the most promising media types for immersive video applications. In this paper, we propose a novel end-to-end spatial-angular-decorrelated network (SADN) for high-efficiency light field image compression. Different from the existing methods that exploit either spatial or angular consistency in the light field image, SADN decouples the angular and spatial information by dilation convolution and stride convolution in spatial-angular interaction, and performs feature fusion to compress spatial and angular information jointly. To train a stable and robust algorithm, a large-scale dataset consisting of 7549 light field images is proposed and built. The proposed method provides 2.137 times and 2.849 times higher compression efficiency relative to H.266/VVC and H.265/HEVC inter coding, respectively. It also outperforms the end-to-end image compression networks by an average of 79.6% bitrate saving with much higher subjective quality and light field consistency.

The ability to capture inter-frame dynamics has been critical to the development of video salient object detection (VSOD). While many works have achieved great success in this field, a deeper insight into its dynamic nature should be developed. In this work, we aim to answer the following questions: How can a model adjust itself to dynamic variations as well as perceive fine differences in the real-world environment; How are the temporal dynamics well introduced into spatial information over time? To this end, we propose a dynamic context-sensitive filtering network (DCFNet) equipped with a dynamic context-sensitive filtering module (DCFM) and an effective bidirectional dynamic fusion strategy. The proposed DCFM sheds new light on dynamic filter generation by extracting location-related affinities between consecutive frames. Our bidirectional dynamic fusion strategy encourages the interaction of spatial and temporal information in a dynamic manner. Experimental results demonstrate that our proposed method can achieve state-of-the-art performance on most VSOD datasets while ensuring a real-time speed of 28 fps. The source code is publicly available at https://github.com/OIPLab-DUT/DCFNet.

Saliency computation has wide applications. It is now being used in almost all vision-related applications. But, identifying saliency is still a problem. Various computational models have been proposed for identifying saliency. Global contrast-based method is used extensively. This method computes the contrast by measuring the color difference between image and specified region. It produces saliency with non-salient points, but, in the process, it loses some structural and spatial information. To address these limitations, the proposed method, i.e., Poisson-based probabilistic contrast, produces saliency with the concave topographical surface. This surface encloses the prominent object with all its structural and spatial information, or with all the salient features. Then, it is used as a reference plane for regional depth, color and spatial saliency integration. The proposed method has three stages. In the first stage, a probabilistic contrast is computed using Poisson-based maximum likelihood estimation by addition of chrominance and luminance contrast. The luminance contrast is normalized by proposed “enhance and suppress luminance method.” In the second stage, the regional color, depth, and spatial saliencies are integrated into the topographical surface to enhance the saliency. In the third and last stages, i.e., saliency enhancement stage, central saliency is used on global color distinction. The proposed method is evaluated on the publicly available datasets. Their performance is compared with 12 state-of-the-art methods. The experimental result presented here shows that the proposed method performs better.

Modern industrial plants generally consist of multiple manufacturing units, and the local correlation within each unit can be used to effectively alleviate the effect of spurious correlation and meticulously reflect the operation status of the process system. Therefore, the local correlation, which is called spatial information here, should also be taken into consideration when developing the monitoring model. In this study, a cascaded monitoring network (MoniNet) method is proposed to develop the monitoring model with concurrent analytics of temporal and spatial information. By implementing convolutional operation to each variable, the temporal information that reveals dynamic correlation of process data and spatial information that reflects local characteristics within individual operation unit can be extracted simultaneously. For each convolutional feature, a submodel is developed and then all the submodels are integrated to generate a final monitoring model. Based on the developed model, the operation status of the newly collected sample can be identified by comparing the calculated statistics with their corresponding control limits. Similar to the convolutional neural network (CNN), the MoniNet can also expand its receptive field and capture deeper information by adding more convolutional layers. Besides, the filter selection and submodel development in MoniNet can be replaced to generalize the proposed network to many existing monitoring strategies. The performance of the proposed method is validated using two real industrial processes. The illustration results show that the proposed method can effectively detect process anomalies by concurrent analytics of temporal and spatial information.

The brain can be viewed as a complex modular structure with features of information processing through knowledge storage and retrieval. Modularity ensures that the knowledge is stored in a manner where any complications in certain modules do not affect the overall functionality of the brain. Although artificial neural networks have been very promising in prediction and recognition tasks, they are limited in terms of learning algorithms that can provide modularity in knowledge representation that could be helpful in using knowledge modules when needed. Multi-task learning enables learning algorithms to feature knowledge in general representation from several related tasks. There has not been much work done that incorporates multi-task learning for modular knowledge representation in neural networks. In this paper, we present multi-task learning for modular knowledge representation in neural networks via modular network topologies. In the proposed method, each task is defined by the selected regions in a network topology (module). Modular knowledge representation would be effective even if some of the neurons and connections are disrupted or removed from selected modules in the network. We demonstrate the effectiveness of the method using single hidden layer feedforward networks to learn selected n-bit parity problems of varying levels of difficulty. Furthermore, we apply the method to benchmark pattern classification problems. The simulation and experimental results, in general, show that the proposed method retains performance quality although the knowledge is represented as modules.

Accurate localization of seizure onset zones (SOZs) in patients with drug-resistant epilepsy is essential for improving prognostic outcomes. This process can be significantly enhanced through effective network representation and analysis of functional dependencies among brain regions. However, traditional network construction methods often lack generalizability due to individual variability. Furthermore, the independent design of the network construction and analysis modules restricts the overall optimization of localization frameworks. In this study, we propose a novel deep learning framework that integrates graph building and analysis modules for seizure localization. The graph building block adaptively generates customized network representations from Stereo-Electroencephalography (SEEG) data of individual patients by extracting feature vectors of each channel and calculating functional connectivity weights among channels with these vectors. While the GCN-and-LSTM-based graph analysis block identifies abnormal nodes corresponding to SOZs by aggregating spatial and temporal information in the network representations. The graph analysis block alongside with the graph building block through the seizure prediction task. Attention weights assigned to each channel are utilized to characterize epileptogenicity, facilitating precise localization of the SOZ. Our method demonstrates superior performance, surpassing baseline and state-of-the-art approaches in 9 of 13 patients from a public dataset and 11 of 14 patients from a clinical dataset. Visualization of the identified brain regions aligns well with labeled SOZs. Furthermore, the adaptive functional brain network reveals that the connectivity density among SOZ channels is greater than that of other brain regions, corroborating existing clinical findings and reinforcing the model’s reliability and interpretability.

This paper addresses the problem of common object detection, which aims to detect objects of similar categories from a set of images. Although it shares some similarities with the standard object detection and co-segmentation, common object detection, recently promoted by \cite{Jiang2019a}, has some unique advantages and challenges. First, it is designed to work on both closed-set and open-set conditions, a.k.a. known and unknown objects. Second, it must be able to match objects of the same category but not restricted to the same instance, texture, or posture. Third, it can distinguish multiple objects. In this work, we introduce the Single Stage Common Object Detection (SSCOD) to detect class-agnostic common objects from an image set. The proposed method is built upon the standard single-stage object detector. Furthermore, an embedded branch is introduced to generate the object's representation feature, and their similarity is measured by cosine distance. Experiments are conducted on PASCAL VOC 2007 and COCO 2014 datasets. While being simple and flexible, our proposed SSCOD built upon ATSSNet performs significantly better than the baseline of the standard object detection, while still be able to match objects of unknown categories. Our source code can be found at \href{https://github.com/cybercore-co-ltd/Single-Stage-Common-Object-Detection}{(URL)}

Background and Objective. Automatic segmentation of MRI brain tumor area is a key step in the diagnosis and treatment of brain tumor. In recent years, the improved network based on UNet encoding and decoding structure has been widely used in brain tumor segmentation. However, due to continuous convolution and pooling operations, some spatial context information in existing networks will be discontinuous or even missing. It will affect the segmentation accuracy of the model. Therefore, the method proposed in this paper is to alleviate the lack of spatial context information and improve the accuracy of the model. Approach. This paper proposes a context attention module (multiscale contextual attention) to capture and filter out high-level features with spatial context information, which solves the problem of context information loss in feature extraction. The channel attention mechanism is introduced into the decoding structure to realize the fusion of high-level features and low-level features. The standard convolution block in the encoding and decoding structure is replaced by the pre-activated residual block to optimize the network training and improve the network performance. Results. This paper uses two public data sets (BraTs 2017 and BraTs 2019) to evaluate and verify the proposed method. Experimental results show that the proposed method can effectively alleviate the lack of spatial context information, and the segmentation performance is better than other existing methods. Significance. The method improves the segmentation performance of the model. It will assist doctors in making accurate diagnosis and provide reference basis for tumor resection. As a result, the proposed method will reduce the operation risk of patients and the postoperative recurrence rate.

An image segmentation method based on optimized spatial texture information is proposed in this article. Spatial information, including the relative position of neighbouring pixels and texture features of the multiscale neighbourhood, is incorporated into the similarity measure of the fuzzy c-means (FCM) clustering algorithm, in which the Gaussian kernel is adopted to diminish the local incorrect segmentation. The FCM clustering is spatially adjusted and optimized by the particle swarm optimization (PSO) algorithm. The purpose of optimization is to obtain the appropriate control parameters influencing spatial information, which can improve segmentation results. Experimental results demonstrate that the proposed method achieves better segmentation performance and is capable of effectively segmenting synthetic images and synthetic aperture radar (SAR) images.