Scene Recomposition by Learning-Based ICP

Super-resolution is a fundamental problem in computer vision which aims to overcome the spatial limitation of camera sensors. While significant progress has been made in single image super-resolution, most algorithms only perform well on synthetic data, which limits their applications in real scenarios. In this paper, we study the problem of real-scene single image super-resolution to bridge the gap between synthetic data and real captured images. We focus on two issues of existing super-resolution algorithms: lack of realistic training data and insufficient utilization of visual information obtained from cameras. To address the first issue, we propose a method to generate more realistic training data by mimicking the imaging process of digital cameras. For the second issue, we develop a two-branch convolutional neural network to exploit the radiance information originally-recorded in raw images. In addition, we propose a dense channel-attention block for better image restoration as well as a learning-based guided filter network for effective color correction. Our model is able to generalize to different cameras without deliberately training on images from specific camera types. Extensive experiments demonstrate that the proposed algorithm can recover fine details and clear structures, and achieve high-quality results for single image super-resolution in real scenes.

Local feature matching using deep learning: A survey

Goal: Over the decades, there have been improvements in the sequence alignment algorithm, with significant advances in various aspects such as complexity and accuracy. However, human-defined algorithms have an explicit limitation in view of developmental completeness. This paper introduces a novel local alignment method to obtain optimal sequence alignment based on reinforcement learning. Methods: There is a DQNalign algorithm that learns and performs sequence alignment through deep reinforcement learning. This paper proposes a DQN x-drop algorithm that performs local alignment without human intervention by combining the x-drop algorithm with this DQNalign algorithm. The proposed algorithm performs local alignment by repeatedly observing the subsequences and selecting the next alignment direction until the x-drop algorithm terminates the DQNalign algorithm. This proposed algorithm has an advantage in view of linear computational complexity compared to conventional local alignment algorithms. Results: This paper compares alignment performance (coverage and identity) and complexity for a fair comparison between the proposed DQN x-drop algorithm and the conventional greedy x-drop algorithm. Firstly, we prove the proposed algorithm's superiority by comparing the two algorithms’ computational complexity through numerical analysis. After that, we tested the alignment performance actual HEV and E.coli sequence datasets. The proposed method shows the comparable identity and coverage performance to the conventional alignment method while having linear complexity for the n}{}X parameter. Conclusions: Through this study, it was possible to confirm the possibility of a new local alignment algorithm that minimizes computational complexity without human intervention.

Monocular spherical depth estimation with explicitly connected weak layout cues

This study proposes a sound event localization and detection (SELD) method using imbalanced real and synthetic data via a multi-generator. The proposed method is based on a residual convolutional neural network (RCNN) and a transformer encoder for real spatial sound scenes. SELD aims to classify the sound event, detect the onset and offset of the classified event, and estimate the direction of the sound event. In Detection and Classification of Acoustic Scenes and Events (DCASE) 2022 Task 3, SELD is performed with a few real spatial sound scene data and a relatively large number of synthetic data. When a model is trained using imbalanced data, it can proceed by focusing only on a larger number of data. Thus, a multi-generator that samples real and synthetic data at a specific rate in one batch is proposed to prevent this problem. We applied the data augmentation technique SpecAugment and used time-frequency masking to the dataset. Furthermore, we propose a neural network architecture to apply the RCNN and transformer encoder. Several models were trained with various structures and hyperparameters, and several ensemble models were obtained by “cherry-picking” specific models. Based on the experiment, the single model of the proposed method and the model applied with the ensemble exhibited improved performance compared with the baseline model.

SADGFeat: Learning local features with layer spatial attention and domain generalization

Non-coding RNAs (ncRNAs) show a unique evolutionary process in which the substitutions of distant bases are correlated in order to conserve the secondary structure of the ncRNA molecule. Therefore, the multiple alignment method for the detection of ncRNAs should take into account both the primary sequence and the secondary structure. Recently, there has been intense focus on multiple alignment investigations for the detection of ncRNAs; however, most of the proposed methods are designed for global multiple alignments. For this reason, these methods are not appropriate to identify locally conserved ncRNAs among genomic sequences. A more efficient local multiple alignment method for the detection of ncRNAs is required. We propose a new local multiple alignment method for the detection of ncRNAs. This method uses a local multiple alignment construction procedure inspired by ProDA, which is a local multiple aligner program for protein sequences with repeated and shuffled elements. To align sequences based on secondary structure information, we propose a new alignment model which incorporates secondary structure features. We define the conditional probability of an alignment via a conditional random field and use a gamma-centroid estimator to align sequences. The locally aligned subsequences are clustered into blocks of approximately globally alignable subsequences between pairwise alignments. Finally, these blocks are multiply aligned via MXSCARNA. In benchmark experiments, we demonstrate the high ability of the implemented software, SCARNA_LM, for local multiple alignment for the detection of ncRNAs. The C++ source code for SCARNA_LM and its experimental datasets are available at http://www.ncrna.org/software/scarna_lm/download. Supplementary data are available at Bioinformatics online.

In recent years, nonlinear unmixing of hyperspectral data has become an attractive topic in hyperspectral image analysis, because nonlinear models appear as more appropriate to represent photon interactions in real scenes. For this challenging problem, nonlinear methods operating in reproducing kernel Hilbert spaces have shown particular advantages. In this paper, we derive an efficient nonlinear unmixing algorithm based on a recently proposed linear mixture/ nonlinear fluctuation model. A multi-kernel learning support vector regressor is established to determine material abundances and nonlinear fluctuations. Moreover, a low complexity locally-spatial regularizer is incorporated to enhance the unmixing performance. Experiments with synthetic and real data illustrate the effectiveness of the proposed method.

The explosive growth of vehicles has increased the importance of intelligent traffic system. However, compared with face recognition, vehicle retrieval has not attracted the attention of researchers in vision community. Precise vehicle retrieval has always been a challenging task because it requires the retrieval of all vehicles with the same visual attributes from a large number of vehicles with subtle visual differences. To handle this, the authors propose to implement precise vehicle retrieval using an improved fast affine matching colour image retrieval method based on the features of annual inspection label area. Moreover, regional colour constant and hue and saturation feature are introduced to the proposed method so as to settle the illumination change problem in the real surveillance scene. To fully evaluate the proposed algorithm, they perform experiments on the ReIDcar and VehicleID datasets, which differ in data scale and image quality. The experimental results show that the presented algorithm outperforms traditional methods in vehicle retrieval. It is verified that the extracted features and the feature matching method can distinguish subtle differences between vehicles.

Dual-View Semantic Inference Network for image-text matching

Automated market makers (AMMs) have revolutionized decentralized finance (DeFi), enabling trustless asset exchange through algorithmic liquidity pools. One notable AMM is Uniswap, which improved capital efficiency by introducing concentrated liquidity, allowing Liquidity Providers (LPs) to allocate capital within specific price ranges. However, this flexibility requires active management to maximize fee generation while effectively addressing impermanent loss (IL) and gascosts. We formulate concentrated liquidity management (CLM) as a stochastic control problem and propose deep reinforcement learning (RL) strategies, specifically Proximal Policy Optimization (PPO) and Deep Q-Network (DQN), to optimize liquidity positioning. Unlike heuristics, RL agents adapt dynamically to market conditions. We adopt a two-stage synthetic-to-historical evaluation strategy, involving RL model training on synthetic data and evaluation on historical data. Results show that training on synthetic data and testing historic data allows RL-based strategies to perform well across regimes, capturing more fees while keeping net performance ahead of a passive buy-and-hold strategy. These findings highlight the robustness and practical viability of such strategies for CLM.

The centralized traffic grid signal control by the reinforcement learning method is challenging due to the difficulties of searching policy in the large state and action space. In order to solve these problems, a deep reinforcement learning (DRL) method via the action and state decomposition mechanism is proposed. We apply long short‐term memory to construct the agent which decomposes the high‐dimensional state and action space into sub‐spaces and makes decisions incrementally. This is a significant difference between our method and other methods. Through the specifically designed structure of the agent, the difficulty of searching policies can be mitigated, and our method can effectively control the traffic lights in a grid with hundreds of intersections. Experiments on synthetic data and real‐world data show that our method has better performance than traditional control methods and state‐of‐the‐art DRL‐based methods with an improvement of 21% on the queue length on synthetic data and the best travel time with an improvement of 9.35% on real‐world data.

Harvesting dense pixel-level annotations to train deep neural networks for semantic segmentation is extremely expensive and unwieldy at scale. While learning from synthetic data where labels are readily available sounds promising, performance degrades significantly when testing on novel realistic data due to domain discrepancies. We present Dual Channel-wise Alignment Networks (DCAN), a simple yet effective approach to reduce domain shift at both pixel-level and feature-level. Exploring statistics in each channel of CNN feature maps, our framework performs channel-wise feature alignment, which preserves spatial structures and semantic information, in both an image generator and a segmentation network. In particular, given an image from the source domain and unlabeled samples from the target domain, the generator synthesizes new images on-the-fly to resemble samples from the target domain in appearance and the segmentation network further refines high-level features before predicting semantic maps, both of which leverage feature statistics of sampled images from the target domain. Unlike much recent and concurrent work relying on adversarial training, our framework is lightweight and easy to train. Extensive experiments on adapting models trained on synthetic segmentation benchmarks to real urban scenes demonstrate the effectiveness of the proposed framework.

Person re-identification is a critical method in video surveillance, which has a wide range of applications. However, because of the shift in shooting posture, the same person's physical attributes can become badly misplaced, and feature matching is challenging. A pose-guided attention and alignment (PGAA) technique is presented as a solution to this. The core of this technique is a multi-branch deep network. The first branch is based on keypoint multi attention technology, filtering the image's actual information and then executing horizontal segmentation to extract local characteristics. The second branch is a method based on local alignment. The pose estimation is realized using a pre-trained pose estimation model and local feature matching at keypoints. We may employ part features while also alleviating the offset of the part feature map caused by the occlusion of the part detection frame and the erroneous position, and improving the discrimination and robustness of the part features with this technique. The method can produce good outcomes in the situation of posture change and human image misalignment, according to testing results on some popular datasets.

CorMatcher: A corners-guided graph neural network for local feature matching