Image vectorization using a sparse patch layout

Multimodal aspect-based sentiment analysis (MABSA) is an essential task in the field of sentiment analysis, which still confronts several critical challenges. The first challenge is how to effectively capture key information within both image and text features to enhance the recognition and understanding of complex sentiment expressions. The second challenge is how to achieve cross-modal alignment of multi-granularity text features and image features. The third challenge is how to narrow the semantic gap between image modality and text modality through effective cross-modal feature fusion. To address these issues, a framework that leverages external knowledge and multi-granularity image and text features (EKMG) is proposed. Firstly, an external knowledge enhanced semantic extraction module is introduced to fuse external knowledge with image features and text features, thereby capturing the key information from texts and images. Secondly, we design a multi-granularity image-text contrastive learning module. This module initially introduces a graph attention network and a novel cross-modal fusion mechanism to align image features and text features at multiple granularities. Additionally, the module employs an image-text contrastive learning strategy to narrow the semantic gap between different modalities. Experimental results on two public benchmark datasets demonstrate that EKMG achieves significant performance improvements compared to state-of-the-art baseline models.

With a very large size image database, there should be a way to search for the similar images efficiently. However, the traditional approaches that have been adopted to retrieve text or scalar values cannot be used because of the multi-dimensionality of the image feature vectors. A new retrieval method that uses one or more scalar values induced from the image vector is proposed. In the proposed method, the scalar values are independently stored as indices to the target feature vectors. With the proposed method, the searching speed is improved approximately 25 times compared to the sequential searching.

For the synthesis of ultra-large scene and ultra-high resolution videos, in order to obtain high-quality large-scene videos, high-quality video stitching and fusion are achieved through multi-scale unstructured array cameras. This paper proposes a network model image feature point extraction algorithm based on symmetric auto-encoding and scale feature fusion. By using the principle of symmetric auto-encoding, the hierarchical restoration of image feature location information is incorporated into the corresponding scale feature, along with deep separable convolution image feature extraction, which not only improves the performance of feature point detection but also significantly reduces the computational complexity of the network model. Based on the calculated high-precision feature point pairing information, a new image localization method is proposed based on area ratio and homography matrix scaling, which improves the speed and accuracy of the array camera image scale alignment and positioning, realizes high-definition perception of local details in large scenes, and obtains clearer synthesis effects of large scenes and high-quality stitched images. The experimental results show that the feature point extraction algorithm proposed in this paper has been experimentally compared with four typical algorithms using the HPatches dataset. The performance of feature point detection has been improved by an average of 4.9%, the performance of homography estimation has been improved by an average of 2.5%, the amount of computation has been reduced by 18%, the number of network model parameters has been reduced by 47%, and the synthesis of billion-pixel videos has been achieved, demonstrating practicality and robustness.

An enhanced time efficient technique for image watermarking using ant colony optimization and light gradient boosting algorithm

Magnetic resonance imaging plays a crucial role in clinical diagnosis due to its ability to provide information on soft tissue structure. At present, multi-contrast super-resolution (SR) methods for magnetic resonance (MR) images have been widely studied and have achieved good results. However, most studies overlook the impact of modal differences between reference and low-resolution (LR) image features on feature reconstruction, which may result in inaccurate reconstruction of detail features due to improper alignment of structural features. To address this issue, we propose a high-frequency feature transfer network for MR image SR task (HFTN-SR), which consists of two feature extraction branches and one high-frequency feature transfer branch. Considering the modal differences between the target and reference images, a high-low frequency decomposition method is used to decompose the reference image into high-frequency and low-frequency components, where the low-frequency components are used in the subsequent network to match LR target image features. A feature extraction block (FEB) is constructed to extract and integrate high-frequency and low-frequency features of the reference image, as well as features of the LR target image. In response to the modal differences between two image features, a feature transfer block (FTB) is designed to establish the correlation between the low-frequency features of the reference image and the target image features, and use the correlation matrix to transfer the high-frequency features of the reference image to the target image features. To further reduce the loss of shallow features caused by the increase in network depth, a standardized combined residual feature module (SCRFM) is constructed to supplement the shallow features of the target image into the final reconstructed features. Experiments on the public dataset FastMRI and the self-built dataset AXA show that the performance of HFTN-SR is superior to some state-of-the-art (SOTA) methods. Notably, HFTN-SR achieves the highest PSNR and SSIM scores across all tested datasets, with significant improvements in visual quality and detail reconstruction.

Traditional multimodal contrastive learning brings text and its corresponding image closer together as a positive pair, where the text typically consists of fixed sentence structures or specific descriptive statements, and the image features are generally global features (with some fine-grained work using local features). Similar to unimodal self-supervised contrastive learning, this approach can be seen as enforcing a strict identity constraint in a multimodal context. However, due to the inherent complexity of remote sensing images, which cannot be easily described in a single sentence, and the fact that remote sensing images contain rich ancillary information beyond just object features, this strict identity constraint may be insufficient. To fully leverage the characteristics of remote sensing images, we propose a multimodal contrastive learning method for remote sensing image feature extraction, based on positive sample tripartite relaxation, where the model is relaxed in three aspects. The first aspect of relaxation involves both the text and image inputs. By introducing learnable parameters in the language and image branches, instead of relying on fixed sentence structures and fixed image features, the network can achieve a more flexible description of remote sensing images in text and extract ancillary information from the image features, thereby relaxing the input constraints. Second relaxation is achieved through multimodal alignment of various features. By aligning semantic information with the corresponding semantic regions in the images, the method allows for the relaxation of local image features under semantic constraints. This approach addresses the issue of selecting image patches in unimodal settings, where there is no semantic constraint. The proposed method for remote sensing image feature extraction has been validated on four datasets. On the PatternNet dataset, it achieved a 91.1% accuracy with just one-shot.

AFC HERA simulated images along COP trajectory for Ground Navigation support and Dimorphos 3D reconstruction

In order to improve the accuracy of brain magnetic resonance imaging (MRI) classification and to reduce classification time, this paper proposes a brain MRI image classification algorithm combined with transfer learning and support vector machine (SVM). Firstly, the three deep convolutional neural networks pre-trained on the ImageNet database, AlexNet, VGG16 and GoogleNet, are used as feature extractors to extract the features of brain MRI images in the Harvard Medical School Website database. The feature extraction process does not require to fine-tune the transferred networks. Then, the features extracted in each convolutional neural network are combined to form a feature vector of each brain MRI image and it is input to the SVM for classification. Finally the SVM classified the brain MRI images into healthy, Alzheimer's disease, and stroke. The experimental results show that the classification accuracy can achieve 100%, and the classification time is only 26 seconds. Compared with the brain MRI image classification algorithm proposed by the literature, the accuracy of the proposed method is increased by 8.67%, 1.09%, 0.55%, respectively. The proposed method can provide effective help for the diagnosis of brain diseases.

Fusion-based hyperspectral image (HSI) super-resolution has become increasingly prevalent for its capability to integrate high-frequency spatial information from the paired high-resolution (HR) RGB reference (Ref-RGB) image. However, most of the existing methods either heavily rely on the accurate alignment between low-resolution (LR) HSIs and RGB images or can only deal with simulated unaligned RGB images generated by rigid geometric transformations, which weakens their effectiveness for real scenes. In this article, we explore the fusion-based HSI super-resolution with real Ref-RGB images that have both rigid and nonrigid misalignments. To properly address the limitations of existing methods for unaligned reference images, we propose an HSI fusion network (HSIFN) with heterogeneous feature extractions, multistage feature alignments, and attentive feature fusion. Specifically, our network first transforms the input HSI and RGB images into two sets of multiscale features with an HSI encoder and an RGB encoder, respectively. The features of Ref-RGB images are then processed by a multistage alignment module to explicitly align the features of Ref-RGB with the LR HSI. Finally, the aligned features of Ref-RGB are further adjusted by an adaptive attention module to focus more on discriminative regions before sending them to the fusion decoder to generate the reconstructed HR HSI. Additionally, we collect a real-world HSI fusion dataset, consisting of paired HSI and unaligned Ref-RGB, to support the evaluation of the proposed model for real scenes. Extensive experiments are conducted on both simulated and our real-world datasets, and it shows that our method obtains a clear improvement over existing single-image and fusion-based super-resolution methods on quantitative assessment as well as visual comparison. The code and dataset are publicly available at https://zeqiang-lai.github.io/HSI-RefSR/.

High quality face image plays an important role for face recognition. In this paper, we discuss various factors of bad face images in video surveillance systems, and propose a high quality face image acquisition method based on multiple cameras. Two main ideas about the multi-cameras deployment and face pose estimation are proposed. Firstly, the deployment method of multi-cameras in video surveillance systems is descripted in engineering application, and the multi-view face images can captured based on these cameras. Secondly, a face pose estimation method is proposed based on edge features of face image and support vector machines. The experiment results show that the approach we proposed can improve the quality of acquired face imaged.

Multilayer assembly is a commonly used technique to construct multilayer polydimethylsiloxane (PDMS)-based microfluidic devices with complex 3D architecture and connectivity for large-scale microfluidic integration. Accurate alignment of structure features on different PDMS layers before their permanent bonding is critical in determining the yield and quality of assembled multilayer microfluidic devices. Herein, we report a custom-built desktop aligner capable of both local and global alignments of PDMS layers covering a broad size range. Two digital microscopes were incorporated into the aligner design to allow accurate global alignment of PDMS structures up to 4 in. in diameter. Both local and global alignment accuracies of the desktop aligner were determined to be about 20 μm cm(-1). To demonstrate its utility for fabrication of integrated multilayer PDMS microfluidic devices, we applied the desktop aligner to achieve accurate alignment of different functional PDMS layers in multilayer microfluidics including an organs-on-chips device as well as a microfluidic device integrated with vertical passages connecting channels located in different PDMS layers. Owing to its convenient operation, high accuracy, low cost, light weight, and portability, the desktop aligner is useful for microfluidic researchers to achieve rapid and accurate alignment for generating multilayer PDMS microfluidic devices.

This paper proposes a method of facial expression recognition based on Zernike moments and the minimum classification error (MCE) based hidden Markov model (HMM). In the feature extraction of face, the method of Zernike moments feature extraction based on local feature regions is adopted. First, eyes and mouths are segmented from the facial expression image and Zernike moments feature vectors of eyes and mouth are extracted. In the classification of expression, using MCE criteria is adopted to improve the performance of HMM. The results show that the performance of expression recognition system can be improved to some extent by using Zernike moments and MCE criterion.

The emergence of Alzheimer's Disease (AD) as a consequence of increasing aging population makes urgent the availability of methods for the early and accurate diagnosis. Magnetic Resonance Imaging (MRI) could be used as in vivo, non invasive tool to identify sensitive and specific markers of very early AD progression. In recent years, multivariate pattern analysis (MVPA) and machine- learning algorithms have attracted strong interest within the neuroimaging community, as they allow automatic classification of imaging data with higher performance than univariate statistical analysis. An exhaustive search of PubMed, Web of Science and Medline records was performed in this work, in order to retrieve studies focused on the potential role of MRI in aiding the clinician in early diagnosis of AD by using Support Vector Machines (SVMs) as MVPA automated classification method. A total of 30 studies emerged, published from 2008 to date. This review aims to give a state-of-the-art overview about SVM for the early and differential diagnosis of AD-related pathologies by means of MRI data, starting from preliminary steps such as image pre-processing, feature extraction and feature selection, and ending with classification, validation strategies and extraction of MRI-related biomarkers. The main advantages and drawbacks of the different techniques were explored. Results obtained by the reviewed studies were reported in terms of classification performance and biomarker outcomes, in order to shed light on the parameters that accompany normal and pathological aging. Unresolved issues and possible future directions were finally pointed out.

High precision magnetic field computer programs are required to accurately calculate the multipole content and magnetic field strength associated with a given accelerator magnet design. In this work, a specialized version of the finite element computer program WEMAP (Westinghouse Electric and Magnetic Analysis Program) is applied to field harmonic calculations. The computer program relies on a higher-order finite element formulation and high-precision computer representation to minimize calculation error. Several calculations are made for a SSC CDM cross section design. The effects of mesh fineness and finite element order are investigated by examining finite element grids with increasing mesh density and element order. A key feature of this computer program is the capability to minimize the truncation error associated with a finite element mesh by increasing the order of the finite element interpolating polynomial. Finite elements of order one through six are incorporated. Inaccuracies which may occur using the lower-order finite elements with a coarse mesh are identified in this paper. Care must be taken in meshing the yoke iron region to accurately account for field harmonics due to nonuniform iron permeability. Methods of extracting the spatial harmonics from the finite element solution are discussed. The field strength and harmonics are calculated for a full-section model of a CDM magnet cross section.

Cross-domain pedestrian detection aims to generalize pedestrian detectors from one label-rich domain to another label-scarce domain, which is crucial for various real-world applications. Most recent works focus on domain alignment to train domain-adaptive detectors either at the instance level or image level. From a practical point of view, one-stage detectors are faster. Therefore, we concentrate on designing a cross-domain algorithm for rapid one-stage detectors that lacks instance-level proposals and can only perform image-level feature alignment. However, pure image-level feature alignment causes the foreground-background misalignment issue to arise, i.e., the foreground features in the source domain image are falsely aligned with background features in the target domain image. To address this issue, we systematically analyze the importance of foreground and background in image-level cross-domain alignment, and learn that background plays a more critical role in image-level cross-domain alignment. Therefore, we focus on cross-domain background feature alignment while minimizing the influence of foreground features on the cross-domain alignment stage. This paper proposes a novel framework, namely, background-focused distribution alignment (BFDA), to train domain adaptive one-stage pedestrian detectors. Specifically, BFDA first decouples the background features from the whole image feature maps and then aligns them via a novel long-short-range discriminator. Extensive experiments demonstrate that compared to mainstream domain adaptation technologies, BFDA significantly enhances cross-domain pedestrian detection performance for either one-stage or two-stage detectors. Moreover, by employing the efficient one-stage detector (YOLOv5), BFDA can reach 217.4 FPS ( 640×480 pixels) on NVIDIA Tesla V100 (7~12 times the FPS of the existing frameworks), which is highly significant for practical applications. The code from this study will be made publicly available.