An Efficient Prototype-Based Model for Handwritten Text Recognition with Multi-loss Fusion

To achieve high scene classification performance of high spatial resolution remote sensing images (HSR-RSIs), it is important to learn a discriminative space in which the distance metric can precisely measure both similarity and dissimilarity of features and labels between images. While the traditional metric learning methods focus on preserving interclass separability, label consistency (LC) is less involved, and this might degrade scene images classification accuracy. Aiming at considering intraclass compactness in HSR-RSIs, we propose a discriminative distance metric learning method with LC (DDML-LC). The DDML-LC starts from the dense scale invariant feature transformation features extracted from HSR-RSIs, and then uses spatial pyramid maximum pooling with sparse coding to encode the features. In the learning process, the intraclass compactness and interclass separability are enforced while the global and local LC after the feature transformation is constrained, leading to a joint optimization of feature manifold, distance metric, and label distribution. The learned metric space can scale to discriminate out-of-sample HSR-RSIs that do not appear in the metric learning process. Experimental results on three data sets demonstrate the superior performance of the DDML-LC over state-of-the-art techniques in HSR-RSI classification.

Salient object detection (SOD) is an important task in computer vision that aims to identify visually conspicuous regions in images. RGB-Thermal SOD combines two spectra to achieve better segmentation results. However, most existing methods for RGB-T SOD use boundary maps to learn sharp boundaries, which lead to sub-optimal performance as they ignore the interactions between isolated boundary pixels and other confident pixels. To address this issue, we propose a novel position-aware relation learning network (PRLNet) for RGB-T SOD. PRLNet explores the distance and direction relationships between pixels by designing an auxiliary task and optimizing the feature structure to strengthen intra-class compactness and inter-class separation. Our method consists of two main components: A signed distance map auxiliary module (SDMAM), and a feature refinement approach with direction field (FRDF). SDMAM improves the encoder feature representation by considering the distance relationship between foreground-background pixels and boundaries, which increases the inter-class separation between foreground and background features. FRDF rectifies the features of boundary neighborhoods by exploiting the features inside salient objects. It utilizes the direction relationship of object pixels to enhance the intra-class compactness of salient features. In addition, we constitute a transformer-based decoder to decode multispectral feature representation. Experimental results on three public RGB-T SOD datasets demonstrate that our proposed method not only outperforms the state-of-the-art methods, but also can be integrated with different backbone networks in a plug-and-play manner. Ablation study and visualizations further prove the validity and interpretability of our method.

Deep residual networks (ResNets) can learn deep feature representation from hyperspectral images (HSIs), and therefore have been widely used for HSI classification. Despite their high accuracies, there still exist a lot of challenging cases, such as open world recognition, limited-sample learning and visualization of learned classification features, which cannot be well addressed. Most of the challenges in HSI classification can be attributed to the dependence on softmax based loss function and classifiers, which cause the lack of robustness for deep learning models and the hardness to visualize the learned classification features. To improve the robustness and achieve the visualization of learned classification features, we propose a novel learning framework called Residual Prototype Learning Network, a combination of residual network and prototypes learning mechanism. Under the framework, a prototype learning based loss function is proposed to enhance intra-class compactness and the inter-class separation of these feature representations; in addition, a prototype learning based classifier is simultaneously proposed to achieve the 2D or 3D visualization of the classification features. The effectiveness of our proposed learning framework is evaluated on several publicly available HSI benchmarks, and the experimental results show that our approach achieve better results than traditional softmax based ResNets.

Fisher discrimination dictionary sparse learning (FDDL) has led to interesting image recognition results where the Fisher discrimination criterion is subject to the coding coefficients. But Fisher discrimination criterion has the limitations of data distribution assumptions and does not consider the local manifold structure of the coding coefficients. In this paper, we will introduce a novel Fisher discrimination sparse learning based on graph embedding (GE-FDSL) scheme. First, we utilizes graph embedding framework to define intra-class compact matrix and inter-class separable matrix imposed on the coding coefficients of training samples to preserving the intra-class compactness and the inter-class separability for the training samples, which simultaneously consider the local manifold structure and label information of the coding coefficients. Then, a new Fisher discrimination criterion based on graph embedding is added to the object function of the sparse coding problem so that the coding coefficients have more discriminative power, where the dictionary atoms in the sparse coding model are associated with the class labels so that the reconstructed error is applied to classification. This method can learn a structured dictionary and sparse coefficients, and in the meantime, it will also keep the local manifold structure of the coding coefficients. So, they will be more discriminative. Experiments on many image databases show that the our algorithm has good classification and recognition performance.

Deep learning has been successfully applied in classification of white blood cells (WBCs), however, accuracy and processing time are found to be less than optimal hindering it from getting its full potential. This is due to imbalanced dataset, intra-class compactness, inter-class separability and overfitting problems. The main research idea is to enhance the classification and prediction accuracy of blood images while lowering processing time through the use of deep convolutional neural network (DCNN) architecture by using the modified loss function. The proposed system consists of a deep neural convolution network (DCNN) that will improve the classification accuracy by using modified loss function along with regularization. Firstly, images are pre-processed and fed through DCNN that contains different layers with different activation function for the feature extraction and classification. Along with modified loss function with regularization, weight function aids in the classification of WBCs by considering weights of samples belonging to each class for compensating the error arising due to imbalanced dataset. The processing time will be counted by each image to check the time enhancement. The classification accuracy and processing time are achieved using the dataset-master. Our proposed solution obtains better classification performance in the given dataset comparing with other previous methods. The proposed system enhanced the classification accuracy of 98.92% from 96.1% and a decrease in processing time from 0.354 to 0.216 s. Less time will be required by our proposed solution for achieving the model convergence with 9 epochs against the current convergence time of 13.5 epochs on average, epoch is the formation white blood cells (WBCs) and the development of granular cells. The proposed solution modified loss function to solve the adverse effect caused due to imbalance dataset by considering weight and use regularization technique for overfitting problem.

Orthogonal neighborhood preserving discriminant analysis with patch embedding for face recognition

Automatic medical image segmentation plays a crucial role in many medical applications, such as disease diagnosis and treatment planning. Existing deep learning based models usually regarded the segmentation task as pixel-wise classification and neglected the semantic correlations of pixels across different images, leading to vague feature distribution. Moreover, pixel-wise annotated data is rare in medical domain, and the scarce annotated data usually exhibits the biased distribution against the desired one, hindering the performance improvement under the supervised learning setting. In this paper, we propose a novel Labeled-to-unlabeled Distribution Translation (L2uDT) framework with Semantic-oriented Contrastive Learning (SoCL), mainly for addressing the aforementioned issues in medical image segmentation. In SoCL, a semantic grouping module is designed to cluster pixels into a set of semantically coherent groups, and a semantic-oriented contrastive loss is advanced to constrain group-wise prototypes, so as to explicitly learn a feature space with intra-class compactness and inter-class separability. We then establish a L2uDT strategy to approximate the desired data distribution for unbiased optimization, where we translate the labeled data distribution with the guidance of extensive unlabeled data. In particular, a bias estimator is devised to measure the distribution bias, then a gradual-paced shift is derived to progressively translate the labeled data distribution to unlabeled one. Both labeled and translated data are leveraged to optimize the segmentation model simultaneously. We illustrate the effectiveness of the proposed method on two benchmark datasets, EndoScene and PROSTATEx, and our method achieves state-of-the-art performance, which clearly demonstrates its effectiveness for medical image segmentation. The source code is available at https://github.com/CityU-AIM-Group/L2uDT.

Drug–drug interactions (DDIs) are a significant source of adverse drug events and pose critical challenges to patient safety and clinical decision-making. Extracting DDIs from biomedical literature plays an essential role in pharmacovigilance, yet remains difficult due to data sparsity and high annotation costs. This study presents BioMCL-DDI, a novel few-shot learning framework that integrates meta-learning with contrastive embedding strategies to enable efficient DDI extraction under limited supervision. BioMCL-DDI jointly optimizes prototype-based classification and supervised contrastive representation learning within a unified architecture. The model captures both intra-class compactness and inter-class separability, enhancing its generalization in sparse biomedical settings. We evaluate BioMCL-DDI on three benchmark datasets: DDI-2013, DrugBank, and the more recent TAC 2018 DDI Extraction corpus. The model achieves F1 scores of 87.80% on DDI-2013, 86.00% on DrugBank, and 74.85%/74.82% on the two official test sets of TAC 2018, consistently outperforming competitive baselines. Our model significantly outperforms state-of-the-art baselines in low-resource scenarios. BioMCL-DDI provides a scalable and effective solution for DDI extraction from biomedical texts, with strong potential for integration into clinical decision support systems and biomedical knowledge bases. All our code and data have been publicly released at: https://github.com/Hero-Legend/BioMCL-DDI.

Convolutional neural networks (CNNs) have shown great advantages in computer vision fields, and loss functions are of great significance to their gradient descent algorithms. Softmax loss, a combination of cross-entropy loss and Softmax function, is the most commonly used one for CNNs. Hence, it can continuously increase the discernibility of sample features in classification tasks. Intuitively, to promote the discrimination of CNNs, the learned features are desirable when the inter-class separability and intra-class compactness are maximized simultaneously. Since Softmax loss hardly motivates this inter-class separability and intra-class compactness simultaneously and explicitly, we propose a new method to achieve this simultaneous maximization. This method minimizes the distance between features of homogeneous samples along with Softmax loss and thus improves CNNs' performance on vision-related tasks. Experiments on both visual classification and face verification datasets validate the effectiveness and advantages of our method.

In feature representation learning, robust features are expected to have intra-class compactness and inter-class separability. The traditional softmax loss concept ignores the intra-class compactness. Hence the discriminative power of deep features is weakened. This paper proposes a constrained center loss (CCL) to enable CNNs to extract robust features. Unlike the general center loss (CL) concept, class centers are analytically updated from the deep features in our formulation. In addition, we propose to use the entire training set to approximate class centers. By doing so, class centers can better capture the global information of feature space. To improve training efficiency, an alternative algorithm is proposed to optimize the joint supervision of softmax loss and CCL. Experiments are performed on four benchmark datasets. The results demonstrate that the proposed scheme outperforms several existing architectures.

Local feature embedding considers two constraints: intra-image spatial and inter-image feature affinity in the embedding process. However, it does not work well for the image classification task when the images are with intra-class variation, background clutter, etc. In this paper, we enhance the manifold structure by adding the class label of images into the embedding process. Since class labels are used in the training, our method can be considered as supervised. Four constituents are included in our model: feature consistency, spatial consistency, intra-class compactness and inter-class separability. With the defined Hausdorff distance between two images, different classifiers are exploited for classification. Extensive experiments on seven datasets demonstrate the effectiveness of our proposed image classification model.

Soft subspace clustering (SSC), which analyzes high-dimensional data and applies various weights to each cluster class to assess the membership degree of each cluster to the space, has shown promising results in recent years. This method of clustering assigns distinct weights to each cluster class. By introducing spatial information, enhanced SSC algorithms improve the degree to which intraclass compactness and interclass separation are achieved. However, these algorithms are sensitive to noisy data and have a tendency to fall into local optima. In addition, the segmentation accuracy is poor because of the influence of noisy data. In this study, an SSC approach that is based on particle swarm optimization is suggested with the intention of reducing the interference caused by noisy data. The particle swarm optimization method is used to locate the best possible clustering center. Second, increasing the amount of geographical membership makes it possible to utilize the spatial information to quantify the link between different clusters in a more precise manner. In conclusion, the extended noise clustering method is implemented in order to maximize the weight. Additionally, the constraint condition of the weight is changed from the equality constraint to the boundary constraint in order to reduce the impact of noise. The methodology presented in this research works to reduce the amount of sensitivity the SSC algorithm has to noisy data. It is possible to demonstrate the efficacy of this algorithm by using photos with noise already present or by introducing noise to existing photographs. The revised SSC approach based on particle swarm optimization (PSO) is demonstrated to have superior segmentation accuracy through a number of trials; as a result, this work gives a novel method for the segmentation of noisy images.

Rotating machinery is one of the important parts of industrial production equipment, and it is of great practical significance for fault diagnosis. Aiming at the problem of classification difficulty caused by feature interleaving in complex working conditions and high-dimension of rotating machinery fault samples, a rotating machinery fault diagnosis method based on multi-structure fusion discriminative projection (MFDP) is proposed. MFDP constructed intraclass and interclass hypergraph structures with multivariate relationships, fully revealing the higher-order association information among multiple samples. Besides, a tangential graph structure of MFDP is further constructed by combining the tangential affine of local samples to preserve the local tangential information of the manifold space. In the method, a unified objective optimization model of the discriminative hypergraph structures and local tangential graph structures is developed, and by solving the model, we can obtain fault structure features with well intraclass compactness and interclass separability. Extensive experiments on the Case Western Reserve University bearing dataset and Connecticut gear dataset show that the method has a good diagnostic accuracy of rotating machinery in different working conditions.

Generalized zero-shot learning (GZSL) requires a classifier trained on seen classes that can recognize objects from both seen and unseen classes. Due to the absence of unseen training samples, the classifier tends to bias towards seen classes. To mitigate this problem, feature generation based models are proposed to synthesize visual features for unseen classes. However, these features are generated in the visual feature space which lacks of discriminative ability. Therefore, some methods turn to find a better embedding space for the classifier training. They emphasize the inter-class relationships of seen classes, leading the embedding space overfitted to seen classes and unfriendly to unseen classes. Instead, in this paper, we propose an Intra-Class Compactness Enhancement method (ICCE) for GZSL. Our ICCE promotes intra-class compactness with inter-class separability on both seen and unseen classes in the embedding space and visual feature space. By promoting the intra-class relationships but the inter-class structures, we can distinguish different classes with better generalization. Specifically, we propose a Self-Distillation Embedding (SDE) module and a Semantic-Visual Contrastive Generation (SVCG) module. The former promotes intra-class compactness in the embedding space, while the latter accomplishes it in the visual feature space. The experiments demonstrate that our ICCE outperforms the state-of-the-art methods on four datasets and achieves competitive results on the remaining dataset.

Unsupervised Domain Adaptation (UDA) aims to enhance the generalization of the learned model to other domains. The domain-invariant knowledge is transferred from the model trained on labeled source domain, e.g., video game, to unlabeled target domains, e.g., real-world scenarios, saving annotation expenses. Existing UDA methods for semantic segmentation usually focus on minimizing the inter-domain discrepancy of various levels, e.g., pixels, features, and predictions, for extracting domain-invariant knowledge. However, the primary intra-domain knowledge, such as context correlation inside an image, remains under-explored. In an attempt to fill this gap, we revisit the current pixel contrast in semantic segmentation and propose a unified pixel- and patch-wise self-supervised learning framework, called PiPa, for domain adaptive semantic segmentation that facilitates intra-image pixel-wise correlations and patch-wise semantic consistency against different contexts. The proposed framework exploits the inherent structures of intra-domain images, which: (1) explicitly encourages learning the discriminative pixel-wise features with intra-class compactness and inter-class separability, and (2) motivates the robust feature learning of the identical patch against different contexts or fluctuations. Extensive experiments verify the effectiveness of the proposed method, which obtains competitive accuracy on the two widely-used UDA benchmarks, e.g., 75.6 mIoU on GTA→Cityscapes and 68.2 mIoU on Synthia→Cityscapes. Moreover, our method is compatible with other UDA approaches to further improve the performance without introducing extra parameters.