Spatial Location Constraint Prototype Loss for Open Set Recognition

Spatial Location Constraint Prototype Loss for Open Set Recognition

Open set recognition requires models to recognize samples of known classes learned in the training set while reject unknowns not learned. Compared with the structural risk minimization theory for closed-set problems, structural risk in open set tasks remains rarely explored. In this paper, we point out that balancing between structural risk and open space risk is crucial for open set recognition, and re-formalize it as open set structural risk. This brings a new view towards the general relationship between closed set recognition and open set recognition against the common intuition, which argues that a good closed set classifier always benefits for open set recognition. Specifically, we theoretically and experimentally show that recent mix-based data augmentation methods are aggressive closed set regularization methods, which reduce structural risk at cost of sacrificing open space risk. Besides, we show that existing negative data augmentation designed for open space risk reduction also ignore the trade-off problem between structural risk and open space risk, which limits their performance. We propose an efficient negative data augmentation strategy named self-mix and a corresponding method named OpenMix. OpenMix generates high-quality negative samples by mixing samples themselves, which can take care of both risks simultaneously. When combining OpenMix with conservative closed set regularization methods to form OpenMix+, models can achieve lower open set structural risk. Extensive experiments validate the superiority of OpenMix and OpenMix+ in terms of both effectiveness and universality.

Image classification methods are usually trained to perform predictions taking into account a predefined group of known classes. Real-world problems, however, may not allow for a full knowledge of the input and label spaces, making failures in recognition a hazard to deep visual learning. Open set recognition methods are characterized by the ability to correctly identify inputs of known and unknown classes. In this context, we propose GeMOS: simple and plug-and-play open set recognition modules that can be attached to pretrained Deep Neural Networks for visual recognition. The GeMOS framework pairs pre-trained Convolutional Neural Networks with generative models for open set recognition to extract open set scores for each sample, allowing for failure recognition in object recognition tasks. We conduct a thorough evaluation of the proposed method in comparison with state-of-the-art open set algorithms, finding that GeMOS either outperforms or is statistically indistinguishable from more complex and costly models.

The central obstacle in Open Set Recognition (OSR) is striking a balance between minimizing classification errors on known data and managing the risks posed by open space for unknown data. To address these issues, we present three novel frameworks: the Positive–Negative Prototypes Fusion Framework (PNPFF), its adversarial extension (APNPFF), and an enhanced version, APNPFF++. The PNPFF framework incorporates multiple positive prototypes to capture intra-class variability and a single negative prototype to strengthen intra-class compactness and inter-class separation. This approach reduces the classification risk for known data while reserving space for unknowns, partially alleviating open space risk. Building on this, APNPFF and APNPFF++ use Generative Adversarial Network (GAN) and manifold mix-up methods, respectively, to simulate two parts of unknown class data, further reducing open space risk and enhancing the model’s generalization performance. Comprehensive experiments conducted across multiple benchmark datasets demonstrate the effectiveness and robustness of the proposed methods, highlighting their superiority in handling both known and unknown class recognition tasks.

In Synthetic Aperture Radar (SAR) image classification tasks, the performance depends on both the classifier and the dataset itself. However, in comparison with plenty of SAR classification methods, there is little work aimed at analyzing the distinguishability of the dataset. In the classification dataset, some classes are semantically different but their distinguishability is low, the classes are hard to be classified especially in some more practical cases that there are unknown classes without supervision exist. Referring to open set recognition (OSR), in this paper, we proposed the SAR Distinguishability Analysor (SAR-DA) to evaluate the distinguishability of the OpenSARUrban dataset. By modeling each class as a multivariate Gaussian distribution in latent space, SAR-DA can not only classify the classes having been seen in training phase, but also can recognize unknown samples if a test sample is out of each known distribution. Each class in OpenSARUr-ban is set unknown in turn, then we apply the SAR-DA on the split dataset in OSR and supervised setting. The distinguishability can be reflected by the unknown recognition recall rate. The experimental results show that the unknown recognition recall rate in OSR setting significantly decreased compared with those in supervised setting, indicating that even though the classes in OpenSARUrban are semantically different from each other, the latent distributions of some classes are quite similar and hard to be classified, thus these classes are of low distinguishability.

Conditional feature generation for transductive open-set recognition via dual-space consistent sampling

We address the problem of open-set recognition, where the goal is to determine if a given sample belongs to one of the classes used for training a model (known classes). The main challenge in open-set recognition is to disentangle open-set samples that produce high class activations from known-set samples. We propose two techniques to force class activations of open-set samples to be low. First, we train a generative model for all known classes and then augment the input with the representation obtained from the generative model to learn a classifier. This network learns to associate high classification probabilities both when image content is from the correct class as well as when the input and the reconstructed image are consistent with each other. Second, we use self-supervision to force the network to learn more informative featues when assigning class scores to improve separation of classes from each other and from open-set samples. We evaluate the performance of the proposed method with recent open-set recognition works across three datasets, where we obtain state-of-the-art results.

Few-shot open-set recognition (FSOSR) poses a significant challenge as it requires identifying unknown classes while maintaining the classification performance of known classes, despite having limited access to labeled training samples. Current methods often employ non-directional metric-based losses to encapsulate feature attributes within the embedding space, inadvertently disregarding the potential influence of spatial distribution deviations of feature representations on open-set recognition performance. To address this, we present a novel directional metric-based method termed Bilevel Direction Preserving (BiDirP). This method incorporates two direction-preserving regularizers operating at distinct levels, specifically at the instance and prototype levels. The combined application of these two direction-preserving regularizers effectively enhances the spatial separation between prototypes of different classes and refines the classification decision boundaries, which results in an improved discriminative ability to differentiate unknown classes within a broader open space. Comprehensive experiments on public benchmarks show that BiDirP can significantly improve the detection ability of unknown classes while correctly classifying known classes.

Understanding open-set recognition by Jacobian norm and inter-class separation

SAR automatic target recognition (SAR ATR) has been widely studied in recent years. Most ATR models are designed based on the traditional closed-set assumption. This type of ATR model can only identify target categories existing in the training set, and it will result in missed detection or misclassification of unseen target categories encountered in battlefield reconnaissance, posing a potential threat. Therefore, it is of great significance to design a model that can simultaneously achieve known class classification and unknown class judgment. In addition, researchers usually use the obtained unknown class data for model re-learning to enable it to recognize new categories. However, before this process, it is necessary to manually interpret and annotate the obtained unknown class data, which undoubtedly requires a large time cost and is difficult to meet the timeliness requirements. To solve these problems, we propose a framework that integrates the open-set recognition module and the novel class discovery module. By introducing the K-contrast loss, the open-set recognition module can accurately distinguish unknown class data, classify known class data, and then transfer the known class knowledge through deep clustering for clustering annotation of unknown class data. Extensive experimental results on the MSTAR benchmark dataset demonstrate the effectiveness of the proposed methods.

Open set recognition (OSR) in radio frequency fingerprint (RFF) is critical for securing Internet of Things (IoT) systems, where previously unseen or malicious devices may attempt unauthorized access. A widely used approach treats all unknown devices as a single additional class and assumes that they will produce low confidence scores during inference. However, due to the inherently subtle and highly similar RFF features across devices, this assumption often fails, leading to high false acceptance rates. To address this challenge, we propose a novel framework, called Multiple Unknown Classes Generation (MUCG), which replaces the single-class modeling of unknowns with a more expressive structure that simulates multiple distinct unknown classes. MUCG employs a conditional generative adversarial network (cGAN) guided by ideal signal priors to produce diverse and realistic unknown samples. Furthermore, we introduce a soft label perturbation (SLP) strategy that blends label semantics using Feature-wise Linear Modulation (FiLM), encouraging the generator to embed richer feature variations. Experiments on three public IoT datasets demonstrate that MUCG consistently outperforms state-of-the-art (SOTA) methods in OSR tasks, achieving superior accuracy and robustness under varying signal conditions.

Summary form only given. The ability to recognize objects, in general, and living creatures, in particular, in photographs or video clips, is a critical enabling technology for a wide range of applications including health care, human-computer intelligent interaction, search engines for image retrieval and data mining, industrial and personal robotics, surveillance and security, and transportation. Despite almost 50 years of research, however, today's object recognition systems are still largely unable to handle the extraordinary wide range of appearances assumed by common objects [including human faces] in typical images. Some of the challenges for modern pattern recognition that have to be addressed in order to advance and make practical both detection and categorization include open set recognition, occlusion and masking, change detection and time-varying imagery, lack of enough data for training, and proper performance evaluation and error analysis. Open set recognition operates under the assumption that not all the test (unknown) probes have mates in the gallery (training set), occlusion and masking hide and disguise parts of the input, image contents vary across both the spatial and temporal dimensions, the amount of data available for learning and adaptation is limited, and errors are not uniformly distributed across patterns. The recognition-by-parts approach proposed here to address the challenges listed above is driven by transduction and boosting. Transduction employs local estimation and inference to find a compatible labeling of joined training and test data. Active learning further promotes the recognition process by making incremental choices about what is best to learn and when in order to accumulate the evidence needed to disambiguate among alternative interpretations. The interplay between labeled ("training") and unlabeled ("test"') data points mediates between semi-supervised learning and transduction. The additional information coming from the unlabeled data points includes consn-aints and hints about the meaningful relations and regularities affecting their very discrimination. Boosting combines in an iterative fashion part-based, model-free, and non-parametric simple weak classifiers, whose contents and relative ranking are driven by their "strangeness" characteristics. The scope of the proposed approach covers also stream-based data points and includes change detection. The benefits of the proposed discriminative recognition-by-parts approach include a priori setting of rejection thresholds, no need for image segmentation, robustness to occlusion, clutter, and disguise. Examples drawn from biometrics illustrate the proposed approach and show its feasibility and utility.

This paper proposes a method to use deep neural networks as end-to-end open-set classifiers. It is based on intra-class data splitting. In open-set recognition, only samples from a limited number of known classes are available for training. During inference, an open-set classifier must reject samples from unknown classes while correctly classifying samples from known classes. The proposed method splits given data into typical and atypical normal subsets by using a closed-set classifier. This enables to model the abnormal classes by atypical normal samples. Accordingly, the open-set recognition problem is reformulated into a traditional classification problem. In addition, a closed-set regularization is proposed to guarantee a high closed-set classification performance. Intensive experiments on five well-known image datasets showed the effectiveness of the proposed method which outperformed the baselines and achieved a distinct improvement over the state-of-the-art methods.

Current deep learning methods are regarded as favorable if they empirically perform well on dedicated test sets. This mentality is seamlessly reflected in the resurfacing area of continual learning, where consecutively arriving data is investigated. The core challenge is framed as protecting previously acquired representations from being catastrophically forgotten. However, comparison of individual methods is nevertheless performed in isolation from the real world by monitoring accumulated benchmark test set performance. The closed world assumption remains predominant, i.e. models are evaluated on data that is guaranteed to originate from the same distribution as used for training. This poses a massive challenge as neural networks are well known to provide overconfident false predictions on unknown and corrupted instances. In this work we critically survey the literature and argue that notable lessons from open set recognition, identifying unknown examples outside of the observed set, and the adjacent field of active learning, querying data to maximize the expected performance gain, are frequently overlooked in the deep learning era. Hence, we propose a consolidated view to bridge continual learning, active learning and open set recognition in deep neural networks. Finally, the established synergies are supported empirically, showing joint improvement in alleviating catastrophic forgetting, querying data, selecting task orders, while exhibiting robust open world application.

We propose a generalized Sparse Representation-based Classification (SRC) algorithm for open set recognition where not all classes presented during testing are known during training. The SRC algorithm uses class reconstruction errors for classification. As most of the discriminative information for open set recognition is hidden in the tail part of the matched and sum of non-matched reconstruction error distributions, we model the tail of those two error distributions using the statistical Extreme Value Theory (EVT). Then we simplify the open set recognition problem into a set of hypothesis testing problems. The confidence scores corresponding to the tail distributions of a novel test sample are then fused to determine its identity. The effectiveness of the proposed method is demonstrated using four publicly available image and object classification datasets and it is shown that this method can perform significantly better than many competitive open set recognition algorithms.