Torchosr -- a PyTorch extension package for Open Set Recognition models evaluation in Python

Many application scenarios for image recognition require learning of deep networks from small sample sizes in the order of a few hundred samples per class. Then, avoiding overfitting is critical. Common techniques to address overfitting are transfer learning, reduction of model complexity and artificial enrichment of the available data by, e.g., data augmentation. A key idea proposed in this paper is to incorporate additional samples into the training that do not belong to the classes of the target task. This can be accomplished by formulating the original classification task as an open set classification task. While the original closed set classification task is not altered at inference time, the recast as open set classification task enables the inclusion of additional data during training. Hence, the original closed set classification task is augmented with an open set task during training. We therefore call the proposed approach open set task augmentation. In order to integrate additional task-unrelated samples into the training, we employ the entropic open set loss originally proposed for open set classification tasks and also show that similar results can be obtained with a modified sum of squared errors loss function. Learning with the proposed approach benefits from the integration of additional “unknown” samples, which are often available, e.g., from open data sets, and can then be easily integrated into the learning process. We show that this open set task augmentation can improve model performance even when these additional samples are rather few or far from the domain of the target task. The proposed approach is demonstrated on two exemplary scenarios based on subsets of the ImageNet and Food-101 data sets as well as with several network architectures and two loss functions. We further shed light on the impact of the entropic open set loss on the internal representations formed by the networks. Open set task augmentation is particularly valuable when no additional data from the target classes are available—a scenario often faced in practice.

Image classification methods are usually trained to perform predictions taking into account a predefined group of known classes. Real-world problems, however, may not allow for a full knowledge of the input and label spaces, making failures in recognition a hazard to deep visual learning. Open set recognition methods are characterized by the ability to correctly identify inputs of known and unknown classes. In this context, we propose GeMOS: simple and plug-and-play open set recognition modules that can be attached to pretrained Deep Neural Networks for visual recognition. The GeMOS framework pairs pre-trained Convolutional Neural Networks with generative models for open set recognition to extract open set scores for each sample, allowing for failure recognition in object recognition tasks. We conduct a thorough evaluation of the proposed method in comparison with state-of-the-art open set algorithms, finding that GeMOS either outperforms or is statistically indistinguishable from more complex and costly models.

Open set recognition requires models to recognize samples of known classes learned in the training set while reject unknowns not learned. Compared with the structural risk minimization theory for closed-set problems, structural risk in open set tasks remains rarely explored. In this paper, we point out that balancing between structural risk and open space risk is crucial for open set recognition, and re-formalize it as open set structural risk. This brings a new view towards the general relationship between closed set recognition and open set recognition against the common intuition, which argues that a good closed set classifier always benefits for open set recognition. Specifically, we theoretically and experimentally show that recent mix-based data augmentation methods are aggressive closed set regularization methods, which reduce structural risk at cost of sacrificing open space risk. Besides, we show that existing negative data augmentation designed for open space risk reduction also ignore the trade-off problem between structural risk and open space risk, which limits their performance. We propose an efficient negative data augmentation strategy named self-mix and a corresponding method named OpenMix. OpenMix generates high-quality negative samples by mixing samples themselves, which can take care of both risks simultaneously. When combining OpenMix with conservative closed set regularization methods to form OpenMix+, models can achieve lower open set structural risk. Extensive experiments validate the superiority of OpenMix and OpenMix+ in terms of both effectiveness and universality.

Most of the progresses achieved in radar high range resolution profile (HRRP) recognition rely on the closed set condition, where the test sample is from a known class. In realistic scenario, however, the test sample may be drawn from unknown classes, which is regarded as an open set recognition task. In such cases, conventional recognition algorithms will inevitably make a wrong prediction. In this paper, the open set problem is addressed by incorporating the extreme value theory (EVT) into the random forest (RF) classifier. The outputs of RF are analyzed to determine whether the test sample should be rejected as an unknown class. At the training phase, a Weibull-based extreme-value meta-recognition is introduced to describe the statistical characteristics of the known classes. At the testing phase, a probability estimation method is introduced to compute the probabilities of the test sample belonging to known and unknown classes based on trained Weibull distributions. The test sample is assigned to the class of highest probability. Experimental results demonstrate that the proposed method outperforms the state-of-art NN, 1-vs-set machine and W-SVM in rejecting unknown classes.

In the unsupervised open set domain adaptation (UOSDA), the target domain contains unknown classes that are not observed in the source domain. Researchers in this area aim to train a classifier to accurately: 1) recognize unknown target data (data with unknown classes) and 2) classify other target data. To achieve this aim, a previous study has proven an upper bound of the target-domain risk, and the open set difference, as an important term in the upper bound, is used to measure the risk on unknown target data. By minimizing the upper bound, a shallow classifier can be trained to achieve the aim. However, if the classifier is very flexible [e.g., deep neural networks (DNNs)], the open set difference will converge to a negative value when minimizing the upper bound, which causes an issue where most target data are recognized as unknown data. To address this issue, we propose a new upper bound of target-domain risk for UOSDA, which includes four terms: source-domain risk, ϵ -open set difference ( ∆ϵ ), distributional discrepancy between domains, and a constant. Compared with the open set difference, ∆ϵ is more robust against the issue when it is being minimized, and thus we are able to use very flexible classifiers (i.e., DNNs). Then, we propose a new principle-guided deep UOSDA method that trains DNNs via minimizing the new upper bound. Specifically, source-domain risk and ∆ϵ are minimized by gradient descent, and the distributional discrepancy is minimized via a novel open set conditional adversarial training strategy. Finally, compared with the existing shallow and deep UOSDA methods, our method shows the state-of-the-art performance on several benchmark datasets, including digit recognition [modified National Institute of Standards and Technology database (MNIST), the Street View House Number (SVHN), U.S. Postal Service (USPS)], object recognition (Office-31, Office-Home), and face recognition [pose, illumination, and expression (PIE)].

It is important to predict critical alarms in the manufacturing process that can reduce the utilization rate of the production facility. In recent years, deep neural networks have been widely used for prediction of alarm types in the manufacturing processes. However, the existing deep neural network classifiers follow a closed-set assumption that all predictable categories should be learned in the training stage. For this reason, when an unknown alarm type comes into the classifier, it should be classified as one of the predefined alarm types. In the actual manufacturing processes, it is extremely difficult to collect all possible types of alarm data. Therefore, a model with open set recognition is required to identify unknown alarm types. In addition, because the alarm type data collected from production facilities occurs simultaneously, a multi-label classification model is necessary. In this study, we propose a multi-label open set recognition model combined with background data that can improve the ability to identify unknown alarm types. We demonstrated the usefulness and applicability of the proposed method by comparing it with existing open set classification methods using real process data obtained from an automobile industry.

Interest in emerging diseases is increasing due to recent global outbreaks like COVID-19. Unlike general image classification tasks, medical imaging is a multi-label classification that can have multiple diseases simultaneously. Then, suppose the results of all classes do not exceed the thresholds. In that case, it is classified as normal rather than unknown. This is why the existing open-set recognition (OSR) methods cannot be applied to open-set medical diagnosis. In this paper, we propose a novel open-set medical diagnosis method to solve the fundamental problem of OSR in multi-label classification. To solve this problem, we employ Copycat and entropy-based thresholds. To our knowledge, open-set multi-label medical diagnosis has not yet been addressed in research. We conduct experiments to confirm that our proposed method performs well in both multi-label classification and recognizing normal and unknown.

The use of the neural networks in the recognition of the austenitic steel types

Current deep learning methods are regarded as favorable if they empirically perform well on dedicated test sets. This mentality is seamlessly reflected in the resurfacing area of continual learning, where consecutively arriving data is investigated. The core challenge is framed as protecting previously acquired representations from being catastrophically forgotten. However, comparison of individual methods is nevertheless performed in isolation from the real world by monitoring accumulated benchmark test set performance. The closed world assumption remains predominant, i.e. models are evaluated on data that is guaranteed to originate from the same distribution as used for training. This poses a massive challenge as neural networks are well known to provide overconfident false predictions on unknown and corrupted instances. In this work we critically survey the literature and argue that notable lessons from open set recognition, identifying unknown examples outside of the observed set, and the adjacent field of active learning, querying data to maximize the expected performance gain, are frequently overlooked in the deep learning era. Hence, we propose a consolidated view to bridge continual learning, active learning and open set recognition in deep neural networks. Finally, the established synergies are supported empirically, showing joint improvement in alleviating catastrophic forgetting, querying data, selecting task orders, while exhibiting robust open world application.

Evaluations of large-scale recognition methods typically focus on overall performance. While this approach is common, it often fails to provide insights into performance across individual classes, which can lead to fairness issues and misrepresentation. Addressing these gaps is crucial for accurately assessing how well methods handle novel or unseen classes and ensuring a fair evaluation. To address fairness in Open-Set Recognition (OSR), we demonstrate that per-class performance can vary dramatically. We introduce Gaussian Hypothesis Open Set Technique (GHOST), a novel hyperparameter-free algorithm that models deep features using class-wise multivariate Gaussian distributions with diagonal covariance matrices. We apply Z-score normalization to logits to mitigate the impact of feature magnitudes that deviate from the model’s expectations, thereby reducing the likelihood of the network assigning a high score to an unknown sample. We evaluate GHOST across multiple ImageNet-1K pre-trained deep networks and test it with four different unknown datasets. Using standard metrics such as AUOSCR, AUROC and FPR95, we achieve statistically significant improvements, advancing the state-of-the-art in large-scale OSR. Source code is provided online.

Open-set deep learning enabled LIBS sensor for label-free and on-site identification of unknown pathogen powders.

Rolling bearings are key components of rotating machinery, and their health status directly affects the safety and stability of equipment operation. However, in actual industrial scenarios, due to factors such as high sensor deployment costs and low fault frequency, the number and types of fault samples are scarce, resulting in the coexistence of small sample problems and open set problems, forming the challenge of small sample open set fault diagnosis. Traditional deep learning methods are prone to overfitting when samples are insufficient, and the closed set assumption is difficult to deal with unknown faults, which limits the practicality of diagnosis. This paper takes rolling bearings as the research object and proposes an open set fault diagnosis method Dense-ProtoNet-AE under small sample conditions. Dense-ProtoNet introduces a prototype network based on CNN, realizes classification through embedded space representation and distance measurement, adopts a meta-learning framework for training, and introduces the DenseNet mechanism to enhance feature extraction capabilities, so as to realize closed set fault diagnosis of rolling bearings under small sample conditions. Autoencoder (AE) is used to design the open set recognition module, and AE is used to calculate the reconstruction error of known class samples. An adaptive multi-class 3σ method threshold selection strategy based on the validation set error is proposed to distinguish known classes from unknown classes. The known classes are then classified by Dense-ProtoNet closed set, thereby realizing small sample open set fault diagnosis.

Open set recognition (OSR) in radio frequency fingerprint (RFF) is critical for securing Internet of Things (IoT) systems, where previously unseen or malicious devices may attempt unauthorized access. A widely used approach treats all unknown devices as a single additional class and assumes that they will produce low confidence scores during inference. However, due to the inherently subtle and highly similar RFF features across devices, this assumption often fails, leading to high false acceptance rates. To address this challenge, we propose a novel framework, called Multiple Unknown Classes Generation (MUCG), which replaces the single-class modeling of unknowns with a more expressive structure that simulates multiple distinct unknown classes. MUCG employs a conditional generative adversarial network (cGAN) guided by ideal signal priors to produce diverse and realistic unknown samples. Furthermore, we introduce a soft label perturbation (SLP) strategy that blends label semantics using Feature-wise Linear Modulation (FiLM), encouraging the generator to embed richer feature variations. Experiments on three public IoT datasets demonstrate that MUCG consistently outperforms state-of-the-art (SOTA) methods in OSR tasks, achieving superior accuracy and robustness under varying signal conditions.

Skin disease is the most common disease and the fastest to infect the human body. This happens because the skin is the first organ to receive external stimuli in the form of touch, temperature and other stimuli. Skin diseases consist of several types that have almost the same color and texture with the naked eye. Thus, an approach is needed to identify the type of skin disease with the help of an image processing system, and an artificial neural network. The identification method used in this research is Convolutional Neural Network (CNN). The infected skin image is used as an input image for image processing. Before being identified, image preprocessing is done, namely resizing, grayscalling, using the Convolutional Neural Network method. The testing process in this study uses 70 types of skin disease images, validation data and 35 types of skin disease images for testing data. The results of this study are the Convolutional Neural Network method can recognize each type of skin disease image with an accuracy of 98% in the validation testing process and 85% in the testing process.
 Keywords : Skin disease, Convolutional Neural Network, Digital Image Processing

Implementation and Evaluation of Analytical Attenuation Correction for Non-uniform Attenuator