GHOST: Gaussian Hypothesis Open-Set Technique

In this paper, we attack a few-shot open-set recognition (FSOSR) problem, which is a combination of few-shot learning (FSL) and open-set recognition (OSR). It aims to quickly adapt a model to a given small set of labeled samples while rejecting unseen class samples. Since OSR requires rich data and FSL considers closed-set classification, existing OSR and FSL methods show poor performances in solving FSOSR problems. The previous FSOSR method follows the pseudo-unseen class sample-based methods, which collect pseudo-unseen samples from the other dataset or synthesize samples to model unseen class representations. However, this approach is heavily dependent on the composition of the pseudo samples. In this paper, we propose a novel unknown class sample detector, named SnaTCHer, that does not require pseudo-unseen samples. Based on the transformation consistency, our method measures the difference between the transformed prototypes and a modified prototype set. The modified set is composed by replacing a query feature and its predicted class prototype. SnaTCHer rejects samples with large differences to the transformed prototypes. Our method alters the unseen class distribution estimation problem to a relative feature transformation problem, independent of pseudo-unseen class samples. We investigate our SnaTCHer with various prototype transformation methods and observe that our method consistently improves unseen class sample detection performance without closed-set classification reduction.

Open set recognition requires models to recognize samples of known classes learned in the training set while reject unknowns not learned. Compared with the structural risk minimization theory for closed-set problems, structural risk in open set tasks remains rarely explored. In this paper, we point out that balancing between structural risk and open space risk is crucial for open set recognition, and re-formalize it as open set structural risk. This brings a new view towards the general relationship between closed set recognition and open set recognition against the common intuition, which argues that a good closed set classifier always benefits for open set recognition. Specifically, we theoretically and experimentally show that recent mix-based data augmentation methods are aggressive closed set regularization methods, which reduce structural risk at cost of sacrificing open space risk. Besides, we show that existing negative data augmentation designed for open space risk reduction also ignore the trade-off problem between structural risk and open space risk, which limits their performance. We propose an efficient negative data augmentation strategy named self-mix and a corresponding method named OpenMix. OpenMix generates high-quality negative samples by mixing samples themselves, which can take care of both risks simultaneously. When combining OpenMix with conservative closed set regularization methods to form OpenMix+, models can achieve lower open set structural risk. Extensive experiments validate the superiority of OpenMix and OpenMix+ in terms of both effectiveness and universality.

Image classification methods are usually trained to perform predictions taking into account a predefined group of known classes. Real-world problems, however, may not allow for a full knowledge of the input and label spaces, making failures in recognition a hazard to deep visual learning. Open set recognition methods are characterized by the ability to correctly identify inputs of known and unknown classes. In this context, we propose GeMOS: simple and plug-and-play open set recognition modules that can be attached to pretrained Deep Neural Networks for visual recognition. The GeMOS framework pairs pre-trained Convolutional Neural Networks with generative models for open set recognition to extract open set scores for each sample, allowing for failure recognition in object recognition tasks. We conduct a thorough evaluation of the proposed method in comparison with state-of-the-art open set algorithms, finding that GeMOS either outperforms or is statistically indistinguishable from more complex and costly models.

In Synthetic Aperture Radar (SAR) image classification tasks, the performance depends on both the classifier and the dataset itself. However, in comparison with plenty of SAR classification methods, there is little work aimed at analyzing the distinguishability of the dataset. In the classification dataset, some classes are semantically different but their distinguishability is low, the classes are hard to be classified especially in some more practical cases that there are unknown classes without supervision exist. Referring to open set recognition (OSR), in this paper, we proposed the SAR Distinguishability Analysor (SAR-DA) to evaluate the distinguishability of the OpenSARUrban dataset. By modeling each class as a multivariate Gaussian distribution in latent space, SAR-DA can not only classify the classes having been seen in training phase, but also can recognize unknown samples if a test sample is out of each known distribution. Each class in OpenSARUr-ban is set unknown in turn, then we apply the SAR-DA on the split dataset in OSR and supervised setting. The distinguishability can be reflected by the unknown recognition recall rate. The experimental results show that the unknown recognition recall rate in OSR setting significantly decreased compared with those in supervised setting, indicating that even though the classes in OpenSARUrban are semantically different from each other, the latent distributions of some classes are quite similar and hard to be classified, thus these classes are of low distinguishability.

Interest in emerging diseases is increasing due to recent global outbreaks like COVID-19. Unlike general image classification tasks, medical imaging is a multi-label classification that can have multiple diseases simultaneously. Then, suppose the results of all classes do not exceed the thresholds. In that case, it is classified as normal rather than unknown. This is why the existing open-set recognition (OSR) methods cannot be applied to open-set medical diagnosis. In this paper, we propose a novel open-set medical diagnosis method to solve the fundamental problem of OSR in multi-label classification. To solve this problem, we employ Copycat and entropy-based thresholds. To our knowledge, open-set multi-label medical diagnosis has not yet been addressed in research. We conduct experiments to confirm that our proposed method performs well in both multi-label classification and recognizing normal and unknown.

Spatial Location Constraint Prototype Loss for Open Set Recognition

Spatial Location Constraint Prototype Loss for Open Set Recognition

Generalized zero-shot learning (GZSL) aims to classify samples under the assumption that some classes are not observable during training. To bridge the gap between the seen and unseen classes, most GZSL methods attempt to associate the visual features of seen classes with attributes or to generate unseen samples directly. Nevertheless, the visual features used in the prior approaches do not necessarily encode semantically related information that the shared attributes refer to, which degrades the model generalization to unseen classes. To address this issue, in this paper, we propose a novel semantics disentangling framework for the generalized zero-shot learning task (SDGZSL), where the visual features of unseen classes are firstly estimated by a conditional VAE and then factorized into semantic-consistent and semantic-unrelated latent vectors. In particular, a total correlation penalty is applied to guarantee the independence between the two factorized representations, and the semantic consistency of which is measured by the derived relation network. Extensive experiments conducted on four GZSL benchmark datasets have evidenced that the semantic-consistent features disentangled by the proposed SDGZSL are more generalizable in tasks of canonical and generalized zero-shot learning. Our source code is available at https://github.com/uqzhichen/SDGZSL.

It is important to predict critical alarms in the manufacturing process that can reduce the utilization rate of the production facility. In recent years, deep neural networks have been widely used for prediction of alarm types in the manufacturing processes. However, the existing deep neural network classifiers follow a closed-set assumption that all predictable categories should be learned in the training stage. For this reason, when an unknown alarm type comes into the classifier, it should be classified as one of the predefined alarm types. In the actual manufacturing processes, it is extremely difficult to collect all possible types of alarm data. Therefore, a model with open set recognition is required to identify unknown alarm types. In addition, because the alarm type data collected from production facilities occurs simultaneously, a multi-label classification model is necessary. In this study, we propose a multi-label open set recognition model combined with background data that can improve the ability to identify unknown alarm types. We demonstrated the usefulness and applicability of the proposed method by comparing it with existing open set classification methods using real process data obtained from an automobile industry.

This paper describes an algorithm for reducing the amount of arithmetic operations in the likelihood computation of continuous mixture HMM (CMHMM) with diagonal covariance matrices while retaining high performance. The key points are the use of the scalar quantization of the input observation vector components and table look-up. These make multiplication, squaring and division operations entirely unnecessary in the whole HMM computation (i.e., output probability calculation and trellis/Viterbi computation). It is experimentally proved in an large-vocabulary isolated word recognition task that scalar quantization into no less than 16 levels does not cause significant degradation in the speech recognition performance. Scalar quantization is also utilized in the computation truncation for unlikely distributions; the total number of distribution likelihood computations can be reduced by 66% with only a slight performance degradation. This multiplication-free HMM algorithm has high potentiality in speech recognition applications on personal computers.

This paper introduced an approach to design and implement an embedded SoPC (System on a Programmable Chip) technique with Altera Nios II processor for real-time speech recognition system by developing hardware/software with minimum usage of resources (hardware components) and relatively small size software to reduce memory utilization. This is achieved by using Mel Frequency Cepstral Coefficients (MFCCs) technique as speech signal feature extraction (observation vector). Using Gaussian Mixture Model (GMM) to model the observation vector of voice information. Finally, this model passed to the Hidden Markov Model (HMM) as probabilistic model to process the GMM statistically to make decision on utterance words recognition, whether a single or composite, one or more syllable words (i.e. one, six, target). The total framework was implemented on Altera Cyclone II EP2C70F896C6N FPGA chip sitting on ALTERA DE2-70 Development Board. The utility software which are used as tools for design and development hardware/software are Quartus II 11.0sp1 (32-Bit) and Nios II 11.0spl IDE/C++ respectively. Each word model (template) stored as Transition Matrix, Diagonal Covariance Matrices, and Mean Vectors in the system memory. Each word model utilizes only 4.45Kbytes regardless of the spoken word length. Accuracy of the recognition words (digit/0 to digit/10) given 100% for the individual speaker. Training and recognition software has size (code + initialized data) equal to 312Kbytes.

Exploring similarity-based classification of larynx disorders from human voice

In the field of speaker recognition, the Gaussian mixture model with diagonal covariance matrices is a popular technique, in this way, it simplified model and reduced the amount of computation, but lost the correlation information between feature vectors, and then influenced the classification performance. In this paper, in order to compensate the correlation between feature elements, we proposed a novel method based on clustering transformation algorithm, we calculate the similarity between Gaussian components, and the cluster of same components will share one transformation matrix, thus multi-transformation matrices, together with weights and means vectors are obtained simultaneously by maximum likelihood estimation. Theory analysis and experimental results demonstrated that this proposed method can get a better balance between training speed and recognition rate, improve the performance of classifier and reduce the complexity and memory burden relatively.

In the statistical approach to offline handwritten numeral recognition, we use the Gaussian mixture model (GMM) to approximate arbitrary class conditional probability density. For simplification, the GMM is assumed to be diagonal covariance matrices. In the case of the features of handwritten numerals being correlated statistically, a large number of mixture components are usually needed to obtain a good approximation. To solve this problem, the feature vectors are first transformed to the space spanned by the eigenvectors of the covariance matrix so that the correlation among the elements is reduced, namely orthogonal transformation. This GMM is defined as orthogonal Gaussian mixture model (OGMM). Finally, the effectiveness of this algorithm is demonstrated by applying it to the NIST database.

Current deep learning methods are regarded as favorable if they empirically perform well on dedicated test sets. This mentality is seamlessly reflected in the resurfacing area of continual learning, where consecutively arriving data is investigated. The core challenge is framed as protecting previously acquired representations from being catastrophically forgotten. However, comparison of individual methods is nevertheless performed in isolation from the real world by monitoring accumulated benchmark test set performance. The closed world assumption remains predominant, i.e. models are evaluated on data that is guaranteed to originate from the same distribution as used for training. This poses a massive challenge as neural networks are well known to provide overconfident false predictions on unknown and corrupted instances. In this work we critically survey the literature and argue that notable lessons from open set recognition, identifying unknown examples outside of the observed set, and the adjacent field of active learning, querying data to maximize the expected performance gain, are frequently overlooked in the deep learning era. Hence, we propose a consolidated view to bridge continual learning, active learning and open set recognition in deep neural networks. Finally, the established synergies are supported empirically, showing joint improvement in alleviating catastrophic forgetting, querying data, selecting task orders, while exhibiting robust open world application.