A Sparse Representation Based Classification Algorithm for Chinese Food Recognition

We propose a generalized Sparse Representation-based Classification (SRC) algorithm for open set recognition where not all classes presented during testing are known during training. The SRC algorithm uses class reconstruction errors for classification. As most of the discriminative information for open set recognition is hidden in the tail part of the matched and sum of non-matched reconstruction error distributions, we model the tail of those two error distributions using the statistical Extreme Value Theory (EVT). Then we simplify the open set recognition problem into a set of hypothesis testing problems. The confidence scores corresponding to the tail distributions of a novel test sample are then fused to determine its identity. The effectiveness of the proposed method is demonstrated using four publicly available image and object classification datasets and it is shown that this method can perform significantly better than many competitive open set recognition algorithms.

The visual analysis of foods on social media by using food recognition algorithm provides valuable insight from the health, cultural and marketing. Food recognition offers a means to automatically recognise foods as well the useful information such as calories and nutritional estimation by using image processing and machine learning technique. The interest points in food image can be detected effectively by using Maximally Stable Extremal Region (MSER). As MSER used global segmentation and many food images have a complex background, there are numerous irrelevant interest points are detected. These interest points are considered as noises that lead to computation burden in the overall recognition process. Therefore, this research proposes an Extremal Region Selection (ERS) algorithm to improve MSER detection by reducing the number of irrelevant extremal regions by using unsupervised learning based on the k-means algorithm. The performance of ERS algorithm is evaluated based on the classification performance metrics by using classification rate (CR), error rate (ERT), precision (Prec.) and recall (rec.) as well as the number of extremal regions produced by ERS. UECFOOD-100 and UNICT-FD1200 are the two food datasets used to benchmark the proposed algorithm. The results of this research have found that the ERS algorithm by using optimum parameters and thresholds, be able to reduce the number of extremal regions with sustained classification performance.

Recently, deep learning convolutional neural network has been widely used in food recognition and other fields, but there are still some problems such as poor semantic feature extraction ability and low recognition accuracy of Chinese food images. Based on the above reasons, this paper proposed a DenseNet model based on attention mechanism for image recognition of Chinese food. Using the idea of transfer learning, this paper applied the DenseNet pre-training model with excellent classification performance to the Chinese food image dataset. In order to improve the ability of extracting distinguishable features and learning fine-grained features of Chinese food images, this paper proposed an attention mechanism based on the DenseNet-169 model. This paper used the attention module to extract the key semantic feature map of the image, and then carried out the adaptive multi-fine-grained region tailoring. Finally, based on the region weight fusion scheme, the multi-fine-grained region feature map was integrated into an image feature description and sent to DenseNet-169 for recognition, so as to realize the end-to-end image recognition network. Experiments were carried out on VIREO Food-172 dataset, and the results shows that our method achieves considerable and comparable recognition performance to the state-of-art.

The theory of compressive sensing applies the sparse representation to the extraction of useful information from signals and brings a breakthrough to the theory of signal sampling. Based on compressive sensing, sparse representation-based classification (SRC) is proposed. SRC uses the compressibility of the image data to represent the facial image sparsely and could solve the problems of both massive calculation and information loss in dealing with signals. SRC does not, however, deal with the effects of variable illumination, posture and incomplete face image, which could result in severe performance degradation. This paper studies the differences between SRC recognition and human recognition. We find that there is an obvious disadvantage in the SRC algorithm, and it will significantly affect the face recognition performance in actual environment, especially for the variable illumination, posture and incomplete face image. To overcome the disadvantage of SRC algorithm, we propose an SRC-based twice face recognition algorithm named T_SRC. T_SRC uses bidirectional PCA, linear discriminant analysis and GradientFace to execute multichannel analysis, which could extract more "holistic/configural" face features in actual environment than by using SRC algorithm directly. Based on the multichannel analysis, we identify the test image by SRC firstly. Then, by analyzing the residual, this algorithm could decide whether the twice recognition is needed. If the twice recognition is needed, T_SRC extracts the facial details ("featural" face features) by the improved Harris point and Gabor filter detector. We suppose that the facial details are more stable than the whole face in actual environment, and later experiments verify our assumption. At last, this algorithm identifies the class of the test image by SRC again. The results of the experiments prove that the T_SRC algorithm has better recognition rate than SRC.

The recent works in cross-modal image-to-recipe retrieval pave a new way to scale up food recognition. By learning the joint space between food images and recipes, food recognition is boiled down as a retrieval problem by evaluating the similarity of embedded features. The major drawback, nevertheless, is the difficulty in applying an already-trained model to recognize different cuisines of dishes unknown to the model. In general, model updating with new training examples, in the form of image-recipe pairs, is required to adapt a model to new cooking styles in a cuisine. Nevertheless, in practice, acquiring sufficient number of image-recipe pairs for model transfer can be time-consuming. This paper addresses the challenge of resource scarcity in the scenario that only partial data instead of a complete view of data is accessible for model transfer. Partial data refers to missing information such as absence of image modality or cooking instructions from an image-recipe pair. To cope with partial data, a novel generic model, equipped with various loss functions including cross-modal metric learning, recipe residual loss, semantic regularization and adversarial learning, is proposed for cross-domain transfer learning. Experiments are conducted on three different cuisines (Chuan, Yue and Washoku) to provide insights on scaling up food recognition across domains with limited training resources.

In content based image retrieval (CBIR) process, every image has been represented in a compact set of local visual features i.e. color, texture, and/or shape of images. This set of local visual features is known as feature vector. In the CBIR process, feature vectors of images have been used to represent or to identify similar images in adequate way. As a result, feature vector construction has always been considered as an important issue since it must reflect proper image semantics using minimal amount of data. The proposed CBIR scheme is based on the combination of color and texture features. In this work initially, we have converted the given RGB image into HSV color image. Subsequently, we have considered H (hue), S (saturation), and V (intensity) components for extraction of visual image features. The texture features have been extracted from the V component of the image using 2D dual-tree complex wavelet transform (2D DT-CWT) where it analyzes the textural patterns in six different directions i.e. ±15\(^{\circ }\), ±45\(^{\circ }\), and ±75\(^{\circ }\). At the same time, we have computed the probability histograms of H and S components of the image respectively and subsequently those are divided into non-uniform bins based on cumulative probability for extraction of color based features. So, in this work both the color and texture features have been extracted simultaneously. Finally, the obtained features have been concatenated to attain the final feature vector and same is considered in image retrieval process. We have tested the novelty and performance of the proposed work in two Corel, two objects, and, a texture image datasets. The experimental results reveal the acceptable retrieval performances for different types of datasets.

Joint discriminative and shared dictionary learning with dictionary extension strategy for bearing fault classification

In order to improve the classification performance for hyperspectral image (HSI), a sparse representation classifier based on spectral information divergence (SID) is proposed. SID measures the discrepancy of probabilistic behaviors between the spectral signatures of two pixels from the aspect of information theory, which can be more effective in preserving spectral properties. Thus, the new method measures the similarity between the reconstructed pixel and the true pixel by SID instead of by the L2 norm used in traditional sparse model. Moreover, the spatial coherency across neighboring pixels sharing a common sparsity pattern is taken into account during the construction of SID-based joint sparse representation model. We propose a new version of the orthogonal matching pursuit method to solve SID-based recovery problems. The proposed SID-based algorithms are applied to real HSI for classification. Experimental results show that our algorithms outperform the classical sparse representation based classification algorithms in most cases.

In this paper, we present a facial expression recognition method based on Gabor feature and sparse representation. Sparse Representation based Classification (SRC) has been widely used in computer vision and pattern recognition. And Gabor filter banks can be used to approximately model the signal processing in visual primary cortex. We believe that the nature of the attractive performance of SRC and Gabor feature lies in that they both followed the natures of signal perception of retina and information processing of cortex in human vision. Therefore, we combined the Gabor feature and SRC for facial expression recognition. The comparison experiments of proposed Gabor+SRC algorithm and straightforward SRC application are conducted on JAFFE database. And the experimental results showed the attractive performance of the proposed Gabor+SRC method.

Traditional machine learning algorithms using hand-crafted feature extraction techniques (such as local binary pattern) have limited accuracy because of high variation in images of the same class (or intraclass variation) for food recognition tasks. In recent works, convolutional neural networks (CNNs) have been applied to this task with better results than all previously reported methods. However, they perform best when trained with large amount of annotated (labeled) food images. This is problematic when obtained in large volume, because they are expensive, laborious, and impractical. This article aims at developing an efficient deep CNN learning-based method for food recognition alleviating these limitations by using partially labeled training data on generative adversarial networks (GANs). We make new enhancements to the unsupervised training architecture introduced by Goodfellow et al., which was originally aimed at generating new data by sampling a dataset. In this article, we make modifications to deep convolutional GANs to make them robust and efficient for classifying food images. Experimental results on benchmarking datasets show the superiority of our proposed method, as compared to the current state-of-the-art methodologies, even when trained with partially labeled training data.

In this paper, we propose a deep learning based system for food recognition from personal life archive im- ages. The system first identifies the eating moments based on multi-modal information, then tries to focus and enhance the food images available in these moments, and finally, exploits GoogleNet as the core of the learning process to recognise the food category of the images. Preliminary results, experimenting on the food recognition module of the proposed system, show that the proposed system achieves 95.97% classification accuracy on the food images taken from the personal life archive from several lifeloggers, which potentially can be extended and applied in broader scenarios and for different types of food categories.

The National Nutrition Survey (NNS) is the Philippines' official source of data on nutritional status, diet, and lifestyle-related risk factors for non-communicable diseases. A food consumption survey is one of its major components that collects individual dietary data through 24-h food recall interviews and household weighing of food intake using a dietetic scale. Conventional manual surveys are costly and tedious to conduct, which can also compound the inconsistencies and biases in the data gathered. Technological advances in mobile devices and networking have made it possible to gather data more economically and more efficiently with emerging technologies such as machine learning (ML) and increased use of automation open vast opportunities. In this paper, we explored the digitalization of food component forms of the NNS and developed a fully digital dietary survey collection system from collection to transmission of data. Google AutoML Vision, an AutoML platform based on NAS was used to create a Filipino food image recognition model as an added feature of the developed system. A Filipino food image dataset comprises 32 food classes and 3,295 images were created for this process. The developed food image recognition model scored a modest 76% on the precision metric in correctly identifying foods, which performed at par with other custom food recognition models. Nutrient approximation based on the Philippine Food Composition Table database also proved to be useful in displaying nutrition facts of recognized foods. This study showed the potential of a digital dietary survey system, combined with a unique and specific food image recognition tool, is a very encouraging proposition. Expansion of the food image dataset, exploration of other ML techniques, automated food weighta approximation, and continuous enhancement of the developed system are advocated.

Single-sample face recognition is one of the most challenging problems in face recognition. We propose a novel algorithm to address this problem based on a sparse representation based classification (SRC) framework. The new algorithm is robust to image misalignment and pixel corruption, and is able to reduce required gallery images to one sample per class. To compensate for the missing illumination information traditionally provided by multiple gallery images, a sparse illumination learning and transfer (SILT) technique is introduced. The illumination in SILT is learned by fitting illumination examples of auxiliary face images from one or more additional subjects with a sparsely-used illumination dictionary. By enforcing a sparse representation of the query image in the illumination dictionary, the SILT can effectively recover and transfer the illumination and pose information from the alignment stage to the recognition stage. Our extensive experiments have demonstrated that the new algorithms significantly outperform the state of the art in the single-sample regime and with less restrictions. In particular, the single-sample face alignment accuracy is comparable to that of the well-known Deformable SRC algorithm using multiple gallery images per class. Furthermore, the face recognition accuracy exceeds those of the SRC and Extended SRC algorithms using hand labeled alignment initialization.

A new scene classification method is proposed based on the combination of local Gabor features with a spatial pyramid matching model. First, new local Gabor feature descriptors are extracted from dense sampling patches of scene images. These local feature descriptors are embedded into a bag-of-visual-words (BOVW) model, which is combined with a spatial pyramid matching framework. The new local Gabor feature descriptors have sufficient discrimination abilities for dense regions of scene images. Then the efficient feature vectors of scene images can be obtained byK-means clustering method and visual word statistics. Second, in order to decrease classification time and improve accuracy, an improved kernel principal component analysis (KPCA) method is applied to reduce the dimensionality of pyramid histogram of visual words (PHOW). The principal components with the bigger interclass separability are retained in feature vectors, which are used for scene classification by the linear support vector machine (SVM) method. The proposed method is evaluated on three commonly used scene datasets. Experimental results demonstrate the effectiveness of the method.

The analysis of dietary pattern is important in health-care to reduce the risk factors of getting diet-related chronic diseases. An automatic dietary pattern assessment via food recognition algorithm provide an alternative way to improve the overall quality of dietary pattern assessment. However, due to very high variability of food images, the fusion of multiple type of features have become inevitable. The Bag of Features (BoF) model to encode the features are originally designed to consider a single type of features only. Therefore, this paper investigates the methods to fuse multiple features extracted from food images. Specifically, three fusion methods have been evaluated namely descriptor-level fusion, representation-level fusion and score-level fusion. The fusion is performed on the feature detector-level between Different of Hessian (DoH) and MSER detector. Speeded-up Robust Feature (SURF) is employed for feature description. The features are encoded by using k-means clustering and Support Vector Machine with linear kernel has been employed for classification. The evaluation has been performed on UECIOO-Food dataset.