Large-scale Food Image Datasets and Food Recognition Systems: A Survey

Scene images usually involve semantic correlations, particularly when considering large-scale image data sets. This paper proposes a novel generative image representation, correlated topic vector, to model such semantic correlations. Oriented from the correlated topic model, correlated topic vector intends to naturally utilize the correlations among topics, which are seldom considered in the conventional feature encoding, e.g., Fisher vector, but do exist in scene images. It is expected that the involvement of correlations can increase the discriminative capability of the learned generative model and consequently improve the recognition accuracy. Incorporated with the Fisher kernel method, correlated topic vector inherits the advantages of Fisher vector. The contributions to the topics of visual words have been further employed by incorporating the Fisher kernel framework to indicate the differences among scenes. Combined with the deep convolutional neural network (CNN) features and Gibbs sampling solution, correlated topic vector shows great potential when processing large-scale and complex scene image data sets. Experiments on two scene image data sets demonstrate that correlated topic vector improves significantly the deep CNN features, and outperforms existing Fisher kernel-based features.

While deep learning (DL) has proven to be a powerful paradigm for the classification of the large-scale big image data set. Deep learning network such as CNN requires a large number of labeled samples for training the network. However, often times the labeled data are difficult, expensive, and time-consuming. In this paper, we propose a semi-supervised approach to fused fuzzy-rough C-mean clustering with convolutional neural networks (CNNs) to knowledge learning from simultaneously intra-model and inter-model relationships forming final data representation to be classified, which can be achieved by better performance. Conception behind this is to reduce the uncertainty in terms of vagueness and indiscernibility by using fuzzy-rough C-mean clustering and specifically removing noise samples by using CNN from raw data. The framework of our proposed semi-supervised approach uses all the training data with abundant unlabeled data with few labeled data to train FRCNN model. To show the effectiveness of our model, we used four benchmark large-scale image datasets and also compared it with the state-of-the-art supervised, unsupervised, and semi-supervised learning methods for image classification.

Pepper is a common vegetable, with a wide range of applications and market demand. Pepper recognition is an important task when harvesting peppers by a robot. Due to the fact that chili peppers are usually planted very densely, the fruits often grow in clusters. As a result, identifying and locating peppers that are obscured by other peppers, branches, and leaves becomes a challenging task. In this task, the robot needs to overcome visual obstacles in order to accurately detect and locate the target pepper. This paper proposes a pepper recognition method based on the improved You Only Look Once version 3 (YOLOv3) convolutional neural network. By adding Res2Net backbone into the feature extraction network of YOLOv3, the extraction performance is improved, and the accuracy and efficiency of pepper identification are also improved. In this study, large-scale pepper image data sets were used for training and testing, and the generalizability and robustness of the model were improved through data enhancement and transfer learning techniques. For the test samples, the recognition rate of the traditional Backpropagation Neural Network (BPNN) algorithm is 66.25%, while the recognition rate of the improved YOLOv3 convolutional network for the same test samples is 91.25%. The experimental results show that this method has good performance in the pepper recognition task, with high accuracy and fast processing speed.

With the rapid growth of image data in recent years, efficient management and retrieval of image data have become increasingly important. In this paper, we propose an image warehouse architecture in the era of big data that combines data preprocessing, compression and deduplication, distributed processing and parallel computing, machine learning, and deep learning, and security and privacy techniques to improve the efficiency and scalability of image warehouses. We conducted experiments on a large-scale image dataset, and the results show that our approach significantly outperforms existing methods in terms of retrieval accuracy and efficiency. The proposed architecture provides a promising solution for managing and retrieving large-scale image data in the era of big data

Classification of food images is now very popular in recent years with the development of vision technology. It is quite challenging work to classify food images as images are consist of different shape, size and color. We proposed a new food recognition system by using local features in this paper. We selected UEC-100 food dataset. It consist 100 types of food images which are popular food in Japan. We considered GIST and SURF as local features. We applied pre-processing task on the input image to improve image quality. Classification is performed by Support Vector Machine (SVM). GIST and SURF features are extracted separately. Classification shows poor result while features are used separately. Later when both features are combined together and creates hybrid feature, it results in higher accuracy. We classified images with different classifier. However, SVM as a classifier shows comparable and promising result.

Space situational awareness (SSA) plays an imperative role in maintaining safe space operations, especially given the increasingly congested space traffic around the Earth. Space-based SSA offers a flexible and lightweight solution compared to traditional ground-based SSA. With advanced machine learning approaches, space-based SSA can extract features from high-resolution images in space to detect and track resident space objects (RSOs). However, existing spacecraft image datasets, such as SPARK, fall short of providing realistic camera observations, rendering the derived algorithms unsuitable for real SSA systems. In this work, we introduce SpaceDet, a large-scale realistic space-based image dataset for SSA. We consider accurate space orbit dynamics and a physical camera model with various noise distributions, generating images at the photon level. To extend the available observation window, four overlapping cameras are simulated with a fixed rotation angle. SpaceDet includes images of RSOs observed from 19 km to 63,000 km, captured by a tracker operating in LEO, MEO, and GEO orbits over a period of 5,000 seconds. Each image has a resolution of 4418 x 4418 pixels, providing detailed features for developing advanced SSA approaches. We split the dataset into three subsets: SpaceDet-100, SpaceDet-5000, and SpaceDet-full, catering to various image processing applications. The SpaceDet-full corpus includes a comprehensive dataloader with 781.5 GB of images and 25.9 MB of ground truth labels. Furthermore, we adapted detection and tracking algorithms on the collected dataset using a specified splitting method to accelerate the training process. The trained model can detect RSOs from real-world space observations with zero-shot capability.

Learning effective representations that exhibit semantic content is crucial to image retrieval applications. Recent advances in deep learning have made significant improvements in performance on a number of visual recognition tasks. Studies have also revealed that visual features extracted from a deep network learned on a large-scale image data set (e.g., ImageNet) for classification are generic and perform well on new recognition tasks in different domains. Nevertheless, when applied to image retrieval, such deep representations do not attain performance as impressive as used for classification. This is mainly because the deep features are optimized for classification rather than for the desired retrieval task. We introduce the cross-batch reference (CBR), a novel training mechanism that enables the optimization of deep networks with a retrieval criterion. With the CBR, the networks leverage both the samples in a single minibatch and the samples in the others for weight updates, enhancing the stochastic gradient descent (SGD) training by enabling interbatch information passing. This interbatch communication is implemented as a cross-batch retrieval process in which the networks are trained to maximize the mean average precision (mAP) that is a popular performance measure in retrieval. Maximizing the cross-batch mAP is equivalent to centralizing the samples relevant to each other in the feature space and separating the samples irrelevant to each other. The learned features can discriminate between relevant and irrelevant samples and thus are suitable for retrieval. To circumvent the discrete, nondifferentiable mAP maximization, we derive an approximate, differentiable lower bound that can be easily optimized in deep networks. Furthermore, the mAP loss can be used alone or with a classification loss. Experiments on several data sets demonstrate that our CBR learning provides favorable performance, validating its effectiveness.

With the advance of the industrial Internet of Things, automated environments such as smart cities are foreseeable. Vehicle re-identification (ReID) tries to recall images of the same vehicle in large-scale image data sets. The main challenges of vehicle ReID are intra-identity differences caused by different views of the same vehicle and inter-identities similarity caused by the same view of similar vehicles. To deal with these challenges, we propose a novel Transformer-based Vehicle ReID model with View Information (TVRVI). First, we annotate and divide the vehicle images into five different view parts and train a parsing network with view information to separate images and give out their corresponding view labels. Second, we design a dual branches transformer-based parsing network to extract features of different views separately and reduce their entanglement. With the help of view information, the local branch of the transformer network can learn fine-grained representations of different views and the overall feature of the whole image is learned in the global branch at the same time. By comparing features from the same view and leaving out features of different views, the TVRVI makes good use of common-view features and avoids interference from uncommon-view features, which helps reduce intra-identity differences and inter-identities similarities between vehicle images. Experiments on four public vehicle datasets show the effectiveness and state-of-the-art results of our method, and ablation studies are specifically designed to testify to the effectiveness of view information.

ABSTRACTThis study explores how people respond to images of junk versus healthy food as a function of their eating habits and food knowledge. The experiment reported here proposed and tested the idea that those with unhealthy eating habits but highly knowledgeable about healthy eating would feel more positive and also more negative toward junk food images compared to images of healthy food because they may perceive them as risky—desirable but potentially harmful. The psychophysiological data collected from participants during their exposure to pictures of junk versus healthy food supported this idea. In addition, unhealthy eaters compared to healthy eaters with the same degree of food knowledge responded more positively to all food items. The findings are critical from a health communication perspective. Because unhealthy eaters produce stronger emotional responses to images of junk food, they are more likely to process information associated with junk food with more cognitive effort and scrutiny. Thus, when targeting this group and using images of junk food, it is important to combine these images with strong message claims and relevant arguments; otherwise, if the arguments are perceived as irrelevant or weak, the motivational activation associated with junk food itself may transfer into an increased desire to consume the unhealthy product.

A High School Soccer Player with Energy Deficiency

Assessing students’ participation in science practices presents several challenges, especially when aiming to differentiate meaningful (vs. rote) forms of participation. In this study, we sought to use machine learning (ML) for a novel purpose in science assessment: developing a construct map for students’ consideration of generality, a key epistemic understanding that undergirds meaningful participation in knowledge-building practices. We report on our efforts to assess the nature of 845 students’ ideas about the generality of their model-based explanations through the combination of an embedded written assessment and a novel data analytic approach that combines unsupervised and supervised machine learning methods and human-driven, interpretive coding. We demonstrate how unsupervised machine learning methods, when coupled with qualitative, interpretive coding, were used to revise our construct map for generality in a way that allowed for a more nuanced evaluation that was closely tied to empirical patterns in the data. We also explored the application of the construct map as a framework for coding used as a part of supervised machine learning methods, finding that it demonstrates some viability for use in future analyses. We discuss implications for the assessment of students’ meaningful participation in science practices in terms of their considerations of generality, the role of unsupervised methods in science assessment, and combining machine learning and human-driven approach for understanding students’ complex involvement in science practices.

Deep learning (DL) methods have demonstrated remarkable effectiveness in assisting with lung cancer risk prediction tasks using computed tomography (CT) scans. However, the lack of comprehensive comparison and validation of state-of-the-art (SOTA) models in practical settings limits their clinical application. This study aims to review and analyze current SOTA deep learning models for lung cancer risk prediction (malignant-benign classification). To evaluate our model’s general performance, we selected 253 out of 467 patients from a subset of the National Lung Screening Trial (NLST) who had CT scans without contrast, which are the most commonly used, and divided them into training and test cohorts. The CT scans were preprocessed into 2D-image and 3D-volume formats according to their nodule annotations. We evaluated ten 3D and eleven 2D SOTA deep learning models, which were pretrained on large-scale general-purpose datasets (Kinetics and ImageNet) and radiological datasets (3DSeg-8, nnUnet and RadImageNet), for their lung cancer risk prediction performance. Our results showed that 3D-based deep learning models generally perform better than 2D models. On the test cohort, the best-performing 3D model achieved an AUROC of 0.86, while the best 2D model reached 0.79. The lowest AUROCs for the 3D and 2D models were 0.70 and 0.62, respectively. Furthermore, pretraining on large-scale radiological image datasets did not show the expected performance advantage over pretraining on general-purpose datasets. Both 2D and 3D deep learning models can handle lung cancer risk prediction tasks effectively, although 3D models generally have superior performance than their 2D competitors. Our findings highlight the importance of carefully selecting pretrained datasets and model architectures for lung cancer risk prediction. Overall, these results have important implications for the development and clinical integration of DL-based tools in lung cancer screening.

The 2014 workshop on medical computer vision (MCV): algorithms for big data took place in Cambridge, MA, USA in connection with MICCAI (Medical Image Computing for Computer Assisted Intervention). It is the fourth MICCAI MCV workshop after those held in 2010, 2012 and 2013 with another edition held at CVPR 2012. This workshop aims at exploring the use of modern computer vision technology in tasks such as automatic segmentation and registration, localisation of anatomical features and extraction of meaningful visual features. It emphasises questions of harvesting, organising and learning from large-scale medical imaging data sets and general-purpose automatic understanding of medical images. The workshop is especially interested in modern, scalable and efficient algorithms which generalise well to previously unseen images.The strong participation in the workshop of over 80 persons shows the importance of and interest in Medical Computer Vision. This overview article describes the papers presented in the workshop as either oral presentations or short presentations and posters. It also describes the invited talks and the results of the VISCERAL session in the workshop on the use of big data in medical imaging.

The feature models used by existing Thermal InfraRed (TIR) tracking methods are usually learned from RGB images due to the lack of a large-scale TIR image training dataset. However, these feature models are less effective in representing TIR objects and they are difficult to effectively distinguish distractors because they do not contain fine-grained discriminative information. To this end, we propose a dual-level feature model containing the TIR-specific discriminative feature and fine-grained correlation feature for robust TIR object tracking. Specifically, to distinguish inter-class TIR objects, we first design an auxiliary multi-classification network to learn the TIR-specific discriminative feature. Then, to recognize intra-class TIR objects, we propose a fine-grained aware module to learn the fine-grained correlation feature. These two kinds of features complement each other and represent TIR objects in the levels of inter-class and intra-class respectively. These two feature models are constructed using a multi-task matching framework and are jointly optimized on the TIR object tracking task. In addition, we develop a large-scale TIR image dataset to train the network for learning TIR-specific feature patterns. To the best of our knowledge, this is the largest TIR tracking training dataset with the richest object class and scenario. To verify the effectiveness of the proposed dual-level feature model, we propose an offline TIR tracker (MMNet) and an online TIR tracker (ECO-MM) based on the feature model and evaluate them on three TIR tracking benchmarks. Extensive experimental results on these benchmarks demonstrate that the proposed algorithms perform favorably against the state-of-the-art methods.

Unconditional human image generation is an important task in vision and graphics, enabling various applications in the creative industry. Existing studies in this field mainly focus on “network engineering” such as designing new components and objective functions. This work takes a data-centric perspective and investigates multiple critical aspects in “data engineering”, which we believe would complement the current practice. To facilitate a comprehensive study, we collect and annotate a large-scale human image dataset with over 230K samples capturing diverse poses and textures. Equipped with this large dataset, we rigorously investigate three essential factors in data engineering for StyleGAN-based human generation, namely data size, data distribution, and data alignment. Extensive experiments reveal several valuable observations w.r.t. these aspects: 1) Large-scale data, more than 40K images, are needed to train a high-fidelity unconditional human generation model with a vanilla StyleGAN. 2) A balanced training set helps improve the generation quality with rare face poses compared to the long-tailed counterpart, whereas simply balancing the clothing texture distribution does not effectively bring an improvement. 3) Human GAN models that employ body centers for alignment outperform models trained using face centers or pelvis points as alignment anchors. In addition, a model zoo and human editing applications are demonstrated to facilitate future research in the community. Code and models are publicly available (Project page: https://stylegan-human.github.io/. Code and models: https://github.com/stylegan-human/StyleGAN-Human.)KeywordsHuman image generationData-centricStyleGAN