Diverse Concept-Level Features for Multi-Object Classification

Vision-language integration using constrained local semantic features

Neural ordinary differential equations (neural ODEs) have emerged as a novel network architecture that bridges dynamical systems and deep learning. However, the gradient obtained with the continuous adjoint method in the vanilla neural ODE is not reverse-accurate. Other approaches suffer either from an excessive memory requirement due to deep computational graphs or from limited choices for the time integration scheme, hampering their application to large-scale complex dynamical systems. To achieve accurate gradients without compromising memory efficiency and flexibility, we present a new neural ODE framework, PNODE, based on high-level discrete adjoint algorithmic differentiation. By leveraging discrete adjoint time integrators and advanced checkpointing strategies tailored for these integrators, PNODE can provide a balance between memory and computational costs, while computing the gradients consistently and accurately. We provide an open-source implementation based on PyTorch and PETSc, one of the most commonly used portable, scalable scientific computing libraries. We demonstrate the performance through extensive numerical experiments on image classification and continuous normalizing flow problems. We show that PNODE achieves the highest memory efficiency when compared with other reverse-accurate methods. On the image classification problems, PNODE is up to two times faster than the vanilla neural ODE and up to 2.3 times faster than the best existing reverse-accurate method. We also show that PNODE enables the use of the implicit time integration methods that are needed for stiff dynamical systems.

We propose a computerized framework that, given a region of interest (ROI) circumscribing a lesion, not only predicts radiological observations related to the lesion characteristics with 83.2% average prediction accuracy but also derives explicit association between low-level imaging features and high-level semantic terms by exploiting their statistical correlation. Such direct association between semantic concepts and low-level imaging features can be leveraged to build a powerful annotation system for radiological images that not only allows the computer to infer the semantics from diverse medical images and run automatic reasoning for making diagnostic decision but also provides “human-interpretable explanation” of the system output to facilitate better end user understanding of computer-based diagnostic decisions. The core component of our framework is a radiological observation detection algorithm that maximizes the low-level imaging feature relevancy for each high-level semantic term. On a liver lesion CT dataset, we have implemented our framework by incorporating a large set of state-of-the-art low-level imaging features. Additionally, we included a novel feature that quantifies lesion(s) present within the liver that have a similar appearance as the primary lesion identified by the radiologist. Our framework achieved a high prediction accuracy (83.2%), and the derived association between semantic concepts and imaging features closely correlates with human expectation. The framework has been only tested on liver lesion CT images, but it is capable of being applied to other imaging domains.

Classification task is one of the most common tasks in machine learning. This supervised learning problem consists in assigning each input to one of a finite number of discrete categories. Classification task appears naturally in numerous applications, such as medical image processing, speech recognition, maintenance systems, accident detection, autonomous driving etc.In the last decade methods of deep learning have proven to be extremely efficient in multiple machine learning problems, including classification. Whereas the neural network architecture might depend a lot on data type and restrictions posed by the nature of the problem (for example, real-time applications), the process of its training (i.e. finding model’s parameters) is almost always presented as loss function optimization problem.Cross-entropy is a loss function often used for multiclass classification problems, as it allows to achieve high accuracy results.Here we propose to use a generalized version of this loss based on Renyi divergence and entropy. We remark that in case of binary labels proposed generalization is reduced to cross-entropy, thus we work in the context of soft labels. Specifically, we consider a problem of image classification being solved by application of convolution neural networks with mixup regularizer. The latter expands the training set by taking convex combination of pairs of data samples and corresponding labels. Consequently, labels are no longer binary (corresponding to single class), but have a form of vector of probabilities. In such settings cross-entropy and proposed generalization with Renyi divergence and entropy are distinct, and their comparison makes sense.To measure effectiveness of the proposed loss function we consider image classification problem on benchmark CIFAR-10 dataset. This dataset consists of 60000 images belonging to 10 classes, where images are color and have the size of 32×32. Training set consists of 50000 images, and the test set contains 10000 images.For the convolution neural network, we follow [1] where the same classification task was studied with respect to different loss functions and consider the same neural network architecture in order to obtain comparable results.Experiments demonstrate superiority of the proposed method over cross-entropy for loss function parameter value α < 1. For parameter value α > 1 proposed method shows worse results than cross-entropy loss function. Finally, parameter value α = 1 corresponds to cross-entropy.

The article focuses on studying the effectiveness of two different Hybrid Neural Networks (HNNs) architectures for solving real-world image classification problems. The first approach investigated in the research is a hybridization technique that allows creation of HNN based on a classical neural network by replacing a number of hidden layers of the neural network with a variational quantum circuit, which allows to reduce the complexity of the classical part of the neural network and move part of computations to a quantum device. The second approach is a hybridization technique based on utilizing quanvolutional operations for image processing as the first quantum convolutional layer of the hybrid neural network, thus building a Quanvolutional Neural Network (QNN). QNN leverages quantum phenomena to facilitate feature extraction, enabling the model to achieve higher accuracy metrics than its classical counterpart. The effectiveness of both architectures was tested on several image classification problems. The first one is a classical image classification problem of CIFAR10 images classification, widely used as a benchmark for various imagery-related tasks. Another problem used for the effectiveness study is the problem of geospatial data analysis. The second problem represents a real-world use case where quantum computing utilization can be very fruitful in the future. For studying the effectiveness, several models were assembled: HNN with a quantum device that replaces one of the hidden layers of the neural network, QNN based on quanvolutional operation and utilizes VGG-16 architecture as a classical part of the model, and also an unmodified VGG-16 was used as a reference model. Experiments were conducted to measure the models' key efficiency metrics: maximal accuracy, complexity of a quantum part of the model and complexity of a classical part of the model. The results of the research indicated the feasibility of both approaches for solving both proposed image classification problems. Results were analyzed to outline the advantages and disadvantages of every approach in terms of selected key metrics. Experiments showed that QNN architectures proved to be a feasible and effective solution for critical practical tasks requiring higher levels of model prediction accuracy and, simultaneously, can tolerate higher processing time and significantly increased costs due to a high number of quantum operations required. Also, the results of the experiments indicated that HNN architectures proved to be a feasible solution for time-critical practical tasks that require higher processing speed and can tolerate slightly decreased accuracy of model predictions.

The use of Convolutional Neural Networks (CNNs) to solve Domain Adaptation (DA) image classification problems in the context of remote sensing has proven to provide good results but at high computational cost. To avoid this problem, a deep learning network for DA in remote sensing hyperspectral images called TCANet is proposed. As a standard CNN, TCANet consists of several stages built based on convolutional filters that operate on patches of the hyperspectral image. Unlike the former, the coefficients of the filter are obtained through Transfer Component Analysis (TCA). This approach has two advantages: firstly, TCANet does not require training based on backpropagation, since TCA is itself a learning method that obtains the filter coefficients directly from the input data. Second, DA is performed on the fly since TCA, in addition to performing dimensional reduction, obtains components that minimize the difference in distributions of data in the different domains corresponding to the source and target images. To build an operating scheme, TCANet includes an initial stage that exploits the spatial information by providing patches around each sample as input data to the network. An output stage performing feature extraction that introduces sufficient invariance and robustness in the final features is also included. Since TCA is sensitive to normalization, to reduce the difference between source and target domains, a previous unsupervised domain shift minimization algorithm consisting of applying conditional correlation alignment (CCA) is conditionally applied. The results of a classification scheme based on CCA and TCANet show that the DA technique proposed outperforms other more complex DA techniques.

The problem of color image classification has been studied in number of research articles and the same can be applied to the problem of fruit image classification and fruit grading. There are many methods has been deliberated for the problem of fruit grading but suffers with the problem of poor classification accuracy and fruit grading. The proposed region based color histogram approximation technique enhances the input image for its clarity using histogram equalization technique at the first stage. In the second stage, the method splits the image in to number of sectional images and extracts the features of each region. From the regional feature extracted in the second stage, the method computes the histogram of the regional feature. The extracted regional histogram will be indexed as the training set and the same will be iterated for the input image. Finally the method performs histogram approximation by computing the sectional similarity between each trained vectors. Using computed sectional similarity the method computes the fruit class weight to classify the image towards number of fruit classes. The method has produced efficient results in fruit classification and increases the efficiency of fruit grading.

As the global population is increasing and agricultural resources are diminishing, effective management of plant diseases become critical. Convolutional neural network-based models have been widely adopted in agriculture cyber–physical systems due to their high accuracy. However, these models require large, labeled datasets for training, limiting their practicality in plant disease detection where data are heterogeneous, scarce, and subjected to continual evolution. To address these challenges, we propose semantic-search, a knowledge-driven system that interprets plant diseases based on high-level semantic features, such as visual patterns, colors, and crop type, rather than relying solely on low-level image features. The system classifies diseases by querying a structured knowledge base using these semantics. “Semantic search” here refers to classification based on semantic feature extraction and knowledge base querying. It enables greater adaptability, interpretability, and robustness to intraclass variability caused by differing environmental conditions. The system can incorporate new disease classes through simple knowledge base updates, eliminating the need for retraining. This facilitates efficient scaling to evolving set of diseases. We validated the proposed approach on a dataset of 11 000 images encompassing 21 diseases across 11 plant species, achieving an accuracy of 90%, thereby demonstrating its effectiveness, scalability.

In this paper, we propose a dynamic technique for selecting the most informative samples in classification problems as coming in two stages: the first stage conducts sample selection in batch off-line mode based on unsupervised criteria extracted from cluster partitions, the second phase proposes an active learning scheme during on-line adaptation of classifiers in non-stationary environments. This is based on the reliability of the classifiers in their output responses (confidences in their predictions). Both approaches contribute to a reduction of the annotation effort for operators, as operators only have to label/give feedback on a subset of the off-line/online. At the same time they are able to keep the accuracy on almost the same level as when the classifiers would have been trained on all samples. This will be verified based on real-world data sets from two image classification problems used in on-line surface inspection scenarios.

With the rapid growth of multimedia application technologies and network technologies, especially the proliferation of Web 2.0 and digital cameras, there has been an explosion of images and videos in the Internet. For example, the volume of videos uploaded to the YouTube every minute is amounting to 48 hours by May 2011, having doubled in the last two years. Such huge video collections hold useful yet implicit and nontrivial knowledge about various domains. To manage and utilize these resources effectively, video concept detection becomes a very important subject of intensive research by a large research community (Over et al., 2008). It is an integral part of visual data mining that is automatically extracting such knowledge from the huge unstructured visual data. It aims to automatically annotate video shots or keyframes with respect to a semantic concept (Tang et al., 2012). Ranging from objects like airplane and car to scenes like urban street and sky, semantic concepts serve as good intermediate semantic features for video content indexing and understanding, and thus, spurring much research attention (Jiang et al., 2010; Naphade & Smith, 2004; Snoek et al., 2006; Zheng et al., 2008). Essentially, concept detection is a classification task, in which a binary classifier is usually learned to predict the presence of a certain concept in a video shot or keyframe (image). Traditional concept detection methods are mainly global classification: use supervised machine learning techniques, such as single Support Vector Machine (SVM), etc., over whole training dataset.

Machine learning involves the task of training systems to be able to make decisions without being explicitly programmed. Important among machine learning tasks is classification involving the process of training machines to make predictions from predefined labels. Classification is broadly categorized into three distinct groups: single-label (SL), multi-class, and multi-label (ML) classification. This research work presents an application of a multi-label classification (MLC) technique in automating Quranic verses labeling. MLC has been gaining attention in recent years. This is due to the increasing amount of works based on real-world classification problems of multi-label data. In traditional classification problems, patterns are associated with a single-label from a set of disjoint labels. However, in MLC, an instance of data is associated with a set of labels. In this paper, three standard <em>MLC</em> methods: <span>binary relevance (BR), classifier chain (CC), and label powerset (LP) algorithms are implemented with four baseline classifiers: support vector machine (SVM), naïve Bayes (NB), k-nearest neighbors (k-NN), and J48. The research methodology adopts the multi-label problem transformation (PT) approach. The results are validated using six conventional performance metrics. These include: hamming loss, accuracy, one error, micro-F1, macro-F1, and avg. precision. From the results, the classifiers effectively achieved above 70% accuracy mark. Overall, SVM achieved the best results with CC and LP algorithms.</span>

Distinct texture classes are often sharing several visual concepts. Texture instances from different classes are sharing regions in the feature hyperspace, which results in ill-defined classification configurations. In this work, we detect rotation-covariant visual concepts using steerable Riesz wavelets and bags of visual words. In a first step, K-means clustering is used to detect visual concepts in the hyperspace of the energies of steerable Riesz wavelets. The coordinates of the clusters are used to construct templates from linear combinations of the Riesz components that are corresponding to visual concepts. The visualization of these templates allows verifying the relevance of the concepts modeled. Then, the local orientations of each template are optimized to maximize their response, which is carried out analytically and can still be expressed as a linear combination of the initial steerable Riesz templates. The texture classes are learned in the feature space composed of the concatenation of the maximum responses of each visual concept using support vector machines. An experimental evaluation using the Outex TC 00010 test suite allowed a classification accuracy of 97.5%, which demonstrates the feasibility of the proposed approach. An optimal number K = 20 of clusters is required to model the visual concepts, which was found to be fewer than the number of classes. This shows that higher-level classes are sharing low-level visual concepts. The importance of rotation-covariant visual concept modeling is highlighted by allowing an absolute gain of more than 30% in accuracy. The visual concepts are modeling the local organization of directions at various scales, which is in accordance with the bottom{up visual information processing sequence of the primal sketch in Marr's theory on vision.

Lately, multi-label classification (MLC) problems have drawn a lot of attention in a wide range of fields including medical, web, and entertainment. The scale and the diversity of MLC problems is much larger than single-label classification problems. Especially we have to face all possible combinations of labels. To solve MLC problems more efficiently, we focus on three kinds of locality hidden in a given MLC problem. In this paper, first we show how large degree of locality exists in nine datasets, then examine how closely they are related to labels, and last propose a method of reducing the problem size using one kind of locality.

Object recognition is a method in the computer vision to identify and recognize objects in the picture or video. When humans see photos or watch videos, they can quickly recognize some object like a car, bus, human, cat, food, and other visual artifacts. However, how do we apply it to the computer? Classification is the technique or method in object recognition that can be used on a computer to distinguish one object from another object contained in the image or video. In this paper, the author proposes about testing some popular image binary classification algorithms used along with the results of the performance matrix of each algorithm, among these are Logistic Regression with Perceptron, Multi-Layer Perceptron (MLP), Deep Multi-Layer Perceptron, and Convolutional Neural Network (ConvNet). The author uses the Food-5K dataset to distinguish two classes of objects, namely food /non-food, and then try to train and test how accurate the computer is in recognizing food and non-food objects, where it will be useful to anyone who needs to identify a food object using auto recognizing tools. This paper is expected to contribute in the field of computer vision related algorithm that is used to solve the problem in image classification, with the state of optimal hyperparameter and validation accuracy level above 90%. From the test results obtained the level of testing accuracy using ConvNet reached above 90% and loss function less than 25% while indicating that ConvNet has a significant advantage on the image classification problem compared to the generic artificial neural network.

Nowadays, Internet of Things (IoT) is becoming an irreplaceable role in human life. Moreover, the meter reading is becoming the procedure composed of picture collection and image recognition. Previous works treat meter reading as a problem of image classification. They only focus on the accuracy of classification but ignore numerical accuracy, which is the measurement’s essential performance. In this paper, we address that the meter reading is a hybrid regression and classification (HRC) problem. Under this definition, the resulting algorithm considers the targets of both measurement and digits recognition. To solve the HRC problem, we designed a hybrid regression and classification loss function and a multi-branch convolutional neural networks for numbers (N-CNNs). To further verify the effectiveness of the model’s classification and regression, we constructed two kinds of datasets: standard dataset and carry dataset. The N-CNNs establishes new state-of-the-art metrics both on regression and classification. Notably, the numerical precision of N-CNNs outperforms the classification-based methods. The numerical accuracy of the model has one order of magnitude higher than other models. Furthermore, we deployed N-CNNs in a realistic meter reading system based on smart meter shells and cloud computing.