High-throughput-derived biologically-inspired features for unconstrained face recognition

Many modern computer vision algorithms are built atop of a set of low-level feature operators (such as SIFT [1], [2]; HOG [3], [4]; or LBP [5], [6]) that transform raw pixel values into a representation better suited to subsequent processing and classification. While the choice of feature representation is often not central to the logic of a given algorithm, the quality of the feature representation can have critically important implications for performance. Here, we demonstrate a large-scale feature search approach to generating new, more powerful feature representations in which a multitude of complex, nonlinear, multilayer neuromorphic feature representations are randomly generated and screened to find those best suited for the task at hand. In particular, we show that a brute-force search can generate representations that, in combination with standard machine learning blending techniques, achieve state-of-the-art performance on the Labeled Faces in the Wild (LFW) [7] unconstrained face recognition challenge set. These representations outperform previous state-of-the-art approaches, in spite of requiring less training data and using a conceptually simpler machine learning backend. We argue that such large-scale-search-derived feature sets can play a synergistic role with other computer vision approaches by providing a richer base of features with which to work.

Recently, centre loss that aiming to assist Softmax loss with the objectives of both inter‐class dispension and intra‐class compactness simultaneously, has achieved remarkable performance on convolutional neural network‐based face recognition. However, its advantages highly rely on the centre feature assumption, which influences the capacity of the final obtained face features. Inspired by the centre loss approach, a novel Orientation Truncated Centre Learning is proposed, which takes advantage of an orientation truncated centre function to make the centre feature learning have more suitable orientation for deep face recognition. Three metrics are proposed to evaluate how discriminative are the distributions of the learned features for MNIST visualisation. Experimental results on several challenging benchmarks, including fine‐grained labelled faces in the wild (FGLFW), labelled faces in the wild (LFW), YouTube faces (YTF), and benchmark of large‐scale unconstrained face recognition (BLUFR), show that the proposed approach can easily generate more favourable results than several state‐of‐the‐art competitors.

A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic

The rise of IoT devices has led to a surge in data generation, necessitating efficient processing solutions. Traditional cloud-centric approaches face challenges like latency, bandwidth and privacy issues. Edge computing, a promising paradigm, enables data processing closer to the source, enhancing IoT-driven computer vision applications. This shift integrates edge computing frameworks with a study that proposed a novel drosophila food search-tuned convolutional neural network (DFS-CNN) and computer vision algorithms for real-time tasks like object detection and anomaly detection. We collected acquisition data from labeled Faces in the Wild (LFW) video and image dataset, the acquisition data were preprocessed using a bilateral filter for minimizing noise while maintaining sharp edges, and then filtered data were extracted using a histogram of oriented gradients (HOG) — the DFS-CNN model using the HOG feature set to detection the computer vision solution. The optimal DFS-CNN model was deployed on edge computing and is used in an IoT-based architecture to compute and transport data for real-time performance simulation using Tensor Flow Lite. The proposed method is compared to other traditional algorithms. The proposed DFS-CNN model detects objects in the LWT image with 96% accuracy. The proposed DFS-CNN model was used to address students’ inactivity status during the online exam, and that the outcome of the suggested method was tested with data latency, and real-time response, according to a comparative performance analysis.

Face Recognition (FR) problem is one of the significant fields in computer vision. FR is used to identify the faces that appear over distributed cameras over the network. The problem of face recognition can be divided into two categories, the first is recognition with more than one sample per person, which can be called traditional face recognition problem. The second is the recognition of faces using only a Single Sample Per Person (SSPP). The efficiency of face recognition systems decreases because of limited references especially (SSPP) and faces taken in the Operational Domain (OD) different from faces in the Enrollment Domain (ED) in illumination, pose, low-resolution, and blurriness. This paper proposed a method that deals with all problems related to face recognition with SSPP. 3D face reconstruction is used to increase the reference gallery set with different poses and generate a design domain dictionary to overcome the problem of limited reference. Besides, the design domain dictionary is used to feed different deep learning models. Face illumination transfer techniques are utilized to overcome the illumination problem. Labeled Faces in the Wild (LFW) dataset is used to train Super-Resolution Generative Adversarial Network (SRGAN) to overcome the low-resolution problem. Deblur Generative Adversarial Network (DeblurGAN) is trained on the LFW dataset to overcome the problem of blurriness. The proposed method is evaluated using the Chokepoint dataset and COX-S2V dataset. The final results confirm an overall enhancement in accuracy compared to techniques that use SSPP for face recognition (generic learning and face synthesizing approaches). Also, the proposed method outperforms of Traditional and Deep Learning (TDL) method accuracy, which uses SSPP for face recognition.

Face recognition remains a crucial issue in computer vision with various applications. This paper introduces an adapted method to tackle the challenges posed by mask-wearing during the COVID-19 pandemic. We propose modifications to the bag of convolutional neural networks (BoCNN) framework, which combines CNNs and the bag of words (BoW) approach. Our main contribution is customizing the BoCNN algorithm to identify faces with masks by focusing on the visible facial regions, particularly the eyes and eyebrows. Facial landmarks are detected, and the region of interest (ROI) is extracted using techniques such as Media Pipe. A pre-trained CNN is then applied to sections within the ROI, enabling robust feature extraction that captures intricate details such as lighting variations and facial expressions while reducing the impact of mask occlusions. The extracted features are pooled to create a comprehensive representation for recognition. Extensive experiments on the labeled faces in the wild (LFW) dataset, including masked face images, demonstrate the superiority of our adapted BoCNN approach over traditional BoW and deep feature extraction methods, especially accurately recognizing masked faces. Additionally, we assess the generalizability of our method across multiple datasets and discuss potential limitations and future research directions. The proposed BoCNN-based solution proves effective in recognizing masked faces, making it highly relevant for applications in security, human-computer interaction, and various other domains affected by the COVID-19 pandemic.

Most existing state-of-the-art deep learning algorithms discover sophisticated representations in huge datasets using convolutional neural networks (CNNs) that mainly adopt backpropagation (BP) algorithm as the backbone for training the face recognition problems. However, since decades ago, BP has been debated for causing trivial issues such as iterative gradient-descent operation, slow convergence rate, local minima, intensive human intervention, exhaustive computation, time-consuming, and so on. On the other hand, a competitive machine learning algorithm called extreme learning machine (ELM) emerged with extreme fast implementation and simple in theory has overcome the challenges faced by BP. The ELM advocates the convergence of machine learning and biological learning for pervasive learning and intelligence and has been extensively researched in widespread applications. Nonetheless, till date, none of the work of ELM has proved its competency in tackling face verification problem. Hence, in this paper, we are going to probe for the first time the feasibility of ELM-based network in handling the face verification task. We devise and propose a novel and distinguished hybrid local receptive field-based extreme learning machine with DeepID (hereinafter denoted as H-ELM-LRF-DeepID), to discriminate face pairs. The experimental results on the YouTube face database, labeled faces in the wild (LFW), and CelebFaces datasets have shed light upon the feasibility and usefulness of the H-ELM-LRF-DeepID in the face verification task.

Recent work in machine learning has significantly benefited semantic extraction tasks in computer vision, particularly for object recognition and image retrieval. We argue that the computer vision techniques that have been successfully applied in those settings can effectively be translated to other domains, such as audio. This claim is supported by recent results in music vs. speech classification, structure from sound, robust music identification and sound object recognition. This paper focuses on two such audio applications and demonstrates how ideas from computer vision map naturally to these problems.

Deep learning based computer vision approaches for smart agricultural applications

Despite the importance of few-shot learning, the lack of labeled training data in the real world makes it extremely challenging for existing machine learning methods because this limited dataset does not well represent the data variance. In this research, we suggest employing a generative approach using variational autoencoders (VAEs), which can be used specifically to optimize few-shot learning tasks by generating new samples with more intra-class variations on the Labeled Faces in the Wild (LFW) dataset. The purpose of our research is to increase the size of the training dataset using various methods to improve the accuracy and robustness of the few-shot face recognition. Specifically, we employ the VAE generator to increase the size of the training dataset, including the basic and the novel sets while utilizing transfer learning as the backend. Based on extensive experimental research, we analyze various data augmentation methods to observe how each method affects the accuracy of face recognition. The face generation method based on VAEs with perceptual loss can effectively improve the recognition accuracy rate to 96.47% using both the base and the novel sets.

Visual Object and Place Recognition are very important issues in computer vision and mobile robotics. In the literature, many approaches have been introduced to solve these problems. Still, one of the most widely used approaches in computer vision refers to as combining the machine learning algorithms to learn objects for optimal recognition and also its image descriptors to describe the image content completely. Thus, in this way, the system is able to learn and describe the structural features of objects or places more effectively, which in turn; it leads to a correct recognition of objects. This paper introduces a method that uses the Kernel Principle Component Analysis (KPCA) approach to extract features from the visual scene. According to this approach, a set of SIFT features is extracted from a given image, and then, the minimum Euclidean distance between all local features is computed from the visual codebook which was constructed by K-means previously. The kernel analysis applied to the distance result to get better computational performance in recognition. SVM was used for data analysis and the results indicate that KPCA method significantly outperforms PCA and BOW approaches on Caltech-101 object dataset and IDOL visual place dataset.

Quickly accessing the contents of a video is challenging for users, particularly for unstructured video, which contains no intentional shot boundaries, no chapters, and no apparent edited format. We approach this problem in the domain of lecture videos though the use of machine learning, to gather semantic information about the videos; and through user interface design, to enable users to fully utilize this new information. First, we use machine learning techniques to gather the semantic information. We develop a system for rapid automatic semantic tagging using a heuristic-based feature selection algorithm called Sort-Merge, by using large initial heterogeneous low-level feature sets (cardinality greater than 1K). We explore applying Sort-Merge to heterogeneous feature sets though two methods: early fusion and late fusion. Each takes different approaches to handling the different kinds of features in the heterogeneous set. We determine the most predictive feature sets for key-frame filters such as “has text”, “has computer source code”, or “has instructor motion”. Specifically we explore the usefulness of Harr Wavelets, Fast Fourier Transforms, Color Coherence Vectors, Line Detectors, Ink Features and Pan/Tilt/Zoom detectors. For evaluation, we introduce a “keeper” heuristic for feature sets, which provides a method of performance comparison against a baseline. Second, we create a user interface to allow the user to make use of the semantic tags we gathered though our computer vision and machine learning process. The interface is integrated into an existing video browser, which detected shot-like boundaries and presented a multitimeline view. The content within shot-like boundaries is represented by frames to which our new interface applies the generated semantic tags. Specifically, we make accessible the semantic concepts of 'text', 'code', 'presenter', and 'person motion.' The tags are detected in the simulated shots using the filters generated with our machine learning approach and are displayed to users using a user-customizable multi-timeline view. We also generate tags based on ASRgenerated transcripts that have been limited to the words provided in the index of the course text book. Each of these occurrences is aligned with the simulated shots. Each spoken word becomes a tag analogous to the visual concepts. A full Boolean algebra over the tags is provided to enable new composite tags such as 'text or code, but no presenter.' Finally, we quantify the effectiveness of our features and our browser through user studies, both observational and task driven. We find that users that use the full suite of tools performed a search task in 60% of the time of users without access to tags. We find that when users are asked to perform search tasks they follow a nearly fixed pattern of accesses, alternating between the use of tags and Keyframes, or between the use of Word Bubbles and the media player. Based on user behavior and feedback, we redesigned the interface to group spatially interface components that are used together, removed un-used components, and redesigned the display of Word Bubbles to match that of the Visual Tags. We found that users strongly preferred the Keyframe tool, as well as both kinds of tags. Users also either found the algebra very useful or not useful at all.

Supervised learning algorithms are to discover the hidden patterns of the statistics, assuming that the training data and the test data are from the same distribution. There are two challenges in the traditional supervised machine learning. One is that the test data distribution always differs largely from the training data distribution in the real world, while another is that there is usually very few labeled data to train a machine learning model. In such cases, transfer learning, which emphasizes the transfer of the previous knowledge from different but related domains and tasks, is recommended to deal with these problems. Traditional transfer learning methods care more about the data itself rather than the task. In fact, there is no one universal feature representation can perfectly benefit the model training work. But different feature representations can discover some independent latent knowledge from the original data. In this paper, we propose an instance-based transfer learning method, which is a weighted ensemble transfer learning framework with multiple feature representations. In our work, mutual information is applied as the smart weighting schema to measure the weight of each feature representation. Extensive experiments have been conducted on three facial expression recognition data sets: JAFFE, KDEF and FERG-DB. The experimental results demonstrate that our approach achieves better performance than the traditional transfer learning method and the non-transfer learning method.

Background: Making an automatic diagnosis based on virtual slides and whole slide imaging or even determining whether a case belongs to a single class, representing a specific disease, is a big challenge. In this work we focus on WHO Classification of Tumours of the Central Nervous System. We try to design a method which allows to automatically distinguish virtual slides which contain histopathologic patterns characteristic of glioblastoma – pseudopalisading necrosis and discriminate cases with neurinoma (schwannoma), which contain similar structures – palisading (Verocay bodies).Methods: Our method is based on computer vision approaches like structural analysis and shape descriptors. We start with image segmentation in a virtual slide, find specific patterns and use a set of features which can describe pseudopalisading necrosis and distinguish it from palisades. Type of structures found in a slide decides about its classification.Results: Described method is tested on a set of 49 virtual slides, captured using robotic microscope. Results show that 82% of glioblastoma cases and 90% of neurinoma cases were correctly identified by the proposed algorithm.Conclusion: Our method is a promising approach to automatic detection of nervous system tumors using virtual slides.

This initiative aims to quickly identify individuals who violate traffic laws and notify them of their violations via What’s App, reducing the burden on law enforcement officers." The major goal is to improve traffic law enforcement. The rise in road accidents is mostly due to widespread noncompliance with traffic rules, which includes failure to wear a helmet, exceeding speed limits, and taking the wrong route. By integrating cutting-edge approaches in computer vision, image processing, and machine learning, the system overcomes considerable problems connected with traffic rule breaches. The primary goals include accurately identifying individuals operating automobiles, validating helmet usage, and recognizing license plates using image processing techniques. Following that, automatic traffic challans are issued in compliance with the violations. In essence, the Intelligent Traffic Management System combines these technological breakthroughs to provide a complete strategy to improving traffic safety, reducing traffic offenses, and optimizing traffic law enforcement. This groundbreaking technology announces a path towards safer and better organized streets, representing a significant advancement in the field of intelligent traffic management.