• Home
  • Search
  • Local visual pattern modelling for image and video classification
  • https://doi.org/10.14264/uql.2017.521Copy DOI Icon

Local visual pattern modelling for image and video classification

Show More
  • Abstract
  • Literature Map
  • Similar Papers
Abstract

Local visual pattern modelling is one of the most important problems in the era of hand-crafted features where the aim is to bridge the gap between a set of local visual representations and some high-level vision tasks such as image and/or video classification. Recent years witnesses the over-whelming success of deep Convolutional Neural Networks (CNN), which poses new challenges and open problems for local pattern modelling. The thesis focuses on the following problems. They are 1) How to leverage the CNN architecture to effectively model the local patterns? 2) How to combine the merits of supervised coding and Fisher vector coding, which are two state-f-the-art coding schemes, to model the high-dimensional local representations, e.g., CNN descriptors, to lead to better classification performance? 3) How to discard the strong assumption that local patterns are distributed i.i.d. and model the dependencies between region-level CNN features for some high-level tasks? For the first question, we propose two CNN architectures to model the local features of videos for action recognition. While one piece of work focuses on how to design a network to effectively encode and aggregate multiple kinds of local features of a video, the other work proposes a convolutional pooling strategy to explore the temporal information hidden within the frame-level representations. These two works raise flexible CNN architectures that are compatible with video format and lead to promising action recognition performance. Supervised encoding and Fisher vector encoding are two representative schemes to create im- age representations. Both of them can achieve state-of-the-art image classification performance but through different strategies: the former extracts discriminative patterns from local features at the encoding stage while the latter preserves rich information into high-dimensional signatures derived from a generative model of local features. For the second problem, we propose a hybrid Fisher vector encoding scheme for image classification which combines the strategies from both of the above two encoding methods. The key idea is to leverage supervised encoding to decompose local features into a discriminative part and a residual part and then build a generative model based on this decomposition. For the third problem, we study a challenging problem of identifying unusual instances of known objects in images within an “open-world ” setting. That is, we aim to find objects that are members of a known class, but which are not typical of that class. We propose to identify unusual objects by inspecting the distributions of local visual patterns at multiple image regions. Considering the promising performance of Region CNN [37], we represent an image by a set of local CNN features and then map them into scalar detection scores to get rid of the distraction influence of irrelevant content. To model the region-level score distribution we propose to use Gaussian Process (GP) to iii construct two separate generative models, one for “regular object” and the other for “other objects”. We design a new covariance function to simultaneously model the detection score at a single location and the score dependencies between multiple regions. This treatment allows our method to capture the spatial dependencies between local regions, which turns out to be crucial for identifying unusual objects

Similar Papers
  • Supplementary Content

Texture Patterns for Object Recognition and Content-Based Color Image Retrieval

  • Dec 21, 2020
  • Griffith Research Online (Griffith University, Queensland, Australia)
  • Muhammad Asif
  • Conference Article
  • Citations2

Facial Expression Recognition using Local Directional Pattern variants and Deep Learning

  • Dec 21, 2018
  • Kennedy Chengeta +1
  • Research Article
  • Citations6

A FUSION OF HAND-CRAFTED FEATURES AND DEEP NEURAL NETWORK FOR INDOOR SCENE CLASSIFICATION

  • Apr 30, 2023
  • Malaysian Journal of Computer Science
  • Basavaraj S Anami +1
  • PDF
  • Research Article
  • Citations48

P2FEViT: Plug-and-Play CNN Feature Embedded Hybrid Vision Transformer for Remote Sensing Image Classification

  • Mar 26, 2023
  • Remote Sensing
  • Guanqun Wang +7
  • Conference Article
  • Citations14

Processing global and local features in convolutional neural network (CNN) and primate visual systems

  • May 14, 2018
  • Jun Huang +4
  • Research Article
  • Citations23

HCNN: A Neural Network Model for Combining Local and Global Features Towards Human-Like Classification

  • Dec 30, 2015
  • International Journal of Pattern Recognition and Artificial Intelligence
  • Tielin Zhang +2
  • Dissertation

Evolving Convolutional Neural Networks for Image and Text Classification Tasks

  • Nov 24, 2024
  • Trevor Londt
  • Research Article
  • Citations25

MSHT: Multi-stage Hybrid Transformer for the ROSE Image Analysis of Pancreatic Cancer.

  • Apr 01, 2023
  • IEEE Journal of Biomedical and Health Informatics
  • Tianyi Zhang +11
  • Research Article
  • Citations60

Compositional Model Based Fisher Vector Coding for Image Classification.

  • Jan 10, 2017
  • IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Lingqiao Liu +6
  • Book Chapter
  • Citations5

A Comparative Study of Modern Machine Learning Approaches for Focal Lesion Detection and Classification in Medical Images: BoVW, CNN and MTANN

  • Jan 01, 2018
  • Nima Tajbakhsh +1
  • Research Article
  • Citations77

Multi-scale hierarchical recurrent neural networks for hyperspectral image classification

  • Mar 13, 2018
  • Neurocomputing
  • Cheng Shi +1
  • Conference Article
  • Citations9

GPCNN: Evolving Convolutional Neural Networks using Genetic Programming

  • Dec 01, 2020
  • Abigail Mcghie +2
  • PDF
  • Research Article
  • Citations2

Application of convolutional neural networks in image classification and applications of improved convolutional neural networks

  • Nov 08, 2024
  • Applied and Computational Engineering
  • Taoyu Liu
  • Research Article
  • Citations26

Joint Coding of Local and Global Deep Features in Videos for Visual Search.

  • Jan 01, 2020
  • IEEE Transactions on Image Processing
  • Lin Ding +4
  • Conference Article
  • Citations41

Combining CNN with Hand-Crafted Features for Image Classification

  • Aug 01, 2018
  • Zhou Tianyu +2
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.