Unsupervised Multi-Index Semantic Hashing

Fast similarity search is a key component in large-scale information retrieval, where semantic hashing has become a popular strategy for representing documents as binary hash codes. Recent advances in this area have been obtained through neural network based models: generative models trained by learning to reconstruct the original documents. We present a novel unsupervised generative semantic hashing approach, Ranking based Semantic Hashing (RBSH) that consists of both a variational and a ranking based component. Similarly to variational autoencoders, the variational component is trained to reconstruct the original document conditioned on its generated hash code, and as in prior work, it only considers documents individually. The ranking component solves this limitation by incorporating inter-document similarity into the hash code generation, modelling document ranking through a hinge loss. To circumvent the need for labelled data to compute the hinge loss, we use a weak labeller and thus keep the approach fully unsupervised. Extensive experimental evaluation on four publicly available datasets against traditional baselines and recent state-of-the-art methods for semantic hashing shows that RBSH significantly outperforms all other methods across all evaluated hash code lengths. In fact, RBSH hash codes are able to perform similarly to state-of-the-art hash codes while using 2-4x fewer bits.

We present ElasticHash, a novel approach for high-quality, efficient, and large-scale semantic image similarity search. It is based on a deep hashing model to learn hash codes for fine-grained image similarity search in natural images and a two-stage method for efficiently searching binary hash codes using Elasticsearch (ES). In the first stage, a coarse search based on short hash codes is performed using multi-index hashing and ES terms lookup of neighboring hash codes. In the second stage, the list of results is re-ranked by computing the Hamming distance on long hash codes. We evaluate the retrieval performance of ElasticHash for more than 120,000 query images on about 6.9 million database images of the OpenImages data set. The results show that our approach achieves high-quality retrieval results and low search latencies.

Hashing can compress heterogeneous high-dimensional data into compact binary codes while preserving the similarity to facilitate efficient retrieval and storage, and thus hashing has recently received much attention from information retrieval researchers. Most of the existing hashing methods first predefine a fixed length (e.g., 32, 64, or 128 bit) for the hash codes before learning them with this fixed length. However, one sample can be represented by various hash codes with different lengths, and thus there must be some associations and relationships among these different hash codes because they represent the same sample. Therefore, harnessing these relationships will boost the performance of hashing methods. Inspired by this possibility, in this study, we propose a new model jointly multiple hash learning (JMH), which can learn hash codes with multiple lengths simultaneously. In the proposed JMH method, three types of information are used for hash learning, which come from hash codes with different lengths, the original features of the samples and label. In contrast to the existing hashing methods, JMH can learn hash codes with different lengths in one step. Users can select appropriate hash codes for their retrieval tasks according to the requirements in terms of accuracy and complexity. To the best of our knowledge, JMH is one of the first attempts to learn multi-length hash codes simultaneously. In addition, in the proposed model, discrete and closed-form solutions for variables can be obtained by cyclic coordinate descent, thereby making the proposed model much faster during training. Extensive experiments were performed based on three benchmark datasets and the results demonstrated the superior performance of the proposed method.

With an ever increasing amount of data available on the web, fast similarity search has become the critical component for large-scale information retrieval systems. One solution is semantic hashing which designs binary codes to accelerate similarity search. Recently, deep learning has been successfully applied to the semantic hashing problem and produces high-quality compact binary codes compared to traditional methods. However, most state-of-the-art semantic hashing approaches require large amounts of hand-labeled training data which are often expensive and time consuming to collect. The cost of getting labeled data is the key bottleneck in deploying these hashing methods. Motivated by the recent success in machine learning that makes use of weak supervision, we employ unsupervised ranking methods such as BM25 to extract weak signals from training data. We further introduce two deep generative semantic hashing models to leverage weak signals for text hashing. The experimental results on four public datasets show that our models can generate high-quality binary codes without using hand-labeled training data and significantly outperform the competitive unsupervised semantic hashing baselines.

Semantic hashing is a crucial component of content based search and retrieval systems. To achieve an effective semantic hashing for images, it is essential to map them to hash space in a way that preserves the semantic information. Most state-of-the-art deep semantic hashing approaches do not fully take into account the structural information and the inherent hierarchy in the dataset. Also, the distribution of hash codes is primarily driven by semantic information that comes from the supervision labels. We propose a semantic hashing framework which utilizes the hyperbolic metric learning to learn the structural and hierarchical information. This information is leveraged in the form of proxy labels for training the hashing network with the proposed novel Structure-Semantic Disagreement (SSD) loss. SSD enforces the model to learn to hash with semantic as well as structural information, leading to more robust and uniformly distributed hash codes. Tests on multiple public domain datasets establish the effectiveness of the proposed approach. Moreover, the developed SSD loss can also be applied to other classification models to improve the representation by enforcing the model to use the structure information more effectively.

Recently, with the discoveries in neurobiology, high-dimensional sparse hashing has attracted increasing attention. In contrast with general hashing that generates low-dimensional hash codes, the high-dimensional sparse hashing maps inputs into a higher dimensional space and generates sparse hash codes, achieving superior performance. However, the sparse hashing has not been fully studied in hashing literature yet. For example, how to fully explore the power of sparse coding in cross-modal retrieval tasks; how to discretely solve the binary and sparse constraints so as to avoid the quantization error problem. Motivated by these issues, in this paper, we present an efficient sparse hashing method, i.e., High-dimensional Sparse Cross-modal Hashing, HSCH for short. It not only takes the high-level semantic similarity of data into consideration, but also properly exploits the low-level feature similarity. In specific, we theoretically design a fine-grained similarity with two critical fusion rules. Then we take advantage of sparse codes to embed the fine-grained similarity into the to-be-learnt hash codes. Moreover, an efficient discrete optimization algorithm is proposed to solve the binary and sparse constraints, reducing the quantization error. In light of this, it becomes much more trainable, and the learnt hash codes are more discriminative. More importantly, the retrieval complexity of HSCH is as efficient as general hash methods. Extensive experiments on three widely-used datasets demonstrate the superior performance of HSCH compared with several state-of-the-art cross-modal hashing approaches.

Similarity search plays an important role in many applications involving high-dimensional data. Due to the known dimensionality curse, the performance of most existing indexing structures degrades quickly as the feature dimensionality increases. Hashing methods, such as locality sensitive hashing (LSH) and its variants, have been widely used to achieve fast approximate similarity search by trading search quality for efficiency. However, most existing hashing methods make use of randomized algorithms to generate hash codes without considering the specific structural information in the data. In this paper, we propose a novel hashing method, namely, robust hashing with local models (RHLM), which learns a set of robust hash functions to map the high-dimensional data points into binary hash codes by effectively utilizing local structural information. In RHLM, for each individual data point in the training dataset, a local hashing model is learned and used to predict the hash codes of its neighboring data points. The local models from all the data points are globally aligned so that an optimal hash code can be assigned to each data point. After obtaining the hash codes of all the training data points, we design a robust method by employing l2,1 -norm minimization on the loss function to learn effective hash functions, which are then used to map each database point into its hash code. Given a query data point, the search process first maps it into the query hash code by the hash functions and then explores the buckets, which have similar hash codes to the query hash code. Extensive experimental results conducted on real-life datasets show that the proposed RHLM outperforms the state-of-the-art methods in terms of search quality and efficiency.

With the rapid proliferation of large-scale web images, recent years have witnessed more and more images labeled with user-provided tags, which leads to considerable effort made on hashing based image retrieval in huge databases. Current research efforts focus mostly on learning semantic hashing functions which designs compact binary codes to map semantically similar images to similar codes, and the visual similarity is not well explored for constructing semantic hashing functions. Here a novel approach is proposed to learn hashing functions that preserve semantic and visual similarity between images. Specifically, semantic hashing codes are first learned by leveraging the similarity between textual structure and visual structure; then, maximum entropy principle is exploited to achieve compact binary codes; finally, function decay principle is introduced to remove noisy visual attributes. Experimental results conducted on a widely-used image dataset demonstrate the proposed approach can effectively improve the performance in image retrieval.

Supervised discrete hashing (SDH) has widely attracted attention because it can directly obtain hash codes rather than relaxing the discrete constraint, which reduces information loss during the quantization process. However, SDH and its extensions tend to ignore the underlying geometry structure of data, while in many cases, data is lying on a low-dimensional manifold. Therefore, based on SDH, we propose a novel manifold learning-based supervised discrete hashing method in this paper to further improve the performance. Our goal is to learn compact hash codes by discovering latent information and using the local structure of data. To this end, we utilize the anchor graph to capture local neighborhood relationship embedding in data for manifold learning. Besides, since the previous methods use a strict binary label matrix for classification, we jointly optimize a relaxed label matrix to increase the discriminative ability of the proposed model. As such, the learned hash codes are more suitable for classification. Compared with SDH and other popular hashing methods, the experimental results on four large-scale datasets show that the proposed hashing method has better performance.

NSDH: A Nonlinear Supervised Discrete Hashing framework for large-scale cross-modal retrieval

Unsupervised hashing can desirably support scalable content-based image retrieval for its appealing advantages of semantic label independence, memory, and search efficiency. However, the learned hash codes are embedded with limited discriminative semantics due to the intrinsic limitation of image representation. To address the problem, in this paper, we propose a novel hashing approach, dubbed as discrete semantic transfer hashing (DSTH). The key idea is to directly augment the semantics of discrete image hash codes by exploring auxiliary contextual modalities. To this end, a unified hashing framework is formulated to simultaneously preserve visual similarities of images and perform semantic transfer from contextual modalities. Furthermore, to guarantee direct semantic transfer and avoid information loss, we explicitly impose the discrete constraint, bit-uncorrelation constraint, and bit-balance constraint on hash codes. A novel and effective discrete optimization method based on augmented Lagrangian multiplier is developed to iteratively solve the optimization problem. The whole learning process has linear computation complexity and desirable scalability. Experiments on three benchmark data sets demonstrate the superiority of DSTH compared with several state-of-the-art approaches.

Multi-view hashing efficiently integrates multi-view data for learning compact hash codes, and achieves impressive large-scale retrieval performance. In real-world applications, multi-view data are often stored or collected in different locations, where hash code learning is more challenging yet less studied. To fulfill this gap, this paper proposes a novel supervised multi-view distributed hashing (SMvDisH) for hash code learning from multi-view data in a distributed manner. SMvDisH yields the discriminative latent hash codes by joint learning of latent factor model and classifier. With local consistency assumption among neighbor nodes, the distributed learning problem is divided into a set of decentralized sub-problems. The sub-problems can be solved in parallel, and the computational and communication costs are low. Experimental results on three large-scale image datasets demonstrate that SMvDisH achieves competitive retrieval performance and trains faster than state-of-the-art multi-view hashing methods.

Deep hashing enables image retrieval by end-to-end learning of deep representations and hash codes from training data with pairwise similarity information. Subject to the distribution skewness underlying the similarity information, most existing deep hashing methods may underperform for imbalanced data due to misspecified loss functions. This paper presents Deep Priority Hashing (DPH), an end-to-end architecture that generates compact and balanced hash codes in a Bayesian learning framework. The main idea is to reshape the standard cross-entropy loss for similarity-preserving learning such that it down-weighs the loss associated to highly-confident pairs. This idea leads to a novel priority cross-entropy loss, which prioritizes the training on uncertain pairs over confident pairs. Also, we propose another priority quantization loss, which prioritizes hard-to-quantize examples for generation of nearly lossless hash codes. Extensive experiments demonstrate that DPH can generate high-quality hash codes and yield state-of-the-art image retrieval results on three datasets, ImageNet, NUS-WIDE, and MS-COCO.

With the rapid development of the Internet, recent years have seen the explosive growth of social media. This brings great challenges in performing efficient and accurate image retrieval on a large scale. Recent work shows that using hashing methods to embed high-dimensional image features and tag information into Hamming space provides a powerful way to index large collections of social images. By learning hash codes through a spectral graph partitioning algorithm, spectral hashing (SH) has shown promising performance among various hashing approaches. However, it is incomplete to model the relations among images only by pairwise simple graphs which ignore the relationship in a higher order. In this paper, we utilize a probabilistic hypergraph model to learn hash codes for social image retrieval. A probabilistic hypergraph model offers a higher order representation among social images by connecting more than two images in one hyperedge. Unlike a normal hypergraph model, a probabilistic hypergraph model considers not only the grouping information, but also the similarities between vertices in hyperedges. Experiments on Flickr image datasets verify the performance of our proposed approach.

Deep hashing has been widely applied in large scale image retrieval due to its high computation efficiency and retrieval performance. Recently, training deep hashing networks with a triplet ranking loss become a common framework. However, most of the triplet ranking loss based deep hashing methods cannot obtain satisfactory retrieval performance due to their ignoring the relative similarities among the objects. In this paper, we propose a method to learn the discriminative object features and utilize these features to compute the adaptive margins of the proposed loss for learning powerful hash codes. Experimental results show that our learned hash codes can yield state-of-the-art retrieval performance on three challenging datasets