Content‐based image retrieval for super‐resolutioned images using feature fusion: Deep learning and hand crafted

To Identify the Particular image from a huge dataset is key problem in Image Processing. To retrieve the exact image a block truncation coding based method is used. To derive feature descriptors Dot Diffusion Block Truncation Coding (DDBTC) is used. The image features derived from color quantizer as well as Bitmap image. Using color quantizer Color Histogram Features (CHF) is obtained and using Bitmap image Bit Pattern Feature (BPF) is obtained. The various distance measures are employed to match the sameness between images. Calculated results are better in term of Average Recall Rate (ARR) and Average Precision rate (APR).

Content-Based Image Retrieval (CBIR) has become one of the trending areas of research in computer vision. In traditional CBIR, the features in a spatial domain, such as color, texture, shape, and point features, are extracted. It is often considered that apart from the spatial features, the features extracted from the frequency domain of the images can give further information on the features of an image. In this paper, we are proposing two novel methods for the purpose of feature extraction from the 2-dimensional Discrete Cosine Transform (DCT) of an image: DCT_256_Zigzag and DCT_256_2 × 2. These methods take into consideration the lower frequencies in order to determine the features in the frequency domain. The advantage of using the zigzag scanning is to have the maximum low frequency values having Higher Energies comparatively. These two features are combined with two of the existing spatial domain features: Local Binary Patterns (LBP) and Interchannel voting features to generate a global feature vector for an image. For a query image, its feature vector is compared with feature vectors of every other image in the database using d1-distance, and the images with least distance are considered most similar images to the query image. To evaluate the efficiency of these two methods, five standard performance measures such as Average Precision Rate (APR), Average Recall Rate (ARR), F-Measure, Average Normalized Modified Retrieval Rank (ANMRR), and Total Minimum Retrieval Epoch (TMRE) are used. Six benchmark image datasets Corel-1000, Corel-5000, Corel-10000, VisTex, STex, and Color-Brodatz are used to corroborate the performance of these methods.KeywordsCBIRDCTFrequency domainInterchannel votingLBPTMRE

Due to the advancements in digital technologies and social networking, image collections are growing exponentially. The important aim in content-based image retrieval (CBIR) is to reduce the semantic gap issue that improves the performance of image retrieval. In this paper, the objective is achieved by introducing effective visual words fusion technique based on speeded-up robust features (SURF) and histograms of oriented gradients (HOG) feature descriptors. HOG is used to extract global features, whereas SURF is used for the extraction of local features. Global features are preferred for large-scale image retrieval, whereas local features perform better on those systems that support semantic queries with close visual appearance. Moreover, SURF is scale and rotation-invariant as compared to HOG descriptor and it works better for low illumination. On the contrary, HOG performs better for scene-recognition- or activity-recognition-based applications. In the proposed technique, visual words fusion of SURF and HOG feature descriptors is carried which performed better than features fusion of SURF and HOG feature descriptors as well as state-of-the-art CBIR techniques. The proposed technique based on visual words fusion gives classification accuracy of 98.40% using support vector machine while image retrieval accuracy of 80.61%. Qualitative and quantitative analyses performed on four standard image collections namely, Corel-1000, Corel-1500, Corel-5000, and Caltech-256 demonstrate the effectiveness of the proposed technique based on visual words fusion of SURF and HOG feature descriptors.

Cast shadow affects computer vision tasks such as image segmentation, object detection and tracking since objects and shadows share the same visual motion characteristics. This unavoidable problem decreases video surveillance system performance. The basic idea of this paper is to exploit the evidence that shadows darken the surface which they are cast upon. For this reason, we propose a simple and accurate method for detection of moving cast shadows based on chromatic properties in RGB, HSV and YUV color spaces. The method requires no a priori assumptions regarding the scene or lighting source. Starting from a normalization step, we apply canny filter to detect the boundary between self-shadow and cast shadow. This treatment is devoted only for the first sequence. Then, we separate between background and moving objects using an improved version of Gaussian mixture model. In order to remove these unwanted shadows completely, we use three change estimators calculated according to the intensity ratio in HSV color space, chromaticity properties in RGB color space, and brightness ratio in YUV color space. Only pixels that satisfy threshold of the three estimators are labeled as shadow and will be removed. Experiments carried out on various video databases prove that the proposed system is robust and efficient and can precisely remove shadows for a wide class of environment and without any assumptions. Experimental results also show that our approach outperforms existing methods and can run in real-time systems.

Multi-joint histogram based modelling for image indexing and retrieval

This research proposed the identification of chicken freshness level based on its color and texture features. Color Features used are the RGB (Red, Green, and Blue) and HSV (Hue, Saturation, Value) channel histogram value. texture features used are GLCM (Grey Level Co-Occurrence Matrix), Gabor kernel, and HOG (Histogram of Oriented Gradients). The freshness level of a chicken meat is categorized into three labels, fresh (0-4 hours after slaughtered), medium-fresh (4-6 hours after slaughtered), and not-fresh (more than 6 hours after slaughtered). The experiments will identify the freshness using several classification methods and different camera resolution and magnification. The highest classification accuracy using SVM (Support Vector Machines) achieves 58,33% with a smartphone camera, 98% with a webcam camera, and 79.1% with a 200 magnification digital microscope. From the experiment results, we can conclude that using webcam camera with normal resolution have better classification accuracy compared with a 200 magnification digital microscope or standard smartphone camera. It is also shown that SVM is superior compared with other methods tested in this experiments which are Decision Tree and Naive Bayes.

Pipeline right-of-way (ROW) monitoring and safety pre-warning is an important way to guarantee a safe operation of oil/gas transportation. Any construction equipment or heavy vehicle intrusion is a potential safety hazard to the pipeline infrastructure. Therefore, we propose a novel technique that can detect and classify an intrusion on oil/gas pipeline ROW. The detection part has been done based on our previous work, where we built a robust feature set using a pyramid histogram of oriented gradients in the Fourier domain with corresponding weights. Then a support vector machine (SVM) with radial basis kernel is used to distinguish threat objects from background. For the classification part, the object can be represented by an integrated color, shape and texture (ICST) feature set, which is a combination of three different feature extraction techniques viz. the color histogram of HSV (hue, saturation, value), histogram of oriented gradient (HOG), and local binary pattern (LBP). Then two decision making models based on K-nearest neighbor (KNN) and SVM classifier are utilized for automatic object identification. Using real-world dataset, it is observed that the proposed method provides promising results in identifying the objects that are present on the oil/gas pipeline ROW.

Image retrieval is the process of finding images in the database that are similar to the query image by measuring how close the feature values of the query image are to other images. Image retrieval is currently dominated by approaches that combine several different representations or features. The optimal weight of each feature is needed in combining the image features such as color features, texture features, and shape features. In this study, we use a multi-layer perceptron artificial neural network (MLP) method to obtain feature weights automatically and simultaneously look for optimal weights. The color moment is used to find nine color features, Gray Level Co-occurrence Matrix (GLCM) to find four texture features, and Hu Moment to find seven shape features totaling 20 neurons and all of these features become the input layer in our MLP network. Three neurons in output layers become the automatic weight of each feature. These weights are used to combine each feature's role in obtaining the relevant image. Euclidean Distance is used to measure similarity. The average precision values obtained using automatic feature weights are 93.94% for the synthetic dataset, 91.19% for the Coil-100 dataset, and 54.31% for the Wang dataset. These results have an average difference of 5.06% with the target so automatic feature weighting works well. This value is obtained at a hidden layer size of 11 and a learning rate of 0.1. In addition, the use of automatic feature weighting gives more accurate results compared to manual feature weighting.

Retrieval of similar images from large dataset of brain images across patients would help the experts in the decision diagnosis process of diseases. Generally used feature extraction methods are color, texture and shape. In medical images texture and shape features are most efficient. Histogram of Oriented Gradients (HOG) and Local Binary Pattern (LBP) are good descriptor for brain MR image retrieval. But there are many challenges facing in medical application. An empirical study of the impact of increasing bins number in the HOG descriptor concluded that larger the number is more accurate the descriptor is. In fact this is due to the reduction of orientations range that each bin covers. Despite the efficiency of augmenting the bins number, this technique has limited spatial support as the augmentation of the number of bins used leads to increase the histogram dimension. So here proposed a method called Histogram of Fuzzy Oriented Gradients (HFOG), in which a pixel can belong several bins with different degrees. The Local Binary Patterns feature extraction method is widely used for texture analysis; however, the original LBP is based on hard thresholding the neighborhood of each pixel. Therefore, texture representation with LBP is very sensitive to noise and cannot distinguish between a strong and a weak pattern. In this study, Fuzzy Local Binary Patterns was introduced to improve the original LBP.

Skin cancer is the most common type of cancer, a life-threatening condition that leads to serious health problems if not detected early, and its incidence is increasing worldwide. In recent years, computer vision and decision support systems have been used for disease detection in dermoscopic images. Furthermore, it has been observed that data representation methods affect the detection performance of these models, and the effect of color information on transformer-based models has not been sufficiently investigated. This study used the International Skin Imaging Collaboration (ISIC) 2017 dataset consisting of RGB images, and these images were converted into the HSV, LAB, and YCbCr color spaces. Transformer-based models, including visual transformer (ViT), swin transformer, data efficient image transformer (DeiT), and label-free self-distillation (DINO), were used for benign and malignant classification. According to the classification performance results, RGB and HSV color spaces particularly in the DeiT and Swin models, stable and high accuracy values ​​were obtained. It was observed that the ViT and DINO models were more sensitive to color space transformations and achieved lower classification performance compared to other models. The highest performance in skin cancer classification was achieved with the DeiT model trained in the RGB color space, with the highest accuracy (0.7668). Furthermore, the explainability-based gradient-weighted class activation mapping (Grad-CAM) method was used to analyze where the models focused in image regions when making classification decisions. This study shows the effect and usability of color space transformations in transformer-based models for skin cancer classification and offers a comparative contribution to the literature.

As online forums contain a vast amount of information that can aid in the early detection of fraud and extremist activities, accurate and efficient information extraction from forum sites is very important. In this paper, we discuss the limitations of existing works in the extraction of information from generic web sites and forum sites. We also identify the need for better suited, generalized and lightweight algorithms to carry out a more accurate and efficient information extraction while eliminating noisy data from forum sites. In this paper, we propose three generalized and lightweight algorithms to carry out accurate thread and post content extraction from web forums. We evaluate our algorithms based on two strict criteria and to the granularity of the (DOM tree) node level correctness. We consider a thread or post as successfully extracted by our algorithms only if (i) all the contents in its text and anchor nodes are extracted correctly, and (ii) each content node is grouped correctly according to its respective thread or post. Our experiments on ten different forum sites show that our proposed thread extraction algorithm achieves an average recall and precision rate of 100% and 98.66%, respectively, while our core post extraction algorithm achieves an average recall and precision rate of 99.74% and 99.79%, respectively.

According to the recent literature, it is pragmatic that image storage and retrieval through the World Wide Web (WWW) has made impressive progress. Practical searching for image still confronts us with present retrieval systems. A Content Based Image Retrieval (CBIR) system provides an efficient way of retrieving related images from image collections. In this paper we present a new technique to extract the images from the database to achieve better performance and efficiency. The novel method uses an approach, which combines texture information and modified Block Truncation Coding (BTC) method to extract the color features from the image. The whole image is divided into a fixed block of size 4 × 4, 8 × 8, 16 × 16, 32 × 32, 64 × 64, 128 × 128 and 256 × 256 non overlapped blocks. For each block, we calculated the threshold as a mean of the pixel values and the number of pixels which are lower than threshold value and upper than the threshold value are counted and mean of these lower and upper values have been taken and these two are considered as features. Now this process is repeated for the all blocks to extract the features of the image. The second step will compute the four texture features using the Gray-Level Co-occurrence Matrix (GLCM) as a statistical approach to extract energy, contrast, entropy and uniformity. We have used this modified BTC and texture features of the image to retrieve images. Finally, we have conducted experiments on a ground truth image database that has 1000 images of different categories. A different block sizes have been used to demonstrate the experimental results and compared with tabulation. From this we have observed that the performance is better and efficiency is low with smaller block sizes and vice versa with higher block sizes. So we have to choose the middle ranges of block sizes for better performance with better efficiency. As we have observed from the results that the performance of the retrieval system is improved because of combination of texture and color features. The proposed system's performance has been improved by an average precision of 2.41% and recall of 8.22% when compared to existing systems.

There is a growing trend for using content‐based image retrieval (CBIR) systems these days because of the constantly growing interest in digital content. Therefore, the ability of the CBIR to perform the CBIR process will depend on the feature extraction process and its basis, for the retrieval will be done on. Numerous researchers put forward various techniques for feature extraction to enhance the nature of the system. Since features play a very key role in enhancing performance, various features can be used collectively to attain the requisite goal. To retain this in mind, we present in this paper a multifeature fusion system, where three features are integrated and form one feature to improve the situation of retrieval. For this purpose, scale‐invariant feature transform (SIFT), speeded‐up robust features (SURF), and histogram of oriented gradients (HOG) features are adopted. These features are common features that deliver information about the shape of the object and for matching purposes, two techniques of distance matching such as Euclidean and Hausdrauff distance are adopted. To assess the performance of the proposed multifeature‐based CBIR approach, experiments were conducted with the usage of a MATLAB simulator. The Corel‐1000 dataset, consisting of 10,000 images in 100 semantic classes, turned into applied, with each magnificence containing 100 images. A subset of 2500 images across 50 semantic classes was used to train the system. This research aligns with industry, innovation, and infrastructure by contributing to advancements in image processing and retrieval systems. Key characteristic descriptors, along with SIFT, SURF, HOG, texture, and multicharacteristic combinations, were extracted for retrieval functions. The results display that the usage of the Hausdrauff distance as a similarity degree outperforms Euclidean distance, accomplishing retrieval accuracies of 80.02% for HOG, 77.9% for SIFT, 79. 8% for SURF, 77.2% for texture, and 84.2% for multicharacteristic combinations, surpassing Euclidean distance results via 1.7%–3.6% across capabilities. These findings underscore the effectiveness of Hausdrauff distance in enhancing retrieval precision within the CBIR framework.

The advances in biomedical imaging equipment have produced a massive amount of medical images that are generated by the different modalities. Consequently, a huge volume of data has been produced and caused a complex and time-consuming retrieving process of the relevant cases. To resolve this issue, the Content-Based Biomedical Image Retrieval (CBMIR) system is applied to retrieve the related images from the earlier patients’ databases. However, the previous handcrafted features methods that applied the CBMIR model have shown poor performance in many multimodal databases. In this paper, we focus on designing CBMIR technique using Deep Learning (DL) models. We present a new Multimodal Biomedical Image Retrieval and Classification (M-BMIRC) technique for retrieving and classifying the biomedical images from huge databases. The proposed M-BMIRC model involves three dissimilar processes as following: feature extraction, similarity measurement, and classification. It uses an ensemble of handcrafted features from Zernike Moments (ZM) and deep features from Deep Convolutional Neural Networks (DCNN) for feature extraction process. Additionally, the Hausdorff Distance based similarity measure is employed to identify the resemblance between the queried image and the images that exist in the database. Moreover, the classification process gets executed on the retrieval images using the Probabilistic Neural Network (PNN) model, which allocates the class labels of the tested images. Finally, the experimental studies are conducted using two benchmark medical datasets and the results ensure the superior performance of the proposed model in terms of different measures include Average Precision Rate (APR), Average Recall Rate (ARR), F-score, accuracy, and Computation Time (CT).

Medical image indexing and storage is gaining increased importance. Indexing and retrieval of these images efficiently is becomes an essential task. Indexing of medical images using text or numbers is a cumbersome task, difficult to memorize and time consuming. Indexing and Retrieval of images can be done through query by text and query by image which is also known as content based image retrieval (CBIR). Different medical images possess different texture and shape features. This paper presents a novel and hybrid approach of managing a huge medical image database and retrieving a medical image from the database by fusion of shape and texture features. Image is retrieved by comparing the features of query image and images of database. We attach some document information with each medical image such as patient identity, diseases, age, and case history. It can be used for further diagnostic and analysis purpose. Further the proposed algorithm also updates the database automatically if new query is found. MATLAB ® 7.01 and its image processing toolbox have been used to implement the algorithm.