Generalized and lightweight algorithms for automated web forum content extraction

Content-Based Image Retrieval (CBIR) has become one of the trending areas of research in computer vision. In traditional CBIR, the features in a spatial domain, such as color, texture, shape, and point features, are extracted. It is often considered that apart from the spatial features, the features extracted from the frequency domain of the images can give further information on the features of an image. In this paper, we are proposing two novel methods for the purpose of feature extraction from the 2-dimensional Discrete Cosine Transform (DCT) of an image: DCT_256_Zigzag and DCT_256_2 × 2. These methods take into consideration the lower frequencies in order to determine the features in the frequency domain. The advantage of using the zigzag scanning is to have the maximum low frequency values having Higher Energies comparatively. These two features are combined with two of the existing spatial domain features: Local Binary Patterns (LBP) and Interchannel voting features to generate a global feature vector for an image. For a query image, its feature vector is compared with feature vectors of every other image in the database using d1-distance, and the images with least distance are considered most similar images to the query image. To evaluate the efficiency of these two methods, five standard performance measures such as Average Precision Rate (APR), Average Recall Rate (ARR), F-Measure, Average Normalized Modified Retrieval Rank (ANMRR), and Total Minimum Retrieval Epoch (TMRE) are used. Six benchmark image datasets Corel-1000, Corel-5000, Corel-10000, VisTex, STex, and Color-Brodatz are used to corroborate the performance of these methods.KeywordsCBIRDCTFrequency domainInterchannel votingLBPTMRE

An innovative image retrieval agenda by concatenating deep learning features from GoogleNet and low‐level features from HSI and RGB color space is proposed in this article. Most of the CNN features suffer from loss of information due to image resize as a pre‐processing stage. To reduce this information loss super‐resolution technic is used for resizing images. An improved form of dot‐diffused block truncation coding is used for extracting RGB handcraft features. To discover the interdependencies between color and intensity component of an image, interchannel voting between hue, saturation, and intensity component is calculated as a color feature in HSI space. Histogram of orientated gradient (HOG) feature is used as shape feature. Five standard performance parameters, average precision rate, average recall rate, F‐Measure, Average Normalized Modified Retrieval Rank, and Total Minimum Retrieval Epoch, are applied on nine image datasets: Corel‐1K, Corel‐5K, Corel‐10K, VisTex, STex, ColorBrodatz and three subsets of ImageNet dataset for evaluation process of proposed method. For all dataset the best performance is achieved by the proposed method with respect to all performance parameters.

The advances in biomedical imaging equipment have produced a massive amount of medical images that are generated by the different modalities. Consequently, a huge volume of data has been produced and caused a complex and time-consuming retrieving process of the relevant cases. To resolve this issue, the Content-Based Biomedical Image Retrieval (CBMIR) system is applied to retrieve the related images from the earlier patients’ databases. However, the previous handcrafted features methods that applied the CBMIR model have shown poor performance in many multimodal databases. In this paper, we focus on designing CBMIR technique using Deep Learning (DL) models. We present a new Multimodal Biomedical Image Retrieval and Classification (M-BMIRC) technique for retrieving and classifying the biomedical images from huge databases. The proposed M-BMIRC model involves three dissimilar processes as following: feature extraction, similarity measurement, and classification. It uses an ensemble of handcrafted features from Zernike Moments (ZM) and deep features from Deep Convolutional Neural Networks (DCNN) for feature extraction process. Additionally, the Hausdorff Distance based similarity measure is employed to identify the resemblance between the queried image and the images that exist in the database. Moreover, the classification process gets executed on the retrieval images using the Probabilistic Neural Network (PNN) model, which allocates the class labels of the tested images. Finally, the experimental studies are conducted using two benchmark medical datasets and the results ensure the superior performance of the proposed model in terms of different measures include Average Precision Rate (APR), Average Recall Rate (ARR), F-score, accuracy, and Computation Time (CT).

To Identify the Particular image from a huge dataset is key problem in Image Processing. To retrieve the exact image a block truncation coding based method is used. To derive feature descriptors Dot Diffusion Block Truncation Coding (DDBTC) is used. The image features derived from color quantizer as well as Bitmap image. Using color quantizer Color Histogram Features (CHF) is obtained and using Bitmap image Bit Pattern Feature (BPF) is obtained. The various distance measures are employed to match the sameness between images. Calculated results are better in term of Average Recall Rate (ARR) and Average Precision rate (APR).

We study in this paper the problem of incremental crawling of web forums, which is a very fundamental yet challenging step in many web applications. Traditional approaches mainly focus on scheduling the revisiting strategy of each individual page. However, simply assigning different weights for different individual pages is usually inefficient in crawling forum sites because of the different characteristics between forum sites and general websites. Instead of treating each individual page independently, we propose a list-wise strategy by taking into account the site-level knowledge. Such site-level knowledge is mined through reconstructing the linking structure, called sitemap, for a given forum site. With the sitemap, posts from the same thread but distributed on various pages can be concatenated according to their timestamps. After that, for each thread, we employ a regression model to predict the time when the next post arrives. Based on this model, we develop an efficient crawler which is 260% faster than some state-of-the-art methods in terms of fetching new generated content; and meanwhile our crawler also ensure a high coverage ratio. Experimental results show promising performance of Coverage, Bandwidth utilization, and Timeliness of our crawler on 18 various forums.

Obtaining canopy area, crown width, position, and other information from UAV aerial images and adjusting spray parameters in real-time according to this information is an important way to achieve precise pesticide application in orchards. However, the natural illumination environment in the orchard makes extracting the fruit tree canopy difficult. Hereto, an effective unsupervised image segmentation method is developed in this paper for fast fruit tree canopy acquisition from UAV images under natural illumination conditions. Firstly, the image is preprocessed using the shadow region luminance compensation method (SRLCM) that is proposed in this paper to reduce the interference of shadow areas. Then, use Naive Bayes to obtain multiple high-quality color features from 10 color models was combined with ensemble clustering to complete image segmentation. The segmentation experiments were performed on the collected apple tree images. The results show that the proposed method’s average precision rate, recall rate, and F1-score are 95.30%, 84.45%, and 89.53%, respectively, and the segmentation quality is significantly better than ordinary K-means and GMM algorithms.

The objective of Web forums is to create a shared space for open communications and discussions of specific topics and issues. The tremendous information behind forum sites is not fully-utilized yet. Most links between forum pages are automatically created, which means the link-based ranking algorithm cannot be applied efficiently. In this paper, we proposed a novel ranking algorithm which tries to introduce the content information into link-based methods as implicit links. The basic idea is derived from the more focused random surfer: the surfer may more likely jump to a page which is similar to what he is reading currently. In this manner, we are allowed to introduce the content similarities into the link graph as a personalization bias. Our method, named Fine-grained Rank (FGRank), can be efficiently computed based on an automatically generated topic hierarchy. Not like the topic-sensitive PageRank, our method only need to compute single PageRank score for each page. Another contribution of this paper is to present a very efficient algorithm for automatically generating topic hierarchy and map each page in a large-scale collection onto the computed hierarchy. The experimental results show that the proposed method can improve retrieval performance, and reveal that content-based link graph is also important compared with the hyper-link graph.

The representation of design information through ontologies has proven to be effective in fostering creative ideation within product design. Consequently, researchers have developed databases comprising models of engineering and biological systems by leveraging ontologies. However, the manual construction of a large number of models from technical documents is an effort-intensive task that demands specialized expertise. To address this challenge, researchers have investigated automatic information extraction methods utilizing data-intensive machine-learning models. However, previous research has not fully documented the end-to-end process of information extraction and representation and has not reported the end-to-end accuracy. This study introduces a novel method for automatic information extraction pertinent to the State change-Action-Part-Phenomenon-Input-oRgan-Effect (SAPPhIRE) model of causality alternative to creating data-intensive machine-learning models. This method employs the dependency parsing technique of natural language processing, along with rules supported by a lexical database, to extract words relevant to the SAPPhIRE model. Unlike previous approaches that rely on supervised learning methods, this new technique does not require extensive datasets for the training and validation of machine-learning models. Furthermore, it reports the end-to-end accuracy of information extraction, rather than focusing solely on the word classification task, which is preceded by manual pre-processing in prior research. The results of this newly developed method have been validated against SAPPhIRE models reported in the literature and through input provided by SAPPhIRE specialists and design researchers.

Although hyperspectral remotely sensed data are believed to be nonlinear, they are often modeled and processed by algorithms assuming that the data are realizations of some linear stochastic processes. This is likely due to the reason that either the nonlinearity of the data may not be strong enough, and the algorithms based on linear data assumption may still do the job, or the effective algorithms that are capable of dealing with nonlinear data are not widely available. The simplification on data characteristics, however, may compromise the effectiveness and accuracy of information extraction from hyperspectral imagery. In this paper, we are investigating the existence of non- linearity in hyperspectral data represented by a 4-m Airborne Visible/Infrared Imaging Spectrometer image acquired over an area of coastal forests on Vancouver Island. The method employed for the investigation is based on the statistical test using surrogate data, an approach often used in nonlinear time series analysis. In addition to the high-order autocorrelation, spectral angle is utilized as the discriminating statistic to evaluate the differences between the hyperspectral data and their surrogates. To facilitate the statistical test, simulated data sets are created under linear stochastic constraints. Both simulated and real hyperspectral data are rearranged into a set of spectral series where the spectral and spatial adjacency of the original data is maintained as much as possible. This paper reveals that the differences are statistically significant between the values of discriminating statistics derived from the hyperspectral data and their surrogates. This indicates that the selected hyperspectral data are nonlinear in the spectral domain. Algorithms that are capable of explicitly addressing the nonlinearity are needed for processing hyperspectral remotely sensed data.

This paper represents automatic facial expression analysis method named Local Directional Stigma Mean Patterns (LDSMP) for automatic facial expression analysis and image retrieval using content based facial expression image retrieval and CNN. The traditional local patterns such as Local Binary Patterns (LBP) and Local Ternary Patterns (LTP) are applied for face recognition and expression analysis, calculated using relationship between the center pixel and neighboring pixels. The proposed method calculates the eight directional difference values then divided into the three ranges based on threshold values. Thus, the values are substituted with basic three positive values (+3, +2, +1) and three negative values (-3, -2, -1) to get more sensitive information from an image rather than aforementioned methods. The threshold can be select either static which is selected by user or dynamic is evaluated from image itself and supports to improve the efficiency. The performance of the proposed method is further improved by giving this patterns as input to the Convolutional Neural Networks (CNN) and compared with the existing methods LBP, LTP, and Directional Binary Code (DBC) in terms of Average Precision (AP), Average Recall (AR), and Average Retrieval Rate (ARR) using standard databases COREL 10K (DB1) and JAFFE (The Japanese Female Facial Expression) (DB2) and Extended Cohn-Kanade (CK +) (DB3) dataset.

In this paper, a convolutional neural network model based on YOLO9000 is introduced to meet the need of real-time engineering computing. This network model can study and classify the targets in depth, aiming at the characteristics of scissors and aerosols. The characteristics have various kinds such as overlap, cover and multiscale. At the present stage, the average speed is 68 FPS on the windows platform with GPU (Geforce GTX Titan X) acceleration. In addition, the average precision and recall rate are 94. 5%, 92. 6%, respectively.

Ground points of lidar is the redundant information in the perception process of autonomous driving. Removal of ground points is beneficial to the improvement of accuracy and real-time performance in the perception process. Aiming at the problems of under-segmentation and over-segmentation in current methods, proposes a segmentation method based on point cloud normal clustering. First, the original point cloud is preprocessed to reduce the time cost of the whole process; then, the preprocessed point cloud is quickly pre-segmented by the ray threshold method, and the ground point cloud containing some non-ground points is obtained; finally, according to the normal vector features of ground point and non-ground points, we filter out the non-ground points in the ground point cloud to obtain the final ground point cloud. The experimental results show that the average precision and recall rate of this method in various road conditions can reach 90%, and the average time consumption is 50 ms, which has good real-time performance and adaptability.

Online advertisements (ads) have taken over the web, nowedays most websites contain some sort of ads. While ads produce revenue for the server maintainer or to businesses, they have become intrusive and dangerous as ever. The ads use more bandwidth, show inappropriate content, and spread malware such as adware and ransomware. Although there are many products to block ads, also known as ad blockers, most depend on static filter lists that must be managed manually and frequently updated. When malicious advertisers can produce millions of new URLs within minutes, this is not the most effective method against ads. In this paper we propose our own ad blocker, CIC-AB, which uses machine learning techniques to detect new and unknown ads without needing to update a filter list. The proposed ad blocker has been developed as an extension for the common browsers (e.g. Firefox and Chrome). It classifies URLs, both HTTP and HTTPS, as: non-ad, normal-ad and malicious-ad. The analysis showed the average precision, recall and False Positive rate of CIC-AB for five classifiers namely; Naive Bayes (NB), Support Vector Machine (SVM), K-Nearest Neighbour (KNN), Random Forest (RF) and Decision Tree (DT) is 97.16%, 94.96% and 3.38% respectively.

Multi-joint histogram based modelling for image indexing and retrieval

Extraction of left ventricular ejection fraction information from various types of clinical reports