Text Similarity Based on Post-training BERT

Measuring textual semantic similarity has been a subject of intense discussion in NLP and AI for many years. A new area of research has emerged that applies semantic similarity measures within Twitter. However, the development of these measures for the semantic analysis of tweets imposes fundamental challenges. The sparsity, ambiguity, and informality present in social media are hampering the performance of traditional textual similarity measures as have special syntactic and semantic characteristics. This paper reviews and evaluates the performance of topological, statistical, and hybrid similarity measures, in the context of Twitter analysis. Furthermore, the performance of each measure is compared against a naive keyword-based similarity computation method to assess the significance of semantic computation in capturing the meaning in tweets. An experiment is designed and conducted to evaluate the different measures through examining various metrics, including correlation, error rates, and statistical tests on a benchmark dataset. The potential weaknesses of semantic similarity measures in relation to Twitter applications of textual similarity assessment and the research contributions are discussed. This research highlights challenges and potential improvement areas for the semantic similarity of tweets, a resource for researchers and practitioners.

As ESG investment principles gain widespread traction in capital markets, discrepancies in ratings among agencies have drawn increasing attention. This study examines ESG reports from Chinese A-share listed companies between 2018 and 2023, employing natural language processing techniques to quantify textual manipulation characteristics across three dimensions: linguistic complexity, tone bias, and textual similarity. It systematically analyzes the impact of these factors on ESG rating divergence. The findings reveal that textual complexity, tone bias, and content similarity all significantly and positively influence rating divergence. Specifically, reports characterized by greater complexity, more positive tone, and higher content homogeneity are more likely to trigger interpretive differences among rating agencies. Concurrently, the level of analyst coverage plays a significant moderating role: higher information attention amplifies the effect of textual manipulation on rating divergence. Heterogeneity analysis reveals this effect is more pronounced in private enterprises and companies with higher separation of ownership and management responsibilities. This study sheds light on the formation mechanism of ESG rating divergence from a textual disclosure perspective, providing empirical evidence and policy recommendations for refining ESG reporting standards, enhancing rating system consistency, and improving information transparency in capital markets.

The rise of social media usage in the form of multimedia is on an exponential increase owing to the increased internet bandwidths in the recent past. As a result, people communicate in the form of videos and images a lot more than ever. One such video sharing and content developer platform is YouTube. YouTube has many features on video analytics in the form of recommendation systems, monetisation etc. It also offers many features for developers to evaluate their content and offers insights on the performance of their videos. Though these features are available, there is not even a single feature for developers to evaluate their content based on the performance of other's videos, which share the same nature of the content - the similarity between any two videos. Here, the similarity between two videos has a statistical measure apart from the content, which includes description and comments of a video. Thus, we propose an analysis of a query video and a range of videos to determine the most similar videos using statistical and textual similarity. The statistical similarity is evident from the number of derived features extracted from a video and the textual similarity is found by analysing the text data from the description and comments of a video. Experimental results show that the resultant similar videos are highly representative of both the statistical and textual similarity and can be used as a measure to compare two videos.

Intelligent communication processing in English aims to obtain effective information from unstructured text data using various text processing techniques. Text vector representation and text similarity calculation are important fundamental tasks in the whole field of natural language processing. In response to the shortcomings of existing sentence vector representation models and the singularity of text similarity algorithms, improved models and algorithms are proposed based on a thorough study of related domain technologies. This paper presents an in‐depth and comprehensive study of text vectorization representation and text similarity calculation algorithms in the field of natural language processing. The existing text vectorized representation models and text similarity computation algorithms are described, and their shortcomings are summarized to provide a basis for the background and significance of this paper, as well as to provide ideas for improvement directions. It is experimentally verified that the sentence vector model proposed in this paper achieves higher accuracy than the SIF sentence vector model for text classification tasks. In the task of text similarity computation, it achieves better results in three evaluation metrics: accuracy, recall, and F1 value. The algorithm also improves the computational efficiency of the model to a certain extent by removing feature words with low feature contribution. The algorithm first improves the deficiencies of the traditional word‐shift distance algorithm by defining multifeature fusion weights and realizes a text similarity calculation algorithm based on multifeature weighted fusion with better similarity calculation results. Then, a linear weighting model is constructed to further combine the similarity calculation results of the hierarchical pooled IIG‐SIF sentence vectors to realize the multimodel fusion text similarity calculation algorithm.

This paper presents P-GTM, a privacy-preserving text similarity algorithm that extends the Google Tri-gram Method (GTM). The Google Tri-gram Method is a high-performance unsupervised semantic text similarity method based on the use of context from the Google Web 1T n-gram dataset. P-GTM computes the semantic similarity between two input bag-of-words documents on public cloud hardware, without disclosing the documents' contents. Like the GTM, P-GTM requires the uni-gram and tri-gram lists from the Google Web 1T n-gram dataset as additional inputs. The need for these additional lists makes private computation of GTM text similarities a challenging problem. P-GTM uses a combination of pre-computation, encryption, and randomized preprocessing to enable private computation of text similarities using the GTM. We discuss the security of the algorithm and quantify its privacy using standard and real life corpora.

Do patent citations indicate knowledge linkage? The evidence from text similarities between patents and their citations

BackgroundComputational methods have been used to find duplicate biomedical publications in MEDLINE. Full text articles are becoming increasingly available, yet the similarities among them have not been systematically studied. Here, we quantitatively investigated the full text similarity of biomedical publications in PubMed Central.Methodology/Principal Findings72,011 full text articles from PubMed Central (PMC) were parsed to generate three different datasets: full texts, sections, and paragraphs. Text similarity comparisons were performed on these datasets using the text similarity algorithm eTBLAST. We measured the frequency of similar text pairs and compared it among different datasets. We found that high abstract similarity can be used to predict high full text similarity with a specificity of 20.1% (95% CI [17.3%, 23.1%]) and sensitivity of 99.999%. similarity and full text similarity have a moderate correlation (Pearson correlation coefficient: −0.423) when the similarity ratio is above 0.4. Among pairs of articles in PMC, method sections are found to be the most repetitive (frequency of similar pairs, methods: 0.029, introduction: 0.0076, results: 0.0043). In contrast, among a set of manually verified duplicate articles, results are the most repetitive sections (frequency of similar pairs, results: 0.94, methods: 0.89, introduction: 0.82). Repetition of introduction and methods sections is more likely to be committed by the same authors (odds of a highly similar pair having at least one shared author, introduction: 2.31, methods: 1.83, results: 1.03). There is also significantly more similarity in pairs of review articles than in pairs containing one review and one nonreview paper (frequency of similar pairs: 0.0167 and 0.0023, respectively).Conclusion/SignificanceWhile quantifying abstract similarity is an effective approach for finding duplicate citations, a comprehensive full text analysis is necessary to uncover all potential duplicate citations in the scientific literature and is helpful when establishing ethical guidelines for scientific publications.

The navigational safety of ships on waterways plays a crucial role in ensuring the operational efficiency of ports. Ship anomalous behavior detection is an important method of water traffic surveillance that can effectively identify abnormal ship behavior, such as sudden acceleration or deceleration. In order to detect potential anomalous ship behavior in real time, a method for ship anomalous behavior detection in waterways is proposed based on text similarity and kernel density estimation. Under the assumption of known traffic patterns entering and leaving the port, this method can identify ship behaviors that violate traffic patterns in real time. Firstly, kernel density estimation is applied to construct a traffic pattern density model for ship trajectories entering and leaving the port, used to estimate the density values of ship motion states. Simultaneously, a semantic transformation method is used to convert traffic pattern trajectory into pattern trajectory text, which is used to identify the ship’s traffic pattern. Subsequently, the historical trajectory data of the target ship are transformed into textual trajectories, and text similarity is used to identify ship inbound and outbound traffic patterns. Furthermore, the constructed traffic pattern density model is used to estimate real-time density values of the state of ship motion, and the trajectory points that exceed the threshold of the anomaly factor are marked as anomalies. Finally, the effectiveness of the proposed method is validated using simulation data, and the results indicate an accuracy of more than 90% for the comprehensive detection of anomalous behavior. This study, approaching the detection of potential ship anomalous behavior from the perspective of port traffic patterns, enriches the methods of ship anomalous behavior detection in port waterways.

Detection of similarity in text documents is needed to avoid the accumulation of information in a database or file system, one way to find out plagiarism between two documents by detecting the similarity of two documents to be compared to detect plagiarism, the Shingling Algorithm is an algorithm used for near-duplicate document search processes. Shingling's algorithm is implemented to detect similarities in text documents. The ease of copying information from one document to another is one of the effects of advances in information technology. And this leads to unwanted plagiarism, so that the document detection process can be minimized properly. In this study, the test results show that the application created can detect the similarity of text documents that have gone through various manipulations, namely scaling (enlarge / reduce), rotation, cropping (partially cut), and manipulating documents.

Geographic objects associated with descriptive texts are becoming prevalent, justifying the need for spatial-keyword queries that consider both locations and textual descriptions of the objects. Specifically, the relevance of an object to a query is measured by spatial-textual similarity that is based on both spatial proximity and textual similarity. In this article, we introduce the Reverse Spatial-Keyword k -Nearest Neighbor (RSK k NN) query, which finds those objects that have the query as one of their k -nearest spatial-textual objects. The RSK k NN queries have numerous applications in online maps and GIS decision support systems. To answer RSK k NN queries efficiently, we propose a hybrid index tree, called IUR-tree (Intersection-Union R-tree) that effectively combines location proximity with textual similarity. Subsequently, we design a branch-and-bound search algorithm based on the IUR-tree. To accelerate the query processing, we improve IUR-tree by leveraging the distribution of textual description, leading to some variants of the IUR-tree called Clustered IUR-tree (CIUR-tree) and combined clustered IUR-tree (C 2 IUR-tree), for each of which we develop optimized algorithms. We also provide a theoretical cost model to analyze the efficiency of our algorithms. Our empirical studies show that the proposed algorithms are efficient and scalable.

PurposeRevenue maximization through improving click‐throughs is of great importance for price comparison shopping services (PCSSs) whose revenues directly depend on the number of click‐throughs of items in their itemsets. The purpose of this paper is to present an approach aiming to maximize the revenue of a PCSS by proposing effective itemset construction methods that can maximize the click‐throughs.Design/methodology/approachThe authors suggest three itemset construction methods, namely naïve method (NM), exhaustive method (EM), and local update method (LM). Specifically, NM searches for the best itemset for an item in terms of textual similarity between an item and an itemset, while EM produces the best itemset for each item for maximizing click‐throughs by considering all the possible memberships of the item. Finally, through combining NM and EM, the authors propose an LM that attempts to improve click‐throughs by locally updating the memberships of items according to their ranks in each itemset.FindingsThrough evaluation of the proposed methods based on a real‐world dataset, it has been found that improvement of click‐throughs is small when itemsets are constructed by using the textual similarity alone. However, significant improvement in the number of click‐throughs was achieved when considering items' membership updates dynamically.Originality/valueUnlike the previous studies that mainly focus on the textual similarity, the authors attempt to maximize the revenue through constructing itemsets that can result in more click‐throughs. By using the proposed methods, it is expected that PCSSs will be able to automatically construct itemsets that can maximize their revenues without the need for manual task.

This paper proposes an enhanced text matching model with augmented recurrent attention that utilizes interactive attention mechanisms. During vector encoding, the proposed model employs attention to interact between two input texts. Following the interaction, it leverages Bi-LSTM to re-encode the sequence at a more advanced level, enabling the model to comprehensively learn global information. Additionally, an attention mechanism is incorporated to emphasize the importance of high-weights words. Furthermore, a fusion layer is added to better integrate the two text segments into a single result, which facilitates subsequent text similarity computations. The model demonstrates a high accuracy in text similarity calculations.

Paraphrase identification (PI) is to recognize whether given two sentences are restatements of each other or not. In our study we propose an approach that exploits machine translation and text similarity metrics as features for PI. Machine learning algorithms like Support Vector Machine (SVM) with three different kernels, C4.5 Decision tree and Multinomial Naïve Bayes (NB) are trained with these features. We evaluated our system on Parder, Turkish paraphrase corpus. The experimental results show that the proposed approach offers state-of-the-art results.

Text similarity is an important index to measure the similarity between two or more texts. It is widely used in many fields of natural language processing tasks. With the maturity of deep learning technology, a large number of neural network models have been used to calculate text similarity and have achieved good results in similarity calculation task of sentences or short texts. Among them, Bert model has become a research hotspot in this field due to its excellent performance. However, the application effect of existing similarity algorithms on long texts is not ideal, and they cannot truly extract richer semantic information hidden in the structure of long text documents. This paper takes Chinese long text as the research object, proposes a long text similarity calculation method using sentence sequence instead of word level sequence, constructs a long text semantic representation model with semantic progressive fusion, solves the practical problems faced by applications or natural language processing tasks related to long text semantics, in order to breaks through the bottleneck of long text similarity calculation.

Patients' education is a vital strategy for understanding a disease by patients and proper management of the condition. Physicians and academicians frequently make customized education materials for their patients. An artificial intelligence (AI)-based writer can help them write an article. Chat Generative Pre-Trained Transformer (ChatGPT) is a conversational language model developed by OpenAI (openai.com). The model can generate human-like responses. We aimed to evaluate the generated text from ChatGPT for its suitability in patients' education. We asked the ChatGPT to list common dermatological diseases. It provided a list of 14 diseases. We used the disease names to converse with the application with disease-specific input (e.g., write a patient education guide on acne). The text was copied for checking the number of words, readability, and text similarity by software. The text's accuracy was checked by a dermatologist following the structure of observed learning outcomes (SOLO) taxonomy. For the readability ease score, we compared the observed value with a score of 30. For the similarity index, we compared the observed value with 15% and tested it with a one-sample t-test. The ChatGPT generated a paragraph of text of 377.43 ± 60.85 words for a patient education guide on skin diseases. The average text reading ease score was 46.94 ± 8.23 (P < 0.0001), and it indicates that this level of text can easily be understood by a high-school student to a newly joined college student. The text similarity index was higher (27.07 ± 11.46%, P = 0.002) than the expected limit of 15%. The text had a "relational" level of accuracy according to the SOLO taxonomy. In its current form, ChatGPT can generate a paragraph of text for patients' educational purposes that can be easily understood. However, the similarity index is high. Hence, doctors should be cautious when using the text generated by ChatGPT and must check for text similarity before using it.