Identification of Source Code Plagiarism Using a Natural Language Processing (NLP) Approach Based on Code Writing Style Analysis

Inspired by the great success of Deep Neural Networks (DNNs) in natural language processing (NLP), DNNs have been increasingly applied in source code analysis and attracted significant attention from the software engineering community. Due to its data-driven nature, a DNN model requires massive and high-quality labeled training data to achieve expert-level performance. Collecting such data is often not hard, but the labeling process is notoriously laborious. The task of DNN-based code analysis even worsens the situation because source code labeling also demands sophisticated expertise. Data augmentation has been a popular approach to supplement training data in domains such as computer vision and NLP. However, existing data augmentation approaches in code analysis adopt simple methods, such as data transformation and adversarial example generation, thus bringing limited performance superiority. In this paper, we propose a data augmentation approach MixCode that aims to effectively supplement valid training data, inspired by the recent advance named Mixup in computer vision. Specifically, we first utilize multiple code refactoring methods to generate transformed code that holds consistent labels with the original data. Then, we adapt the Mixup technique to mix the original code with the transformed code to augment the training data. We evaluate MixCode on two programming languages (Java and Python), two code tasks (problem classification and bug detection), four benchmark datasets (JAVA250, Python800, CodRep1, and Refactory), and seven model architectures (including two pretrained models CodeBERT and GraphCodeBERT). Experimental results demonstrate that MixCode outperforms the baseline data augmentation approach by up to 6.24% in accuracy and 26.06% in robustness.

Background. There is growing evidence that social and behavioral determinants of health (SBDH) play a substantial effect in a wide range of health outcomes. Electronic health records (EHRs) have been widely employed to conduct observational studies in the age of artificial intelligence (AI). However, there has been limited review into how to make the most of SBDH information from EHRs using AI approaches.Methods. A systematic search was conducted in six databases to find relevant peer-reviewed publications that had recently been published. Relevance was determined by screening and evaluating the articles. Based on selected relevant studies, a methodological analysis of AI algorithms leveraging SBDH information in EHR data was provided.Results. Our synthesis was driven by an analysis of SBDH categories, the relationship between SBDH and healthcare-related statuses, natural language processing (NLP) approaches for extracting SBDH from clinical notes, and predictive models using SBDH for health outcomes.Discussion. The associations between SBDH and health outcomes are complicated and diverse; several pathways may be involved. Using NLP technology to support the extraction of SBDH and other clinical ideas simplifies the identification and extraction of essential concepts from clinical data, efficiently unlocks unstructured data, and aids in the resolution of unstructured data-related issues.Conclusion. Despite known associations between SBDH and diseases, SBDH factors are rarely investigated as interventions to improve patient outcomes. Gaining knowledge about SBDH and how SBDH data can be collected from EHRs using NLP approaches and predictive models improves the chances of influencing health policy change for patient wellness, ultimately promoting health and health equity.

Nowadays, the dissemination of news information has become more rapid, liberal, and open to the public. People can find what they want to know more and more easily from a variety of sources, including traditional news outlets and new social media platforms. However, at a time when our lives are glutted with all kinds of news, we cannot help but doubt the veracity and legitimacy of these news sources; meanwhile, we also need to guard against the possible impact of various forms of fake news. To combat the spread of misinformation, more and more researchers have turned to natural language processing (NLP) approaches for effective fake news detection. However, in the face of increasingly serious fake news events, existing detection methods still need to be continuously improved. This study proposes a modified proof-of-concept model named NER-SA, which integrates natural language processing (NLP) and named entity recognition (NER) to conduct the in-domain and cross-domain analysis of fake news detection with the existing three datasets simultaneously. The named entities associated with any particular news event exist in a finite and available evidence pool. Therefore, entities must be mentioned and recognized in this entity bank in any authentic news articles. A piece of fake news inevitably includes only some entitlements in the entity bank. The false information is deliberately fabricated with fictitious, imaginary, and even unreasonable sentences and content. As a result, there must be differences in statements, writing logic, and style between legitimate news and fake news, meaning that it is possible to successfully detect fake news. We developed a mathematical model and used the simulated annealing algorithm to find the optimal legitimate area. Comparing the detection performance of the NER-SA model with current state-of-the-art models proposed in other studies, we found that the NER-SA model indeed has superior performance in detecting fake news. For in-domain analysis, the accuracy increased by an average of 8.94% on the LIAR dataset and 19.36% on the fake or real news dataset, while the F1-score increased by an average of 24.04% on the LIAR dataset and 19.36% on the fake or real news dataset. In cross-domain analysis, the accuracy and F1-score for the NER-SA model increased by an average of 28.51% and 24.54%, respectively, across six domains in the FakeNews AMT dataset. The findings and implications of this study are further discussed with regard to their significance for improving accuracy, understanding context, and addressing adversarial attacks. The development of stylometric detection based on NLP approaches using NER techniques can improve the effectiveness and applicability of fake news detection.

Determining the programming language of a source code file has been considered in the research community; it has been shown that Machine Learning (ML) and Natural Language Processing (NLP) algorithms can be effective in identifying the programming language of source code files. However, determining the programming language of a code snippet or a few lines of source code is still a challenging task. Online forums such as Stack Overflow and code repositories such as GitHub contain a large number of code snippets. In this paper, we describe Source Code Classification (SCC), a classifier that can identify the programming language of code snippets written in 21 different programming languages. A Multinomial Naive Bayes (MNB) classifier is employed which is trained using Stack Overflow posts. It is shown to achieve an accuracy of 75% which is higher than that with Programming Languages Identification (PLI-a proprietary online classifier of snippets) whose accuracy is only 55.5%. The average score for precision, recall and the F1 score with the proposed tool are 0.76, 0.75 and 0.75, respectively. In addition, it can distinguish between code snippets from a family of programming languages such as C, C++ and C#, and can also identify the programming language version such as C# 3.0, C# 4.0 and C# 5.0.

Nowadays, most of the information available on the web is in Natural Language. Extracting such knowledge from Natural Language text is an essential work and a very remarkable research topic in the Semantic Web field. The logic programming language Prolog, based on the definite-clause formalism, is a useful tool for implementing a Natural Language Processing (NLP) systems. However, web-based services for NLP have also been developed recently, and they represent an important alternative to be considered. In this paper we present the comparison between two different approaches in NLP, for the automatic creation of an OWL ontology supporting the semantic annotation of text. The first one is a pure Prolog approach, based on grammar and logic analysis rules. The second one is based on Watson Relationship Extraction service of IBM Cloud platform Bluemix. We evaluate the two approaches in terms of performance, the quality of NLP result, OWL completeness and richness.

How to identify class comment types? A multi-language approach for class comment classification

Geriatric syndromes (GS) are complex conditions that affect older adults and often require multidisciplinary assessment. Natural language processing (NLP) has emerged as a promising tool for extracting relevant clinical information from unstructured text in electronic health records (EHRs). However, the application of NLP in detecting and monitoring GS remains an evolving area of research. This systematic review explores the role of NLP in the identification and analysis of GS, examining its applications, methodologies, and effectiveness. Furthermore, this review discusses the existing challenges, limitations, and future directions to advance NLP applications in the GS research. We conducted a systematic literature search across ten databases to identify studies that applied NLP to GS detection. Articles were screened using predefined inclusion and exclusion criteria, and relevant studies were evaluated for quality using PROBAST. Data were extracted on study characteristics, datasets, annotation processes, NLP approaches, performance metrics, population demographics, and clinical applications. A PRISMA flow diagram was used to illustrate the study selection process. A total of 65 studies were included, where the majority of the studies used traditional rule-based and machine learning approaches. Publicly available datasets were scarce, and most studies used their private dataset, leading to significant variability in data sources and formats. Annotation methodologies differed across studies, with minimal shared guidelines or standards, making direct comparisons challenging. Performance metrics varied across syndromes, with F1-score, precision, and recall as the most commonly reported. Key challenges included the lack of dataset uniformity, differences in annotation practices, and the absence of external validation. NLP has shown potential in GS analysis, particularly for the detection of syndromes and epidemiological research. However, the majority of studies only focused on one syndrome, and variability in dataset availability, annotation processes, and model performance present challenges to broader implementation. Future research should focus on improving the comprehensiveness of GS identification, dataset standardisation, enhancing model generalisability, and integrating NLP approaches into clinical workflows.

BackgroundLarge language models have had a huge impact on natural language processing (NLP) in recent years. However, their application in epidemiological research is still limited to the analysis of electronic health records and social media data.ObjectivesTo demonstrate the potential of NLP beyond these domains, we aimed to develop prediction models based on texts collected from an epidemiological cohort and compare their performance to classical regression methods.MethodsWe used data from the British National Child Development Study, where 10,567 children aged 11 years wrote essays about how they imagined themselves as 25-year-olds. Overall, 15% of the data set was set aside as a test set for performance evaluation. Pretrained language models were fine-tuned using AutoTrain (Hugging Face) to predict current reading comprehension score (range: 0-35) and future BMI and physical activity (active vs inactive) at the age of 33 years. We then compared their predictive performance (accuracy or discrimination) with linear and logistic regression models, including demographic and lifestyle factors of the parents and children from birth to the age of 11 years as predictors.ResultsNLP clearly outperformed linear regression when predicting reading comprehension scores (root mean square error: 3.89, 95% CI 3.74-4.05 for NLP vs 4.14, 95% CI 3.98-4.30 and 5.41, 95% CI 5.23-5.58 for regression models with and without general ability score as a predictor, respectively). Predictive performance for physical activity was similarly poor for the 2 methods (area under the receiver operating characteristic curve: 0.55, 95% CI 0.52-0.60 for both) but was slightly better than random assignment, whereas linear regression clearly outperformed the NLP approach when predicting BMI (root mean square error: 4.38, 95% CI 4.02-4.74 for NLP vs 3.85, 95% CI 3.54-4.16 for regression). The NLP approach did not perform better than simply assigning the mean BMI from the training set as a predictor.ConclusionsOur study demonstrated the potential of using large language models on text collected from epidemiological studies. The performance of the approach appeared to depend on how directly the topic of the text was related to the outcome. Open-ended questions specifically designed to capture certain health concepts and lived experiences in combination with NLP methods should receive more attention in future epidemiological studies.

All across the world, social media is one of the most widely used platforms for information exchange. Comments on relevant information might be made in response to a video or any other piece of information. A remark may include an emotion that may be recognized by an automated recognition system. On Facebook, Twitter, and YouTube comments, we performed studies to determine their emotional categorization. A set of comments is gathered and manually classified using six fundamental emotion labels (happy, sad, angry, surprised, disgust, and fear) and one neutral label, with each emotion label representing a different emotion category. A prominent approach in natural language processing (NLP), deep learning has been used in a wide range of categorization applications. This procedure begins by preprocessing the input data with normalization, followed by categorizing characteristics in feature extraction utilizing the Linguistic and word count analysis (LIWC). Finally, for the categorization stage, the classify features might be supplied. Finally, for categorizing emotions, the Bidirectional Encoder based recurrent neural network classification approach is used. The studies have been carried out with the use of typical social media data that has been acquired from the kaggle data repository. The findings show that the suggested model outperforms all other existing mechanisms in terms of overall performance.

This chapter focuses on the computational and cognitive models that enhance Natural Language Processing, including machine learning algorithms, memory models, and attention mechanisms. It explores the latest advances in neural architectures like transformers and highlights the impact of transfer learning on Natural Language Processing. The chapter also examines human-like reasoning and memory models, which are essential for advancing the field.

This paper explores the fractal properties of token embedding spaces in GPT-2 language models by analyzing the stability of the correlation dimension, a measure of geometric complexity. Token embeddings represent words or subwords as vectors in a high-dimensional space. We hypothesize that the correlation dimension D2 remains consistent across different vocabulary subsets, revealing fundamental structural characteristics of language representation in GPT-2. Our main objective is to quantify and analyze the stability of D2 in these embedding subspaces, addressing the challenges posed by their high dimensionality. We introduce a new theorem formalizing this stability, stating that for any two sufficiently large random subsets S1,S2⊂E, the difference in their correlation dimensions is less than a small constant ε. We validate this theorem using the Grassberger–Procaccia algorithm for estimating D2, coupled with bootstrap sampling for statistical consistency. Our experiments on GPT-2 models of varying sizes demonstrate remarkable stability in D2 across different subsets, with consistent mean values and small standard errors. We further investigate how the model size, embedding dimension, and network depth impact D2. Our findings reveal distinct patterns of D2 progression through the network layers, contributing to a deeper understanding of the geometric properties of language model representations and informing new approaches in natural language processing.

One of the most widely used approaches in natural language processing and information retrieval is the so-called bag-of-words model. A common component of such methods is the removal of uninformative words, commonly referred to as stopwords. Currently, most practitioners use manually curated stopword lists. This approach is problematic because it cannot be readily generalized across knowledge domains or languages. As a result of the difficulty in rigorously defining stopwords, there have been few systematic studies on the effect of stopword removal on algorithm performance, which is reflected in the ongoing debate on whether to keep or remove stopwords. Here we address this challenge by formulating an information theoretic framework that automatically identifies uninformative words in a corpus. We show that our framework not only outperforms other stopword heuristics, but also allows for a substantial reduction of document size in applications of topic modelling. Our findings can be readily generalized to other bag-of-words-type approaches beyond language such as in the statistical analysis of transcriptomics, audio or image corpora. To better extract meaning from natural language, some less informative words can be removed before a model is trained, which is usually done by using manually curated lists of stopwords. A new information theoretic approach can identify uninformative words automatically and more accurately.

Challenges and Opportunities of Using Transformer-Based Multi-Task Learning in NLP Through ML Lifecycle: A Position Paper

Deep learning subsumes algorithms that automatically learn compositional representations. The ability of these models to generalize well has ushered in tremendous advances in many fields such as natural language processing (NLP). Recent research in the software engineering (SE) community has demonstrated the usefulness of applying NLP techniques to software corpora. Hence, we motivate deep learning for software language modeling, highlighting fundamental differences between state-of-the-practice software language models and connectionist models. Our deep learning models are applicable to source code files (since they only require lexically analyzed source code written in any programming language) and other types of artifacts. We show how a particular deep learning model can remember its state to effectively model sequential data, e.g., Streaming software tokens, and the state is shown to be much more expressive than discrete tokens in a prefix. Then we instantiate deep learning models and show that deep learning induces high-quality models compared to n-grams and cache-based n-grams on a corpus of Java projects. We experiment with two of the models' hyper parameters, which govern their capacity and the amount of context they use to inform predictions, before building several committees of software language models to aid generalization. Then we apply the deep learning models to code suggestion and demonstrate their effectiveness at a real SE task compared to state-of-the-practice models. Finally, we propose avenues for future work, where deep learning can be brought to bear to support model-based testing, improve software lexicons, and conceptualize software artifacts. Thus, our work serves as the first step toward deep learning software repositories.

Mark Watson Morgan Kaufmann Publishers, San Francisco, 1997, 377 pp. ISBN 1-55860-420-0, $44.95 This book is well written and organized to cover how to apply AI (Artificial Intelligence) techniques for practical Internet and Intranet tools by using the Java programming language. In the book, the author uses neural network and genetic algorithms to develop distributed systems that are applied for the Internet and Intranets, such as client/server model and information retrieval tool and so on. Part 1 begins with a good general introduction to AI. The history and survey of Artificial Intelligence research are given briefly. In addition, symbolic processing versus neural networks approaches are discussed. This author opts to use neural network to develop example Internet/Intranet systems in the book. Part 2 consists of Java programs developed for AI, neural networks, genetic algorithms, natural language processing, and expert systems. Chapter 2 illustrates Graphical User Interface (GUI) class that is derived from Applet and Canvas classes in Java, which is used in most example programs in the book. Chapter 3 introduces a frame data structure that is used as a message in distributed systems in the later chapters. Chapter 4 shows neural network classes developed in Java and its theory and implementation. Chapter 5 explains the genetic algorithms and the Java genetic algorithm classes. Chapter 6 describes how to develop and implement natural language processing classes using Java. Chapter 7 shows how to share data objects in distributed systems. For example, distributed AI agent classes are developed to share data objects, WWW data collection agent classes are shown to retrieve documents in the Internet, and small POP (Post Office Protocol) mail classes are developed to send and receive mails. Chapter 8 illustrates how to develop expert systems by using Java Jess, which is based on OPS5/CLIPS expert systems. Part 3 explains how to generate and implement the projects using the Java classes that are developed in Part 2. Chapter 9 shows the project that demonstrates the use of genetic algorithms in chapter 5 to control enemy ships in a scrolling arcade game. Chapter 10 describes the real-time handwriting recognition system developed by using the neural network classes in chapter 4. Chapter 11 illustrates a greedy algorithm to optimize neural networks that is based on the author’s hand-tuning neural training data. Chapter 12 improves neural network training data by applying genetic algorithms, to discard bad training data and to avoid discarding good training data. Chapter 13 describes the distributed natural language processing system that uses natural language processing classes in chapter 6. The data structures in chapter 2 are used to store historical data in a separate server application on any other computer in the Internet or Intranets. The final chapter provides the design and implementation of a complete agent-based information retrieval tool for the Internet. The retrieved documents are stored in ASCII or HTML format. This book is a very good reference book for anyone interested in developing systems using Java in distributed environments, especially applying AI techniques. Also, the author provides the CD-ROM that includes the source code for Java classes developed in the book. Haklin Kimm, University of Tennessee