Improvement and Evaluation of Pseudo Natural Programming Language

A knowledge-based system approach is applied to the design of a document retrieval system for online retrieval of bibliographic material. The main components of the system are a language interface that understands user queries in pseudo natural language (i.e. queries that are restricted to subject domain concepts), a retrieval component that combines the expertise of subject-domain and retrieval experts, a help system that aids users in formulating and reformulating queries, and a user model builder that infers specific user characteristics. A comprehensive mathematical model based on fuzzy sets and fuzzy relations has been defined for the thesaurus and the retrieval mechanism and a prototypical system has been implemented in a modular fashion to test system response to changes in model parameters.

A search of the web reveals that natural language is considered to be language which a human would consider natural and unnatural language is language a human would consider artificial. This clearly raises interpretation questions and plenty of leeway so I am going to take the latter view here and bend it somewhat to consider programming languages as particularly good examples of unnatural language.Programming languages are something like 60 years old. Their evolution tells us much about the triumph of creativity over parsimony; the need for diversity and the frequent triumph of politics over common sense or logic. Yet their minimum set of features is defined by the beauty and simplicity of the Böhm-Jacopini theorem [1] and their underlying similarity in information theoretic terms is shown in the theorem proved by myself in [3],[4].This essay is a reflection on programming languages both old and new, engineering, systems evolution and the role of education.KeywordsProgramming LanguageUnnatural LanguageDisparate SystemUnderlying SimilarityStandard PageThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Code retrieval is to find the code snippet from a large corpus of source code repositories that highly matches the query of natural language description. Recent work mainly uses natural language processing techniques to process both query texts (i.e., human natural language) and code snippets (i.e., machine programming language), however, neglecting the deep structured features of query texts and source codes, both of which contain rich semantic information. In this article, we propose an end-to-end deep graph matching and searching (DGMS) model based on graph neural networks for the task of semantic code retrieval. To this end, we first represent both natural language query texts and programming language code snippets with the unified graph-structured data, and then use the proposed graph matching and searching model to retrieve the best matching code snippet. In particular, DGMS not only captures more structural information for individual query texts or code snippets, but also learns the fine-grained similarity between them by cross-attention based semantic matching operations. We evaluate the proposed DGMS model on two public code retrieval datasets with two representative programming languages (i.e., Java and Python). Experiment results demonstrate that DGMS significantly outperforms state-of-the-art baseline models by a large margin on both datasets. Moreover, our extensive ablation studies systematically investigate and illustrate the impact of each part of DGMS.

Research shows that errors in natural language can be corrected by translating texts to another language and back using language models. We explore to what extent this latent correction capability extends to Automated Program Repair (APR) by investigating Round-Trip Translation (RTT): translating code from one programming language into another programming or natural language and back, using Large Language Models (LLMs). We hypothesize that RTT restores patterns most commonly seen in the LLM’s training corpora through regression toward the mean , replacing infrequent bugs with more frequent, natural , bug-free code. To test this hypothesis, we employ nine LLMs and four common APR benchmarks in Java, and perform a detailed quantitative and qualitative analysis of RTT-generated patches. We find that RTT through English generates plausible patches for 100 of 164 bugs with GPT-4 on the HumanEval-Java benchmark, and 97 are found to be correct in our manual assessment. Moreover, RTT uniquely generates plausible patches for 46 bugs that were missed by LLMs specifically fine-tuned for APR. While this demonstrates the viability of RTT for APR, we also observe limitations, such as a lower overall bug fix rate than the state-of-the-art and diluting the original coding style. We analyze the impact of these limitations and discuss the potential of using RTT as a complementary component in APR frameworks.

Chapter 44 - Natural Language Interface Systems

Translation between programming languages and natural languages is an important solution to improve the understanding of coding in informatics students who often have difficulty understanding the syntax rules of programming languages. This research develops a translator application that is integrated with OCR technology and uses GPT Chat API to automatically translate text. This application supports seven programming languages (Python, Kotlin, CSS, Dart, HTML, Javascript, and Java) and two natural languages (Indonesian and English). This research uses the Waterfall software development method, starting from requirements analysis, system design, implementation, to testing using the blackbox method. The results show that this application can help users understand coding more effectively and efficiently through translation features, interactive quizzes, and user activity history. Thus, this application has the potential to be an innovative learning media that improves the programming skills of informatics students.

Programming language, natural language? Supporting the diverse computational activities of novice programmers

Code corpora, as observed in large software systems, are now known to be far more repetitive and predictable than natural language corpora. But why? Does the difference simply arise from the syntactic limitations of programming languages? Or does it arise from the differences in authoring decisions made by the writers of these natural and programming language texts? We conjecture that the differences are not entirely due to syntax, but also from the fact that reading and writing code is un-natural for humans, and requires substantial mental effort; so, people prefer to write code in ways that are familiar to both reader and writer. To support this argument, we present results from two sets of studies: 1) a first set aimed at attenuating the effects of syntax, and 2) a second, aimed at measuring repetitiveness of text written in other settings (e.g. second language, technical/specialized jargon), which are also effortful to write. We find that this repetition in source code is not entirely the result of grammar constraints, and thus some repetition must result from human choice. While the evidence we find of similar repetitive behavior in technical and learner corpora does not conclusively show that such language is used by humans to mitigate difficulty, it is consistent with that theory. This discovery of “non-syntactic” repetitive behaviour is actionable, and can be leveraged for statistically significant improvements on the code suggestion task. We discuss this finding, and other future implications on practice, and for research.

SEML: A Simplified English Modeling Language for Constructing Biological Models in Julia

In this paper we present Caesar, an intelligent domestic service robot. In domestic settings for service robots complex tasks have to be accomplished. Those tasks benefit from deliberation, from robust action execution and from flexible methods for human---robot interaction that account for qualitative notions used in natural language as well as human fallibility. Our robot Caesar deploys AI techniques on several levels of its system architecture. On the low-level side, system modules for localization or navigation make, for instance, use of path-planning methods, heuristic search, and Bayesian filters. For face recognition and human---machine interaction, random trees and well-known methods from natural language processing are deployed. For deliberation, we use the robot programming and plan language Readylog, which was developed for the high-level control of agents and robots; it allows combining programming the behaviour using planning to find a course of action. Readylog is a variant of the robot programming language Golog. We extended Readylog to be able to cope with qualitative notions of space frequently used by humans, such as "near" and "far". This facilitates human---robot interaction by bridging the gap between human natural language and the numerical values needed by the robot. Further, we use Readylog to increase the flexible interpretation of human commands with decision-theoretic planning. We give an overview of the different methods deployed in Caesar and show the applicability of a system equipped with these AI techniques in domestic service robotics.

Smart assistants are a technology that has become very popular today, due to the multiple functions they have, and somehow allow a natural interaction between devices and human beings. The objective is to develop an assistant based on NLP (Natural Language Processing) focused on the Java programming language, with the feature of guiding the user on the use of this language from reliable and validated sources, through speech recognition and speech synthesis by the assistant, allowing communication between the software and the user through natural language, in order to make the time in the consultation of information more efficient since it is considered that from the implementation of the speech to text engine and the subsequent processing of natural language, The search will be carried out on the web obtaining relevant results of precision. In the implemented methodology, the general requirements were defined, such as the voice recognition and identification modules, the search module and the internal storage structure, the design was based on a file-based controller view architecture, implemented through technologies such as PyQt5, Speech Recognition, Pyttsx3, Beautiful Soap among others, developed with the python programming language with the paradigm of object-oriented programming obtaining satisfactory results of precision in the searches by concepts and syntax carried out

The task of semantic code search is to retrieve code snippets from a source code corpus based on an information need expressed in natural language. The semantic gap between natural language and programming languages has for long been regarded as one of the most significant obstacles to the effectiveness of keyword-based information retrieval (IR) methods. It is a common assumption that “traditional” bag-of-words IR methods are poorly suited for semantic code search: our work empirically investigates this assumption. Specifically, we examine the effectiveness of two traditional IR methods, namely BM25 and RM3, on the CodeSearchNet Corpus, which consists of natural language queries paired with relevant code snippets. We find that the two keyword-based methods outperform several pre-BERT neural models. We also compare several code-specific data pre-processing strategies and find that specialized tokenization improves effectiveness.

Code summarization generates brief natural language description given a source code snippet, while code retrieval fetches relevant source code given a natural language query. Since both tasks aim to model the association between natural language and programming language, recent studies have combined these two tasks to improve their performance. However, researchers have yet been able to effectively leverage the intrinsic connection between the two tasks as they train these tasks in a separate or pipeline manner, which means their performance can not be well balanced. In this paper, we propose a novel end-to-end model for the two tasks by introducing an additional code generation task. More specifically, we explicitly exploit the probabilistic correlation between code summarization and code generation with dual learning, and utilize the two encoders for code summarization and code generation to train the code retrieval task via multi-task learning. We have carried out extensive experiments on an existing dataset of SQL and Python, and results show that our model can significantly improve the results of the code retrieval task over the-state-of-art models, as well as achieve competitive performance in terms of BLEU score for the code summarization task.

Pre-trained models for Natural Languages (NL) like BERT and GPT have been recently shown to transfer well to Programming Languages (PL) and largely benefit a broad set of code-related tasks. Despite their success, most current methods either rely on an encoder-only (or decoder-only) pre-training that is suboptimal for generation (resp. understanding) tasks or process the code snippet in the same way as NL, neglecting the special characteristics of PL such as token types. We present CodeT5, a unified pre-trained encoder-decoder Transformer model that better leverages the code semantics conveyed from the developer-assigned identifiers. Our model employs a unified framework to seamlessly support both code understanding and generation tasks and allows for multi-task learning. Besides, we propose a novel identifier-aware pre-training task that enables the model to distinguish which code tokens are identifiers and to recover them when they are masked. Furthermore, we propose to exploit the user-written code comments with a bimodal dual generation task for better NL-PL alignment. Comprehensive experiments show that CodeT5 significantly outperforms prior methods on understanding tasks such as code defect detection and clone detection, and generation tasks across various directions including PL-NL, NL-PL, and PL-PL. Further analysis reveals that our model can better capture semantic information from code.

The first international Workshop on Programming Language Processing presents interdisciplinary contributions that address programming language procession problems with machine learning and data mining techniques. Recently, there are lots of successful natural language processing methods. But the mining of programming languages could not exactly follow the manner of natural language processing. The difference between natural language and programming language brings in new research challenges and opportunities. The workshop will bring together researchers from machine learning, data mining and software engineering to discuss and debate the path forward for mining the value of programming languages.