Interactive Plot Manipulation using Natural Language

Robot grasping under the instruction of natural language has attracted increasing attention in various applications for its advantages in enabling natural and smooth human-robot interaction. At present, mainstream algorithms mainly solve problems of utilizing simple natural language instructions to guide the robot arm to perform some specific grasping. However, for two natural language instructions with different temporal logic and the same semantics, it is usually difficult for the robot to achieve semantic disambiguation, which further leads to the failure of the grasping task. In order to address this problem, we propose a new natural language instruction disambiguation method for robot grasping by combining sentence vector similarity calculation model and sentence temporal logic model. Firstly, the word vector is obtained through the Skip-gram model in Word2vec and a sentence vector is constructed. The semantic similarity of the sentence is then calculated by using the proposed cost function. Based on the semantic similarity of the sentence, the correct temporal logic form of the sentence is then extracted according to the temporal adverbial priority to further guide the grabbing process of the robot arm. The experimental results show that our method can successfully realize the semantic disambiguation for natural language instructions with different temporal logics and the same semantics, and further guide the robot arm to complete more complicated tasks than previous tasks.

Nowadays most of people of developed economy countries interact with software every day. As a result of computer systems expansion to all scope of people's activity the problem of transition from visual and command interfaces to natural language user interfaces is thrown into the sharp relief. Computational linguistics and natural language processing methods are described in this article. Represent a formal definition of a dialogue system as a Markov decision process. Methods of natural language manipulation are applied in machine translation software systems, systems of search and exchange the data, text annotation and expert systems. Prototype of natural language user interface to structured data source is developed. As a result it is convert natural language user's query to SQL query to database. Also showed application of Levenshtein distance for processing natural language query to SQL query to database. Natural language user interface is created to predefined subject field. User interface interacts to database that contain information about existent program libraries and frameworks. Consequently, using natural language processing methods it is possible to develop user natural language user interface providing capability to interact with machine.

Interactive programming environments , Interactive programming environments , مرکز فناوری اطلاعات و اطلاع رسانی کشاورزی

Social scientists are increasingly interested in analyzing the semantic information (e.g., emotion) of unstructured data (e.g., Tweets), where the semantic information is not natively present. Performing this analysis in a cost-efficient manner requires using machine learning (ML) models to extract the semantic information and subsequently analyze the now structured data. However, this process remains challenging for domain experts. To demonstrate the challenges in social science analytics, we collect a dataset, QUIET-ML, of 120 real-world social science queries in natural language and their ground truth answers. Existing systems struggle with these queries since (1) they require selecting and applying ML models, and (2) more than a quarter of these queries are vague, making standard tools like natural language to SQL systems unsuited. To address these issues, we develop LEAP, an end-to-end library that answers social science queries in natural language with ML. LEAP filters vague queries to ensure that the answers are deterministic and selects from internally supported and user-defined ML functions to extend the unstructured data to structured tables with necessary annotations. LEAP further generates and executes code to respond to these natural language queries. LEAP achieves a 100% pass @ 3 and 92% pass @ 1 on QUIET-ML, with a $1.06 average end-to-end cost, of which code generation costs $0.02.

A robot as a coworker or a cohabitant is becoming mainstream day-by-day with the development of low-cost sophisticated hardware. However, an accompanying software stack that can aid the usability of the robotic hardware remains the bottleneck of the process, especially if the robot is not dedicated to a single job. Programming a multi-purpose robot requires an on the fly mission scheduling capability that involves task identification and plan generation. The problem dimension increases if the robot accepts tasks from a human in natural language. Though recent advances in NLP and planner development can solve a variety of complex problems, their amalgamation for a dynamic robotic task handler is used in a limited scope. Specifically, the problem of formulating a planning problem from natural language instructions is not studied in details. In this work, we provide a non-trivial method to combine an NLP engine and a planner such that a robot can successfully identify tasks and all the relevant parameters and generate an accurate plan for the task. Additionally, some mechanism is required to resolve the ambiguity or missing pieces of information in natural language instruction. Thus, we also develop a dialogue strategy that aims to gather additional information with minimal question-answer iterations and only when it is necessary. This work makes a significant stride towards enabling a human-like task understanding capability in a robot.

: If we want to build intelligent information retrieval systems, we will have to give them the capabilities of understanding Natural language, automatically organizing and reorganizing their memories, and using intelligent heuristics for searching their memories. These systems will have to analyze and understand both new text and Natural Language queries. In answering questions, they will have to direct memory search to reasonable places. This requires good organization of both the conceptual content of text and knowledge necessary for understanding those texts and accessing memory. The CYRUS and FRUMP systems (Kolodner (1978), Schank and Kolodner (1979), Dejong (1979)) comprise an information retrieval system called CyFr. Together, they have the analysis and retrieval capabilities mentioned above. FRUMP analysis news stories from the UPI wire for their conceptual content, and produces summaries of those stories. It sends summaries of stories about important people to CYRUS automatically adds those stories to its memory, and can then retrieve that information to answer question posed to it in natural language. This paper describes the problems involved in building such an intelligent system. It proposes solutions to some of those problems bases on recent research in Artificial Intelligence and Natural Language processing, and describes the CyFr system, which implements those solutions. The solutions we propose and implement are based on a model of human understanding and memory retrieval. (Author)

Extraction of knowledge data from knowledge database using natural language query is a difficult task. Different types of natural language processing (NLP) techniques have been developed to handle this knowledge data extraction task. This paper proposes an automated query-response model termed Extended Automated Knowledge Provider System (EAKPS) that can manage various types of natural language queries from user. The EAKPS uses combination based technique and it can handle assertive, interrogative, imperative, compound and complex type query sentences. The algorithm of EAKPS generates structure query language (SQL) for each natural language query to extract knowledge data from the knowledge database resident within the EAKPS. Extraction of noun or noun phrases is another issue in natural language query processing. Most of the times, determiner, preposition and conjunction are prefixed to a noun or noun phrase and it is difficult to identify the noun/noun phrase with prefix during query processing. The proposed system is able to identify these prefixes and extract exact noun or noun phrases from natural language queries without any manual intervention.

This paper describes two approaches to the automatic generation of behavioral VHDL models from descriptions written in natural language. Both approaches are based on a modeling style in which behavior is represented by a system of interconnected processes. The first approach employs a semantic grammar to directly generate a single VHDL process from a paragraph written in a restricted English called ModelSpeak. The second approach accepts more general English and generates models consisting of multiple processes. series of sentences describing the behavior of a single process and generated correct VHDL code for it. This interface requires the behavior be expressed in a highly restricted English called ModelSpeak (10). Conditional expressions (if statements) may be nested to two levels, and interaction is invoked by the interface to resolve scope ambiguity of loops. This interface is implemented with a semantic grammar (3) in Prolog. The second interface (Approach II) to the Modeler's Assistant for natural language is substantially more complex, but accepts more general English expressions and generates multiple-process models. In this approach, each sentence is parsed by a bottom-up, parallel chart parser to generate a set of valid parse trees. The trees are examined by a Semantic Analyzer (7) to determine one which is semantically correct, and a conceptual graph (14) (semantic network) of its meaning is generated. Once the conceptual graph of individual sentences are joined together (12), a program called the Linker (8) analyzes the graph to produce a Process Model Graph and VHDL process code for the Modeler's Assistant. Nodes in the conceptual graph representing verbs typically generate processes or the conditions which govern them. Nodes representing objects generate signals. The modeling style underlying the Linker produces a sizable collection of processes, each consisting of a single if_then statement. To avoid unnecessary bus resolutions, multiple processes assigning to the same signal are merged. Since signal types are generally not expressed in the English description, they may be added interactively later. Before describing the approaches in more detail, related research on natural language interfaces is reviewed briefly.

Natural language interface (NLI) enables an efficient and effective interaction by allowing a user to submit a single phrase in natural language to the system. Free hand gestures can be added to an NLI to specify the referents for deictic terms in speech. By combining NLI with other modalities to a multimodal user interface, speech utterance length can be reduced, and users need not clearly specify the referent verbally. Integrating deictic terms with deictic gestures is a critical function in multimodal user interface. This paper presents a novel approach to extend chart parsing used in natural language processing (NLP) to integrate multimodal input based on speech and manual deictic gesture. The effectiveness of the technique has been validated through experiments, using a traffic incident management scenario where an operator interacts with a map on large display at a distance and issues multimodal commands through speech and manual gestures. The preliminary experiment of the proposed algorithm shows encouraging results.

Tables are a ubiquitous data format for insight communication. However, transforming data into consumable tabular views remains a challenging and time-consuming task. To lower the barrier of such a task, research efforts have been devoted to developing interactive approaches for data transformation, but many approaches still presume that their users have considerable knowledge of various data transformation concepts and functions. In this study, we leverage natural language (NL) as the primary interaction modality to improve the accessibility of average users to performing complex data transformation and facilitate intuitive table generation and editing. Designing an NL-driven data transformation approach introduces two challenges: 1) NL-driven synthesis of interpretable pipelines and 2) incremental refinement of synthesized tables. To address these challenges, we present NL2Rigel, an interactive tool that assists users in synthesizing and improving tables from semi-structured text with NL instructions. Based on a large language model and prompting techniques, NL2Rigel can interpret the given NL instructions into a table synthesis pipeline corresponding to Rigel specifications, a declarative language for tabular data transformation. An intuitive interface is designed to visualize the synthesis pipeline and the generated tables, helping users understand the transformation process and refine the results efficiently with targeted NL instructions. The comprehensiveness of NL2Rigel is demonstrated with an example gallery, and we further confirmed NL2Rigel's usability with a comparative user study by showing that the task completion time with NL2Rigel is significantly shorter than that with the original version of Rigel with comparable completion rates.

The database interface, called the "natural language interface", uses natural language to allow users to access data without the need for SQL queries. Using query language to interact with databases is always a sophisticated issue. Due to this complexity, the client must use the clear reports that are included in some pre-executed programming projects when using information that is already present in database points of confinement. But, users can make it possible for each nonprofessional user to ask questions and provide requirements in natural language, which the computer can then process to produce the appropriate data. This study introduces a system developed using the "Natural Language Interface to Database" (NLIDB). This approach has led to the emergence of "experts" who can detect compound sentences in English and use them in lessons. After parsing the input sentences, the natural language is converted to SQL.

Natural language interfaces for robot control aspire to find the best sequence of actions that reflect the behavior intended by the instruction. This is difficult because of the diversity of language, variety of environments, and heterogeneity of tasks. Previous work has demonstrated that probabilistic graphical models constructed from the parse structure of natural language can be used to identify motions that most closely resemble verb phrases. Such approaches however quickly succumb to computational bottlenecks imposed by construction and search the space of possible actions. Planning constraints, which define goal regions and separate the admissible and inadmissible states in an environment model, provide an interesting alternative to represent the meaning of verb phrases. In this paper we present a new model called the Distributed Correspondence Graph (DCG) to infer the most likely set of planning constraints from natural language instructions. A trajectory planner then uses these planning constraints to find a sequence of actions that resemble the instruction. Separating the problem of identifying the action encoded by the language into individual steps of planning constraint inference and motion planning enables us to avoid computational costs associated with generation and evaluation of many trajectories. We present experimental results from comparative experiments that demonstrate improvements in efficiency in natural language understanding without loss of accuracy.

Natural Language Interfaces (NLIs) are a viable, human-readable alternative to complex, formal query languages like SPARQL, which are typically used for accessing semantically structured data (e.g. RDF and OWL repositories). However, in order to cope with natural language ambiguities, NLIs typically support a more restricted language. A major challenge when designing such restricted languages is habitability-how easily, naturally and effectively users can use the language to express themselves within the con-strains imposed by the system. In this paper, we investigate two methods for improving the habitability of a Natural Language Interface: feedback and clarification dialogues. We model feedback by showing the user how the system interprets the query, thus suggesting repair through query reformulation. Next, we investigate how clarification dialogues can be used to control the query interpretations generated by the system. To reduce the cognitive overhead, clarification dialogues are coupled with a learning mechanism. Both methods are shown to have a positive effect on the overall performance and habitability.

An authoring tool for decision support systems in context questions of ecological knowledge

Structured Query Language (SQL) is commonly used in Relational Database Management Systems (RDBMS) and is currently one of the most popular data definition and manipulation languages. Its core functionality is implemented, with only some minor variations, throughout all RDBMS products. It is an effective tool in the process of managing and querying data in relational databases. This paper describes a method to effectively automate the conversion of a data query from a Natural Language Query (NLQ) to Structured Query Language (SQL) with Online Analytical Processing (OLAP) cube data warehouse objects. To obtain or manipulate the data from relational databases, the user must be familiar with SQL and must also write an appropriate and valid SQL statement. However, users who are not familiar with SQL are unable to obtain relevant data through relational databases. To address this, we propose a Natural Language Processing (NLP) model to convert an NLQ into an SQL query. This allows novice users to obtain the required data without having to know any complicated SQL details. The model is also capable of handling complex queries using the OLAP cube technique, which allows data to be pre-calculated and stored in a multi-dimensional and ready-to-use format. A multi-dimensional cube (hypercube) is used to connect with the NLP interface, thereby eliminating long-running data queries and enabling self-service business intelligence. The study demonstrated how the use of hypercube technology helps to increase the system response speed and the ability to process very complex query sentences. The system achieved impressive performance in terms of NLP and the accuracy of generating different query sentences. Using OLAP hypercube technology, the study achieved distinguished results compared to previous studies in terms of the speed of the response of the model to NLQ analysis, the generation of complex SQL statements, and the dynamic display of the results. As a plan for future work, it is recommended to use infinite-dimension (n-D) cubes instead of 4-D cubes to enable ingesting as much data as possible in a single object and to facilitate the execution of query statements that may be too complex in query interfaces running in a data warehouse. The study demonstrated how the use of hypercube technology helps to increase system response speed and process very complex query sentences.