Rewarding Coreference Resolvers for Being Consistent with World Knowledge

In many Multi Agent Systems, under-education agents investigate their environments to discover their targets. Any agent can also learn its strategy. In multitask learning, one agent studies a set of related problems together simultaneously, by a common model. In reinforcement learning exploration phase, it is necessary to introduce a process of trial and error to learn better rewards obtained from environment. To reach this end, anyone can typically employ the uniform pseudorandom number generator in exploration period. On the other hand, it is predictable that chaotic sources also offer a random-like series comparable to stochastic ones. It is useful in multitask reinforcement learning, to use teammate agents' experience by doing simple interactions between each other. We employ the past experiences of agents to enhance performance of multitask learning in a nondeterministic environment. Communications are created by operators of evolutionary algorithm. In this paper we have also employed the chaotic generator in the exploration phase of reinforcement learning in a nondeterministic maze problem. We obtained interesting results in the maze problem.

The performance of modern relation extraction systems is to a great degree dependent on the size and quality of the underlying training corpus and in particular on the labels. Since generating these labels by human annotators is expensive, Distant Supervision has been proposed to automatically align entities in a knowledge base with a text corpus to generate annotations. However, this approach suffers from introducing noise, which negatively affects the performance of relation extraction systems. To tackle this problem, we propose a probabilistic graphical model which simultaneously incorporates different sources of knowledge such as domain experts knowledge about the context and linguistic knowledge about the sentence structure in a principled way. The model is defined using the declarative language provided by Probabilistic Soft Logic. Experimental results show that the proposed approach, compared to the original distantly supervised set, not only improves the quality of such generated training data sets, but also the performance of the final relation extraction model.

In many Multi-Agent Systems, under-education agents investigate their environments to discover their target(s). Any agent can also learn its strategy. In multi-task learning, one agent studies a set of related problems together simultaneously, by a common model. In reinforcement learning exploration phase, it is necessary to introduce a process of trial and error to learn better rewards obtained from environment. To reach this end, anyone can typically employ the uniform pseudorandom number generator in exploration period. On the other hand, it is predictable that chaotic sources also offer a random-like series comparable to stochastic ones. It is useful in multi-task reinforcement learning, to use teammate agents' experience by doing simple interactions between each other. We employ the past experiences of agents to enhance performance of multi-task learning in a nondeterministic environment. Communications are created by operators of evolutionary algorithm. In this paper we have also employed the chaotic generator in the exploration phase of reinforcement learning in a nondeterministic maze problem. We obtained interesting results in the maze problem.

Distant supervision (DS) has been widely used for relation extraction (RE), which automatically generates large‐scale labeled data. However, there is a wrong labeling problem, which affects the performance of RE. Besides, the existing method suffers from the lack of useful semantic features for some positive training instances. To address the above problems, we propose a novel RE model with sentence selection and interaction representation for distantly supervised RE. First, we propose a pattern method based on the relation trigger words as a sentence selector to filter out noisy sentences to alleviate the wrong labeling problem. After clean instances are obtained, we propose the interaction representation using the word‐level attention mechanism‐based entity pairs to dynamically increase the weights of the words related to entity pairs, which can provide more useful semantic information for relation prediction. The proposed model outperforms the strongest baseline by 2.61 in F1‐score on a widely used dataset, which proves that our model performs significantly better than the state‐of‐the‐art RE systems.

Relation Extraction (RE) aims at harvesting relational facts from texts. A majority of existing research targets at knowledge acquisition from sentences, where subject-verb-object structures are usually treated as the signals of existence of relations. In contrast, relational facts expressed within noun phrases are highly implicit. Previous works mostly relies on human-compiled assertions and textual patterns in English to address noun phrase-based RE. For Chinese, the corresponding task is non-trivial because Chinese is a highly analytic language with flexible expressions. Additionally, noun phrases tend to be incomplete in grammatical structures, where clear mentions of predicates are often missing. In this article, we present an unsupervised Noun Phrase-based Open RE system for the Chinese language (NPORE), which employs a three-layer data-driven architecture. The system contains three components, i.e., Modifier-sensitive Phrase Segmenter, Candidate Relation Generator and Missing Relation Predicate Detector. It integrates with a graph clique mining algorithm to chunk Chinese noun phrases, considering how relations are expressed. We further propose a probabilistic method with knowledge priors and a hypergraph-based random walk process to detect missing relation predicates. Experiments over Chinese Wikipedia show NPORE outperforms state-of-the-art, capable of extracting 55.2 percent more relations than the most competitive baseline, with a comparable precision at 95.4 percent.

The World Wide Web includes semantic relations of numerous types that exist among different entities. Extracting the relations that exist between two entities is an important step in various Web-related tasks such as information retrieval (IR), information extraction, and social network extraction. A supervised relation extraction system that is trained to extract a particular relation type (source relation) might not accurately extract a new type of a relation (target relation) for which it has not been trained. However, it is costly to create training data manually for every new relation type that one might want to extract. We propose a method to adapt an existing relation extraction system to extract new relation types with minimum supervision. Our proposed method comprises two stages: learning a lower dimensional projection between different relations, and learning a relational classifier for the target relation type with instance sampling. First, to represent a semantic relation that exists between two entities, we extract lexical and syntactic patterns from contexts in which those two entities co-occur. Then, we construct a bipartite graph between relation-specific (RS) and relation-independent (RI) patterns. Spectral clustering is performed on the bipartite graph to compute a lower dimensional projection. Second, we train a classifier for the target relation type using a small number of labeled instances. To account for the lack of target relation training instances, we present a one-sided under sampling method. We evaluate the proposed method using a data set that contains 2,000 instances for 20 different relation types. Our experimental results show that the proposed method achieves a statistically significant macroaverage F-score of 62.77. Moreover, the proposed method outperforms numerous baselines and a previously proposed weakly supervised relation extraction method.

Ontology instances are typically stored as triples which associate two named entities with a pre-defined relational description. Sometimes such triples can be incomplete in that one entity is known but the other entity is missing. The automatic discovery of the missing values is closely related to relation extraction systems that extract binary relations between two identified entities. Relation extraction systems rely on the availability of accurately named entities in that mislabelled entities can decrease the number of relations correctly identified. Although recent results demonstrate over 80% accuracy for recognising named entities, when input texts have less consistent patterns, the performance decreases rapidly. This paper presents OntotripleQA which is the application of question-answering techniques to relation extraction in order to reduce the reliance on the named entities and take into account other assessments when evaluating potential relations. Not only does this increase the number of relations extracted, but it also improves the accuracy of extracting relations by considering features which are not extractable with only comparisons of the named entities. A small dataset was collected to test the proposed approach and the experiment demonstrates that it is effective on sentences from Web documents with an accuracy of 68% on average.

Natural language understanding depends heavily on assessing veridicality – whether the speaker intends to convey that events mentioned are actual, non-actual, or uncertain. However, this property is little used in relation and event extraction systems, and the work that has been done has generally assumed that it can be captured by lexical semantic properties. Here, we show that context and world knowledge play a significant role in shaping veridicality. We extend the FactBank corpus, which contains semantically driven veridicality annotations, with pragmatically informed ones. Our annotations are more complex than the lexical assumption predicts but systematic enough to be included in computational work on textual understanding. They also indicate that veridicality judgments are not always categorical, and should therefore be modeled as distributions. We build a classifier to automatically assign event veridicality distributions based on our new annotations. The classifier relies not only on lexical features like hedges or negations, but also structural features and approximations of world knowledge, thereby providing a nuanced picture of the diverse factors that shape veridicality

Recently, Multitask learning, which can cope with several tasks, has attracted much attention. Multitask Reinforcement Learning introduced by Tanaka et al is a problem class where number of problem instances of Markov Decision Processes sampled from the same probability distributions is sequentially given to reinforcement learning agents. The purpose of solving this problem is to realize adaptive agents for newly given environments by using knowledge acquired from past experience. Evolutionary Algorithms are often used to solve reinforcement learning problems if problem classes are quite different with Markov Decision Processes or state-action space is quite huge. From the viewpoint of Evolutionary Algorithms studies, the Multitask Reinforcement Learning problems are regarded as dynamic problems whose fitness landscape has changed temporally. In this paper, a memory-based Evolutionary Programming which is suitable for Multitask Reinforcement Learning problems is proposed.

Dexterous grasping and manipulation with multifingered robotic hands presents a significant challenge due to their high degrees of freedom and the need for task-specific adaptations. Existing methods usually adopt single-task learning framework or focus on simple stable wrap grasping, limiting their efficiency and generalization ability when encountering new task or precise functional grasping pose. In this article, we introduce MetaGrasp, a novel approach that defines dexterous functional grasping as a multitask reinforcement learning (RL) problem based on hand grasp pose classification. Our method features a unique gradual skill curriculum learning (GSCL) framework, which structures the learning process into three stages: beginner, intermediate, and advanced curriculum learning according to the level of difficulty. MetaGrasp leverages this hierarchical learning structure to develop a versatile, adaptive grasping policy that can grasp objects based on hand grasp pose and object point cloud inputs. Taking five hand grasp types as research cases, the trained policy with our MetaGrasp can be easily adpated to grasp different object instances from different object categories according to functional grasp intentions specified by one expert demonstration without requiring extensive system interaction. We categorize the dexterous functional grasping tasks of a five-fingered robotic hand into multiple tasks based on hand poses for RL, and to combine meta imitation learning (IL) with curriculum learning. The experimental results show that the MetaGrasp has better one-shot generalization ability on new grasp tasks, and outperforms state-of-the-art single-task dexterous grasping methods.

We propose a fully distributed actor-critic architecture, named diffusion-distributed-actor-criticDiff-DAC, with application to multitask reinforcement learning (MRL). During the learning process, agents communicate their value and policy parameters to their neighbours, diffusing the information across a network of agents with no need for a central station. Each agent can only access data from its local task, but aims to learn a common policy that performs well for the whole set of tasks. The architecture is scalable, since the computational and communication cost per agent depends on the number of neighbours rather than the overall number of agents. We derive Diff-DAC from duality theory and provide novel insights into the actor-critic framework, showing that it is actually an instance of the dual-ascent method. We prove almost sure convergence of Diff-DAC to a common policy under general assumptions that hold even for deep neural network approximations. For more restrictive assumptions, we also prove that this common policy is a stationary point of an approximation of the original problem. Numerical results on multitask extensions of common continuous control benchmarks demonstrate that Diff-DAC stabilises learning and has a regularising effect that induces higher performance and better generalisation properties than previous architectures.

In this paper, a new distributed multi-agent off-policy Actor-Critic algorithm for collaborative multitask reinforcement learning is proposed. The Critic stage is based on the distributed gradient temporal difference algorithm GTD(1), while the Actor stage is derived from a predefined global criterion function and consists of a complementary consensus-based exact policy gradient algorithm. A proof that the Feller-Markov properties hold for the derived algorithm at the Actor stage is derived. The weak convergence of the algorithm to the set of stationary points of an attached ODE is proved under mild conditions using the two-time-scale stochastic approximation arguments. An experimental verification of the algorithm properties is given, demonstrating its high efficiency and practical applicability.

In pursuit of reinforcement learning systems that could train in physical environments, we investigate multi-task approaches as a means to alleviate the need for massive data acquisition. In a tabular scenario where the Q-functions are collected across tasks, we model our learning problem as optimizing a higher order tensor structure. Recognizing that close-related tasks may require similar actions, our proposed method imposes a low-rank condition on this aggregated Q-tensor. The rationale behind this approach to multi-task learning is that the low-rank structure enforces the notion of similarity, without the need to explicitly prescribe which tasks are similar, but inferring this information from a reduced amount of data simultaneously with the stochastic optimization of the Q-tensor. The efficiency of our low-rank tensor approach to multi-task learning is demonstrated in two numerical experiments, first in a benchmark environment formed by a collection of inverted pendulums, and then into a practical scenario involving multiple wireless communication devices.

Robotic shepherding has become indispensable in animal husbandry and crowd management, offering a modern solution to traditional challenges. Drone Automated Shepherding leverages advanced maneuverability and an extensive field of view to improve the efficiency of these tasks, which are typically labor-intensive and time-consuming. Existing methods for managing large herds face significant challenges due to insufficient coordination among multiple drones and the complexities involved in simultaneously executing diverse shepherding tasks. This paper aims to enhance the execution of multiple shepherding tasks by optimizing drone coordination, designing optimal flight paths, and reducing flight time. To harness the potential of reinforcement learning, we develop a multi-drone collaborative shepherding environment that facilitates efficient drone training using a dense reward system. Additionally, we employ a multi-task deep reinforcement learning algorithm that enhances the sample efficiency and reward performance by leveraging shared information across tasks within this environment. Two specific tasks, driving and collecting, are used to assess the performance of our methodology. The effectiveness of our approach is measured against a classical solution named CTRL, examining metrics such as success rate, completion time, and flight path length. Results indicate that our approach significantly outperforms the CTRL in all measured metrics. Visualization of drone trajectories provides further evidence of our enhanced collaboration and efficiency in shepherding operations. Real-world experiments are conducted on a square of 300 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\text{m}^{3}$</tex-math></inline-formula>, where two drones utilize our method to guide four small autonomous vehicles from the starting area to the goal area within 30 seconds.

Multitask reinforcement learning with metadata-guided adaptive routing