A graph model with integrated pattern and query-based technique for extracting answer to questions in community question answering system

In this work, a best answer recommendation model is proposed for a Question Answering (QA) system. A Community Question Answering System was subsequently developed based on the model. The system applies Brouwer Fixed Point Theorem to prove the existence of the desired voter scoring function and Normalized Google Distance (NGD) to show closeness between words before an answer is suggested to users. Answers are ranked according to their Fixed-Point Score (FPS) for each question. Thereafter, the highest scored answer is chosen as the FPS Best Answer (BA). For each question asked by user, the system applies NGD to check if similar or related questions with the best answer had been asked and stored in the database. When similar or related questions with the best answer are not found in the database, Brouwer Fixed point is used to calculate the best answer from the pool of answers on a question then the best answer is stored in the NGD data-table for recommendation purpose. The system was implemented using PHP scripting language, MySQL for database management, JQuery, and Apache. The system was evaluated using standard metrics: Reciprocal Rank, Mean Reciprocal Rank (MRR) and Discounted Cumulative Gain (DCG). The system eliminated longer waiting time faced by askers in a community question answering system. The developed system can be used for research and learning purposes.

The task of finding- and ranking-related questions plays the most important role for any real-world Community Question Answering (cQA) systems. This paper proposes a new method to solve this problem by considering multi-views for measuring the similarities between the input questions and the question-answering pairs in the database. Our model will investigate various aspects for understanding questions. Beside the traditional features such as bag of n-grams, we will use more efficient aspects that include word embeddings and question categories. We will use a word representation model for generating word embeddings, a question classification module for determining the category for an input question. Then all these obtained features are combined into a machine learning-based framework for getting similarity existing question-answering pairs as well as for ranking these pairs. We tested our proposed approach on the dataset SemEval 2016 and the experiment shows obtained results with the Accuracy and MAP of 80.43% and 77.43%, respectively, which are the highest accuracies in comparison with previous studies.

Community question answering (CQA) systems aim to provide users with high-quality answers. Nevertheless, unreliable answers are often returned to users in CQA systems, and the phenomenon causes that users have to browse multiple answers to find the best one. To improve such problem, we design a novel scheme, named PW-CrossCMN. The scheme ranks the candidate answers by pair-wise approach based on numerous historical documents. In the scheme, we apply the preference relationship into deep learning framework. Specifically, the scheme consists of two phases. In phase 1, the scheme extracts the features via automated feature engineering to construct the preference vectors and then divides the vectors into balanced positive and negative training samples based on the preference relationship. In phase 2, we build the CrossCMN model, which implements the multi-network parallel convolution and the cross forward propagation of full-connected layers, to achieve training and prediction tasks. Moreover, the multi-layer perception (MLP) is introduced to extract combination features in the prediction module. We perform extensive experiments on two typical datasets, and the results show that our scheme has more excellent performance in answer ranking task compared with several state-of-the-art baselines. In addition, we have released the relevant codes.

User votes are important signals in community question-answering (CQA) systems. Many features of typical CQA systems, e.g. the best answer to a question, status of a user, are dependent on ratings or votes cast by the community. In a popular CQA site, Yahoo! Answers, users vote for the best answers to their questions and can also thumb up or down each individual answer. Prior work has shown that these votes provide useful predictors for content quality and user expertise, where each vote is usually assumed to carry the same weight as others. In this paper, we analyze a set of possible factors that indicate bias in user voting behavior -- these factors encompass different gaming behavior, as well as other eccentricities, e.g., votes to show appreciation of answerers. These observations suggest that votes need to be calibrated before being used to identify good answers or experts. To address this problem, we propose a general machine learning framework to calibrate such votes. Through extensive experiments based on an editorially judged CQA dataset, we show that our supervised learning method of content-agnostic vote calibration can significantly improve the performance of answer ranking and expert ranking.

Intelligent Community Question Answering (CQA) system is a popular research topic in Natural Language Processing (NLP) area. Recently, the development of distributed data mining has significantly improved the efficiency of the searching and sorting operations in CQA database. However, problems are left unanswered such as how to choose the best answer among multiple candidates for a single question. To solve this issue, we design a CQA system called UB-CQA. First, in database refinement part, we construct an optimized and structured database by the best answer choosing method leveraging user attribute provided by the response provider. Second, in human-computer part, the UB-CQA is able to search and provide a more satisfying answer to users by similarity calculation and re-ranking method leveraging text categorization information. Empirical evaluations show that the best answer choosing and the candidate question re-ranking methods bring great improvements in accuracy and reliability.

There have been increasing interests in Community Question Answering (CQA) recently. CQA websites such as Yahoo! Answers and Baidu Knows are increasingly popular, attracting tens of thousands of users to submit questions and answers every day. However, we find that there is a gap in the study of what kinds of questions are more likely to attract answers, which makes sense for users in practice when asking questions in the CQA systems. In this paper, we investigate the factors that may affect users’ questions to attract answers. We also introduce a general framework to predict how many replies a question is expected to receive. Evaluation results of the framework on real data prove its effectiveness.

At present, people no longer meet the way of communication between users and the Internet. And more and more people choose the interaction between users and users to get information. The community question answering system is one of the new information sharing model. In the community question answering system, users are not only the questioner but also the answer and the question is the link between the users. With the increasing number of users and the increasing number of questions and answers, it makes many questions which just were raised disappear in the category pages of the home page. Leading to the efficiency of the questions be answered greatly reduce. Aim at the recommended user’s interest, ability and time. In this paper we construct a dynamic user interest model and user expertise model. Experimental results show that the recommendation mechanism improves the efficiency of the recommendation to a certain extent.

Incorporating a duplicate question detection system can be beneficial for various systems such as community forums or question answering systems. Detecting question that have already an answer improves the user experience by reducing the search time and returning the correct answer. In this paper, we construct several methods for Arabic duplicate question detection based on machine learning. First, the pre-processing step is applied to clean and normalize questions. Next, we use Term Frequency Inverse Document Frequency (TF-IDF), Word2Vec, and FastText methods to map questions from their textual format into a vector space. Then, we trained various shallow learning methods (SVM, XGBoost, Random Forest, Logistic Regression) and deep learning methods (CNN, RNN, LSTM, GRU) with the objective of detecting if a pair of questions is duplicate or not. Various experiments were conducted to evaluate the performances of our models. The results obtained show that the deep learning model based on GRU with FastText representation performed better compared to the other models.

Community question answering (cQA) systems have enjoyed the benefits of advances in neural information retrieval, some models of which need annotated documents as supervised data. However, in contrast with the amount of supervised data for cQA systems, user-generated data in cQA sites have been increasing greatly with time. Thus, focusing on unsupervised models, we tackle a task of retrieving relevant answers for new questions from existing cQA data and propose two frameworks to exploit a Question Retrieval (QR) model for Answer Retrieval (AR). The first framework ranks answers according to the combined scores of QR and AR models and the second framework ranks answers using the scores of a QR model and best answer flags. In our experiments, we applied the combination of our proposed frameworks and a classical fusion technique to AR models with a Japanese cQA data set containing approximately 9.4M question-answer pairs. When best answer flags in the cQA data cannot be utilized, our combination of AR and QR scores with data fusion outperforms a base AR model on average. When best answer flags can be utilized, the retrieval performance can be improved further. While our results lack statistical significance, we discuss effect sizes as well as future sample sizes to attain sufficient statistical power.

Ranking question-answering pairs according to their similarities to each input question is very important for any real-world community Question Answering system. To address this problem we will propose the models which use Convolutional Neural Network and Bi-Directional Long Short Term Memory. The proposed models are formulated for both representation learning and question similarity score detection. Especially in this paper we will utilize various feature kinds including both abstract features (i.e. high level representation) and conventional features. We test our proposed model on the dataset SemEval 2016 and obtain the results with the Accuracy and MAP of 82.86% and 78.43% respectively, which are best in comparison with previous studies.

The measurement of semantic similarity is a fundamental task in natural language processing. In the settings of a community question answering (cQA) system, it is essentially a classification problem: given a pair of questions, label it similar, relevant, or irrelevant. Traditional methods, either those at word level or at sentence level, typically require many lexical and syntactic resources, which are not available in languages other than English. In addition, there does not exist a finely annotated dataset for our purpose. In this paper, we constructed a dataset containing 4,322 labelled question pairs in Chinese, which is, to the best of our knowledge, the first open Chinese dataset for question similarity classification. We propose a novel framework for measuring the semantic similarity between sentences based on the architecture of a recurrent neural network (RNN) encoderdecoder, which does not require lexical or syntactic resources. We solve the problem of lacking labelled data by first training the RNN using a larger dataset of question pairs that are automatically labelled with heuristic scores, and then fine-tuning it with our smaller, manually labelled dataset. The two-step training scheme improves the accuracy of classification compared to single-step training, and also outperforms other traditional models. The proposed model is capable of both classification and candidate ranking.

Search engine is a program that performs a search in the documents for finding out the response to the user’s query in form of keywords. It then provides a list of web pages comprising of those keywords. Search engines cannot differentiate between the variable documents and spams. Some search engine crawler retrieves only document title not the entire text in the document. The major objective of Question Answering system is to develop techniques that not only retrieve documents, but also provide exact answers to natural language questions. Many Question Answering systems developed are able to carry out the processing needed for attaining higher accuracy levels. However, there is no major progress on techniques for quickly finding exact answers. Existing Question Answering system is unable to handle variety of questions and reasoning-based question. In case of absence of data sources, QA system fails to answer the query. This paper investigates a novel technique for indexing the semantic Web for efficient Question Answering system. Proposed techniques include manual constructed question classifier based on , retrieval of documents specifically for Question Answering, semantic type answer extraction, answer extraction via manually constructed index for every category of Question.

We regard answer extraction of Question Answering (QA) system as a classification problem, classifying answer candidate sentences into positive or negative. To confirm the feasibility of this new approach, we first extract features concerning question sentences and answer words from question answer pairs (QA pair), then we conduct experiments based on these features, using Maximum Entropy Model (MEM) as a Machine Learning (ML) technique. The first experiment conducted on the class-TIME_YEAR achieves 81.24% in precision and 78.48% in recall. The second experiment expanded to two other classes-OBJ_SUBSTANCE and LOC_CONTINENT also shows good performance.KeywordsQuery WordCombine FeatureQuestion AnsweringInterrogative WordMaximum Entropy ModelThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Search engines can return ranked documents as a result for any query from which the user struggle to navigate and search the correct answer. This process wastes user's navigation time and due to this the need for automated question answering systems becomes more urgent. We need such a system which is capable of replying the exact and concise answer to the question posed in natural language. The best way to address this problem is use of Question answering systems (QAS). The basic aim of QAS is to provide short and correct answer to the user saving his/her navigation time. The concept of Natural Language Processing plays an important role in developing any QAS. This paper provides an implementation approaches for various categories of QAS such as Closed Domain based QAS, Open Domain based QAS, WEBBASED QAS, Information Retrieval or Information Extraction (IR/IE) based QAS, and Rule based QAS which will be helpful for new directions of research in this area.

Open-domain question answering is an emerging field of natural language processing. Application areas of question answering (QA) systems are increasing with the availability of new datasets, more powerful machines and easy access to voice-enabled devices that interact with humans in their daily lives. Users can get into interactions with QA systems without the need to be limited by any domain. However, there is still need for QA systems that can answer questions in a specific field such as international criminal law, violence of human rights or bill of rights. The recent approaches on open-domain question answering made it possible to extend open-domain QA systems towards specific fields. This paper presents Quill-QA, the new feature of the Quill Platform. Quill Platform is a web platform providing set of tools that digitize and visualize negotiated texts, any text in the form of formal negotiations. Quill-QA is based on Facebook's DrQA system and as a result it adopts a flexible Ranker-Reader architecture. Quill-QA answers questions addressed on the samples of negotiated texts such as U.S. Constitutional Convention of 1787, Utah State Constitutional Convention of 1895 and European Union Act 2017. Without limiting the scope of the input questions to a finite set of templates, Quill-QA is the first QA system that focuses on negotiated texts.