Coreference Resolution for Russian: Taking Stock and Moving Forward

While it has been claimed that anaphora or coreference resolution plays an important role in opinion mining, it is not clear to what extent coreference resolution actually boosts performance, if at all. In this paper, we investigate the potential added value of coreference resolution for the aspect-based sentiment analysis of restaurant reviews in two languages, English and Dutch. We focus on the task of aspect category classification and investigate whether including coreference information prior to classification to resolve implicit aspect mentions is beneficial. Because coreference resolution is not a solved task in NLP, we rely on both automatically-derived and gold-standard coreference relations, allowing us to investigate the true upper bound. By training a classifier on a combination of lexical and semantic features, we show that resolving the coreferential relations prior to classification is beneficial in a joint optimization setup. However, this is only the case when relying on gold-standard relations and the result is more outspoken for English than for Dutch. When validating the optimal models, however, we found that only the Dutch pipeline is able to achieve a satisfying performance on a held-out test set and does so regardless of whether coreference information was included.

Opinion Mining and Sentiment Analysis in Natural Language Processing (NLP) are challenging, as they require deep understanding. Understanding involves methods that could differentiate between the facts of explicit and implicit, regular and irregular, syntactical and semantic language rules. Researches oriented towards Natural Language Processing and Sentiment Analysis have many unresolved problems like co-reference resolution, negation handling, anaphora resolution, named-entity recognition, and word-sense disambiguation. This paper is proposed to develop an Optimized Partial Ancestral Graph (O-PAG) which is capable of mining patterns in human interactions and compare it with an existing tree based pattern mining approach. The experimental results are exposed to number of frequent interactions made and execution time. Results indicate that the overall performance can reach considerable improvements on using O-PAG approach.

In this paper we present our first attempt on anaphora resolution for a resource poor language, namely Bengali. We address the issue of adapting a state-of-the-art system, BART, which was originally developed for English. Overall performance of co-reference resolution greatly depends on the high accurate mention detectors. We develop a number of models based on the heuristics used as well as on the particular machine learning employed. Thereafter we perform a series of experiments for adapting BART for Bengali. Our evaluation shows, a language-dependant system (designed primarily for English) can achieve a good performance level when re-trained and tested on a new language with proper subsets of features. The system produces the recall, precision and F-measure values of 56.00%, 46.50% and 50.80%, respectively. The contribution of this work is two-fold, viz. (i). attempt to build a machine learning based anaphora resolution system for a resource-poor Indian language; and (ii). domain adaptation of a state-of-the-art English co-reference resolution system for Bengali, which has completely different orthography and characteristics.

Under statistical learning framework, the paper focuses on how to use traditional linguistic findings on anaphora resolution as a guide for mining and organizing contextual features for Chinese co-reference resolution. The main achievements are as follows. (1) In order to simulate "syntactic and semantic parallelism factor", we extract "bags of word form and POS" feature and "bag of semes" feature from the contexts of the entity mentions and incorporate them into the baseline feature set. (2) Because it is too coarse to use the feature of bags of word form, POS tag and seme to determine the syntactic and semantic parallelism between two entity mentions, we propose a method for contextual feature reconstruction based on semantic similarity computation, in order that the reconstructed contextual features could better approximate the anaphora resolution factor of "Syntactic and Semantic Parallelism Preferences". (3) We use an entity-mention-based contextual feature representation instead of isolated word-based contextual feature representation, and expand the size of the contextual windows in addition, in order to approximately simulate "the selectional restriction factor" for anaphora resolution. The experiments show that the multi-level contextual features are useful for co-reference resolution, and the statistical system incorporated with these features performs well on the standard ACE datasets.

The prediction of heart failure survivors is a challenging task and helps medical professionals to make the right decisions about patients. Expertise and experience of medical professionals are required to care for heart failure patients. Machine Learning models can help with understanding symptoms of cardiac disease. However, manual feature engineering is challenging and requires expertise to select the appropriate technique. This study proposes a smart healthcare framework using the Internet-of-Things (IoT) and cloud technologies that improve heart failure patients’ survival prediction without considering manual feature engineering. The smart IoT-based framework monitors patients on the basis of real-time data and provides timely, effective, and quality healthcare services to heart failure patients. The proposed model also investigates deep learning models in classifying heart failure patients as alive or deceased. The framework employs IoT-based sensors to obtain signals and send them to the cloud web server for processing. These signals are further processed by deep learning models to determine the state of patients. Patients’ health records and processing results are shared with a medical professional who will provide emergency help if required. The dataset used in this study contains 13 features and was attained from the UCI repository known as Heart Failure Clinical Records. The experimental results revealed that the CNN model is superior to other deep learning and machine learning models with a 0.9289 accuracy value.

We propose to use segment graph convolutional and recurrent neural networks (Seg-GCRNs), which use only word embedding and sentence syntactic dependencies, to classify relations from clinical notes without manual feature engineering. In this study, the relations between 2 medical concepts are classified by simultaneously learning representations of text segments in the context of sentence syntactic dependency: preceding, concept1, middle, concept2, and succeeding segments. Seg-GCRN was systematically evaluated on the i2b2/VA relation classification challenge datasets. Experiments show that Seg-GCRN attains state-of-the-art micro-averaged F-measure for all 3 relation categories: 0.692 for classifying medical treatment-problem relations, 0.827 for medical test-problem relations, and 0.741 for medical problem-medical problem relations. Comparison with the previous state-of-the-art segment convolutional neural network (Seg-CNN) suggests that adding syntactic dependency information helps refine medical word embedding and improves concept relation classification without manual feature engineering. Seg-GCRN can be trained efficiently for the i2b2/VA dataset on a GPU platform.

The aim of the present work was to investigate the processing of coreferential relations, focusing on their relationship with the working memory. In a reading process, it is essential that readers continuously perform mental operations that involve the working memory, such as storing, retrieving, and manipulating information. For this reason, it is understood that the processing of coreference resolution is closely related to the working memory. In this relationship between memory and anaphora resolution, the distance factor between the referent and anaphor can play a significant role as distance increases the processing costs when reading sentences. It is known that the working memory is limited; yet there is still no consensus among researchers regarding the nature of storable items. Moreover, the syntactic node, here taken as a representative of the structure of a phrase, was observed as possible storage and recoverable unit in the working memory. Thus, the objective of the study was to verify whether the syntactic distance factor, embeddings with 1 and 2 intervening syntactic nodes, is a storable unit in the working memory, investigating its influence in the processing of different anaphora resolution. The self-paced reading experimental technique was used in an experiment. Seventy participants, all university students, were tested individually and underwent a training session before the task. The results suggest that the syntactic distance is a relevant factor in the working memory with significant differences between conditions with syntactic distance concerning those with linear distance, indicating the formation of chunks. There was an increase in reading times depending on the number of intervening embeddings, and different processing costs for each type of anaphora resolution investigated . The aim of the present work was to investigate the processing of coreferential relations, focusing on their relationship with the working memory. In a reading process, it is essential that readers continuously perform mental operations that involve the working memory, such as storing, retrieving, and manipulating information. For this reason, it is understood that the processing of coreference resolution is closely related to the working memory. In this relationship between memory and anaphora resolution, the distance factor between the referent and anaphor can play a significant role as distance increases the processing costs when reading sentences. It is known that the working memory is limited; yet there is still no consensus among researchers regarding the nature of storable items. Moreover, the syntactic node, here taken as a representative of the structure of a phrase, was observed as possible storage and recoverable unit in the working memory. Thus, the objective of the study was to verify whether the syntactic distance factor, embeddings with 1 and 2 intervening syntactic nodes, is a storable unit in the working memory, investigating its influence in the processing of different anaphora resolution. The self-paced reading experimental technique was used in an experiment. Seventy participants, all university students, were tested individually and underwent a training session before the task. The results suggest that the syntactic distance is a relevant factor in the working memory with significant differences between conditions with syntactic distance concerning those with linear distance, indicating the formation of chunks. There was an increase in reading times depending on the number of intervening embeddings, and different processing costs for each type of anaphora resolution investigated .

Most recent studies on coreference resolution advocate accurate yet relatively complex models, relying on, for example, entitymention or graph-based representations. As it has been convincingly demonstrated at the recent CoNLL 2012 shared task, such algorithms considerably outperform popular basic approaches, in particular mention-pair models. This study advocates a novel approach that keeps the simplicity of a mention-pair framework, while showing state-of-the-art results. Apart from being very efficient and straightforward to implement, our model facilitates experimental work on the pairwise classifier, in particular on feature engineering. The proposed model achieves the performance level of up to 61.82% (MELA F, v4 scorer) on the CoNLL test data, on par with complex state-of-the-art systems.

In this paper, we consider the coreference resolution problem in the context of information extraction as envisioned by the DARPA Automatic Content Extraction (ACE) program Given a set of entity mentions referring to real world entities and a similarity matrix that characterizes how similar those mentions are, we seek a set of entities that are uniquely co-referred to by those entity mentions The quality of the clustering of entity mentions into unique entities significantly depends on the quality of (1) the similarity matrix and (2) the clustering algorithm We explore the coreference resolution problem along those two dimensions and clearly show the tradeoff among several ways of learning similarity matrix and using it while performing clustering.

Advancements in modern technology have significantly increased the availability of educational data, presenting researchers with new challenges in extracting meaningful insights. Educational Data Mining offers analytical methods to support the prediction of student outcomes, development of intelligent tutoring systems, and curriculum optimization. Prior studies have highlighted the potential of semi-supervised approaches that incorporate feature selection to identify factors influencing academic success, particularly for improving model interpretability and predictive performance. Many feature selection methods tend to exclude variables that may not be individually powerful predictors but can collectively provide significant information, thereby constraining a model’s capabilities in learning environments. In contrast, Deep Learning (DL) models paired with Automated Machine Learning techniques can decrease the reliance on manual feature engineering, thereby enabling automatic fine-tuning of numerous model configurations. In this study, we propose a reproducible methodology that integrates DL with AutoML to evaluate student performance. We compared the proposed DL methodology to a semi-supervised approach originally introduced by Yu et al. under the same evaluation criteria. Our results indicate that DL-based models can provide a flexible, data-driven approach for examining student outcomes, in addition to preserving the importance of feature selection for interpretability. This proposal is available for replication and additional research.

In today's cyber environment, modern botnets and malware are increasingly employing domain generation mechanisms to circumvent conventional detection solutions reliant on blacklisting or statistical methods for malicious domains. These outdated methods prove inadequate against algorithmically generated domain names, presenting significant challenges for cyber security. Domain Generation Algorithms (DGAs) have become essential tools for many malware families, allowing them to create numerous DGA domain names to establish communication with C&C servers. Consequently, detecting such malware has become a formidable task in cyber security. Traditional approaches to domain name detection rely heavily on manual feature engineering and statistical analysis, with classifiers designed to differentiate between legitimate and DGA domain names. In this study, we propose a novel approach to classify and detect algorithmically generated domain names. The deep learning architectures, including LSTM, RNN and GRU are trained and evaluated for their effectiveness in distinguishing between legitimate and malicious domain names. The performance of each model is evaluated using standard metrics such as precision, recall, and F1-score. The findings of this research have significant implications for cyber security defense strategies. Our experimental findings illustrate that the proposed model outperforms current state-of-the-art methods in both DGA domain name classification and detection. Our proposed model achieved 99% accuracy for DGA classification. By integrating additional feature extraction and knowledge-based methods our proposed model surpasses existing models. The experimental outcomes suggest that our proposed model gated recurrent unit can achieve 99% accuracy, a 94% recall rate, and a 98% F1-score for the detection and classification of DGA-generated domain names.

Individual tree segmentation (ITS) from terrestrial laser scanning (TLS) point clouds is foundational for deriving detailed forest structural parameters, crucial for precision forestry, biomass calculation, and carbon accounting. Conventional ITS algorithms often struggle in complex forest stands due to reliance on heuristic rules and manual feature engineering. Deep learning methodologies proffer more efficacious and automated solutions, but their segmentation accuracy is restricted by imprecise center offset predictions, particularly in intricate forest environments. To address this issue, we proposed a deep learning method, SPA-Net, for achieving tree instance segmentation of forest point clouds. Unlike methods heavily reliant on potentially error-prone global offset vector predictions, SPA-Net employs a novel sampling-shifting-grouping paradigm within its sparse geometric proposal (SGP) module to directly generate initial proposal candidates from raw point data, aiming to reduce dependence on the offset branch. Subsequently, an affinity aggregation (AA) module robustly refines these proposals by assessing inter-proposal relationships and merging fragmented segments, effectively mitigating oversegmentation of large or complex trees; integrating with SGP eliminates the postprocessing step of scoring/NMS. SPA-Net was rigorously validated on two different forest datasets. On both BaiMa and Hong-Tes Lake datasets, the approach demonstrated superior performance compared to several contemporary segmentation approaches evaluated under the same conditions. It achieved 95.8% precision, 96.3% recall, and 92.9% coverage on BaiMa dataset, and achieved 92.6% precision, 94.8% recall, and 88.8% coverage on the Hong-Tes Lake dataset. This study provides a robust tool for individual tree analysis, advancing the accuracy of individual tree segmentation in challenging forest environments.

Malicious URLs have become an important attack vector used by attackers to perpetrate cybercrimes, how to effectively detect malicious URLs is an important and urgent problem to be solved. Due to current feature based malicious URLs detection models need manual feature engineering, and deep learning based models have their limit on processing long sequences, which reduces the detection performance. We proposed an attentional based BiLSTM model AB-BiLSTM for the Malicious URLs detection in this paper. Firstly, the URLs were preprocessed and converted into word vectors by using pre-trained Word2Vec, then BiLSTM combined with an attention mechanism was trained to extract URL sequences features and classify them. The model was tested on collected dataset, the experimental results show that our proposed model can achieve the accuracy of 98.06%, the precision rate of 96.05, the recall rate of 95.79% and the F1 Score of 95.92%, which achieved better performance than other comparison traditional machine learning based and deep learning based models.

Smart health is quickly boosted by technological advancements: smart sensors, body sensor network, internet of medical things and big data. Vast amounts of smart health big data from ubiquitous sensors pose unprecedented challenges to the security and privacy protection, which is extremely critical in healthcare applications. The vital signs, user daily behaviors, medicine recommendations, and so many other health data are vulnerable to different attacks, due to the fact that wearable/mobile monitors have very strict performance/power constraints, which limit the complexity of security protocols. In this paper, we study how to leverage a natural vital signal (Electrocardiogram - ECG) for user identification purpose, without introducing new hardware sensing devices. ECG is not only a gold standard cardiac signal, but also unique to each individual. We investigate a phase-domain deep patient-ECG image learning framework, to tackle key challenges in ECG biometric user identification: high diversities of ECG morphologies due to heart diseases, and time-consuming/ineffective heartbeat localization methods & manual feature engineering. The ultimate goal is to make the smart health security zero-effort: use `phase-domain transformation' to enable blind signal segmentation without localizing heartbeats; create a computer image processing-like task by `pixelating' phase-domain ECG trajectories to ECG images; and enable automatic (non-manual) `deep feature learning' using a deep convolutional neural network. Evaluated on two patient-ECG databases, this zero-effort framework achieves an accuracy as high as 97.2%, and greatly outperforms state-of-the-art studies in terms of the generalization ability and/or performance. This study is expected to enable highly challenging patient-ECG biometric user identification, by generalizable blind signal segmentation and deep feature learning strategies, in the era of smart health boosted by internet of medical things and big medical data.

In recent years, machine learning algorithms have been developing rapidly, becoming increasingly powerful tools in decoding physiological and neural signals. The aim of this dissertation is to develop computational tools, and especially machine learning techniques, to identify the most effective methods for feature extraction and classification of these signals. This is particularly challenging due to the highly non-linear, non-stationery, and artifact- and noise-prone nature of these signals. Among basic human-control tasks, reaching and grasping are ubiquitous in everyday life. I investigated different linear and non-linear dimensionality reduction techniques for feature extraction and classification of electromyography (EMG) during a reach-grasp-lift task. The results highlighted the advantages of completely automated feature-learning by Laplacian Eigenmaps over manual feature engineering, especially when combined with classification, to achieve high accuracy with few training samples. The ability to decode and reduce the complexity of EMG could enable new practical applications for EMG in basic science and in the clinic. It could also help design humanoid and other robots. Beyond EMG, a key objective of my dissertation was to decode brain activity during the decision-making processes that lead to voluntary action. This was based on electroencephalography (EEG) and holds the promise for improved brain-computer interfaces (BCIs), particularly related to motor imagery (MI). We developed an end-to-end convolutional neural network with attentional mechanism together with different data augmentation techniques on two benchmark MI datasets. I also collected a new dataset, recorded using high-density EEG, and containing both MI and motor execution (ME) tasks. This enabled us to directly compare the decodability of MI and ME, investigate optimal channel configurations, and much more. In particular, this facilitates the analysis and decoding of MI on the fly—online and in in real time. Another potential use of EEG is measuring brain activity of people who are floating in body-temperature water in a sensory-deprivation-tank float pod. We compared lying in the float pod versus lying in bed (a control condition). And we found differences between the two, especially in the gamma band. More research is required to understand what these findings mean for levels of stress in the float pod.