• Home
  • Search
  • Learned Text Representation for Amharic Information Retrieval and Natural Language Processing
  • Cite Icon16
  • https://doi.org/10.3390/info14030195Copy DOI Icon

Learned Text Representation for Amharic Information Retrieval and Natural Language Processing

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

Over the past few years, word embeddings and bidirectional encoder representations from transformers (BERT) models have brought better solutions to learning text representations for natural language processing (NLP) and other tasks. Many NLP applications rely on pre-trained text representations, leading to the development of a number of neural network language models for various languages. However, this is not the case for Amharic, which is known to be a morphologically complex and under-resourced language. Usable pre-trained models for automatic Amharic text processing are not available. This paper presents an investigation on the essence of learned text representation for information retrieval and NLP tasks using word embeddings and BERT language models. We explored the most commonly used methods for word embeddings, including word2vec, GloVe, and fastText, as well as the BERT model. We investigated the performance of query expansion using word embeddings. We also analyzed the use of a pre-trained Amharic BERT model for masked language modeling, next sentence prediction, and text classification tasks. Amharic ad hoc information retrieval test collections that contain word-based, stem-based, and root-based text representations were used for evaluation. We conducted a detailed empirical analysis on the usability of word embeddings and BERT models on word-based, stem-based, and root-based corpora. Experimental results show that word-based query expansion and language modeling perform better than stem-based and root-based text representations, and fastText outperforms other word embeddings on word-based corpus.

Loading PDF

Similar Papers
  • Abstract
  • Citations5

Bert model fine-tuning for text classification in knee OA radiology reports

  • Apr 01, 2020
  • Osteoarthritis and Cartilage
  • L Chen +5
  • PDF
  • Research Article
  • Citations34

MenuNER: Domain-Adapted BERT Based NER Approach for a Domain with Limited Dataset and Its Application to Food Menu Domain

  • Jun 28, 2021
  • Applied Sciences
  • Muzamil Hussain Syed +1
  • Research Article

Cybershield Monitor Using Deep Learning and BERT Model

  • Feb 24, 2025
  • INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT
  • A Nandhini
  • Research Article

T-­BERT:臺灣語言模型–以臺灣在地語言預訓練BERT模型

  • Jan 01, 2020
  • 鍾明諺
  • Research Article
  • Citations6

Prediction of Personal Experience Tweets of Medication Use via Contextual Word Representations.

  • Jul 01, 2019
  • Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference
  • Keyuan Jiang +3
  • Research Article
  • Citations20

Oversampling effect in pretraining for bidirectional encoder representations from transformers (BERT) to localize medical BERT and enhance biomedical BERT

  • May 05, 2024
  • Artificial Intelligence In Medicine
  • Shoya Wada +6
  • PDF
  • Research Article
  • Citations18

Bidirectional encoders to state-of-the-art: a review of BERT and its transformative impact on natural language processing

  • Mar 02, 2024
  • Информатика. Экономика. Управление - Informatics. Economics. Management
  • Rajesh Gupta
  • Conference Article
  • Citations3

Engineering Document Summarization Using Sentence Representations Generated by Bidirectional Language Model

  • Aug 17, 2021
  • Yunjian Qiu +1
  • Research Article
  • Citations8

Cost-Efficient Domain-Adaptive Pretraining of Language Models for Optoelectronics Applications.

  • Feb 11, 2025
  • Journal of chemical information and modeling
  • Dingyun Huang +1
  • Research Article
  • Citations15

Bioformer: an efficient transformer language model for biomedical text mining

  • Feb 03, 2023
  • ArXiv
  • Li Fang +4
  • Research Article
  • Citations26

BERT models for Brazilian Portuguese: Pretraining, evaluation and tokenization analysis

  • Oct 07, 2023
  • Applied Soft Computing
  • F.C Souza +2
  • Video Transcripts

BERTimbau: pretrained BERT models for Brazilian Portuguese

  • Oct 15, 2020
  • Underline Science Inc.
  • Rodrigo Nogueira +2
  • Book Chapter
  • Citations597

BERTimbau: Pretrained BERT Models for Brazilian Portuguese

  • Jan 01, 2020
  • Fábio Souza +2
  • Research Article
  • Citations1

Hybrid Feature Combination of TF-IDF and BERT for Enhanced Information Retrieval Accuracy

  • Jun 27, 2025
  • JISA(Jurnal Informatika dan Sains)
  • Pajri Aprilio +3
  • Research Article

Spam Detection Based on BERT Model: A Descriptive Study

  • Jan 01, 2023
  • Theory and practice of science and technology
  • Hailin Xu
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.