• Home
  • Search
  • METAHEURISTIC-DRIVEN HYPERPARAMETER OPTIMIZATION FOR BERT IN SENTIMENT ANALYSIS
  • https://doi.org/10.70102/afts.2025.1833.176Copy DOI Icon

METAHEURISTIC-DRIVEN HYPERPARAMETER OPTIMIZATION FOR BERT IN SENTIMENT ANALYSIS

Show More
  • Abstract
  • Literature Map
  • Similar Papers
Abstract

Sentiment analysis has come out as an important activity in natural language processing (NLP) applications whose data analysis is in high demand at present in the modern world. The BERT (Bidirectional Encoder Representations from Transformers) algorithm has proved to be extremely efficient when it comes to sentiment analysis tasks, and its potential is far exceeding that of conventional algorithms, unlocking their potential however would require fine tuning of their hyperparameters. It is quite a feat to optimise the BERT’s various hyperparameters due to the complicated interaction between them (e.g. the learning rate, batch size, dropout rate, attention heads). In this paper, the Salp Swarm Algorithm (SSA) is used as a bio-inspired metaheuristic optimization technique to optimize the fine-tuning process. Through SSA’s exceptionally efficient search capabilities in modelling multidimensional search space, BERT hyperparameters are optimized systematically to the sentiment classification tasks. A benchmark dataset for sentiment analysis (Sentiment140) is used to evaluate the proposed model. The novelty of the presented model is the fact that it dynamically adjusts its search behaviour in response to performance signals, thus it identifies better-performing parameter sets than conventional methods, leading to successful exploitation of the BERT algorithm that has produced high performing configurations. Extensive evaluations against 3 state-of-the-art search algorithms, namely manual tuning, grid search, and random search are conducted on the Sentiment140 benchmark dataset, demonstrating the superiority of the proposed SSA BERT optimization technique over state-of-the-art methods. The SSA-BERT model achieved a maximum accuracy of 96.4 percent, which is far better than manual tuning, grid search, and random search (65.0 percent, 69.5 percent and 72.0 percent respectively). It also performed better than other existing BERT models used in related literature, which showed accuracy levels between 46.4 and 75.7 percent in accordance with different benchmarks Sentiment analysis has come out as an important activity in natural language processing (NLP) applications whose data analysis is in high demand at present in the modern world. The BERT (Bidirectional Encoder Representations from Transformers) algorithm has proved to be extremely efficient when it comes to sentiment analysis tasks, and its potential is far exceeding that of conventional algorithms, unlocking their potential however would require fine tuning of their hyperparameters. It is quite a feat to optimise the BERT’s various hyperparameters due to the complicated interaction between them (e.g. the learning rate, batch size, dropout rate, attention heads). In this paper, the Salp Swarm Algorithm (SSA) is used as a bio-inspired metaheuristic optimization technique to optimize the fine-tuning process. Through SSA’s exceptionally efficient search capabilities in modelling multidimensional search space, BERT hyperparameters are optimized systematically to the sentiment classification tasks. A benchmark dataset for sentiment analysis (Sentiment140) is used to evaluate the proposed model. The novelty of the presented model is the fact that it dynamically adjusts its search behaviour in response to performance signals, thus it identifies better-performing parameter sets than conventional methods, leading to successful exploitation of the BERT algorithm that has produced high performing configurations. Extensive evaluations against 3 state-of-the-art search algorithms, namely manual tuning, grid search, and random search are conducted on the Sentiment140 benchmark dataset, demonstrating the superiority of the proposed SSA BERT optimization technique over state-of-the-art methods. The SSA-BERT model achieved a maximum accuracy of 96.4 percent, which is far better than manual tuning, grid search, and random search (65.0 percent, 69.5 percent and 72.0 percent respectively). It also performed better than other existing BERT models used in related literature, which showed accuracy levels between 46.4 and 75.7 percent in accordance with different benchmarks.

Similar Papers
  • PDF
  • Research Article
  • Citations18

Bidirectional encoders to state-of-the-art: a review of BERT and its transformative impact on natural language processing

  • Mar 02, 2024
  • Информатика. Экономика. Управление - Informatics. Economics. Management
  • Rajesh Gupta
  • Research Article

Financial Report Sentiment Analysis Using Loughran-mcdonald Dictionary and BERT

  • Jun 24, 2024
  • Financial Engineering
  • Sheetal R +1
  • Research Article

Sentiment Analysis on Twitter Social Media Regarding Covid-19 Vaccination with Naive Bayes Classifier (NBC) and Bidirectional Encoder Representations from Transformers (BERT)

  • Sep 30, 2024
  • Recursive Journal of Informatics
  • Angga Riski Dwi Saputra +1
  • Research Article
  • Citations1

Sentiment Analysis on Twitter Social Media Regarding Covid-19 Vaccination with Naive Bayes Classifier (NBC) and Bidirectional Encoder Representations from Transformers (BERT)

  • Sep 30, 2024
  • Recursive Journal of Informatics
  • Angga Riski Dwi Saputra +1
  • Book Chapter
  • Citations8

Natural Language Processing with “More Than Words – BERT”

  • Jan 01, 2021
  • Saranlita Chotirat +1
  • Conference Article
  • Citations10

Classification of Fire Related Tweets on Twitter Using Bidirectional Encoder Representations from Transformers (BERT)

  • Nov 28, 2021
  • Jairus Mingua +2
  • PDF
  • Research Article
  • Citations28

Automatic text classification of actionable radiology reports of tinnitus patients using bidirectional encoder representations from transformer (BERT) and in-domain pre-training (IDPT)

  • Jul 30, 2022
  • BMC Medical Informatics and Decision Making
  • Jia Li +10
  • Research Article
  • Citations13

Bidirectional Encoder Representations from Transformers (BERT) Language Model for Sentiment Analysis task: Review

  • Apr 19, 2021
  • Turkish Journal of Computer and Mathematics Education (TURCOMAT)
  • D Deepa
  • Abstract
  • Citations5

Bert model fine-tuning for text classification in knee OA radiology reports

  • Apr 01, 2020
  • Osteoarthritis and Cartilage
  • L Chen +5
  • Research Article
  • Citations20

Oversampling effect in pretraining for bidirectional encoder representations from transformers (BERT) to localize medical BERT and enhance biomedical BERT

  • May 05, 2024
  • Artificial Intelligence In Medicine
  • Shoya Wada +6
  • Research Article
  • Citations7

Sentiment Analysis in Healthcare: A Comparison of VADER, BERT, and Flair NLP Models on Patient Reviews of Pain Management Physicians

  • Jul 28, 2025
  • Cureus
  • Ishan Aggarwal +5
  • Research Article
  • Citations10

Fine-Tuning BERT for Automated News Classification

  • Jun 04, 2025
  • Engineering, Technology & Applied Science Research
  • Mohammed I Salih +4
  • Research Article

Performance Analysis of Transformer Based Models for Automatic Short Answer Grading

  • May 30, 2025
  • Journal of Information Systems Engineering and Management
  • Rupal Chaudhari
  • Research Article

Performance Analysis of Transformer Based Models for Automatic Short Answer Grading

  • May 30, 2025
  • Journal of Information Systems Engineering and Management
  • Rupal Chaudhari
  • Conference Article

Predictive Sentiment Analysis of Skincare Product Reviews: A Natural Language Processing Approach with BERT-Enhanced Sequential Model (BESM)

  • Dec 01, 2025
  • Sky Nurimba +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.