• Home
  • Search
  • Integrating deep learning for visual question answering in Agricultural Disease Diagnostics: Case Study of Wheat Rust
  • Cite Icon13
  • https://doi.org/10.1038/s41598-024-79793-2Copy DOI Icon

Integrating deep learning for visual question answering in Agricultural Disease Diagnostics: Case Study of Wheat Rust

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

This paper presents a novel approach to agricultural disease diagnostics through the integration of Deep Learning (DL) techniques with Visual Question Answering (VQA) systems, specifically targeting the detection of wheat rust. Wheat rust is a pervasive and destructive disease that significantly impacts wheat production worldwide. Traditional diagnostic methods often require expert knowledge and time-consuming processes, making rapid and accurate detection challenging. We drafted a new, WheatRustDL2024 dataset (7998 images of healthy and infected leaves) specifically designed for VQA in the context of wheat rust detection and utilized it to retrieve the initial weights on the federated learning server. This dataset comprises high-resolution images of wheat plants, annotated with detailed questions and answers pertaining to the presence, type, and severity of rust infections. Our dataset also contains images collected from various sources and successfully highlights a wide range of conditions (different lighting, obstructions in the image, etc.) in which a wheat image may be taken, therefore making a generalized universally applicable model. The trained model was federated using Flower. Following extensive analysis, the chosen central model was ResNet. Our fine-tuned ResNet achieved an accuracy of 97.69% on the existing data. We also implemented the BLIP (Bootstrapping Language-Image Pre-training) methods that enable the model to understand complex visual and textual inputs, thereby improving the accuracy and relevance of the generated answers. The dual attention mechanism, combined with BLIP techniques, allows the model to simultaneously focus on relevant image regions and pertinent parts of the questions. We also created a custom dataset (WheatRustVQA) with our augmented dataset containing 1800 augmented images and their associated question-answer pairs. The model fetches an answer with an average BLEU score of 0.6235 on our testing partition of the dataset. This federated model is lightweight and can be seamlessly integrated into mobile phones, drones, etc. without any hardware requirement. Our results indicate that integrating deep learning with VQA for agricultural disease diagnostics not only accelerates the detection process but also reduces dependency on human experts, making it a valuable tool for farmers and agricultural professionals. This approach holds promise for broader applications in plant pathology and precision agriculture and can consequently address food security issues.

Loading PDF

Similar Papers
  • Conference Article

Bridging the Gap: Leveraging Textual and Visual Contexts for PreciseMedical Visual Question Answering

  • Aug 08, 2024
  • Pratiksh Kumar +3
  • Research Article
  • Citations18

From known to the unknown: Transferring knowledge to answer questions about novel visual and semantic concepts

  • Aug 04, 2020
  • Image and Vision Computing
  • Moshiur R Farazi +2
  • Research Article
  • Citations2

Multi-Granularity Feature Interaction and Multi-Region Selection Based Triplet Visual Question Answering

  • Jun 01, 2025
  • IEEE Transactions on Big Data
  • Heng Liu +6
  • Conference Article
  • Citations11

Grad-Cam Aware Supervised Attention for Visual Question Answering for Post-Disaster Damage Assessment

  • Oct 16, 2022
  • Argho Sarkar +1
  • Research Article
  • Citations23

Dual Attention and Question Categorization-Based Visual Question Answering

  • Feb 01, 2023
  • IEEE Transactions on Artificial Intelligence
  • Aakansha Mishra +2
  • Research Article
  • Citations74

Radial Graph Convolutional Network for Visual Question Generation.

  • Apr 01, 2021
  • IEEE Transactions on Neural Networks and Learning Systems
  • Xing Xu +4
  • PDF
  • Research Article
  • Citations20

Visual question answering model for fruit tree disease decision-making based on multimodal deep learning

  • Jan 05, 2023
  • Frontiers in Plant Science
  • Yubin Lan +5
  • Conference Article
  • Citations8

Tourism Object Detection Around Monumen Nasional (Monas) Using YOLO and RetinaNet

  • Oct 17, 2020
  • Median Hardiv Nugraha +1
  • Research Article
  • Citations10

Robust visual question answering via semantic cross modal augmentation

  • Oct 16, 2023
  • Computer Vision and Image Understanding
  • Akib Mashrur +3
  • PDF
  • Research Article
  • Citations2

Enhancing visual question answering with common sense knowledge: a data-driven neurosymbolic graph routing approach

  • Jun 14, 2025
  • International Journal of Data Science and Analytics
  • Muhammad Junaid Khan +3
  • Book Chapter
  • Citations86

VQA-LOL: Visual Question Answering Under the Lens of Logic

  • Jan 01, 2020
  • Tejas Gokhale +3
  • Research Article
  • Citations7

Graph-enhanced visual representations and question-guided dual attention for visual question answering

  • Nov 07, 2024
  • Neurocomputing
  • Abdulganiyu Abdu Yusuf +5
  • Research Article

An Adaptive Multimodal Fusion Network Based on Multilinear Gradients for Visual Question Answering

  • Dec 24, 2024
  • Electronics
  • Chengfang Zhao +3
  • Research Article
  • Citations5

From Application-Driven Growth to Paradigm Shift: Scientific Evolution and Core Bottleneck Analysis in the Field of UAV Remote Sensing

  • Jul 25, 2025
  • Applied Sciences
  • Denghong Huang +6
  • Research Article
  • Citations7

A comprehensive image dataset for the identification of eggplant leaf diseases and computer vision applications.

  • Apr 01, 2025
  • Data in brief
  • Shakib Howlader +5
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.