- Conference Article
- 10.1109/icdiss68238.2025.11320662
A RAG-Based Evaluation of Large Language Models for Financial Insights
- Nov 14, 2025
- Chaitali Bhowmik + 4 more +4
Large Language Models (LLMs) have transformed financial research by providing sophisticated analysis of complex textual data. This research examines the performance differences between GPT-4 and Gemini Pro and Claude and DeepSeek through a Retrieval-Augmented Generation (RAG) framework designed to interpret financial news. We built a Streamlit-based application through the integration of LangChain with HuggingFace embeddings and FAISS vector stores and Gemini APIs to create a system that mirrors real-world financial analysis requirements. Multiple metrics including contextual accuracy, relevance, factual consistency, latency and cost-efficiency serve as evaluation criteria for the models. The outcomes of other models are extrapolated from Gemini's complete implementation through extensive documentation and public benchmarks along with qualitative evaluation methods. The evaluation demonstrates that Gemini delivers better contextual relevancy along with faster inference speeds but GPT-4 demonstrates stronger coherence combined with more precise factual accuracy. This work presents actionable guidelines to choose appropriate LLMs for financial research alongside an analysis of how RAG systems boost model effectiveness.
Read more