• Home
  • Search
  • CrossLM: A Data-Free Collaborative Fine-Tuning Framework for Large and Small Language Models
  • https://doi.org/10.1145/3711875.3729128Copy DOI Icon

CrossLM: A Data-Free Collaborative Fine-Tuning Framework for Large and Small Language Models

  • Jun 23, 2025
  • Yongheng Deng +5 more
Show More
  • Abstract
  • Literature Map
  • References
  • Similar Papers
Abstract

While large language models (LLMs) are endowed with broad knowledge, their task-specific performance is often suboptimal. Fine-tuning LLMs with task-specific data from diverse nodes is necessary, but this data is typically safeguarded and not shared publicly due to privacy concerns. A common solution involves downstream nodes downloading the LLM locally and fine-tuning it with their proprietary data. However, owners often regard pre-trained LLMs as valuable assets and are reluctant to share them. Additionally, the significant computational resources required by LLMs make local fine-tuning impractical for many nodes. To mitigate these problems, this paper proposes CrossLM, a data-free collaborative fine-tuning framework for large and small language models. CrossLM enables resource-constrained nodes to train smaller language models (SLMs) using their private task-specific data. These SLMs are subsequently leveraged to promote the task-specific natural language generation and understanding capabilities of the LLMs. Simultaneously, the SLMs of nodes also benefit from enhancement by the fine-tuned LLMs. In this way, CrossLM avoids sharing private data and proprietary LLMs, and also reduces the resource requirements of nodes. Through extensive experiments across a range of benchmark tasks and popular language models, we demonstrate that CrossLM significantly boosts the task-specific performance of both LLMs and SLMs while preserving the generalization capabilities of LLMs.

Similar Papers
  • Research Article
  • Citations6

Tax Intelligent Decision-Making Language Model

  • Jan 01, 2024
  • IEEE Access
  • Yan Zhong +2
  • Research Article
  • Citations11

How Can IJDS Authors, Reviewers, and Editors Use (and Misuse) Generative AI?

  • Apr 01, 2023
  • INFORMS Journal on Data Science
  • Galit Shmueli +7
  • Research Article
  • Citations7

Evaluating the performance of large language & visual-language models in cervical cytology screening

  • May 23, 2025
  • npj Precision Oncology
  • Qi Hong +15
  • Supplementary Content
  • Citations1

Large language and vision-language models for robot: safety challenges, mitigation strategies and future directions

  • Jul 29, 2025
  • Industrial Robot: the international journal of robotics research and application
  • Xiangyu Hu +1
  • Research Article
  • Citations2

Evaluating the Accuracy and Diagnostic Reasoning of Multimodal Large Language Models in Interpreting Neuroradiology Cases From RadioGraphics.

  • Jan 01, 2026
  • Korean journal of radiology
  • Pae Sun Suh +6
  • Conference Article
  • Citations9

Looking Right is Sometimes Right: Investigating the Capabilities of Decoder-only LLMs for Sequence Labeling

  • Jan 01, 2024
  • David Dukić +1
  • Research Article
  • Citations1

Extraction of geoprocessing modeling knowledge from crowdsourced Google Earth Engine scripts by coordinating large and small language models

  • Nov 01, 2025
  • International Journal of Geographical Information Science
  • Anqi Zhao +7
  • Research Article
  • Citations1

Unlocking the Potential of Large Language Models in Education: Factors Influencing Adoption by Instructional Designers and Academics

  • Jan 01, 2026
  • Journal of Information Technology Education: Research
  • Katherine L Fourie +2
  • Conference Article
  • Citations135

Jigsaw

  • May 21, 2022
  • Naman Jain +6
  • Research Article
  • Citations13

A Large and Diverse Arabic Corpus for Language Modeling

  • Jan 01, 2023
  • Procedia Computer Science
  • Abbas Raza Ali +3
  • Research Article

Вдосконалена класифікація великих мовних моделей

  • Jan 01, 2025
  • Scientific papers of Donetsk National Technical University. Series: Informatics, Cybernetics and Computer Science
  • С.М Бабчук
  • Discussion
  • Citations2

Large Language Models: A Historical and Sociocultural Perspective.

  • Mar 01, 2024
  • Cognitive Science
  • Eugene Yu Ji
  • Conference Article
  • Citations6

Exploring Spatial Schema Intuitions in Large Language and Vision Models

  • Jan 01, 2024
  • Philipp Wicke +1
  • Research Article
  • Citations6

High Throughput Phenotyping of Physician Notes with Large Language and Hybrid NLP Models.

  • Jul 15, 2024
  • Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference
  • Syed I Munzir +2
  • Research Article

An Iterative Comprehensive Evaluation of Large Language and Vision Models in Medical AI: Benchmarks, Adaptability, and Deployment Challenges

  • Dec 22, 2025
  • International Journal on Advanced Computer Engineering and Communication Technology
  • Wani H Bisen +1
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.