• Home
  • Search
  • Detecting \u201clarge language models fingerprint\u201d for Japanese texts generated by six LLMs
  • https://doi.org/10.3389/frai.2026.1771115Copy DOI Icon

Detecting \u201clarge language models fingerprint\u201d for Japanese texts generated by six LLMs

Show More
  • Abstract
  • Literature Map
  • Similar Papers
Abstract

BackgroundLarge language models (LLMs) have developed rapidly since the release of ChatGPT by OpenAI on November 30, 2022. This vogue comes to mind us “Can we distinguish LLMs-generated texts each other?” “What stylometric features are effective for differentiating LLMs such as fingerprints?” The purpose of this study was to distinguish the 300 Japanese texts (e.g., public comments) generated by six LLMs [ChatGPT (GPT5), Claude 3.5, Gemini, Microsoft Copilot, Llama 3.1, and Perplexity].MethodsTo this end, we explored the effective stylometric features [e.g., function-word unigrams, part of speech (POS) bigrams, and phrase patterns] using the following analyses: (1) UMAP (Uniform Manifold Approximation and Projection) to visually explore distributional differences among LLMs, (2) Random Forest (RF) and XGBoost with leave-one-out cross-validation to differentiate LLMs, and (3) SHAP (SHapley Additive exPlanations) based on Random Forest to identify effective stylometric features.ResultsFirst, UMAP demonstrated the separation of texts among the five LLMs except for Llama 3.1, which displayed substantial overlap with the other five LLMs. Second, RF achieved the highest performance across all stylometric features, with macro F1 scores exceeding 0.95 and reaching 1.00 for several LLMs. The detection performance of XGBoost was lower than that of RF, with the macro F1 scores ranging from 0.88 to 0.94. Finally, SHAP revealed LLM-specific patterns in function-word unigrams, POS bigrams, and phrase patterns.ConclusionThese findings indicate that Japanese public comments generated by the six LLMs can be accurately distinguished by focusing on the combination or patterns of stylometric features, suggesting LLM-specific linguistic fingerprints regardless of similarities in the underlying transformer architectures.

Similar Papers
  • Book Chapter
  • Citations3

Enhancing Understandability of Omics Data with SHAP, Embedding Projections and Interactive Visualisations

  • Jan 01, 2022
  • Zhonglin Qu +5
  • Research Article
  • Citations7

Stylometry can reveal artificial intelligence authorship, but humans struggle: A comparison of human and seven large language models in Japanese.

  • Oct 27, 2025
  • PloS one
  • Wataru Zaitsu +4
  • Research Article

Stylometry can reveal artificial intelligence authorship, but humans struggle: A comparison of human and seven large language models in Japanese

  • Oct 27, 2025
  • PLOS One
  • Wataru Zaitsu +5
  • Research Article
  • Citations5

GeneSetCart: assembling, augmenting, combining, visualizing, and analyzing gene sets.

  • Jan 06, 2025
  • GigaScience
  • Giacomo B Marino +4
  • Research Article
  • Citations9

Stylometry recognizes human and LLM-generated texts in short samples

  • Jan 01, 2026
  • Expert Systems with Applications
  • Karol Przystalski +3
  • Research Article

Prompting and Fine-Tuning Large Language Models for Parkinson Disease Diagnosis: Comparative Evaluation Study Using the PPMI Structured Dataset.

  • Jan 15, 2026
  • JMIR medical informatics
  • Hyun-Ji Shin +3
  • Research Article
  • Citations2

Large Language Models in Crisis Informatics for Zero and Few-Shot Classification

  • May 22, 2025
  • ACM Transactions on the Web
  • Cinthia Sánchez +2
  • Research Article

Machine Learning and Artificial Intelligence in Nutrition Research: Analytical Methods, Applications, and Key Considerations.

  • Apr 01, 2026
  • The Journal of nutrition
  • Nicole L Southey +2
  • Research Article
  • Citations133

Harmonizome 3.0: integrated knowledge about genes and proteins from diverse multi-omics resources

  • Nov 20, 2024
  • Nucleic Acids Research
  • Ido Diamant +4
  • Research Article

Mapping the mHealth Nexus: A Semantic Analysis of mHealth Scholars’ Research Propensities Following an Interdisciplinary Training Institute

  • Jun 01, 2025
  • Applied sciences (Basel, Switzerland)
  • Junpeng Ren +4
  • Research Article
  • Citations2

Enhancing large language model clinical support information with machine learning risk and explainability: a feasibility study.

  • Apr 21, 2026
  • Intensive care medicine experimental
  • Yu-Chang Yeh +5
  • PDF
  • Research Article
  • Citations16

Explaining Misinformation Detection Using Large Language Models

  • Apr 26, 2024
  • Electronics
  • Vishnu S Pendyala +1
  • PDF
  • Research Article
  • Citations111

Uniform Manifold Approximation and Projection (UMAP) Reveals Composite Patterns and Resolves Visualization Artifacts in Microbiome Data.

  • Oct 05, 2021
  • mSystems
  • George Armstrong +6
  • Research Article
  • Citations3

Development of an explainable machine learning model for predicting device-related pressure injuries in clinical settings

  • Jul 09, 2025
  • BMC Medical Informatics and Decision Making
  • Yijie Qian +7
  • Research Article
  • Citations19

Ophthalmological Question Answering and Reasoning Using OpenAI o1 vs Other Large Language Models

  • Jul 31, 2025
  • JAMA Ophthalmology
  • Sahana Srinivasan +18
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.