Discourse-Driven Detection of Multimodal Misinformation

Abstract

Translate Article

The spread of misinformation on social platforms is amplified by multimodal posts and the discourse they trigger. We present <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LANTERN</monospace>, a lightweight framework that detects fake posts by pairing a compact LVLM (Large Visio-Language Model) with a graph-to-text rendering of reply threads. Comment trees are pruned breadth-first and serialized with depth-based indentation and in-line metadata (author, engagement), then concatenated with image tokens; the LVLM is fine-tuned with low-rank adapters and a linear head, using the end-of-sequence state for classification. On <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Fakeddit</i>, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LANTERN</monospace> attains state-of-the-art accuracy with only 150 k training samples, achieving <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">97.16%</b> (2-way), <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">97.05%</b> (3-way), <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">93.72%</b> (6-way). To assess versatility beyond Reddit, we additionally evaluate <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LANTERN-TEXT</monospace> on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PHEME</i> (text-only rumor cascades) and adapt the pipeline to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Twitter15/16</i> (structure-only replies), observing robust gains from shallow discourse and and measurable gains even when reply text is absent. Large-scale multimodal pretraining plus lightweight adaptation and structured conversational context yields a practical and reproducible approach to misinformation detection.