Hallucination Detection in Natural Language Generation via Knowledge-Transfer Distillation

dc.AffiliationOctober University for modern sciences and Arts MSA
dc.contributor.authorAbdelrahman Radwan
dc.contributor.authorMohammad Emad
dc.contributor.authorAli Hamdi
dc.date.accessioned2026-11-16T06:00:55Z
dc.date.issued2027-02-02
dc.descriptionSJR 2025 0.165 Q4 H-Index 57 Subject Area and Category: Computer Science Computer Networks and Communications Signal Processing Engineering Control and Systems Engineering
dc.description.abstractIn Natural Language Generation (NLG), hallucinations occur when the models’ output content is semantically unsupported or factually incorrect, creating real serious challenges and critical problems for many real-world applications. In this research work, a novel framework for hallucination detection is proposed. Our method integrates the principles of knowledge transfer and knowledge distillation by utilizing the hard labels from a pre-trained NLI transformer model with soft probabilistic labels from a large teacher model (GPT-OSS-120B). We introduce a novel hybrid dual-objective loss function, termed Focal–Distillation Loss, which combines a modified Cross-Entropy—Focal Cross-Entropy—for hard labels with Kullback–Leibler divergence for soft probabilistic labels. This loss function is used to fine-tune the DeBERTa-v3 base model as the student model. This dual supervision with the new dual-objective loss enables the student model to capture complex patterns in the generated text and learn them. Our novel framework is demonstrated by an experiment on the SHROOM 2024 hallucination detection dataset, which achieves state-of-the-art performance, reaching an accuracy of 70.0% and outperforming several existing systems, and there is still room for improvement. These findings demonstrate the effectiveness of combining model-based knowledge transfer with LLM-based distillation, supported by the proposed Focal–Distillation Loss, in advancing hallucination detection within Natural Language Generation.
dc.description.urihttps://www.scimagojr.com/journalsearch.php?q=21100901469&tip=sid&clean=0
dc.identifier.citationRadwan, A., Emad, M., & Hamdi, A. (2027). Hallucination detection in natural language generation via knowledge-transfer distillation. Lecture Notes in Networks and Systems, 138–149. https://doi.org/10.1007/978-3-032-34434-2_12
dc.identifier.doihttps://doi.org/10.1007/978-3-032-34434-2_12
dc.identifier.otherhttps://doi.org/10.1007/978-3-032-34434-2_12
dc.identifier.urihttps://repository.msa.edu.eg/handle/123456789/6900
dc.language.isoen_US
dc.publisherLecture Notes in Networks and Systems
dc.relation.ispartofseriesLecture Notes in Networks and Systems; pp 138–149
dc.subjectHallucination Detection
dc.subjectKnowledge Distillation
dc.subjectTeacher-Student Model
dc.subjectTransformer Models
dc.titleHallucination Detection in Natural Language Generation via Knowledge-Transfer Distillation
dc.typeArticle

Files

Original bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
IMG-20231214-WA0000.jpg
Size:
16.8 KB
Format:
Joint Photographic Experts Group/JPEG File Interchange Format (JFIF)

License bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
license.txt
Size:
51 B
Format:
Item-specific license agreed upon to submission
Description: