Consistency of News Titles and Contents With Multi Level Classification Using Bidirectional Encoder Representations from Transformer (BERT)
Abstract
News is written in a standard format that consist of a title followed by the content. The title must be present a content of news. Sometimes found that news title did not represent the content, causing readers to be disappointed when they finish reading the whole news. This research proposes multi-class and multi level classification using BERT algorithm to detect the suitability of news titles and their contents. The first level performs classification for complete data and the second level performs classification for data that has been separated into 2 parts, Title dan Conten. The dataset pertains to the realm of Covid-19 news because its substantial volume and diverse range of conversation topics. It is sufficient to be used as a dataset. We select several news topic classes that are often discussed, that are health, politic, and economy. In order to determine suitability, we categorise both the news Title and the news Content into predefined topic classes. The dataset exhibits an inbalance for each topic class. This research proposed a model to determine the appropriateness of both the news Title and the news Content. We use Bidirectional Encoder Representations from Transformers for building model. BERT is one of the state-of-the-art in the field of Natural Language Processing (NLP). Multi-layer classification that separates dataset (title and content) determine the consistency between them is very effective in increasing accuracy, and the BERT model is very supportive in solving linguistic problems well. This research succeeded in providing an accuracy 15% greater compared to previous research.
Downloads
References
Yoon S, Park K, Lee M, Kim T, Cha M, Jung K. Learning to Detect Incongruence in News Headline and Body Text via a Graph Neural Network. IEEE Access. 2021;9:36195–206. https://doi.org/10.1109/ACCESS.2021.3062029
Goebel JT, Susmann MW, Parthasarathy S, El Gamal H, Garrett RK, Wegener DT. Belief-consistent information is most shared despite being the least surprising. Sci Rep., Vol 14, No.1, 2024. https://doi.org/10.1038/s41598-024-56086-2
Chandra A. Konstruksi Berita Olahraga Pada Media Massa Internet (Analisis Framing Berita Konflik Marquez dan Rossi pada Sindonews.com). Jurusan Ilmu Komunikasi – Konsentrasi Jurnalistik, 4(2), pp.1–15, 2017.
Andersen K, Shehata A, Skovsgaard M, Strömbäck J. Selective News Avoidance: Consistency and Temporality. Communication Research. Vol. 53, No. 3, pp. 291–318 2026. https://doi.org/ 10.1177/00936502231221689
Yoon S, Park K, Shin J, Lim H, Won S, Cha M, et al. Detecting Incongruity between News Headline and Body Text via a Deep Hierarchical Encoder. AAAI. Vol 33, No.01, pp. 791–800 Jul 2019. https://doi.org/10.1609/aaai.v33i01.3301791
Zheng HT, Chen JY, Yao X, Sangaiah AK, Jiang Y, Zhao CZ. Clickbait Convolutional Neural Network. Symmetry. Vol. 10, No. 5, 2018 May 1, pp.138. https://doi.org/10.3390/sym10050138
Dijk TA van. News as discourse. Hillsdale (NJ): L. Erlbaum associates; 1988. (Communication).
Wirawan C. Indonesian BERT base model (uncased). Hugging Face [Internet]. 2021 [cited 2025 Apr 24]. Available from: https://huggingface.co/cahya/bert-base-indonesian-522M
Musman A, Mulyadi N. Jurnalisme Dasar: Panduan Praktis Para Jurnalis. Yogyakarta: Citra Media; 2022.
Chesney S, Liakata M, Poesio M, Purver M. Incongruent Headlines: Yet Another Way to Mislead Your Readers. In: Proceedings of the 2017 EMNLP Workshop: Natural Language Processing meets Journalism [Internet]. Copenhagen, Denmark: Association for Computational Linguistics; 2017 [cited 2026 Apr 24]. p. 56–61. https://doi.org/10.18653/v1/W17-4210
Jang J, Cho YS, Kim M, Kim M. Detecting incongruent news headlines with auxiliary textual information. Expert Systems with Applications. 2022 Aug;199:116866. https://doi.org/10.1016/j.eswa.2022.116866
Devlin J, Chang MW, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Proceedings of the 2019 Conference of the North [Internet]. Minneapolis, Minnesota: Association for Computational Linguistics; 2019 [cited 2026 Apr 24]. p. 4171–86. https://doi.org/10.18653/v1/N19-1423
Guo M, Ainslie J, Uthus D, Ontanon S, Ni J, Sung YH, et al. LongT5: Efficient Text-To-Text Transformer for Long Sequences. In: Findings of the Association for Computational Linguistics: NAACL 2022 [Internet]. Seattle, United States: Association for Computational Linguistics; 2022 [cited 2026 Apr 24]. pp. 724–36. https://doi.org/10.18653/v1/2022.findings-naacl.55
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I. Attention Is All You Need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17). Red Hook, NY, USA: Curran Associates Inc.; 2017. pp. 6000–6010.
Velavan TP, Meyer CG. The COVID‐19 epidemic. Tropical Med Int Health. vol. 25, no. 3, pp.278–280, 2020. https://doi.org/10.1111/tmi.13383
Ahmad T, Khan M, Haroon, Musa TH, Nasir S, Hui J, et al. COVID-19: Zoonotic aspects. Travel Medicine and Infectious Disease. Vol. 36. 101607 2020. https://doi.org/10.1016/j.tmaid.2020.101607
Department of Biological Sciences, Federal University Gusau Zamfara, Nigeria., Mohammed M, Umar AY, Department of Biological Sciences, Nigerian Defence Academy, Kaduna State, Nigeria. COVID-19 prevention and health promotion for the whole community. J Public Health Dis. vol. 6, no. 2, pp. 31–38, 2023. https://doi.org/10.31248/JPHD2023.131
Shamrat FMJM, Tasnim Z, Rahman AKMS, Nobel NI, Hossain SA. An effective implementation of web crawling technology to retrieve data from the world wide web (www). International Journal of Scientific & Technology Research. Vol. 9, No. 01, pp.1252–1256, 2020
Zhao B. Web Scraping. In: Schintler LA, McNeely CL, editors. Encyclopedia of Big Data [Internet]. Cham: Springer International Publishing; 2017 [cited 2026 Apr 24]. p. 1–3. https://doi.org/10.1007/978-3-319-32001-4_483-1
GitHub [Internet]. 2022. Scrapy at a glance. Available from: https://docs.scrapy.org/en/latest/intro/overview.html#scrapy-at-a-glance
Selenium [Internet]. 2021. The Selenium Browser Automation Project. Available from: https://www.selenium.dev/documentation/
Alzahrani E, Jololian L. How Different Text-Preprocessing Techniques Using the BERT Model Affect the Gender Profiling of Authors. In: Proceedings of the 3rd International Conference on Machine Learning & Applications (CMLA). Toronto, Canada; September 2021. https://doi.org/10.5121/csit.2021.111501
Santos G. What’s Hugging Face? Towards Data Science [Internet]. [cited 2022 Jun 19; updated 2025 Dec 28]. Available from: https://towardsdatascience.com/whats-hugging-face-122f4e7eb11a
Gardazi NM, Daud A, Malik MK, et al. BERT Applications in Natural Language Processing: A Review. Artificial Intelligence Review. 2025;58:166. https://doi.org/10.1007/s10462-025-11162-5
Jain SM. Introduction to Transformers. In: Introduction to Transformers for NLP [Internet]. Berkeley, CA: Apress; 2022 [cited 2026 Apr 24]. p. 19–36. Available from: https://link.springer.com/10.1007/978-1-4842-8844-3_2. https://doi.org/10.1007/978-1-4842-8844-3_2
Valkov V. Sentiment Analysis with BERT and Transformers by Hugging Face using PyTorch and Python. Curiousily [Internet]. 2020 Apr 20 [cited 2022 Jul 4]. Available from: https://curiousily.com/posts/sentiment-analysis-with-bert-and-hugging-face-using-pytorch-and-python/
Guan L. Weight Prediction Boosts the Convergence of AdamW [Internet]. arXiv; 2023 [cited 2026 Apr 24]. Available from: https://arxiv.org/abs/2302.00195. https://doi.org/10.48550/ARXIV.2302.00195
Smith LN. Cyclical Learning Rates for Training Neural Networks. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). 2017. pp. 464–472.
Zeng G. Invariance Properties and Evaluation Metrics Derived from the Confusion Matrix in Multiclass Classification. Mathematics. 2025;13(16):2609. https://doi.org/10.3390/math13162609
Aguilar-Ruiz JS, Michalak M. Multiclass Classification Performance Curve. IEEE Access. vol.10, pp. 68915–68921, 2022. https://doi.org/10.1109/ACCESS.2022.3186444
Alaros E, Marjani M, Shafiq DA, Asirvatham D. Predicting Consumption Intention of Consumer Relationship Management Users Using Deep Learning Techniques: A Review. Indonesian J Sci Technol. Vol. 8, no. 2, pp. 307–328, 2022. https://doi.org/10.17509/ijost.v8i2.55814
Brahmi AE, Abderafi S, Ellaia R. Artificial Neural Network Analysis of Sulfide Production in A Moroccan Sewerage Network. Indonesian J Sci Technol. vol. 6, no.1, pp:193–204, 2021. https://doi.org/10.17509/ijost.v6i1.32322.
Copyright (c) 2026 EMITTER International Journal of Engineering Technology
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The copyright to this article is transferred to Politeknik Elektronika Negeri Surabaya(PENS) if and when the article is accepted for publication. The undersigned hereby transfers any and all rights in and to the paper including without limitation all copyrights to PENS. The undersigned hereby represents and warrants that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. The undersigned represents that he/she has the power and authority to make and execute this assignment. The copyright transfer form can be downloaded here .
The corresponding author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors. This agreement is to be signed by at least one of the authors who have obtained the assent of the co-author(s) where applicable. After submission of this agreement signed by the corresponding author, changes of authorship or in the order of the authors listed will not be accepted.
Retained Rights/Terms and Conditions
- Authors retain all proprietary rights in any process, procedure, or article of manufacture described in the Work.
- Authors may reproduce or authorize others to reproduce the work or derivative works for the author’s personal use or company use, provided that the source and the copyright notice of Politeknik Elektronika Negeri Surabaya (PENS) publisher are indicated.
- Authors are allowed to use and reuse their articles under the same CC-BY-NC-SA license as third parties.
- Third-parties are allowed to share and adapt the publication work for all non-commercial purposes and if they remix, transform, or build upon the material, they must distribute under the same license as the original.
Plagiarism Check
To avoid plagiarism activities, the manuscript will be checked twice by the Editorial Board of the EMITTER International Journal of Engineering Technology (EMITTER Journal) using iThenticate Plagiarism Checker and the CrossCheck plagiarism screening service. The similarity score of a manuscript has should be less than 25%. The manuscript that plagiarizes another author’s work or author's own will be rejected by EMITTER Journal.
Authors are expected to comply with EMITTER Journal's plagiarism rules by downloading and signing the plagiarism declaration form here and resubmitting the form, along with the copyright transfer form via online submission.
