Particle Swarm Optimization–Tuned Random Forest for Multi-Class Mental Health Sentiment Classification from Textual Data
Abstract
Mental health sentiment classification from textual data has attracted increasing attention as a computational approach to support large-scale psychological assessment; however, multi-class classification remains challenging due to noisy text, class imbalance, and semantic overlap among categories. This study proposes and evaluates a machine learning framework for seven-class mental health sentiment classification that integrates enhanced text preprocessing with lemmatization, data augmentation via back-translation, TF-IDF feature extraction, and systematic model evaluation across multiple classifiers, including Logistic Regression, Decision Tree, Random Forest, Gradient Boosting, K-Nearest Neighbors, AdaBoost, and XGBoost, under three hyperparameter tuning strategies: Grid Search, Random Search, and Particle Swarm Optimization (PSO). Experimental results indicate that ensemble-based models consistently outperform single classifiers, with the PSO-optimized Random Forest achieving the best numerical performance, attaining an accuracy of 0.933, a macro F1-score of 0.923, and a ROC AUC of 0.989, demonstrating strong generalization and balanced class-level performance despite dataset imbalance. These findings confirm that the combination of robust preprocessing and metaheuristic-based hyperparameter optimization significantly enhances multi-class mental health sentiment classification and supports its potential use as a scalable decision-support tool for large-scale mental health screening, while not intended to replace clinical diagnosis.
Downloads
References
N. K. Iyortsuun, S. Kim, M. Jhon, and H. Yang, A Review of Machine Learning and Deep Learning Approaches on Mental Health Diagnosis, Healthcare, Vol. 11, No. 3, pp. 1–27, 2023, https://doi.org/10.3390/healthcare11030285.
U. Madububambachu, A. Ukpebor, and U. Ihezue, Machine Learning Techniques to Predict Mental Health Diagnoses : A Systematic Literature Review, Clin Pract Epidemiol Ment Health. no. Ml, pp. 1–16, 2024, https://doi.org/10.2174/0117450179315688240607052117.
M. Garg, Mental Health Analysis in Social Media Posts: A Survey, Archives of Computational Methods in Engineering, vol. 30, no. 3, pp. 1819–1842, 2023, https://doi.org/10.1007/s11831-022-09863-z.
J. Kim, D. Lee, and E. Park, Machine Learning for Mental Health in Social Media. J Med Internet Res, vol. 23, pp. 1–17, 2021, https://doi.org/10.2196/24870.
G. Fu et al., Distant supervision for mental health management in social media: Suicide risk classification system development study, J. Med. Internet Res., vol. 23, no. 8, 2021, https://doi.org/10.2196/26119.
A. Le Glaz, Y. Haralambous, P. Lenca, and R. Billot, Machine Learning and Natural Language Processing in Mental Health, J. Med. Internet Res, vol. 23, https://doi.org/10.2196/15708.
Palanivinayagam, C. Z. El-Bayeh, and R. Damaševičius, Twenty Years of Machine-Learning-Based Text Classification: A Systematic Review, Algorithms, vol. 16, no. 5, pp. 1–28, 2023, https://doi.org/10.3390/a16050236.
A.G. Putrada, N. Alamsyah, S. F. Pane, and M. Nurkamal Fauzan, Feature Importance on Text Analysis for a Novel Indonesian Movie Recommender System, in 2023 11th International Conference on Information and Communication Technology (ICoICT), 2023, pp. 34–39. https://doi.org/10.1109/ICoICT58202.2023.10262504.
O. D. Adeniji, S. O. Adeyemi, and S. A. Ajagbe, An Improved Bagging Ensemble in Predicting Mental Disorder Using Hybridized Random Forest - Artificial Neural Network Model, Informatica, vol. 46, pp. 543–550, 2022, https://doi.org/10.31449/inf.v46i4.3916.
Konda Vaishnavi, U Nikhitha Kamath, Ashwath Rao B, N V Subba Reddy, Predicting Mental Health Illness using Machine Learning Algorithms, J. of Physics: Conference Series, Vol. 2161, 1st International Conference on Artificial Intelligence, Computational Electronics and Communication System (AICECS 2021) 28-30 October 20212022, https://doi.org/10.1088/1742-6596/2161/1/012021
Q. Li and J. Zhou, A Comparative Analysis of Extreme Gradient Boosting, Decision Tree, Support Vector Machines, and Random Forest Algorithm in Data Analysis of College Students’ Psychological Health, Informatica, vol. 49, No. 15, pp. 127–134, 2025, doi: https://doi.org/10.31449/inf.v49i15.7004.
Y. Xin and X. Ren, Predicting depression among rural and urban disabled elderly in China using a random forest classifier, BMC Psychiatry, pp. 1–11, 2022, https://doi.org 10.1186/s12888-022-03742-4.
B. Ramadhan and S. F. Pane, Pengaruh Hyperparameter Tuning untuk Efektivitas pada Pendekatan Hybrid dalam Mendiagnosis Stres dan Depresi: Tinjauan Studi Literatur, Jurnal Tekno Insentif, vol. 18, no. 2, pp. 104–118, 2024, https://doi.org/10.36787/jti.v18i2.1516.
R. Choirunnisa, M. Anshori, and W. T. Kusuma, Improving Random Forest Evaluation in Mental Health Disorder Identification with Cross Validation, RIGGS Journal of Artificial Intelligence and Digital Business, vol. 4, no. 2, pp. 3526–3534, 2025, https://doi.org/10.31004/riggs.v4i2.1053.
M. Jain, V. Saihjpal, N. Singh, and S. B. Singh, An Overview of Variants and Advancements of PSO Algorithm, Applied Sciences (Switzerland), vol. 12, no. 17, pp. 1–21, 2022, doi: 10.3390/app12178392.
D.-D. Ramírez-Ochoa, L. A. Pérez-Domínguez, E.-A. Martínez-Gómez, and D. Luviano-Cruz, PSO, a Swarm Intelligence-Based Evolutionary Algorithm as a Decision-Making Strategy: A Review, Symmetry, Vol. 14, No. 3, 2022, https://doi.org/10.3390/sym14030455
J. Chung and J. Teo, Mental Health Prediction Using Machine Learning: Taxonomy, Applications, and Challenges, Applied Computational Intelligence and Soft Computing, vol. 2022, 2022, https://doi.org/10.1155/2022/9970363.
N. Dhariwal et al., A pilot study on AI-driven approaches for classification of mental health disorders, Front. Hum. Neurosci., vol. 18, 2024, https://doi.org/10.3389/fnhum.2024.1376338.
T. Kallstenius, A. J. Capusan, G. Andersson, and A. Williamson, Comparing traditional natural language processing and large language models for mental health status classification: a multi-model evaluation, Sci. Rep., vol. 15, no. 1, pp. 1–13, 2025, https://doi.org/10.1038/s41598-025-08031-0.
T. Xiao, D. Ren, S. Lei, J. Zhang, and X. Liu, Based on grid-search and PSO parameter optimization for Support Vector Machine, in Proceeding of the 11th World Congress on Intelligent Control and Automation, 2014, pp. 1529–1533. https://doi.org/10.1109/WCICA.2014.7052946.
K. Shiomi, T. Sato, and E. Kita, Comparison of Particle Swarm Optimization Algorithms in Hyperparameter Optimization Problem of Multi Layered Perceptron, Computer Assisted Methods in Engineering and Science, vol. 32, no. 1, pp. 3–24, 2025, https://doi.org/10.24423/cames.2025.1730.
H. S. Salem, M. A. Mead, and G. S. El-Taweel, Particle Swarm Optimization-Based Hyperparameters Tuning of Machine Learning Models for Big COVID-19 Data Analysis, Journal of Computer and Communications, vol. 12, no. 03, pp. 160–183, 2024, https://doi.org/10.4236/jcc.2024.123010.
S. Ghosal and A. Jain, Depression and Suicide Risk Detection on Social Media using fastText Embedding and XGBoost Classifier, Procedia Comput. Sci., vol. 218, pp. 1631–1639, 2022, https://doi.org/10.1016/j.procs.2023.01.141.
M. M. Hossain, M. S. Hossain, M. F. Mridha, M. Safran, and S. Alfarhood, Multi task opinion enhanced hybrid BERT model for mental health analysis, Sci. Rep., vol. 15, no. 1, pp. 1–20, 2025, https://doi.org/10.1038/s41598-025-86124-6.
S. F. Pane, F. Abdullah, and R. Habibi, Deteksi Emosi Pada Teks Berbahasa Indonesia Menggunakan Pendekatan Ensemble, JTT (Jurnal Teknologi Terapan), vol. 10, no. 2, p. 80, 2024, https://doi.org/10.31884/jtt.v10i2.551.
A. N. Sutranggono, R. Sarno, and I. Ghozali, Multi-Class Multi-Level Classification of Mental Health Disorders Based on Textual Data from Social Media, The Journal of Information and Communication Technology, vol. 1, no. v. 1, 1, pp. 117–148, 2023, [Online]. Available: https://books.google.co.id/books?id=5ySg3D3s-sIC
S. Juanita, A. H. Daeli, M. Syafrullah, W. Anggraeni, and M. H. Purnomo, A machine learning approach for text pattern diagnosis in mental health consultations, Decision Analytics Journal, vol. 15, no. March, p. 100572, 2025, https://doi.org/10.1016/j.dajour.2025.100572.
T. Zhang, A. M. Schoene, S. Ji, and S. Ananiadou, Natural language processing applied to mental illness detection: a narrative review, NPJ Digit. Med., vol. 5, no. 1, pp. 1–13, 2022, https://doi.org/10.1038/s41746-022-00589-7.
H. T. Duong and T. A. Nguyen-Thi, A review: preprocessing techniques and data augmentation for sentiment analysis, Comput. Soc. Netw., vol. 8, no. 1, pp. 1–16, 2021, https://doi.org/10.1186/s40649-020-00080-x.
M. A. Palomino and F. Aider, Evaluating the Effectiveness of Text Pre-Processing in Sentiment Analysis, Applied Sciences (Switzerland), vol. 12, no. 17, 2022, https://doi.org/10.3390/app12178765.
M. N. Fauzan, A. G. Putrada, N. Alamsyah, and S. F. Pane, PCA-AdaBoost Method for a Low Bias and Low Dimension Toxic Comment Classification, in 2022 International Conference on Advanced Creative Networks and Intelligent Systems (ICACNIS), 2022, pp. 1–6. https://doi.org/10.1109/ICACNIS57039.2022.10055017.
D. A. Anggoro and S. S. Mukti, Performance Comparison of Grid Search and Random Search Methods for Hyperparameter Tuning in Extreme Gradient Boosting Algorithm to Predict Chronic Kidney Failure, International Journal of Intelligent Engineering and Systems, vol. 14, no. 6, pp. 198–207, 2021, https://doi.org/10.22266/ijies2021.1231.19.
L. Abualigah, Particle Swarm Optimization: Advances, Applications, and Experimental Insights, Computers, Materials and Continua, vol. 82, no. 2, pp. 1539–1592, 2025, https://doi.org/10.32604/ cmc.2025.060765.
J. Veintimilla-Reyes, B. Guerrero, D. Maldonado-Segarra, and R. Ortíz-Gaona, Application of a Heuristic Model (PSO—Particle Swarm Optimization) for Optimizing Surface Water Allocation in the Machángara River Basin, Ecuador, Water (Switzerland), vol. 17, no. 16, pp. 1–26, 2025, https://doi.org/10.3390/w17162481.
N. Maldini et al., “Deteksi Dini Gangguan Kesehatan Mental berdasarkan Sentimen menggunakan Metode Stacking Early Detection of Mental Health Disorders based on Sentiment using Stacking Method,” Sistemasi: Jurnal Sistem Informasi, vol. 14, no. 1, pp. 2540–9719, 2025, doi: https://doi.org/10.32520/stmsi.v14i1.4842.
S. Basuki, R. Indrabayu, and N. A. Effendy, Automatic Categorization of Mental Health Frame in Indonesian X (Twitter) Text using Classification and Topic Detection Techniques, Khazanah Informatika : Jurnal Ilmu Komputer dan Informatika, vol. 10, no. 2, pp. 104–110, 2025, https://doi.org/10.23917/khif.v10i2.3328.
C. Sintiya, G. H. Hutagaol, D. Bate`e, and S. Irviantina, Evaluasi Teknik Resampling untuk Class Balancing dalam Analisis Sentimen Kesehatan Mental Berbasis Bi-LSTM, Jurnal Sifo Mikroskil, vol. 26, no. 2, pp. 257–274, 2025, https://doi.org/10.55601/jsm.v26i2.1799.
I. Villanueva-Miranda, Y. Xie, and G. Xiao, Sentiment analysis in public health: a systematic review of the current state, challenges, and future directions, Front. Public Health, vol. 13, 2025, https://doi.org/10.3389/fpubh.2025.1609749.
M. R. Alfreda Cecio Salwa Alexita, Pramesti Kusumaningtyas, Optimasi Algoritma Random Forest Menggunakan PSO Untuk Klasifikasi Kanker Payudara Dengan Citra Mammograms, Teknika STTKD Jurnal Teknik Elektronik Engine, vol. 11, no. 1, pp. 47–54, 2025, doi: https://doi.org/10.56521/teknika.v11i1.1346.
N. Agrawal, M. Mahrishi, M. K. Gupta, and M. K. Bohra, Large scale multi-class pest image classification using structurally adapted DenseNet architecture, Sci. Rep., vol. 16, no. 1, Dec. 2026, https://doi.org/10.1038/s41598-026-49685-8.
M. Salmi, D. Atif, D. Oliva, A. Abraham, and S. Ventura, Handling imbalanced medical datasets: review of a decade of research, Artif. Intell. Rev., vol. 57, no. 10, Oct. 2024, https://doi.org/10.1007/s10462-024-10884-2.
D. Remawati, E. Noersasongko, A. Marjuni, and Pujiono, “Mental Health Detection with TF-IDF Feature Extraction,” in 2024 IEEE International Conference on Artificial Intelligence and Mechatronics Systems (AIMS), 2024, pp. 1–6. doi: 10.1109/AIMS61812.2024.10512480.
J. Chung and J. Teo, Single classifier vs. ensemble machine learning approaches for mental health prediction, Brain Inform., vol. 10, no. 1, pp. 1–10, 2023, https://doi.org/10.1186/s40708-022-00180-6.
T. Zhang, K. Yang, S. Ji, and S. Ananiadou, “Emotion fusion for mental illness detection from social media: A survey,” Information Fusion, vol. 92, no. December 2022, pp. 231–246, 2023, doi: 10.1016/j.inffus.2022.11.031.
A. V. Lakra and S. Jena, Optimization of Random Forest Hyperparameter Using Improved PSO for Handwritten Digits Classification, In: Panda, S.K., Rout, R.R., Sadam, R.C., Rayanoothala, B.V.S., Li, KC., Buyya, R. (eds) Computing, Communication and Learning. CoCoLe 2022. Communications in Computer and Information Science, vol 1729. Springer, Cham. https://doi.org/10.1007/978-3-031-21750-0_23S.
A. Salloum, A. Almomani, K. M. Alomari, T. Khdour, and M. Alauthman, Predicting Thyroid Dysfunction Using Classical Machine Learning with Rigorous Statistical Evaluation, IEEE Access, 2026, https://doi.org/10.1109/ACCESS.2026.3666210.
Copyright (c) 2026 EMITTER International Journal of Engineering Technology
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The copyright to this article is transferred to Politeknik Elektronika Negeri Surabaya(PENS) if and when the article is accepted for publication. The undersigned hereby transfers any and all rights in and to the paper including without limitation all copyrights to PENS. The undersigned hereby represents and warrants that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. The undersigned represents that he/she has the power and authority to make and execute this assignment. The copyright transfer form can be downloaded here .
The corresponding author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors. This agreement is to be signed by at least one of the authors who have obtained the assent of the co-author(s) where applicable. After submission of this agreement signed by the corresponding author, changes of authorship or in the order of the authors listed will not be accepted.
Retained Rights/Terms and Conditions
- Authors retain all proprietary rights in any process, procedure, or article of manufacture described in the Work.
- Authors may reproduce or authorize others to reproduce the work or derivative works for the author’s personal use or company use, provided that the source and the copyright notice of Politeknik Elektronika Negeri Surabaya (PENS) publisher are indicated.
- Authors are allowed to use and reuse their articles under the same CC-BY-NC-SA license as third parties.
- Third-parties are allowed to share and adapt the publication work for all non-commercial purposes and if they remix, transform, or build upon the material, they must distribute under the same license as the original.
Plagiarism Check
To avoid plagiarism activities, the manuscript will be checked twice by the Editorial Board of the EMITTER International Journal of Engineering Technology (EMITTER Journal) using iThenticate Plagiarism Checker and the CrossCheck plagiarism screening service. The similarity score of a manuscript has should be less than 25%. The manuscript that plagiarizes another author’s work or author's own will be rejected by EMITTER Journal.
Authors are expected to comply with EMITTER Journal's plagiarism rules by downloading and signing the plagiarism declaration form here and resubmitting the form, along with the copyright transfer form via online submission.
