Evaluating YOLOv5s and YOLOv8s for Kitchen Fire Detection: A Comparative Analysis
Abstract
Accurate and timely detection of kitchen fires is crucial for enhancing safety and reducing potential damage. This paper discusses comparative analysis of two cutting-edge object detection models, YOLOv5s and YOLOv8s, focusing on each performance in the critical application of kitchen fire detection. The performance of these models is evaluated using five main key metrics including precision, F1 score, recall, mean Average Precision across various thresholds (mAP50-95) and mean Average Precision at 50 percent threshold (mAP50). Results indicate that YOLOv8s significantly outperforms YOLOv5s in several metrics. YOLOv8s achieves a recall of 0.814 and an mAP50 of 0.897, compared to YOLOv5s' recall of 0.704 and mAP50 of 0.783. Additionally, YOLOv8s attains an F1 score of 0.861 and an mAP50-95 of 0.465, whereas YOLOv5s records an F1 score of 0.826 and mAP50-95 of 0.342. However, YOLOv5s shows a higher precision of 0.952 compared to YOLOv8s' 0.914. This detailed evaluation underscores YOLOv8s as a more effective model for precise fire detection in kitchen settings, highlighting its potential for enhancing real-time fire safety systems. Additionally, by offering the future work of integration of sensors with latest YOLO involvement can further optimize efficiency and fast detection rate.
Downloads
References
S. Hall, and T. Mcgree, Home cooking fires: supporting tables September 2023 (NFPA ®)
M. Ahrens, Home fires involving cooking equipment. National Fire Protection Association Fire Analysis and Research Division, 2009
A. Dinesh, T. Polanco, and R. Engdahl, Burns from ignited household aerosols in the kitchen: a case series. Scars Burn Heal, 3, 205951311772820, 2017
M. Spearpoint, C. Hopkin, and D. Hopkin, Modelling the thermal radiation from kitchen hob fires. Journal Fire Science, 38 (4), 377–394, 2020
M.B. Hamida, and M.A. Hassanain, Fire safety in the built-environment: a case study in a residential facility, Architecture, Civil Engineering, Environment, 12 (2), 2019
H.B. Choi, E.H. Hwang, and D.M. Choi, Indoor air quality sensor utilization for unwanted fire alarm improvement in studio-type apartments. Fire, 6 (7), 2023
J. Milke, and R. Zevotek, Analysis of the response of smoke detectors to smoldering fires and nuisance sources. Fire Technology, 52 (5), 1235–1253, 2016
J. Johnston, K. Zeng, and N. Wu, An evaluation and embedded hardware implementation of yolo for real-time wildfire detection. 2022 IEEE World AI IoT Congress, AIIoT 2022, 138–144, 2022
L. An, L. Chen, and X. Hao, Indoor fire detection algorithm based on second-order exponential smoothing and information fusion. Information, 14 (5), 258. 2023
E. Casas, L. Ramos, E. Bendek, and F. Rivas-Echeverria, YOLOv5 vs. YOLOv8: Performance benchmarking in wildfire and smoke detection scenarios. Journal of Image and Graphics, 12 (2), 127–136, 2024
A. Rehman, D. Kim, and A. Paul, Convolutional neural network model for fire detection in real-time environment. Computers, Materials & Continua, 77 (2), 2289–2307, 2023
Y. Li, J. Shang, M. Yan, B. Ding, and J. Zhong, Real-time early indoor fire detection and localization on embedded platforms with fully convolutional one-stage object detection. Sustainability, 15 (3), 1794, 2023
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, You Only Look Once: Unified, real-time object detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 779–788, 2016
J. Redmon, and A. Farhadi, YOLO9000: Better, faster, stronger. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 6517–6525, 2017
J. Redmon, and A. Farhadi, YOLOv3: An incremental improvement, 2018
A. Bochkovskiy, C.-Y. Wang, and H.-Y.M. Liao, YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934, 2020
G. Jocher, YOLOv5 by Ultralytics, 2020
C. Li, L. Li, H. Jiang, K. Weng, Y. Geng, L. Li, Z. Ke, Q. Li, M. Cheng, W.; Li, Y. Nie, B. Zhang, Y. Liang, L. Zhou, X. Xu, X. Chu, X. Wei, and X. Wei, YOLOv6: A single-stage object detection framework for industrial applications. arXiv preprint arXiv:2209.02976, 2022
C.-Y. Wang, A. Bochkovskiy, and H.-Y.M. Liao, YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 7464-7475), 2022
C.-Y. Wang, I.-H. Yeh, and H.-Y.M. Liao, YOLOv9: Learning what you want to learn using programmable gradient information. arXiv preprint arXiv:2402.13616, 2024
L. Zhao, L. Zhi, C. Zhao, W. Zheng, Fire-YOLO: A Small Target Object Detection Method for Fire Inspection. Sustainability, 14, 4930, 2022
H. Wu, Y. Hu, X. Mei, and J. Xian, Ship fire detection based on an improved yolo algorithm with a lightweight convolutional neural network model. Sensors, 22(19):7420. 2022
S. Wang, and X. Wang, ES-YOLO: A new lightweight fire detection model. Third International Conference on Computer Vision and Data Mining (ICCVDM 2022), 125111F, 2023
X. Wan, J. Cai, S. Luo, Z. Tian, L. Zhang and X. Xia, Gaussian Process for the Machine Learning-based Smart fire Detection System. 2022 IEEE 6th Information Technology and Mechatronics Engineering Conference (ITOEC), Chongqing, China, 100-104, 2022
M. A. Rahman, S. T. Hasan and M. A. Kader, Computer Vision Based Industrial and Forest Fire Detection Using Support Vector Machine (SVM). International Conference on Innovations in Science, Engineering and Technology (ICISET), Chittagong, Bangladesh, 233-238, 2022
J. Zhang, J. Zhang, K. Zhou, Y. Zhang, H. Chen, and X. Yan, An improved yolov5-based underwater object-detection framework. Sensors, 23 (7), 3693, 2023
M.L. Mekhalfi, C. Nicolo, Y. Bazi, M.M. Al. Rahhal, N.A. Alsharif, and E. Al. Maghayreh, Contrasting Yolov5, transformer, and efficientdet detectors for crop circle detection in desert. IEEE Geoscience and Remote Sensing Letters, 19, 2022
Z. Liu, X. Gao, Y. Wan, J. Wang, and H. Lyu, An improved Yolov5 method for small object detection in UAV capture scenes. IEEE Access, 11, 14365–14374, 2023
W. Liu, K. Quijano, and M.M. Crawford, YOLOv5-Tassel: Detecting tassels in RGB UAV imagery with improved yolov5 based on transfer learning. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15, 8085–8094, 2022
S. Wu, Z. Li, S. Li, Q. Liu, and W. Wu, Static gesture recognition algorithm based on improved yolov5s. Electronics (Basel), 12 (3), 596, 2023
G. Jocher, A. Chaurasia, and J. Qiu, YOLO by Ultralytics, 2023
J.H. Kim, N. Kim, and C.S. Won, High-speed drone detection based on yolo-v8. ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2023-June, 2023
A. Vats, and D.C. Anastasiu, Enhancing retail checkout through video inpainting, Yolov8 detection, and deepsort tracking. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 5530–5537, 2023
M. Hussain, H. Al-Aqrabi, M. Munawar, R. Hill, and T. Alsboui, (). Domain feature mapping with yolov7 for automated edge-based pallet racking inspections. Sensors, 22 (18), 6927, 2022
G. Zeng, On the confusion matrix in credit scoring and its analytical properties. Communications in Statistics- Theory and Methods, 49 (9), 2080–2093, 2020
Y. Zhang, T. Zuo, L. Fang, J. Li, and Z. Xing, An improved MAHAKIL oversampling method for imbalanced dataset classification. IEEE Access, 9, 16030–16040, 2021
E. Casas, L. Ramos, E. Bendek, and F. Rivas-Echeverria, Assessing the effectiveness of yolo architectures for smoke and wildfire detection. IEEE Access, 11, 96554–96583, 2023
M. Vergara, L. Ramos, N.D. Rivera-Campoverde, and F. Rivas-Echeverria, EngineFaultDB: A novel dataset for automotive engine fault classification and baseline results. IEEE Access, 11, 126155–126171, 2023
R. Padilla, S.L. Netto, and E.A.B. da Silva, A survey on performance metrics for object-detection algorithms. 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), 237–242, 2020
R. Padilla, W.L. Passos, T.L.B. Dias, S.L. Netto, and E.A.B. da Silva, A comparative analysis of object detection metrics with a companion open-source toolkit. Electronics (Basel), 10 (3), 279, 2021
Z. Ning, X. Wu, J. Yang, and Y. Yang, MT-YOLOv5: Mobile terminal table detection model based on YOLOv5. Journal of Physics: Conference Series, 1978 (1), 012010, 2021
H. Zhu, H. Wei, B. Li, X. Yuan, and N. Kehtarnavaz, A review of video object detection: datasets, metrics and methods. Applied Sciences, 10 (21), 7834, 2020
Copyright (c) 2024 EMITTER International Journal of Engineering Technology
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The copyright to this article is transferred to Politeknik Elektronika Negeri Surabaya(PENS) if and when the article is accepted for publication. The undersigned hereby transfers any and all rights in and to the paper including without limitation all copyrights to PENS. The undersigned hereby represents and warrants that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. The undersigned represents that he/she has the power and authority to make and execute this assignment. The copyright transfer form can be downloaded here .
The corresponding author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors. This agreement is to be signed by at least one of the authors who have obtained the assent of the co-author(s) where applicable. After submission of this agreement signed by the corresponding author, changes of authorship or in the order of the authors listed will not be accepted.
Retained Rights/Terms and Conditions
- Authors retain all proprietary rights in any process, procedure, or article of manufacture described in the Work.
- Authors may reproduce or authorize others to reproduce the work or derivative works for the author’s personal use or company use, provided that the source and the copyright notice of Politeknik Elektronika Negeri Surabaya (PENS) publisher are indicated.
- Authors are allowed to use and reuse their articles under the same CC-BY-NC-SA license as third parties.
- Third-parties are allowed to share and adapt the publication work for all non-commercial purposes and if they remix, transform, or build upon the material, they must distribute under the same license as the original.
Plagiarism Check
To avoid plagiarism activities, the manuscript will be checked twice by the Editorial Board of the EMITTER International Journal of Engineering Technology (EMITTER Journal) using iThenticate Plagiarism Checker and the CrossCheck plagiarism screening service. The similarity score of a manuscript has should be less than 25%. The manuscript that plagiarizes another author’s work or author's own will be rejected by EMITTER Journal.
Authors are expected to comply with EMITTER Journal's plagiarism rules by downloading and signing the plagiarism declaration form here and resubmitting the form, along with the copyright transfer form via online submission.
