Evaluating YOLOv5s and YOLOv8s for Kitchen Fire Detection: A Comparative Analysis
Abstract
Accurate and timely detection of kitchen fires is crucial for enhancing safety and reducing potential damage. This paper discusses comparative analysis of two cutting-edge object detection models, YOLOv5s and YOLOv8s, focusing on each performance in the critical application of kitchen fire detection. The performance of these models is evaluated using five main key metrics including precision, F1 score, recall, mean Average Precision across various thresholds (mAP50-95) and mean Average Precision at 50 percent threshold (mAP50). Results indicate that YOLOv8s significantly outperforms YOLOv5s in several metrics. YOLOv8s achieves a recall of 0.814 and an mAP50 of 0.897, compared to YOLOv5s' recall of 0.704 and mAP50 of 0.783. Additionally, YOLOv8s attains an F1 score of 0.861 and an mAP50-95 of 0.465, whereas YOLOv5s records an F1 score of 0.826 and mAP50-95 of 0.342. However, YOLOv5s shows a higher precision of 0.952 compared to YOLOv8s' 0.914. This detailed evaluation underscores YOLOv8s as a more effective model for precise fire detection in kitchen settings, highlighting its potential for enhancing real-time fire safety systems. Additionally, by offering the future work of integration of sensors with latest YOLO involvement can further optimize efficiency and fast detection rate.
Downloads
References
S. Hall, and T. Mcgree, Home cooking fires: supporting tables September 2023 (NFPA ®)
M. Ahrens, Home fires involving cooking equipment. National Fire Protection Association Fire Analysis and Research Division, 2009
A. Dinesh, T. Polanco, and R. Engdahl, Burns from ignited household aerosols in the kitchen: a case series. Scars Burn Heal, 3, 205951311772820, 2017 DOI: https://doi.org/10.1177/2059513117728202
M. Spearpoint, C. Hopkin, and D. Hopkin, Modelling the thermal radiation from kitchen hob fires. Journal Fire Science, 38 (4), 377–394, 2020 DOI: https://doi.org/10.1177/0734904120923566
M.B. Hamida, and M.A. Hassanain, Fire safety in the built-environment: a case study in a residential facility, Architecture, Civil Engineering, Environment, 12 (2), 2019 DOI: https://doi.org/10.21307/acee-2019-020
H.B. Choi, E.H. Hwang, and D.M. Choi, Indoor air quality sensor utilization for unwanted fire alarm improvement in studio-type apartments. Fire, 6 (7), 2023 DOI: https://doi.org/10.3390/fire6070255
J. Milke, and R. Zevotek, Analysis of the response of smoke detectors to smoldering fires and nuisance sources. Fire Technology, 52 (5), 1235–1253, 2016 DOI: https://doi.org/10.1007/s10694-015-0465-2
J. Johnston, K. Zeng, and N. Wu, An evaluation and embedded hardware implementation of yolo for real-time wildfire detection. 2022 IEEE World AI IoT Congress, AIIoT 2022, 138–144, 2022 DOI: https://doi.org/10.1109/AIIoT54504.2022.9817206
L. An, L. Chen, and X. Hao, Indoor fire detection algorithm based on second-order exponential smoothing and information fusion. Information, 14 (5), 258. 2023 DOI: https://doi.org/10.3390/info14050258
E. Casas, L. Ramos, E. Bendek, and F. Rivas-Echeverria, YOLOv5 vs. YOLOv8: Performance benchmarking in wildfire and smoke detection scenarios. Journal of Image and Graphics, 12 (2), 127–136, 2024 DOI: https://doi.org/10.18178/joig.12.2.127-136
A. Rehman, D. Kim, and A. Paul, Convolutional neural network model for fire detection in real-time environment. Computers, Materials & Continua, 77 (2), 2289–2307, 2023 DOI: https://doi.org/10.32604/cmc.2023.036435
Y. Li, J. Shang, M. Yan, B. Ding, and J. Zhong, Real-time early indoor fire detection and localization on embedded platforms with fully convolutional one-stage object detection. Sustainability, 15 (3), 1794, 2023 DOI: https://doi.org/10.3390/su15031794
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, You Only Look Once: Unified, real-time object detection. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 779–788, 2016 DOI: https://doi.org/10.1109/CVPR.2016.91
J. Redmon, and A. Farhadi, YOLO9000: Better, faster, stronger. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 6517–6525, 2017 DOI: https://doi.org/10.1109/CVPR.2017.690
J. Redmon, and A. Farhadi, YOLOv3: An incremental improvement, 2018
A. Bochkovskiy, C.-Y. Wang, and H.-Y.M. Liao, YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934, 2020
G. Jocher, YOLOv5 by Ultralytics, 2020
C. Li, L. Li, H. Jiang, K. Weng, Y. Geng, L. Li, Z. Ke, Q. Li, M. Cheng, W.; Li, Y. Nie, B. Zhang, Y. Liang, L. Zhou, X. Xu, X. Chu, X. Wei, and X. Wei, YOLOv6: A single-stage object detection framework for industrial applications. arXiv preprint arXiv:2209.02976, 2022
C.-Y. Wang, A. Bochkovskiy, and H.-Y.M. Liao, YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 7464-7475), 2022 DOI: https://doi.org/10.1109/CVPR52729.2023.00721
C.-Y. Wang, I.-H. Yeh, and H.-Y.M. Liao, YOLOv9: Learning what you want to learn using programmable gradient information. arXiv preprint arXiv:2402.13616, 2024 DOI: https://doi.org/10.1007/978-3-031-72751-1_1
L. Zhao, L. Zhi, C. Zhao, W. Zheng, Fire-YOLO: A Small Target Object Detection Method for Fire Inspection. Sustainability, 14, 4930, 2022 DOI: https://doi.org/10.3390/su14094930
H. Wu, Y. Hu, X. Mei, and J. Xian, Ship fire detection based on an improved yolo algorithm with a lightweight convolutional neural network model. Sensors, 22(19):7420. 2022 DOI: https://doi.org/10.3390/s22197420
S. Wang, and X. Wang, ES-YOLO: A new lightweight fire detection model. Third International Conference on Computer Vision and Data Mining (ICCVDM 2022), 125111F, 2023 DOI: https://doi.org/10.1117/12.2660040
X. Wan, J. Cai, S. Luo, Z. Tian, L. Zhang and X. Xia, Gaussian Process for the Machine Learning-based Smart fire Detection System. 2022 IEEE 6th Information Technology and Mechatronics Engineering Conference (ITOEC), Chongqing, China, 100-104, 2022 DOI: https://doi.org/10.1109/ITOEC53115.2022.9734697
M. A. Rahman, S. T. Hasan and M. A. Kader, Computer Vision Based Industrial and Forest Fire Detection Using Support Vector Machine (SVM). International Conference on Innovations in Science, Engineering and Technology (ICISET), Chittagong, Bangladesh, 233-238, 2022 DOI: https://doi.org/10.1109/ICISET54810.2022.9775775
J. Zhang, J. Zhang, K. Zhou, Y. Zhang, H. Chen, and X. Yan, An improved yolov5-based underwater object-detection framework. Sensors, 23 (7), 3693, 2023 DOI: https://doi.org/10.3390/s23073693
M.L. Mekhalfi, C. Nicolo, Y. Bazi, M.M. Al. Rahhal, N.A. Alsharif, and E. Al. Maghayreh, Contrasting Yolov5, transformer, and efficientdet detectors for crop circle detection in desert. IEEE Geoscience and Remote Sensing Letters, 19, 2022 DOI: https://doi.org/10.1109/LGRS.2021.3085139
Z. Liu, X. Gao, Y. Wan, J. Wang, and H. Lyu, An improved Yolov5 method for small object detection in UAV capture scenes. IEEE Access, 11, 14365–14374, 2023 DOI: https://doi.org/10.1109/ACCESS.2023.3241005
W. Liu, K. Quijano, and M.M. Crawford, YOLOv5-Tassel: Detecting tassels in RGB UAV imagery with improved yolov5 based on transfer learning. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15, 8085–8094, 2022 DOI: https://doi.org/10.1109/JSTARS.2022.3206399
S. Wu, Z. Li, S. Li, Q. Liu, and W. Wu, Static gesture recognition algorithm based on improved yolov5s. Electronics (Basel), 12 (3), 596, 2023 DOI: https://doi.org/10.3390/electronics12030596
G. Jocher, A. Chaurasia, and J. Qiu, YOLO by Ultralytics, 2023
J.H. Kim, N. Kim, and C.S. Won, High-speed drone detection based on yolo-v8. ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2023-June, 2023 DOI: https://doi.org/10.1109/ICASSP49357.2023.10095516
A. Vats, and D.C. Anastasiu, Enhancing retail checkout through video inpainting, Yolov8 detection, and deepsort tracking. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 5530–5537, 2023 DOI: https://doi.org/10.1109/CVPRW59228.2023.00585
M. Hussain, H. Al-Aqrabi, M. Munawar, R. Hill, and T. Alsboui, (). Domain feature mapping with yolov7 for automated edge-based pallet racking inspections. Sensors, 22 (18), 6927, 2022 DOI: https://doi.org/10.3390/s22186927
G. Zeng, On the confusion matrix in credit scoring and its analytical properties. Communications in Statistics- Theory and Methods, 49 (9), 2080–2093, 2020 DOI: https://doi.org/10.1080/03610926.2019.1568485
Y. Zhang, T. Zuo, L. Fang, J. Li, and Z. Xing, An improved MAHAKIL oversampling method for imbalanced dataset classification. IEEE Access, 9, 16030–16040, 2021 DOI: https://doi.org/10.1109/ACCESS.2020.3047741
E. Casas, L. Ramos, E. Bendek, and F. Rivas-Echeverria, Assessing the effectiveness of yolo architectures for smoke and wildfire detection. IEEE Access, 11, 96554–96583, 2023 DOI: https://doi.org/10.1109/ACCESS.2023.3312217
M. Vergara, L. Ramos, N.D. Rivera-Campoverde, and F. Rivas-Echeverria, EngineFaultDB: A novel dataset for automotive engine fault classification and baseline results. IEEE Access, 11, 126155–126171, 2023 DOI: https://doi.org/10.1109/ACCESS.2023.3331316
R. Padilla, S.L. Netto, and E.A.B. da Silva, A survey on performance metrics for object-detection algorithms. 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), 237–242, 2020 DOI: https://doi.org/10.1109/IWSSIP48289.2020.9145130
R. Padilla, W.L. Passos, T.L.B. Dias, S.L. Netto, and E.A.B. da Silva, A comparative analysis of object detection metrics with a companion open-source toolkit. Electronics (Basel), 10 (3), 279, 2021 DOI: https://doi.org/10.3390/electronics10030279
Z. Ning, X. Wu, J. Yang, and Y. Yang, MT-YOLOv5: Mobile terminal table detection model based on YOLOv5. Journal of Physics: Conference Series, 1978 (1), 012010, 2021 DOI: https://doi.org/10.1088/1742-6596/1978/1/012010
H. Zhu, H. Wei, B. Li, X. Yuan, and N. Kehtarnavaz, A review of video object detection: datasets, metrics and methods. Applied Sciences, 10 (21), 7834, 2020 DOI: https://doi.org/10.3390/app10217834
Copyright (c) 2024 EMITTER International Journal of Engineering Technology
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The copyright to this article is transferred to Politeknik Elektronika Negeri Surabaya(PENS) if and when the article is accepted for publication. The undersigned hereby transfers any and all rights in and to the paper including without limitation all copyrights to PENS. The undersigned hereby represents and warrants that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. The undersigned represents that he/she has the power and authority to make and execute this assignment. The copyright transfer form can be downloaded here .
The corresponding author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors. This agreement is to be signed by at least one of the authors who have obtained the assent of the co-author(s) where applicable. After submission of this agreement signed by the corresponding author, changes of authorship or in the order of the authors listed will not be accepted.
Retained Rights/Terms and Conditions
- Authors retain all proprietary rights in any process, procedure, or article of manufacture described in the Work.
- Authors may reproduce or authorize others to reproduce the work or derivative works for the author’s personal use or company use, provided that the source and the copyright notice of Politeknik Elektronika Negeri Surabaya (PENS) publisher are indicated.
- Authors are allowed to use and reuse their articles under the same CC-BY-NC-SA license as third parties.
- Third-parties are allowed to share and adapt the publication work for all non-commercial purposes and if they remix, transform, or build upon the material, they must distribute under the same license as the original.
Plagiarism Check
To avoid plagiarism activities, the manuscript will be checked twice by the Editorial Board of the EMITTER International Journal of Engineering Technology (EMITTER Journal) using iThenticate Plagiarism Checker and the CrossCheck plagiarism screening service. The similarity score of a manuscript has should be less than 25%. The manuscript that plagiarizes another author’s work or author's own will be rejected by EMITTER Journal.
Authors are expected to comply with EMITTER Journal's plagiarism rules by downloading and signing the plagiarism declaration form here and resubmitting the form, along with the copyright transfer form via online submission.
