Visual Similarity Detection for Intellectual Property using Deep Transfer Learning
Abstract
Trademarks examination can benefit from deep transfer learning. Utilizing pretrained models to extract image features can significantly improve the trademarks registration process. This approach can facilitate and accelerate image detection. This study aims to enhance the trademark similarity examination process by detecting marks’ visual similarities using deep transfer learning. Deep transfer learning has the potential to develop the registration process of trademarks through the implementation of an automated image detection system, which can enhance detection accuracy. To the best of our knowledge, no automated approach has been used locally to determine the similarities between local trademarks. This study proposes an image similarity detection system to make the trademark examination process more efficient and assist examiners in their decision-making. The proposed system was validated using a dataset provided by the Saudi authority for intellectual property (SAIP). To extract the features, we employed a residual network-based convolutional neural network model (ResNet-50). Then principal component analysis (PCA) was used to reduce the number of extracted features. The proposed system reached a mean average precision (MAP) of 0.774, which indicates a promising result in distinguishing the similarity of trademarks. The findings of this research suggested that an image similarity detection system can support decision-making in trademark examination contexts. Trademark examiners, legal professionals, and intellectual property offices can use the results of this research to enhance their evaluation processes and improve the accuracy and efficiency of trademark registration.
Downloads
References
S. Russell and P. Norvig, Artificial Intelligence A Modern Approach, Pearson (Hoboken), Ed. 4, 2020.
S. Albawi, T. A. Mohammed, and S. Al-Zawi, Understanding of a convolutional neural network, Proceedings of the 2017 International Conference on Engineering and Technology (ICET), Antalya, pp. 1-6, 2017.
X. Gu et al., EEG based Brain-Computer Interfaces (BCIs): A Survey of Recent Studies on Signal Sensing Technologies and Computational Intelligence Approaches and their Applications, Signal Processing and Machine Learning for Brain-Machine Interfaces, pp. 1-22, 2020.
F. Mohd Anuar, R. Setchi, and Y. K. Lai, Trademark image retrieval using an integrated shape descriptor, Expert Systems with Applications, Vol. 40, No. 1, pp. 105-121, 2013.
C. V. Trappey, A. J. C. Trappey, and B. H. Liu, Identify trademark legal case precedents - Using machine learning to enable semantic analysis of judgments, World Patent Information, Vol. 62, pp. 101980, 2020.
S. Y. Arafat, M. Saleem, and S. A. Hussain, Comparative analysis of invariant schemes for logo classification, Proceedings of the 2009 International Conference on Emerging Technologies, Islamabad, pp. 256-261, 2009.
C. V. Trappey, A. J. C. Trappey, and S. C. C. Lin, Intelligent trademark similarity analysis of image, spelling, and phonetic features using machine learning methodologies, Advanced Engineering Informatics, Vol. 45, pp. 101120, 2020.
W. Macías and J. Cerviño, Trademark dilution and its practical effect on purchase decision, Spanish Journal of Marketing - ESIC, Vol. 21, No. 1, pp. 1-13, 2017.
Y. LeCun, Y. Bengio, and G. Hinton, Deep learning, Nature, Vol. 521, No. 7553, pp. 436-444, 2015.
C. C. Aggarwal, Neural Networks and Deep Learning, Springer International Publishing (Cham), Ed. 1, 2023.
G. Kumar and P. K. Bhatia, A Detailed Review of Feature Extraction in Image Processing Systems, Proceedings of the 2014 Fourth International Conference on Advanced Computing Communication Technologies, Rohtak, pp. 5-12, 2014.
N. Singh and H. Sabrol, Convolutional Neural Networks: An Extensive arena of Deep Learning. A Comprehensive Study, Archives of Computational Methods in Engineering, Vol. 28, No. 7, pp. 4755-4780, 2021.
S. Russell and P. Norvig, Artificial Intelligence A Modern Approach, Pearson Education, Inc. (Hoboken), Ed. 4, 2010.
O. Russakovsky et al., ImageNet Large Scale Visual Recognition Challenge, International Journal of Computer Vision (IJCV), Vol. 115, No. 3, pp. 211-252, 2015.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, ImageNet Classification with Deep Convolutional Neural Networks, Advances in Neural Information Processing Systems, Vol. 25, pp. 1097-1105, 2012.
K. Simonyan and A. Zisserman, Very deep convolutional networks for large-scale image recognition, Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015), San Diego, pp. 1-14, 2014.
K. He, X. Zhang, R. S. Ren, and J. Sun, Deep Residual Learning for Image Recognition, Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, pp. 770-778, 2016.
A. J. C. Trappey, C. V. Trappey, and S. C. C. Lin, Detecting Trademark Image Infringement Using Convolutional Neural Networks, Proceedings of the 26th ISTE International Conference on Transdisciplinary Engineering, Tokyo, pp. 477-486, 2019.
C. S. Chen and C. M. Weng, An Efficient Retrieval Technique for Trademarks Based on the Fuzzy Inference System, Applied Sciences, Vol. 7, No. 8, pp. 849, 2017.
O. Tursun et al., Component-Based Attention for Large-Scale Trademark Retrieval, IEEE Transactions on Information Forensics and Security, Vol. 17, pp. 2350-2363, 2022.
K. Abe, H. Morita, and T. Hayashi, Similarity Retrieval of Trademark Images by Vector Graphics Based on Shape Characteristics of Components, Proceedings of the 2018 10th International Conference on Computer and Automation Engineering, New York, pp. 826, 2018.
T. Lan, X. Feng, Z. Xia, S. Pan, and J. Peng, Similar Trademark Image Retrieval Integrating LBP and Convolutional Neural Network, Proceedings of the 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), Shenzhen, pp. 231-242, 2017.
G. Showkatramani, S. Nareddi, C. Doninger, G. Gabel, and A. Krishna, Trademark image similarity search, Communications in Computer and Information Science, Vol. 850, pp. 199-205, 2018.
T. Lan, X. Feng, L. Li, and Z. Xia, Similar Trademark Image Retrieval Based on Convolutional Neural Network and Constraint Theory, Proceedings of the 2018 Eighth International Conference on Image Processing Theory, Tools and Applications (IPTA), Xi'an, pp. 1-6, 2018.
S. Khamkar, User Retrieval of Trademarks System Using Conceptual Similarity Approach, HELIX, Vol. 8, No. 5, pp. 3754-3758, 2018.
I. Mosseri, M. Rusanovsky, and G. Oren, TradeMarker - Artificial Intelligence Based Trademarks Similarity Search Engine, Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Long Beach, pp. 97-105, 2019.
S. B. K. Aires, C. O. A. Freitas, and M. L. Sguario, Dynamic Selection Feature Extractor for Trademark Retrieval, Computational Science and Its Applications – ICCSA 2018, pp. 219-231, 2018.
C. A. Perez et al., Trademark Image Retrieval Using a Combination of Deep Convolutional Neural Networks, Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, pp. 1-7, 2018.
A. K. Bhunia, A. K. Bhunia, S. Ghose, A. Das, P. P. Roy, and U. Pal, A deep one-shot network for query-based logo retrieval, Pattern Recognition, Vol. 96, pp. 106965, 2019.
H. M. Y, S. A, and Jaber, Trademark image recognition utilizing deep convolutional neural network, Annals of Electrical and Electronic Engineering, Vol. 2, No. 3, pp. 13-20, 2019.
Z. Xia, J. Lin, and X. Feng, Trademark image retrieval via transformation-invariant deep hashing, Journal of Visual Communication and Image Representation, Vol. 59, pp. 108-116, 2019.
S. Jardim, J. António, C. Mora, and A. Almeida, A Novel Trademark Image Retrieval System Based on Multi-Feature Extraction and Deep Networks, Journal of Imaging, Vol. 8, No. 9, pp. 238, 2022.
J. Wan and Y. Xiaobo, Intelligent Retrieval Method of Approximate Painting in Digital Art Field, Scientific Programming, Vol. 2021, pp. 1-8, 2021.
J. Cao, Y. Huang, Q. Dai, and W. K. Ling, Unsupervised Trademark Retrieval Method Based on Attention Mechanism, Sensors, Vol. 21, No. 5, pp. 1894, 2021.
World Intellectual Property Organization, Nice Agreement Concerning the International Classification of Goods and Services for the Purposes of the Registration of Marks, WIPO (Geneva), 2009.
H. Alshowaish, Y. Al-Ohali, and A. Al-Nafjan, Trademark Image Similarity Detection Using Convolutional Neural Network, Applied Sciences, Vol. 12, No. 3, 2022.
S. Gkelios, A. Sophokleous, S. Plakias, Y. Boutalis, and S. A. Chatzichristofis, Deep convolutional features for image retrieval, Expert Syst. Appl., Vol. 177, pp. 114940, 2021.
O. Tursun and S. Kalkan, METU dataset: A big dataset for benchmarking trademark retrieval, Proceedings of the 2015 14th IAPR International Conference on Machine Vision Applications (MVA), Tokyo, pp. 514-517, 2015.
Saudi Authority for Intellectual Property (SAIP), About the Saudi Authority for Intellectual Property, 2022.
J. M. Czum, Dive Into Deep Learning, Journal of the American College of Radiology, Vol. 17, No. 5, pp. 637-638, 2020.
A. Tharwat, Principal component analysis - a tutorial, International Journal of Applied Pattern Recognition, Vol. 3, No. 3, pp. 197, 2016.
L. Deng, Deep Learning: Methods and Applications, Foundations and Trends in Signal Processing, Vol. 7, No. 3-4, pp. 197-387, 2014.
J. K. Min, R. T. Ng, and K. Shim, Efficient Aggregation Processing in the Presence of Duplicately Detected Objects in WSNs, Journal of Sensors, Vol. 2019, pp. 1-15, 2019.
Y. Feng, C. Shi, C. Qi, J. Xu, B. Xiao, and C. Wang, Aggregation of reversal invariant features from edge images for large-scale trademark retrieval, Proceedings of the 2018 4th International Conference on Control, Automation and Robotics (ICCAR), Auckland, pp. 384-388, 2018.
Copyright (c) 2025 EMITTER International Journal of Engineering Technology
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
The copyright to this article is transferred to Politeknik Elektronika Negeri Surabaya(PENS) if and when the article is accepted for publication. The undersigned hereby transfers any and all rights in and to the paper including without limitation all copyrights to PENS. The undersigned hereby represents and warrants that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. The undersigned represents that he/she has the power and authority to make and execute this assignment. The copyright transfer form can be downloaded here .
The corresponding author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors. This agreement is to be signed by at least one of the authors who have obtained the assent of the co-author(s) where applicable. After submission of this agreement signed by the corresponding author, changes of authorship or in the order of the authors listed will not be accepted.
Retained Rights/Terms and Conditions
- Authors retain all proprietary rights in any process, procedure, or article of manufacture described in the Work.
- Authors may reproduce or authorize others to reproduce the work or derivative works for the author’s personal use or company use, provided that the source and the copyright notice of Politeknik Elektronika Negeri Surabaya (PENS) publisher are indicated.
- Authors are allowed to use and reuse their articles under the same CC-BY-NC-SA license as third parties.
- Third-parties are allowed to share and adapt the publication work for all non-commercial purposes and if they remix, transform, or build upon the material, they must distribute under the same license as the original.
Plagiarism Check
To avoid plagiarism activities, the manuscript will be checked twice by the Editorial Board of the EMITTER International Journal of Engineering Technology (EMITTER Journal) using iThenticate Plagiarism Checker and the CrossCheck plagiarism screening service. The similarity score of a manuscript has should be less than 25%. The manuscript that plagiarizes another author’s work or author's own will be rejected by EMITTER Journal.
Authors are expected to comply with EMITTER Journal's plagiarism rules by downloading and signing the plagiarism declaration form here and resubmitting the form, along with the copyright transfer form via online submission.
