Human-machine Translation Model Evaluation Based on Artificial Intelligence Translation

  • Ruichao Li Universiti Teknologi Malaysia, Malaysia; Xi'an Fanyi University, China
  • Abdullah Mohd Nawi Universiti Teknologi Malaysia, Malaysia
  • Myoung Sook Kang Universiti Teknologi Malaysia, Malaysia
DOI: https://doi.org/10.24003/emitter.v11i2.812
Keywords: Artificial Intelligence, AI-based translation, attention mechanism, Statistical Machine Translation, translation model

Abstract

As artificial intelligence (AI) translation technology advances, big data, cloud computing, and emerging technologies have enhanced the progress of the data industry over the past several decades. Human-machine translation becomes a new interactive mode between humans and machines and plays an essential role in transmitting information. Nevertheless, several translation models have their drawbacks and limitations, such as error rates and inaccuracy, and they are not able to adapt to the various demands of different groups. Taking the AI-based translation model as the research object, this study conducted an analysis of attention mechanisms and relevant technical means, examined the setbacks of conventional translation models, and proposed an AI-based translation model that produced a clear and high quality translation and presented a reference to further perfect AI-based translation models. The values of the manual and automated evaluation have demonstrated that the human-machine translation model improved the mismatchings between texts and contexts and enhanced the accurate and efficient intelligent recognition and expressions. It is set to a score of 1-10 for evaluation comparison with 30 language users as participants, and the achieved 6 points or above is considered effective. The research results suggested that the language fluency score rose from 4.9667 for conventional Statistical Machine Translation to 6.6333 for the AI-based translation model. As a result, the human-machine translation model improved the efficiency, speed, precision, and accuracy of language input to a certain degree, strengthened the correlation between semantic characteristics and intelligent recognition, and pushed the advancement of intelligent recognition. It can provide accurate and high-quality translation for language users and achieve an understanding of natural language input and output and automatic processing.

Downloads

Download data is not yet available.

Author Biographies

Ruichao Li, Universiti Teknologi Malaysia, Malaysia; Xi'an Fanyi University, China

Ruichao Li is PhD in linguistics at Language Academy, Faculty of Social Sciences & Humanities, Universiti Teknologi Malaysia. He is also an associate professor at Xi’an Fanyi University. His research interests include translation technology, cultural translation, and translation teaching.

Abdullah Mohd Nawi, Universiti Teknologi Malaysia, Malaysia

Abdullah Mohd Nawi is a senior lecturer at Universiti Teknologi Malaysia. He is also the assistant dean for the Faculty of Social Sciences & Humanities. His research interests are drama in ESL/ELT, second language acquisition, and teacher training.

Myoung Sook Kang , Universiti Teknologi Malaysia, Malaysia

Myoung Sook Kang is a senior lecturer at Universiti Teknologi Malaysia. Her areas of academic interest include studies on the translation profession and teaching Korean as a foreign language.

References

P. Xiu and L. Xeauyin, Human translation vs machine translation: The practitioner phenomenology, Linguistics and Culture Review, vol. 2, no. 1, pp.13-23, 2018. DOI: https://doi.org/10.37028/lingcure.v2n1.8

A. Fan, S. Bhosale, H. Schwenk, Z. Ma, A. El-Kishky, et al. Beyond English-centric multilingual machine translation, The Journal of Machine Learning Research, vol. 22. no. 1, pp. 4839-4886, 2021.

S. Huang, Design and development of educational robot teaching resources using artificial intelligence technology, International Journal of Emerging Technologies in Learning (IJET), vol. 16, no. 05, p. 116, 2021. DOI: https://doi.org/10.3991/ijet.v16i05.20311

B. Yi and D. Mandal, English teaching practice based on AI technology, Journal of Intelligent and Fuzzy Systems, vol. 37, no. 1, pp. 1–11, 2019.

L. Yu and N. Peng, Research on English teaching reform based on artificial intelligence matching model, Journal of Intelligent and Fuzzy Systems, no. 1, pp. 1–10, 2021. DOI: https://doi.org/10.3233/JIFS-219131

J. Zhang, and C. Zong, Neural machine translation: Challenges, progress, and future, Science China Technological Sciences, vol. 63, no. 10, pp. 2028-2050, 2020. DOI: https://doi.org/10.1007/s11431-020-1632-x

P. Lohar, G. Xie, D. Gallagher, and A. Way, Building Neural Machine Translation Systems for Multilingual Participatory Spaces, Analytics, vol. 2, no. 2, pp. 393-409, 2023. DOI: https://doi.org/10.3390/analytics2020022

E. De Vries, M. Schoonvelde, and G. Schumacher, No longer lost in translation: Evidence that Google Translate works for comparative bag-of-words text applications, Political Analysis, vol. 26, no. 4, pp. 417-430, 2018. DOI: https://doi.org/10.1017/pan.2018.26

I. Rivera-Trigueros, Machine translation systems and quality assessment: a systematic review, Language Resources and Evaluation, vol. 56, no. 2, pp. 593-619, 2022. DOI: https://doi.org/10.1007/s10579-021-09537-5

J. Su, J. Zeng, D. Xiong, Y. Liu, M. Wang, and J. Xie. A hierarchy-to-sequence attentional neural machine translation model, IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no.3, pp. 623-632, 2018. DOI: https://doi.org/10.1109/TASLP.2018.2789721

B. Klimova, M.Pikhart, A. D. Benites, C. Lehr, and C. Sanchez-Stockhammer, Neural machine translation in foreign language teaching and learning: a systematic review, Education and Information Technologies, vol. 28, no.1, pp. 663-682, 2023. DOI: https://doi.org/10.1007/s10639-022-11194-2

S. Wu, D. Zhang, Z. Zhang, N. Yang, M. Li, and M. Zhou. Dependency-to-dependency neural machine translation, IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no.11, pp. 2132-2141, 2018. DOI: https://doi.org/10.1109/TASLP.2018.2855968

B. Zhang, De. Xiong, and J. Su, Neural machine translation with deep attention, IEEE Transactions on pattern analysis and machine intelligence, vol. 42, no.1, pp. 154-163, 2018. DOI: https://doi.org/10.1109/TPAMI.2018.2876404

S. Ranathunga, E. A. Lee, M. P. Skenduli, R. Shekhar, M. Alam, R. Kaur, Neural machine translation for low-resource languages: A survey, ACM Computing Surveys, vol. 55, no.11, pp. 1-37, 2023. DOI: https://doi.org/10.1145/3567592

S. Maruf, F. Saleh, and G. Haffari, A survey on document-level neural machine translation: methods and evaluation, ACM Computing Surveys (CSUR), vol. 54, no. 2, pp. 1-36, 2021. DOI: https://doi.org/10.1145/3441691

B. Yi and D. Mandal, English teaching practice based on AI technology, Journal of Intelligent and Fuzzy Systems, vol. 37, no. 1, pp. 1–11, 2019.

W. Che, Z. Yu, Z. Yu, Y. Wen, and J. Guo, Towards integrated classification lexicon for handling unknown words in Chinese-Vietnamese neural machine translation, ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), vo. 19, no. 3, pp. 1-17, 2020. DOI: https://doi.org/10.1145/3373267

S. Shi, X. Wu, R. Su, and H. Huang, Low-Resource Neural Machine Translation: Methods and Trends, ACM Transactions on Asian and Low-Resource Language Information Processing, vol. 21, no. 5, pp. 1-22, 2022. DOI: https://doi.org/10.1145/3524300

M. Sepesy and M. J. Brest, Slavic languages in phrase-based statistical machine translation: a survey, Artificial intelligence review, vol. 51, no.1, pp. 77-117, 2019. DOI: https://doi.org/10.1007/s10462-017-9558-2

S. K. Mahata and D. Das, S. Bandyopadhyay, Mtil2017: Machine translation using recurrent neural network on statistical machine translation, Journal of Intelligent Systems, vol. 28, no. 3, pp. 447-453, 2019. DOI: https://doi.org/10.1515/jisys-2018-0016

K. Ralph, Some Translation Studies informed suggestions for further balancing methodologies for machine translation quality evaluation, Translation Spaces, vol. 11, no. 2, pp. 213-233, 2022. DOI: https://doi.org/10.1075/ts.21026.kru

M. Kolhar and A. Abdalla, Artificial Intelligence Based Language Translation Platform, Intelligent Automation & Soft Computing, vol. 28, no. 1, pp.1-9, 2021. DOI: https://doi.org/10.32604/iasc.2021.014995

Z.A. Cahyaningtyas, D. Purwitasari, C. Fatichah, Deep Learning Approaches for Automatic Drum Transcription, EMITTER International Journal of Engineering Technology, vol. 11, no. 1, pp. 21-34, 2023.

V. H. Vu, Q. P. Nguyen, K. H. Nguyen, J. C. Shin, and C. Y. Ock, Korean-Vietnamese neural machine translation with named entity recognition and part-of-speech tags, IEICE TRANSACTIONS on Information and Systems, vol. 103, no. 4, pp. 866-873, 2020. DOI: https://doi.org/10.1587/transinf.2019EDP7154

M. Zheng, Research on Intelligent English Translation Methods Based on Improved Attention Mechanism Models, Electronic Technology, vol. 33, no. 11 pp. 84-87, 2020.

J. Li and F. Yang, Text level Machine translation based on joint attention mechanism, Chinese Journal of Information Technology, vol. 33, no.12, pp. 45-53, 2019.

Published
2023-12-20
How to Cite
Li, R., Mohd Nawi, A., & Kang , M. S. (2023). Human-machine Translation Model Evaluation Based on Artificial Intelligence Translation. EMITTER International Journal of Engineering Technology, 11(2), 145-159. https://doi.org/10.24003/emitter.v11i2.812
Issue
Vol 11 No 2 (2023)
Section
Articles

Copyright (c) 2023 EMITTER International Journal of Engineering Technology

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

The copyright to this article is transferred to Politeknik Elektronika Negeri Surabaya(PENS) if and when the article is accepted for publication. The undersigned hereby transfers any and all rights in and to the paper including without limitation all copyrights to PENS. The undersigned hereby represents and warrants that the paper is original and that he/she is the author of the paper, except for material that is clearly identified as to its original source, with permission notices from the copyright owners where required. The undersigned represents that he/she has the power and authority to make and execute this assignment. The copyright transfer form can be downloaded here .

The corresponding author signs for and accepts responsibility for releasing this material on behalf of any and all co-authors. This agreement is to be signed by at least one of the authors who have obtained the assent of the co-author(s) where applicable. After submission of this agreement signed by the corresponding author, changes of authorship or in the order of the authors listed will not be accepted.

Retained Rights/Terms and Conditions

  1. Authors retain all proprietary rights in any process, procedure, or article of manufacture described in the Work.
  2. Authors may reproduce or authorize others to reproduce the work or derivative works for the author’s personal use or company use, provided that the source and the copyright notice of Politeknik Elektronika Negeri Surabaya (PENS) publisher are indicated.
  3. Authors are allowed to use and reuse their articles under the same CC-BY-NC-SA license as third parties.
  4. Third-parties are allowed to share and adapt the publication work for all non-commercial purposes and if they remix, transform, or build upon the material, they must distribute under the same license as the original.

Plagiarism Check

To avoid plagiarism activities, the manuscript will be checked twice by the Editorial Board of the EMITTER International Journal of Engineering Technology (EMITTER Journal) using iThenticate Plagiarism Checker and the CrossCheck plagiarism screening service. The similarity score of a manuscript has should be less than 25%. The manuscript that plagiarizes another author’s work or author's own will be rejected by EMITTER Journal.

Authors are expected to comply with EMITTER Journal's plagiarism rules by downloading and signing the plagiarism declaration form here and resubmitting the form, along with the copyright transfer form via online submission.