IRAWNET: A Method for Transcribing Indonesian Classical Music Notes Directly from Multichannel Raw Audio

Keywords: Indonesian Classical Music, Gamelan Notes Transcription, Music Signal Processing, Multi-Channel Raw Audio, Deep Learning, IRawNet


A challenging task when developing real-time Automatic Music Transcription (AMT) methods is directly leveraging inputs from multichannel raw audio without any handcrafted signal transformation and feature extraction steps. The crucial problems are that raw audio only contains an amplitude in each timestamp, and the signals of the left and right channels have different amplitude intensities and onset times. Thus, this study addressed these issues by proposing the IRawNet method with fused feature layers to merge different amplitude from multichannel raw audio. IRawNet aims to transcribe Indonesian classical music notes. It was validated with the Gamelan music dataset. The Synthetic Minority Oversampling Technique (SMOTE) overcame the class imbalance of the Gamelan music dataset. Under various experimental scenarios, the performance effects of oversampled data, hyperparameters tuning, and fused feature layers are analyzed. Furthermore, the performance of the proposed method was compared with Temporal Convolutional Network (TCN), Deep WaveNet, and the monochannel IRawNet. The results proved that proposed method almost achieves superior results in entire metric performances with 0.871 of accuracy, 0.988 of AUC, 0.927 of precision, 0.896 of recall, and 0.896 of F1 score.


Download data is not yet available.


E. Benetos, S. Dixon, Z. Duan, and S. Ewert, Automatic music transcription: An overview, IEEE Signal Process. Mag., vol. 36, no. 1, pp. 20–30, 2018. DOI:

A. van den Oord et al., {WaveNet}: A Generative Model for Raw Audio, no. {arXiv}:1609.03499. 2016. Accessed: Jul. 15, 2022. [Online]. Available:

S. Bai, J. Z. Kolter, and V. Koltun, An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling, ArXiv, vol. abs/1803.0, 2018.

E. P. MatthewDavies and S. Böck, Temporal convolutional networks for musical audio beat tracking, in 2019 27th European Signal Processing Conference (EUSIPCO), pp. 1–5, 2019. DOI:

L. S. Martak, M. Sajgalik, and W. Benesova, Polyphonic note transcription of time-domain audio signal with deep wavenet architecture, in 2018 25th International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 1–5, 2018. DOI:

S. L. Oh et al., Classification of heart sound signals using a novel deep WaveNet model, Comput. Methods Programs Biomed., vol. 196, pp. 105604, 2020. DOI:

L. Chen, M. Yu, D. Su, and D. Yu, Multi-band pit and model integration for improved multi-channel speech separation, in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 705–709, 2019. DOI:

S. Gul, M. S. Khan, and S. W. Shah, Integration of deep learning with expectation maximization for spatial cue-based speech separation in reverberant conditions, Appl. Acoust., vol. 179, pp. 108048, 2021. DOI:

W. Zhang, Y. Zhang, Y. She, and J. Shao, Stereo feature enhancement and temporal information extraction network for automatic music transcription, IEEE Signal Process. Lett., vol. 28, pp. 1500–1504, 2021. DOI:

S. Kristiawan, THE GAMELAN AND ITS IMPACT ON DEBUSSY’S PAGODES, Mahasara-Journal Interdiscip. Music Stud., vol. 1, no. 1, pp. 24–32, 2021.

J. Becker and A. H. Feinstein, Karawitan: Source Readings in Javanese Gamelan and Vocal Music, Volume 1. University of Michigan Press, 2020. DOI:

K. Tanaka, T. Nakatsuka, R. Nishikimi, K. Yoshii, and S. Morishima, Multi-Instrument Music Transcription Based on Deep Spherical Clustering of Spectrograms and Pitchgrams., in ISMIR, pp. 327–334, 2020.

A. Huaysrijan and S. Pongpinigpinyo, Deep Convolution Neural Network for Thai Classical Music Instruments Sound Recognition, in 2021 25th International Computer Science and Engineering Conference (ICSEC), pp. 283–288, 2021. DOI:

V. S. Pendyala, N. Yadav, C. Kulkarni, and L. Vadlamudi, Towards building a Deep Learning based Automated Indian Classical Music Tutor for the Masses, Syst. Soft Comput., vol. 4, pp. 200042, 2022. DOI:

C. Hawthorne, I. Simon, R. Swavely, E. Manilow, and J. Engel, Sequence-to-sequence piano transcription with Transformers, arXiv Prepr. arXiv2107.09142, 2021.

B. Bahmei, E. Birmingham, and S. Arzanpour, CNN-RNN and Data Augmentation Using Deep Convolutional Generative Adversarial Network for Environmental Sound Classification, IEEE Signal Process. Lett., vol. 29, pp. 682–686, 2022. DOI:

H.-S. Choi, J. Lee, and K. Lee, Spec2Spec: Towards the general framework of music processing using generative adversarial networks, Acoust. Sci. Technol., vol. 41, no. 1, pp. 160–165, 2020. DOI:

A. Ycart and E. Benetos, Learning and Evaluation Methodologies for Polyphonic Music Sequence Prediction With LSTMs, IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 28, pp. 1328–1341, 2020. DOI:

Q. Wang, R. Zhou, and Y. Yan, Polyphonic piano transcription with a note-based music language model, Appl. Sci., vol. 8, no. 3, pp. 470, 2018. DOI:

A. K. Sharma et al., Classification of Indian classical music with time-series matching deep learning approach, IEEE Access, vol. 9, pp. 102041–102052, 2021. DOI:

A. Sadekar and S. P. Mahajan, Polyphonic Piano Music Transcription using Long Short-Term Memory, in 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), pp. 1–7, 2019. DOI:

S. Shahriar and U. Tariq, Classifying maqams of Qur’anic recitations using deep learning, IEEE Access, vol. 9, pp. 117271–117281, 2021. DOI:

M. A. Román, A. Pertusa, and J. Calvo-Zaragoza, Data representations for audio-to-score monophonic music transcription, Expert Syst. Appl., vol. 162, pp. 113769, 2020. DOI:

D. Nurdiyah, Y. K. Suprapto, and E. M. Yuniarno, Gamelan Orchestra Transcription Using Neural Network, in 2020 International Conference on Computer Engineering, Network, and Intelligent Multimedia (CENIM), pp. 371–376, Nov. 2020. DOI:

D. Nurdiyah et al, September 12, 2023, Gamelan Music Dataset, Zenodo repository, doi: 10.5281/zenodo.8333916

A. Arafa, N. El-Fishawy, M. Badawy, and M. Radad, RN-SMOTE: Reduced noise smote based on DBSCAN for enhancing imbalanced data classification, J. King Saud Univ. Inf. Sci., vol. 34, no. 8, pp. 5059–5074, 2022. DOI:

H. Gao, H. Yuan, Z. Wang, and S. Ji, Pixel Transposed Convolutional Networks, IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 5, pp. 1218–1227, 2020.

L. Cheng, R. Khalitov, T. Yu, J. Zhang, and Z. Yang, Classification of long sequential data using circular dilated convolutional neural networks, Neurocomputing, vol. 518, pp. 50–59, 2023. DOI:

M. Segu, A. Tonioni, and F. Tombari, Batch normalization embeddings for deep domain generalization, Pattern Recognit., vol. 135, pp. 109115, 2023. DOI:

Y.-N. Hung and A. Lerch, Multitask learning for instrument activation aware music source separation, arXiv Prepr. arXiv2008.00616, 2020.

A. K. Sharma et al., Polyphonic note transcription of time-domain audio signal with deep wavenet architecture, IEEE Access, vol. 28, no. 1, pp. 1–5, 2020.

How to Cite
Dewi Nurdiyah, Eko Mulyanto Yuniarno, Yoyon Kusnendar Suprapto, & Mauridhi Hery Purnomo. (2023). IRAWNET: A Method for Transcribing Indonesian Classical Music Notes Directly from Multichannel Raw Audio. EMITTER International Journal of Engineering Technology, 11(2), 246-264.

Funding data