• Home
  • Search
  • HapticSOUND: An Interactive Learning Experience with a Digital Musical Instrument
  • Cite Icon7
  • https://doi.org/10.3390/app13127149Copy DOI Icon

HapticSOUND: An Interactive Learning Experience with a Digital Musical Instrument

Show More
  • Abstract
  • Highlights & Summary
  • PDF
  • Literature Map
  • References
  • Citations
  • Similar Papers
Abstract

In this paper, an interactive learning experience is proposed, aiming to involve museum visitors in a personalized experience of the transmittal of cultural knowledge in an active and creative way. The proposed system, called HapticSOUND, consists of three subsystems: (a) the Information, where visitors are informed about the traditional musical instruments; (b) the Entertainment, where visitors are entertained by playing serious games to virtually assemble traditional musical instruments by a set of 3D objects; and (c) the Interaction, where visitors interact with a digital musical instrument which is an exact 3D-printed replica of a traditional musical instrument, where cameras have been placed to capture user gestures and machine learning algorithms have been implemented for gesture recognition. The museum visitor can interact with the lifelike replica to tactilely and aurally explore the instrument’s abilities, producing sounds guided by the system and receiving real-time visual and audio feedback. Emphasis is given to the Interaction Subsystem, where a pilot study was conducted to evaluate the usability of the subsystem. Preliminary results were promising since the usability was satisfactory, indicating that it is an innovative approach that utilizes sensorimotor learning and machine learning techniques in the context of playing sounds based on real-time gesture and fingering recognition.

Loading PDF

Similar Papers
  • Research Article
  • Citations1

AUGMENTED REALITY BASED IMAGE TRACKING FOR INTRODUCE PUPPET SHADOW TRADITIONAL MUSICAL INSTRUMENT

  • Feb 10, 2023
  • Jurnal Teknik Informatika (Jutif)
  • Krisna Subarkah +2
  • Research Article
  • Citations7

Digital Musical Instruments in Special Educational Needs Schools: Requirements from the Music Teachers’ Perspective and the Status Quo in Germany

  • Sep 21, 2023
  • ACM Transactions on Accessible Computing
  • Andreas Förster +1
  • PDF
  • Research Article
  • Citations1

Designing Android Based Game to Educate The Central Java Traditional Music Instrument for Children

  • Nov 23, 2020
  • SISFORMA
  • Helarius Panji Aruna +2
  • Research Article

Preference Evaluation of Reverberation Times for Traditional Inner Mongolian Musical Instruments in Performance Spaces

  • Jan 13, 2026
  • Buildings
  • Xiaoyun Yue +5
  • Conference Article
  • Citations6

Construction of basic concepts of waves through a “gambo” (traditional musical instrument)

  • Jan 01, 2018
  • AIP conference proceedings
  • Khairil Anwar +3
  • Research Article
  • Citations5

Applications of the phase vocoder in the control of real‐time electronic musical instruments

  • Aug 01, 1993
  • Interface
  • Nicholas J Bailey +3
  • Research Article
  • Citations1

THE MEANING OF KELAMBUT MUSIC FOR THE SENTANI TRIBE, PAPUA

  • Jan 01, 2023
  • Journal of Southwest Jiaotong University
  • I Wayan +1
  • Supplementary Content
  • Citations1

Real Time Gesture Learning And Recognition : Towards Automatic Categorization

  • Jun 01, 2008
  • Surrey Research Insight Open Access (The University of Surrey)
  • Jean-Baptiste Thiébaut +4
  • Research Article

Contemporary technology in the production of ‘Ariwa’, and ‘Igede-Eze’ traditional musical instruments of Edem Community in Nsukka L.G.A. of Enugu State, Nigeria

  • Jul 23, 2025
  • Journal of the Association of Nigerian Musicologists
  • Emelda Chinasa Nnanyelugo +1
  • Research Article

About the meaning of the terms autochthonous and authentic in relation to the term «traditional musical instrument»

  • Jan 01, 2019
  • Scientific collections of the Lviv National Music Academy named after M.V. Lysenko
  • Liliia Pasichniak
  • Research Article

MENYULAM NUSANTARA: EKSPLORASI ALAT MUSIK RINDING/KNOBE OH KABUPATEN TTS PADA PEMBELAJARAN BAHASA INDONESIA

  • Feb 28, 2025
  • Jurnal Pendidikan Dasar Borneo (Judikdas Borneo)
  • Maria Dolorosa Fransiska Tefa +2
  • Research Article

News

  • Jun 01, 2021
  • Computer Music Journal
  • PDF
  • Research Article
  • Citations22

The EyeHarp: A Gaze-Controlled Digital Musical Instrument

  • Jun 21, 2016
  • Frontiers in Psychology
  • Zacharias Vamvakousis +1
  • Research Article
  • Citations11

Exploring the motivations for building new digital musical instruments

  • Oct 16, 2018
  • Musicae Scientiae
  • Gina Emerson +1
  • Research Article

Comprehensive dataset of Portuguese folk instruments for computer vision and heritage research

  • May 31, 2025
  • Data in Brief
  • Luis Augusto Silva Zendron +4
Cactus Communications logo

Copyright 2026 Cactus Communications. All rights reserved.