RESEARCH
Musical Dynaformics
Musical Dynaformics / ダイナフォーミックス
Sustainable Development of Art & Culture
NeuroPiano is an organization that aims to create a society where all music lovers can create rich and diverse musical expressions. It is essential not only to have the ability to read sheet music with knowledge and aesthetics, but also to be able to embody our musical creativity through freely controlling the brain, mind, and body, without suffering from any physical and mental constraints. We therefore propose a new academic discipline called “Musical Dynaformics” to create synergies between music and physical education. Musical Dynaformics enhances sensory-motor-cognitive functions and skills of their mind and body along with nurturing artistic aesthetics and interpretations in musicians. We work together with musicians and support their artistic and physical development by establishing and developing research and education foundations. We apply the knowledge gained through our cutting-edge research directly to musical performance and teaching through our educational program, so called “PEAC”.
The new academic field “Musical Dynaformics” and the Music Excellence Project, whose aim is to contribute to the future of musicians by building upon research and development, were launched by Shinichi Furuya, a researcher at Sony Computer Science Laboratories, Inc.. The nonprofit organization NeuroPiano is dedicated to further promoting scientific research in these areas and engaging in social and educational contribution and outreach activities based on the research outcomes.
Sony Computer Science Laboratories, Inc. / Musical Dynaformics
新たな学術領域である Musical Dynaformics(ダイナフォーミックス)および、この研究開発活動とその成果を基盤としてアーティストに還元するプロジェクト Music Excellence Project は、株式会社ソニーコンピュータサイエンス研究所の古屋 晋一 研究員により発足されました。一般社団法人NeuroPiano では、これらの科学研究を更に推進すると共に、その成果の社会還元と普及活動に従事しています。
株式会社ソニーコンピュータサイエンス研究所
ダイナフォーミックス(外部サイト)

Piano performance is an extremely difficult motor skill that requires skillful manipulation of 88 keys and multiple pedals. Mastering how to use the body to produce diverse music expressions that one envisions requires pianists to spend an enormous amount of time practicing from a young age. However, many pianists suffer from injuries such as tendinitis and focal dystonia during this process. To enable pianists to continue exploring their musical expressions throughout their lifetime, it is essential to clarify the optimal ways of moving the body as the foundation for creating musical expressions, to understand the mechanisms by which the nervous system acquires dexterous skills, and to establish the most effective training based on evidence. We conduct research that targets the neural mechanisms underlying the specialized sensory and motor skills of pianists by integrating our original multi-modal sensing system that simultaneously records the movements of the keys and pedals with multiple video cameras, audio, and skeletal information on the body posture, with non-invasive techniques such as electroencephalography (EEG), transcranial magnetic stimulation (TMS), and functional magnetic resonance imaging (fMRI). The research findings will be applied to supporting for pianists to acquire optimal performance techniques based on the understanding of the human body and to developing evidence-based training.
JSPS KAKENHI
FY2023~FY2025 Grant-in-Aid for Scientific Research (B), “Expert-specific somatosensory inhibitory functions that facilitate the execution and perception of complex movements.” 23H03300
FY2020~FY2022 Grant-in-Aid for Transformative Research Areas (B), “Surmounting the limit of psycho-motor skills of expert pianists” 20H05716
FY2020~FY2021 Grant-in-Aid for Challenging Research (Exploratory), “Assessment and communication of musical virtuosity” 20K21835
FY2019~FY2021 Grant-in-Aid for Scientific Research (B), “Elucidating individual differences in sensorimotor skills of pianists” 19H04020
FY2019~FY2020 Grant-in-Aid for Challenging Research (Exploratory), “Quantitative analysis of expression design in piano performance” (Research director::Prof. Haruhiro Katayose, Kwansei Gakuin University) 19K21622
FY2026~FY2028 Grant-in-Aid for Scientific Research (A),“Elucidating the Formation Process of Motor Memory Through Passive Movement” 26H02542
FY2024~FY2025 Grant-in-Aid for Challenging Research (Exploratory),“Catalyst effects of sensorimotor training on motor learning” 24K22263
Private grant
FY2021~FY2022 Tateisi Science and Technology Foundation, Research Grant (C), “Development and evaluation of neuromuscular training to promote expert hand fine motor skills”
FY2018 Tateisi Science and Technology Foundation, Research Grant (A), “Mechanisms of sensory-motor integration function based on a novel measurement system”
Japan Science and Technology Agency (JST)
FY2019 JST CREST AIP Challenge program, “3D markerless posture estimation of fingers in skillful playing movements using deep learning”
FY2025~FY2030 JST ASPIRE “Establishing an International Research Consortium for the Transmission, Breakthrough, and Scientific Understanding of Expertise” JPMJAP2503
FY2024~FY2030 JST FOREST “Control and Understanding of ‘Sensory Interference’ in High-Speed, Multi-Degree-of-Freedom Motion” JPMJFR234U
Achievement
Erwin Wu, Hayato Nishioka, Shinichi Furuya and Hideki Koike (2023) Marker-removal Networks to Collect Precise 3D Hand Data for RGB-based Estimation and its Application in Piano. IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
Masato Hirano, Shinichi Furuya (2022) Multisensory interactions on auditory and somatosensory information in expert pianists. Scientific Reports 12: 12503
Takanori Oku, Shinichi Furuya (2022) Noncontact and high-precision sensing system for piano keys identified fingerprints of virtuosity. Sensors 22(13): 4891
Yudai Kimoto, Masato Hirano, Shinichi Furuya (2022) Adaptation of the corticomuscular and biomechanical systems of pianists. Cerebral Cortex 32(4): 709-724
Shinichi Furuya, Takanori Oku, Yudai Kimoto, Hayato Nishioka, Masato Hirano (2020) Skillful and pathological movement coordination in musical performance. Advances in Exercise & Sports Physiology 26(2): 23-26
Masato Hirano, Yudai Kimoto, Shinichi Furuya (2020) Specialized somatosensory-motor integration functions in musicians. Cerebral Cortex 30(3): 1148-1158
Yudai Kimoto, Takanori Oku, Shinichi Furuya (2019) Neuromuscular and biomechanical functions subserving finger dexterity in musicians. Scientific Reports 9: 12224
Shinichi Furuya, Sayuri Yokota (2018) Temporal exploration of sequential movements in sequential movements shapes efficient neuromuscular control. Journal of Neurophysiology 120(1): 196-210
Shinichi Furuya, Yuta Furukawa, Kazumasa Uehara, Takanori Oku (2018) Probing sensory-motor integration during musical performance. Annals of the New York Academy of Science 1423(1): 211-218
Shinichi Furuya (2018) Individual differences in sensorimotor skills among musicians. Current Opinion in Behavioral Science 20: 61-66
Vincent K. M. Cheung, Tamaka Harada, Shu Sakamoto and Shinichi Furuya (2025) Time-warped representational similarity analysis reveals acoustic contributions to musical pleasure are weakly shaped by autonomic neural inputs. iScience 28(12): 114072.
Thomas S. Binns, Shinichi Furuya and Vincent K. M. Cheung (2025) A real-time multimodal system for music preference decoding combining EEG and acoustic features. Extended Abstracts for the Late-Breaking Demo Session of the 26th International Society for Music Information Retrieval Conference.
Masato Hirano, Yudai Kimoto, Sachiko Shiotani and Shinichi Furuya (2025) Enhanced somatosensory inhibition sharpens hand representation and sensorimotor skills in pianists. The Journal of Neuroscience 45(8): e1486242024.
Masaki Yasuhara, Kazumasa Uehara, Takanori Oku, Sachiko Shiotani, Isao Nambu and Shinichi Furuya (2024) Robustness and adaptability of sensorimotor skills in expert piano performance. iScience 27(8): 110400.
Masato Hirano and Shinichi Furuya (2024) Active perceptual learning involves motor exploration and adaptation of predictive sensory integration. iScience 27(1): 108604.
Kazumasa Uehara, Masaki Yasuhara, Junya Koguchi, Takanori Oku, Sachiko Shiotani, Masanori Morise and Shinichi Furuya (2023) Brain network flexibility as a predictor of skilled musical performance. Cerebral Cortex 33(20): 10492-10503.
Vincent K. M. Cheung, Lana Okuma, Kazuhisa Shibata, Kosetsu Tsukuda, Masataka Goto and Shinichi Furuya (2023) Decoding drums, instrumentals, vocals, and mixed sources in music using human brain activity with fMRI. Proceedings of the 24th International Society for Music Information Retrieval Conference: 197-206.
In the field of sports, cutting-edge training methods utilizing technology have been put into practical use and many top athletes are taking advantage of them. Despite the fact that piano performance requires dexterous body control that is no less challenging than in sports, such training methods are currently rarely utilized. We are working to identify the optimal use of technology to further improve the skills of pianists and develop and validate state-of-the-art training methods. For example, we aim to establish training methods that are optimal for piano performance from the viewpoint of technology and science, such as robotic training using hand exoskeletons, haptic training that enhances tactile and muscle sense abilities at the fingertips, auditory training that enhances the ability to discriminate each note in a chord, and biofeedback training that visualizes and sound difficult-to-perceive states of the body and brain.
Japan Science and Technology Agency (JST)
FY2022 JST CREST AIP Challenge program, “Precise estimation of hand anatomical features using generative models and depth images”
FY2020~FY2025 CREST, ”Building a trusted explorable recommendation foundation technology” Group leader (Research director: Masataka Goto, National Institute of Advanced Industrial Science and Technology)
FY2020 JST CREST AIP Challenge program, “Developing a hand haptic skills training method using AI technology”
FY2017~FY2022 CREST, “A study on skill acquisition mechanism and development of skill transfer systems” Group leader (Research director: Prof. Hideki Koike, Tokyo Institute of Technology)
FY2024~FY2029 JST CRONOS “Democratization of Skill-Communication Technologies Based on an Understanding of the Operating Principles of the Brain and Body” JPMJCS24N8
JSPS KAKENHI
FY2020~FY2023 Grant-in-Aid for Scientific Research (B), “Elucidation of neural mechanisms underlying the promotion of motor execution, learning, and recovery associated with active touch training” 20H04093
FY2024~FY2026 Grant-in-Aid for Scientific Research (B),“Development of Personalized Skill-Acquisition Feedback Training Through Human-AI Collaboration” 24K02992
FY2023~FY2025 Grant-in-Aid for Challenging Research (Pioneering),“Development of MRI-compatible hand haptic robot that opens the way for futuristic training methods” 23K17453(PI: Eiichi Naito, NICT; Co-Investigator: Shinichi Furuya, NeuroPiano)
Achievement
Shinichi Furuya, Ryuya Tanibuchi, Hayato Nishioka, Yudai Kimoto, Masato Hirano, Takanori Oku (2023) Passive somatosensory training enhances motor skill of piano playing in adolescent and adult pianists. Annals of the New York Academy of Science 1519(1): 167-172
Masato Hirano, Muzuha Sakurada, Shinichi Furuya (2020) Overcoming the ceiling effects of experts’ motor expertise through somatosensory training. Science Advances 6(47): eabd2558
Nobuhiro Takahashi, Shinichi Furuya, Hideki Koike (2020) Soft exoskeleton glove with human anatomical architecture: production of dexterous finger movements and skillful piano performance. IEEE Transaction on Haptics 13(49): 679-690
Kazuki Kawamura, Fujiki Nakamura, Hayato Nishioka, Momoko Shioki, Shinichi Furuya and Jun Rekimoto (2026) Interpretable Visualization of Expertise-Dependent Motor Skills Toward Supporting Piano Practice. Proceedings of the 2026 ACM Designing Interactive Systems Conference: 2868-2885.
Ruofan Liu, Yichen Peng, Takanori Oku, Chen-Chieh Liao, Erwin Wu, Shinichi Furuya and Hideki Koike (2025) From Pose to Muscle: Multimodal Learning for Piano Hand Muscle Electromyography. Advances in Neural Information Processing Systems 38: 115558-115587.
Ruofan Liu, Yichen Peng, Takanori Oku, Chen-Chieh Liao, Erwin Wu, Shinichi Furuya and Hideki Koike (2025) PiaMuscle: Improving Piano Skill Acquisition by Cost-effectively Estimating and Visualizing Activities of Miniature Hand Muscles. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, Article 364: 1-16.
Huan Zhang, Vincent K. M. Cheung, Hayato Nishioka, Simon Dixon and Shinichi Furuya (2025) LLaQo: Towards a Query-Based Coach in Expressive Music Performance Assessment. Proceedings of the 2025 IEEE International Conference on Acoustics, Speech and Signal Processing: 1-5.
Yudai Kimoto, Masato Hirano and Shinichi Furuya (2025) Decomposition of a complex motor skill with precise error feedback and intensive training breaks expertise ceiling. Communications Biology 8: 118.
Shinichi Furuya, Takanori Oku, Hayato Nishioka and Masato Hirano (2025) Surmounting the ceiling effect of motor expertise by novel sensory experience with a hand exoskeleton. Science Robotics 10(98): eadn3802.
Ruofan Liu, Yichen Peng, Takanori Oku, Erwin Wu, Shinichi Furuya and Hideki Koike (2024) PianoKeystroke-EMG: Piano Hand Muscle Electromyography Estimation from Easily Accessible Piano Keystroke. SIGGRAPH Asia 2024 Posters, Article 6: 1-2.
Huan Zhang, Vincent K. M. Cheung, Hayato Nishioka, Simon Dixon and Shinichi Furuya (2024) How does the teacher rate? Observations from the NeuroPiano dataset. Extended Abstracts for the Late-Breaking Demo Session of the 25th International Society for Music Information Retrieval Conference.
Ruofan Liu, Erwin Wu, Chen-Chieh Liao, Hayato Nishioka, Shinichi Furuya and Hideki Koike (2023) PianoSyncAR: Enhancing Piano Learning through Visualizing Synchronized Hand Pose Discrepancies in Augmented Reality. 2023 IEEE International Symposium on Mixed and Augmented Reality: 859-868.
Ruofan Liu, Erwin Wu, Chen-Chieh Liao, Hayato Nishioka, Shinichi Furuya and Hideki Koike (2023) PianoHandSync: An Alignment-based Hand Pose Discrepancy Visualization System for Piano Learning. Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, Article 228: 1-7.
Patent
Yuki Ogasawara, Hayato Nishioka and Shinichi Furuya (2025) Detection device and detection method. U.S. Patent Application No. 18/854,550.
Hayato Nishioka, Yuki Ogasawara and Shinichi Furuya (2025) Information processing device, information processing method, and information processing program. U.S. Patent Application No. 18/864,280.
Hayato Nishioka, Takanori Oku, Shinichi Furuya, Takahisa Ishikawa and Yuki Ogasawara (2025) Information processing apparatus, information processing method, and information processing program. U.S. Patent No. 12,350,036.
Yuki Ogasawara, Hayato Nishioka and Shinichi Furuya (2025) Key depression amount sensor device. U.S. Patent Application No. 18/842,596.
Hayato Nishioka, Takanori Oku and Shinichi Furuya (2025) Measurement device and measurement method, and program for measuring a moveable range of a finger. U.S. Patent No. 12,299,205.
Shinichi Furuya and Takanori Oku (2024) Information processing apparatus. U.S. Patent No. 12,142,249.
Takanori Oku, Yu Nishimura, Hayato Nishioka, Shinichi Furuya, Yukako Uematsu, Junichiro Sakata, Takaharu Suzuki, Reigo Ban and Ryo Nakabayashi (2024) Information processing device, information processing method, and information processing system. U.S. Patent Application No. 18/685,732.
Shinichi Furuya and Takanori Oku (2024) Information processing device, information processing method, and program. U.S. Patent No. 12,067,891.
Hayato Nishioka, Shinichi Furuya, Masato Hirano and Takanori Oku (2023) Information processing device, information processing method, and program. U.S. Patent No. 11,741,925.
Hayato Nishioka, Takanori Oku and Shinichi Furuya (2023) Information processing device, information processing method, and information processing program. U.S. Patent Application No. 17/792,327.
Pianists must memorize multiple lengthy pieces for concerts, and it is difficult to completely eliminate the fear of forgetting their memorized pieces during a performance even for experts. In addition, in stressful situations such as competitions or auditions, memories may become unstable, causing memory lapses or performance deterioration under pressure. To enable pianists to perform their best on stage, we aim to clarify the mechanisms of memory and anxiety in piano performance. Our goal is to develop training strategies to help that memorized pieces can be reliably recalled during performance under various environments.
Cabinet Office, Government
FY2020~FY2025 Moonshot Research and Development Program, ”Liberation from Biological Limitations via Physical, Cognitive and Perceptual Augmentation” Group leader (Project Manager: Ryota Kanai, CEO, Araya Inc.)
Private grant
FY2018 YAMAHA MUSIC FOUNDATION, Research Activity Support, “Elucidation of the mechanism of loss of hand motor dexterity associated with state anxiety”
JSPS KAKENHI
FY2025~FY2026 Grant-in-Aid for Early-Career Scientists,“Impact of Physiological Cycles on Performance Stability of Learned Motor Sequences” 25K21067
Achievement
Shinichi Furuya, Reiko Ishimaru, Takanori Oku, Noriko Nagata (2021) Back to feedback: aberrant sensorimotor control in music performance under pressure. Communications Biology 4(1): 1367
Shinichi Furuya, Reiko Ishimaru, Noriko Nagata (2021) Factors of choking under pressure in musicians. PLoS One 16(1): e0244082
Shuntaro Kotani, Shinichi Furuya (2018) State anxiety disorganizes finger movements during musical performance. Journal of Neurophysiology 120(2): 439-451
Pei-Cheng Shih, Masato Hirano and Shinichi Furuya (2026) Bridging chunks during complex movement sequence execution. iScience 29(2): 114562.
Pei-Cheng Shih, Adrien Verhulst, Masato Hirano, Shunichi Kasahara and Shinichi Furuya (2024) Non-linear impact of reward size on pianists' performance and anxiety-related physiological responses. Psychophysiology 61(S1): S165.
During daily practice, many musicians experience physical and mental problems. One representative example is a neurological disorder known as "focal dystonia", which makes it difficult to move the body as intended. This disorder occurs at body portions such as the fingers of pianists, the left fingers and right arms of violinists, the feet of drummers and organists, and the mouths and tongues of wind instrumentalists, causing them to lose control of their body movements and experience sensory abnormalities. This is a terrifying disease that sometimes even terminates musicians' careers, but there is currently no established diagnostic method or treatment that can realize perfect cure. We aim to create preventive methods for physical and mental problems caused by excessive musical practice through unveiling the pathological mechanisms, as well as developing retraining and rehabilitation methods.
JSPS KAKENHI
FY2019~FY2022 Fund for the Promotion of Joint International Research (Fostering Joint International Research (B)), “Unveiling neuroplasticity of sensorimotor functions subserving acquisition and loss of musical skills” 19KK0252
FY2016~FY2017 Grant-in-Aid for Challenging Exploratory Research, “Closed-loop non-invasive brain stimulation system for assessment of neuromuscular control of sequential movements”
Private grant
FY 2023 Tateisi Science and Technology Foundation, Research Grant (A), “Creation of a training system to normalize the function of movement disorders caused by overtraining”
FY 2017 KAWAI FOUNDATION FOR SOUND TECHNOLOGY & MUSIC, Research grant, “Elucidating the mechanism of the loss of hand dexterity caused by dystonia in musicians”
Japan Science and Technology Agency (JST)
FY 2022 JST CREST AIP Challenge program, “Development of a diagnostic aid system for dystonia in musicians”
Achievement
Ken Takiyama, Shuta Mugikura, Shinichi Furuya (2022) Impaired feedforward control of movements in pianists with focal dystonia. Frontiers in Neurology 13: 983448
Kahori Kita, Shinichi Furuya, Rieko Osu, Takashi Sakamoto, Takashi Hanakawa (2021) Aberrant cerebello-cortical connectivity in pianists with focal task-specific dystonia. Cerebral Cortex 31(10): 4853–4863
Shinichi Furuya, André Lee, Takanori Oku, Eckart Altenmüller (2020) Aberrant somatosensory-motor adaptation in musicians' dystonia. Movement Disorders 35(5): 808-815
Takanori Oku, Shinichi Furuya (2019) Neuromuscular incoordination in musicians' dystonia. Parkinsonism & Related Disorders 65: 97-104
Kazumasa Uehara, Shinichi Furuya, Hidemi Numasawa, Kahori Kita, Takashi Sakamoto, Takashi Hanakawa (2019) Distinct roles of brain activity and somatotopic representation in pathophysiology of focal dystonia. Human Brain Mapping 40(6): 1738-1749
Kahori Kita, Jaroslav Rokicki, Shinichi Furuya, Takashi Sakamoto, Takashi Hanakawa (2018) Resting-state basal ganglia connectivity codes a motor musical skill and its disruption due to a disease process. Movement Disorders 33(9): 1472-1480
Shinichi Furuya, Kazumasa Uehara, Takashi Sakamoto, Takashi Hanakawa (2018) Aberrant cortical excitability explains the loss of hand dexterity in musician's dystonia. The Journal of Physiology 596(12): 2397-2411
André Lee, Tobias Mantel, Shinichi Furuya, Masanori Morise, Eckart Altenmüller and Bernhard Haslinger (2026) Quantification of Loudness Instability in Tone Production in Embouchure Dystonia. Journal of Movement Disorders 19(2): 199-202.
Takanori Oku, Shinichi Furuya, André Lee and Eckart Altenmüller (2024) Video-based diagnosis support system for pianists with musician's dystonia. Frontiers in Neurology 15: 1409962.
Mitsuaki Takemi, Mai Akahoshi, Junichi Ushiba and Shinichi Furuya (2023) Behavioral and physiological fatigue-related factors influencing timing and force control learning in pianists. Scientific Reports 13(1): 21646.
Shinichi Furuya and Takanori Oku (2023) Sensorimotor Incoordination in Musicians' Dystonia. Advances in Neurobiology 31: 61-70.
Tobias Mantel, André Lee, Shinichi Furuya, Masanori Morise, Eckart Altenmüller and Bernhard Haslinger (2023) Reliability and Validity of the Embouchure Dystonia Severity Rating Scale. Journal of Movement Disorders 16(2): 191-195.
Johanna Doll-Lee, André Lee, Shinichi Furuya, Bernhard Haslinger and Eckart Altenmüller (2023) Musician's Dystonia: Family History as a Predictor for Onset and Course of the Disease. Movement Disorders 38(8): 1559-1560.
Musical expression in piano performance is shaped by a wide range of elements, including dynamics, note duration, timbre, timing, and pedaling. Pianists create entirely different expressions from the same score by finely controlling how they depress and release the keys, as well as how they use their fingers, wrists, and arms. However, it remains unclear what kinds of body movements give rise to specific acoustic features, and how these features are perceived by listeners.
We investigate the bodily mechanisms that generate musical expression using multimodal sensing technologies that simultaneously record key motion, pedal operation, performance sound, multi-view video, skeletal information, and muscle activity. By quantifying keystrokes, key release, and finger and arm movements involved in the production of timbre and expressive nuance by skilled pianists, we aim to clarify the relationships among body movements, acoustic features, and listeners' perception.
Our findings will contribute to the development of new practice methods for approaching an intended timbre or expression, feedback technologies for visualizing musical expression, and a scientific foundation for sharing sensory terms used in pedagogy, such as “touch,” “timbre,” and “nuance.”
Achievement
Jingjing Tang, Shinichi Furuya, Hayato Nishioka, Momoko Shioki, Geraint A. Wiggins, George Fazekas and Vincent K. M. Cheung (2026) Visualising Pianists' Touch: Transcribing Expressive Piano Performance from Audio to Piano Key Motion. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems, Article 857: 1-11.
Kaori Kuromiya, Yuya Kobayashi, Masato Hirano and Shinichi Furuya (2025) Motor origins of timbre in piano performance. Proceedings of the National Academy of Sciences 122(39): e2425073122.
Jingjing Tang, Shinichi Furuya, George Fazekas and Vincent K. M. Cheung (2025) S2H2S: Modelling Expressive Piano Performances via Continuous Piano Key Motion Signals. Extended Abstracts for the Late-Breaking Demo Session of the 26th International Society for Music Information Retrieval Conference.
Find out more about Musical Dynaformics
ダイナフォーミックスについてより詳しく知る
© 1993-2024 Sony Computer Science Laboratories, Inc.
© 2021-2024 NeuroPiano Inst.