Applications of Sound Technology in Entrainment, Virtual Reality, and Embodied Medicine

Introduction

This paper explores the multifaceted applications of sound technology, spanning from therapeutic interventions in neurology to its pivotal role in enhancing VR experiences and its potential in embodied medicine through sonoception. Recently, papers [1] and [2] have discussed the fundamentals of ultrasound stimulation, its physics, therapeutic uses, and its applications in neuromodulation.

The paper also delves into the potential therapeutic intervention of transcranial ultrasound-based gamma entrainment. According to research, modulating gamma oscillations using low-intensity focused ultrasound can restore impaired neuronal activity, potentially reducing Aβ load and improving brain function in Alzheimer’s disease (AD).

Additionally, sound technology has made significant strides in the realm of Virtual Reality (VR).

Moreover, embodied medicine introduces the concept of sonoception to modify body experiences for health benefits. This approach utilizes sound and vibration to manipulate internal bodily sensations, from stimulating mechanoreceptors in the stomach to inducing the perception of one’s heartbeat. Sonoception holds potential in altering subjective inner body experiences and understanding psychosomatic processes.

Entrainment

Entrainment is a phenomenon characterized by the synchronization of neural activity in response to external stimuli. It involves the alignment of oscillatory patterns, particularly gamma oscillations, across different brain regions, leading to enhanced communication and information processing. Electrical vibrations called theta and gamma take place in the brain and are crucial for neuronal activation. Information is moved between various parts of the brain thanks to theta vibrations. They typically operate at a frequency of 4 to 8 Hz, are particularly active in settings that promote mental focus, rest, and sleep, and are involved in cognitive functions like learning, memory, and concentration. Gamma vibrations, which vary in frequency from 30 to 100 Hz, are higher frequency than theta vibrations. Neuronal coordination and rapid and simultaneous connections between various brain regions are both facilitated by gamma vibrations. They participate in brain functions like problem-solving, sustained concentration, and the processing of auditory and visual information. The dysregulation of gamma oscillations has been observed in neurodegenerative disorders, underscoring their significance in maintaining healthy brain function [3],[4],[5].

Study demonstrates that ultrasound neuromodulation and gamma entrainment have the potential to serve as promising therapeutic interventions for neurological disorders [6]. To address the disruption of gamma oscillations in Alzheimer’s disease (AD), transcranial ultrasound-based gamma entrainment has emerged as a potential intervention. This technique employs the non-invasive application of low-intensity focused ultrasound to modulate and synchronize gamma oscillations in specific brain regions. The hypothesis is that by stimulating neural circuits at gamma frequencies, gamma entrainment can restore impaired neuronal activity.

To explore the validity of this hypothesis, several studies have investigated the effects of transcranial ultrasound-based gamma entrainment in AD models. By examining the outcomes of these investigations, we can gain insights into the effectiveness of transcranial ultrasound-based gamma entrainment as a promising intervention for AD. A different research paper examines the potential therapeutic effects of transcranial ultrasound stimulation at a gamma frequency of 40 Hz on Alzheimer’s disease (AD) in a mouse model. AD is characterized by the presence of amyloid-beta (Aβ) plaques and tauopathy. The study involved implanting EEG electrodes and ultrasound transducers on the skulls of both AD mice and control mice. The AD mice received two-hour ultrasound stimulation at 40 Hz daily for two weeks, while the control mice received sham treatment. The results demonstrated that the treatment group exhibited decreased levels of Aβ, particularly insoluble Aβ, in specific brain regions. Additionally, the number of Aβ plaques in the hippocampus, a critical area affected in AD, was also reduced. Importantly, the ultrasound stimulation improved brain connectivity, as evidenced by increased spontaneous gamma power and normalized cross-frequency coupling. These findings suggest that transcranial ultrasound-based gamma-band entrainment could be an effective therapy for AD by reducing Aβ load and improving brain function [7].

Another study focused on the use of focused ultrasound (FUS) to modulate the activity of cerebellar Purkinje cells (PCs) with varying pulse durations and frequencies. Using anesthetized rats, the researchers recorded the single-unit activity of PCs while applying ultrasonic stimulation. The results revealed that FUS entrained the spike activity of PCs, with the probability of spike occurrences peaking around 1ms following the stimulus onset, regardless of pulse duration. Notably, shorter pulse durations resulted in stronger entrainment at a pulse repetition frequency of 50 Hz, while longer durations showed delayed and wider spread effects at 100 Hz. However, no significant change was observed in the average firing rates. These findings provide valuable insights into the modulatory effects of FUS on single-cell activity in the cerebellar cortex, potentially through the aggregate effects on the entire cortical network. This research highlights the importance of pulse duration in entraining neural spikes and offers potential applications of FUS for neuromodulation [8].

Figure 1 provides essential information on the experimental setup and methodology used in the study. Figure 1A presents a schematic view illustrating the placement of the recording electrode and ultrasound probe. Figure 1B shows a sample recording of cerebellar Purkinje cells (PCs) with a marked complex spike. Figure 1C displays the pulse train generated to control the ultrasonic stimulation pattern, with each train consisting of 500ms on-periods separated by 500ms off-periods. The pulse repetition frequency (PRF) refers to the frequency at which the pulses are repeated.

Ultrasound in Virtual Reality (VR)

Virtual Reality (VR) technology has revolutionized the way we interact with digital environments, offering immersive experiences that engage our senses and transport us to virtual worlds. To achieve true immersion, it is crucial to involve sensory systems and even internal bodily sensations. In the human body, we have different sensory systems that allow us to perceive and interact with the world around us. These include Exteroception (sight, hearing, touch, smell, and taste), proprioception (awareness of body posture and movements), vestibular input (sense of motion and body position), and Interoception (sense of the physiological condition of the body). Research has shown that the more sensory inputs we can replicate in VR, the more immersive and embodied the experience becomes. Therefore, incorporating multiple sensory inputs, including haptic feedback, is essential in creating more realistic and engaging VR experiences. Ultrasound technology has emerged as a promising solution for incorporating haptic feedback and stimulating internal bodily experiences in VR environments. Ultrasound utilizes high-frequency sound waves beyond the audible range of humans to create physical sensations on the user’s body, providing a realistic sense of touch and interaction with virtual objects.

Ultrasound Haptic Feedback and Stimulation in VR

• Haptic wearable devices

Researchers have explored integrating ultrasound technology into VR headsets to provide haptic feedback without the need for additional wearable devices. By attaching an array of ultrasonic transducers to the headset, sound waves can be emitted and directed towards specific areas of the user’s face or body. This enables the creation of various tactile sensations, such as the feeling of objects touching the skin or the sensation of wind on the face, enhancing the realism and immersion in virtual environments.

Ultrasound technology offers a wide range of sensations that can be replicated in VR. For example, users can experience the crawling of insects on their skin, the splash of water on their face, or the sensation of drinking from a waterfall. For instance, in [9”] they used ultrasound in a VR headset without adding a new wearable, and they attached it to the VR headset. An array of several ultrasonic transducers, which emit sound waves outside the audible range, generate the feedback. Those sound waves cause a minor physical sensation when they touch the lips, teeth, or tongue.

By precisely manipulating the ultrasound waves, developers can simulate different textures, pressures, and movements, further enhancing the sense of realism and immersion in virtual environments [9].

• mid-air haptic

A formal research investigation was conducted with the primary objective of examining the influence of tactile interactions in virtual reality (VR) environments, particularly within the context of virtual scenarios. The experimental framework adopted for this study employed a 2×3 within-group design, incorporating two levels of virtual representation and multisensory feedback figure2.

To ensure scientific rigor, the study introduced a randomization of the sequence in which the virtual representation and multisensory feedback conditions were presented. Significantly, the condition involving actual physical touch was consistently administered as the final stage of the experiment. This strategic decision served to establish a reference point for gauging participants’ subjective experiences and physiological responses, all while considering the immersive, virtual nature of the environment. Furthermore, the allocation of the physical touch condition was randomized between scenarios featuring avatars and feather-based interactions.

Participants in the study were seated in front of a tangible table while their visual perspective encompassed a virtual representation of a human body from a first-person avatar standpoint. This avatar, attired in a neutral fashion, enabled precise tracking of hand and finger movements through the utilization of a Leap Motion sensor. Real-time motion capture technology, inclusive of PhaseSpace Impulse X2E, MotionBuilder, and Blender, was harnessed to animate the movements of the avatar’s hands. Notably, the avatar’s facial expressions remained concealed behind a translucent screen, allowing participants to view the hands and body while concealing any facial features. This conscious design choice aimed to circumvent emotional cues, the so-called “uncanny valley” effects, and the possibility of discomfort or aversion when a humanoid avatar imperfectly resembles a genuine human being.

The focal point of the investigation centered on delivering tactile sensations to the participants’ left hand due to its heightened sensitivity to ultrasound waves and cutaneous afferent nerves. Calibration of ultrasonic waves was meticulously adjusted to align with the participants’ specific hand size, thereby preventing mismatches between the virtual touch and the felt feedback. A visual reference illustration of the left hand was created to elucidate the trajectory of haptic feedback. Participants, unbeknownst to the true purpose of the experiment, were instructed to place their left hand on the table with the palm facing upward.

A custom-designed ultrasound board, fabricated with a 3D printer, was positioned above the participants’ palms. In this setup, participants perceived two discrete points of tactile stimulation on their left palm, aligning these points with reference markers in the visual representation of the left hand. Synchronization between the ultrasound waves and the visual animations featuring avatars and feathers was carefully orchestrated. The velocity of the ultrasound waves was adjusted to account for differences in palm size, with optimal velocity settings ranging between 1 to 10 centimeters per second. Importantly, participants had the ability to control and reposition the location of the stimulation using keyboard inputs. The central objective was to gain insights into the impact of tactile stimulation on the quality of affective touch experiences.

To assess the physiological responses of participants, the study integrated the use of ultrasound to measure skin conductance response (SCR). The ultrasound board was configured with unchanging parameters including intensity, frequency, and distance. SCR data was recorded through the deployment of a ProComp Infiniti Encoder and two galvanic skin response sensors. Data preprocessing was executed via MATLAB, and subsequent analysis was conducted using the Statistical Package for the Social Sciences (SPSS).

Within the experimental paradigm, event markers were systematically introduced to the skin conductance signal when avatars or feathers engaged in caressing participants for a duration of 60 seconds. The results of the study revealed that congruent spatiotemporal visuotactile feedback significantly heightened the illusion of being touched and incorporated into an artificial body, eliciting elevated sensations of pleasure, arousal, and corresponding physiological responses. Additionally, the study underscored the pivotal role of somatosensory stimuli by establishing that genuine interpersonal touch and ultrasonic mid-air haptic stimulation both enhance illusions related to the body. This study supported the viability of utilizing ultrasonic mid-air haptic stimulation for simulating lifelike affective touch in the context of VR and mediated communication. Although physical skin-to-skin contact or interaction with a feather produced the most profound illusory touch sensations, subjective evaluations of being touched or caressed in conditions involving mid-air ultrasonic tactile stimulation consistently rated highly [11],[12],[13].

Embodied medicine and Sonoception

Embodied medicine is a field that aims to alter the experience of being in a body and improve health and well-being using advanced technologies. Our body is unique because it can sense both internal and exterior cues. Researchers are investigating the integration of exteroceptive and interoceptive inputs to connect VR with bio/neurofeedback and brain/body stimulation technologies. As an extension of this non-invasive method, the idea of “Sonoception” is put forth, which modifies the internal/inner body sensation through sound and vibration. With recent developments in our understanding of sound and vibration, acoustics has come to be recognized as a technological science.

Through mechanoreceptors, which convert sensory information into particular somatosensory experiences, these interconnected physical occurrences can have an impact on the human body. In the inner ear, for instance, sound and vibration can produce fluid pressure waves that can result in vertigo and vestibular disorders. Similarly, the heart is susceptible to both internal and external mechanical pressures. All inner body functions, such as vestibular input, proprioception, and Interoception, are intended to be replicated via Sonoception technology. In order to induce the perception of movement in the stomach and heart, this method will stimulate mechanoreceptors in the chest and belly using contactless acoustic transducers. They have used acoustic levitation to translate the food particles with the motion of the stomach walls to perceive the movement of the stomach. The ultrasonic transducer array was 40 to 60 kHz. They have also used the low bass frequency of 50 to 120 Hz to stimulate mechanoreceptors in the chest to ultimately experience their own heartbeat sensation. Mechanoreceptors located on muscles and otolith organs inside the vestibular system will be stimulated by vibrotactile transducers. Through inner body modulation, researchers can investigate how these modifications impact internal and inner subjective experiences and comprehend the relationship between inner signal fluctuations and BSC. By modifying psychological and behavioral aspects, embodied medicine seeks to reverse engineer psychosomatic processes. Psychophysiological signals are converted to vibrationary signals and returned to the body in real time by contactless acoustic transducers. These signals can be processed and classified by a closed-loop software module.

Companies like Doppel1 are creating wearable technology that uses haptic input to change heart rhythms. In order to generate the illusion of inhabiting a distinct synthetic or surrogate body, the method uses exteroceptive inputs from the body, such as touch and vision, to create a multimodal conflict. Recent research has demonstrated that embodiment over a virtual body can influence the body’s natural reactions to unpleasant stimuli and enhance depressed patients’ sense of self-compassion. Future pain and depression treatments may benefit from the use of embodied virtual bodies, and body-swap illusions might alter an individual’s view of their body, memories, emotions, and desire to adopt healthy eating habits [10 ].

Conclusion

Ultrasound stimulation has several uses and benefits in the field of virtual reality; therefore, this technology has the potential to play a significant role in the advancement of the VR user experience. Subsequent clinical research endeavors ought to delve into the psychophysiological and neurological mechanisms that facilitate the amalgamation of internal bodily signals with external stimuli.

REFERENCES

[1]. “The basic of ultrasound stimulation”

[2]. “Ultrasound Noromodulation”

[3]. Hannah F. Iaccarino, Annabelle C. Singer, Anthony J. Martorell, Andrii Rudenko, Fan Gao, Tyler Z. Gillingham, Hansruedi Mathys, Jinsoo Seo, Oleg Kritskiy, Fatema Abdurrob, Chinnakkaruppan Adaikkan, Rebecca G. Canter, Richard Rueda, Emery N. Brown, Edward S. Boyden & Li-Huei Tsai: “Gamma frequency entrainment attenuates amyloid load and modifies microglia”. Nature volume 540, pages 230–235 (2016)

[4]. Artemis Traikapi and Nikos Konstantinou: “Gamma Oscillations in Alzheimer’s Disease and Their Potential Therapeutic Role”. Front. Syst. Neurosci. 15:782399. doi: 10.3389/fnsys.2021.782399

[5]. Anthony J. Martorell, Abigail L. Paulson, Ho-Jun Suk, …, Edward S. Boyden, Annabelle C. Singer, Li-Huei Tsai: “Multi-sensory Gamma Stimulation Ameliorates Alzheimer’s-Associated Pathology and Improves Cognition”. 2019 Elsevier Inc.

[6].  Doi:10.33552/ANN.2022.13.000804

[7]. Mincheol Park1, Gia Minh Hoang1, Thien Nguyen, Eunkyung Lee, Hyun Jin Jung, Youngshik Choe, Moon Hwan Lee, Jae Youn Hwang, Jae Gwan Kim and Tae Kim: “Effects of transcranial ultrasound stimulation pulsed at 40 Hz on Aβ plaques and brain rhythms in 5×FAD mice”. Translational Neurodegeneration (2021)

[8].  Ahmet S. Asan, Qi Kang, Ömer Oralkan, Mesut Sahin: “Entrainment of cerebellar Purkinje cell spiking activity using pulsed ultrasound stimulation”. Brain Stimulation 14 (2021) 598e606   DOI:  https://doi.org/10.1016/j.brs.2021.03.004

[9]. Vivian Shen , Craig Shultz ,Chris Harrison : “Mouth Haptics in VR using a Headset Ultrasound Phased Array”. Proceedings of the 2022 CHI Conference on Human Factors in Computing SystemsApril 2022Article No.: 275Pages 1–14 https://doi.org/10.1145/3491102.3501960

[9”] https://www.figlab.com/research/2022/mouth-haptics

[10]. Sofia Seinfeld, Ivette Schmidt, Jörg Müller: “Evoking realistic affective touch experiences in virtual reality”. arXiv:2202.13389. https://doi.org/10.48550/arXiv.2202.13389

[11]. Fusaro, Martina, Lisi, M. P., Tieri, G., & Aglioti, S. M. (2021). Heterosexual, gay, and lesbian people’s reactivity to virtual caresses on their embodied avatars’ taboo zones. Scientific Reports, 11(1), 2221. https://doi.org/10.1038/s41598-021-81168-w

[12]. Shin, M., Kim, S. J., & Biocca, F. (2019). The uncanny valley: No need for any further judgments when an avatar looks eerie. Computers in Human Behavior, 94, 100–109. https://doi.org/10.1016/j.chb.2019.01.016

[13]. Watkins, R. H., Dione, M., Ackerley, R., Wasling, H. B., Wessberg, J., & Löken, L. S. (2021). Evidence for sparse C-tactile afferent innervation of glabrous human hand skin. Https://Doi.Org/10.1152/Jn.00587.2020, 125(1), 232–237. https://doi.org/10.1152/JN.00587.2020

[14]. Giuseppe Riva, Silvia Serino, Daniele Di Lernia, Enea Francesco Pavone and Antonios Dakanalis: “Embodied Medicine: Mens Sana in Corpore Virtuale Sano”. Frontiers in Human Neuroscience, March 2017 doi: 10.3389/fnhum.2017.00120