Professor Nishiura, thank you for your time today. Can you please tell us more about your area of research?

Prof. Nishiura: I joined the College of Information Science and Engineering at Ritsumeikan University in 2004. My research focuses on acoustic signal processing, acoustic systems, and sound interfaces, amongst others.

I conduct integrated research on the analysis, understanding, reproduction, and synthesis of a sound environment to improve society through sound. My goal as a researcher is to create a sound environment for people to live comfortably.

Could you please elaborate on this?

Prof. Nishiura: A practical example may be the visit to the dentist. Many people feel uncomfortable with the dental drilling sound. I have researched and developed a technology of sound esthetics, also called noise-masking so that people are not bothered by the unwanted noise by overlapping it with a pleasant sound.

Furthermore, I have been investigating audio spot technology, a technique to transmit sound only in a specific area. Loudspeakers utilizing ultrasound waves have a higher directivity and can form a narrow audible area to a particular listener, i.e., the audio spot.

This is useful for exhibitions at museums, for example, when you would like to give visitors information relevant to each exhibited object.

When and how did you join the Art Research Center (ARC)?

Prof. Nishiura: The Art Research Center has been conducting advanced research in Japanese arts and culture by merging the humanities with information science. I first joined the ARC around ten years ago when we started the project to build a Digital Museum of the Gion Festival , adopted by the Ministry of Education, Culture, Sports, Science and Technology (MEXT).

An Intangible Cultural Heritage designated by UNESCO, the Gion Festival is one of the most treasured festivals in Japan.

For an authentic virtual experience of the festival, we have not only created ultra-high-quality visualizations but also reproduced the unique music and sounds that surround this festival in a highly realistic way.

Can you please tell us a little more about this project?

Prof. Nishiura: As part of the digital museum, we have virtually reproduced the Yamahoko Junko--the grand parade of decorated floats and highlight of the festival. (--> click here for the video)

My colleague Prof. Satoshi Tanaka created ultra-high-quality 3D see-through visualization models of some festival floats while I was in charge of reproducing the sound of the Gion bayashi, the floats, crews, and spectators using a high-fidelity acoustic recording.

Gion bayashi is the unique orchestral music accompanying the festival and played with the traditional instruments of drums, flutes, and bells.

We built a binaural system to record the musical performances of Gion bayashi indoors and at the actual parade. Binaural recording of sound refers to the technique of placing two microphones in a dummy head to reproduce the acoustics of a human head as it reaches the eardrums. Hence, a 3D sound impression is generated for listeners as if they were at the festival themselves.

Finally, we integrated all these recorded sounds, including the ambient noise of the creaking of wheels and the buzzing crowd to create a highly immersive virtual experience.

What is your current research project under the ARC's common theme 「"遊び": アート・エンターテインメント研究」 ("Play": Art Entertainment)?

Prof. Nishiura: Our current project examines the acoustic sound reconstruction and digital archiving of traditional Japanese music using the example of the shikokin, an antique music box popular towards the end of the 19^th century (Meiji period).

Through investigating the shikokin, we can learn about Japanese instruments from the past and what kind of music was popular during the Meiji period-- something difficult to determine from historiography.

What is interesting about the research on the shikokin?

Prof. Nishiura: To play a melody, the shikokin requires a hand-made rolled music sheet with holes that takes time to make. Likewise, to edit a melody, you are required to modify the holes by hand.

Digital technology has not only enabled us to recreate the sound of music from the past in the way in which it was composed and to edit melodies easily but has also made it possible to express tones beyond the actual scale of the instrument.

By developing a method to expand the musical pitch range and constructing a web-based system, we can play and listen to contemporary music with the traditional sound of a shikokin.

Furthermore, we are working on reconstructing the sound propagation, i.e., how sound waves travel, by measuring sound pressure distribution around the shikokin. Our goal is to develop a new interactive performance interface to visualize its sound in a three-dimensional way.

You have also been active in industry-academia collaboration.

Prof. Nishiura: Yes. Recently, we have collaborated with Nike to develop golf balls with an improved sound when hitting them.

First, we analyzed the sound of their golf balls. Then, we conducted trials based on different materials for the inner core. A decisive factor behind the sound is the hardness or softness of the inner core material.

Furthermore, we are developing ultrasound directional loudspeakers that we can utilize for exercise instructions, for example.

People should be able to exercise in a sound environment comfortably to maintain a healthy lifestyle. However, in Japan--particularly the metropolitan areas--the lack of space has been a permanent problem. I advocate the concept of space-sharing to promote exercising while sharing a common space.

Could you please explain how this space-sharing concept works?

Prof. Nishiura: We base this concept on flexible audio spot technology. As mentioned at the beginning, sound waves travel with a different directivity depending on their frequencies, and the higher the frequency, the straighter the sound travels at a specific angle.

Conventional loudspeakers can change the volume of sound but cannot change the range of the sound.

We have developed parametric loudspeakers with a sound audible to humans but the directivity of ultrasound waves. Hence, we can control the sound transmitted through the loudspeaker within a straight direction at a specific range. We can flexibly generate multiple sounds in a space with a different sound range according to the number of users and their positions.

Therefore, we can divide space by sound and use it effectively. Various groups of people can gather in the same space while receiving different audio exercise instructions.

These ultrasound loudspeakers have already been in use at the gymnasium at Ritsumeikan University's Biwako-Kusatsu Campus.

You are engaged in various research fields. In conclusion to this interview, what would you say is important to you as a researcher?

Prof. Nishiura: The more we push technology, the more we encounter the greatness and depth of human functions and mechanisms. And the more I learn about humans, the more I realize that technology has not yet caught up. For instance, the human voice can cover a wide range of sounds from high to low. However, current speaker technology still cannot completely cover the same range of sound as humans with a single diaphragm.

So, I would like to further learn from humans and discover the potential of sound through my research.

Prof. Nishiura, thank you very much for your time. We are very much looking forward to the further development of your projects.

(This interview was conducted by Yinzi Emily Li.)