Hi, the Sound Design team of Al Crepuscolo VR is back!

Here's our second instalment about the decision making process in crafting the soundscape of our poetic VR game.

Grabbing and focusing the player’s attention

Opposed to the contextual and narrative sounds, a good sound design also needs to pay attention to what the player is focusing on or what the player should be paying attention to. Sometimes it needs to be very persuasive in telling the players that something is happening and it deserves their attention. This is even more important in a virtual reality environment as it improves the immersion of the experience and helps the player to be a part of the story.    

These sound cues are so important for the story flow, that sometimes we have to cheat and call the player’s attention using a sound that doesn't even relate to a visible element on the screen. These types of sounds anticipate an action or event and help us direct the player to important parts happening out of their attention cone.

For example, during the experience a player has the chance to meet a bear who is just waking up in the nearby forest, outside of the player’s line of sight. To nudge the player into stopping and waiting to see the bear showing, we've created a bunch of vocalizations and grunts coming from the area where the bear will show up eventually.

Like this examples, there are many other animals who emit vocalizations at a specific time to make sure the players don’t miss them. The journey becomes almost like a guided safari in an amusement park, where you want to make sure the player has always something interesting to look at or discover.

Define space, create atmosphere/mood

Different spaces sound differently and sometimes, depending on the narrative, it’s important to highlight those changes of space and intended mood. This is very important to control the emotion that a player feels while experiencing a game.   

The game has five different mood zones spread out across the big lake. Each mood is linked to an animal spirit. Some zones have to give the impression of happiness and joyfulness, while others make the players contemplate feelings of loneliness, isolation and, sometimes, fear. The sounds had to reflect these emotions according to the script.

Below, you can see one of the initial maps that helped us to start creating the different soundscapes for each zone. The level design has changed during production, but we kept these five mood zones (the blue circles).

For happy sections of the journey, we used many lively animals like various birds, squirrels and other forest creatures; for quieter and lonelier moments, we muted all of the birds and other “happy” animals, and added some night animals and insects, like crickets, frogs, flies and various owls.

During the experience the player will enter a dark cave that has to feel whimsical, mysterious but beautiful. In this confined space we passed the wind and tree sounds through a strong low pass filter (to dampen their sound) and we added some frogs and bats sounds, even though those animals are not visible anywhere.

Technically, to achieve these sound transitions between different zones, we used FMOD’s snapshots. The FMOD snapshots allow us to have a lot of flexibility when it comes to changing attributes of the mix in real-time.

The sound transitions between moods are triggered by delineating invisible zones throughout the river: the respective snapshots of each zone is activated when the kayak trespasses them.

The sound ambience implementation of the game was technically challenging sometimes. For random birds, wind and tree sounds the process was fairly easy, since we only had to drag and drop some sound emitters around the map to fill the space.

However, the river shore was a bit more complex than that. The shore had to have wave sounds in its entire length, which added an extra layer of difficulty. We couldn’t simply drag and drop various wave emitters along the shore because that would create phasing problems (if all instances started at the same time), and the player would hear gaps of sound in the shore as well, if there wasn’t an emitter close by. 

Our implementation involved the use of two splines running along the two sides of the shore. Two instances of the water waves were attached to each spline. 

The wave’s sound was restricted to the spline’s path and followed the kayak’s position in real-time. This way, we only needed two sound sources to create the entire shore’s soundscape. This is an effective technique for implementing sound for sources that have a large extension, like rivers.

Emphasise/intensify action

Besides realistic sounds, we also created some fictional sounds to support the narrative, like effects for some magical encounters that happen during the experience. Sometimes this is important to help emphasize an action.  Another type of non-realistic sounds are the “non-diegetic” sounds, which are sounds that do not represent an action or an object that is visible on the screen. For example the bats inside the cave don't really exist in the game, but we imply their presence via found. These non-diegetic sounds help us to recreate a mental image of the environment from the user's real life experience.

Promote immersion in virtual reality

Given that the game is built for VR, the sound is extremely important for the player’s immersion in the virtual environment. Our eyes see a stereoscopic view of the environment, and the head’s movement in the virtual space trick the brain into thinking that virtual world is the actual player’s reality. The sound has to convincingly support this illusion. Luckily, the Oculus headsets support a plugin called Oculus Spatializer, that recreates a version of the sound that surrounds the player from all directions and delivers it into the pair of stereo headphones integrated in the headset. Technically speaking, this plugin uses a function called HRTF (Head Related Transfer Function) that gives the system the capability to convincingly simulate human hearing in a virtual environment. Compared to the standard FMOD spatializer, which lacks directionality in the Up, Down, Front and Back axis, the Oculus Spatializer allows the player to hear sounds from those axis pretty clearly. All in all, the spatializer worked wonders in giving the illusion of 3D sound for Al Crepuscolo rich soundscape. 

An extra tip for tech savvy: never use this spatializer with stereo file sources, stick to mono sources, or your spatialized sounds will sound awful because of phasing problems… you’ve been warned! 😊

Sound Dynamics

To have a credible experience in this game, the sounds had to follow the movement of some objects or actions in the game. For example, while paddling the kayak down a river, the player can paddle fast or slow and the sound has to reflect those dynamics. The technical implementation of this “dynamic sound” uses the parameters of FMOD, which allow the designer to create adaptive sound, by modulating certain attributes of the sound wave via the variables of the game. In this case,  the velocity of the kayak was used to influence the dynamics of the sounds of the water waves and the amount of creaking noises of the kayak. The sound of bumping the kayak against the terrain and the paddling through the water also had different sound dynamics depending on the values from the game.  

It’s crucial to create a sound environment that is “believable” so that the player doesn’t question the audio and stays immersed in the experience. Another useful technique for this is to use several versions of each sound in order to prevent repeating sounds for each animal or action. Sound repetition can cause hear fatigue and it could affect the immersion of the player, giving him/her an overall unsatisfying experience. It’s also interesting to add certain randomness to most sound emitters, such as volume and pitch randomizations to keep them fresh at each execution. 

Smooth transitions 

The transitions between different areas throughout the level should sound seamless, almost like being inside a movie. For this reason it's imperative to avoid stopping the sound during transitions. For entering or exiting a level, we decided to use simple fade-ins and fade-outs, leaving some reverb of the previous sounds, thus avoiding an uncomfortable silence between levels while smoothly and seamlessly transporting the player to a new ambience.

Obstacles encountered during the production

We ran into some technical issues while implementing sound for Al Crepuscolo VR. The most annoying issue we fought against was that some sounds that were supposed to play at a certain time, didn't. The technical reason was connected to the concept of FMOD event instances. In the code, these event instances need to be "released" by the developer after being stopped, otherwise the system stacks several instances along the way and it quickly reaches the maximum number of sound instances that can be kept in memory. This leads to some sounds not playing at the right time at run-time. We had to review all of the sound scripts made for the game and release the instances every time they weren't needed anymore (after a playback for example). It was extra work that we could have avoided if we had known in advance, shame on us! :)

Another simple, but stressful problem we encountered with the implementation was that every time we played the game, after 10 seconds, the game would raise an API error and would crash. We banged our head over several days trying to figure out what the problem was. After 3 days of intense research, sweat and tears Francisco found that the issues stemmed from one simple setting that was checked. In the FMOD integration plugin there is a tiny parameter that reads “API error log” on/off. When we turned that off, the problem disappeared and so did our tears. Again, in the heat of production, we feel sometimes very dumb for a few hours because of these types of frustrating issues that we cannot quite find the reason why. Sometimes the solution is right under our noses, we just can’t see it because it is too close. 

But then again, we have learned a lot from this experience and we will not make the same mistake twice. 

Besides those particular problems, there weren’t that many others problems that we had to sort out regarding the implementation of the soundscape of Al Crepuscolo, thanks to the great integration of FMOD and Unity that made everything work together pretty smoothly. 

So that's all about our journey into the creation of the soundscape of Al Crepuscolo VR. The game is almost finished, with the final touches being added at the time of this writing and we cannot wait to let you try it and get your responses about it. We hope you'll enjoy playing Al Crespuscolo VR as much as we enjoyed making it! 

Until next time, take care and have fun!

Al Crepuscolo VR - Sound Team