At notefornote an exercise that we constantly do is to contemplate the next steps for technology in the music industry. General audio developments and developments in VR and similar technology point to a potential for more immersive music experiences. Let’s take a peek at the development, and how it could affect musicians, songwriters, and producers.
Spatial Audio
Another word for spatial audio is virtual surround sound. It differs from surround sound in the sense that different sounds are placed at different points in a room, whereas surround sound still comes from directional speakers.
The concept of spatial audio has been around since the Byzantine era with the practice of the antiphon – a religious chant sung as call-and-response, thus taking advantage of the surrounding space. The actual term «spatial audio» was introduced in 1928, in Germany, as «Raummusik», which directly translated into «space music.»
The notion of more immersive music experiences has obviously fascinated artists throughout the history. Many experiments followed the German «space music» concept. One such experiment was The Flaming Lips album Zaireeka, released in 1997, and contained four unique CDs designed to be simultaneously played on four different CD players for a surround experience.
Around the same period, the band organized so-called «Parking Lot Experiments». The band recorded 40 cassette tapes and invited people to bring their cars to a parking lot where they were handed cassette tapes to be played in unison.
Fast forward to this century, and in 2021 Apple introduced its spatial sound concept, built on the Dolby Atmos technology, claiming it to be a game-changer in music consumption, and it’s now a feature on Apple Music. Tidal has followed suit with their «Ambient in Dolby Atmos» feature.
Spatial Audio in Your Home
Having a proper spatial set-up in your home takes dedication and a solid budget. The Dolby standard is often based on a 5.1.2. set-up or even 7.1.4. The first number in the sequence refers to the total number of speakers, the second to sub-woofers, and the third to ceiling speakers.
Even the most fanatic of Hi-Fi enthusiasts are not about to hang speakers from the ceiling and will use «up-firing» speakers as a substitute. These are upward-facing speakers that bounce the sound of the ceiling. The systems that include up-firing speakers include algorithms to measure that delay and compensate for it on the audio system.
Immersive Music Innovation and Adaptation
Spatial Audio is becoming more accessible to mainstream audiences thanks to wider support on popular streaming platforms like Apple Music and Tidal. It’s fair to assume that pretty quickly, this will not be viewed as a «novelty technology», but just another way to consume music.
Several interesting innovative initiatives in the field will affect not only how we experience music, but likely how it’s produced and mixed as well. There are advances in binaural rendering techniques, which create immersive 3D audio experiences using just two channels. Additionally, head tracking technology integrated into headphones allows for a more realistic and dynamic soundscape that changes as the listener moves her head.
The launch of Apple Vision Pro with its emphasis on immersive experiences has set a new benchmark for spatial audio integration in mixed reality headsets. This indicates a growing trend of spatial audio being a core component of future extended reality devices.
Artificial Reality (AR) music apps are also on the rise, one example being Gorillaz’ Mixed Reality app that launched in 2017. The app superimposed elements from their music videos onto the user’s surroundings.
In today’s market, you’ll find apps linking your location to relevant music («Music Everywhere»), apps generating holograms of artists allowing the user to create their own custom music videos («Jadu»), and many other variations of apps creating AR environments «in synch» with the music.
Artists and researchers are also exploring using sensors and biofeedback to create music that responds to the listener’s physiological and emotional states. These sensors monitor physiological data like heart rate and brainwaves, creating music that responds and adapts to the listener’s emotional state.
What does more immersive MusicTech mean for Artists?
When listening habits change, so will production and performance praxis. From the birth of the recording industry in 1909 (with the release of the «Nutcracker Suite» as a quadruple 78 disc package), up until the 21st century, the technological development was in the hands of the industry, the performers, the instrument-makers, the labels, the vinyl plants, etc.etc.
What we are experiencing in this century is a situation where artists have to respond to technology less than create and explore it creatively. However, we often see artists who are in a privileged position to be early adopters of technology like NFTs, virtual concerts etc. gain a huge monetary, as well as name recognition, momentum.
In this context, artists can perform in virtual worlds, inviting fans from all over the world to attend, and AR can also overlay digital elements onto real-world concerts, enhancing the visual and auditory experience. Multi-sensory concerts could easily be labeled «the future of concerts», but it’s not like your average indie band will put on a «multi-sensory experience» at your local club anytime soon.
Another interactive concept is biofeedback-based installations. Artists are exploring the use of biofeedback in interactive installations, where the audience’s physiological responses influence the music and visuals in real-time.
Some of these concepts for immersive music seem far-fetched or resource-heavy for the average artist. However, two take-aways from this field that are relevant to all, music lovers are getting used to more immersive experiences, and listening through a spatial audio set-up is becoming increasingly commonplace.
The latter influences how albums are produced and will be mixed as well.
Mixing an Album in 3D
First order of business is to make sure your Digital Audio Workstation (DAW) supports Dolby Atmos and compatible monitoring equipment. This will allow you to place instruments and vocals precisely within the 3D space.
The second challenge is to think beyond left-right panning and to get into a
360-degree sound field, including height channels. One common technique is to employ «object-based mixing» where each sound element is treated as an independent object that can be moved and manipulated within the space.
A cool effect of this is that there is more depth to the aural field while somewhat leveling the «sound hierarchy» of traditional mixing. Everyone who has ever mixed a track has felt the frustration that some of the gems have been «buried in the mix», sacrificed on the “altar of better clarity.» On that note, use reverb and spatial effects carefully to enhance the sense of space and create a realistic aural space that is not «ping-ponging» sounds into the stratosphere.
Billie Eilish’s album «Happier Than Ever» was recently released in Dolby Atmos, and The Weeknd’s album “After Hours” is another example of a mainstream pop album that has been custom mixed for spatial audio. And, even the Beatles are on the «spatial bandwagon» with a Dolby Atmos mix of «Abbey Road»!
Outro
Overall, spatial audio and immersive music are rapidly evolving, shaping the future of how we experience and interact with sound. The increased accessibility, technological advancements, and wider industry adoption indicate a future where immersive audio experiences become the norm.
Musicians should look at this trend as an opportunity to experiment. New innovation equals opportunities for new sound. Artists could choose to go full-on meta-verse and aim to create music for virtual environments or embrace the interactive and immersive characteristics in a more physical setting, with live experiences offering mixed-media concepts. And, like the Flaming Lips demonstrated almost 30 years ago, you don’t necessarily need the most high-end gear if you add imagination to the mix!
Bands, labels, and artists can also become more creative as regards partnerships and take the idea of interactivity and run with it to other creative corners and professionals in the world of sound design, engineering, and multi-media expression.
