The future of experience: Immersive

In my last piece, I introduced our four Genuine X Innovation Platforms: Immersive, Responsive, Curated and Connected.

• Immersive recasts the physical and virtual environment to create brave new worlds.
• Responsive anticipates user needs through experiences that sense, measure, and react.
• Curated responsibly harnesses data to create hyperpersonalised, ultra-relevant experiences.
• Connected creates compelling, consistent and continuous experiences across brand ecosystems.

In this article, my focus is on Immersive. We are entering a new era of technology-driven storytelling, where emerging mediums are blending together to create new hybrid platforms. The boundaries between creatives, technologists, makers and storytellers are disappearing, creating a culture of experimentation in which the rules are constantly being rewritten. Virtual worlds are taking on new levels of multisensory fidelity, through full-body haptics and sound. ‘Ordinary’ surfaces are being reimagined as interactive canvases for content that comes alive at a touch. 5G will lay the foundations for the ‘mirrorworld’ – futurist Kevin Kelly’s term for a digital mirror of our reality, teeming with data.

These developments will create new paradigms for how we learn, play, connect and live. We will move away from flat, static environments and the tyranny of the screen to embrace a new experiential age: embodied, emotive, and populated with a cascade of unique and memorable experiences.

We will look back on this era as a golden age, when everything was open and anything was possible. We’re here as your guide to creating experiences and stories for the brands of the future.

Throughout the rest of this paper, we will be exploring three ‘landscape shifts’ – tectonic, global disruptions to shape the new Immersive age. Each landscape shift is in turn made up of three ‘realisations’ – learnings that will help inform and define audience expectations in this new era.

Three landscape shifts

1. Shaping the immersive economy

Cutting-edge developments in the fields of AR, VR, MR – and the ways in which they’re merging to create ever more fluid, hybrid experiences.

2. Building dynamic sensory worlds

Harnessing developments in tech-enabled multisensory design to create new levels of immersion for audiences.

3. Connecting content and culture

Creating emotionally powerful content that attains cultural relevance in an accelerating, multiplatform landscape.

Landscape shift 01: Shaping the immersive economy

The surge of excitement that accompanied the launch of the Oculus Rift and HTC Vive in 2016 was rapidly followed by a sobering realisation: the technology had some way to go before being ready for widespread adoption. However, cutting-edge developments in the fields of AR, VR, MR are leading to an increase in adoption from creators and storytellers, with the mediums themselves merging to create ever more fluid, hybrid experiences.

Realisation 01: Our brains encode VR as personal experience

Rumours about the death of VR may be greatly exaggerated, as new hardware is finally approaching the form, fidelity and ease of use necessary to break into the mass market. The Oculus Quest is due to launch in Spring 2019, and whilst it’s not as powerful as the Rift, it’s still set to offer a vision of virtual reality we’ve been collectively dreaming about since Stanley G. Weinbaum’s 1935 short story Pygmalion’s Spectacles: ‘Suppose that you could experience a story that was all about you, and you are in it. Would that be to make real a dream?’

Certainly, a dream real enough to remember, according to research commissioned by Miami’s Children’s Hospital. They found that those who are trained in VR retain up to 80% of that information a year later, compared to 10-20% of written information – when recall is tested after a few weeks. When we interviewed Jack Maddelena of VRCraftworks for our X Podcast, he told us that this is because we relate to VR as a personal experience. It is something that ‘happens’ to us, a lived experience of the kind our species has been remembering and learning from for hundreds of thousands of years. The written word is relatively young at approximately 5,000 years – and we’re not yet ‘trained’ to remember and encode it in the same way.

Whichever way you look at it, it’s time to pay attention to VR, as new form  factors, the proliferation of virtual arcades, and hyperreal hybrids create exciting new opportunities for businesses and storytellers alike.

Realisation 02: Hyperreality will unlock new levels of ‘presence’

‘Presence’ is an oft-used term in the world of VR, used to describe the varying degrees of ‘being-in-the-world’ that VR can induce. Greater presence means less awareness of being in a virtual construct, and therefore a more immersive, believable experience.

Cutting-edge developments in the fields of AR, VR, MR.

Iterative technological enhancements – such as incrementally higher-resolution screens that banish the so-called ‘screen door’ effect – are one way of creating a greater sense of presence. But ‘hyperreal’ experiences go one step further, creating an embodied experience where users can reach out and touch a virtual world that is superimposed upon the physical one. A physical set – bland and featureless IRL – becomes a virtual playground. Press a button to open a virtual spaceship door, and feel it move beneath your fingers. Take cover from enemy fire behind a stack of crates.

We are constantly testing and experimenting with the technologies that may comprise our hyperreal future. Some members of our team have been fortunate enough to wear the Teslasuit: a full-body haptic suit that brings to mind the lavish VR rigs of Ready Player One. Featuring haptic feedback, a motion capture and avatar system, climate control (10 °C to 40 °C) and biometric sensors, it is a startling leap forward that anticipates a liberating and completely untethered virtual experience. Scientific research is also uncovering ways of creating the sensation of weight when lifting virtual objects, by applying electrical currents that make the material more rigid – and harder to bend – at the joints.

This is a rapidly-evolving field, with technological innovations announced constantly. We’re excited to see how they develop – and to experience the new levels of presence and immersion they will enable.

Realisation 03: AR will paint the world with data

We’ve come a long way with AR in a few short years: Pokémon Go exploded onto the scene in 2016, Snap and Facebook introduced AR functionality in 2017, and now Google’s ARCore and Apple’s ARKit are powering the dramatic  ubiquity of AR on our mobile devices. According to Digi-Capital, VR experiences will be worth $15Bn by 2022 – a number dwarfed by the projected $90Bn global industry value of AR by the same year.

Futurist Charlie Fink predicts that in the next stage of AR’s evolution, ‘the world will be painted with data’. An invisible virtual layer will be revealed first through our phones, and then through head-mounted displays that integrate phone, audio and AI assistants, powered by gesture, voice and wearables. Whilst this total integration is years away, this virtual layer is something we can build and leverage now. Technology is moving forward at a startling pace. Geolocation means AR objects can be anchored in ‘physical’ places. Computer vision systems are evolving to detect surfaces and planes, but also people, animals, objects and more.

We’re striving to identify and develop use cases for emerging (and now established) technologies like AR that enliven experiences and tell more memorable stories. At this year’s Mobile World Congress, we created an AR world for one of our clients, in which 3D printed objects became triggers for compelling brand stories. We’re trialling an AR navigation system in our offices as we type. There’s also huge scope to integrate AR into existing solutions, as The New York Times did when they enabled users of their app to explore David Bowie’s costumes and visual legacy in AR. Coachella just debuted its first interactive AR stage, with space ships and meteors soaring above festival-goers.

The list goes on.

But perhaps the most exciting elements of AR are the ones we haven’t dared to dream up yet. We saw a keynote from a leading AR company in which they recreated a physical shop in AR, complete with all of the analogue banalities of a typical retail environment. Who wants to experience neatly folded clothes, shelving units and hangers, when there is a virtual metaverse to build?

A group of artists hinted at AR’s disruptive potential when they created a ‘takeover’ of MOMA New York’s Jackson Pollock room, turning it into an AR playground that featured their own works. In order to embrace these emerging mediums, we need to untether ourselves from the design restraints of the  physical world. Fink predicts that ‘Soon, ‘Augmented Reality’ will sound a lot like ‘the world wide web’. The world is about to be painted with data. The canvas is as broad as the physical world, and blank as a fresh sheet of paper.’

Landscape shift 02: Building dynamic sensory worlds

For a long time, our physical experience of moving through brand environments has been stuck in a kind of stasis. Colossal screens dominate the environment, audio is added often as an afterthought, and the ‘interactive’ layer is treated as an optional bolt-on. It certainly doesn’t help that physical spaces are designed on flat computer screens that turn the process into a visual journey. ‘Vision,’ writes Finnish architect Juhani Pallasmaa, ‘separates us from the world whereas the other senses unite us with it.’ We need to reconnect with our common senses in both the real and virtual environments, using technology as a tool to enhance our feeling of being in the world, not detract from it. By harnessing data and technology we can create a new form of multisensory design and new levels of immersion and participation for audiences. Regardless of whether you’re creating an exhibition or an exotic virtual world, designing for the senses creates memories by rooting people in their bodies – by making them present and aware in the ‘here and now’.

Realisation 01: We are experiencing a spatial sound revolution

Sound is one of our most fundamental human senses, but it’s often taken for granted when it comes to creating brand experiences and virtual worlds where visual cues have typically dominated the approach. But we’re on the brink of a sonic revolution thanks to the emergence of ‘spatial sound’ – a technology that transforms the flat, canned audio we’re unconsciously accustomed to into a rich, 3D soundscape.

This is a world of sound that you can physically move through, in real life or virtual reality. Imagine being able to ‘walk’ around a virtual orchestra to hear how your position affected your experience. Or step into a piece of immersive theatre and hear the sound of the sea grow Landscape shift 02: Building dynamic sensory worlds louder when you approach an open ‘window’. Whatever the application, spatial sound is set to greatly increase our sense of immersion and ‘presence’ in the worlds we create, and could be the key to our brains accepting the reality of virtual environments.

To help power this new sonic boom, we’re collaborating with the academics and students at York University’s Immersive Narrative facility to unlock how this exciting, and often undervalued, medium can help to transform experiences.

Want to know more? Listen to the Episode 2 of the X Podcast, where we speak to Gareth and Jon from MagicBeans about their ground-breaking work in this area.

Realisation 02: Tactile interfaces merge the digital & physical

We are a society addicted to big screens. But screens often have an uneasy relationship with the physical environment around them, breaking up warm and carefully considered spaces with their harsh rectangular glare. They can struggle with people too, demanding to have their glass surfaces jabbed by human fingers, only to dully respond at a far slower pace than their sleek mobile counterparts.

These screens are a part of an ‘interactive’ or ‘content’ layer that is usually considered as distinctive from the environment – a bolt-on that sits on top of the space, rather than being seamlessly integrated into it. But this way of thinking is being challenged by the emergence of capacitive, touch-based technologies that mean that every surface has the potential to become an interface or tell a story.

These products are being pioneered by companies like Bare Conductive, a material science and smart surface company. We interviewed their Head of Technology, Stefan Smith, for our X Podcast series. Bare Conductive’s core product is a pot of paint that conducts electricity, and which can be applied to most surfaces to create interactive circuits that come to life at a touch. The product is a creative catalyst that has been used by engineers, inventors and creatives worldwide to create interactive art, theatre sets, and user interfaces, activating everything from animation and soundscapes to lamps and motors with the utmost simplicity.

These tactile interfaces create an intuitive canvas for immersive experiences. Capacitive surfaces can also be engineered to trigger projected content – the principle behind the extraordinary touch wall in our X Lab, a wooden surface that comes to life with bespoke content to unpack the Jack Morton brand story. Touch the volcano to cause it to erupt. Blow into the circle to see projected bubbles stream across the surface. Simple principles that can be scaled to create immersive, projection-mapped environments that transform at the touch of a, well, anything but a button.

These technologies merge the physical and digital worlds to create a sense of a magic, and to reintroduce a sense of discovery and play to the way that we immerse people in brand stories. In our work with our clients we seek to continue to push the possibilities of inventions like these, discovering new use cases and new opportunities for memorable storytelling.

Realisation 03: Experiential science unlocks psychological spaces

The colour blue can boost our creativity, according to a report from the University of British Columbia. Participants were asked to list creative uses for an ordinary object, and generated significantly higher scores in blue rooms. This led American author (and trained neuroscientist) Jonah Lehrer to speculate about the creation of ‘thinking rooms’: spaces specifically designed to facilitate a certain cognitive function, in this case ‘blue-sky’ thinking.

This opens up an interesting avenue of thought. How else might we harness emerging research to create new kinds of immersive, psychologically-attuned spaces?

New pieces of the multisensory code are constantly falling into place, and can be utilised to induce immersion and enhance engagement. Rosemary boosts recall. Birdsong can help us to concentrate. Different wavelengths and intensities of light can energise an audience or enhance their focus. The field of embodied cognition is expanding in influence, broadening our understanding of how we don’t just ‘think’ with our brains, but with our bodies, with a wealth of studies informing our understanding of how textures, materials and sensations can communicate abstract values at a deeply intuitive level. Scientists recently analysed Cirque du Soleil’s performances to see if neuroscience could unlock the art of awe.

By combining this multisensory code with a ‘data layer’, we will be able to orchestrate responsive spaces that become what people need, when they need them. What if a space could learn what environmental settings make you most effective at certain tasks – and calibrate its light levels, sound design, and digital vistas accordingly?

The design of these spaces will be driven by a new breed of cross-disciplinary experts and designers, who will synthesise this research to create an ‘experiential science’. Their insights will not only help to justify our design decisions, but feed new creative possibilities, and new kinds of spaces – both physical and virtual.

Landscape shift 03: Connecting content and culture

How do you get people to engage with your brand when more content is produced every day than a single person could consume in their entire lifetime? When people consume one billion hours of YouTube content every day – but the algorithm favours user-generated content? When Netflix created 700 original shows in 2018? And when 5% of the content created gets 90% of our attention? The answer lies in the creation of culturally relevant stories: the currency of our experience, the enduring vehicle for captivating attention in a loud and saturated world.

Realisation 01: Be the program, not the adbreak

A common refrain in the X Studio is ‘Be the program, not the adbreak.’ This means creating something that people want to actually watch in its own right – rather than the unskippable ‘branded content’ people are forced to endure. This is an ambition that needs to be defined at the beginning of project: the ambition to create something that is respectful of people’s time and which contributes to the ever-shifting landscape of knowledge, beliefs and ideas that populate our culture.

One way to achieve this is to collaborate and co-create with people who already craft content for the audiences you want to reach, and who will help define a powerful narrative approach or distinctive visual style. People who have made adverts in the past will make adverts in the future, whereas people who have a history in TV are more likely to drive content that could get picked up by the BBC, or Netflix, or The Verge – and which will generate broader relevance and traction.

Distribution and channel planning also come into play. Yes, you could spend all of your media budget on YouTube ads – but it might be better to create content that can be distributed via a platform that champions short films, or a guest spot on a podcast that is relevant to your audience.

Our new X Podcast series is a good example of how we have created programming to encourage our audiences to spend more time with us compared to our social channels or website. Not only that, they provide a greater level of value for the people that matter most to us.

Realisation 02: Real-life is the foundation for great content

Too many live experiences are filmed as an afterthought, creating content that might look good on the agency showreel, but which isn’t watchable for a broader audience. However, live brand experiences can be a powerful platform for watching video content – but this content needs to be carefully considered and baked into the experience itself.

‘Live from your event’ is the most used form of experience content and again we should take cues from TV. But the key lesson is that getting a host is not enough. You need a writer, an editor and a director who have experience in the field to avoid a very dry video and instead, create watchable experience content that immerses a much wider audience in your live brand experience.

We create experiences that become a playground for audience content, in which attendees become the stars in your video. Think like a reality TV producer and harness the power of real people interacting with your brand. Think Gogglebox, X-Factor, Love Island. Reality TV has dominated the ratings for twenty years, and it can work for a brand experience too.

Realisation 03: 5g will unlock the internet of experience

2019 is the year that 5G will be unleashed upon the world. Yes, this will provide transformational speed boosts for consumers – with an HD film downloading in 20 seconds, not 30 minutes. But its real significance lies in the foundation 5G provides for a new age of immersive content. Current testing and research shows that 5G will be able to support up to 1000 additional devices per metre (yes, you read that right), and everyday download speeds will be in the region of 100MB per second, compared to 10MB per second with 4G. (5G at peak speeds can achieve a massive 20GB per second.) These kinds of speeds and capabilities will ultimately replace the ‘representational media’ we have today, with the ‘experiential media’ of the future.

That’s according to Toshi Hoo, director of the IFTF’s Emerging Media Lab, who anticipates the emergence of the ‘internet of experience’. In this future, flat & static content will be replaced with an internet that you can touch, smell and experience. Hoo predicts this change will be in effect come 2030 – but the deployment of 5G may see the first wave of this new era arrive much sooner.

5G is already unlocking exciting new experiences where creativity is no longer hamstrung by technical limitations. London Fashion Week saw the first 5G-enabled mixed reality catwalk show. Sports fans will be able to see player stats and watch instant AR replays of goals from their stadium seats. And full-blown, immersive experiences could be streamed from the cloud, with no latency, and experienced on consumers’ headsets and smartphones.

We hope you found X Paper 02 informative, thought-provoking and exciting. Keep your eyes peeled for future X articles that explore how innovation in content, technology and data is transforming the evolving brand experience landscape.