The best of the Craft Track

A look inside Aeronaut VR: the tech

The creators: Isobar and Viacom
Recognition: Winner of this year’s Cannes Lions Grand Prix for Digital Craft
The tech: A VR music experience featuring Smashing Pumpkin’s frontman, Billy Corgan was created using:

• 106 cameras shot in VR with Microsoft’s Mixed Reality Capture Studio to record and produce a photorealistic volumetric capture
• Google Tilt Brush for all the animations in the endlessly expanding lake where Corgan plays, created by VR artist Danny Bittman
• Unity to combine all the components in one coherent experience

An orchestra of digital artistry

The creators brought the pieces of Aeronaut together in a way that thrusts the user into a stunning experience that points at the amazing potential that digital technology holds for art.

The real-time revolution: experiences versus stories

Creating worlds in VR experiences is quite different than in movies. Like movies, VR experiences have a beginning, middle, and end, but each person who goes through them sees different things – based on what they do or where they look and in what order. When making these pieces, creators need to see how all the different components come together in real-time to deliver the experience. The audience, on the other hand, gets real-time feedback to their actions, which determines the path their story takes.

A new medium, yes, but with precedents you can learn from

The question for creators then becomes: How do you create content for a real-time world? Our answer is: Set it up like a game, the original real-time storytelling medium.

• Set up who the user is: passive observer or a main character
• Set up their goal for the experience: explore or accomplish certain tasks
• Build the supporting characters: who guides them on the journey
• Create the environment: what is their perspective
• Storyboard their emotional journey: what visuals, sounds, and cues evoke that path

Cost of production

Creating non-linear content is comprised of two key components: The assets and the production.

What you need to consider regarding assets

Assets in Virtual and Augmented reality need to be in 3D. This extra dimension adds a lot more weight, detail, and complexity, which you need to take into consideration. These are a number of ways you can produce or source 3D assets:

• Use model objects or characters from tools like Maia
  This is usually quite time-intensive, and you need to balance the need for detail with file size, for example, when working with textures, which add depth and realism to objects.
• Take volumetric video or photos of real objects and characters
  Video is of course much more complex, and creators therefore sometimes choose to purposefully attain a more retro or simplified look. 3D scans of real objects are a lot faster, but depending on the source, they can also lack detail. Lighting choices can help obscure these issues.
• Utilise existing assets – e.g. from the Unity Asset Store
  These ready-to-use 3D assets allow you to build worlds a lot faster by focusing primarily on your iconic objects and characters.

VR and AR production tools and resources

Production for virtual and augmented reality requires the right tools and technical knowledge. A 3D content creation engine like Unity can really speed up the process. You can use Unity to carry out all of the following processes efficiently in real-time:

• Import assets and place them in a 3D virtual world
• Move assets and determine their reactions based on user input
• Make directorial decisions like lighting and camera perspective
• Render 2D movies in real-time by recording the 3D movement
• Build once and deploy across multiple platforms

The art: a look inside Coco VR

The creators: Magnopus and Pixar Animation Studios
Recognition: Silver Lion in the Digital Craft category; nominated for an Emmy this year in the Outstanding Original Interactive Program category
The art: In order to bring the detailed, extravagant and brilliant visualizations of the Land of the Dead from Disney/Pixar's Coco to life in VR, Magnopus cultivated new techniques that blended a variety of 360-degree projections for the environment.

The content is created, now what? Distribution and reach

While they may seem very different, AR, VR, and 360 have a lot in common when it comes to creating content for each medium. Where they vary wildly is distribution and reach.

Augmented Reality

Integrates digital information into a user’s environment in real-time using a device. While AR can also exist in a headset (e.g. Google Glass), most AR today is done on smartphones. And here the device penetration is massive.

Virtual Reality

Immerses the user into a virtual environment with non-linear storytelling. VR can only exist in a headset, and today there are several on the market with more portable, more affordable versions, coming out every few months.

How to reach users with the relevant devices

Just because a user has the right device does not mean that they will automatically see your content. The most common and flexible way to get content in front of users is to build it into a standalone app or experience.

• AR and 360 video apps: Distributed in the iOS App and Google Play Stores. Discoverability can be heightened if you receive a promotion from the stores and run your own app-download campaigns.
• VR apps: Distributed by any of the following providers. Same considerations apply for discoverability.
  • Google Play for Cardboard and Daydream apps
  • Oculus Store for Oculus Rift and Samsung VR apps
  • Steam for HTC Vive apps

Events, storefronts, or live installations also provide an excellent way to promote app usage, especially for VR where users will often stand in line just to try the hardware due to its scarcity.

How do you promote long-form VR and AR? With short-form VR and AR

The winning pieces of 2018 we’ve been highlighting here are full-length pieces of VR content, ranging from 5 to 24 minutes. But just like every movie has a trailer, it’s possible to make short-form VR, AR, and 360 content either by extracting it from the full piece or as a standalone project.

The benefits? Distribution! Unlike apps and experiences, short-form content can be distributed with minimal or no friction to users in places that already capture their attention.

The pitfalls? Length and weight. Since this new content has to be distributed as part of existing, heavier content, it has to be short and lightweight enough to fit the medium.

AR and 360 short-form experiences can be distributed via existing content – i.e. websites or existing apps. YouTube and Facebook both provide platforms for distributing 360 content. Unity is also the first provider to distribute true AR, utilising ARKit and ARCore, content within existing AR and non-AR apps.

• Length: 30 seconds
• Weight: 5-10 MB

VR short-form experiences can also be distributed via portals or existing apps. For example, Unity ran a pilot program with Lionsgate Jigsaw, where an interactive VR branded experience was integrated as a Unity scene into existing Unity apps – including the Samsung Internet for VR app which provides a hub for 360 and VR content for Gear users.

• Length: 1-2 minutes
• Weight: Under 100 GB

User immersion from 0 to 60 in 3s

Due to their interactive nature, AR and VR usually require a user tutorial at the beginning of the content. This can be one of the most daunting parts of creating short-form immersive content – getting the user in the action while still having enough time to tell your story. Best practices for short-form content flow:

1. Use 2D for informational screens and UI
2. Rely on actions that users often do in the real world – e.g. walk around, zoom, rotate objects, take photos
3. Ensure the experience can play through without any user input. Even if they do nothing, they will feel like they’re in the action. But their actions should not be required to complete their emotional journey and get the message.