How to build VR experiences with a headless CMS: an interview with a developer

Virtual reality has been a hot topic in the tech industry for the past few years. Almost all types of businesses are interested in implementing this technology to create a "wow effect" on their audience. However, for many people, creating VR experiences remains a mystery and seems complicated and expensive.

In order to bust this myth, we interviewed Ilham Achmad Al-hafidz – a Virtual Reality developer who has already built multiple VR projects. We'll talk about his experience, the frameworks he uses, and how to manage VR content with a headless CMS. 

Let's get started.

What's your background in building VR apps?

I have over 2 years of experience in developing VR applications. I still remember my first project, which involved creating a digital twin of an arm robot. In this context, a digital twin refers to a virtual replica of the physical arm robot. This twin, existing in a virtual space, can mimic the actions and behaviors of the real robot, allowing for simulations, tests, and modifications to be performed in the virtual world before implementing them in reality. This ability to bridge the real world with the virtual one has extensive applications in different industries. 

Since then, I have developed more than 50 VR applications, spanning diverse domains such as training, simulation, medicine, gaming, manufacturing, industry, tourism, and meditation. Currently, I am working for a German AI company, managing their Sports Virtual AI Assistant project called "SAIVA Football."

How is content usually managed in VR?

When talking about VR development, managing content can be quite a complex task. Content within VR applications can take various forms, including text, images, audio, video, and 3D assets. The dynamic and immersive nature of VR applications requires careful organization and management.

Traditionally, content in VR apps can be managed in a hard-coded manner – meaning the content would be directly embedded into the application's code. While this approach can work, it can often be time-consuming and inflexible. Every time a change needs to be made to the content, the code may need to be adjusted, and the application recompiled, which could slow down the development process and make it harder to adapt to changes quickly.

CMSs, on the other hand, provide a more streamlined and efficient way to manage content in VR apps. Instead of hard-coding the content into the application, the content is stored separately and can be accessed and managed via an Application Programming Interface (API). This approach allows developers to update and modify the content without changing the application's code directly.

Not only does using a CMS make managing content more efficient, but it also opens up the possibility of collaborative work. Multiple team members can access and manage the content, making it easier to work on larger projects or coordinate efforts across different teams. This can lead to improved workflow, quicker turnaround times, and, ultimately, more effective and engaging VR applications.

What types of content work the best in VR?

In the realm of Virtual Reality (VR), content is the cornerstone of immersive experiences. VR can handle many content types, each contributing to VR's immersive environments and interactive experiences.

1. Text: While VR is primarily a visual medium, text still plays an important role, often providing contextual information, instructions, or narrative elements to the user. This can range from simple labels and descriptions to complex, story-driving dialogue.

2. Images: VR can utilize both 2D and 3D images to construct the visual aspect of the virtual environment. This includes background images, textures applied to 3D models, user interface elements, and other graphic assets contributing to the overall experience.

3. Audio: Sound is a critical component of VR experiences. It adds depth and realism to the virtual environment, helping to fully immerse users. This can include music, sound effects, ambient noise, and spoken dialogue.

4. Video: VR can incorporate both traditional 2D video and 360-degree video. These can be used for various applications, from playing a video on a virtual screen within the VR environment to creating immersive, spherical video experiences surrounding the user.

5. 3D Assets: These are the backbone of any VR experience. 3D assets are digital models that make up the "objects" you see in the virtual world. This can include everything from the environment itself (like buildings and landscapes) to interactive objects (like tools or vehicles) and even characters.

Thus, a CMS for VR needs to be capable of managing all these diverse types of content, as they each contribute to the overall VR experience in their own way.

Have you ever used a CMS to manage content in VR? If so, which ones?

I've used several CMSs depending on the project. 

For example, I've had the opportunity to work with an in-house built CMS. This CMS was explicitly designed to suit our unique requirements and specifications, providing a tailored solution to content management in our VR applications.

Beyond the corporate setting, I have also collaborated with my freelance team on various projects that demanded the usage of a CMS. We've utilized multiple systems depending on the specific project needs, constantly adapting and learning new platforms as necessary.

Among the numerous CMSs I have worked with, Storyblok holds a significant place. Storyblok is a headless CMS that provides the back-end content infrastructure while allowing us to build a unique front-end user experience. Its versatile nature allows for managing various forms of content within VR applications. It provides an interface to create, manage, and distribute content, making it a powerful tool for VR development. From text and images to complex 3D models and interactive elements, Storyblok has proven to be an effective solution in managing the diverse range of content we encounter in VR applications.

What's it like to use Storyblok as a CMS for VR?

Storyblok is an easy-to-use CMS, designed with user-friendliness in mind. One of its standout features is its intuitive user interface, which simplifies the process of managing and organizing diverse types of content. Whether you're handling text, images, audio, video, or even 3D assets, Storyblok offers streamlined solutions for content management.

One aspect that makes Storyblok particularly accessible is its comprehensive and user-friendly documentation. Whether you're a seasoned developer or a newcomer to the CMS world, the guidance provided by Storyblok's documentation can help you navigate its features and capabilities with ease. From setting up your project to implementing complex features, the documentation serves as a helpful manual every step of the way.

Additionally, Storyblok offers a straightforward setup process. It doesn't require extensive technical knowledge to get your CMS up and running. This ease of setup means you can quickly start creating, managing, and publishing content, saving time and effort.

Furthermore, Storyblok is designed to integrate seamlessly with various technologies and platforms, including VR applications. This compatibility further enhances its ease of use, as it allows you to manage your VR content efficiently without needing to constantly switch between different systems or tools.

Overall, Storyblok's simplicity, robust documentation, and effortless setup process make it an ideal choice for managing content across a wide range of applications, including VR.

How long did it take for you to figure out how to use Storyblok with VR?

Getting started with Storyblok in the context of Virtual Reality (VR) was a surprisingly swift process for me. Essentially, it took me just about an hour to familiarize myself with its capabilities, thanks to its highly accessible nature.

One of the standout attributes of Storyblok is its comprehensive documentation. This encompasses a wide range of topics, from simple setup instructions to detailed tutorials on more complex features. Not only is it thorough, but it's also written in a way that's easy to understand, even for those who might be new to using Content Management Systems (CMSs) or working in VR. This made the learning process much more efficient for me.

Furthermore, the user interface and design of Storyblok are quite intuitive. I found that the layout of tools and options was logical and easy to navigate, which helped me quickly understand the system's functions. The intuitive nature of the platform cuts down on the time required to get used to the system and increases productivity.

In addition to its user-friendliness, Storyblok also supports seamless integration with VR technologies. The platform allows for effortless management of VR content, which made my transition into using Storyblok within a VR context quite smooth.

In summary, Storyblok's comprehensive documentation, user-friendly design, and easy VR integration meant that it took me just an hour to grasp how to effectively utilize the platform in a VR setting.

How can companies get started with building VR experiences? Do you have any resources or tools to recommend?

Unity is a powerful game engine widely used for creating interactive content, including 3D games, simulations, and VR experiences. Its versatility, robust features, and user-friendly interface make it a popular choice among developers, both seasoned professionals and beginners.

One of the major strengths of Unity is its extensive Asset Store, a marketplace where developers can buy and sell various assets, tools, and scripts to aid in their development process. Within the Unity Asset Store, there are numerous VR frameworks available for use. These frameworks provide pre-built solutions and functionalities that significantly streamline the process of creating VR experiences.

For instance, a VR framework might include scripts for common VR interactions (like grabbing objects or teleporting), pre-configured settings optimized for VR, or even complete example projects that you can learn from. You can create a VR experience by leveraging these frameworks without writing all the code from scratch. This can be a massive time-saver, especially for those who may not have extensive coding knowledge.

Furthermore, Unity provides extensive documentation and a large online community, which can be incredibly helpful if you're just getting started or if you encounter any difficulties during development. This combination of a powerful game engine, readily available VR frameworks, and supportive resources makes Unity a viable option for creating VR experiences, regardless of your coding expertise level.

What does the partnership between Apple and Unity mean for the creation of AR/VR content experiences on the Apple Vision Pro?

As an AR/VR developer, the revelations made at WWDC23 were incredibly exciting. One of the most notable announcements was the demonstration of the Apple Vision Pro. This piece of technology surprised me with its advanced features specifically designed for AR and VR experiences, indicating a significant leap forward for the industry.

Even more thrilling was the announcement of a partnership between Apple and Unity, one of the most widely used game development platforms. This was not merely surprising; it was a moment of genuine joy. Unity is an essential tool for many AR/VR developers, including myself, due to its flexibility, user-friendly interface, and vast ecosystem of tools and assets. The prospect of an alliance between Apple and Unity promises an even more streamlined and enhanced development process.

With this partnership, it is expected that the Apple Vision Pro will feature high compatibility with Unity. This compatibility means that developers would have an easier time creating AR/VR experiences by leveraging the advanced features of the Apple Vision Pro within the familiar environment of Unity. Unity-based applications could benefit from features like pass-through and Dynamic Foveated Rendering, which can improve the performance and visual quality of VR experiences.

Moreover, developers could also tap into Unity's popular AR Foundation and XR Interaction Toolkit, making it easier to create interactive AR experiences. Such integration could make the development process more efficient and allow for the creation of more immersive and engaging AR/VR applications.

The integration between Unity's new PolySpatial technology and Apple's visionOS would potentially allow your apps to exist alongside other apps in a Shared Space. This means that multiple applications could interact within the same virtual environment, creating more cohesive and immersive user experiences.

Additionally, by combining Unity's powerful authoring and simulation capabilities with the rendering features of Apple's RealityKit, content created with Unity could feel right at home within the Apple ecosystem. This means that Unity-developed applications could take advantage of the aesthetic and performance optimizations of Apple's platforms, leading to more visually appealing and responsive apps.

In essence, this partnership between Apple and Unity promises a new horizon for AR/VR development, opening up opportunities for more streamlined development processes and the creation of high-quality, immersive applications.