Building VR software is not so straightforward. There are different variables, e.g., platforms, operating systems, devices, etc., to consider. To add context, if you decide you want to create a VR app or experience, you have to figure out what platform you want your users to have the experience on, whether VR headset or mobile device. Even at that, there are platforms within platforms; if they’re viewing it on a VR headset, is it as a native app developed specifically for the headset? Or is it a Web based VR app via the headsets browser? What’s their internet speed like? Answering these questions will give you the clarity you need to start development.
That said, there are 3 central aspects of VR app development. They are:
- SDK
- Game Engine
- VR Content Creation
Let’s take a deeper dive.
- SDK
You can think of the SDK as a kit containing things like code libraries and tutorials, that allows you to connect your VR item, whether it’s an experience or a virtual object, to a platform or device. To illustrate, if you build a game in a game engine and want it to play on the Meta Quest 3 and have access to its features, you’d need the Oculus SDK. If you then decide you also want it on an HTC headset, you’d use a STEAMVR SDK.
Choosing which SDK to use depends on which game engine you use to build your app, which in turn depends on the platform you’re building on and finally, depends on the audience you’re building for. Revisiting the example above, if you wanted to target Apple Vision Pro users instead, you’d need the visionOS development kit, which limits your development options to just the Unity game engine.
Hence, many developers target users via Web XR (the API that allows VR experiences on web browsers) because all VR devices can access the web.
We’ll do a write up on SDKs soon, this piece is just a general overview.
- Game Engine
If the SDK connects the VR experience to the hardware, the game engine is where most of the building takes place. Game engines, like SDKs, are tools containing code libraries, blueprints, builds, and so on, to help you create a virtual reality environment. This youtube video about Unreal, one of the most popular game engines, is a short and excellent illustration of how game engines work.
SDKs focus on hardware support, game engines focus on things like physics, landscape, characters, and rules of the VR environment. Basically the game engine is what ensures that virtual objects behave like their physical counterparts - for example, if you push a ball in a virtual environment the game engine makes sure the ball is round and that it rolls. That said, some SDKs offer extra physics and 3D support as well but that function is mainly down to game engines.
The most popular game engines are Unity and Unreal, both platforms offer support for multiple platforms and work with a wide range of SDKs. Unity even has its own SDK called the XR Interaction Toolkit. Both engines have their pros and cons, for example Unity is easier to learn, while Unreal offers more realistic 3D graphics. Your choice will be based on a mix of project goals and familiarity with the software.
NOTE: Different game engines work with different SDKs, for example, Meta has two separate SDKs for working with experiences created on Unity and Unreal respectively. Thus the SDKs you use will be determined by the game engine you use to build your VR experience.
- VR Content Creation
VR content creation somewhat overlaps with choosing a game engine in the sense that game engines provide pre-built 3D models and environments, however, these offerings can be limited and newer models or environments may be required. That said, not all VR experiences require 3D modeling software, content like virtual tours require panoramic video which can only be captured with special types of cameras.
The two categories of tools for VR content creation are 3D software and 360 cameras. 3D modeling software like Blender, Maya and 3DS Max allow artists to create realistic 3D models and environments which can then be imported into a game engine for building. 360 cameras are used to shoot videos/pictures that capture the entire environment without having to move the camera; these videos/pictures can then be imported into a VR headset or any supported device and used to explore the environment that was captured. Examples of 360 cameras include the Insta360 X3 and the Insta360 ONE RS 1-Inch 360 Edition.
Conclusion
As seen, there are many tools for producing VR software. This article was a rundown of the 3 major components that must be present in any VR development venture. We’ll release a series of articles covering the tech stack for AR and MR and will also release articles doing deeper dives on the tools in each tech stack. Till next time, we remain your go to source for everything XR related.
