How to prepare for virtual reality development
Virtual reality (VR) has been getting a lot of publicity, particularly for games. Over the last couple of years, developers have explored ways to develop 3D applications — called augmented reality (AR) and mixed reality (MR) — that provide an overlay on the real world. The whole ensemble of these various types of 3D applications is starting to be referred to as extended reality (XR).
Hardware is becoming more affordable and XR capabilities are being baked into Android and iOS development frameworks. Enterprises are starting to take note as these techniques promise to streamline many industrial processes. For example, IBM has recently partnered with Unity to improve tooling for enterprise app developers.
Most recently, Magic Leap has been getting a lot of attention from the release of its new display, which presents an AR overlay in its display that emulates the depth information in the real world. If this technique plays out well, it may mean that enterprise XR apps might overcome one of the last hurdles to simulator sickness.
We caught up with Katie Stern, general manager of the Game Developers Conference to find out what is in store for developers curious about building AR/VR/MR apps. The landscape of 3D app development has changed so much, the company now calls its VR focused conference XRDC. The conference recently published a survey looking at some of the trends in this field.
What are the biggest challenges in developing AR/VR/MR apps compared to mobile or web apps?
Katie Stern: These are difficult to compare, but the challenges of XR, in general, are wide-ranging. For one, it’s still a relatively new field, so a lot of the applications are experimental. We’re not sure what real world, mass market, practical XR looks like yet. And as this new generation of XR is in its infancy, technical and design challenges abound: there are locomotion and motion sickness issues, hand tracking and controller challenges, and UI problems to be solved. That’s not even getting into challenges you come across when doing business in a fledgling market. It’s still such a new space — it’s hard to know where to even begin in terms of “challenges.” Everything is a challenge!
What are some of the big differences in developing AR versus VR and MR apps?
Stern: While we tend to lump these together, they do have notable differences. AR involves placing a digital layer on top of reality; MR is an offshoot of AR where objects — say, a basketball — is digitally inserted into a real environment and can respond to that environment; VR, of course, totally blocks out the real world to create a wholly separate digital environment. Understanding the differences between various forms of XR is important in identifying their distinct challenges when designing their respective applications, as they each involve varying levels of user immersion. A VR game is highly immersive, Pokémon Go on your phone less-so. Long story short, the big differences tend to have to do with how people use different kinds of XR. The challenge is designing with humans in mind in order to create the most intuitive experiences.
Why did you change the name of the conference to XRDC?
Stern: XRDC began as the Virtual Reality Developers Conference (VRDC), held alongside GDC in March 2016. To meet the growing demand for high-quality content in the space, VRDC launched an expanded, standalone event later that year. Following five successful editions of VRDC, the next edition will be called XRDC, encompassing the rich diversity of immersive experiences using augmented, virtual, and mixed reality applications for innovative applications in a variety of industries.
What are some of the factors driving platform exclusive development?
Stern: One reason is funding. If Oculus says, “We’ll give you $X so we can say your game is a Rift exclusive,” studios will go that route, because there’s less financial risk. Other than that, there’s also the issue that a lot of developers making games in XR are small with limited resources, and targeting one platform is more manageable than targeting multiple platforms.
What big changes have you observed in the development of enterprise-oriented AR/VR/MR?
Stern: AR and MR in particular are turning out to be useful in terms of helping employees become more efficient and helping companies sell and market their products and services. The Ikea AR app lets customers drop virtual furniture into a room, some factories use AR for data visualization to help workers identify their surroundings accurately. Surgeons use VR to practice new techniques. Everything is happening incrementally right now. There aren’t big changes happening, but the industry is definitely trending more towards highly practical applications that will work in the real world and benefit a lot of people.
What are the top unsolved problems today and what technological and business advances do you see emerging to help address them?
Stern: The challenges are broad, and so are the solutions. Perhaps the biggest challenge is related to hardware — making XR hardware that is affordable, practical, and that is highly user-friendly is difficult. Aside from smartphones, XR hardware is generally expensive, bulky, difficult to use, and unattractive. Those are not the traits of a mass market product. However, companies are working every day to shrink down these devices, trimming costs, and working toward mass market appeal. It’s just a matter of time before we see mass market XR devices, and once this hardware clicks with a larger audience, the business will be more viable, and we’ll see more developers making new and interesting applications.
How do you expect the field of AR/VR/XR to evolve over the next year?
Stern: We’re going to be seeing interesting smartphone-based AR applications that are increasingly entertaining and useful, Magic Leap One is going to launch and it will give developers new inspiration as to the possibilities of MR, and VR will be showing up at more arcades and location-based installments. Exciting stuff!