XR encompasses augmented, virtual, and mixed reality technologies. This is an upcoming industry that may be good to get into. An XR Enthusiast can be of 2 types—
1. Developer
2. Designer
It depends on the enthusiast’s thought process.
In this article, I have enlisted some of the skills that will help you grow in XR, as well as serve as an answer for “How do I become an XR Developer?”
Like many other technologies, there is no perfect roadmap to learning XR. Nevertheless, I have tried my level best to come up with a generalized learning process for the same.
Feel free to ask questions in the comments below this article.
Skill 1 — Programming
Basic programming experience in languages like Java, C#, Swift, Javascript etc is essential for developing any piece of software. Get a good grasp on any one of these languages. Concepts like loops, control-logic, OOP principles like inheritance, abstraction, encapsulation, etc should be at your fingertips before moving forward.
If you already have a bachelors degree in Computer Science, you have a good foundation. If you don’t, then who cares? Learn programming from YouTube, Udemy, Udacity and a thousand other free platforms.
Skill 2 — Software Development
Once you are through with programming, your thinking will automatically shift towards building stuff rather than practicing problems.
3 types of Software Development are pretty mainstream —
1. Web Development
2. Mobile App Development
3. Game Development
You should have a basic idea of any one of them at least. There are tons of online resources to learn these.
Note that once you pickup a domain, you should stick to it for 6 months at least. That’s 6 months of committed learning.
There is no use of all that knowledge that you have accumulated in the past 6 months if you don’t apply it to a real world project. Therefore add to that another 6 months of applying your learning in a real world project through an internship or a job.
The more time you spend working on a real world project under an organization, the more you sharpen your domain knowledge.
Based on your learning pace you can shorten the 6 months learning process + 6 months real world project experience into a shorter period. I am just providing you with a rough estimate.
In these 6 months you are expected to know stuff like —
1. UI/UX Development based on a given mock i.e if you are given a design you are able to develop that design using code. Naturally, the code depends on the domain that you have chosen. For web development, you should be familiar with HTML, CSS & JS. Then you can dive into a UI Framework such as React.
2. Connecting your client side app to some API to show dynamic data using JSON Parsing techniques.
3. Adding 3rd party libraries so that you don’t have to reinvent the wheel. I am not pinpointing any particular library because that again depends on your domain and your app requirements.
Also, I am not diving deep into things like code architecture, testing, security and fixing memory leaks in your mobile apps. These skills are required to improve the quality of your app, as well as scaling it up.
If in the future, you are working on an XR application that requires reaching out to a billion users, then you cannot ignore these quality aspects. For developing small AR/VR prototypes, you can compromise on the quality aspect.
The more knowledge you have in your domain the more you can think about how to combine things and think out of the box. This is one of the many ways to flex your creative muscles.
Skill 3 — General XR Knowledge
Now it is time to explore XR. Get familiar with the basic terminologies of XR. Understand the constraints of this technology. The objective behind this step is to get you familiar with terminologies of XR and become comfortable with the words used loosely in the XR domain.
There are 3 domains that come under the umbrella of XR
1. AR i.e Augmented Reality
2. VR i.e Virtual Reality
3. MR i.e Mixed Reality
Here is a course from the University of Michigan. This should be a good starting point.
Developer Mindset
Most XR developers tend to use a pre-existing SDK to develop customized XR experiences for consumers.
For those who are interested in building custom SDKs and libraries that can help fellow XR developers to develop those customized XR experiences, should have —
Skill 4 — XR SDKs
SDKs like ARCore, ARKit, ARFoundation, Vuforia and Wikitude are well renowned for creating AR experiences for Mobile Platforms i.e Android & iOS. Apart from these are dozens of SDKs present. Depending upon your app’s feature requirements and your present code architecture, select an SDK and then continue with the development.
Most of the SDKs are still evolving. Once you start using some SDK, there is a high chance you will fall into problems that need more time investment from you, to solve those problems. That’s how the XR landscape is at the moment.
In VR, there is an SDK for each VR headset. The Oculus headsets have the Oculus SDK. The HTC Vive headsets have its own SDK. So again, it depends upon the headset that you are going to use for developing your VR experiences.
This course from the University of Michigan focuses mainly on developing XR apps with WebXR, Unity & Unreal.
Skill 5 — Machine Learning & Computer Vision
XR works on underlying Computer Vision Algorithms that are abstracted away in methods & functions of existing SDKs. There is a lot of research going on here in terms of optimizing plane searching, depth mapping, occlusion features,etc. These features can be incredibly tricky to develop. For some, it may even seem like reinventing the wheel.
Designer Mindset
Those who are interested in developing consumer facing XR applications and want their experiences to look better than others fall under this category.
It’s the same as saying, “Both A and B are note taking apps. I like App A more than App B because I feel it’s easier to use. I wonder why App B does not have this cool interaction that solves so-and-so problem.”
Likewise there are various design problems in XR Experiences.
If you are have a mobile development background, it is similar to getting the UI design mocks for your android app from your designer, rather than you designing and developing the screens yourself.
Skill 6 — UX Design Knowledge
If you are not that interested in making your XR experiences look beautiful or brainstorm on how to solve design problems, then skip to the next skill.
For the ones who are interested in transforming good XR experiences to great XR experiences by incorporating interactions, studying users and understanding where an XR experience falls short, then study UX Design first. Understand topics like interaction design principles, usability, design thinking process, storytelling, HCI, accessibility, etc.
Once you are done with UX Design, think how you can apply the design principles in XR.
Here is another course from the University of Michigan
Skill 7–3D Modelling & Animation
Tools like Blender, Maya and Cinema4D are great for prototyping experiences with animated 3D Models and environment designs. These prototypes serve as a good starting point for showing a concept to your potential clients.
Many a times, you will find yourself in a position where you are not being able to communicate your thought process to someone. If developers start working on something that does not match the requirements, it wastes precious time & energy. This is exactly why a prototype can prove incredibly helpful for the stakeholders of a project. Once the overall design is finalized then you can continue with the rest of the development.
Moreover, 3D models have to be optimized and should have certain file requirements before being added to any XR experience. Not doing so can cause performance issues. For example) Take a look at the file requirements for ARCore —
Using Scene Viewer to display interactive 3D models in AR from an Android app or browser
Interested in both XR Design and Development?
Skill 8— Graphics Rendering
Shader programming languages like Metal, HLSL and GLSL help developers to customize the look and feel of the models, animation effects and vfx in your XR experiences. This skill lies at the cross-intersection of design and development.
If you have done a Computer Graphics Course in your bachelors, you are at an advantage here. If not, then, you have a learning curve.
Here is an excellent YouTube playlist to help you, if you have no idea about Shader Development.
Anyone with the ability to think in terms of both Design and Development is a UX Engineer. There are multiple roles where design and development play a vital role. I have seen Creative Technologists, Technical Artists, AR/VR Developer, AR/VR Researcher, etc in many XR job postings. I believe the focus should be more on the skill set and the responsibilities that you get, rather than the name of the role itself.
Sidenote
There are so many platforms like Lens Studio and SparkAR that allows you to develop XR experiences without code, but there is a limit to the amount of customization. XR Experiences developed using these platforms run on their own platforms(Snapchat and Instagram) rather than your own platform. You won’t be able to embed the XR Experiences developed using these platforms in your own mobile app.
There are already a bunch of drag and drop platforms for Web Development and Mobile App Development. Why do you think developers are still writing code to develop web apps and mobile apps?
Soft Skills
Skill 9 — Social Media
I strongly believe it is good to always stay updated with the upcoming trends in whatever field you are interested in. Your social media feed shows you what you have liked in the past. If you could trick the algorithm to show you only XR stuff by liking, sharing and commenting only about XR content, then you are bound to absorb a lot of information revolving around XR. You get to know about events(which are mostly webinars, due to the pandemic these days!), courses, news, tips from industry experts, jobs, community competitions and much more.
Conclusion
The above skills should be enough to become an XR Designer or an XR Developer. I understand that the learning curve is steep, but if you are passionate about this field and ready to invest your time into it, then go ahead. XR is growing. It might take some time to become stable for end users, but as a technologist you can surely invest your weekends into XR.
