#aNewLeaf #13 -The First Look at Digital Realities [ VR, AR, MR ]
This aNewLeaf #13 (a New Learning from this week) is special, because here we are talking about the technology which is going to connect society and technology in reality, with the title “The First Look at Digital Realities[ VR, AR, MR, XR ]”.
Digital Realities is growing significantly in many industries today to bring the great user reality. And this articles converse the pros, cons, and overall differences between augmented, virtual, and mixed reality, along with an overview of their respective architect and developer tools.
By end of this document we will able to have clear understanding on the strengths and weakness of digital realities. The Practice of Digital Realities can be focused in the area of three pillars like VR, AR, and MR. And MR will be having most impact by using the tools like C#, Unity 3D, Blender, HoloLens, and Visual Studio.
What is Digital Reality ? Is it VR + AR + MR = Digital Realities ?
There is lot of buzz around virtual, augmented and the last one on mixed reality and of course all the description have virtual reality at core. Top Industries like Education, Engineering, Healthcare, and, Entertainment are exploring and experiencing the fullest of Digital Realities.
Let’s first understand the VR in the past, the 1st virtual reality came into experience in 1837 by Charles Wheatstone for the Stereoscopic Image which can bring 2D and 3D brain. Then the Link Trainer comes into the picture as second VR experience, then the third one is Sensorama a fully experienced for the everyday person, which didn’t create impact in the market due to cost and size of the machine, but we lot of geeks wrote that it’s a great piece of technology.
Then let’s understand the Modern Era VR in the period of 1980s/1990s with bulky headsets, very low quality graphics. But today we application and instrument with smaller in size, better sound, and more of better graphics. So the aim of the Virtual Reality is to replace the users reality with a fabricated one. But with all the above we have few of concerns and challenges for virtual reality experience like no visibility to the real world, where user can’t see where they are going with clunky navigation and might be with motion sickness. So, to make the the VR to next level we need to smooth out and dealt with no visibility, clunky navigation, and motion sickness challenges. The ultimate aim of the VR is to immerse the user completely into this world that we are designing. And with the final note where VR excludes the real world completely, that put the path forward for Augmented Reality.
AR uses the real world as a background for the digital experience, generally augmented reality uses a trigger to start in particularly like shape or symbol to start which acts as point of reference for the entire user experience. The Augmented Reality experiment is first created in the 1990s in US Air Force to improve the human performance using fixtures. And today AR is much more approachable and much more portable, and most AR experiments and applications are done on mobile devices like marketing brochure when the user points the camera to particular point, or game of cards where the characters pop out of the cards. And today any devices with camera can be a potential to be an AR device, and may AR experiences are pair with a smartphone app to bring 360 degree and complete experience. With this information the industry and academia started many researches on future Augmented Reality can work and benefits the industries, not merely the entertainment. The main challenges of AR are Interaction with essentially an overlay of data, Capturing input from users is still a barrier, and the final challenges is educating the user of the consumer market about the potential of AR. But where as the VR is already in presence and most people understand the concept, but AR is put many times in the same category which is incorrect. So with the overlay of data, the user can get completely the different experience in the AR than from VR where the world is virtual.
The new kid in the digital reality ecosystem is Mixed Reality or MR, which is the merging of the real and virtual ( digital world) to produce a new environment and visualization in which the physical and digital objects are coexists and interact in real time. Where VR replaces the entire world of the user, and AR creates and overlay on the real world, but the MR takes one step further. Mixed Reality is the merging of real and virtual worlds to produce new environments and visualizations where physical and digital objects co-exist and interact in real time. It means the actual physical objects are being used in the experience, which makes the MR a unique. Implementing the spatial mapping requires customized and specific hardware that isn’t found in augmented or virtual reality by default. Spatial mapping is done by using environmental cameras like UV light empowered, or Infrared Light powered, and the spatial mapping is what makes or breaks the experience of the Mixed Reality, so we the architects, designer, and developers should bring the most of spatial mapping to the Mixed Reality applications must be convincing and true realistic. The use will not forget the first experience of true MR.
VR vs. AR vs. MR
We have many similarities such as digital 3D assets and use of the hardwares to project the experience, but we also have significant differences.
Development and Tooling:
Above we have discussed about the overview but let’s discuss about the developer and architect side of the Digital Reality App. We have many and significant reality tooling to make the apps great and immersive.
VR Dev Tooling are many in the market, and to list few Unity 3D, Oculus Rift, Google Daydream, Samsung Gear VR, HTC Vive, and Unity is very commonly used tool especially for hobby and very small projects, which is readily with even a free tier to explore the IDE with zero cost. Benefits of the Unity are we have great tutorials and designed around 2D and 3D gaming, also very active community and load with many asset stores. The programming languages used in the Unity 3D are the JavaScript called Unity Script, C# Scripting, Boo which is similar to Python is deprecated now. And other notable tool in the VR ecosystem called Unreal Engine which supports many platform from medium to large enterprise, which comes with the full scale editor that allows and even it had marketplace where we can acquire music, sound effects, animations and much more with the C# as an programming language. And last in the queue is Web VR which can be used in any modern web browser and programming language used are JavaScript and A-Frame which had native performance of rendering it where it can run very easily 90 frames per second in the modern browser with API features and support.
AR Tools uses the real world of their choice and it works everywhere and it should be adaptive anywhere with the surroundings. It uses the real world to overlay data on; so since the backdrop is with the real world data we no need to create many 3D modelling, which can be used very less 3D Modelling. The traditional augmented experience has three steps one with downloaded app, two with the trigger point and the final and three is with action. Apple ARKit and ARCore from Google are the two exciting kits to build the great AR porjects. This ARKit is totally depends on the TrueDepth camera and VIO ( Visual Inertial Odometry ) is excellent features which can be navigate purely using camera only and the ARCore uses SLAM ( Simultaneous localization and mapping) approach.
MR Tool uses mainly two ways to develop experiences of mixed reality apps called HoloLens Apps and Immersive Apps. Both have same development cycle but slightly a different versions of the tooling. Where VR relies on the designer and developer create the environment, and and AR uses the camera and motion detection to understand the users world. Where as MR is bit more complex and sophisticated, HoloLens especially builds up a 3D model of the real world using environmental cameras with that whole real world is mapped into a detailed 3D model made up of polygons and triangles, once we have the model then we can place the digital assets or holograms. Because of the extremely convincing MR is set apart from VR and AR. Both HoloLens and Immersive apps is done using Unity 3D. So, Unity 3D is used for most of the MR app creation journey with visual studio thrown into the mix of development cycle.
With the above we could able to understand the tooling choice for the digital realities, and as 3D is a large component in VR, AR, as well as MR and it’s very clear we need the 3D tool to create the experience. And overall there are only two choices left with to creating a digital reality experiences when it comes to tooling, they are Unreal Engine and Unity 3D. And if we need to create a MR or HoloLens app then we have only one option which is Unity.
Few tools to look for 3D modeling are Maya is great but expensive , Blender is open source tool including entire development flow like modeling, rigging, animation, simulation, rendering, composition, and immersion tracking with video editing and game creation. Blender is great tool must explore for for our digital realities road maps. Then coming to Sculptris which is very mature product and aimed at first time users in the 3D modeling. And if it is goes for SketchUp which will allows to build both 2D and 3D modeling and even it can be print and exported as pdf file.
Industry Implementations:
With the above detailed exploration of VR, AR, and MR and tools around the same. Let’s explore the true prospects of making an impact with any industry with the digital enhancement with the right idea and the right people.
So to conclude along with our technical geek around big data, cloud, ai, and blockchain. We need to focus on the digital realty too, by overlapping Virtual Reality, Augmented Reality and Mixed Reality to bring the customer with unique and personalized experience and take advantage of the spectrum of digital enhancement.
Appendix:
HoloLens in Healthcare & Pharmaceutical has great use case of visualising patient health data using HoloLens. Build an app to consume data from HealthVault by leveraging API.
Reference:
https://www.realitytechnologies.com/mixed-reality/
https://docs.microsoft.com/en-us/windows/mixed-reality/mixed-reality
https://en.wikipedia.org/wiki/Mixed_reality
Keep yourself up-to-the-minute on ABCDE of Data (Artificial Intelligence, Automation, Big Data, Blockchain, Cloud Computing, Collaborative Tech, Digital, Design Thinking, Edge Computing ) by following us on https://medium.com/dataottam Happy Learning, Happy Sharing !
See you in the next #aNewLeaf edition…..
