3D-Native Working
 |
| Image via wikia.com |
It hit me a while ago that even with VR, XR, AR, it seems that most applications revolve around a single application working space. Also, while there are many cases of users collaborating, again this seems to be within the context of a single application.
With the massive increase in cloud compute and especially edge compute, it seems that we are not that far away from being able to attempt true 3D-native multi-tasking.
Currently, our operating systems are very advanced in securely and efficiently handling the interoperation of multiple applications. And those applications themselves need not be processed locally (ok that's even older that multitasking, right?). But, ultimately, even 3D apps are for the most part rendered to a 2D workspace. And this dictates how we interact with our applications.
Think back to the Star Trek "holodeck". I don't mean just running a fully immersive simulation of a story for entertainment. I mean being able to call into existence some process, say, a simulation, then being able to verbally instruct the computer to insert that simulation within some environment to observe the effects. Being able to have processes dynamically created and cooperate, while being able to view from all angles.
This really isn't so far from what we do already. The two fundamental differences are:
- Being able to instruct operation verbally in natural language.
- Instead of processes rendering to a 2D window, transforming their presentation in real-time to the appropriate perspective of the viewer in 3D space.
Regarding the first one, after all that has happened in the past year, I don't think we can say there is a fundamental impossibility here! 😉
Regarding the second, the requirement for adding that extra dimension is the sheer compute power required to have multiple 3D processes, which also potentially need to render for multiple viewpoints (one for each participant in the workspace). However, with massive increase in the availability of compute power in the cloud and even on the edge, we need not have the work done locally by each client.
In addition, while having processes interact is a nominally case-by-case requirement, the equivalent of 3D primitives (I'm not sure about spatial primitive at the time of writing) has had some standardization recently which should help ease the process.
The NVIDIA Omniverse ecosystem looks very promising. Providing at-scale rendering, even on the edge, while working natively with the Univeral Scene Description specification, seems a solid foundation to build on.
I made a "back of a napkin" overview of what I was thinking about. It's nowhere near a proper HLD but I am very much looking forward to diving into the specifications to see what the possibilities are!
