Developing a Metaverse Strategy
Poor application performance has been one of the greatest limiting factors to significant consumer adoption of immersive Metaverse experiences. Depending on improvements in hardware alone is insufficient. Rethinking data architecture and using a data-oriented technology stack (DOTS) holds incredible potential for dramatic throughput and performance improvements that can scale to never before seen levels on hardware that exists today.
Launched in 2016, Unity Technology has been developing DOTS as an alternative to object-oriented design (OOD) for some time. While they have continued to develop improvements over the past 6 years, use in production has been slow in great part because of the technical learning curve. For technical teams with the skills, however, using DOTS creates a compelling competitive advantage for early adopters to implement advanced parallel processing without adding risk to core applications. Dimension X has developed expertise thanks to Scott Robertson, co-founder and CEO of Dimension X, who was an early DOTS developer participating as a client leader with strong ties into the Unity DOTS working team.
This post serves as an entry point for companies or developers incorporating this technology into projects to understand its value and take action.
DOTS background
DOTS is a Unity-specific implementation of Data-Oriented Design (DOD) that began in 2016. This stack consists of three core components plus a host of support packages that expand its feature set further:
- Job System lets you convert functions into highly performant and thread safe ”jobs” that can then be executed across all available threads on the host CPU.
- Burst Compiler is a compiler that translates from IL/.NET bytecode to highly optimized native code using LLVM. Burst is used in conjunction with Jobs to provide highly optimized vectorized assembly code capable of executing SIMD and MIMD operations across all available threads.
- Entity Component System (ECS) is the core of the new Unity DOTS workflow that includes 3 sub components:
- Entities – simple thread safe data structures, comprised primarily of pointers and identifiers, that represent all objects within your application’s “world”,
- Components – simple thread safe data structures stored in like type arrays that contain all of your entity data in fast native continuous memory blocks and
- Systems–the logic that transforms the component data from its current state to its next state through Bursted Jobs.
Additional Packages – Unity provides and readily releases new game packages routinely; providing additional functionality like Physics, Rendering. Beta and experimental packages.
The core Unity DOTS team regularly publishes roadmap updates including this Sept 2022 post from Laurent Gibert. We expect to hear more updates at Unity Unite, Nov 1-2, 2022.
Data Oriented Design (DOD) originally saw mass utilization starting in 2000 with the XBOX 360 and the Playstation 3. Born out of the need to dedicate more transistors to GPU over CPU, DOD made it possible to leverage parallel processing efficiently and accomplish complex workloads without increasing CPU costs.
Software programs have relied heavily on the single-core clock speeds of CPUs and often run into the Von Neumann Bottleneck, in which throughput is limited due to the relative ability of processors compared to top rates of data transfer. This bottleneck relates to the speed of memory access. Computer scientists have developed ways to increase the bandwidth of memory in and out of the processor including multi-threading and parallel processing. DOTS is one of these parallel processing strategies.
You can learn more from this Unity DOTS tutorial
DOTS’ relevance to the Metaverse?
The Metaverse is the next evolution of the internet providing additional layers of immersion and interaction across a spectrum of devices and environments from simple to complex including full 3D worlds with hundreds of thousands of real-time interactions between players, objects, and other Non-Playable Characters (NPC). Such environments require massive processing power that overwhelms all but the beefiest of today’s consumer computing devices. Meanwhile, innovators are pushing new boundaries in nanotechnology to create Metaverse devices in ever smaller form factors that unobtrusively fit into the real world. DOTS together with edge computing–placing cloud computational processing nearest to the user via high internet bandwidth speeds–are critical to reduce overall costs and to massive consumer adoption of Metaverse experiences.
Current web 2.0 interactive applications that don’t evolve along with these new technologies will find themselves displaced by others that do.
Important things to know about DOTS
DOTS shrinks the amount of code required to be processed in order to execute a given function and does so in parallel. Smaller instruction sets executed in parallel mean fewer cache misses, fewer CPU cycles required, faster processing times, faster data transfers, greater stability, more headroom for computing, more features, and more complex logic.
Is using DOTS an all-or-nothing gambit? No. It’s entirely possible to leverage DOD without having to commit to it fully. To manage risk, DOD should be implemented where heavy processing happens in every single frame or for multi-frame workloads that process mass quantities of data.
Even though the core technology stack continues to evolve and present some risk, the jobs system and burst compiler are very stable and can be used in production environments. ECS enables complex applications to be run on low-end devices. Also, ECS Entities 1.0 does not replace GameObjects.
In a world of rapidly rising costs and ongoing shortages, DOTS provides a path for more sustainable and efficient applications that can continue to keep pace with growing user demands without growing hardware and energy costs to match.
Applications using DOTS
- V Rising – A settlement building and survival game set in an open world. The player is a newly resurrected vampire that gathers materials to build a personal castle, defeats enemies to unlock new abilities and technologies and feeds on victims to maintain blood levels. Lead developers Rasmus Höök and Fredrik Haraldsson in a 3-hr Twitch Stream presented their use of ECS, from world building in the editor with custom visual scripting, to scalable open-world streamingReview:https://www.youtube.com/watch?v=3oxLTIw2lGI
- HardSpace: Shipbreaker – An action-adventure simulation video game which combines elements of simulation and adventure games. The player explores and dismantles abandoned spacecraft in search for useful materials. Review:https://www.youtube.com/watch?v=3tip-FrNU9c
- Zenith: The Last City – A virtual reality open world massively multiplayer online role-playing game (MMORPG). Lauren Frazier, CTO explains at GDC how ECS helped them scale up their gameplay for managing a VR MMO game architecture.
- Detonation Racing – A unique and very literally explosive racing experience. Lead Developer Jonas De Maeseneer speaks at GDC about the use of ECS for deterministic gameplay for QA and design loops, as well as streaming for a high-pace Apple Arcade racing game.
- IXION – A space journey to find a new home for humanity that includes space station building, population management, exploration and survival. Technical Director Jérémy Guéry walks us through how ECS has powered heavy NPC simulation for their space city building.
Getting Started Using DOTS
DOTS is a transformational shift in software architecture from objects to data and includes a steep learning curve. A computer science education or equivalent self-learning will likely be necessary.
It’s a fundamental shift in thinking compared to OOD, so expect more veteran developers to have just as much difficulty as newcomers.
There are great educational resources from well established, and reputable sources:
- Unity itself has published a getting started course:https://learn.unity.com/course/dots-best-practices and https://unity.com/roadmap/unity-platform/dots
- TurboMakesGames has published a 2.5-hr tutorial using ECS 1.0https://www.youtube.com/watch?v=IO6_6Y_YUdE
- GameDev.tv–instructors of more than 900k students globally has developed 24 lectures in a 2.5 hr course (wait for a promotion price of $14.99 before buying this course) https://www.udemy.com/course/intro-to-data-oriented-tech-stack-dots-ecs-in-unity/
Data-Oriented Design by R. Fabian. Buy on Amazon.
Pro .Net Memory Management by Konrad Kokosa. Buy on Amazon.
- Laurent Gibert – Unity DOTS product manager https://forum.unity.com/threads/dots-development-status-and-next-milestones-september-2022.1341077/
- Fabrice Lete – Unity Engine DOTS developer – https://www.linkedin.com/in/letef/
- Matt Fuad – https://twitter.com/matt_fuad
- Joe Valenzuela – https://forum.unity.com/search/18558712/
- Jason Weimann – In June, 2022, Jason, discussed the components of DOTS with Laurent Gibert and Andrew Parsons, both who are on the Unity DOTS team. https://youtu.be/19bAWtyEZsk
- Code Monkey – Youtube game developer who compiled a playlist of DOTS and DOD topics https://www.youtube.com/playlist?list=PLzDRvYVwl53s40yP5RQXitbT–IRcHqba
- Johnny Thompson – The developer behind Turbo Makes Games
We’ll try to keep this article current with the latest DOTS technology advances. Send us an email if you know of a resource we should add to this page. Updated as of Oct 28, 2022.