Augmented Reality – multi-user application as first step to Industrial Metaverse

For an interactive collaboration with international and national project partners, Fraunhofer IKTS develops solutions that include the virtual world – a metaverse. The goal is for project partners to come together and work on hardware and software solutions in the metaverse. These will then be transferred to the real world. In this context, the metaverse is not a virtual match for reality, but an active workplace. In it, all the necessary resources and equipment are to be made available. It is important to simulate the actual properties and conditions of devices and materials in the virtual world. For this purpose, a connection to real devices is imaginable.

Currently, a multi-user application is being developed for the metaverse. This first step deals with virtual collaboration using data glasses.

Acting together via multi-user application

A multi-user application with multiple users allows them to work together in the virtual environment. The prerequisite for this is that both the participant and the user have the same scene view. The use of augmented reality data glasses enables an overlay of virtual infor­mation with reality. With existing CAD data, models of machines or devices can be pre­pared for the virtual world and enhanced with application-specific animations and interactions. Image 1 shows an example of this in the form of a model of a rail testing device in a labora­tory. In connection with real components as well as an integrated gesture control in the data glasses, a real handling can be simulated in the virtual environment. In this way, test results can be projected locally onto three-dimensional objects (image 2).

For the current application, only one PC can be used as well. Image 3 shows that the partici­pants are represented by a sphere in the virtual environment. The position is based on the camera position of the terminal device. The start location is used for orientation or posi­tioning in the room during shared use. This is clearly defined in the application and is identi­cal for all participants.

For the current version of the application, it is defined that the first user as master controls the main activities. This includes the manipula­tion of objects via gesture control and moving them around the room. The operation of but-tons is possible for all participants. With this, additional information can be accessed and/or exploratory displays selected. Each of these actions of the master or participant is visible for all others and allows a detailed explanation or discussion.

Communication in the metaverse

Communication in the virtual environment is currently enabled exclusively by a chat function. On the one hand, the participants can use a predefined text field with the virtual keyboard and leave comments for the others (image 4). On the other hand, markings can be placed in the room as 3D writing or characters. After the multi-user application, further iterative steps towards the metaverse follow. It will be investi­gated to what extent avatars will be developed (how important is the representation of the body or arms) and which further communica­tion channels must be found to allow natural collaboration. Furthermore, the focus will be more and more on the integration and connec­tion between real and virtual devices to pro­mote the development of new components.