Extended Reality and Digital Twins – The Future Is Now

By Janice Abel

Category:
ARC Report Abstract

Overview

While the concept might seem somewhat futuristic, numerous research reports indicate that a large number of forward-thinking industrial, municipal, and other organizations are already testing aspects of extended reality technology in their operations.  This is due to the technology’s significant potential to help humans do their jobs better and more efficiently.   The overall market for extended reality, which includes augmented reality (AR), virtual reality (VR), and mixed reality (MR), has been growing by leaps and bounds.  The technologies are being used mostly for design, engineering, operations, manufacturing, and training.

extended reality Extended%20Reality%20-%20A%20Combination%20of%20VR%2C%20AR%2C%20and%20MR.JPGWhile AR, VR, and MR are often used interchangeably, they each have distinct meanings. The increasingly popular (but not generally uniformly defined) digital twin concept also comes into play in this discussion.

What Is Extended Reality?

Extended reality is an umbrella term referring to all real-and-virtual combined environments and human-machine interactions generated by computer technology and wearables.   Extended reality encompasses VR, AR, and MR and the term is likely to be used when referring to future reality technology.  To understand extended reality, it is necessary to understand its three main components: augmented, virtual, and mixed realities. 

Augmented Reality

Augmented (or assisted) reality has been used in manufacturing for some time now. AR refers to the ability to overlay digital graphics and/or digitally generated sounds onto a real-world environment in real time using real-world sensor information. For example, you can overlay the digital representation of a piece of equipment onto the manufacturing floor to see where it would fit best or overlay technical instructions over an image of the equipment to assist technicians.  Equipment OEMs can use augmented reality (AR) in the form of a display of real-time data layered on a live video to further improve their field service capabilities and offerings. 

Another example would be a “heads up” display projected onto a vehicle’s windshield that provides the driver with supplemental sensor-based information.

Virtual Reality

Virtual reality technology, pioneered by the gaming industry, provides a more immersive, computer-simulated experience. With VR, everything is digitized and can be manipulated, enabling people to interact as if they were right there.  VR relies on headsets and goggles to create the real-world illusion and often integrates sound and vibrations.  The leading headset, Microsoft HoloLens, engages people with virtual experiences that can incorporate visual, sound, acceleration, and balance elements.   The early uses of VR in industry was for operator training simulators (OTS).  For some hazardous industries, like the nuclear industry, OTS training is mandated, since it has proven to be more effective in preparing operations to respond quickly and effectively to abnormal situations.

Mixed Reality

Mixed (or “hybrid”) reality includes elements of both VR and AR.  MR digitizes virtual objects that interact with the environment.  MR blends the real and virtual worlds to produce new digital environments and visualizations so that physical and digital elements can interact in real time.  Like AR, it overlays digital display images and information onto a real-world environment.  Like VR, users can manipulate digital objects in the physical environment.  Microsoft has produced Spectator View for MR using its HoloLens and AR capabilities.  Rockwell Automation used its Studio 5000 development platform with Microsoft’s HoloLens VR headset to create a next-gen, mixed reality experience for designers.  DAQRI and RealWear have environmentally hardened smart glasses and helmets for field service use in hazardous environments. 

The Extended Reality of the Future

Extended reality is a catch-all term.  In the future, AR, VR, and MR will work together and disrupt how we do business. Extended reality is a fundamental shift in the way people will interact with technology and media.  Not only will the physical world be combined with the virtual world, but it will integrate chemical and biological attributes of materials and things.  Underlying technology are advanced analytics and artificial intelligence.  The smartphone, mobile VR headset, and AR glasses will converge into a single extended reality wearable replacing all other screens in your life.  These wearables will provide immersive, cognitive, and connected experiences.

Extended reality will also play a larger role in manufacturing from product design and prototyping to helping operators and other workers solve process problems, maintain a safe environment, and maintain equipment. 

There will be challenges to using the technology such as cost, compute capacity, storage, and performance.  Extended reality requires massive volumes of data and often at fast speeds.  Currently, creating an application like this is relatively expensive, even with today’s IIoT platforms, related services, and AR application development software.  Databases will need to integrate, contextualize, and store data in a data lake that enables data management, analytics, and other applications.  New types of cybersecurity will need to be in place to do this securely.  But technology will advance with a convergence of computer vision, machine learning, enhanced security, people, and things.  Better computing performance, better battery technology, integrated materials and science, and more standardization will further advance extended reality.  Seamless connectivity complemented by 5G and gigabyte LTE technology and advanced 3D visualization will lead to additional usage of extended reality technologies. 

Along with robotics, manufacturing and remote services will be done virtually.  Extended reality will support robotics-based manufacturing, remote services, and training, including training for hazardous environments that are too risky to train in physically; and train people to respond to potential disasters that may never happen. 

Digital Twin and Simulation

So how does the digital twin fit in?  Different companies define the digital twin slightly differently depending upon their respective technology stack. 

Generally speaking, the digital twin is a virtual or digital representation of physical or real-world assets that includes modeling behavior and real-time analytics and machine learning.  Digital twin visualization is a combination of virtual and augmented and mixed extended reality that simulate the real-world counterpart.  The software digitalizes and simulates a real process, machine, or thing using real sensor data and models that enable people to interact in a virtual manner.

How Suppliers See the Digital Twin 

Digital twin technology is being used on the plant floor to improve efficiencies for commissioning, product design and innovation, production, and for predictive maintenance.  Here are how a few suppliers view the digital twin:

Emerson Automation Solutions views the digital twin as a tool to validate and optimize control systems and automation processes in the virtual world.  Use cases include operator training and virtual commissioning using Emerson’s DeltaV Simulate and Ovation Simulation.  Emerson’s dynamic simulator gives companies a tool to test control logic and operator graphics using a virtual commissioning scenario.  This approach helps minimize potential errors, particularly during startups and shutdowns. Emerson’s customers also use digital twins to support the initial design of a facility and build control systems and production processes.  The digital twin allows companies to identify and predict potential design or production flaws and fix them prior to commissioning.

GE Digital describes the digital twin as an “ever-evolving” dynamic digital model/software representation of a physical thing, system, or process designed to detect, prevent, predict, and optimize a customer’s industrial environment.  It provides insight into how and why individual assets, systems or processes are performing based on past history, current condition, real-time operational context, and environment. It then combines the data with machine learning and other advanced analytics to understand the present and predict the future. This  allows customers to build feedback loops for continuous improvement with predictive maintenance, improved anomaly detection, and enhanced performance for assets and operations.  For GE, the digital twin use cases are more around predictive and preventive maintenance.

For Dassault Systèmes a “digital twin” includes the concept of its 3DEXPERIENCE twin.  This represents an object, system, facility, or environment that exists (or will exist) in the real world. However, unlike some other interpretations of the digital twin, a 3DEXPERIENCE twin replicates an object as a dynamic 3D model at every stage of its life.  According to the company, from the regulatory requirements and materials that influence its design and manufacture to the customer’s experience; every stage can be simulated, manipulated, and experimented upon in a 3DEXPERIENCE twin. Because it is generated from a single data model on a unified platform – a 3DEXPERIENCE twin also helps ensure accuracy and fidelity. When these simulated environments are used to analyze real-time data from operating devices, the users can experiment in the digital world, which creates a replica of the experience in the real world.

For Siemens, the digital twin concept spans product and process design; production planning, engineering, and simulation; and performance. The software enables data and feedback to be exchanged  between engineering, operations, and other parts of the value chain. In a production capacity for process industries, the digital twin allows seamless handover between engineering and operations, using a common database of all the plant assets in a physical plant: instrument data, logic diagrams, control schemes, alarms, piping, design attributes, etc.  These are enhanced with simulation capabilities that support use cases like virtual commissioning, process design, and operator training.  A common digital twin is used with COMOS, Siemens’ plant engineering software, and Simit, its operator training and simulation software, which is used to validate control systems and train operators. According to the company, these support efficient plant engineering, shorter commissioning phases, and improved operator training as well as more innovative and better product and process design.

Both Dassault Systèmes and Siemens use the digital twin to depict and optimize factory floor layouts and production processes in a virtual world prior to putting physical assets in place and starting up production.  

Rockwell Automation uses the digital twin to remove the need for the actual physical manufacturing assets to be available to commission and test the control systems. The company’s Studio 5000 Logix Emulate removes the physical asset for actual hardware or control system testing.  According to the company, this software enables users to validate, test, and optimize application code independent of physical hardware, while also allowing connectivity to third-party simulation and operator training systems.  This enables teams to simulate processes and train operators in a virtual environment.  Among other things, the Rockwell Automation digital twin can enable a safer environment for startups and shutdowns. 

ARC Advisory Group clients can view the complete report at  ARC Client Portal

If you would like to buy this report or obtain information about how to become a client please Contact Us

Keywords: Extended Reality, Virtual Reality, Augmented Reality, Mixed Reality, Digital Twins, Process Manufacturing, ARC Advisory Group.

 

Engage with ARC Advisory Group