Augmented reality for a real industrial world

Industry today faces huge challenges. Even more than in consumer and high-tech fields, the skills gap created by the retirement of older industry experts, along with complex and time-consuming hiring challenges, means that having enough trained and informed people to get the job done is a daily problem.

At the same time, the pressure to increase or at least maintain profitability and viability doesn’t decrease, although budgets may. The critical nature of operations makes industry risk averse and slow to change, and every expenditure has to justify itself many times over.

Still, the only way to balance the lack of expert personnel and the need for profitability is an increase in productivity and a decrease in downtime that can only be achieved through technology. But, by definition, that technology has to solve problems and pay for itself from day one.

Unexpectedly enough, that’s where augmented reality comes in. Today, industry professionals are realising that a technology that once only existed in gaming is the most practical, highest ROI tool that they can consider.

Augmented reality in the industrial world

Huge amounts of data are collected through hundreds or thousands of sensors on every plant floor, but how can that data get into the hands of people on the plant floor who need it, and how can it be interpreted so that personnel can use it? New staff line up at workstations to check instructions, manuals and diagrams, and at any given time, only one or two experts on any procedure may be in the plant. Training new workers is a critical process that requires time and effort and the right expertise and tools. Maintenance personnel constantly face machines they can’t evaluate and data that is trapped behind gates and in inaccessible locations.

AR tools can help close the knowledge gap, connecting less experienced personnel to tools and experienced mentors while they work — rapidly upskilling a new generation of operators and technicians and ultimately enhancing productivity and the bottom line.

Many a large plant has hundreds of disparate assets and a maintenance team, many of whom are new and not yet fully trained to recognise and fix problems with each of these assets. The maintenance tech may need to consult a central location or a distant workstation to get access to the key performance indicators on that machine and the experts may not be available to support them.

Now imagine maintenance personnel on duty carrying tablets that enable them to walk to a machine and see KPIs superimposed over the asset to help them with equipment inspection and overhaul. With this information, they can evaluate if the machine is performing optimally according to specifications or if any action is needed.

Maintenance staff may approach a group of machines that are inaccessible, whether they are behind a gate or too high to reach. Traditionally, they would have to obtain permission to access the machine and cause a temporary process shutdown to view the current performance indicators and determine the correct course of action. However, with augmented reality, the maintenance technician can simply hold up a tablet and zoom in on the machine and all of the status indicator information is instantly available without having to open a gate or cause any interruption in service.

This is a very simple example of what augmented reality can do — not in the future, but today.

Industry and AR: a match made in profitability

With such an example, it’s easy to understand why AR technology offers so many practical benefits today in the industrial space. Having the opportunity to superimpose information typically accessed through a HMI on to a view of the equipment that operators and service technicians are looking at in real time is crucial to enabling the next generation of workers who are far more often found in the plant than sitting in a central control room. The new technologies allow operators to move freely, while at the same time having additional real-time information relevant to the task in hand presented to them. In this way, mobile and wearable AR technology supports and simplifies operational activities, creating solutions that perfectly integrate with supervision and HMI technologies commonly used to manage automation systems and perform maintenance activities.

AR is suitable for both the manufacturing and process industries, allowing operators to use technology integrated with standard IIoT, SCADA/HMI and plant analytics tools to perform actions needed within various production stages.

Wearable and mixed reality AR devices supplied to operators can come in a kit that includes glasses or helmets fitted with displays, cameras, audio hardware and Wi-Fi connectivity that maximise flexibility. More frequently, mobile devices can be used as operators walk through the plant. These systems can display dynamic, real-time information relevant to the task and the location where the operator is working as described in the example above, as well as process data, systems status, alert-related information, required actions or other helpful data. Operators can interpret the required commands, such as start, stop, set-point modification and instruction requests based on alerts.

Although AR systems will tend to be complementary solutions, rather than completely replacing workstations that support traditional work processes, the mobile nature of these devices reduces costs and provides much greater flexibility and speed during operational phases. Many facilities can’t install standard HMI systems in every location, so AR technology provides access to data they would not have otherwise.

For maintenance activities, AR systems make it possible to view on-screen documentation, such as manuals, guided videos and tutorials for troubleshooting the part or machine being maintained, and also enable communication with the operations centre using a wireless connection. Consequently, the instant visibility of relevant data significantly reduces the risk of human error when performing a procedure.

The key benefits of AR include:

• access to information that otherwise wouldn’t be available
• instant information on operations to perform according to the situation
• the receiving of alert notifications and relevant instructions
• access to guided maintenance procedures
• barcode or QR code scanning with real-time context-related feedback
• connectivity and interaction with other operators at remote operations centres
• interactive — or holographic — 3D views of automation systems
• access to information via the web.
   

It is important to note that these benefits accrue without the need to have a local HMI (panel PC or other) at each asset, allowing visibility to all the machines in the plant easily and quickly. The savings in infrastructure costs is substantial.

The present and the future

The present value of AR in industry is its unique ability to solve pressing and impactful problems such as improving productivity in the light of a constrained workforce and unpredictable economies. At the same time, AR has unlimited potential for solving industrial problems and enhancing the bottom line in the future.

A scenario currently exists in which the operator speaks to a machine, asks for information, addresses commands or directly interacts with the system using hand motions to launch or adjust a process, whether or not a plant or enterprise needs to make use of it immediately. Moreover, the machine may ask the operator to execute required commands or manual operations as needed by the circumstances, guiding the operators in their activity, preventing any inaccuracy or lack of efficiency.

Another AR tool for present or future is the HoloLens. The Microsoft HoloLens is actually a wearable holographic computer, although it is more commonly referred to as an AR device, given its ability to overlay a virtual interface to the actual environment to ensure remarkably intuitive interaction. HoloLens has a CPU, GPU (graphics unit) and a third chip called the Holographic Processing Unit (HPU). It is a standalone autonomous device, requiring no cables to connect to a PC or other devices.

The display is actually a computer worn on the head with a transparent lens that allows the user to see the surrounding environment, and surround audio (spatial sound) which locates the source of the sound. Advanced sensors include an inertia measurement, light sensor and four cameras for environmental analysis. An additional camera can detect depth in order to recreate the surrounding environment, while a 2 MP HD camera is used for taking pictures and recording video. Vocal commands are captured by four microphones embedded in the display. According to Microsoft, HoloLens can gather terabytes of data from sensors and process it in real time. The device senses where the operator is looking, interprets their gestures and understands vocal commands.

Alternatively, the term ‘smart glasses’ is used to define a type of AR wearable device that is worn like a pair of glasses. Visual content and dynamic data are viewed on a ‘heads up’ display, which overlays information on whatever is being looked at. Several devices are available on the market, including Google Glass and the Epson Moverio.

Google Glass has an integrated camera, inertia sensors, speakers and microphone, enabling control via vocal commands. The application can be displayed directly on the glass and does not need any other external device.

Epson Moverio is glasses designed for AR applications and offers images displayed for both eyes. As with Google Glass, an SDK is available for creating applications. The Moverio device also has a front-facing HD camera for shooting video and a battery that provides six hours of operation. Motion sensors integrated in the display and controller, plus GPS, microphone and video camera, help to support more sophisticated applications.

These are just a few examples of technologies available to extend the application of AR throughout plants and industrial enterprises — once the shift to a new way of mobile working has been embraced.

How to put AR to work

Putting AR to work solving problems today is remarkably easy and cost-effective. An AR solution must obtain dynamic data from a data server running an application that physically communicates with the system or machine of interest. This requires suitable data communication protocols to be used to access data from different monitoring and control devices, such as PLCs, CNC machines and sensors. Consequently, a server platform is needed to publish system-related data and make it available to the AR device.

Typically, a separate IIoT or SCADA/HMI supervisory workstation is required to provide:

• an I/O data server
• historisation of events and process-related data
• alert notification and event notification management
• data analysis management
• supervisory control logic management.
   

The AR device should be connected to the supervision system whenever system-related real-time data is available. Consequently, the supervision system and the AR system are linked to one another. The ideal solution is a standard, open, user-friendly platform that is compatible with all industrial protocols, is modular and expandable, and includes dashboard and screen design tools typically found in SCADA/HMI and AR systems.

From nice-to-have to must-have

Today’s industrial challenges have catapulted augmented reality from an excellent addition to an industrial plant’s operating technologies to a necessity. Few technological enhancements can provide more benefit at less cost in a short time. And, of course, the potential for AR to simplify, speed and enhance operations and maintenance in a plant is just the beginning. The integration of AR into SCADA/HMI platforms and the subsequent availability of actionable information via mobile, wearable and mixed reality devices offers the potential for new levels of efficiency and output.

Having the opportunity to instantly access a HMI while working in the industrial plant gives operators greater situational awareness, enables them to intervene promptly when and where the process demands it, and helps them perform tasks faster and make better decisions. Augmented reality HMI has the potential to revolutionise how operators manage processes, optimise productivity and reduce downtime, making it possible to do more with less every day.

Image: ©iStockPhoto.com/NanoStockk