Wearable computing with safety benefits

Humans exhibiting enhanced capabilities through the use of implanted computers and electronics - otherwise known as cyborgs - have long been standard fare in popular science fiction movies and stories. This idea is no longer confined to creative Hollywood minds but is gradually finding its way into industries where it is proving to be very effective in terms of plant maintenance and personnel safety. Building on recent advances in mobile computing and sensor technology, the concept of wearable computing combines these technologies to produce invisible sensing devices that are capable of providing accurate information about the very often complex environment in which they operate. ABB has looked at how industrial environments, and in particular the service industry, can benefit from wearable computing solutions and solutions combined with augmented reality. The science fiction of yesterday has become the technology of today.

Recent advances in mobile computing and sensor technologies have enabled innovative solutions in the form of mobile apps and wearable computing. Mobile devices, such as smartphones and tablets, allow information to be quickly and cheaply accessed, processed and communicated without being confined to a single location. Sensor technologies can measure information in a particular environment and transmit the data to a mobile device for closer inspection. Modern and rugged mobile devices, equipped with different types of sensors (eg, light sensor, gyro, GPS, Wi-Fi and accelerometer) and extended functionality are proving very useful in industrial environments.

Image 1: Industrial environments are benefiting from wearable computing and augmented reality solutions to improve plant maintenance and ensure personnel safety.

Wearable computing takes the combination of mobile devices and sensors to a higher level by making the computer invisible (it is embedded into clothing or everyday items) and always on ^[1]. Though not an entirely new concept, costs and technical reliability were serious obstacles to the widespread practical implementation of wearable computing in the past ^[2]. A full-scale wearable computing solution consisting of a head-mounted-display (HMD)-based augmented reality (AR) and hand-gesture-based interaction was expensive. However, the popularity of mobile apps and the availability of advanced mobile technologies have now enabled the development of more affordable applications that provide field personnel with timely, accurate and detailed information on the move.

Supporting service efficiency

Proper service is vital in ensuring that industrial plants are safe to work in and operate without unexpected shutdowns. While the servicing of a particular device or system area may be routine to experienced service engineers, complex cases often mean additional and up-to-date information, such as customer products, application domains, the history of the installed equipment and service procedures and processes is required to support problem-solving and reduce the impact on the rest of the plant. The quicker this information can be accessed, the faster the problem can be solved. Therefore, the instant availability of up-to-date information is a vital prerequisite in any business environment today.

In global operating enterprises, this information is stored in many databases of the enterprise information technology infrastructure. ServIS, ABB’s installed base information system, is an example of such an enterprise information system that keeps track of all ABB products and systems at a customer site, including technical and project details. It is integrated with other ABB information systems, such as ABB Product, ABB People and the global customer identification system.

Using mobile and wearable systems, such as HMDs, eyeglasses or contact lenses, to access installed base information provides an opportunity for more efficient service delivery and execution.

These systems could be used to:

• Locate industrial equipment in large plants: AR can be used to overlay a real-world view of the plant with information related to the location of the equipment. The current location of the worker can be obtained via a GPS sensor built into a mobile device while the GPS position of the equipment can be loaded from an installed base management system such as ServIS.
• Identify industrial equipment: Advanced identification and labelling techniques, such as barcodes and NFC- or RFID-based tags (near-field communication or radio-frequency identification) can be used to identify equipment. The data read from the barcode or tag can be used to request further information from backend systems, such as ServIS.
• Access different types of information: As well as being able to access information such as previous service reports, technical drawings, manuals and checklists, field service workers could access the process control system of a plant to view real-time values of different process devices without being confined to a single location. Moreover, AR features could be used to overlay real-world images with work instructions or equipment or safety-related information ^[3].
• Situational awareness: Recent environmental changes or updates affecting service execution can be directly pushed to the service worker via wearable devices (eg, wristwatch displays and smart watches) in a non-intrusive way.
• Monitor work quality and documentation: Cameras and microphones could be used to continuously collect information (eg, sound recordings or movies) for plant analysis and auditing. Video recordings of work demonstrations can help to improve the quality of the work ^[4, 5].
• Seamlessly integrate the worker: Mobile and wearable solutions would enable the seamless integration of field service workers into service processes, allowing asset information to be retrieved and updated instantly. Moreover, they could enable service workers to connect to remote diagnostics and optimisation applications, or expert systems hosted by either backend systems or in a cloud environment.

Increasing safety

In industrial working environments, field service engineers face different types of hazards. A wearable system that can sense, collect information and issue warnings about the environment around it - such as temperature, humidity, oxygen level, poisonous gases, noise or radiation - as well as the vital signs of the wearer - such as heart or pulse rate, tiredness, consciousness and cognitive load or stress level - would increase the safety of maintenance and service staff ^[6]. Existing wrist-worn devices, such as Basis B1, can already measure skin resistance, pulse rate, temperature and even the worker’s stress level.

Image 2: The mobile app for displaying the values of the sensors embedded in safety clothing.

Wearable systems can support rescue and self-rescue operations. They can warn personnel of impending dangers, such as fire, water or lack of breathable atmosphere, and direct them to safe escape routes, even if exit signs are not available, are broken or are invisible because of the smoke and fire. Additionally, rescue teams can utilise the location functionality of such devices to locate personnel still in the plant.

ABB solutions

ABB provides various solutions that can be used on mobile devices:

• The Ventyx Service Suite enables field operators to maintain assets and reduce costs.
• The Ventyx Shift Operations Management System (eSOMS) ensures the safe, efficient and reliable operation and maintenance of facility assets.
• The Ventyx Advanced Work Management (AWM) Mobile Inspector collects and manages data relating to all physical assets.

In addition to these, several prototypes have been successfully developed in the search for new ways to implement mobile and wearable computing. In one prototype, sensors were sewn into a high-visibility vest, which was then operated via a smartphone. The sensors collected environmental conditions (carbon monoxide level, temperature and humidity) as well as the vital signs of  the worker (heart rate and skin temperature) - see image 2. They were complemented with feedback devices, such as a vibrator and speaker as well as an emergency/panic button. All the components were connected via a body area network (BAN) to a microcontroller⁷. The wearable safety suit could be connected via Bluetooth to the smartphone, which runs a control app to collect sensor data, display alerts and send notifications to a remote control centre and/or a supervisor when abnormal conditions are detected. The corresponding messages contained GPS coordinates of the last location of the field service worker, allowing him to be quickly located.

Another project investigated the possibility of using AR for control systems in industrial environments. In AR, live images are shown on the camera display of, for example, a mobile device and this view is then augmented by computer-generated content (eg, graphics). Several prototypes have been produced including one where a maintenance engineer points a mobile device at a water tank. After identifying the tank, the camera display is then augmented with live status values ‘- see image 3.

Image 3: The prototype for showing tank values in a camera view of a mobile device.

In another prototype, AR and sensor technologies were combined by adding a sensor to measure the temperature of an object. When the camera view of the mobile device is directed towards the object, the display shows the object’s temperature trend over time. The ability to view historical data enables engineers to perform fault tracing and testing during maintenance work.

A third prototype demonstrated how equipment can be found and identified within plants and factories. Mobile devices with sensors can determine the location of a field technician and from this immediately identify and provide information about devices in the vicinity - see image 4.

Image 4: The prototype for demonstrating how devices can be localised within a plant or factory to provide more information.

The availability of a variety of personal and mobile computing technologies has enabled the creation of new support tools for field service tasks. Using off-the-shelf components, important environmental, health and process information can be instantly obtained and shared with others while the engineer is on the move.

Source: First published in ABB review 3|14, this article was republished with permission.

*Author details: Markus Aleksy,  ABB Corporate Research Ladenburg, Germany, markus.aleksy@de.abb.com; Elina Vartiainen, ABB Corporate Research, Västerås, Sweden, elina.vartiainen@se.abb.com; and Martin Naedele, ABB Power Systems, Network Management, Baden, Switzerland, martin.naedele@ch.abb.com.

References:

[1] T. Kieffner. (Accessed 2013, June 19). Wearable Computers: An Overview [Online]. Available http://misnt.indstate.edu/harper/Wearable_Computers.html
[2] V. Stanford, “Wearable computing goes live in industry,” IEEE Pervasive Computing vol.1, issue 4, pp.14–19, 2002.
[3] S. Henderson and S. Feiner, “Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair”, IEEE Transactions on Visualization and Computer Graphics, vol. 17, issue 10, pp. 1355–1368, 2011.
[4] D. Roggen et al., (2013): “Opportunistic Human Activity and Context Recognition,” IEEE Computer vol. 46, issue 2, pp. 36–45, 2013.
[5] F. Naya et al., “Workers’ Routine Activity Recognition using Body Movement and Location Information”, Proceedings of the 10th IEEE International Symposium on Wearable Computers, Montreux, Switzerland, 2006.
[6] A. Pantelopoulos and N. G. Bourbakis, “A Survey on Wearable Sensor-Based Systems for Health Monitoring and Prognosis”, IEEE Transactions on Systems, Man, and Cybernetics – Part C: Applications and Reviews, vol. 40, issue 1, pp. 1–12, 2010.

Footnotes:

[7]The researchers used e-textile technology for the integration of the sensors and microcontroller inside the safety suit.