Image-generating tech boosts construction site safety

Current safety compliance procedures involve placing cameras onsite to capture images of machinery — such as cranes, scissor lifts, bulldozers and dump trucks — in operation. From there, the images are labelled manually, including key points of each piece of equipment. The generated dataset is then used to train artificial intelligence (AI) models and inferences are made about the classification of different equipment, pose estimation and activity recognition. Now, technology developed by Monash University engineers will generate accurate, synthetic images of machinery, creating large datasets for training deep neural networks to analyse the use of machinery onsite and in a variety of conditions.

The new technology, developed by Dr Mehrdad Arashpour and his team of PhD students and postdoctoral researchers, will streamline safety, productivity and quality monitoring procedures on building, mining and construction sites. This new method, which is currently under Australian patent review, removes the need for cameras to be placed on construction machinery, which has time, cost and privacy implications. The technology can process millions of images of any type of machinery in a short amount of time, and provide detailed information on its key physical and operational features.

For example, before using an excavator at a building site on a wet day, workers can identify all the safety features of this equipment within a digitised version of the working environment, including rain and clouds as well as reduced light and visibility. Over time, this technology can be expanded and used on any and every construction site, depending on the type of machinery in operation. Dr Arashpour said this application has been designed to improve safety, productivity and quality on building sites and other workplaces.

“Operations involving heavy construction equipment are a critical component of most projects. Heavy construction equipment, vehicles and workers are often required to work closely due to spatial limitations and tight schedules, which often leads to suboptimal performance both in terms of safety and productivity,” said Dr Arashpour.

Dr Arashpour added that preparing large datasets for specific tasks is currently a manual process that is time-consuming, labour-intensive, error-prone and subject to privacy concerns. “Our method randomises various critical features of the scene, such as equipment pose and texture, scene texture and lighting, camera location and field of view, and adds other elements to the scene such as simulated dust and occluding objects,” said Dr Arashpour.

The novel Monash-developed method uses game engineering technology to generate CAD models of heavy machinery, which are then accurately and automatically annotated. Dr Arashpour and his team have tested several deep neural networks on the real images of the equipment. The results obtained by training deep neural networks using synthetic images are almost as accurate as real images. “As compared to traditional data preparation pipelines, the proposed method does not require any manual annotation, which is a labour-intensive and time-consuming process,” said Dr Arashpour.

Synthetically generated datasets are also advantageous to manually labelled datasets, as they can produce pixel-level annotations for the key points of interest. In contrast, manually annotated datasets are susceptible to human error, and monitoring the quality of annotation can also be a cumbersome task. Dr Arashpour believes the new technology has the potential to modernise the building and construction industry and improve safety, quality and productivity.

Image credit: Monash University.