A Real-Time Intelligent Acoustic IoT-Enabled Embedded Construction Site Monitoring and Alert System: Integrating Deep Learning-Based Machine-Listening Algorithms, Edge Computing, and Cloud Computing

Acoustic-based construction site monitoring approaches have attracted recent interest due to their advantages compared to other methods. Previous relevant studies have many limitations, including narrowly focusing on sounds related to a specific/limited application without offering comprehensive monitoring capabilities of various construction site-related activities; tackling the software-side (or computational aspects) with limited work on hardware IoT-enabled embedded devices that could be used for real-time data collection, analysis, and edge computation; and lacking cloud-computing and visualization capabilities needed to improve data accessibility, storage, interpretation, and communication. This paper addresses these research gaps by developing a real-time intelligent IoT-enabled embedded acoustic-based sensing system for construction site monitoring by integrating machine listening techniques based on deep learning algorithms, edge computing based on Wi-Fi and bluetooth low energy (BLE)-enabled embedded systems, and cloud computing based on Amazon Web Services EC2. This study designed an automated monitoring system that uses convolutional recurrent neural networks to interpret construction site audio data with an accuracy of 89.13% across 14 classes of various audio related to different construction site aspects categorized into equipment and work activities, weather/environmental conditions, possible hazards, and workforce-related. The paper also developed a smartphone application to facilitate immediate and targeted alerts to relevant stakeholders. Finally, the proposed approach was tested in various construction workshops and environments. This paper's contributions are reflected by offering an unprecedented technological workflow/architecture that integrates software and hardware innovative advancements in audio-based monitoring systems of construction jobsites across various types of sounds. The paper also adds to the body of knowledge by developing an integrated system of real-time IoT-enabled acoustic capabilities, utilizing modern machine listening techniques powered by cloud and edge computing to improve current construction-site surveillance systems. This paper has the promise to change the way construction sites are monitored and managed, thereby contributing to enhanced safety and efficiency in the construction industry.