The talk titled “Revolutionizing Roller Coaster Safety: Integrating Computer Vision, IoT, and Language Models” was presented by Dr. Chiranjeevi Roy at Deep Learning DevCon (DLDC) 2023. This talk is aimed to explore the integration of cutting-edge technologies to enhance roller coaster safety while preserving the thrill and excitement for riders. This research endeavours to revolutionize roller coaster safety standards using data-driven approaches, ensuring an enjoyable and secure experience for riders worldwide.
Dr. Roy commenced the session by emphasizing the importance of roller coaster safety in amusement parks, where the experience of riders depends on the balance between safety and excitement. The talk delved into the integration of computer vision, IoT, and language models (LLM) to address safety concerns in roller coasters. The objective was to provide a comprehensive solution that encompasses both the machines and the human aspect of roller coaster operations.
The research presented in the talk focused on three key components: asset condition monitoring, audio signal analysis, and rider movement detection. For asset condition monitoring, hundreds of sensors deployed on the roller coasters provided data at high frequencies. To predict failures and ensure safety, the team employed generative models and semi-supervised approaches, considering the challenges associated with labelling and the need for interpretability. The lightweight and generalized nature of these models made them well-suited for real-time inferencing and scalable deployment across different geographical locations.
The analysis of audio signals from scientific microphones aimed to filter out human voices and identify anomalies. By converting the audio signals into spectrograms, generative models were used to reconstruct the audio signals and identify problematic areas. This approach allowed for interpretable insights into potential issues while safeguarding personally identifiable information.
Rider movement detection, a critical aspect of safety, was addressed using the YOLO (You Only Look Once) architecture for object detection and classification. Real-time feeds from cameras deployed on the roller coasters enabled the identification of rider misconduct. The solution was deployed on an edge device, the Jetson Nano, and optimized using TensorRT to minimize latency and false alarms.
The integration of computer vision, IoT, and language models presented in this talk exemplifies a data-driven approach to revolutionize roller coaster safety. By combining generative models, semi-supervised learning, and real-time analysis, the research team showcased a comprehensive solution that addresses asset condition monitoring, audio signal analysis, and rider movement detection. The application of these technologies ensures a balance between safety and a thrilling experience for riders. Moving forward, this research has the potential to enhance safety standards in amusement parks and benefit riders worldwide.
