{"DOI":"10.1007/978-3-030-69781-5_6","abstract":"AbstractIn this paper, we present a multimodal deep model for detection of abnormal activity, based on bidirectional Long Short-Term Memory neural networks (LSTM). The proposed model exploits three different input modalities: RGB imagery, thermographic imagery and Channel State Information from Wi-Fi signal reflectance to estimate human intrusion and suspicious activity. The fused multimodal information is used as input in a Bidirectional LSTM, which has the benefit of being able to capture temporal interdependencies in both past and future time instances, a significant aspect in the discussed unusual activity detection scenario. We also present a Bayesian optimization framework that fine-tunes the Bidirectional LSTM parameters in an optimal manner. The proposed framework is evaluated on real-world data from a critical water infrastructure protection and monitoring scenario and the results indicate a superior performance compared to other unimodal and multimodal approaches and classification models.","author":[{"family":"Bakalos","given":"Nikolaos"},{"family":"Voulodimos","given":"Athanasios"},{"family":"Doulamis","given":"Nikolaos"},{"family":"Doulamis","given":"Anastasios"},{"family":"Papasotiriou","given":"Kassiani"},{"family":"Bimpas","given":"Matthaios"}],"id":"unknown","issued":{"date-parts":[[2021]]},"page-first":"77","publisher":"Springer International Publishing","title":"Fusing RGB and Thermal Imagery with Channel State Information for Abnormal Activity Detection Using Multimodal Bidirectional LSTM","type":"chapter"}