Real-Time Adaptive Data-Driven Perception for Anomaly Priority Scoring at Scale

With the aim of ultimately contributing to humanitarian response to disasters and violent events, detecting anomalies in daily human life is a crucial requirement for developing secured smart-home and smart-communities in the context of smart cities. In early security systems, it was necessary for a team of security experts to analyze a vast amount of surveillance data from a network of cameras, for instance to pick out patterns of human behavior identified as potential harmful threats. Now, however, in the big data era, online excavation and interpretation of streamed zettabyte data requires two automated technologies: (1) intelligent models - to extract suspicious patterns and discover latent anomalies; and, (2) agile systems - to take real-time action based on decision- making processes. As such, the two primary contributions of this dissertation are: (1) developing accurate intelligent models that perform much like human precision, and (2) proposing sub-systems of smart city infrastructure that intimately incorporate these intelligent models.

For this dissertation, pattern recognition models with applications in real-time video analysis were developed based on four computer vision tasks: (1) identity recognition, (2) object detection, (3) gesture recognition, and (4) action recognition. Applications of these models include, but are not limited to, recognition of: suspicious identities, active threats (life-threatening events i.e. bomb threats, civil unrest, criminal activity, earthquakes, evacuations, fires, hazardous materials), and suspicious packages. To perform these tasks, the intent is to have the models emulate the processes that take place within a human brain, i.e. with a close resemblance to human neuro processing, albeit in high-powered computational machines. Deep learning, the state-of-the-art concept in artificial intelligence, was the developmental basis for the proposed cognitive models.

In order to run efficiently, these computationally intensive models rely on the use of and co- ordination between high-throughput, high-performance, and many-task (parallel-task) computing- enabled machines that have a high level of computing performance compared to general-purpose computers. This variety of computational resources are served at scale in a cloud system, which serves as a central location with the core building blocks needed for compute, storage and networking. Nevertheless, anomaly detection and then taking real-time actions demands a faster processing speed than what is possible when communicating with cloud networks. It requires the use of physical infrastructure that is closer to the edge, near the source of the data (end-device); otherwise, when the data is centrally processed and stored, there is too much bandwidth required. This edge- computing approach helps reduce latency for critical applications, lower dependence on the cloud, and better manage the massive deluge of data being generated. In addition, security and privacy can also be improved with edge computing by keeping sensitive data within the end-device. In this dissertation, these distributed and decentralized deep learning systems aimed at enabling smart city applications –spread throughout the end-device, edge, and cloud– are designed following the requirements of smart city infrastructure.