Researchers use neural networks to discover quiet agents

People with nefarious motives almost always want to operate undetected – a Soviet-era spy working in the U.S., a terrorist planning an attack in Yemen, even a pickpocket lifting a wallet in London.

These “quiet agents” are silent entities attempting to pass unnoticed through a given environment. Accurately detecting the exact location and movement of a quiet agent by looking for the agent rarely works, but looking at the environment rather than the individual can make the invisible visible.

A person walking quietly through a park might never be heard by others, but if a flock of birds flies away as the person passes it is revealed that someone is there. Using advanced technology to analyze environments for quiet agents in real-time  – when there are no Jedi available  – could prove immensely useful for national security and military operations.

Capstone posterMaster of Science in Data Science (MSDS) students Jennifer Cruser and Court Haworth at the University of Virginia’s Data Science Institute undertook a capstone project aimed to develop a framework of concepts and software to stimulate and detect quiet agents in various environments.

Haworth and Cruser produced a simulation using Unity to place quiet agents in large, noisy environments. This created authentic data that allowed them to study how quiet agents can be found using surrounding noisy agents.

As a theoretical problem, a major project constraint was the absence of data provided by the capstone project client. The researchers therefore developed a simulation to produce data. They set up a virtual room laid out in a grid and evenly distributed microphones throughout the environment. They placed people – the noisy agents – throughout the room as well. During the stimulation, a quiet agent moved around the space and gave the researchers a realistic model for an environment with noisy and quiet agents.

The researchers then placed the data points into different matrices. Cruser and Haworth then fed these matrices into a five-layer Artificial Neural Network (ANN) designed as a Multi-layer Perceptron classifier from the scikit learn library in Python. This ANN used a stochastic gradient-based optimizer called Adam, which was chosen as it performs well in training time and validation score on large datasets. These choices make the framework conducive to an increase in scale. Once the ANN has been trained on a subset of the matrices, it makes predictions on a test set.

This completed proof-of-concept provides a configurable simulation that can be adapted to known spaces for surveillance. The research team was able to detect the quiet agent with a satisfactory level of accuracy and deliver a data product easily understood by the human eye.

J. Cruser, C. Haworth, P. Alonzi, and R. Alvarado. "Quiet agent detection through simulation and classification." 2018 Systems and Information Engineering Design Symposium, Charlottesville, VA, 2018.


Faculty Advisor
Faculty Advisor
Completed in:
Project type: