Testbed Design for Autonomous Security Testing of IoT Devices

Internet of Things [29] (IOT) represents a new class of network connected devices. IOT describes a group of interconnected, common devices, utilizing intra device and cloud services for communication and storage. Manufacturers who wished to leverage the "always on", "always connected" behavior of the Internet, added network functionality to their products and advertised this network connectivity as the next wave of innovation. As an example, IOT refrigerators [30] contain internal cameras. These cameras are used as part of a feature, to track the internal contents of the refrigerator and share this view via a smart phone connection. The feature enables the user to view the current inventory, build a grocery list, and set alarms when a tracked item's quantity is below a chosen threshold. Internally, the application also enables the tracking of a user's purchases; Data which is sold to companies with a vested interest in understanding large scale consumer behavior. This description represents two complementary industries. One which represents the utility of IOT devices in our everyday lives, and a second industry which is far removed from public view. [15] Now that large scale data mining is the new normal [3], we believe a formalized security testing process is needed within the IOT device space. We also propose a broader ownership of IOT behavior with the hope of reducing the damage of maligned behavior, poor security policies and abandoned devices. The choice in focusing on testbed design stems from the belief that a formal testing environment eases the pain of testing disparate IOT devices and allows for the focusing on new ways to group and test the underlying technology. Programmatic software testing is a fascinating field of study, however not all classes of testing fit neatly into autonomous workloads. Proactive security testing such as penetration testing, exploit variation, and reduced proof of concepts are notoriously difficult to automate; leaving these jobs for manual execution. The matrix of work in application and operating system versioning is large and would benefit from autonomous execution. The goal is to describe and implement a cross platform IOT security testbed, leveraging multi machine execution, to provide coverage for tens of operating systems, hundreds of traditional applications, and a focus on web-based applications. The coverage matrix is designed to encompass as many IOT device groups as possible. The novelty in categorizing tests by similar traits, is that it removes versioning as a testing data point, and instead places focus on feature behavior. Accomplishing this goal is as simple as bucketizing IOT devices and creating tests which focus on this common behavior. As an example, researchers can focus on testing security boundaries in the Oauth protocol by building a set of prepared Oauth, network, and string-matching tests; allowing for greater focus on the security problem, and less time on implementing the building blocks of the technology. We can group IOT devices by their use of Oauth as an authentication mechanism and write one suite of tests for all devices. Since the tests are extensible, we can build upon this example by introducing a second authentication mechanism and adding the IOT devices which use this second protocol. Adding test coverage for the second authentication protocol increases the number of testable devices but does not require a change in the original authentication tests. This design approach adds value to the process by limiting the waste of one-off test cases and instead uses each body of work as a portion of coverage for a larger pool of devices. Our second effort is to leverage the modular approach to produce tests which are traditionally left to manual execution. By building up small steps of function, we can create a suite of tests, which cover complex behaviors.