Containerizing Database Queries

Many methods to scale relational databases have been proposed, some implemented. Few have been widely adopted. Cloud virtualization, specifically container technology opens up new opportunities to execute and scale large data intensive applications wherever data is located in the cloud. A software architecture and methodology is presented to distribute(containerize) any SQL query across a cluster of cloud containers. Furthermore for the most part, the essential organization, structure and processing techniques of orthodox RDBMS need not be modified or extended with the exception of all-in-memory and parallelism, which simplifies overall design. An analysis of requirements, issues and advantages proceeds a description of the architecture. The methodology describes how a query's relational operators are packaged and deployed to containers. Both static and dynamic processes comprise the methodology. A containerized query execution planner proceeds from a query's optimized relational tree. Simple algorithms describe how the process of containerizing a query can be automated. A reference test bed for investigating cloud-based containerized queries validates the methodology by providing and analyzing benchmarks. A select-join-project-order(SJPO) example query demonstrates how containerization proceeds and is deployed in the Docker container framework. The example illustrations convey how the planner using the methodology automates the process of deploying a distributed SQL query across a cluster of containers across on multiple cloud hosts. In the reference implementation each container has its own embedded lightweight SQL engine is; SQLite. This allows the planner to delegate many types of query operations to the container's local SQLite data engine. In our reference implementation the containerized query plan off-loads many tasks to a container embedded lightweight SQLite data engine and other utilities. An experimental test-bed reference implementation is created for investigating containerizing a relational database query using cloud hosts. A reference implementation shows how a select-join-project-order(SJPO) query can be containerized and deployed using a commercial container framework., e.g. Docker. Benchmarks demonstrate performance and feasibility.