Soft Embedding Cascades for Real-time Instance Segmentation

Instance segmentation is the process of predicting semantic classes and pixel-wise masks for every object in an image. This is primarily a supervised learning problem but can incorporate various elements from many different areas of Machine Learning. There is an increasing number of works focusing on real-time semantic segmentation, but real-time instance segmentation is a harder and less explored problem. Given that many modern instance segmentation algorithms are computationally expensive, this research outlines a possible approach towards achieving real-time results by reducing the computational load required for instance segmentation without sacrificing competitive accuracy. Two methods are used in order to accomplish this. The first is dividing up the computational load by outfitting a multi-branch parallel semantic segmentation base CNN architecture for instance segmentation. The other is making the segmentation process more tractable by performing critical computations on a lower resolution of superpixels rather than on the full resolution of the image. The results are then upscaled through a cascaded sequence of superpixel embeddings. Although the work is incomplete, preliminary results show potential in this approach.