Understanding User Interface for Intelligent Software Analysis and Exploration

Nowadays, millions of people across more than 190 countries around the world are using smartphones every day. People use them to book hotels, online shopping, and keep track of fitness information. The Graphic User Interface (GUI) is an essential part of an app to communicate with users. GUI analysis will help improve the security, privacy, and reliability of the applications and prevent the potential loss for both app companies and end-users. Existing research works on mobile apps are mainly focusing on system-defined APIs. Unlike well-defined APIs, GUIs are more unstructured and implicit since there is no criterion. Thus it makes analyzing and exploration really hard. To overcome such challenges, we proposed a novel approach to UI analysis and exploration. We applied it to solve the problems in privacy security and GUI testing reusing in mobile applications. We found 39 violations out of 120 apps in privacy analysis, and successfully convert averagely 80.2% of the UI test cases from iOS to Android.