INFERRING IN-SCREEN ANIMATIONS AND INTER-SCREEN TRANSITION FROM USER INTERFACE SCREENSHOTS

Abstract

In practice, many companies have adopted the concept of creating interactive prototypes for explaining workflows and animations. Designing and developing a user interface is a time-consuming process, and the user experience of the application has a major impact on the success of the application itself. User interface designing marks the start of the app development, and it is very expensive regarding cost and time for making any modification after the coding phase kicks in. Currently, companies have adopted UI prototyping as part of the app development process. Third-party tools like Flinto or Invision use the high fidelity screen designs for making interactive prototypes, and other tools like Flash is used to prototype animations and other transition effects. With this approach, there are two major setbacks. Creating the screen designs (acts as the screen specification for color, dimensions, margin, etc.) and the navigations or animations takes a lot of time, but they are not reusable in the app development process. The prototypes could act as a reference for the developers, but none of the output artifacts is reusable in the developing the application With our technique of using REMAUI as a preprocessor to identify different UI elements like images, texts, containers on the input bitmap images. We have developed a user interface that allows users to interact with the preprocessed inputs and create links for inter-screen transitions on click, long click with effects like a v slide, fade, and explode. We would be able to generate code for the intended navigation targeting a specific platform say Android. Additionally, we have developed a heuristic algorithm that analyses REMAUI processed input bitmaps and infers possible in-screen animations such as translation, scaling and fading using perceptual hashing. In our experiment, we evaluated our prototype’s inter- screen transition on screenshots of 10 different third party applications and it generated android code for transition in less than 2s. We evaluated inferring in-screen animation on screenshots of top 30 third party Android application generated user interfaces similar to the original on comparing pixel-by- pixel (SSIM) and it takes 26s on an average for identifying possible animation.