Toward a deeper integration of low-fidelity sketches into mobile application development
Abstract
Mobile application development often starts with creating low-fidelity sketches of user interfaces. Integrating these sketches into the software development process can reduce repetition, narrow the gap between user perception and final implementation, and improve app resilience. In this study, we introduce the DoodleUINet dataset, which comprises over 10K sketches of UI elements. Our Doodle2App tool converts low-fidelity sketches into a single-page, compilable Android app. At the same time, our PSDoodle provides an interactive, partial sketch-based search engine with a top-10 screen retrieval accuracy comparable to the state-of-the-art SWIRE line of work but with a 50% reduction in the average required time. Our TpD tool leverages natural language and sketching to retrieve and display the target screen in its top-10 search results with a success rate of 90%, achieving the top-10 accuracy of the state-of-the-art approach and consistently showing the target screen in the top-30. We have also developed D2S2, a drag-and-drop-based mobile screen search tool that incorporates the backend of TpD. Overall, our tools provide valuable resources for novice software engineers and facilitate the integration of low-fidelity sketches into the software development process.