A Lightweight Approach of Human-Like Playtest for Android Apps

A play test is the process in which testers play video games for software quality assurance. Manual testing is expensive and time-consuming, especially when there are many mobile games to test and every game version requires extensive testing. Current testing frameworks (e.g., Android Monkey) are limited as they adopt no domain knowledge to play games. Learning-based tools (e.g., Wuji) require tremendous manual effort and ML expertise of developers. This paper presents LIT-a lightweight approach to generalize play test tactics from manual testing, and to adopt the tactics for automatic testing. Lit has two phases: tactic generalization and tactic concretization. In Phase I, when a human tester plays an Android game G for a while (e.g., eight minutes), Lit records the tester's inputs and related scenes. Based on the collected data, Lit infers a set of context-aware, abstract play test tactics that describe under what circumstances, what actions can be taken. In Phase II, LIttests G based on the generalized tactics. Namely, given a randomly generated game scene, Lit tentatively matches that scene with the abstract context of any inferred tactic; if the match succeeds, Lit customizes the tactic to generate an action for playtest. Our evaluation with nine games shows Lit to outperform two state-of-the-art tools and a reinforcement learning (RL)-based tool, by covering more code and triggering more errors. Lit complements existing tools and helps developers test various casual games (e.g., match3, shooting, and puzzles).