On the effectiveness of random testing for Android: Or how i learned to stop worrying and love the monkey

Priyam Patel, Gokul Srinivasan, Sydur Rahaman, Iulian Neamtiu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Scopus citations

Abstract

Random testing of Android apps is attractive due to ease-of-use and scalability, but its effectiveness could be questioned. Prior studies have shown that Monkey - a simple approach and tool for random testing of Android apps - is surprisingly effective, "beating" much more sophisticated tools by achieving higher coverage. We study how Monkey's parameters affect code coverage (at class, method, block, and line levels) and set out to answer several research questions centered around improving the effectiveness of Monkey-based random testing in Android, and how it compares with manual exploration. First, we show that random stress testing via Monkey is extremely efficient (85 seconds on average) and effective at crashing apps, including 15 widely-used apps that have millions (or even billions) of installs. Second, we vary Monkey's event distribution to change app behavior and measured the resulting coverage. We found that, except for isolated cases, altering Monkey's default event distribution is unlikely to lead to higher coverage. Third, we manually explore 62 apps and compare the resulting coverages; we found that coverage achieved via manual exploration is just 2 - 3% higher than that achieved via Monkey exploration. Finally, our analysis shows that coarse-grained coverage is highly indicative of fine-grained coverage, hence coarse-grained coverage (which imposes low collection overhead) hits a performance vs accuracy sweet spot.

Original languageEnglish (US)
Title of host publicationProceedings 2018 ACM/IEEE 13th International Workshop on Automation of Software Test, AST 2018
PublisherIEEE Computer Society
Pages34-37
Number of pages4
ISBN (Electronic)9781450357432
DOIs
StatePublished - May 28 2018
Event13th ACM/IEEE International Workshop on Automation of Software Test, AST 2018 - Gothenburg, Sweden
Duration: May 28 2018May 29 2018

Publication series

NameProceedings - International Conference on Software Engineering
ISSN (Print)0270-5257

Other

Other13th ACM/IEEE International Workshop on Automation of Software Test, AST 2018
CountrySweden
CityGothenburg
Period5/28/185/29/18

All Science Journal Classification (ASJC) codes

  • Software

Keywords

  • code coverage
  • google Android
  • mobile applications
  • random testing
  • stress testing

Fingerprint Dive into the research topics of 'On the effectiveness of random testing for Android: Or how i learned to stop worrying and love the monkey'. Together they form a unique fingerprint.

Cite this