Meta Researchers Win ISSTA 2026 Impact Award for Sapienz Android Testing Tool
Ke Mao, Mark Harman, and Yue Jia receive top honor for their 2016 paper on multi-objective automated testing that now powers Meta’s quality assurance.
Key points
- Ke Mao, Mark Harman, and Yue Jia won the ISSTA 2026 Most Influential Paper Award for their 2016 research on Sapienz.
- The tool originally identified 558 unknown crashes in 1,000 Google Play apps and was deployed at Meta within 17 months.
- Sapienz now runs millions of tests daily at Meta and supports dynamic taint analysis for WhatsApp using synthetic conversations.
Ke Mao, Mark Harman, and Yue Jia received the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) 2026 Most Influential Paper Award. The honor recognizes their 2016 research paper titled “Sapienz: multi-objective automated testing for Android applications,” which originated at University College London before becoming core testing infrastructure at Meta, according to Engineering at Meta.
The award celebrates ten years of impact from work that began in academia and evolved into enterprise-grade reliability engineering. Sapienz automatically tests applications to detect crashes and helps engineers reproduce those failures in fewer steps, making investigation more efficient. The original research balanced three objectives: exploring more of an app, finding more crashes, and reducing the number of steps needed to trigger them.
Technical Evolution and Scale
In its initial evaluation on 1,000 popular Google Play apps, Sapienz discovered 558 previously unknown crashes, demonstrating the effectiveness of its multi-objective approach. Within 17 months of publication, Meta deployed the system at scale across its platforms. Today, the tool runs millions of tests daily, surfacing thousands of unique crashes each week and helping safeguard hundreds of app releases each month to support product reliability and quality.
Sapienz’s applications expanded beyond basic crash detection to include dynamic taint analysis at WhatsApp. By using test accounts with synthetic conversations, it became the app-exploration engine for this security-focused analysis. This evolution highlights how automated testing infrastructure can adapt to support privacy and risk management programs by identifying risks earlier in the development lifecycle.
Future Directions and Analysis
Ke Mao delivered the ISSTA 2026 keynote titled “Finding the Bugs Users Would Find: From Sapienz to Autonomous Agents.” He described how large language model (LLM) agents can now explore apps and reason about whether their behavior matches user expectations. This extends automated testing from simple reliability checks to functional correctness and increasingly to automated repair, while Sapienz’s core ideas of multi-objective exploration and actionable fault revelation remain foundational.
That suggests the industry is shifting from static detection methods to autonomous agents capable of reasoning about software behavior. The integration of LLMs into testing frameworks allows for more nuanced evaluation of functional correctness, moving beyond mere crash reporting to assessing whether applications behave as intended by end users.
What should you do?
Security and engineering teams should evaluate how automated testing tools can be integrated into their continuous integration pipelines to catch defects earlier. Organizations should consider adopting multi-objective testing strategies that prioritize both coverage and reproducibility of failures. Developers should monitor advancements in LLM-driven testing agents to understand how they might enhance functional correctness verification in their own workflows.
Background: Large language models
Large language models are a specific type of neural network trained on text. Generative AI is the broader umbrella term for any system that creates new content, including images and code. Not all generative AI uses large language models, but most modern text generators do.
What to do and how to stay safe: Meta
- Review current automated testing strategies to ensure they prioritize reproducible failure scenarios rather than just coverage metrics.
- Investigate how synthetic data and test accounts can be used safely to expand testing coverage without exposing real user data.
- Monitor industry conferences like ISSTA for emerging tools that combine traditional testing with large language model reasoning capabilities.
- Ensure that privacy considerations are embedded in testing infrastructure design to align with broader risk management programs.
Step-by-step guide: Mobile Device Security: How to Protect Data Beyond Passwords
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
Who won the ISSTA 2026 Most Influential Paper Award?
Ke Mao, Mark Harman, and Yue Jia won the award for their 2016 paper on Sapienz, an automated testing tool for Android applications.
What did Sapienz achieve in its initial evaluation?
In its initial evaluation on 1,000 popular Google Play apps, Sapienz found 558 previously unknown crashes.
How is Sapienz used at Meta today?
Sapienz runs millions of tests daily at Meta, surfaces thousands of unique crashes weekly, and supports dynamic taint analysis at WhatsApp.
