Kea2-python 0.2.2

A python library for supporting and customizing automated UI testing for mobile apps

Github repo link

https://github.com/ecnusse/Kea2

点击此处：查看中文文档

About

Kea2 is an easy-to-use tool for fuzzing mobile apps. Its key novelty is able to fuse automated UI testing with scripts (usually written by human), thus empowering automated UI testing with human intelligence for effectively finding crashing bugs as well as non-crashing functional (logic) bugs.

Kea2 is currently built on top of Fastbot, an industrial-strength automated UI testing tool, and uiautomator2, an easy-to-use and stable Android automation library. Kea2 currently targets Android apps.

Novelty & Important features

• Feature 1(查找稳定性问题): coming with the full capability of Fastbot for stress testing and finding stability problems (i.e., crashing bugs);

• Feature 2(自定义测试场景\事件序列\黑白名单\黑白控件[^1]): customizing testing scenarios when running Fastbot (e.g., testing specific app functionalities, executing specific event traces, entering specifc UI pages, reaching specific app states, blacklisting specific activities/UI widgets/UI regions) with the full capability and flexibility powered by python language and uiautomator2;

• Feature 3(支持断言机制^2): supporting auto-assertions when running Fastbot, based on the idea of property-based testing inheritted from Kea, for finding logic bugs (i.e., non-crashing functional bugs).

  For Feature 2 and 3, Kea2 allows you to focus on what app functionalities to be tested. You do not need to worry about how to reach these app functionalities. Just let Fastbot help. As a result, your scripts are usually short, robust and easy to maintain, and the corresponding app functionalities are much more stress-tested!

The ability of the three features in Kea2

	Feature 1	Feature 2	Feature 3
Finding crashes	:+1:	:+1:	:+1:
Finding crashes in deep states		:+1:	:+1:
Finding non-crashing functional (logic) bugs			:+1:

Design & Roadmap

Kea2 currently works with:

• unittest as the testing framework to manage the scripts;
• uiautomator2 as the UI test driver;
• Fastbot as the backend automated UI testing tool.

In the future, Kea2 will be extended to support

• pytest, another popular python testing framework;
• Appium, Hypium (for HarmonyOS/Open Harmony);
• any other automated UI testing tools (not limited to Fastbot)

Installation

Running environment:

• support Windows, MacOS and Linux
• python 3.8+, Android 5.0+ (Android SDK installed)
• VPN closed (Features 2 and 3 required)

Install Kea2 by pip:

python3 -m pip install kea2-python

Find Kea2's options by running

kea2 -h

Quick Test

Kea2 connects to and runs on Android devices. We recommend you to do a quick test to ensure that Kea2 is compatible with your devices.

1. Connect to a real Android device or an Android emulator (only one device is enough) and make sure you can see the connected device by running adb devices.

2. Run quicktest.py to test a sample app omninotes (released as omninotes.apk in Kea2's repository). The script quicktest.py will automatically install and test this sample app for a short time.

Initialize Kea2 under your preferred working directory:

kea2 init

This step is always needed if it is your first time to run Kea2.

Run the quick test:

python3 quicktest.py

If you can see the app omninotes is successfully running and tested, Kea2 works! Otherwise, please help file a bug report with the error message to us. Thank you!

Feature 1(运行基础版Fastbot：查找稳定性错误)

Test your app with the full capability of Fastbot for stress testing and finding stability problems (i.e., crashing bugs);

kea2 run -s "emulator-5554" -p it.feio.android.omninotes.alpha --agent native --running-minutes 10 --throttle 200

To understand the meanings of the options, you can see our manual.

The usage is similar to the the original Fastbot's shell commands.

See more options by

kea2 run -h

Feature 2(运行增强版Fastbot：自定义测试场景\事件序列\黑白控件)

When running any automated UI testing tools like Fastbot to test your apps, you may find that some specifc UI pages or functionalities are difficult to reach or cover. The reason is that Fastbot lacks knowledge of your apps. Fortunately, this is the strength of script testing. In Feature 2, Kea2 can support writing small scripts to guide Fastbot to explore wherever we want. You can also use such small scripts to block specific widgets during UI testing.

In Kea2, a script is composed of two elements:

• Precondition: When to execute the script.
• Interaction scenario: The interaction logic (specified in the script's test method) to reach where we want.

Simple Example

Assuming Privacy is a hard-to-reach UI page during automated UI testing. Kea2 can easily guide Fastbot to reach this page.

    @prob(0.5)
    # precondition: when we are at the page `Home`
    @precondition(lambda self: 
        self.d(text="Home").exists
    )
    def test_goToPrivacy(self):
        """
        Guide Fastbot to the page `Privacy` by opening `Drawer`, 
        clicking the option `Setting` and clicking `Privacy`.
        """
        self.d(description="Drawer").click()
        self.d(text="Settings").click()
        self.d(text="Privacy").click()

• By the decorator @precondition, we specify the precondition --- when we are at the Home page. In this case, the Home page is the entry page of the Privacy page and the Home page can be easily reached by Fastbot. Thus, the script will be activated when we are at Home page by checking whether a unique widget Home exists.
• In script's test method test_goToPrivacy, we specify the interaction logic (i.e., opening Drawer, clicking the option Setting and clicking Privacy) to guide Fastbot to reach the Privacy page.
• By the decorator @prob, we specify the probability (50% in this example) to do the guidance when we are at the Home page. As a result, Kea2 still allows Fastbot to explore other pages.

You can find the full example in script quicktest.py, and run this script with Fastbot by the command kea2 run:

# Launch Kea2 and load one single script quicktest.py.
kea2 run -s "emulator-5554" -p it.feio.android.omninotes.alpha --agent u2 --running-minutes 10 --throttle 200 --driver-name d unittest discover -p quicktest.py

Feature 3(运行增强版Fastbot：加入自动断言)

Kea2 supports auto-assertions when running Fastbot for finding logic bugs (i.e., non-crashing bugs). To achieve this, you can add assertions in the scripts. When an assertion fails during automated UI testing, we find a likely functional bug.

In Feature 3, a script is composed of three elements:

• Precondition: When to execute the script.
• Interaction scenario: The interaction logic (specified in the script's test method).
• Assertion: The expected app behaviour.

Example

In a social media app, message sending is a common feature. On the message sending page, the send button should always appears when the input box is not empty (i.e., has some message).

The expected behavior (the upper figure) and the buggy behavior (the lower figure).

For the preceding always-holding property, we can write the following script to validate the functional correctness: when there is an input_box widget on the message sending page, we can type any non-empty string text into the input box and assert send_button should always exists.

    @precondition(
        lambda self: self.d(description="input_box").exists
    )
    def test_input_box(self):
        from hypothesis.strategies import text, ascii_letters
        random_str = text(alphabet=ascii_letters).example()
        self.d(description="input_box").set_text(random_str)
        assert self.d(description="send_button").exist

        # we can even do more assertions, e.g.,
        #       the input string should exist on the message sending page
        assert self.d(text=random_str).exist

We use hypothesis to generate random texts.

You can run this example by using the similar command line in Feature 2.

Documentations（更多文档）

You can find the user manual, which includes:

• Examples of using Kea2 on WeChat (in Chinese);
• How to define Kea2's scripts and use the decorators (e.g., @precondition、@prob、@max_tries);
• How to run Kea2 and Kea2's command line options
• How to find and understand Kea2's testing results
• How to whitelist or blacklist specific activities, UI widgets and UI regions during fuzzing

Open-source projects used by Kea2

• Fastbot
• uiautomator2
• hypothesis

Relevant papers of Kea2

General and Practical Property-based Testing for Android Apps. ASE 2024. pdf

An Empirical Study of Functional Bugs in Android Apps. ISSTA 2023. pdf

Fastbot2: Reusable Automated Model-based GUI Testing for Android Enhanced by Reinforcement Learning. ASE 2022. pdf

Guided, Stochastic Model-Based GUI Testing of Android Apps. ESEC/FSE 2017. pdf

Maintainers/Contributors

Kea2 has been actively developed and maintained by the people in ecnusse:

• Xixian Liang (@XixianLiang)
• Bo Ma (@majuzi123)
• Chen Peng (@Drifterpc)
• Ting Su (@tingsu)

Zhendong Su, Yiheng Xiong, Xiangchen Shen, Mengqian Xu, Haiying Sun, Jingling Sun, Jue Wang have also been actively participated in this project and contributed a lot!

Kea2 has also received many valuable insights, advices, feedbacks and lessons shared by several industrial people from Bytedance (Zhao Zhang, Yuhui Su from the Fastbot team), OPay (Tiesong Liu), WeChat (Haochuan Lu, Yuetang Deng), Huawei, Xiaomi and etc. Kudos!

[^1]: 不少UI自动化测试工具提供了“自定义事件序列”能力（如Fastbot 和AppCrawler），但在实际使用中存在不少问题，如自定义能力有限、使用不灵活等。此前不少Fastbot用户抱怨过其“自定义事件序列”在使用中的问题，如#209, #225, #286等。