androidtestrunner.qdoc

Go to the documentation of this file.

// Copyright (C) 2024 The Qt Company Ltd.
// SPDX-License-Identifier: LicenseRef-Qt-Commercial OR GFDL-1.3-no-invariants-only
 
/*!
    \page android-test-runner.html
    \brief Overview of the androidtestrunner tool for running Qt tests on Android.
    \title The androidtestrunner Tool
 
    \target androidtestrunner
 
    \ingroup android-platform-extra-topics
 
    \section1 Introduction
 
    The \c androidtestrunner tool runs Qt tests on Android devices and emulators.
    It automates the steps required to execute tests, including managing APK
    installation, test execution, and retrieving results.
 
    Before using the \c androidtestrunner, ensure you have configured your Qt
    test project with CMake or qmake.
 
    \section1 How it works
 
    \list 1
        \li Firstly, it begins by executing the APK build command to generate
            the required APK for the test.
        \li Following this, it installs the test app on the target device and
            starts the test app, setting the testing process in motion.
        \li The test results are written to the app’s data directory on the
            device, ensuring that comprehensive test outputs are captured.
        \li Once the tests conclude, the runner writes an exit code file beside
            the result files.
        \li At this stage, the \c androidtestrunner retrieves these result files
            from the device, examining the exit code for failures.
        \li If issues are detected, it immediately prints the app’s logcat logs,
            including any potential crash stack traces, which are beautified to
            reveal file names and line numbers for each frame. Additionally, it
            captures and reports Application Not Responding (ANR) logs if such
            events occur during the test execution.
    \endlist
 
    To enhance the integration experience, the test runner
    propagates QT or QTEST environment variables from the host environment of
    the test runner process to the app, ensuring consistency and a seamless
    testing workflow.
 
    \section1 Running tests with the test wrapper
 
    Qt generates a per-target wrapper script alongside each Android test (named
    after the test target) that invokes \c androidtestrunner with the right
    paths and forwards any extra arguments to the test binary. The example below
    runs \c tst_android on a specific emulator, forwards a Qt environment
    variable, and executes only the \c testAssets test case:
 
    \badcode
    ANDROID_SERIAL=emulator-5554 QT_DEBUG_PLUGINS=1 ./tst_android testAssets
    \endcode
 
    \section1 Retrieving results
 
    By default, and if the stdout test output is not explicitly disabled, the test
    results are printed to the host as the test is executing. After running the
    tests, any explicitly requested test files are pulled into the specified output
    path with their respective formats.
 
    For comprehensive details about Qt Test Framework, see \l {Qt Test Overview}.
 
    \section1 How to use it
 
    The basic syntax to run the \c androidtestrunner is as follows:
 
    \badcode
    androidtestrunner [ARGUMENTS] -- [TESTARGS]
    \endcode
 
    To run the test on a specific device/emulator, pass \c {--serial <serial>}
    or set the \c adb environment variable \c ANDROID_SERIAL or
    \c ANDROID_DEVICE_SERIAL. The explicit option takes precedence.
 
    \section2 Mandatory Arguments
 
    The test runner always expects the following arguments to be passed:
 
    \list
        \li \c {--path <build-path>}: The path where the Android Gradle
            package is built, typically \c {<build-dir>/android-build-<target>}.
        \li \c {--make <build-command>}: The command used to build the test APK,
            for example, \c {cmake --build <build-dir> --target <target>_make_apk}.
            \note Pass this argument quoted so it's not treated as multiple arguments
                of the test runner but as the value of \c {--make} argument.
        \li One of:
            \list
                \li \c {--apk <apk-path>}: Path to the test APK generated by the
                    build command and installed on the device.
                \li \c {--aab <aab-path>}: Path to the test AAB; requires
                    \c {--bundletool}. Both options cannot be set at the same
                    time, and neither may be specified more than once.
            \endlist
    \endlist
 
    \section2 Optional arguments
 
    You can also pass the following optional arguments:
 
    \list
        \li \c {--bundletool <path>}: Path to the Android \c bundletool jar.
            Required when \c {--aab} is used.
        \li \c {--manifest <path>}: Custom \c AndroidManifest.xml path. Defaults
            to a file discovered under the build path or its \c app/
            subdirectory.
        \li \c {--adb <adb-path>}: Specifies a custom ADB command path.
            Defaults to the \c adb path found in the system's \c $PATH.
        \li \c {--serial <serial>}: Android device serial to target. Overrides
            \c ANDROID_SERIAL and \c ANDROID_DEVICE_SERIAL.
        \li \c {--activity <activity-name>}: Specifies a custom activity to run.
            Defaults to the first activity defined in the \c AndroidManifest.xml.
        \li \c {--timeout <seconds>}: Sets the timeout for running the test.
            Defaults to 600 seconds (10 minutes).
        \li \c {--pre-test-adb-command <command>}: Call the adb \c {<command>}
            after installation and before the test run. Can be passed
            multiple times.
        \li \c --skip-install-root: Suppresses the automatic
            \c {INSTALL_ROOT=<path> install} suffix that is appended to
            \c --make when the build command is a \c make-family tool
            (\c make, \c gmake, \c nmake, \c mingw32-make, \c jom; matched
            case-insensitively). The suffix is never appended for \c cmake,
            \c ninja or other build drivers, so this option only has an
            effect with the listed tools.
        \li \c {--ndk-stack <command-path>}: Specifies the path to the
            \l {Android: ndk-stack}{ndk-stack} tool for symbolizing crash stack
            traces. Defaults to the tool path found under \c $ANDROID_NDK_ROOT.
        \li \c --show-logcat: Prints the logcat output to stdout regardless
            of test failure. Reads the main, system and crash buffers, and
            includes \c system_server lines when an ANR is detected.
        \li \c --verbose: Prints verbose output.
        \li \c {-- <arguments>}: Passes anything after the dashes as test arguments.
        \li \c --help: Displays the help information.
    \endlist
 
    \section2 Example usage
 
    Here is an example that runs the \c tst_android test, executing only
    \c testAssets test case:
 
    \badcode
    androidtestrunner \
        --path ~/tst_android/build/android-build-tst_openssl \
        --make "cmake --build ~/tst_android/build --target apk" \
        --apk ~/tst_android/build/android-build-tst_openssl/tst_openssl.apk \
        testAssets
    \endcode
*/