Running emulator tests on GitLab CIPosted on 2021-02-24 by eighthave
GitLab CI (Continuous Integration) has become an essential part of the F-Droid community processes. It is free software, built on open standards, and works well. The last piece that is missing from our testing ecosystem is a reliable way to run tests in actual Android emulators. Thanks to pushes from a number of people, Google now actually tests running the Android emulators in Docker. And recent releases of the emulator SDK package actually works without immediately crashing! This is the most promising news regarding using free software emulators for Android testing in a long time. Unfortunately, it is not smooth sailing yet, and getting the emulators to run in GitLab CI still requires some dancing around with magic incantations. Our requirements are:
- Works on the default gitlab.com shared runners.
- Works without KVM or any extra privileges.
- Uses KVM when available.
What we have now gives us the foundation to build our standard fdroidclient CI setup upon. Before, we were limited to using old armeabi-v7a emulators, which are run almost unimaginably slow. Those were the only emulator system images that would run in Docker without KVM support. Even with KVM support, the emulator seems to be quite flaky. This has gotten better, but is still not where it should be.
The good news is that running the emulator in Docker is now stable enough that people are actually building things around it, like running emulators in GitHub Actions. As much people use the Google emulators, that should keep Google taking care of them. The F-Droid emulator setup is free software: Debian base image running on GNU/Linux runners. Any project using GitLab CI can use this setup to run emulators on merge request, etc.
It is important to use the
default rather than
they do not contain the Google Play and apps binary blobs, and because the
Google apps seem to slow down the boot process a lot. Also, it seems the
android-27 system images seem to require less
resources than the newer ones, so much so that they are unlikely to work at
all for some projects.
The microg system-image repository is also included in the F-Droid setup. There are currently two images:
system-images;android-23;microg;x86- requires emulator v28 or older because it lacks a “ranchu” kernel.
uses YAML templates to make it easy to choose specific emulator setups,
We use the microg image in fdroidclient like this:
kvm 29 microg x86_64: <<: *kvm-template no-accel 29 microg x86_64: <<: *test-template <<: *connected-template
- To help debugging, the emulator kernel startup log can be found in the
root of the project in kernel.log, and the full logcat output in
logcat.txt. These can be included in
artifacts:for easy access.
- In order to run the jobs that use KVM, first you have to have a GitLab CI
runner that supports KVM and is tagged with
kvm. Then you need to set the variable
RUN_KVM_JOBSin the CI/CD Settings to
To keep development on using emulator images in GitLab CI moving forward, there is a wiki page to document tips and tricks, gotchas, and new developments: