Schedule
For each CI config change, we need to follow:
scope of work (what will run, how frequently)
capacity planning (cost, physical space limitations)
will this replace anything or is this 100% new
puppet/deployment scripts or documentation
setup pool on try server
documented updated on this page, communicate with release management and others as appropriate
Current / Future CI config changes
Start Date |
Completed |
Tracking Bug |
Description |
|---|---|---|---|
TBD |
TBD |
TBD |
Upgrade Ubuntu 18.04 -> Ubuntu 22.04 X11 |
TBD |
TBD |
TBD |
Add Ubuntu 22.04 Wayland |
TBD |
TBD |
TBD |
Upgrade Mac M1 from 11.2.3 -> 13.2.1 |
TBD |
TBD |
TBD |
replace 2017 acer perf laptops with lower end NUCs |
TBD |
TBD |
TBD |
replace windows moonshots with mid level NUCs |
TBD |
TBD |
TBD |
Upgrade android emulators to modern version |
Completed CI config changes
Start Date |
Completed |
Tracking Bug |
Description |
|---|---|---|---|
October 2022 |
March 2023 |
Migrate from win10 -> win11 |
|
November 2022 |
February 2023 |
Migrate Win7 unittests from AWS -> Azure |
|
October 2022 |
February 2023 |
Migrate unittests from pixel2 -> pixel5 |
|
November 2020 |
August 2021 |
Windows tests migrate from AWS -> Datacenter/Azure and 1803 -> 20.04 |
|
May 2022 |
July 2022 |
Migrate perftests from Moto G5 phones to Samsung A51 phones |
|
March 2021 |
October 2021 |
Migrate from OSX 10.14 -> 10.15 |
|
July 2020 |
March 2021 |
upgrade datacenter linux perf machines from ubuntu 16.04 to 18.04 |
|
September 2020 |
January 2021 |
Android phones upgrade from version 7 -> 10 |
|
October 2020 |
February 2021 |
Run tests on MacOSX Aarch64 (subset in parallel) |
|
September 2020 |
March 2021 |
Python 2.7 -> 3.6 migration in CI |
|
July 2020 |
October 2020 |
Remove EDID dongles from MacOSX machines |
|
August 2020 |
September 2020 |
Schedule tests by test selection/manifest |
|
June 2020 |
August 2020 |
Android hardware tests running without rooted phones |
|
August 2019 |
January 2020 |
Upgrade Ubuntu from 16.04 to 18.04 (finished in January) |
Appendix:
OS: base operating system such as Android, Linux, Mac OSX, Windows
Hardware: specific cpu/memory/disk/graphics/display/inputs that we are using, could be physical hardware we own or manage, or it could be a cloud provider.
Platform: a combination of hardware and OS
Configuration: what we change on a platform (can be runtime with flags), installed OS software updates (service pack), tools (python/node/etc.), hardware or OS settings (anti aliasing, display resolution, background processes, clipboard), environment variables,
Test Failure: a test doesn’t report the expected result (if we expect fail and we crash, that is unexpected). Typically this is a failure, but it can be a timeout, crash, not run, or even pass
Greening up: Assuming all tests return expected results (passing), they are green. When tests fail, they are orange. We need to find a way to get all tests green by investigating test failures.