Baylibre announces partnership with to deliver Secure OTA updates to the Internet of Things

BayLibre is delighted to announce a partnership with to bring secure over-the-air (SOTA) updates to our customers & their products. SOTA ensures that every IoT device is patched, protected, and running with the latest fixes. Mender has become a core component for many of our Linux projects, including our AIoT Linux SDK developed in partnership with MediaTek.

Secure and frequent OTA updates have become table stakes for IoT and embedded systems. BayLibre required a solution that would be flexible and extensible enough to integrate with systems built for different purposes. Our customers required an advanced solution that struck a balance between turnkey readiness and configurability. offers multiple levels of security paired with a turn-key backend hosting solution at affordable prices. Above all else, delivers a reliable service that we feel confident recommending to our partners & customers.

“BayLibre has successfully deployed SOTA in the field across our customer base and the diverse set of devices that we help our customers build”, said Michael Turquette, CEO at BayLibre. “In addition to the flexibility and security of Mender’s solution, we have seen a strong customer response to the commercial hosting solution provided by Mender. Of course, customers have the choice to host the SOTA back-end themselves or through, and we’ve seen great success for customers that trust the team at Mender with their SOTA hosting needs.”

Linux Kernel v5.4 released, our contributions

Linux v5.4, the latest version of the kernel, was released on Monday November 25th. An excellent summary of this release can be found over on KernelNewbies. As always, has in-depth coverage of the statistics from the 5.4 development cycle, where BayLibre is once again in the top 20 list of most active employers.

This time Neil Armstrong made it onto the list of most active developers by number of lines changed, coming in at 12th place with a whopping 7370 changed lines. Also, Kevin Hilman was recognized as one of the most prolific testers in the 5.4 cycle and ranked in 5th position for developers providing the most Tested-by tags.

Here’s a summary of our contributions, organized by SoC family, and a summary graph of contributions by developer.

Amlogic SoCs

This was another Amlogic-heavy development cycle for us, and we collectively contributed 119 patches to improve or extend the kernel’s Amlogic support. Those changes were added by Neil Armstrong, Maxime Jourdan, Jerome Brunet, Alexandre Mergnat, Julien Masson, and Kevin Hilman, and they included:

  • Adding support for the SM1-based Khadas VIM3L board
  • Adding support for the SM1-based SEI610 board
  • Enabling DVFS for the SEI610, Odroid-N2, U200, SEI520, and X96-MAX boards
  • Also adding support for USB and HDMI display for the SEI610 board
  • Replacing the VPU-centric power domain controller with the generic Everything-Else power domain controller for G12-based boards
  • Miscellaneous fixes for the GXBB, G12A, and GXL boards
  • Converting various device tree bindings to YAML to take advantage of DT validation


Fabien Parent continued his work on MediaTek’s SoCs and added MT8516 SoC support to the PWM and CPUFreq subsystems in the Linux kernel 5.4 release.

Texas Instruments

This release, Bartosz Golaszewski contributed a variety of patches for the TI DaVinci family of SoCs.

  • Convert DA850 EVM boards to use the GPIO backlight driver
  • Support multiplatform build for ARM
  • Switch all DaVinci boards that use device tree or the old board files to the new clocksource driver
  • A range of cleanups and patches to simply the fbdev code


Jerome Brunet and Julien Masson contributed changes to the DRM subsystem.

  • Add support for more i2s formats to the Synopsys DesignWare I2S audio driver
  • Various fixes and improvements for the Synopsys DesignWare DRM driver

And Neil Armstrong was added to the MAINTAINERS file as co-maintainer of DRM bridge drivers.


Once again, Jerome Brunet provided fixes and cleanups for the ASoC subsystem.

The history of the KernelCI project

The KernelCI project brings automated build and boot testing to anyone wanting to check the quality and stability of Linux kernel releases. What started as a spare time project has gradually grown into a community-driven open source project that’s used by the likes of Google and the Automotive Grade Linux project to verify the quality of the kernels included in their products. Here at BayLibre, we’ve been contributing to the KernelCI project since its founding and we routinely use it in our customer work — it’s become one of our most indispensable tools.

But the success of the project wasn’t always so certain. When the KernelCI project initially started, it was just two kernel hackers sharing bash and expect scripts to automatically apply patches and build ARM kernels. The journey from spare time tool to Linux Foundation project has been one of struggles and uncertainty.

Let’s break the evolution of the KernelCI project down into three phases.

Phase 1: Spare time project

The arm-soc tree is a Linux kernel git repository that’s used to collect patches for the kernel’s ARM sub architectures, and every release tens of developers submit pull requests to the arm-soc tree. When it was initially created in 2012, the co-maintainers (Arnd Bergmann, Olof Johansson and Kevin Hilman) had to apply patches to the tree by hand and there was no way to easily test that the patches they were merging worked correctly for the all of the 120 kernel configs. The end result was that build errors were accidentally incorporated, especially for less popular boards where it was easier for problems to go unnoticed.

Independently, both Kevin and Olof came up with the same solution to stop this from happening: a set of scripts to automate the task of applying patches and make building kernels easier. Over time, the two co-maintainers merged their scripts together to create a single collections of maintainer tools.

This was still early days though, and this embryonic version of KernelCI was missing many features that are taken for granted in a modern CI tool. For one, when the tools detected that a patch caused the kernel to stop building, the maintainers would have to manually send emails to the submitter telling them about the build failure. It was a very time-consuming process.

The next step was to reduce the effort of mailing build reports by automatically notifying authors whenever they submitted git pull requests that didn’t build across all ARM defconfigs. Once building and error reporting were automated, the arm-soc maintainers set their sights on run time testing every kernel that they built. And that meant booting kernels on real hardware.

Over the years, Kevin and Olof had collected a small pile of different ARM boards and platforms, so they diligently wrote the code to automatically boot freshly built kernels on their boards. This code uploaded kernels to the test device and booted it to a shell inside of a ramdisk. With this new functionality supported, the KernelCI code grew a new testing dimension. Now, not only could Kevin and Olof perform basic checks to ensure kernels built , but they could also ensure that those kernels would pass the slightly more strict requirement of booting to a shell. This meant that the automated testing could not only detect when patches “broke the build” but also when patches “broke the boot” too.

Around 2013, other kernel subsystem maintainers started asking for access to the KernelCI build and boot reports. Linaro engineers Tyler Baker and Milo Casagrande registered the domain and began work on building a website using the existing maintainer tools as a backend. To give the wider audience the information they required, a new frontend was written by Milo that would allow users to view the build and boot reports. 

It was at this point that the KernelCI project as we know it today was officially born.

Phase 2: Finding a community

Fast forward to early 2016 and Kevin was now at BayLibre and continuing his work on KernelCI. All of the founding developers had changed companies and were still involved, to varying degrees, with the project in their spare time. But without corporate backing a question naturally arose: is the KernelCI project sustainable?

Now the future of the entire project was in doubt and faced a kind of existential crisis. The developers knew they would be able to allocate dedicated time to work on the KernelCI project, but only if there was enough demand. The project needed to move beyond a spare time effort into one with a self-sustaining community.

KernelCI began to see adoption beyond the arm-soc tree as other kernel maintainers began making heavy use of the project. In particular, Greg Kroah-Hartman and Sasha Levin (the stable tree maintainers) were using KernelCI to ensure that every new release they produced was just as stable as the previous versions. In addition, BayLibre productized for the Automotive Grade Linux project. This proved that the project was a viable choice for projects focused on market verticals such as automotive and IVI.

This was a major turning point and KernelCI became a legitimate open source project. All of the code was moved to GitHub and mailing lists were created for developers and users to report bugs and discuss new features. With the backing of various companies, things began happening much more quickly.

The increased popularity of the project meant that the feature wishlist started growing faster than the small group of developers were able to finish tasks. It turned out that there was so much interest for the project that the developer were unable to keep up. So the next phase of the project was searching for a permanent home and a source of funding.

Phase 3: Going mainstream

Once the project started to be adopted as a foundational continuous integration tool for kernel developers and stable maintainers, the KernelCI developers began looking for a way to permanently fund the work and pay for the infrastructure to take the project to the next level.

A number of methods were considered, but ultimately the project decided on working with the Linux Foundation to achieve their ambitious goals, and there’ll be an official announcement of the project joining the Linux Foundation at the upcoming OSS Summit Europe 2019 conference.

One of the issues that the re-invigorated KernelCI project wants to tackle is the fragmentation that exists  in the Linux kernel testing space. There are currently dozens of kernel testing projects that duplicate work to varying degrees, and consolidating them into a single project — or establishing common test and reporting formats — would go along way towards avoiding that duplicate work.

The next chapter for KernelCI

Even though the KernelCI project has been going for seven years now, we’ve got no plans to slow down. New features are proposed all the time and the list of tasks just keeps growing.

Perhaps the biggest request the project has received is to make it easy for developers and product owners using stable kernels to switch to newer versions. Today, this is a challenge because there’s no standard way to compare the quality of two stable kernel releases, so you never know if you’re going to run into bugs when moving to a newer release. That means products ship with known bugs because developers are discouraged from updating.

If you’d like to help shape the future of Linux Kernel quality and the Continuous Integration ecosystem join the LF KernelCI project today. To find out more about the project, head on over to to see the latest results or join in the discussions on the mailing list.

KernelCI Joins Linux Foundation

We’re thrilled to announce that the KernelCI project has officially joined the Linux Foundation and that BayLibre is a founding member company along with Google, Collabora, Red Hat, and Microsoft. BayLibre’s Kevin Hilman was one of the original founders of in 2014, and since then Kevin and the BayLibre team have continued to contribute changes. This announcement is a milestone for the project, and the support from the Linux Foundation demonstrates how essential automated testing at scale is for the Linux kernel community.

KernelCI is a true battle-tested open-source project and is a vital tool for the upstream Linux kernel maintainers, allowing them to merge large numbers of changes during the merge window while still maintaining quality and stability. Thanks to the collection of board farms located in hardware labs around the world, new changes are boot and runtime tested across a wide range of devices and hardware, and results are checked via the KernelCI infrastructure before being applied to the official maintainer repositories.

We think this is the start of a really exciting time for KernelCI. With the help of the Linux Foundation, infrastructure costs will be covered and the overall operation of the service will be improved. Which means the project’s members will be able to focus on new ways to grow the testing service and make it even better.

We’re looking forward to the next stage of the KernelCI project, and continuing to help increase the long-term maintainability of the Linux kernel.

If you want to learn more about KernelCI, including how you can get involved, check out the website and the mailing list.

Linux Kernel v5.3 released, our contributions

BayLibre has continued our contribution to the Linux community as seen with this new version of Linux Kernel 5.3, released on Sunday, September 15th 2019.  An excellent summary of this release can be found at KernelNewbies.

If you check out’s 5.3 Kernel development cycle statistics article, you’ll see that BayLibre made it onto the top 20 contributors list, showing that we were one of the most active companies (when measured by changes) this development cycle.

Here is a summary of our contributions, organized by SoC family and a summary graph of contributions by developer.

AmLogic SoCs

This release we contributed heavily to Amlogic Meson SoCs mainline support with lots of work from Neil Armstrong, Jerome Brunet, and Maxime Jourdan for the G12A (S905X2/S905D2) and G12B SoCs including:

  • Initial support for new G12B Soc family 
  • Enabling the IR controller on the SEI510, u200, and X96 Max boards
  • Fixups for the AXG TDM formatter driver
  • Adding support for dynamic OTG switching with the ID change interrupt
  • Enabling the sound card on the Hardkernel Odroid-N2
  • Increasing the Bluetooth bus speed to 2Mbaud/s
  • Enabling the WiFi SDIO module
  • Adding gigabit ethernet support for the X96 Max board
  • Enabling the hwrng module for the next generation SM1 SoC family
  • Enabling SD and eMMC on the g12a u200 board
  • A V4L2 m2m video decoder driver
  • Support for the XBGR8888 and ABGR8888 formats to the graphics controller

GPIO subsystem

Bartosz Golaszewski contributed changes for the GPIO subsystem including:

  • Fixes for warnings when the gpiolib is disabled
  • A fix for a use-after-free bug 
  • Various other cleanups and fixes


With the continued interest and adoption of RISC-V, we wanted an easy way to build an upstream kernel for the SiFive Unleashed board. Loys Ollivier submitted a patch that enables support in the default RISC-V kernel config.

DaVinci SoC

We’ve talked about updating the existing TI DaVinci SoC timer driver in the past, and this release Bartosz Golaszewski implemented a much simpler — and more modern — version with clockevents and clocksource support. He also enabled cpufreq support.


Beyond the above, we’ve also contributed patches to various subsystems and drivers.

  • Jerome Brunet contributed miscellaneous fixes and cleanups for the ASoC subsystem
  • Fabien Parent added AUDSYS clock support for MediaTek’s MT8516 SoC
  • Neil Armstorng enabled the Lima driver (ARM Mali 400/450 GPU) for arm64 and ARMv7 boards because it will be useful for KernelCI boot and runtime testing. 
  • Bartosz Golaszewski contributed cleanup patches for the at24 EEPROM driver and a new selector stepping option for voltage regulators.

BayLibre Linux 5.3 contributions

When 32 bits isn’t enough — Porting Zephyr to RISCV64

Conventional wisdom says you should normally apply small microcontrollers to dedicated applications with constrained resources. 8-bit microcontrollers with a few kilobytes of memory are still plentiful today. 32-bit microcontrollers with a couple of dozen kilobytes of memory are also very popular. In the latter case, it is typical to rely on a small RTOS to provide basic software interfaces and services.

The Zephyr Project provides such an RTOS. Many ARM-based microcontrollers are supported, but other architectures including ARC, XTENSA, RISC-V (32-bit) and X86 (32-bit) are also supported.

Yet some people are designing products with computing needs that are simple enough to be fulfilled by a small RTOS like Zephyr, but with memory addressing needs that cannot be described by kilobytes or megabytes, but that actually require gigabytes! So it was quite a surprise when BayLibre was asked to port Zephyr to the 64-bit RISC-V architecture.

Where to start

The 64-bit port required a lot of cleanups. Initially, we were far from concerned by the actual RISCV64 support. Zephyr supports a virtual “board” configuration faking basic hardware on one side and interfacing with a POSIX environment on the other side which allows for compiling a Zephyr application into a standard Linux process. This has enormous benefits such as the ability to use native Linux development tools. For example, it allows you to use gdb to look at core dumps without fiddling with a remote debugging setup or emulators such as QEMU.

Until this point, this “POSIX” architecture only created 32-bit executables. We started by only testing the generic Zephyr code in 64-bit mode. It was only a matter of flipping some compiler arguments to attempt a 64-bit build. But unsurprisingly, it failed.

The 32-bit legacy

Since its inception, the Zephyr RTOS targeted 32-bit architectures. The assumption that everything can be represented by an int32_t variable was everywhere. Code patterns like the following were ubiquitous:

static inline void mbox_async_free(struct k_mbox_async *async)
        k_stack_push(&async_msg_free, (u32_t)async);

Here the async pointer gets truncated on a 64-bit build. Fortunately, the compiler does flag those occurrences:

In function ‘mbox_async_free’:
warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
k_stack_push(&async_msg_free, (u32_t)async);

Therefore the actual work started with a simple task: converting all u32_t variables and parameters that may carry pointers into uintptr_t. After several days of work, the Hello_world demo application could finally be built successfully. Yay!

But attempting to execute it resulted in a segmentation fault. The investigation phase began.

Chasing bugs

While the compiler could identify bad u32_t usage when a cast to or from a pointer was involved, some other cases could be found only by manual code inspection. Still, Zephyr is a significant body of code to review and catching all issues, especially the subtle ones, couldn’t happen without some code execution tracing in gdb.

A much more complicated issue involved linked list pointers that ended up referring to non-existent list nodes for no obvious reason, and the bug only occurred after another item was removed from the list. This issue was only noticeable with a subsequent list search that followed the rogue pointer into Lalaland. And it didn’t trigger every time.

The header file for list operations starts with this:

#ifdef __LP64__
typedef u64_t unative_t;
typedef u32_t unative_t;

So one would quickly presume that the code is already 64-bit ready. From a quick glance, it does use unative_t everywhere. What is easily missed is this:


static inline sys_sfnode_t *z_sfnode_next_peek(sys_sfnode_t *node)
        return (sys_sfnode_t *)(node->next_and_flags & ~SYS_SFLIST_FLAGS_MASK);

Here we return the next pointer after masking out the bottom 2 flag bits. But 0x3U is interpreted by the compiler as an unsigned int and therefore a 32-bit value, meaning that ~0x3U is equal to 0xFFFFFFFC. Because node->next_and_flags is an u64_t, our (unsigned) 0xFFFFFFFC is promoted to 0x00000000FFFFFFFC, effectively truncating the returned pointer to its 32 bottom bits. So everything worked when the next node in the list was allocated in heap memory which is typically below the 4GB mark, but not for nodes allocated on the stack which is typically located towards the top of the address space on Linux.

The fix? Turning 0x3U into 0x3UL. The addition of that single character required many hours of debugging, and this is only one example. Other equally difficult bugs were also found.

The unsuspecting C library

One major change with most 64-bit targets is the width of pointers, but another issue is the change in width of long integer variables. This means that the printf() family of functions have to behave differently when the “l” conversion modifier is provided, as in “%ld”. On a 32-bit only target, all the printf() modifiers can be ignored as they all refer to a 32-bit integer (except for “%lld” but that isn’t supported by Zephyr). For 64-bit, this shortcut can no longer be used.

Alignment considerations are different too. For example, memory allocators must return pointers that are naturally aligned to 64-bit boundaries on 64-bit targets which has implications for the actual allocator design. The memcpy() implementation can exploit larger memory words to optimize data transfer but a larger align is necessary. Structure unions may need adjustments to remain space efficient in the presence of wider pointers and longs.

Test, test and test

One great thing about Zephyr is its extensive test suite. Once all the above was dealt with, it was time to find out if the test suite was happy. And of course it wasn’t. In fact, the majority of the tests failed. At least the Hello_world demo application worked at that point.

Writing good tests is difficult. The goal is to exercise code paths that ought to work, but it is even better when tests try to simulate normal failure conditions to make sure the core code returns with proper error codes. That often requires some programming trickery (read: type casting) within test code that is less portable than regular application code. This means that many tests had to be fixed to be compatible with a 64-bit build. And when core code bugs only affecting 64-bit builds were found, fixing them typically improved results in large portions of the tests all at once.

OK, but where does RV64 fit in this story?

We wrote the RV64 support at the very end of this project. In fact, it represented less than 10% of the whole development effort. Once Zephyr reached 64-bit maturity, it was quite easy to abstract register save/restore and pointer accesses in the assembly code to support RV64 within the existing RV32 code thanks to RISC-V’s highly symmetric architecture. Testing was also easy with QEMU since it can be instructed to use either an RV32 or an RV64 core with the same machine model.

Taking full advantage of 64-bit RISC-V cores on Zephyr may require additional work depending on the actual target where it would be deployed. For example, Zephyr doesn’t support hardware floating point context switching or SMP with either 32-bit or 64-bit RISC-V flavors yet.

But the groundwork is now done and merged into the mainline Zephyr project repository. Our RV64 port makes Zephyr RTOS 2.0.0 a milestone release — it’s the first Zephyr version to support both 32-bit and 64-bit architectures.

Linux Kernel 5.0 released, our contributions

BayLibre has continued our contribution to the Linux community as seen with this new version of Linux Kernel 5.0, released on Sunday, March 3rd 2019.  An excellent summary of this release can be found at KernelNewbies.

Here is a summary of our contributions, organized by SoC family and a summary graph of contributions by developer.

Amlogic SoC family:

  • new board: S805X-based “La Frite” from Libre Computer
  • audio: support S/PDIF input on AXG SoC family
  • audio: enable sound modules for S400 board
  • clocks: added new clock measure driver
  • display: support HDMI 1.4 4K modes

Ti DaVinci SoC Family:

  • fix label names in GPIO lookup entries
  • use nvmem to lookup MAC address


  • Corentin Labbe: several fixes and cleanups in the crypto subsystem
  • Carlo Caione: fix boot from eMMC on i.MX8MQ
  • Bartosz Golaszewski: nexted IRQ fixes in GPIO subsystem

U-Boot v2019.01 released, our contributions

BayLibre has continued contribution to the open-source community as seen with this new version of U-Boot v2019.01, released on 14th January 2019.

This release introduces a big rework for Amlogic Meson SoC support in order to:

  • Add support for the new AXG platform and ease support for G12A/G12B platforms
  • Ease support for new boards based on Amlogic Reference Design

Starting with this release, Neil Armstrong now maintains the Amlogic SoC support with an U-Boot Custodian tree, similar to a maintainer tree, now collecting patches for the Amlogic SoC support (board, arch, clock, pinctrl, …) and sending pull-requests to Tom Rini, the U-Boot maintainer.

Since U-Boot v2017.11, BayLibre engineers have pushed:

  • 75 patches
  • 14804 line changes
  • 292 files changes

Here is a summary of our contributions for this release:

Amlogic SoC family:

  • Add boot device mode, usable with pyamlboot
  • Add support for AXG family
    • pinctrl
    • dts
    • clock
    • network
    • mmc
  • Rework GX support by:
    • moving all P212 boards variants under the P212 board code
    • moving Khadas VIM2 support under the Q200 board code
    • moving Nanopi-K2 support under the Odroid-C2 board code
  • Add SPIFC flash controller for the upcoming “La Frite” board
  • Add VPU Power domain and VPU clocks support for upcoming video support


Linux Kernel 4.20 released, our contributions

BayLibre has continued our contribution to the Linux community as seen with this new version of Linux Kernel 4.20, released on Sunday, 23rd December 2018.  An excellent summary of this release can be found at KernelNewbies.

Here is a summary of our contributions, organized by SoC family and a summary graph of contributions by developer.

Amlogic SoC family:

  • Added AXG PDM Input support, with Devicetree Nodes
  • Added common “Canvas” provider for DRM Display and upcoming V4L2 Decoder Driver
  • Added support for Serial Number readout from sysfs
  • Various Pinctrl, DRM, Audio and clock fixes

Ti DaVinci SoC Family:

  • fixed a GPIO-related regression present since v4.19
  • Finally killed davinci_clk_reset_assert/deassert()


  • Bartosz Golaszewski became co-maintainer of the GPIO sub-system
  • nvmem: Bartosz fixed problems with the nvmem consumer API
  • Still some cleanups from Corentin in Crypto, Network and various other sub-systems
  • Fabien Parent added support for dedicated power supply in ChromeOS’s charger driver



The Top 3 Innovative Features of the Automotive Grade Linux UCB

Last month the Automotive Grade Linux (AGL) Unified Code Base (UCB) was listed as a CES Innovation Awards Honouree. AGL is a game-changing project for the automotive industry focused on bringing a fully-open software stack to the connected car, and we’re proud to have recently been named one of the top contributing companies. CES innovation awards 2019

AGL provides an operating system, middleware and application, known as the Unified Code Base (UCB), on top of which its nearly 140 member companies can rapidly build stable products. With all of those companies contributing to the project, there are plenty of innovative software features to talk about. Here, in our opinion, are the top 3:

1. Over-the-air (OTA) Updates

Today’s cars are collections of embedded systems, each running their own software stacks. As security issues are uncovered, updates need to be rolled out across fleets of cars as quickly as possible. Plus, with product life-cycles growing longer, software needs to be supported in production for longer and that often means shipping new features to older vehicles.

OTA updates prevent the need to recall cars simply to apply software updates. The ability to apply updates with cars still on the road is a major advantage because recalls inconvenience customers and can lead to reputational damage for manufacturers. They’re also extremely expensive.

As of the Electric Eel release, the AGL UCB includes a software layer for OTA upgrades. AGL uses the OSTree and Aktualizr projects to provide atomic, full file-system upgrades with rollback capability. As OSTree uses the file system to perform updates, it minimizes the network bandwidth and data storage requirements.

2. A Wide Range of Hardware Board Support

One of AGL’s goals for the UCB is to provide a single software platform for the entire automotive industry, and the UCB provides 70-80% of a finished product out of the box. That obviously includes applications, connectivity, and graphics, but it also includes a wide range of hardware board support for vendors such as Renesas, Qualcomm Technologies, Intel, Texas Instrument, NXP and Raspberry Pi.

And developers continue to contribute new support. Initial 64-bit ARM support was merged for the Electric Eel release earlier this year with the addition of the Renesas R-CAR Generation 3. Now, Renesas is working on contributing support for the ARM big.LITTLE architecture which can provide substantial performance improvements and power savings when combined with Energy-Aware Scheduler (EAS) patch series that’s part of the Renesas BSP.

3. Automated Continuous Integration and Testing

AGL is a “code first” project which means that it favors working code over writing lengthy specifications. All of that code has to be tested, and testing quickly and thoroughly is an important part of accelerating time-to-market with AGL’s shared software platform.

AGL has built extensive infrastructure for testing code changes. Defining the architecture of that testing infrastructure is the job of the Continuous Integration and Test (CIAT) expert group who, like all AGL expert groups, meet regularly to plan and coordinate development efforts.

Within the CIAT infrastructure, open-source software projects such as Fuego, LAVA, and Kernel CI provide comprehensive testing and allow every gerrit submission to be built and smoke tested on a range of hardware. But new changes aren’t just tested in isolation: daily snapshots are built and tested to catch any integration issues as early as possible.

We’ve written previously about how BayLibre contributed changes to both Kernel CI and AGL to harness the power of scalable automated testing, and we even housed a temporary Kernel CI backend in our office while the AGL project transitioned to their own instance. To see AGL’s Kernel CI instance in action visit

Check out the AGL Booth at CES 2019

These are just some of the technical highlights of the AGL UCB. With over 2000 commits so far in 2018, new features and bug fixes are shipped in every release. If you’re attending CES 2019, January 8-11 in Las Vegas, be sure to check out the AGL Demo Showcase (Westgate Hotel Pavilion, booth 1614) to see the UCB on display.