Linux Kernel 5.0 released, our contributions

BayLibre has continued our contribution to the Linux community as seen with this new version of Linux Kernel 5.0, released on Sunday, March 3rd 2019.  An excellent summary of this release can be found at KernelNewbies.

Here is a summary of our contributions, organized by SoC family and a summary graph of contributions by developer.

Amlogic SoC family:

  • new board: S805X-based “La Frite” from Libre Computer
  • audio: support S/PDIF input on AXG SoC family
  • audio: enable sound modules for S400 board
  • clocks: added new clock measure driver
  • display: support HDMI 1.4 4K modes

Ti DaVinci SoC Family:

  • fix label names in GPIO lookup entries
  • use nvmem to lookup MAC address

Misc:

  • Corentin Labbe: several fixes and cleanups in the crypto subsystem
  • Carlo Caione: fix boot from eMMC on i.MX8MQ
  • Bartosz Golaszewski: nexted IRQ fixes in GPIO subsystem

Our Upstream Kernel Contributions for TI SoCs

Texas Instruments (TI) supplies chips to roughly 100,000 customers around the world, with product life cycles measured in years and decades. These long-tail chips include TI’s DaVinci SoC which is particularly popular in IP network cameras, video communications, and portable media players, and also include the OMAP SoC, featured heavily on single-board computer development platforms such as the BeagleBone and the PandaBoard.

A large portion of the devices containing these SoCs will run Linux, and that makes Linux kernel support a priority. However, because the Linux kernel is routinely updated with bug fixes and new features, ensuring on-going support for TI’s SoCs requires constant development effort to keep out-of-tree patches up to date.

Luckily, there’s a way to lessen the load: by keeping TI’s SoCs supported in the upstream kernel.

The Business Case for Upstreaming

The biggest headache with supporting long-tail chips for years with out-of-tree patches is that it requires forward porting those patches to newer Long-Term Support (LTS) kernel releases. It’s a huge investment of engineering time, not just to update the patches, but also because every time that happens tests need to be run to check that no bugs have been introduced.

Merging TI SoC support upstream doesn’t completely eliminate this problem but does make it much more manageable. Instead of having to deal with migrating patches to a brand new LTS version, code can be updated piecemeal, as and when things like new APIs and bug fixes are merged.

And this is where BayLibre comes in. We’ve been working on the upstream kernel to ensure that, not only does every release work on the DaVinci and OMAP SoCs, but that those SoCs also take advantage of the latest bug fixes and features.

Our Upstream Contributions

BayLibre TI Contributions vs Linux Kernel Release bar chart

The majority of our work has been on the DaVinci SoC. In the last two years, we’ve contributed 99 patches to the Linux kernel. Bartosz Golaszewski and Kevin Hilman added support to the Video Port Interface (VPIF) driver to allow it to be initialised with device tree (DT) bindings; for many SoCs, DT bindings are the preferred way to describe hardware characteristics. In addition, Bartosz and Kevin added support for raw image capture with sensors supporting the 10-bit SGRBG10 format. Along the way, the guys cleaned up a slew of hard-coded assumptions about image formats in the VPIF driver that came from it only previously being tested with a single sensor that supported a single 8-bit format.

We’ve also worked on performance improvements by taking advantage of the DMA hardware in DaVinci SoCs. Fabian Parent enabled DMA channels for the SPI driver so that it correctly maps data buffers passed from the MTD layer. And Alex Baildon made CCPI DMA work on the DA850/OMAP-L138/AM18xx platforms to improve the performance of USB. Both of these efforts reduce the work the CPU has to do when transferring data to and from devices, which means a more stable workload.

Lastly, we helped with a large effort to move DaVinci to the common clock framework (CCF). This work involved rewriting hand-crafted clock drivers to use the CCF. A huge part of the migration to the CCF was done by an independent developer, David Lechner, and he deserves special mention. From BayLibre, Bartosz modified some of the clock driver consumers, including updating the remoteproc driver to use the reset framework and deduplicated code that existed in both the platform-specific support and the generic aemif driver.

Plans for 2019

As we said at the beginning of this post, maintaining support upstream requires on-going effort and we have further plans for the TI SoC code in 2019. We’ll be creating a proper clocksource driver to tidy up the random bits in the platform support, enabling support for sparse interrupts (critical for supporting some hardware), and eventually building DaVinci as part of the multi_v5_defconfig kernel config.

U-Boot v2019.01 released, our contributions

BayLibre has continued contribution to the open-source community as seen with this new version of U-Boot v2019.01, released on 14th January 2019.

This release introduces a big rework for Amlogic Meson SoC support in order to:

  • Add support for the new AXG platform and ease support for G12A/G12B platforms
  • Ease support for new boards based on Amlogic Reference Design

Starting with this release, Neil Armstrong now maintains the Amlogic SoC support with an U-Boot Custodian tree, similar to a kernel.org maintainer tree, now collecting patches for the Amlogic SoC support (board, arch, clock, pinctrl, …) and sending pull-requests to Tom Rini, the U-Boot maintainer.

Since U-Boot v2017.11, BayLibre engineers have pushed:

  • 75 patches
  • 14804 line changes
  • 292 files changes

Here is a summary of our contributions for this release:

Amlogic SoC family:

  • Add boot device mode, usable with pyamlboot
  • Add support for AXG family
    • pinctrl
    • dts
    • clock
    • network
    • mmc
  • Rework GX support by:
    • moving all P212 boards variants under the P212 board code
    • moving Khadas VIM2 support under the Q200 board code
    • moving Nanopi-K2 support under the Odroid-C2 board code
  • Add SPIFC flash controller for the upcoming “La Frite” board
  • Add VPU Power domain and VPU clocks support for upcoming video support

 

Linux Kernel 4.20 released, our contributions

BayLibre has continued our contribution to the Linux community as seen with this new version of Linux Kernel 4.20, released on Sunday, 23rd December 2018.  An excellent summary of this release can be found at KernelNewbies.

Here is a summary of our contributions, organized by SoC family and a summary graph of contributions by developer.

Amlogic SoC family:

  • Added AXG PDM Input support, with Devicetree Nodes
  • Added common “Canvas” provider for DRM Display and upcoming V4L2 Decoder Driver
  • Added support for Serial Number readout from sysfs
  • Various Pinctrl, DRM, Audio and clock fixes

Ti DaVinci SoC Family:

  • fixed a GPIO-related regression present since v4.19
  • Finally killed davinci_clk_reset_assert/deassert()

Misc:

  • Bartosz Golaszewski became co-maintainer of the GPIO sub-system
  • nvmem: Bartosz fixed problems with the nvmem consumer API
  • Still some cleanups from Corentin in Crypto, Network and various other sub-systems
  • Fabien Parent added support for dedicated power supply in ChromeOS’s charger driver

 

 

Hardware Accelerated Video Decoding and System Load Monitoring Demos

In the past few months, BayLibre engineers Maxime Jourdan, Alexandre Bailon and Neil Armstrong showcased two demos to illustrate some of their recent work: fully hardware-accelerated video decoding which was linked to non-intrusive System Load monitoring via JTAG.

While these two demos seem unrelated, the non-intrusive System Load monitoring via JTAG developed by Alexandre Bailon was a good way to prove that the Amlogic Video Decoder driver from Maxime Jourdan and the Amlogic Video Processing Unit graphics output work well together, and that it’s possible to monitor system load without causing video frames to be dropped.

First of all, Maxime Jourdan did a talk at Embedded Recipes 2018 in Paris about his work on developing and upstreaming the Amlogic Video Decoder driver for the Amlogic S905, S905X, S9095D and S9012 SoCs.

You can access the talk here :

And slides at: https://www.slideshare.net/ennael/embedded-recipes-2018-upstream-multimedia-on-amlogic-so-cs-from-fiction-to-reality-maxime-jourdan

Then, Alexandre Bailon and Patrick Titiano spoke about their “libSoCCA” project which gets the real-time statistics of a running system via the well-known JTAG interface without interfering with the system’s execution or requiring any modifications to the code.

The ultimate demonstration was to show, in real-time, the CPU Load and CPU Bus accesses of a Libre Computer AML-S905X-CC system (running the LibreELEC Kodi distribution) with a steady 10% load decoding 50mbps 4K H.264 and 4K H.265 10-bit video samples from the JellyFish Video Bitrate test files http://jell.yfish.us/. And all without changing a single byte of the Linux filesystem or Kodi binaries.

The most interesting fact of this demo is that Kodi doesn’t have any platform-specific code to handle Accelerated Hardware Video decoding, nor does FFmpeg which speaks to the decoder driver.

All of this is made possible thanks to the Linux DRM (Direct Rendering Manager) KMS (Kernel Mode Setting) GBM (Graphics Buffer Management) display support handled in Kodi, and the V4L2 (Video For Linux 2) Memory2Memory Hardware Decoder support from FFmpeg.

With these two graphics subsystems combined, decoded frames from the V4L2 interface can be passed to the DRM Video driver and scaled, blended and displayed. And thanks to the Linux DMA-BUF framework, none of the frames need to be copied.

Linux Kernel 4.19 released, our contributions

BayLibre has continued our contribution to the Linux community as seen with this new version of Linux Kernel 4.19, released on Sunday, 21th October 2018.  An excellent summary of this release can be found at KernelNewbies.

Here is a summary of our contributions, organized by SoC family and a summary graph of contributions by developer.

Amlogic SoC family:

  • Initial support of the audio hardware on the Amlogic AXG SoC : A113D
    • Audio clock controller
    • Audio reset controller
    • Pinctrl missing configurations
    • ALSA SoC card
    • TDM input/output
    • SPDIF output
    • HW FIFOs handling
    • es7134 codec support
    • tas517x support
    • DT audio and pinctrl nodes
  • Add S805X based P241 board
  • Finally enable graphics output on the FriendlyElec Nanopi-K2
  • Enable DMT (Display Monitor Timing) HDMI output to support generic, non-HDMI monitors (DVI, VGA via HDMI->VGA dongle or monitors with custom timings in EDID)

Ti DaVinci SoC Family:

  • switch davinci SoCs to use the ti-aemif driver
  • switch to the reset framework for the DSP remoteproc

Misc:

  • Lots of cleanup and some code removal from Corentin
  • Various Clock & ALSA SoC fixups to make AXG Audio work
  • Add support for the CEC functionality on the upcoming Asus ChromeBox via the Embedded Controller interface and the Linux CEC Framework maintained by Hans Verkuil

 

 

U-Boot v2018.07 released, our contributions

BayLibre has continued contribution to the open-source community as seen with this new version of U-Boot v2018.07, released on 27th July 2018.

This release permits booting over an USB Mass Storage (USB Stick or Hard Drive) with EFI on the Libre Computer AML-S905X-CC board !

With U-Boot 2018.07, you can download the nighly Debian-Installer or OpenSuse  TumbleTweed, copy U-boot to a blank SDCard, copy the installer to an USB, plug USB and SDCard, boot and install !

Here is a summary of our contributions:

Amlogic SoC family:

  • Add support for the USB PHY and Controller on the Meson GXL (S905X, S905D) SoC based on the Linux version done by Martin Blumenstingl
  • Add Analog-to-Digital (ADC) driver based on the Linux version done by Martin Blumenstingl
  • Add Amlogic Reset Controller
  • Add ADC cli command
  • Sync DT with Linux 4.17

Misc :

  • Add “bulk” commands for Reset and Clocks + tests
  • Add dwc3-of-simple USB DWC3 simple Glue driver based on the Linux version
  • Add support for any number of PHYs for the DWC3 controller

Linux Kernel 4.18 released, our contributions

BayLibre has continued our contribution to the Linux community as seen with this new version of Linux Kernel 4.18, released on Sunday, 12th August 2018.  An excellent summary of this release can be found at KernelNewbies.

Here is a summary of our contributions, organized by SoC family and a summary graph of contributions by developer.

Amlogic SoC family:

  • Multiple fixups/enhancements for the AXG A113D SoC (I2C, TDM)
  • Add write support for NVMEM
  • MMC/SDCard Fixups
  • Boot fixup with SCPI

Ti DaVinci SoC Family:

  • Fixup for NAND support
  • Fixups for RemoteProc support
  • Fixups for AEMIF support

Misc:

  • Lots of cleanup and some code removal

 

 

An Overview of Generic Power Domains (genpd) on Linux

The saying goes that most programming problems can be solved with another layer of abstraction. Generic Power domains (genpd) are a (relatively) new Linux kernel power management abstraction that model the way power is controlled for components on SoCs. In other words, genpd describes the relationship between devices and power controllers. Generic power domains allow these relationships to be declared outside of device drivers, which means these device groups can be updated at runtime.

Static versus Dynamic power management

Linux supports two types of idle power management: static and dynamic. Static power management is the term for suspend and resume operations; it controls system-wide power. This is the traditional way to suspend a system and comes from the original support for laptop PCs where the whole system would be suspended at once. A good example of this is closing the laptop lid.

Dynamic power management, on the other hand, controls power to individual devices, allowing flexible power management based on activity in different parts of the SoC/platform.

Dynamic power management for device drivers is implemented inside of the kernel with the runtime PM framework (dynamic power management for CPUs is handled differently, via the CPUfreq and CPUidle subsystems). Device drivers use the struct dev_pm_ops data object to hook up their callbacks:

These callbacks allow devices to be powered up or down, or enter a low-power idle state when they go idle, such as when a transaction is finished. Each individual driver decides when the device is idle.

Device drivers can inform the PM core when devices are going idle. Idleness is controlled using the pm_runtime_get() and pm_runtime_put() functions. This API maintains a reference counter for each device to track whether the device is in-use at any time. pm_runtime_get() blocks the device from entering a low-power mode until the last pm_runtime_put().

But controlling power on a per-device level, while better than system-wide, isn’t the most efficient way to save power. Instead, hardware vendors have long provided a way to control power to groups of devices through power islands, or power domains.

 

What are power domains?

Devices and SoCs have separate power rails with devices connected to them. Power domains are a hardware concept for managing devices sharing power rails. Their power voltages are correlated. While the hardware power islands have existed for many years, the kernel has only recently added support for them.

Linux power domain drivers sit above devices in the PM hierarchy, at the same level as bus_type.

Power management core hierarchy diagram

These relationships need to be modeled inside of the kernel because the same IP blocks and chip families might have different groups depending on the hardware. For example, the same ARM IP can be used in different SoC vendor products and they might group things differently. So you need a way to describe this fixed relationship. Some examples are whether all CPUs/clusters share a power rail or have independent rails, or whether video-centric IP shares a power rail with the display IP.

A solution is needed that abstracts this so that the driver doesn’t need to care. And these power domain relationships need to be described in a way that does not require updating drivers for every new platform and configuration.

Crucially, we don’t want a driver to care whether it’s running on an AmLogic or Qualcomm chip — it should work on both, independent of how the power rails are configured on those platforms.

 

Generic Power Domains (genpd)

Generic power domains are an abstraction on top of Linux kernel power domains. They provide a number of benefits over regular power domains because they support:

  • Domain hierarchies
  • Adding and removing devices from domains at runtime
  • Power governors
  • Device Tree support

Generic power domain drivers control the entire domain’s power when an entire group of devices goes idle. A typical action is sending a command to a microcontroller or writing to a register to disable a voltage rail. The genpd framework takes care of running the callbacks for every genpd driver in a hierarchy, a feature that makes it possible to control the nested power domains available on some SoCs.

genpd drivers can be both IRQ-safe and always-on. The former (enabled with the GENPD_FLAG_IRQ_SAFE) is useful for domains that can be powered on/off in atomic context. This feature is particularly useful for devices that have low suspend and resume latency, and can be powered on or off quickly.

Always-on genpd drivers make sense if you have devices that must not be powered off. For example, CPUs that are behind a power microcontroller. Whether or not genpd drivers set the GENPD_FLAG_ALWAYS_ON flag can be platform-dependent.

In addition, there are optional driver callbacks that can be used if drivers want to know when new devices are attached or detached from a generic power domain. These callbacks are used to execute power domain-specific actions.

Generic power domains are described in the device tree, so the configuration can be described on a per-platform basis. Storing the configuration data inside device tree also makes it easy to update since the genpd driver requires no changes when new SoCs are released; a single driver can support all platform configurations.

Here’s a simple genpd example:

Here’s how it would be used by a device:

genpd also provides governors which add another level of control in between the PM core and the device drivers. Every generic power domain can have its own governor. Governors decide whether power should be gated to a generic power domain, and their decisions are based on simple heuristics. There are two available in the kernel: the simple QoS governor and the always-on governor.

As the name implies, the always-on governor prevents devices from being powered off. The simple QoS governor is more interesting — it considers the time it takes to power devices on and off, known as suspend and resume latency. If a power domain is predicted to be resumed before power can be turned off to all devices, the simple QoS governor will prevent the power domain from suspending.

Governors have a standard plugin-architecture, so you can write custom versions for specific platforms as well.

 

Performance states

A recent addition to the kernel, in v4.15, is support for power domain performance states. This new feature makes it possible to finely tune the power consumption of devices by varying the voltage to power rails. genpd drivers implement this with the (*set_performance_state) callback. Sadly, there are no upstream users of this API yet, but one is currently being reviewed on the kernel mailing lists.

 

Resources

The genpd framework provides an extremely flexible API for finely controlling power domains. If you want to learn more, here is a list of resources:

And here’s the video of Kevin’s talk on generic power domains at Kernel Recipes 2017.

Linux Kernel 4.17 released, our contributions

BayLibre has continued our contribution to the Linux community as seen with this new version of Linux Kernel 4.17, released on June, 3rd 2018.  An excellent summary of this release can be found at KernelNewbies.

We’re happy to report BayLibre is once again featured in the LWN stats for this release. Corentin Labbe from our team is a Top 10 developer and BayLibre is a Top 20 employer, both by Lines Changed.

Here is a summary of our contributions, organized by SoC family and a summary graph of contributions by developer.

Amlogic SoC family:

  • clocks: major rework to switch to regmap; misc. cleanups
  • improve DT support for WeTek hub and play2
  • MMC: reduce max speed for Odroid-C2 boards after multiple problems reported
  • AXG family: enable hardware RNG
  • HDMI: Add support for DMT modes

Misc:

  • Lots of cleanup and dead code removal (see Lines removed by Corentin)