CentOS Hyperscale SIG Quarterly Report for 2024Q2

This report covers work that happened between January 27th 2024 and July 3rd 2024. For previous work, see the 2023Q4 report.

Purpose

The Hyperscale SIG focuses on enabling CentOS Stream deployment on large-scale infrastructures and facilitating collaboration on packages and tooling.

Membership update

Since the last update, the SIG membership has remained the same.

We welcome anybody that’s interested and willing to do work within the scope of the SIG to join and contribute. See the membership page in our user documentation for the current members list and how to join.

Releases and Packages

Unless otherwise specified, packages are available in our main repository, which can be enabled with dnf install centos-hyperscale-release. Please report any issues with these packages on our package-bugs tracker.

CentOS Stream 10 is now available!

CentOS Stream 10 composes started becoming available earlier in June, and SIG support was announced shortly after. Thanks to this we are now able to build Hyperscale content for the new distribution as well! As CentOS Stream 10 is not officially released yet, SIG content should be considered preliminary at this point, and we expect things to stabilize as development continues and we get closer to the release. Our plan is to have a full complement of deliverables for Hyperscale at release time.

CentOS Stream 8 is EOL

Following the official EOL, we have stopped building content for CentOS Stream 8, and will be updating our documentation accordingly. Previously-released packages are still available on vault, but will no longer be updated.

systemd

We released systemd 255 for CentOS Stream 9. Additionally, all systemd integration tests now run in a CentOS Stream 9 Hyperscale image on every pull request in the upstream systemd repository. This additional coverage helps us catch issues in pull requests before they are merged. For systemd 256, we're planning to simplify the backporting workflow by working directly from the stable branch in the upstream systemd repository instead of maintaining our own branch.

We've also done our first build of systemd for CentOS Stream 10. This is still systemd 255 at the moment but we're expecting to do a new build with the newly released systemd 256 very soon.

Finally, as the CentOS Stream 9 SELinux policy development is now done in the open on github (https://github.com/fedora-selinux/selinux-policy/tree/c9s), we are aiming to retire our custom systemd selinux policy when we release systemd v256 and instead propose to have the necessary policy fixes backported to the CentOS Stream SELinux policy from this point onwards.

Kernel

We released kernel 6.8.5-0.hs1 for CentOS Stream 9. This continues the work on releasing kernels based on upstream Fedora release kernels rather than the CentOS/RHEL kernels to ease integrating new features and capabilities into CentOS Hyperscale.

For the foreseeable future, the Hyperscale SIG will be tracking Fedora kernels to build and release into CentOS. The kernel is still built with a RHEL-like configuration, modulo changes for CentOS Hyperscale specifically.

Work has started for the next rebase, targeting CentOS Stream 9 and CentOS Stream 10 simultaneously as part of bringup for CentOS Stream Hyperscale 10.

Spin images

Updated images for CentOS Stream Hyperscale built in May 2024 have been released. In addition to the experimental workstation live media providing KDE Plasma and GNOME desktop experiences, new experimental cloud images compatible with OpenStack and Amazon EC2 are now available.

We intend to explore having cloud images available in public cloud marketplaces as we work through developing our cloud images.

Package updates

As mentioned above, we've started building packages for CentOS Stream 10, including btrfs-progs, compsize and snapper to bootstrap some of the userspace tooling we need for btrfs support in the distribution. We've also build a new kpatch update for both CentOS Stream 10 and CentOS Stream 9, catching up with a number of upstream improvements and bug fixes.

Other updates in CentOS Stream 9 include:

• a new librepo build merging in the latest upstream changes
• updated builds of our qemu and libvirt backports from Fedora; these package are now officially released, but they could use additional testing and we'd appreciate any available feedback
• emacs rebased to 29.4 to pick up the latest security fixes
• mkosi updated to version 22. mkosi is a crucial tool for testing systemd and as systemd generally depends on a very new version of mkosi, we backport it along with systemd to make sure it is up to date and can be used for systemd development.

New packages:

• dracut 059 backported from Fedora which is required by newer releases of systemd to make sure the initramfs contains the necessary systemd binaries.
• ed 1.19 backported from Fedora, to fix regressions in 1.14 (tracked in RHEL-32549)
• elfutils 0.191 backported from Fedora with static subpackages enabled, for developers who need to build binaries portable across OS releases
• iproute 6.7.0 with improved BPF support
• libnvme 1.8 and nvme-cli 2.8 backported from Fedora
• tmux 3.4 backported from Fedora, adding support for OSC 8 hyperlinks
• vim 9.1.113 backported from Fedora, featuring improved syntax highlighting for Python
• traceroute 2.1.5 backported from Fedora, which includes support for ICMP extension headers, improving debuggability in IPv6-only environments

Notable updates in other repositories:

• The facebook build of openssh now incorporates the fix for CVE-2024-6387 (regreSSHion). It has Meta-specific additions that are mostly in the process of being upstreamed, notably LOG_SESSION_ID support for non-PAM environments. Interested parties can rebuild this to pick up the CVE fix, and the Meta patches won't be applied.

DNF/RPM stack with CoW support

We submitted a new version of the Copy-on-Write patchset upstream, but more work is still required. We had a fruitful discussion with the upstream RPM developers at DevConf.CZ, and are in the process of integrating the feedback we've received to move this feature forward.

Health and Activity

The SIG continues to maintain a healthy development pace.

Meetings

The SIG holds regular bi-weekly meetings on Wednesdays at 16:00 UTC. Meetings are logged and the minutes for past meetings are available.

The SIG uses the #centos-hyperscale:fedoraproject.org Matrix room for ad-hoc communication and work coordination; this room used to be bridged to the #centos-hyperscale IRC channel, but the bridge is currently inoperative. For async discussions and announcements we generally use the centos-devel mailing list. The SIG also holds open monthly video conference sessions to promote collaboration and social interaction.

Conference talks

During the last quarter members of the CentOS Hyperscale SIG attended several conferences where they were able to share the work the SIG is doing and meet up in person. We presented an update on SIG activities at CentOS Connect at FOSDEM, where we also hosted a SIG meetup (see below). Later in the year we then talked about the work we do in Hyperscale and how it relates to the wider CentOS project at LinuxFest Northwest and Red Hat Summit. Finally, we also had a presence at FOSDEM, SCALE 21x, Texas Linux Fest, LSFMMBPF and DevConf.CZ.

We maintain a page tracking conference presentations around Hyperscale-related topics. You can find references there to all talks mentioned below, including video recordings where available. If you’d like to meet us in person at a future event please reach out.

Later this year we're planning to attend Flock to Fedora, DevConf.US, Linux Plumbers and All Systems Go.

Face-to-face meetup

The first day of CentOS Connect we held another in-person meetup.

Like the previous events, this meetup gave us an opportunity to socialize face to face, leading to plenty of great conversation and significant progress on several work items. The meetup wasn’t recorded, but there are notes of the discussion if you’re interested.

Live streams

The SIG periodically does work live on Twitch from its official Twitch channel. Interested parties who want to watch and interact with us as we do work should follow us on Twitch to get notified for when we stream.

Planned work

The SIG tracks pending work as issues on our Pagure repository. Notable projects currently in flight include:

• continuing the CentOS Stream 10 bringup
• shipping an updated QEMU package in EPEL
• integrate btrfs transactional updates as an optional feature