SIG authentication retooling

Monday , 4, May 2020 Jim Perrin announcement, General Leave a comment

You may have seen the emails from Aoife about the work the Community Platform Engineering (CPE) team is doing around authentication tooling, and what that might mean for CentOS. Here’s a brief explainer for what’s happening.

The authentication software we use for SIGs (FAS or Fedora Account System) and a few other bits around the project will be EOL fairly soon. This is a 10+ year old, difficult to maintain software project with bugs that can’t be effectively addressed with its old code. The CPE team is writing a replacement for FAS that uses more of the standard distribution components, largely built around FreeIPA. This new tooling is intended to be an upgrade for use by anyone, but particularly Fedora and CentOS to replace both uses of FAS currently. There are a number of feature improvements and standardizations included in the new software, but in the end users shouldn’t notice any real impact in operation.

As we engaged with stakeholders including SIG chairs, the CentOS QA team, and other prominent community members, one issue became quickly apparent. We have many SIG contributors who push their work into both CentOS and Fedora, as well as Fedora’s EPEL repository. Having to work with separate auth systems makes it more difficult with automation, testing, and other parts of the contributor workflow. Because of this chance to improve the lives of our current and incoming contributors, our intention with the new authentication rewrite is for the CentOS and Fedora projects to share a single, unified authentication system. This would allow members of our communities who contribute in multiple places to do so via a single account, while having negligible impact on users who don’t. Group management, permissions, etc. will still be the purview of each project to manage as they see fit.

Fixing this gap between the auth systems the CPE team uses also solves some problems for the team itself. Sharing this system also encourages more cross-team work, which benefits both projects and communities (more hands). These communities are already sharing some resources, such as Fedora making use of the CentOS CI system. This work paves the way for easier resource sharing and management, which will cut down on the amount of duplicative work done across both infrastructures.

Over the next few months as the CPE team works toward its October implementation goal, you’ll see additional communication and messaging about the project. That doesn’t mean you need to wait to get involved though. If you’re interested in how we’re designing the auth, or want to participate in the development, please have a look at the git repository and see where you can help!

Release: CentOS Linux 7 (2003)

Tuesday , 28, April 2020 Rich Bowen announcement Leave a comment

We are very pleased to announce the latest release in the CentOS 7 series. The full release announcements may be seen on the centos-devel mailing list:

These releases were made possible due to the hard work of many people, and we thank all of them for their help as we move this platform forward.

New CentOS Project licensing policy

Monday , 27, April 2020 Karsten Wade announcement Leave a comment

Today the CentOS Project is rolling out a comprehensive licensing policy to document how licensing has been conducted normally in the Project, along with filling gaps that are crucial for being a contributor project. Your feedback and questions are welcome on the centos-devel mailing list. Please read the following for more detail and background.

One of the effects of adding CentOS Stream highlighted the fact we do not have any kind of clear policy about licensing contributions to the project. Obviously people have been contributing code and content to the project for a very long time, but none of those contributions went into the core Linux distribution. With CentOS Stream comes the need to manage a comparatively huge firehose of contributions needing clear guidelines and policies.

This gave us a chance to look at the state of the licenses the CentOS Project and its contributors put on code and content that originates in the project itself. Examples of this might be spec files for RPM packages, documentation on the wiki, or contributions to the branding of the project itself.

Presenting clear statements about how content and code is going to be licensed is a standard part of any open source project. The introduction of CentOS Stream has just raised the visibility of not having a licensing policy. The Board of Directors feels this policy better serves the needs of contributors and users.

https://centos.org/legal/licensing-policy/

An important purpose of this licensing policy is to provide a Default License--we’ve selected the MIT license--and a clear notice of attribution to the project under the MIT license. This Default License is used when a contribution does not have a license attached or is not destined for a repository that is already licensed, e.g. has a LICENSE file.

Otherwise, contributions are under the license of whatever is covering the rest of the content and code base--when you contribute to a software project repository, you put the contribution under the same license as the rest of the content and code in that repository.

This is also an opportunity to upgrade the version of our wiki license to CC BY-SA 4.0 for anything going forward; it is backward compatible with existing 3.0 content. This is a clean upgrade and improvement for the Project.

The CentOS Board of Directors received advice and drafting from Red Hat Legal in crafting this policy.

Your questions about this policy are welcome; the best place to discuss is the centos-devel mailing list.

CPE Weekly 2020-04-18

Monday , 20, April 2020 Rich Bowen Uncategorized Leave a comment

Background:

The Community Platform Engineering group is the Red Hat team combining
IT and release engineering from Fedora and CentOS. Check out our
team's info here https://docs.fedoraproject.org/en-US/cpe/

GitForge Updates

We are still actively engaging with the Fedora Council and have not
made any further progress with this project. We are focusing on
ensuring we have captured the most granular of requirements before
further engaging with GitLab by rereviewing what we have gathered so
far and requesting more specific technical clarifications on some.
We are also as a team discussing the best way to track the updates on
this project publicly and hope to have something published early next
week. We will send an email to the devel list as soon as we have
agreed and created this.
Thank you for your patience and engagement with us thus far.

Fedora Updates

* Fedora 32 release: RC 1.3 was tested this week
* Go/No-Go meeting happened on Thursday 16th April - decision:No Go
https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org/thread/HXSBRI4LRWKKHLUH2OI4UPBKJJKGCDQR/

Data Centre Move

* Please note Communishift is now down and is en route to the
datacentre in RDU-CC.
* We have a list of affected services that will be in effect from May
25th - July 1st (est) as part of the full datacentre move. You can
view that list here https://hackmd.io/hpYYJQRjQy-oHxUS7IonIA
* As always, please view our public schedule here for more a more
detailed overview
https://hackmd.io/@fedorainfra2020/rJpsA4FLL#First-draft-of-schedule-for-PHX2--gt-IAD2-move

AAA Replacement

* The team officially kicked off phase two of the project development today.
* They are going to look at applications that currently use the FAS
client to change their codebase to the FASJSON client post production.
* The team are also working on adding a python library to the API so
that this maps to python objects easily.
* They are also working on API endpoints, specifically to be able to
retrieve the open-api specification in JSON format & provide a health
endpoint for monitoring tools.
* Our work is publicly tracked here if you want to find out more
https://github.com/orgs/fedora-infra/projects/6

CI/CD

* rpmautospec is available for testing in staging, find more info here
https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org/thread/LWE4URIRWVTEZKXKP7QOK5JXFNVJRUNW/

Sustaining Team

* The team are looking at creating a ticket dashboard for tickets and
have reused most the fedora-gather-easyfix code base to work on this
* https://fedorapeople.org/~humaton/cpe/

* Some cool stats about tickets from the infra repo here
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/thread/5OZR3ITYCQ2NYQCIHL35YJKJP22TS4LD/
* Mbbox:
* Review koji-hub CRD PR
https://github.com/fedora-infra/mbbox/pull/19

Starting to play around with koji-ansible collections for releng work (here).
Automation of the openh264 packages.

CentOS Updates

CentOS

* CentOS CI - stable with 0 downtime
* 7.8 QA tree is being tested by CentOS QA folks

CentOS Stream

* The team are working on having Stream be publically consumable in
the coming months, with SIG enablement so watch the devel lists for
more technical updates as and when they are announced!

As always, feedback is welcome, and we will continue to look at ways
to improve the delivery and readability of this weekly report.

Have a great weekend!

Aoife

Source: https://hackmd.io/gSci385uRoeNVEuoNUS1pg?view

CPE Weekly: 2020-04-14

Tuesday , 14, April 2020 Rich Bowen Uncategorized 1 Comment

# CPE Weekly: 2020-04-14

Hi All,

Apologies for the delayed weekly mail, I enjoyed a lovely four-day
weekend with my family over Easter which was important and didn't get
around to sending this email.

The upside is you get two emails from me this week instead

Background:
The Community Platform Engineering group is the Red Hat team combining
IT and release engineering from Fedora and CentOS.Check out our teams
info here https://docs.fedoraproject.org/en-US/cpe/

GitForge Updates

* Initial conversations have started with GitLab to investigate
options and weight them against more refined requirements to make sure
we reach the best possible outcome
* We are still engaging with Fedora Council
https://pagure.io/Fedora-Council/tickets/issue/292
* We are also engaging with the CentOS board too
* And we are looking at ways to have open conversations/Q&A sessions
in public forums too as we move through this journey.

Fedora Updates

* F32 release first go/no-go meeting is scheduled for this week to
review the estimated release date of 21st April
* Calendar for meetings here
https://apps.fedoraproject.org/calendar/Fedora%20release/
* F32 release schedule here https://fedoraproject.org/wiki/Releases/32/Schedule

Data Centre Move

* CommuniShift is now down until May 8th est. to facilitate the data
centre move that has now officially begun.
* Please see our list of affected https://hackmd.io/hpYYJQRjQy-oHxUS7IonIA
* Our first few sets of hardware have been deracked and are awaiting
collection from PHX2 this week.
* Please check our schedule here - we are on Week 06
https://hackmd.io/R3EkjzVyTG2TYwQvkfzYrA

AAA Replacement

* The team are planning phase two of the replacement which will target
UX/UI improvements,

CI/CD

* rpmautospec
* Call for testers has been sent to the devel-announce list:
https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedoraproject.org/thread/LWE4URIRWVTEZKXKP7QOK5JXFNVJRUNW/
* Documentation is also being refined on:
* How it works
* What are its peculiarities
* How to opt-in
* What is working and what isn't yet
* What remains to do before this gets pushed to production
* Thank you for your feedback!

Sustaining Team

* Mbbox Upgrade
* The team are making good progress on the koji-builder & koji-hub
CRD https://github.com/fedora-infra/mbbox/pull/20
* They also have resolved the issue with PVC in the mbox-operator
https://github.com/fedora-infra/mbbox/pull/23
* We are now looking into CI support and the work that this will involve
* Bodhi
* Investigation database lockups, seems related to celery worker
not completing their tasks
* Working on a 5.3 release
* Releng
* fedscm-admin 1.0.13 released
* Future openh264 composes will use ODCS, script is added to releng repo
* Infra
* The daily standup the team has has helped a lot with managing
infra tickets - they are down to 99 tickets!
* Mass update of stg and prod
* Please note you may experience some Kojira slowness
* New review-stats application deployed -
(https://fedoraproject.org/PackageReviewStatus/)

CentOS Updates

CentOS

* CentOS CI is stable and all working fine
* Investigation for cloud vms addition to cico is also underway
* RHEL 8.1 batch update 07-Apr

CentOS Stream

* We have now ~200 packages built in CentOS Stream & we expect a few
hundred more to become available to us soon!
* The team are also still working on general compose config, koji tag
& housekeeping

As always, feedback is welcome, and we will continue to look at ways
to improve the delivery and readability of this weekly report.

Have a great weekend!

Aoife

Source: https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ

-

Aoife Moloney
Product Owner
Community Platform Engineering Team
Red Hat EMEA
Communications House
Cork Road
Waterford

Welcome to our new Board members!

Tuesday , 7, April 2020 Rich Bowen announcement 1 Comment

The CentOS community, along with the Governing Board, is pleased to welcome two new members to the Board. Effective 8th April 2020, Thomas Oulevey and Patrick Riehecky will be joining the project leadership. (See also KB's announcement on the centos-devel mailing list.)

I spoke with Karsten Wade, who is a board member, about how the board selection and appointment process works:

"This is our first time doing this, and in planning it out, it quickly became apparent we didn't give ourselves a detailed roadmap back in 2014 with the new Project governance. The guidance was simply that the Board decides for itself who sits on the Board, candidates must have a body of work that benefits the CentOS Project and recognition as a leader in the community already, and we should review the make-up of the Board on a regular basis.

 

It turned out that such minimal guidance can be useful for remaining flexible, but it also meant we had to gut-check ourselves at every turn that we were following some kind of best practices. In the end, this time, we knew who we wanted and we just kept working things until we get there.

 

One of the activities of the Board in this coming year is going to be improving self-documentation, which I think will be a natural outfall requirement from the open goals refresh we're starting now that Thomas and Patrick are onboard. Our intention is to define CentOS leadership criteria as part of an open discussion process with the community. In the end, that is the true gift we gave ourselves back in 2014--the ability to operate in good faith under a set of governing principles that were good enough and the flexibility to evolve those principles when the time is right."

The two new Directors bring a wealth of experience and technical background to the table. They are also notable in that they come from two organizations which are actively involved in contributing to the project, and so they understand real-world interests and concerns.

Thomas Oulevey works in the Controls group within the CERN Beams department. As a system engineer he contributes his Linux knowledge to improve the exploitation of the Accelerator complex and technical infrastructure. Thomas has been contributing to CentOS since 2012, as a member of the infrastructure team. He helped to bootstrap few Special Interest Groups, helped the QA team with reports, and now mainly design and improve the Community Build Service.

Pat Riehecky works at Fermi National Accelerator Laboratory. He is part of the Scientific Linux team and works on systems that perform Data Acquisition for various experiments at Fermilab. Officially Pat began working with the CentOS Community when Fermilab decided to use CentOS 8 as its EL8 base platform.

We're particularly pleased to enter this period of greater transparency around the CentOS Project with two new directors from outside of Red Hat, to broaden our perspective, get more industry input, and amplify the voice of the community in our decision making process.

Please join us in welcoming Thomas and Pat.

CentOS Community newsletter, April 2020 (#2004)

Tuesday , 7, April 2020 Rich Bowen Newsletter 1 Comment

Dear CentOS enthusiast,

I hope you are all well. I know that this is a very difficult time for all of you, and that you likely have other things on your mind than CentOS, so I'll try to make it interesting this month.

In this edition:

News

We have had a fairly busy month in the CentOS project, with some exciting new developments on the contributor front.

Early in the month, Fabian announced a new CBS/SIG signing process. If you are in a SIG, or contribute in some other way, you'll want to read all about the updates to the workflow, which should make things a lot easier for you. And just a few days ago they announced that it was live, and tested, and ready for you to use. If you have questions, you should drop by the #centos-devel IRC channel.

Back in January, we talked about the decision of a Git forge platform. The CPE - Community Platform Engineering - team, which does a lot of the infrastructure work for Fedora and CentOS, listed requirements for selecting a platform that Fedora and CentOS could agree on, for shared collaboration. That decision has now been made, and you can read up on all the details in our blog post from the end of last week.

Over the past few months, areguera and others have been working on updates to our primary centos.org website, and a few days ago announced a staging website where you can see the progress of that project. See www.stg.centos.org for how the new website is proposed to look, and take your comments (or, better yet, your contributions) back to the centos-devel mailing list.

The process to propose changes to the logo and our visual identity continues, and you should see THIS THREAD (and particularly responses thereto) for updates to how that process is going.

And I want to draw attention to the thread about Unshipped -devel packages in CentOS 8 and CentOS Stream which may address a concern that many of you have had in recent months.

Every week for several months now, Aoife Moloney has been posting the CPE Weekly to the centos-devel mailing list, keeping us appraised of what's going on in the Community Platform Engineering team to support the work of the CentOS project. This covers major projects like the data center move, and the AAA project, as well as daily CI/CD status and other smaller ongoing maintenance efforts. I strongly encourage you to read that each week, for some background on what's involved in keeping a project of this size moving along.

Finally, I want to make mention of the amazing work that's doing on in the Supercomputing community - much of which is powered by CentOS and Red Hat Enterprise Linux - to combat the corona virus using the power of HPC to simulate and test possible vaccines and cures. And if you want to participate in this effort with your spare CentOS computing power, have a look at the Folding@Home project which harnesses your spare CPU cycles to power the complex simulations of virus protein molecules.

Releases and updates

Errata and Enhancements Advisories

We issued the following CEEA (CentOS Errata and Enhancements Advisories) during March:

Errata and Security Advisories

We issued the following CESA (CentOS Errata and Security Advisories) during March:

Errata and Bugfix Advisories

We issued the following CEBA (CentOS Errata and Bugfix Advisories) during March:

Track CentOS 8 and CentOS Stream package updates at feeds.centos.org.

Events

As you may already know, the upcoming Red Hat Summit has been converted to a virtual experience. What you might not know is that it is free to register for it. We'll be there, with video content, live "ask the expert" sessions, and a virtual "booth" chat room where you can drop by and ask questions, or learn about what we're working on.

The virtual event will be April 28-29, at redhat.com/summit. Afterwards, all of the CentOS content we produce will also be available on our YouTube channel.

Beyond Red Hat Summit, we are carefully monitoring the coronavirus situation, and will decide on future events once the danger has passed. We're looking into how we can put on some virtual events over the coming months, particularly in cooperation with our friends in the Fedora project. If you're interested in participating, please do let us know.

Our in-person events are still very important to us, and we want to do more of them, not fewer. But we also have no interest in putting our community at risk. So, stay healthy, and stay tuned.

SIG Reports

CentOS SIGs - Special Interest Groups - are the lifeblood of the CentOS project, and the best place to get involved. These groups work on specific topics or technologies on top of CentOS. You can learn more about SIGs, and see which ones are currently available, at https://wiki.centos.org/SpecialInterestGroup
 
Most SIGs hold periodic meetings on the #centos-meeting IRC channel, on Freenode, where they discuss what they're working on, what challenges they are facing, and where help is needed. Minutes are posted publicly after the meeting, so you can catch up on what was discussed.
The schedule for these meetings is posted at https://centos.org/community/calendar/

 

CPE Weekly 2020-04-04

Saturday , 4, April 2020 Rich Bowen Uncategorized Leave a comment

Background:

The Community Platform Engineering group is the Red Hat team combining
IT and release engineering from Fedora and CentOS. Check out our teams
info here https://docs.fedoraproject.org/en-US/cpe/

GitForge Updates

*There has been a lot of discussion this week on the devel and infra
lists about the decision to move to GitLab in the near future.
Firstly, let us apologize again to the communities for our drop in
communication between the requirement collecting phase and the
decision making phase. As we have said before, it was in no way, shape
or form an intentional lapse of communications. However we do
recognize that it was still nonetheless a decision that was not made
in public, and for that we can only now offer our apologies for this
mistake and learn a hard lesson from it.

We do want to let you know that we deeply appreciate the requirements
you have given us and would like to ask you to continue engaging with
us while we are moving through our next steps with GitLab.
While the discussions on the lists are deeply emotional, they are
still incredibly valuable to us to truly comprehend the importance of
our next steps in ensuring we make the right choices in the coming
months.

Now more than ever, your guidance is needed to make sure we achieve
the best possible result for you and our team from this decision.
CPE management and I, our team's product owner, are also actively
engaging with the Fedora Council and soon the CentOS Board to make
sure that ALL of the developments and progress between us and GitLab
are publicly available.

We have a long way to go in this process and your feedback on our
progress will be vital to make sure we remain on course.

We hope in time you can understand our decision was made in good
intent for the betterment of both our team and the communities we
serve, and we hope to still be able to rely on you all as peers and
friends for feedback and guidance during this journey.*

CentOS

* CentOS CI is stable with 0 downtime!

* RHEL 7.8 released this week too!

CentOS Stream

* qt5, go-toolest (module), container tools (module) are updated to
their 8.2 versions

* We are also looking at easier automation of koji tags

Fedora Updates

* Final Freeze starts 7th April 2020 @ 1400 UTC

* Pagure 5.9.1 release pushed to both staging and pagure.io

* the-new-hotness configuration was updated
https://pagure.io/fedora-infrastructure/issue/8783

* Michal Konečný has been working on mapping the fedora infrastructure
applications, his project, (which sounds really cool and useful!) can
be found here https://github.com/Zlopez/fedora-infra-map

Data Centre Move

* Please note Communishift will be down from 13th April - 8th May to
facilitate the first shipment wave of our datacenter

* We are also still on track to switch to a reduced Fedora offering
from 25th May until est. 1st July\*.

* For a list of services we are planning to have available during this
window, please see mail thread in archive
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/thread/PN6RL7XT3V7DVC7MK46H3QDEJPL5FRI6/

* We will not have staging available so we will not have capacity to
review or deploy new or upgraded features and applications during this
time.

* As always, please view our public schedule here for more a more
detailed overview
https://hackmd.io/@fedorainfra2020/rJpsA4FLL#First-draft-of-schedule-for-PHX2--gt-IAD2-move

* We found a password, we do not know whose it is, but we have turned
it into the lost and found.

AAA Replacement

* First development phase complete & the team worked through 57 tickets in total

* The codebase was sent to our team first for demo and we will be
using feedback to develop the portal further

* During phase two we would like to change some codebases in existing
apps, and write documentation on how to upgrade applications to
redirect to the new API

* We would like to roll this request for feedback out to some
community maintainers during this phase too for another iteration on
the service and documentation

* Our work is publicly tracked here
https://github.com/orgs/fedora-infra/projects/6 so please stop by and
check out the progress we are making, and what we are looking at
working on next

CI/CD

* Monitor-gating is still running in production and giving us some
data about the health of the packager workflow:

* For example, these are the statistics between Monday and Wednesday:
39 messages retrieved
prod.monitor-gating.multi-build.end.failed -- 7
prod.monitor-gating.multi-build.end.succeeded -- 2
prod.monitor-gating.multi-build.start -- 10
prod.monitor-gating.single-build.end.failed -- 3
prod.monitor-gating.single-build.end.succeeded -- 7
prod.monitor-gating.single-build.start -- 10

* rpmautospec 0.0.1 through 0.0.10 have been released and deployed in staging

* We got two builds to go through fine, from the same commit,
getting two different NEVR and an auto-generated changelog

* However, for this to happen, we had to tweak a couple of things
on the builder which is not really ideal/acceptable, so we moved a
part of the processing inside the chroot where the SRPM will be built,
which solved the main issues we faced

* To be done: Make tagging (latest) existing builds a separate
operation executed outside the build root, to avoid having to talk to
Koji inside it.

* In other words: Stay put, we're getting there!

Sustaining Team

* Mbbox

* The team have made some progress on koji-builder CRD

* The team are also working on Bodhi 5.3 with few improvements and bug
fixes from 5.2.

* The team had a reboot and Update cycle this week.

* The team are also discussing ideas around a releng bot to help
process the tickets that require manual interventions.

* They also add more updates to compose-tracker

* And started playing with odcs on new
odcs-backend-releng01.phx2.fedoraproject.org

As always, feedback is welcome, and we will continue to look at ways
to improve the delivery and readability of this weekly report.

Have a great weekend!

Aoife

Source: https://hackmd.io/gSci385uRoeNVEuoNUS1pg?view

CPE Weekly: 2020-03-28

Saturday , 28, March 2020 Rich Bowen Uncategorized Leave a comment

Background:

The Community Platform Engineering group is the Red Hat team combining
IT and release engineering from Fedora and CentOS. Our goal is to keep
core servers and services running and maintained, build releases, and
other strategic tasks that need more dedicated time than volunteers
can give.

For better communication, we will be giving weekly reports to the
CentOS and Fedora communities about the general tasks and work being
done. Also for better communication between our groups we have
created #redhat-cpe on Freenode IRC! Please feel free to catch us
there, a mail has landed on both the CentOS and Fedora devel lists
with context here.

CentOS

* ppc64le and aarch64, 8 and 8-stream nodes now available in cico for
tenants to checkout. -- Email sent to ci-user list

* New signing for SIGs (through https://cbs.centos.org) live this week!

CentOS Stream

* Qt5.12 pushed in response to an internal request
* NetworkManager re-imported

Fedora Updates

* Freeze is over!

* New version of pagure (5.9.0) has been released and deployed in
staging and on pagure.io

* Document being worked on about how to onboard new CI systems in
Fedora - this is a work in progress!
https://hackmd.io/5gk_kHFhSR6sKa1huPA-4Q

Request for Review

* Fedora magazine: Article proposals about Silverblue rebase to F32
https://pagure.io/fedora-magazine-proposals/issue/59

* Packit integration in the-new-hotness
https://github.com/packit-service/packit/issues/689

* KeepassXC flatpak issue https://pagure.io/flatpak-module-tools/issue/6

Data Centre Move

* Communishift will be unavailable from 2020-04-13 until 2020-05-08

* Check out our detailed move shedule here
https://hackmd.io/R3EkjzVyTG2TYwQvkfzYrA?sync=&type=

* Covid-19 has seen restrictions added to the data centres we are
moving from and to, however we are unaffected as of yet for the move.

* If or when our timelines become affected, we will inform you
immediately of any outages, downtimes, etc a delay could cause.

* Thank you for your understanding at this particularly uncertain and
worrying times.

AAA Replacement

* We are nearly complete in our phase one development!

* Below are some of the features the team have developed since
beginning the project in mid-Jan

* UI where people register and login

* Add groups function

* Change personal details

* Enroll OPT

* Reset password

* View group owner details

* Use a search engine

* Our next steps in this project is to demo to the team for feedback
and then continue to develop CentOS authentication, more user focused
features, request for more feedback and test, test test!

CI/CD

* Monitor-gating is now running in production and has already caught a
couple of issues with bodhi (both in stg and in prod)!

* Rpmautospec

* This is in review as a Fedora package:
https://bugzilla.redhat.com/show_bug.cgi?id=1816124

* Work progressing on Koji tagging plugin (post-build), full use
case support for bumping releases

* The team hope to deploy this in staging soon!

Sustaining Team

* Mbbox

* Some progress on CRD for koji-builder and koji-hub components
has been made this week

* Bodhi 5.2.2 released

* Some issues with celery tasks & rawhide monitoring has been
super useful with this.

* Compose Tracker enhancement

* Tagging issues have been resolved

* Ability to ping maintainers

* Fedora Minimal Compose

* Odcs-backend-releng01 has been provisioned to enable testing

GitForge Decision

* After evaluating over 300 user stories from multiple stakeholders we
have aligned on a decision for the Gitforge that CPE will operate for
the coming years. We are opting for Gitlab for our dist git and
project hosting and will continue to run pagure.io with community
assistance.

* Check out our GitForge decision on the Fedora Community blog
https://communityblog.fedoraproject.org/

* And at the CentOS blog page
https://blog.centos.org/2020/03/git-forge-decision/

* Keep an eye out for mails in the coming months to the devel lists as
we plan transitions and next steps with GitLab

* We would like to express our sincere thank you to all who
contributed requirements to us!

As always, feedback is welcome, and we will continue to look at ways
to improve the delivery and readability of this weekly report.

Have a great weekend!

Aoife

Source: https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ

Git Forge decision

Friday , 27, March 2020 Rich Bowen Uncategorized 1 Comment

After evaluating over 300 user stories from multiple stakeholders we have aligned on a decision for the Gitforge that CPE will operate for the coming years. We are opting for Gitlab for our dist git and project hosting and will continue to run pagure.io with community assistance.

Analysis and recommendation:

A lot of comments and concerns were raised about the suitability of Github as a forge of choice. The preference from all stakeholders (Fedora, CentOS, RHEL, CPE)  is that Github is not a contender and not a preference, with that in mind, we have decided to not analyse it as an option and respect the wider wishes of our stakeholders. Therefore the rest of this analysis focuses on Pagure versus Gitlab as our choice.

Looking at the user story list, we have a picture of a standard set of practices that users expect to have from a Gitforge. The basics of storing code, accessing it, merging, forking and the traditional git workflow are satisfied by both Forges under investigation. 

A key requirement coming to us is security. The need for HTTP/S pushes, the need for more stringent branch control via protected and private branches is a key operating requirement of the CentOS stakeholders. The need to interface with internal and external users in a private capacity whereby embargoed content can be worked on in private is a necessary requirement. 

Another key requirement is usability and accessibility. It is clear that our current forge solution is used as a mixture of ticket tracker, work planning, code repository and storage of documents and other artifacts. The barrier to usage needs to be low to attract drive by users and a strong representation was made for the need to have more accessible ways to interface with the system from a GUI to a command line client.

Developer centric needs came from multiple sources. Integrations with daily workflow, integrations within the IDE, integrations in an always ready and always on approach (SLA requirements were high) as well as the ability to use the forge as a means to improve the codebase (auto notifications of issues, interactive PR reviews etc.) and way of working by providing analytical output was also raised.

A big factor in a decision here needs to be both the immediate usability to meet stakeholder needs that includes an immovable deliverable for CentOS Stream which CPE must deliver by the end of the year. 

Another major factor is the stability, availability and responsiveness of the platform chosen. While no Forge meets the full suite of requirements, the issue of stability, availability and some of the richer features that were requested are currently not available in Pagure. Gitlab provides the most feature rich experience out of the box and the recommendation of the CPE Management is to opt for Gitlab as our chosen Forge for dist-git and general project hosting. For pagure.io we want to offer it to the community to maintain. CPE would provide power and ping and the rest of it will be up to the community willing to do the work. If no-one steps up to pick the maintenance of pagure.io, it will be a candidate application to sunset. Some top level requirements which helped us arrive at this decision:

  • There is a need for CentOS Stream to integrate with a kernel workflow that is an automated bot driven merging solution (merge trains). This allows for richer CI capabilities and minimizes the need for human interaction
  • Gitlab provides subgroups allowing for more granular permissions for repos
  • Gitlab allows for project planning capability which could make multiple trackers such as Taiga redundant, allowing for the planning and tracking to reside within the repo. It would enrich the current ticket based solution that Pagure has evolved into for some groups
  • 24/7 availability in an SLA model and not hosted by the CPE team freeing up resourcing and removing the need to staff a dedicated team for a Gitforge SLA which would necessitate a follow- the- sun Ops model and a heavy investment in stability and observability of the Pagure solution.

The opportunity cost to invest our finite resources into bringing Pagure up to the minimum standard that we require by the end of the year would mean feature starving both Fedora and CentOS for the next 18-24 months as we strive for the optimal standard. As a team, we spend 40% of our available resources on keeping the lights on day to day with a very small amount of that improving our technical debt situation. We are spending 30% of our team on delivering CentOS Stream. The available bandwidth for the team is not at a point that we could safely and with confidence deliver the required features to make Pagure work as our Forge of choice. It additionally would have a longer term impact with our lights on work needing to expand to move Pagure to an SLA, tilting our resourcing plan for that body of work towards 60% of our capacity. We feel this is not a responsible decision that we can make as the inward investment in a Forge is not something that we can do at the expense of planned initiatives that are on our backlog. Some of them include a better packager workflow, more investment in CI/CD to remove CPE from manual work and empower the community to do more things in our infrastructure, more observability and monitoring of our infra and services, movement of services towards the Cloud to make use of a modern tech stack and that's before we consider immovable service progression that we simply have to undertake, for example, the new Auth / AAA system.

However, we do not want to abandon Pagure and our plan going forward is thus.

  1. Offer the maintenance of pagure.io to anyone in the community interested in leading it.
  2. Engage with Gitlab on the possibility of a SaaS offering so that CPE can attain key requirements of uptime, availability and throughput as well as ensuring tooling integrations (such as Fedora Messaging among others) are preserved. Legal considerations with respect to control of code will be our first discussion point with them enabling us to make a SaaS Vs self hosted decision.
  3. Keep Pagure running with our oversight while we analyse a sunset timeline which will give a minimum of 12 months notice once we have a plan firmed up. We will fix blocker bugs, address critical vulnerabilities and keep the lights on in the same manner that we have committed to over the last 14 months where Pagure has not been a staffed and supported initiative.
  4. Where possible, when we have to update our tooling, we will attempt to refactor our tooling to be forge agnostic, allowing our Communities the choice of storing their code on Gitlab or continuing to use pagure.io
  5. Watch closely for collaboration with other Communities on Pagure and provide them with guidance and oversight to help the Pagure Community grow. We recognise that this is a growing and unique ecosystem and we genuinely want to see it succeed and will do our best to support it in that capacity. To that end we will publish the roadmap difference between Pagure and Gitlab to allow the Community to focus on feature enhancements to bridge that gap.
  6. Facilitate our Communities and assist them in standing up a version of Pagure that can be driven and maintained by the Community allowing a pure Open Source principles approach for those who seek it.

We recognize how difficult a decision this is and we empathize with the emotional attachment to Pagure. It is why we want to have a mutually beneficial approach to ultimately allow Pagure to grow and flourish and allow our community members to setup and work with any Forge they wish. This ultimately allows the CPE team to focus on adding value to a greater scale of initiatives  . This approach allows us to focus on value added services and initiatives that will benefit a large percentage of our communities instead of focusing on a singular foundational service which would ultimately consume our finite resourcing and limit our impact on both Communities.

-- Jim and Leigh