I'm at SC18 - the premiere international supercomputing event - in Dallas, Texas. Every year at this event, hundreds of companies and universities gather to show what they've been doing in the past year in supercomputing and HPC.
As usual, the highlight of this event for me is the student cluster competition. Teams from around the world gather to compete on which team can make the fastest, most efficient supercomputer within certain constraints. In particular, the machine must be built from commercially available components and not consume more than a certain amount of electrical power while doing so.
This year's teams come from Europe, North America, Asia, and Australia, and come from a pool of applicants of hundreds of universities who have been narrowed down to this list.
Of the 15 teams participating, 11 of them are running their clusters on CentOS. There are 2 running Ubuntu, one Running Debian, and one running fedora. This is, of course, typical at these competitions, with Centos leading as the preferred supercomputing operating system.
The teams are given a variety of projects to work on before they get here, and then there is one surprise project that is presented to them when they arrive. They have 48 hours to work on these projects, and the winner is selected based on benchmarks and power consumption.
You can read more about the competition, and about the teams participating, on the SCC website.
We would like to announce that OKD v3.11 rpms been officially released and are available at http://mirror.centos.org/centos/7/paas/x86_64/openshift-origin311/. 
In order to use the released repo  we have created and published the rpm (contains the yum configuration file)  which is in the main CentOS extra repository. The rpm itself has a dependency on the centos-release-ansible26  which is the ansbile 2.6 version rpm built by CentOS Infra team.
Should you decide not to use the centos-release-openshift-origin3* rpm then will be your responsibility to get ansible 2.6 required to by openshift-ansible installer.
Please note that due to ongoing work on releasing CentOS 7.6, the mirror.centos.org repo is in freeze mode - see https://lists.centos.org/pipermail/centos-devel/2018-November/017033.html  and as such we have not published the rpms to http://mirror.centos.org/centos/7/paas/x86_64/openshift-origin/ . Once the freeze mode will end, we'll publish the rpms.
Kudos goes to CentOS Infra team for being very kind in giving us a waiver to make the current release possible.
PaaS SIG team
We are pleased to announce the (tentative) schedule of talks for the
upcoming CentOS Dojo in Brussels, which will be held on the day before
FOSDEM - February 1, 2019 - at the Grand Place Marriott.
Details, and the schedule, are now available at
https://wiki.centos.org/Events/Dojo/Brussels2019 (Schedule subject to
Registration is free, but we need to know how many people are coming,
for catering and space purposes. You can register today at:
See you in Brussels!
The videos from the recent #CentOSDojo at #CERN are now available on the CentOS YouTube channel. If you have time for only one, be sure to watch the first video, which talks about the challenges that CERN has with the enormous amount of data they produce every day in the LHC.
Also recommended, Fabian's discussion of the coming (and already in place!) changes to the CentOS Git infrastructure.
[UPDATE: The videos which were previously updated were truncated, and we're looking into fixing that. meanwhile you can view the video on the event schedule by clicking the paperclip icon next to each talk title.]
Greetings from the mirror-management department! This notice is for those who employ some sort of an automation to download AltArch (ie. aarch64, armhfp, i386, power9, ppc64, ppc64le) CentOS 7 .iso/.raw.xz images from mirror.centos.org. Those using a regular browser to download these images are not particularly affected, and you can continue to the next post on this blog.
Previously, only main architecture .iso image downloads from mirror.centos.org were redirected to isoredirect.centos.org, which then displayed the user a list of nearby external mirrors. We will shortly extend this configuration to cover AltArch image downloads as well, ie. direct AltArch image downloads from mirror.centos.org will no longer be possible. mirror.centos.org will still serve .rpm downloads for all architectures as before.
There are three reasons for the change. First, to save bandwidth from mirror.centos.org nodes directly managed by the CentOS Project. Most of these mirror.centos.org hosts are also used for seeding the 600+ external mirrors we have. By directing some of that .iso download traffic to external mirrors we can offer faster sync speeds for those external mirrors, and for people downloading individual rpms from mirror.centos.org. Second, most of those external mirrors offer faster download speeds to end users than what could be achieved by downloading from mirror.centos.org, so the users will benefit from this change as well. Finally, because there are much more external mirrors than mirror.centos.org nodes, it is likely that your bits will need to travel a shorter path, conserving bandwidth globally.
The above change will be implemented some time between the releases of RHEL 7.6 and CentOS 7.6.18xx, so that external mirrors syncing CentOS 7.6.18xx content would not need to fight for bandwidth between AltArch .iso downloaders.
The other change, which has already been implemented, is related to how isoredirect.centos.org behaves when accessed with curl or wget. If you now do a
wget http://isoredirect.centos.org/altarch/7/isos/i386/CentOS-7-i386-Everything-1804.iso, isoredirect will notice that you are trying to download the file and will redirect the request to the nearest external mirror. If you access the same URL with a regular browser, you will see a list of nearby mirrors from which you can pick your favourite mirror. wget will follow redirects by default, but curl needs a
--location switch to follow redirects. If a filename is not specified, you will get a list of mirrors regardless of the browser used.
So, combining the effects of the above two changes: If you currently use some sort of a script that downloads AltArch .iso images from mirror.centos.org, those requests will soon be served by external mirrors instead of mirror.centos.org. In the case of wget you will only see one additional request and you probably don't need to change anything. If you use curl, you must add the
--location switch to curl to follow the redirect issued by isoredirect.centos.org. If you want to eliminate one redirect, you can change mirror.centos.org to isoredirect.centos.org in your script. The rest of the URL is the same, ie. /altarch/<release>/isos/<arch>/<filename.iso or .raw.xz>
As an aside, even though mirror.centos.org nodes are managed by the CentOS Project, those servers and their hosting are donations from various organizations. If you think your organization could donate an additional server to share the load and to give us better geographical coverage, please see https://wiki.centos.org/Donate
If you have questions or concerns regarding this change, please let me know. Thanks!
It's been over a year since we published anything about the CentOS Community Container Pipeline. Many interesting things have happened during the past year, many things have changed and there's a complete shift in the architecture of the service that's was rolled out over the last weekend.
If this is the first time you're hearing about CentOS Community Container Pipeline project, it would be best to refer this blog post, or the GitHub repo of the project, or the wiki page. But to put it in short, the service does below things:
RUNlabel in its Dockerfile
When we talked about the project at DevConf.cz '18, we received a positive response from the audience. However, at that time, we knew that our service couldn't handle more build requests and on-boarding more community projects would be counter-productive when our backend didn't have the ability to serve those requests.
Old implementation of the service had a lot of plumbing. There are workers written for most of the features mentioned above.
atomic scanto scan the containers. This in turn spun up a few containers which we needed to delete along with their volumes to make sure that host system disk doesn’t get filled up.
Everything was spread across different hosts and we were using really huge Ansible playbooks to bring up the service. A fresh deployment took 30 minutes on an average. Testing any change in dev environment would require us to do a redeployment of the service which took another 15 minutes on an average. Deploying and maintaining this service was quite a pain!
Since long time we were discussing about developing our service on top of OpenShift. Then, at some point, we read about OpenShift Pipeline and found it interesting. We took the plunge and came up with a proof of concept implementation of CentOS Community Container Pipeline on top of OpenShift OKD using Minishift. Results were exciting! We were able to do parallel builds of container image, Jenkins Pipelines orchestrated the flow really well, build times were faster, we didn't need to use beanstalkd at all and, most importantly, there was very less code written to get things done!
With the POC in place, we went ahead with developing more tangible service on top of a real OpenShift cluster instead of developing on top of Minishift. What used to be individual workers doing their thing in old system is now pretty much all inside OpenShift Pipeline.
We now have an OpenShift Pipeline for every project on CentOS Container Index that does Pre-build, Dockerfile lint, container image build, scan the container image and push it to external registry; all from a single container! We have another OpenShift Pipeline for every project to do their weekly scans. So instead of having five workers to do these tasks and communicate with each other via beanstalkd, we have orchestrated things through OpenShift Pipelines.
We don't have Repo tracking implemented in the new architecture yet. We don't have a UI for the users to take a look at their build logs or weekly scan logs either. We're initially focusing on getting the UI for logs up and then we will start working on Repo tracking. We are also working on setting up a CI job that tests core parts of the service on Minishift so that anyone willing to take the service for a spin should literally be able to do it on a Minishift VM!
This project is solely focused on making things easier for open-source projects and its developers. If you are working on an open-source project that's building on top of CentOS, we would like to know your thoughts. If you need help getting started, you can contact us on IRC (#centos-devel on Freenode) or take a look at project documentation.
Dharmit Shah (dharmit on #centos-devel IRC)
We are pleased to announce new official Vagrant images of CentOS Linux 6.9 and CentOS Linux 7.5.1804 for x86_64 (based on the sources of RHEL 7.5). All included packages have been updated to September 30th, 2018.
config.vm.synced_folder ".", "/vagrant", type: "virtualbox"
config.vm.synced_folder ".", "/vagrant", disabled: true
to their Vagrantfile, to prevent errors on "vagrant up".
vb.customize ["modifyvm", :id, "--natdnshostresolver1", "off"]
Our automatic testing is running on a CentOS Linux 7 host, using Vagrant 1.9.4 with vagrant-libvirt and VirtualBox 5.1.20 (without the Guest Additions) as providers. We strongly recommend using the libvirt provider when stability is required.
The official images can be downloaded from Vagrant Cloud. We provide images for HyperV, libvirt-kvm, VirtualBox and VMware.
If you never used our images before:
vagrant box add centos/6 # for CentOS Linux 6, or... vagrant box add centos/7 # for CentOS Linux 7
Existing users can upgrade their images:
vagrant box update --box centos/6 vagrant box update --box centos/7
The SHA256 checksums of the images are signed with the CentOS 7 Official Signing Key. First, download and verify the checksum file:
$ curl http://cloud.centos.org/centos/7/vagrant/x86_64/images/sha256sum.txt.asc -o sha256sum.txt.asc $ gpg --verify sha256sum.txt.asc
Once you are sure that the checksums are properly signed by the CentOS Project, you have to include them in your Vagrantfile (Vagrant unfortunately ignores the checksum provided from the command line). Here's the relevant snippet from my own Vagrantfile, using v1803.01 and VirtualBox:
Vagrant.configure(2) do |config| config.vm.box = "centos/7" config.vm.provider :virtualbox do |virtualbox, override| virtualbox.memory = 1024 override.vm.box_download_checksum_type = "sha256" override.vm.box_download_checksum = "b24c912b136d2aa9b7b94fc2689b2001c8d04280cf25983123e45b6a52693fb3" override.vm.box_url = "https://cloud.centos.org/centos/7/vagrant/x86_64/images/CentOS-7-x86_64-Vagrant-1803_01.VirtualBox.box" end end
If you encounter any unexpected issues with the Vagrant images, feel free to ask on the centos-devel mailing list, or in #centos on Freenode IRC.
I would like to warmly thank Brian Stinson, Fabian Arrotin and Thomas Oulevey for their work on the build infrastructure, as well as Patrick Lang from Microsoft for testing and feedback on the Hyper-V images. I would also like to thank the CentOS Project Lead, Karanbir Singh, without whose years of continuous support we wouldn't have had the Vagrant images in their present form.
I would also like to thank the following people (in alphabetical order):
After many years of excellent service by the Oregon State University Open Source Lab the CentOS Project has decided to migrate our web-based pastebin instance to a self-hosted platform running on our infrastructure. This has provided us the opportunity to move to a different solution based on the Stikked pastebin server which is a more modern solution with a number of features we felt would best benefit our user communities:
The web interface is available at https://paste.centos.org and from there you can paste content directly into the provided web form and optionally add your name or a paste title and even select the language of the paste if you wish the contents to be syntactically colored when displayed. You are able to select a number of time periods for the paste's lifetime from the dropdown selection and may opt to have the paste delete itself on view, so called "burn on view". The option also exists to encrypt your paste if you wish. After you submit the form you can share the resulting URL with others.
Additionally we've made a command line client, cpaste, available to enable pasting directly from your servers / desktops to our pastebin instance. This client is based on the Stikkit client by Petr Bena. This package is in our "extras" repository and can be installed with:
yum --enablerepo=extras install cpaste
Usage information can be retrieved with:
Examples illustrating how to how the command line client:
Paste a file directly to our server:
Paste a python code snippet with a title of "code snippet" and an author name of "John Q. Public"
cpaste -l python -t "code snippet" -a "John Q. Public" -i ~/src/project/code.py
Paste the standard output of a process and return only the paste's url:
~/bin/process | cpaste -s
One notable difference between the new and old instances is that the new instance supports paste lifetimes of up to one day only.
We hope you find the new service useful.
We would also like to thank OSUOSL for providing the old pastebin instance for the past many years.