Make long running tasks short for development

COmpiling the Linux Kernel is a long running task. Oh, sure, you can put -j 32 and it speeds it up tremendously. But it still is a long running task. And by that, I mean it fits the following definition:

A Long Running task is one that takes so long that you start doing something else before it is done.

“aDAM YOUNG”

If you are familiar with the concept of Flow, then a long running task is one that breaks the flow.

It seems that these kind of “flow Blockers” are common in my line of work. I tend to work on distributed systems, and network based workflows are usually long running tasks. Things like:

  • Ansible Playbooks that allocate a bunch a virtual machines before install software on them.
  • Software install process like ipa-server install
  • Automatically build, install, and test the Linux Kernel

This last one is the thing I am working on now.

The funny part is that I don’t have to do the hard work of making the Kernel compile. This is the responsibility of the team I am building this for, and I can assume that it works OK. This assumption will allow me to skip actually building the Kernel for the majority of my effort.

What do I need to do? I need to write a small yaml file that gitlab will use to kick off a CI/CD Pipeline. This file is a fairly thin wrapper around a bash script (similar pattern to RPM spec files, Ansible, and so forth) that will be executed on the remote system when triggered by a new or modified merge-request. This code needs to install a bunch of RPMs via dnf, kick off the kernel build, and then upload the artifacts to an object store. Right now, I am working on that last bit.

If I wait for the Linux Kernel to build, each iteration of code changes in my development cycle will take more than an hour. Thus, I want to put off actually building the Kernel until I know the rest of the tasks work. All I need to do is test that something gets uploaded, I don’t really care what that thing is. So I comment out the make and replace it with the simplest command I can think of to create a file:

 #- make -j 16 rpm-pkg mkdir ./artifacts
 - touch  ./artifacts/test.test

There is actually a bunch of stuff I comment out including the part where I install packages from dnf. If I know that the code is working, there is little benefit to running it during the debug cycle for the upload stage.
However, I only want to comment out lines, not remove, Why? Because the code needs to be put back in to place in the final assembly, and I don’t want to have to debug the long running process.

With the actual build elided from my process, I can test the system by submiting a no-op merge request and see that gitlab triggers the runner to run my code. My round trip is now about a minute, short enough that I can keep myself on task.

Usually.

Date format suitable for file names

It is rare that you want to write something without later wanting to be able to read it back. One common way of organizing files that are generated regularly is by time stamp. If you want to add a timestamp to a file name, you can do so using the date command.

In order for the filenames to sort in the right order, you want the name to go from largest unit to smallest.

Here is an example that creates a filename-suitable string formed Year->second. I remove all unnecessary formatting characters.

date --rfc-3339=seconds | sed -E -e 's! |-|:!!g'

The date command reads the current date/time on the local system. –rfc-3339=seconds produces output that looks like this:

$ date --rfc-3339=seconds 
2021-11-03 10:57:14-04:00

In order to keep the regular expression concise inside the sed command, the -E switch tells it to use extended regular expressions, including the alternation character ‘|’ . Thus, the regex ‘ |-|:’ matches a space, a dash, and a colon.

Debugging a Clean Failure in Ironic

My team is running a small OpenStack cluster with reposnsibility for providing bare metal nodes via Ironic. Currently, we have a handful of nodes that are not usable. They show up as “Cleaning failed.” I’m learning how to debug this process.

Continue reading

Ampere

Time for a change, and a big one at that. As of September 20th, I am now a full time Employee of Ampere Computing. I am in the Software Development team, working on Open Source stuff. That means Linux Kernel and Open Stack, among other things.

I’ll post more on why in the future. Why I left Red Hat, and why I specifically chose Ampere. Both deserve a well formed explanation, as both are very important to me. My head is not there yet, it is in code and machines and processes.

I’m getting Amped.