Git Lab provides a mechanism to run workloads triggered by a git commit. We try to let the automation do as much work as possible before interrupting a human code reviewer. We want to know if the code is right in as objective a manner as possible before we take the expensive context switch to perform a code review.
Continue readingCOmpiling the Linux Kernel is a long running task. Oh, sure, you can put -j 32 and it speeds it up tremendously. But it still is a long running task. And by that, I mean it fits the following definition:
A Long Running task is one that takes so long that you start doing something else before it is done.
“aDAM YOUNG”
If you are familiar with the concept of Flow, then a long running task is one that breaks the flow.
It seems that these kind of “flow Blockers” are common in my line of work. I tend to work on distributed systems, and network based workflows are usually long running tasks. Things like:
This last one is the thing I am working on now.
The funny part is that I don’t have to do the hard work of making the Kernel compile. This is the responsibility of the team I am building this for, and I can assume that it works OK. This assumption will allow me to skip actually building the Kernel for the majority of my effort.
What do I need to do? I need to write a small yaml file that gitlab will use to kick off a CI/CD Pipeline. This file is a fairly thin wrapper around a bash script (similar pattern to RPM spec files, Ansible, and so forth) that will be executed on the remote system when triggered by a new or modified merge-request. This code needs to install a bunch of RPMs via dnf, kick off the kernel build, and then upload the artifacts to an object store. Right now, I am working on that last bit.
If I wait for the Linux Kernel to build, each iteration of code changes in my development cycle will take more than an hour. Thus, I want to put off actually building the Kernel until I know the rest of the tasks work. All I need to do is test that something gets uploaded, I don’t really care what that thing is. So I comment out the make and replace it with the simplest command I can think of to create a file:
#- make -j 16 rpm-pkg mkdir ./artifacts - touch ./artifacts/test.test |
There is actually a bunch of stuff I comment out including the part where I install packages from dnf. If I know that the code is working, there is little benefit to running it during the debug cycle for the upload stage.
However, I only want to comment out lines, not remove, Why? Because the code needs to be put back in to place in the final assembly, and I don’t want to have to debug the long running process.
With the actual build elided from my process, I can test the system by submiting a no-op merge request and see that gitlab triggers the runner to run my code. My round trip is now about a minute, short enough that I can keep myself on task.
Usually.
It is rare that you want to write something without later wanting to be able to read it back. One common way of organizing files that are generated regularly is by time stamp. If you want to add a timestamp to a file name, you can do so using the date command.
In order for the filenames to sort in the right order, you want the name to go from largest unit to smallest.
Here is an example that creates a filename-suitable string formed Year->second. I remove all unnecessary formatting characters.
date --rfc-3339=seconds | sed -E -e 's! |-|:!!g' |
The date command reads the current date/time on the local system. –rfc-3339=seconds produces output that looks like this:
$ date --rfc-3339=seconds 2021-11-03 10:57:14-04:00 |
In order to keep the regular expression concise inside the sed command, the -E switch tells it to use extended regular expressions, including the alternation character ‘|’ . Thus, the regex ‘ |-|:’ matches a space, a dash, and a colon.
In my quest to automate the testing of the Linux Kernel, I need to automate the build of the Linux Kernel. To build the Kernel, you need the requisite packages. What are they? Let’s find out.
Continue readingHere’s a one liner for showing the status of all your beaker jobs.
for JOB in $( bkr job-list -o $( bkr whoami | jq -r '.username' ) | jq -r ".[]" ) ; do bkr job-results $JOB | xpath -q -e "string(/job/recipeSet/recipe/@status)" ; done |
Ampere Computing chips run the ARM64 instruction set. My laptop is a Dell running x86_64. In order to edit locally, but build remotely, I make use of servers in our datacenter. These are the steps I am taking.
Continue readingssh allows you to run a command on a remote machine. You may want to use a shell variable in a remote command. You have to be aware of when that variable gets evaluated.
Continue readingOne of our ironic baremetal nodes was suffering a cleaing failure. Fixing it was easy…once we knew the cause/
Continue readingEnough of us are looking at the cluster that I want an easy to read snapshot of the node state. So, I convert the csv output of the openstack command into a simple HTML table.
Continue readingSome custom jq for RegEx selection of OpenStack Ironic baremetal nodes. Our Server types show up in their names. I want to be able to build lists of only the Mt. Jade Servers, which have names that look like this:
jade09-r097
openstack baremetal node list --sort provision_state:asc -c UUID -c Name -f json | jq '.[] | select(.Name | test("jade."))' |