Converting a Shell Script to Python

We have a build system that has grown organically. It started as a shell script. We needed to run it from gitlab, so we wrote helper scripts to insulate our code from gitlab. Then we added some helper functions to mimic the gitlab interactions when working with them from the comand line. The helper functions grew until you could not practically run the original shell script without them.

It is a mess.

I want to refactor it.

Refactoring Shell is painful.

I want objects. I want python.

So I am rewriting the gitlab and functions layer in python with an eye to rewriting the whole thing. Here’s what I have learned;

Continue reading

Fetching all packages from a Koji build

#bin/bash

KOJI_URL=https://kojipkgs.fedoraproject.org/packages/kernel/6.12.0/0.rc3.20241015giteca631b8fe80.32.eln143/aarch64/ 

curl $KOJI_URL > aarch64.dir.html
for  PACKAGE in  `xmllint --html  --xpath "//html/body/pre/a/@href" aarch64.dir.html | awk -F\" 'NR % 2 == 0 { print $2 }' `;  
do 
        URL=$KOJI_URL/$PACKAGE ;
        echo $URL; wget $URL;
done

Modify the URL to fetch a different architecture, version, or package.

The version of xmllint I am running does not support fn:string. If it did, the “for PACKAGE” line can be redone as:

for  PACKAGE in  `xmllint --html  --xpath "string(//html/body/pre/a/@href)" aarch64.dir.html` ;

Working with the Booked scheduler API

One benefit of working in a hardware company is that you actually have hardware. I have worked in software for a long time, and I have learned to appreciate when new servers are not such a scarce resource as to impact productivity. However, hardware in our group needs to be shared amongst a large group of developers, and constantly reserved, assigned, and reprovisioned. We use an install of the booked scheduler to reserve servers. As with many tools, I am most interested in using it in a scripted fashion. Booked comes with an API. Here’s some of the things I can do with it.

Continue reading

Building a Custom Fedora Based Kernel with Local Patches

How can I create a binary kernel RPM that has patches that have not yet merged into the mainline kernel? One approach to building the Kernel RPM is to use the Makefile option provided with the Kernel. While we typically do this, it does not provide us with the user land tools like perf and its libraries used to test certain patches.

An alternative approach is to take the Fedora Kernel Source RPM that matches the targeted upstream Kernel version, and modify it to apply the set of patches. Here is a walk-through of the process I just got to succeed.

Continue reading

How not to waste time developing long-running processes

Developing long running tasks might be my least favorite coding activity. I love writing and debugging code…I’d be crazy to be in this profession if I did not. But when a task takes long enough, your attention wanders and you get out of the zone.

Building the Linux Kernel takes time. Even checking the Linux Kernel out of git takes a non-trivial amount of time. The Ansible work I did back in the OpenStack days to build and tear down environments took a good bit of time as well. How do I keep from getting out of the zone while coding on these? It is hard, but here are some techniques.

Continue reading