Category Archives: Software

Merging Config Fragments in the Linux Kernel

Posted on by
Reply

If you have a config fragment that you want included in your Linux config file, you can use the make system to add it to the existing config. For example, I have a file called: kernel/configs/ampere-topic-mctp.config that looks like this:

CONFIG_MCTP_TRANSPORT_PCC=m

This tells the make system to change CONFIG_MCTP_TRANSPORT_PCC from the default (=n) to module (=m) when building. To include this config fragment, I append it to the make file command to build the config file. For example:

make olddefconfig  ampere-topic-mctp.config

This can be done with multiple fragments.

How not to waste time developing long-running processes

Posted on by
Reply

Developing long running tasks might be my least favorite coding activity. I love writing and debugging code…I’d be crazy to be in this profession if I did not. But when a task takes long enough, your attention wanders and you get out of the zone.

Building the Linux Kernel takes time. Even checking the Linux Kernel out of git takes a non-trivial amount of time. The Ansible work I did back in the OpenStack days to build and tear down environments took a good bit of time as well. How do I keep from getting out of the zone while coding on these? It is hard, but here are some techniques.

Continue reading →

Keeping the CI logic in bash

Posted on by
2

As much as I try to be a “real” programmer, the reality is that we need automation, and setting up automation is a grind. A necessary grind.

One thing that I found frustrating was that, in order to test our automation, I needed to kick off a pipeline in our git server (gitlab, but the logic holds for others) even though the majority of the heavy lifting was done in a single bash script.

In order to get to the point where we could run that script in a gitlab runner, we needed to install a bunch of packages (Dwarves, Make, and so forth) as well as do some SSH Key provisioning in order to copy the artifacts off to a server. The gitlab-ci.yml file ended up being a couple doze lines long, and all those lines were bash commands.

So I pulled the lines out of gitlab-ci.yml and put them into the somewhat intuitively named file workflow.sh. Now my gitlab-ci.yml file is basically a one liner that calls workflow.sh.

But I also made it so workflow.sh can be called from the bash command line of a new machine. This is the key part. By doing this, I am creating automation that the rest of my team can use without relying on gitlab. Since the automation will be run from gitlab, no one can check in a change that breaks the CI, but they can make changes that will make life easier for them on the remote systems.

The next step is to start breaking apart the workflow into separate pipelines, due to CI requirements. To do this, I do three things:

  • Move the majority of the logic into functions, and source a functions.sh file. This lets me share across top-level bash scripts
  • Make one top-level function for each pipeline.
  • replace workflow.sh with a script per pipeline. These are named pipeline_<stage>. These scripts merely change to the source directory, and then call top level functions in functions.sh.

The reason for the last split is to keep logic from creeping into the pipeline functions. They are merely interfaces to the single set of logic in functions.sh.

The goal of having the separate functions source-able is to be able to run interior steps of the overall processing without having to run the end-to-end work. This is to save the sitting-around time for waiting for a long running process to complete….more on that in a future article.

Running Keystone in development mode on Ubuntu 22.04

Posted on by
Reply

Things have diverged a bit from the docs. Just want to document here what I got working:

I had already checked out Keystone and run the unit tests.

I needed uwsgi

sudo apt install uwsgi-core
sudo apt install uwsgi-plugin-python3


Then a modified command line to run the server:

uwsgi --http-socket 127.0.0.1:5000    --plugin /usr/lib/uwsgi/plugins/python3_plugin.so   --wsgi-file $(which keystone-wsgi-public)

This got me the last part

https://stackoverflow.com/questions/31330905/uwsgi-options-wsgi-file-and-module-not-recognized

Revision control and sheet music

Posted on by
Reply

Musescore is a wonderful tool. It has made a huge impact on my musical development over the past couple decades. Sheet music is the primary way I communicate and record musical ideas, and Musescore the tool and musecore.com have combined to make a process that works for me and my musical needs.

I have also spent a few years writing software, and the methods that we have learned to use in software development have evolved due to needs of scale and flexibility. I would like to apply some of those lessons to how I manage sheet music. But there are disconnects.

The biggest issue I have is that I want the same core song in multiple different but related formats. The second biggest issue is that I want to be able to make changes to a song, and to collaborate with other composers in making those changes.

Continue reading →

When to Ansible? When to Shell?

Posted on by
Reply

Any new technology requires a mental effort to understand. When trying to automate the boring stuff, one decision I have to make is whether to use straight shell scripting or whether to perform that operation using Ansible. What I want to do is look at a simple Ansible playbook I have written, and then compare what the comparable shell script would look like to determine if it would help my team to use Ansible or not in this situation.

Continue reading →

Cleaning a machine

Posted on by
Reply

After you get something working, you find you might have missed a step in documenting how you got that working. You might have installed a package that you didn’t remember. Or maybe you set up a network connection. In my case, I find I have often brute-forced the SSH setup for later provisioning. Since this is done once, and then forgotten, often in the push to “just get work done” I have had to go back and redo this (again usually manually) when I get to a new machine.

To avoid this, I am documenting what I can do to get a new machine up and running in a state where SSH connections (and forwarding) can be reliably run. This process should be automatable, but at a minimum, it should be understood.

Continue reading →

Remotely checking out from git using ssh key forwarding.

Posted on by
Reply

Much of my work is done on machines that are only on load to me, not permanently assigned. Thus, I need to be able to provision them quickly and with a minimum of fuss. One action I routinely need to do is to check code out of a git server, such as gitlab.com. We use ssh keys to authenticate to gitlab. I need a way to do this securely when working on a remote machine. Here’s what I have found

Continue reading →