Developing long running tasks might be my least favorite coding activity. I love writing and debugging code…I’d be crazy to be in this profession if I did not. But when a task takes long enough, your attention wanders and you get out of the zone.

Building the Linux Kernel takes time. Even checking the Linux Kernel out of git takes a non-trivial amount of time. The Ansible work I did back in the OpenStack days to build and tear down environments took a good bit of time as well. How do I keep from getting out of the zone while coding on these? It is hard, but here are some techniques.

Put everything in the container ahead of time

Build processes necessarily need a lot of tools. On a Fedora System, I often start by doing

yum groupinstall "Development Tools"

And then still may need to install a few more things: Java, Rust, or Python libraries, meson, Dwarves, bison, and others.

If you are doing CI inside a container, as is the norm for the git hosting services, you can specify a custom container. Anything that is not part of your build should be in the container.

Note that by doing this, you are putting another variable in your build process: which version of the container was used. By installing packages at build time, you are using the latest. By using a pre-built container, some of those packages may have gone stale. This is kind of the reason FOR using containers, as you can change package versions at your own cadence, but make sure you are deliberate about it.

Get it working, then disable it

If part of a long running task works, hold on to the output of that task, and disable the task itself. This implies that any information generated from this stage of the task can be deduced from the artifacts.

A Linux Kernel build takes a long time, even with all 100+ processors working on it. Skip the build until you need it to be fresh again.

use mkdir to simulate git checkouts

If you are doing a git checkout, but don’;t actually need that checkout for the follow on stage, you can short circuit that checkout by doing a mkdir of the root directory. This is a more useful hack than I originally realized. Pulling down a huge tree like the Linux kernel can take quite some time. Skipping it until you are ready to test it can save that time.

Use touch or fallocate to simulate build artifacts

If your automation is primarily for moving artifacts around, you can temporarily skip the stage that builds the artifacts until you get the follow on stages working. This is often done in conjunction with the git checkout skip above.

There are many variations to these work-arounds, as well as several more I have not yet documented. The point is to identify where you are waiting on tasks, and to figure out ways to avoid that time while developing the rest of your automation.

If you are working with Ansible, using the flag to start on a specific task can be a powerful time saver. However, that means structuring your playbook such that you don’t gather up all the information you need for a task at the beginning.

The same general idea is true for bash based operations: you can use functions that you call directly, but make sure you don;’t need too much context for that function call. Make it easy to start in the middle, and end in the middle.