I have wasted a lot of time as a developer waiting for long running processes to complete. Whether it is a Linux Kernel compile, and Ansible Playbook tearing down and recreating a system on a remote server, or a gitlab pipeline building and testing code, the common problem is that my head is in the problem being addressed there, but I cannot do anything to verify hypotheses until the process completes. I often get distracted while waiting, and find that what could have been a 5 minute turn around has become a 2 hour turn around.

One of the common aspects of these long running tasks is that they are built out of smaller elements, but need to perform all of the elements in order to validate some prerequisite. Perhaps that prerequisite is starting from a know state, either of the machine of the build.

The problem with automation is that it forces you to go through all the steps. While that is sometimes a necessary and right approach, it is not for all development.

I am going to make a distinction here between tools and automation. The distinction is that a tool is designed to be used by the software craftsman to perform and operation, maybe several, in the course of interactive work. The tools may be as simple as ‘ls’ which lists the contents of a directory, or as complex as tcpdump/wireshare/netstat type tools that show network state in flight.

Automation is explicitly not for this use case: automation is to replace time consuming and error prone tasks with repeatable, deterministic processing. Automation maybe built out of tools, but it is a different use case, and you should not be putting automation in the middle of your code-debug cycle if at all possible.

This leads to some guidelines.

• First, If automation is to replace manual processes, those processes need to be automatable. Building the functionality you need into a gui Window breaks this.
• Make everything executable from the command line.Using curl or python requests to execute a remote HTTP call is acceptable for this. Although you probable want to make the server accept and return JSON or YAML instead of HTML.
• That implies that you build your tools first. Make it possible for a developer to perform each individual step in the process without waiting a long time for the outcome. For compilation, yeah, the first one might take a long time, but support incremental compilation and make it simple to make simple changes.
• Do not lead with automation. If you are kicking things off with an Ansible playbook, you probably have leaned to far in the direction of automation….make that process human executable first, and then automate with Ansible.
• Do not have a dependency on a long running task, or your short task becomes a long running task. You see this in Ansible where a given task needs something that is performed once in the pipeline, much earlier, and an implicit dependency slips in between the stages, and now you need to wait for that unrelated task to complete. Make it easy to fetch just the information you need, or to perform only the essential step, before performing tricky, hard to code sections that need a lot of attention.

There are two benefits to this approach. First, you make your developers much more productive. Second, you prevent burnout and mental exhaustion. The productivity comes from the ability to keep on task, and not have to rebuild the mental model when returning to a task. The mental exhaustion comes from the need to store and retrieve, and rebuilt, that mental model. Over time that contributes to burnout.