Category Archives: revision control

Getting a File from gitlab using a TOKEN

Posted on by
Reply

I work on a system in our lab that I connect to via SSH. Fetching a file via the web and pushing it to this machine is quite slow, and I want to pull on this machine directly. The trick to make this work is to use a TOKEN from gitlab and oauth2 as the username portion of the URL. For example

curl -O   https://oauth2:$CI_JOB_TOKEN@gitlab.com/api/v4/projects/$PROJECT_ID/packages/generic/$PROJECT_NAME/$PROJECT_VERSION/$FILE

The CI_JOB_TOKEN must be set to a token that has appropriate permissions on your project. Most of the values I wrote as envvars could be copy-pasted from the gitlab packages link on the web.

Adding stable to a blobless clone

Posted on by
Reply

We regularly sync various git repos from upstream to our company speicfic Linux repository. However, sometimes when working on a development machine, I need a branch from stable that we have not synced yet. If I did a blobless clone from our repository, doing a fetch from stable is going to pull over lots of blobs that I do not need.

Instead, I want to fetch just a specific tag. I can do this by adding the tag to the fetch command, and it will pull over only those blobs

Continue reading →

More Personal Ansible

Posted on by
Reply

I can do anything. I can’t do everything. –Me

Anything worth doing is worth doing in a way you can check in to git. To recall what I did from the command line, I should turn those actions into a persist-able document. Do I? Not often enough. Often I rely on bash history to remind me of what I did last time. Since the machines I work on are out of a global pool, I have been burned by not recording commands before relinquishing a machine.

For complex series of tasks, it makes sense to execute a bash script to perform those tasks, and I have many of these. Shell scripting excels in doing command line tasks. Where it does not do so well is on tasks that are split over multiple machines. While curl is great for pulling and pushing files to webservers, the majority of my remote work requires ssh and scp to set things up. This is where Ansible comes in: If I can make a playbook that records the commands I use to perform that action, I can repeat it on another machine.

Here is what my workflow looks like as I try to get better at it:

Continue reading →

How not to waste time developing long-running processes

Posted on by
Reply

Developing long running tasks might be my least favorite coding activity. I love writing and debugging code…I’d be crazy to be in this profession if I did not. But when a task takes long enough, your attention wanders and you get out of the zone.

Building the Linux Kernel takes time. Even checking the Linux Kernel out of git takes a non-trivial amount of time. The Ansible work I did back in the OpenStack days to build and tear down environments took a good bit of time as well. How do I keep from getting out of the zone while coding on these? It is hard, but here are some techniques.

Continue reading →

Revision control and sheet music

Posted on by
Reply

Musescore is a wonderful tool. It has made a huge impact on my musical development over the past couple decades. Sheet music is the primary way I communicate and record musical ideas, and Musescore the tool and musecore.com have combined to make a process that works for me and my musical needs.

I have also spent a few years writing software, and the methods that we have learned to use in software development have evolved due to needs of scale and flexibility. I would like to apply some of those lessons to how I manage sheet music. But there are disconnects.

The biggest issue I have is that I want the same core song in multiple different but related formats. The second biggest issue is that I want to be able to make changes to a song, and to collaborate with other composers in making those changes.

Continue reading →

When to Ansible? When to Shell?

Posted on by
Reply

Any new technology requires a mental effort to understand. When trying to automate the boring stuff, one decision I have to make is whether to use straight shell scripting or whether to perform that operation using Ansible. What I want to do is look at a simple Ansible playbook I have written, and then compare what the comparable shell script would look like to determine if it would help my team to use Ansible or not in this situation.

Continue reading →

Remotely checking out from git using ssh key forwarding.

Posted on by
Reply

Much of my work is done on machines that are only on load to me, not permanently assigned. Thus, I need to be able to provision them quickly and with a minimum of fuss. One action I routinely need to do is to check code out of a git server, such as gitlab.com. We use ssh keys to authenticate to gitlab. I need a way to do this securely when working on a remote machine. Here’s what I have found

Continue reading →

Remote git with ssh

Posted on by
Reply

Working on a different architecture from my Laptop means that I am invariably working on a remote machine. My current development can be done on an Ampere AltraMax machine, which means I have 80 processors available; quite a nice benefit when doing a Linux Kernel compile that can use all of the processors available.

Because the machine is a shared resource out of out lab, I want to make sure I can recreate my work there on another machine; this one could be reclaimed or powered down due to lab priorities. Thus, all my remote work is done in git, and I use the ssh protocol to pull changes from my work server to my laptop fairly regularly…whenever I feel I have valuable work that could be potentially lost.

Continue reading →