Refactoring in Ansible: extract Variable

“Let the complexity emerge.” Probably the best advice I ever got in coding. Do something in as straight-forward manner as possible. When you find your self repeating code, extract it. Here’s an example from an ansible playbook I’m working on.

I’ve gotten a couple tasks written so far.

---
- hosts: localhost
  tasks:

    - name: Creates directory
      file:
        path:  /home/ayoung/ocp-ansible/stage
        state: directory

    - name: regen install-config 
      copy:
        src:  /home/ayoung/ocp-ansible/files/install-config.yaml.orig
        dest: /home/ayoung/ocp-ansible/stage/install-config.yaml

    - name: validate install-config
      command: /home/ayoung/apps/ocp4.4/openshift-install create install-config  --dir /home/ayoung/ocp-ansible/stage
      environment:
        OS_CLOUD: fsi-moc

I’ve copied the directory name a few times. Lets start by introducing a variable section into the play book. I’ll use the first task. Ah…but first…git! This project is following the pattern I wrote about several years ago. Here is what I have in my first commit:

Generate the install-config
 
# Please enter the commit message for your changes. Lines starting
# with '#' will be ignored, and an empty message aborts the commit.
#
# Date:      Thu Apr 2 14:13:09 2020 -0400
#
# On branch master
#
# Initial commit
#
# Changes to be committed:
#       new file:   .gitignore
#       new file:   bin/regen.sh
#       new file:   files/install-config.yaml.orig
#       new file:   inventory/rhfsi.yaml
#       new file:   playbooks/regen-install-config.yaml

The playbook I am refactoring is playbooks/regen-install-config.yaml but it will soon be sharing values with others. However, the first step is just to pull the variable out into its own section.

$ git diff
diff --git a/playbooks/regen-install-config.yaml b/playbooks/regen-install-config.yaml
index 77e285c..b0c50c9 100644
--- a/playbooks/regen-install-config.yaml
+++ b/playbooks/regen-install-config.yaml
@@ -1,10 +1,12 @@
 ---
 - hosts: localhost
+  vars:
+    stage_dir: /home/ayoung/ocp-ansible/stage
   tasks:
 
     - name: Creates directory
       file:
-        path:  /home/ayoung/ocp-ansible/stage
+        path:  "{{ stage_dir }}"
         state: directory
 
     - name: regen install-config

To test this out, I rerun the playbook…see if you can find the mistake:

$ bin/regen.sh 
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
 
PLAY [localhost] *****************************************************************************************************************************************************************************
 
TASK [Gathering Facts] ***********************************************************************************************************************************************************************
ok: [localhost]
 
TASK [Creates directory] *********************************************************************************************************************************************************************
ok: [localhost]
 
TASK [regen install-config] ******************************************************************************************************************************************************************
changed: [localhost]
 
TASK [validate install-config] ***************************************************************************************************************************************************************
changed: [localhost]
 
PLAY RECAP ***********************************************************************************************************************************************************************************
localhost                  : ok=4    changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
 
[ayoung@ayoungP40 ocp-ansible]$ ls stage/
install-config.yaml

Did you spot the mistake? I didn’t know what I had prior to running the playbook. I might have fooled myself. So, I remove the stage directory by hand and rerun.


$ rm -rf stage/
$ bin/regen.sh 
...
$ diff stage/
install-config.yaml            .openshift_install.log         .openshift_install_state.json  
[ayoung@ayoungP40 ocp-ansible]$ diff files/install-config.yaml.orig stage/install-config.yaml 
23c23
<   - cidr: 192.0.2.0/24 
---
>   - cidr: 192.0.2.0/24
41d40
<

A minor tweak in formatting, as I expected. But, removing directories by hand is dangerous. Let’s create a new playbook to cleanup. I’ll call it playbooks/cleanup.yaml

- hosts: localhost
  vars:
    stage_dir: /home/ayoung/ocp-ansible/stage
  tasks:
    - name: Creates directory
      file:
        path:  "{{ stage_dir }}"
        state: absent

And hey…there is that duplicated code again….Commit this, but continue to refactor.

This is a case of “manual testing” that I think is OK. Essentially., my test script is:

  • check that stage/ does not exist
  • run regen.sh
  • check that stage/install-config.yaml exists
  • run cleanup.sh
  • check that stage/ does not exist
    run regen.sh

Once this works, commit to git.

Now lets further extract that variable to its own file. First, create a new directory vars and a yaml file to store the variable:

$ cat vars/main.yaml 
---
stage_dir: /home/ayoung/ocp-ansible/stage

And remove the variable from the regen and cleanup playbooks. Run them to ensure that they fail.

[ayoung@ayoungP40 ocp-ansible]$ bin/regen.sh [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match ‘all’ PLAY [localhost] ***************************************************************************************************************************************************************************** TASK [Gathering Facts] *********************************************************************************************************************************************************************** ok: [localhost] TASK [Creates directory] ********************************************************************************************************************************************************************* fatal: [localhost]: FAILED! => {“msg”: “The task includes an option with an undefined variable. The error was: ‘stage_dir’ is undefined\n\nThe error appears to be in ‘/home/ayoung/ocp-ansible/playbooks/regen-install-config.yaml’: line 6, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n – name: Creates directory\n ^ here\n”} PLAY RECAP *********************************************************************************************************************************************************************************** localhost : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 [ayoung@ayoungP40 ocp-ansible]$ bin/cleanup.sh [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match ‘all’ PLAY [localhost] ***************************************************************************************************************************************************************************** TASK [Gathering Facts] *********************************************************************************************************************************************************************** ok: [localhost] TASK [Creates directory] ********************************************************************************************************************************************************************* fatal: [localhost]: FAILED! => {“msg”: “The task includes an option with an undefined variable. The error was: ‘stage_dir’ is undefined\n\nThe error appears to be in ‘/home/ayoung/ocp-ansible/playbooks/cleanup.yaml’: line 5, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n tasks:\n – name: Creates directory\n ^ here\n”} PLAY RECAP *********************************************************************************************************************************************************************************** localhost

Now run them using the external variable file. Here is the git diff:

$ git diff HEAD
diff --git a/bin/cleanup.sh b/bin/cleanup.sh
index 61a2d28..ccc532d 100755
--- a/bin/cleanup.sh
+++ b/bin/cleanup.sh
@@ -1,2 +1,2 @@
 #!/bin/sh 
-ansible-playbook playbooks/cleanup.yaml
+ansible-playbook   -e @vars/main.yaml  playbooks/cleanup.yaml
diff --git a/bin/regen.sh b/bin/regen.sh
index 2b7af04..75b3ef6 100755
--- a/bin/regen.sh
+++ b/bin/regen.sh
@@ -1,2 +1,2 @@
 #!/bin/sh 
-ansible-playbook playbooks/regen-install-config.yaml
+ansible-playbook -e @vars/main.yaml playbooks/regen-install-config.yaml
diff --git a/playbooks/cleanup.yaml b/playbooks/cleanup.yaml
index 213ab37..c48fd7a 100644
--- a/playbooks/cleanup.yaml
+++ b/playbooks/cleanup.yaml
@@ -1,7 +1,6 @@
 ---
 - hosts: localhost
   vars:
-    stage_dir: /home/ayoung/ocp-ansible/stage
   tasks:
     - name: Creates directory
       file:
diff --git a/playbooks/regen-install-config.yaml b/playbooks/regen-install-config.yaml
index b0c50c9..e6103ce 100644
--- a/playbooks/regen-install-config.yaml
+++ b/playbooks/regen-install-config.yaml
@@ -1,7 +1,6 @@
 ---
 - hosts: localhost
:...skipping...
diff --git a/bin/cleanup.sh b/bin/cleanup.sh
index 61a2d28..ccc532d 100755
--- a/bin/cleanup.sh
+++ b/bin/cleanup.sh
@@ -1,2 +1,2 @@
 #!/bin/sh 
-ansible-playbook playbooks/cleanup.yaml
+ansible-playbook   -e @vars/main.yaml  playbooks/cleanup.yaml
diff --git a/bin/regen.sh b/bin/regen.sh
index 2b7af04..75b3ef6 100755
--- a/bin/regen.sh
+++ b/bin/regen.sh
@@ -1,2 +1,2 @@
 #!/bin/sh 
-ansible-playbook playbooks/regen-install-config.yaml
+ansible-playbook -e @vars/main.yaml playbooks/regen-install-config.yaml
diff --git a/playbooks/cleanup.yaml b/playbooks/cleanup.yaml
index 213ab37..c48fd7a 100644
--- a/playbooks/cleanup.yaml
+++ b/playbooks/cleanup.yaml
@@ -1,7 +1,6 @@
 ---
 - hosts: localhost
   vars:
-    stage_dir: /home/ayoung/ocp-ansible/stage
   tasks:
     - name: Creates directory
       file:
diff --git a/playbooks/regen-install-config.yaml b/playbooks/regen-install-config.yaml
index b0c50c9..e6103ce 100644
--- a/playbooks/regen-install-config.yaml
+++ b/playbooks/regen-install-config.yaml
@@ -1,7 +1,6 @@
 ---
 - hosts: localhost
   vars:
-    stage_dir: /home/ayoung/ocp-ansible/stage
   tasks:
 
     - name: Creates directory
diff --git a/vars/main.yaml b/vars/main.yaml
new file mode 100644
index 0000000..5ac011f
--- /dev/null
+++ b/vars/main.yaml
@@ -0,0 +1,2 @@
+---
+stage_dir: /home/ayoung/ocp-ansible/stage

Commit this, and continue to extract the variable from other portions of the file:

$ git diff
diff --git a/playbooks/regen-install-config.yaml b/playbooks/regen-install-config.yaml
index e6103ce..30e76f1 100644
--- a/playbooks/regen-install-config.yaml
+++ b/playbooks/regen-install-config.yaml
@@ -11,10 +11,10 @@
     - name: regen install-config 
       copy:
         src:  /home/ayoung/ocp-ansible/files/install-config.yaml.orig
-        dest: /home/ayoung/ocp-ansible/stage/install-config.yaml
+        dest: "{{ stage_dir }}/install-config.yaml"
 
     - name: validate install-config
-      command: /home/ayoung/apps/ocp4.4/openshift-install create install-config  --dir /home/ayoung/ocp-ansible/stage
+      command: /home/ayoung/apps/ocp4.4/openshift-install create install-config  --dir "{{ stage_dir }}"
       environment:
         OS_CLOUD: fsi-moc

Continue the process with other repeated strings. Lets extrace a base_dir variable that will be used both for finding the source directory and the binaries.

[ayoung@ayoungP40 ocp-ansible]$ git diff
diff --git a/playbooks/regen-install-config.yaml b/playbooks/regen-install-config.yaml
index 30e76f1..5a95ef5 100644
--- a/playbooks/regen-install-config.yaml
+++ b/playbooks/regen-install-config.yaml
@@ -1,6 +1,7 @@
 ---
 - hosts: localhost
   vars:
+    base_dir: /home/ayoung/ocp-ansible
   tasks:
 
     - name: Creates directory
@@ -10,11 +11,11 @@
 
     - name: regen install-config 
       copy:
-        src:  /home/ayoung/ocp-ansible/files/install-config.yaml.orig
+        src:  "{{ base_dir }}/files/install-config.yaml.orig"
         dest: "{{ stage_dir }}/install-config.yaml"
 
     - name: validate install-config
-      command: /home/ayoung/apps/ocp4.4/openshift-install create install-config  --dir "{{ stage_dir }}"
+      command: "{{ base_dir }}/bin/openshift-install create install-config --dir {{ stage_dir }}"
       environment:
         OS_CLOUD: fsi-moc

And keep going. Move the variable to the common variables file so we can use base_dir to build the stage_dir variable.

$ git diff
diff --git a/playbooks/regen-install-config.yaml b/playbooks/regen-install-config.yaml
index 5a95ef5..78dbfdc 100644
--- a/playbooks/regen-install-config.yaml
+++ b/playbooks/regen-install-config.yaml
@@ -1,7 +1,6 @@
 ---
 - hosts: localhost
   vars:
-    base_dir: /home/ayoung/ocp-ansible
   tasks:
 
     - name: Creates directory
diff --git a/vars/main.yaml b/vars/main.yaml
index 5ac011f..01de10c 100644
--- a/vars/main.yaml
+++ b/vars/main.yaml
@@ -1,2 +1,3 @@
 ---
-stage_dir: /home/ayoung/ocp-ansible/stage
+base_dir: /home/ayoung/ocp-ansible
+stage_dir: "{{ base_dir }}/stage"

The basic rules: Test often. Small steps. Commit Successes to Git. Extract repetition. Have fun.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.