My cloud may not look like your cloud. The contract between the application deployment and the Kubernetes installation is a set of manifest files that guide Kubernetes in selecting, naming, and exposing resources. In order to make the generation of the Manifests sane in KubeVirt, we’ve provided a little bit of build system support.
The manifest files are templatized in a jinja style. I say style, because the actual template string replacement is done using simmple bash scripting. Regardless of the mechanism, it should not be hard for a developer to understand what happens. I’ll assume that you have your source code checked out in $GOPATH/src/kubevirt.io/kubevirt/
The template files exist in the manifests subdirectory. Mine looks like this:
haproxy.yaml.in squid.yaml.in virt-manifest.yaml.in iscsi-demo-target.yaml.in virt-api.yaml.in vm-resource.yaml.in libvirt.yaml.in virt-controller.yaml.in migration-resource.yaml.in virt-handler.yaml.in |
The simplest way to generate a set of actual manifest files is to run make manifests
make manifests ./hack/build-manifests.sh $ ls -l manifests/*yaml -rw-rw-r--. 1 ayoung ayoung 672 Aug 21 10:17 manifests/haproxy.yaml -rw-rw-r--. 1 ayoung ayoung 2384 Aug 21 10:17 manifests/iscsi-demo-target.yaml -rw-rw-r--. 1 ayoung ayoung 1707 Aug 21 10:17 manifests/libvirt.yaml -rw-rw-r--. 1 ayoung ayoung 256 Aug 21 10:17 manifests/migration-resource.yaml -rw-rw-r--. 1 ayoung ayoung 709 Aug 21 10:17 manifests/squid.yaml -rw-rw-r--. 1 ayoung ayoung 832 Aug 21 10:17 manifests/virt-api.yaml -rw-rw-r--. 1 ayoung ayoung 987 Aug 21 10:17 manifests/virt-controller.yaml -rw-rw-r--. 1 ayoung ayoung 954 Aug 21 10:17 manifests/virt-handler.yaml -rw-rw-r--. 1 ayoung ayoung 1650 Aug 21 10:17 manifests/virt-manifest.yaml -rw-rw-r--. 1 ayoung ayoung 228 Aug 21 10:17 manifests/vm-resource.yaml |
Looking at the difference between, say the virt-api template and final yaml file:
$ diff -u manifests/virt-api.yaml.in manifests/virt-api.yaml --- manifests/virt-api.yaml.in 2017-07-20 13:29:00.532916101 -0400 +++ manifests/virt-api.yaml 2017-08-21 10:17:10.533038861 -0400 @@ -7,7 +7,7 @@ - port: 8183 targetPort: virt-api externalIPs : - - "{{ master_ip }}" + - "192.168.200.2" selector: app: virt-api --- @@ -23,14 +23,14 @@ spec: containers: - name: virt-api - image: {{ docker_prefix }}/virt-api:{{ docker_tag }} + image: kubevirt/virt-api:latest imagePullPolicy: IfNotPresent command: - "/virt-api" - "--port" - "8183" - "--spice-proxy" - - "{{ master_ip }}:3128" + - "192.168.200.2:3128" ports: - containerPort: 8183 name: "virt-api" @@ -38,4 +38,4 @@ securityContext: runAsNonRoot: true nodeSelector: - kubernetes.io/hostname: {{ primary_node_name }} + kubernetes.io/hostname: master |
make manifests, it turns out, just calls a bash script ./hack/build-manifests.sh. This script uses two files to determine the values to use for template string substitution. First, the defaults: hack/config-default.sh. This is where master_ip get the value of 192.168.200.2. This file also gives priority to the $DOCKER_TAG environment variable. However, if you need to customize values further, you can create and manage them in the file hack/config-local.sh. The goal is that any of the keys from the -default file that are specified in the hack/config-local.sh will use the value from the latter file. The set of keys with their defaults (as of this writing) that you can customize are:
binaries="cmd/virt-controller cmd/virt-launcher cmd/virt-handler cmd/virt-api cmd/virtctl cmd/virt-manifest" docker_images="cmd/virt-controller cmd/virt-launcher cmd/virt-handler cmd/virt-api cmd/virt-manifest images/haproxy images/iscsi-demo-target-tgtd images/vm-killer images/libvirt-kubevirt images/spice-proxy cmd/virt-migrator cmd/registry-disk-v1alpha images/cirros-registry-disk-demo" optional_docker_images="cmd/registry-disk-v1alpha images/fedora-atomic-registry-disk-demo" docker_prefix=kubevirt docker_tag=${DOCKER_TAG:-latest} manifest_templates="`ls manifests/*.in`" master_ip=192.168.200.2 master_port=8184 network_provider=weave primary_nic=${primary_nic:-eth1} primary_node_name=${primary_node_name:-master} |
Not all of these are for Manifest files. The docker_images key is used in selecting the set of generating Docker images to generate in a command called from a different section of the Makefile. The network_provider is used in the Vagrant setup, and so on.However, most of the values are used in the manifest files. So, If I want to set a master IP Address of 10.10.10.10, I would have a hack/config-local.sh file that looks like this:
master_ip=10.10.10.10 |
$ diff -u manifests/virt-api.yaml.in manifests/virt-api.yaml --- manifests/virt-api.yaml.in 2017-07-20 13:29:00.532916101 -0400 +++ manifests/virt-api.yaml 2017-08-21 10:42:28.434742371 -0400 @@ -7,7 +7,7 @@ - port: 8183 targetPort: virt-api externalIPs : - - "{{ master_ip }}" + - "10.10.10.10" selector: app: virt-api --- @@ -23,14 +23,14 @@ spec: containers: - name: virt-api - image: {{ docker_prefix }}/virt-api:{{ docker_tag }} + image: kubevirt/virt-api:latest imagePullPolicy: IfNotPresent command: - "/virt-api" - "--port" - "8183" - "--spice-proxy" - - "{{ master_ip }}:3128" + - "10.10.10.10:3128" ports: - containerPort: 8183 name: "virt-api" @@ -38,4 +38,4 @@ securityContext: runAsNonRoot: true nodeSelector: - kubernetes.io/hostname: {{ primary_node_name }} + kubernetes.io/hostname: master |