Merging Kubernetes client configs at run time

Last time I walked through the process of merging two sets of Kubernetest client configurations into one. For more ephemeral data, you might not want to munge it all into your main configuration. The KUBECONFIG environment variables lets you specify muiltiple configuration files and merge them into a single set of configuration data.

From

kubectl config --help

If $KUBECONFIG environment variable is set, then it is used [as] a list of paths (normal path delimiting rules for your system). These paths are merged. When a value is modified, it is modified in the file that defines the stanza. When a value is created, it is created in the first file that exists. If no files in the chain exist, then it creates the last file in the list.

 

So, lets start with the file downloaded by the kubevirt build system yesterday.

 

[ayoung@ayoung541 vagrant]$ echo $PWD
/home/ayoung/go/src/kubevirt.io/kubevirt/cluster/vagrant
[ayoung@ayoung541 vagrant]$ export KUBECONFIG=$PWD/.kubeconfig
[ayoung@ayoung541 vagrant]$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kubernetes-admin@kubernetes kubernetes kubernetes-admin

Contrast this with what a get without the environment variable set, if I use the configuration in ~/.kube, which I synced over from my OpenShift cluster:

[ayoung@ayoung541 vagrant]$ unset KUBECONFIG
[ayoung@ayoung541 vagrant]$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
 default/munchlax:8443/ayoung munchlax:8443 ayoung/munchlax:8443 default
* default/munchlax:8443/system:admin munchlax:8443 system:admin/munchlax:8443 default
 kube-system/munchlax:8443/system:admin munchlax:8443 system:admin/munchlax:8443 kube-system

I want to create a new configuration for the vagrant managed machines for Kubevirt.  IT turns out that the API server specified there is actually a proxy, a short term shim we put in place as we anxiously awate the Amagalmated Api Server of 1.7.  However, sometimes this proxy is broken or we just need to by-pass it.  The only difference between this setup and the proxied setup is the server URL.

So…I create a new file, based on the .kubeconfig file, but munged slightly.  Here is the diff:

[ayoung@ayoung541 vagrant]$ diff -Nurd .kubeconfig .kubeconfig-core 
--- .kubeconfig 2017-05-24 19:49:24.643158731 -0400
+++ .kubeconfig-core 2017-05-26 11:10:49.359955538 -0400
@@ -3,13 +3,13 @@
 - cluster:
 certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRFM01EVXlOREl4TWpnek5sb1hEVEkzTURVeU1qSXhNamd6Tmxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTmRmCnVINkE3Q1JVRVQ5VzhpSGgyam9EQUNxVGZXQ01ITFN3dzc5Q01DZXlyWFhZazVvR0lIbnZkeVB5aEVNZ2xYYysKczUwZVJZRDBkWTYrYUlnVmtVaElIVitSUHltVHE0WklrS3EzNnV5MXk0TFYzSDNTaGt2eVZBbitjL3htYldaZQp5bEZaZHhSMTFoVjRac0h4WXdzWTR4bmVoaWpkMnkwWUFaQnkwellkQm5xTmE4cFpDb3BNbStLdmtjVEJ1UERGCkp5ZWkzU0tJd3R1R0gxU3ByUCsxdi9OSGFCOTNXR0g0MFQxbm1HZTRGWWQ2SzErcWNNdndpdmY1dVQ4Nk10M2YKVWhEQWZNUlk3aW5maXVsVW1HeUNPWlNsbFhpWlRMWmpoOGZiUW1FdmZvOFJjMm1lOGtwTXJpMDdIWUQ4ZjZFNQpScjNhT05mcTkwd2s1VDM5YWxjQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFNTll1R3N1bGNpY3REQ0pFZ3R3K2ZQSUU3S04KQnRvV2RuZWZZdktya1l1WUVTRkk5alFXTFNmVGN1MnpibzNRWnYzZDg3WnkvNjYyK2R0SWloWFA5V3NJWVhHUApDUXNuTUMyZXY5djlmOU9WbVhZbEhuUUx0YXRiSDZVTFZPdWJZUXlFRlRSa21XV1dwcXpoR1pNWk1pbG8wRzhLCnBNd29Ia0dDWm5tUytyUVVEVWF6QlprcVdzRFNabW5jWUhtdFRtMEJ6RUJpa002SEFsNzAvT21rNGpHcmtHZEQKS2tMWU16UjJkZnlkSklCVGxKdGlGYjRhZ3R5amlFb3NDSGY0Z1oyY0xUMTRyOENud0QrOWxSbVk3dDNDRjIrdgpFOGxxb3RSYVI2TVRyWnZkUXUrOWtFYnNKWVZUN1NQR3pqeEpMZ1BmTGprK0g1YUJWQU9od0tvdTV0QT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
 server: https://192.168.200.2:6443
- name: kubernetes
+ name: core
 contexts:
 - context:
- cluster: kubernetes
+ cluster: core
 user: kubernetes-admin
- name: kubernetes-admin@kubernetes
-current-context: kubernetes-admin@kubernetes
+ name: kubernetes-admin@core
+current-context: kubernetes-admin@core
 kind: Config
 preferences: {}
 users:

Now I have a couple choices. I can just specify this second config file on the command line:

[ayoung@ayoung541 vagrant]$ kubectl --kubeconfig=$PWD/.kubeconfig-core config get-contexts
 CURRENT NAME CLUSTER AUTHINFO NAMESPACE
 kubernetes-admin@core core kubernetes-admin

Or I can munge the two together and provide a flag which states which context to use.

[ayoung@ayoung541 vagrant]$ export KUBECONFIG=$PWD/.kubeconfig:$PWD/.kubeconfig-core
[ayoung@ayoung541 vagrant]$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kubernetes-admin@kubernetes kubernetes kubernetes-admin 
 kubernetes-admin@core core kubernetes-admin

Note that this gives a different current context (with the asterix) than if I reverse the order of the files in the env-var:

[ayoung@ayoung541 vagrant]$ export KUBECONFIG=$PWD/.kubeconfig-core:$PWD/.kubeconfig
[ayoung@ayoung541 vagrant]$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kubernetes-admin@core core kubernetes-admin 
 kubernetes-admin@kubernetes kubernetes kubernetes-admin

Whichever one declared the default first wins.

However, regardless of the order, I can explicitly set the context I want to use on the command line:

[ayoung@ayoung541 vagrant]$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
 kubernetes-admin@core core kubernetes-admin 
* kubernetes-admin@kubernetes kubernetes kubernetes-admin 
[ayoung@ayoung541 vagrant]$ kubectl --context=kubernetes-admin@core config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* kubernetes-admin@kubernetes kubernetes kubernetes-admin 
 kubernetes-admin@core core kubernetes-admin

Again, notice the line where asterix specifies which context is in use.

With only two files, it might be easier to just specify the –kubeconfig option, but as the number of configs you work with grows, you might find you want to share the user data between two of them, or have a bunch of scripts that work between them, and it is easier to track which context to use than to track which file contains which set of data.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.