Kubernetes is moving from Third Party Resources to the Aggregated API Server. In order to work with this and continue to deploy on OpenShift Origin, we need to move from working with the shipped and stable version that is in Fedora 26 to the development version in git. Here are my notes to get it up and running.
Process
It took a couple tries to realize that the go build process needs a fairly bulky virtual machine. I ended up using one that has 8 GB Ram and 50 GB disk. In order to minimize churn, I also went with a Centos7 deployment.
Once I had a running VM here are the configuration changes I had to make.
Use nmcli c eth0 on and set ONBOOT. This can also be done via editing the network config files or using older tools.
yum update -y yum groupinstall "Development Tools" yum instyall -y origin-clients |
Ensure that the Docker daemon is running with the following argument:
–insecure-registry 172.30.0.0/16 By editing the file /etc/sysconfig/docker. My Options line looks like this:
OPTIONS='--selinux-enabled --log-driver=journald --signature-verification=false --insecure-registry 172.30.0.0/16' |
Followed dirs from here In order to set up the development environment.
cd $GOPATH/src/github.com/openshift/origin hack/env hack/build-base-images.sh OS_BUILD_ENV_PRESERVE=_output/local/releases hack/env make release |
Note that the KubeVirt code I want to run on top of this requires a later version of go, and thus I upgrade to go version go1.8.3 linux/amd64Â via the tarball install method.
The Hash that gets generated by the build depends on when you run.  To see the images run
docker images |
I expand the terminal to full screen lots of columns of data. Here is a subset:
REPOSITORY TAG IMAGE ID CREATED SIZE openshift/hello-openshift 413eb73 911092241b5a 35 hours ago 5.84 MB openshift/hello-openshift latest 911092241b5a 35 hours ago 5.84 MB openshift/openvswitch 413eb73 c53aae019d81 35 hours ago 1.241 GB openshift/openvswitch latest c53aae019d81 35 hours ago 1.241 GB openshift/node 413eb73 af6135fc50c9 35 hours ago 1.239 GB openshift/node latest af6135fc50c9 35 hours ago 1.239 GB |
The tag is the second column. This is what I use in order to install. I don’t use “Latest” as that changes over time, and it might accidentally succeed using a remote package when the local build failed.
I want to be able to edit configuration values. A also want the etcd store to persist across reboots. Thus,
sudo mkdir /var/lib/origin/etcd sudo chown ayoung:ayoung /var/lib/origin/etcd |
And then my final command line to bring up the cluster is:
oc cluster up --use-existing-config --loglevel=5 --version=413eb73 --host-data-dir=/var/lib/origin/etcd/ | tee /tmp/oc.log 2>&1 |
Notes:
Below are some of my Troubleshooting notes. I am going to leave them in here so they show up on future searches for people that have the same problems. They are rough, and you don’t need to read them.
hack/env make release errored
[WARNING] Copying _output/local/releases from the container failed!
[WARNING] Error response from daemon: lstat /var/lib/docker/devicemapper/mnt/fb199307b2f95649066c42f55e5487c66eb3421e5407c8bd6d2f0a7058bc8cd5/rootfs/go/src/github.com/openshift/origin/_output/local/releases: no such file or directory
Tried with OS_BUILD_ENV_PRESERVE=_output/local but no difference.
Should have been
OS_BUILD_ENV_PRESERVE=_output/local/releases hack/env make release
This did not work (basename error)
export PATH=”${PATH}:$( source hack/lib/init.sh; echo “${OS_OUTPUT_BINPATH}/$( os::util::host_platform )/” )”
But I was able to do
export PATH=$PATH:$PWD/_output/local/bin/linux/amd64/
and then
oc cluster up –version=latest
failed due to docker error
— Checking Docker daemon configuration … FAIL
Error: did not detect an –insecure-registry argument on the Docker daemon
Solution:
used https://wiki.centos.org/SpecialInterestGroup/PaaS/OpenShift-Quickstart
To fix
oc seems to be running OK now. But not using my git commit
Told to run:
Then
oc cluster up –version=8d96d48