Testing OpenShift Network Policy

The first principle is that you must not fool yourself and you are the easiest person to fool.

Richard P. Feynman

Test before you deploy. Treat configuration as code. These are precepts of DevOps that we want to make real. When dealing with network policy, we want to test it out in a development context before deploying it in production.

When one service calls another, it can avoid the overhead of passing through the external load balancer by using the cluster IP assigned to the other service. If both services are in the same project, the default policies allows this with no changes.

If we need to cross project boundaries, we need to customize policy. Specifically, we need to create policies that will mirror the default, while also allowing access from another project.

For my example, I am going to create two projects. One is my demo-servicet, that has the app ab-deploy running in it. This application is the target. We want to show we can get access to it from services both inside and outside demo-project. The second project is called check-project and exists solely to run the code to check the cross-project call.

I am going to use the simple Flask service I wrote about in my previous post. I will have two deployments of this code, one running in demo-service and one running in check-project. The one running in demo-project is part of our control; it shows that we can access the ab-deploy application. The one running in check-project will show what happens when we make cross-project calls.

I added three policies to the demo-service project.

  • allow-from-openshift-ingress
  • allow-same-namespace
  • allow-from-openshift-global

The first two mimic default policy: they allow access from the ingress controller and from services in the same namespace. There are others, specifically monitoring, that I should add as well, but this is enough to prove my point.

The last policy allows access from any namespace with a special label on it. That policy looks like this:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  creationTimestamp: "2020-07-31T20:12:12Z"
  generation: 1
  name: allow-from-openshift-global
  namespace: demo-service
  resourceVersion: "9196451"
  selfLink: /apis/networking.k8s.io/v1/namespaces/demo-service/networkpolicies/allow-from-openshift-global
  uid: 144e751e-f71f-416b-8d7e-3938e52cb9b9
spec:
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          network.openshift.io/policy-group: global
  podSelector: {}
  policyTypes:
  - Ingress

The line to take note of here is:

network.openshift.io/policy-group: global

This is what I want to test: If a project has that label, it should be able to access the ab-service. If it does not, I should not be able to access the service.

For my first round of testing, I am going to hard code values. In the future, I will be able to look these up. The values I am going to hard code are the Cluster IP of the ab-deploy service and the URLs of the ckaccess service that calls them. I’ve collected them all into this script:

#!/bin/python3
 
import requests
import os
 
services = ["http://172.21.193.8:8080", "https://www.redhat.com"]
checkers={
    'ckaccess1-demo-service':
    'http://ckaccess1-demo-service.ayoung-rhfsi-f2c6cdc6801be85fd188b09d006f13e3-0000.us-south.containers.appdomain.cloud/',
     'ckaccess-check-project':
    'http://ckaccess-check-project.ayoung-rhfsi-f2c6cdc6801be85fd188b09d006f13e3-0000.us-south.containers.appdomain.cloud/'
    }
 
 
def run_tests():
    for project, url in checkers.items():
        for service in services:
            print("--------------------------------------------\n")
            print('CHECKER    : %s' % project) 
            print('SERVICE_URL: %s' % service)
            try:
                r = requests.get(url,
                                 params={'service_url': service},
                )
                print('status=%d' % (r.status_code))
                print("%s" % response.text)
                print("\n")
            except requests.exceptions.RequestException:
                print('Error Calling Service %s!' % url)
 
print("Start\n")
print("make sure the project is unlabeled")
os.system("oc label namespace check-project 'network.openshift.io/policy-group-'")
run_tests()
 
print("Label the project")
os.system("oc label namespace check-project 'network.openshift.io/policy-group=global'")
 
run_tests()
 
print("Unlabel the project")
os.system("oc label namespace check-project 'network.openshift.io/policy-group-'")
run_tests()

Note that I added in a call to redhat.com just to make sure I had a sanity check on my script: I was pretty sure I could call that from inside my cluster.

The output is below. This test requires a human to analyze it, which is not what I would do for an automated test. However, it makes it easier to show the example here.

Note that the places where check_project is unlabeled, the calls from ckaccess-check-project return the message “Could not reach Service http://172.21.193.8:8080!” But the others all succeed.

When the project is labeled, those calls return “Testing Service for http://172.21.193.8:8080! for status 200”

ayoung@ayoungP40 ~]$ ./run_tests.py
Start
 
make sure the project is unlabeled
label "network.openshift.io/policy-group" not found.
namespace/check-project labeled
--------------------------------------------
 
CHECKER    : ckaccess1-demo-service
SERVICE_URL: http://172.21.193.8:8080
status=200
Testing Service for http://172.21.193.8:8080!  for status 200 
 
 
--------------------------------------------
 
CHECKER    : ckaccess1-demo-service
SERVICE_URL: https://www.redhat.com
status=200
Testing Service for https://www.redhat.com!  for status 200 
 
 
--------------------------------------------
 
CHECKER    : ckaccess-check-project
SERVICE_URL: http://172.21.193.8:8080
status=200
Could not reach Service http://172.21.193.8:8080!
 
 
--------------------------------------------
 
CHECKER    : ckaccess-check-project
SERVICE_URL: https://www.redhat.com
status=200
Testing Service for https://www.redhat.com!  for status 200 
 
 
--------------------------------------------
 
Label the project
--------------------------------------------
 
namespace/check-project labeled
--------------------------------------------
 
CHECKER    : ckaccess1-demo-service
SERVICE_URL: http://172.21.193.8:8080
status=200
Testing Service for http://172.21.193.8:8080!  for status 200 
 
 
--------------------------------------------
 
CHECKER    : ckaccess1-demo-service
SERVICE_URL: https://www.redhat.com
status=200
Testing Service for https://www.redhat.com!  for status 200 
 
 
--------------------------------------------
 
CHECKER    : ckaccess-check-project
SERVICE_URL: http://172.21.193.8:8080
status=200
Testing Service for http://172.21.193.8:8080!  for status 200 
 
 
--------------------------------------------
 
CHECKER    : ckaccess-check-project
SERVICE_URL: https://www.redhat.com
status=200
Testing Service for https://www.redhat.com!  for status 200 
 
 
--------------------------------------------
 
Unlabel the project
--------------------------------------------
 
namespace/check-project labeled
--------------------------------------------
 
CHECKER    : ckaccess1-demo-service
SERVICE_URL: http://172.21.193.8:8080
status=200
Testing Service for http://172.21.193.8:8080!  for status 200 
 
 
--------------------------------------------
 
CHECKER    : ckaccess1-demo-service
SERVICE_URL: https://www.redhat.com
status=200
Testing Service for https://www.redhat.com!  for status 200 
 
 
--------------------------------------------
 
CHECKER    : ckaccess-check-project
SERVICE_URL: http://172.21.193.8:8080
status=200
Could not reach Service http://172.21.193.8:8080!
 
 
--------------------------------------------
 
CHECKER    : ckaccess-check-project
SERVICE_URL: https://www.redhat.com
status=200
Testing Service for https://www.redhat.com!  for status 200

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.