You have a cloud, I have a cloud.

Neither of use are willing to surrender control of our OpenStack deployments, but we need to inter-operate.

We both have Keystone servers. Those servers are the system of record for user authorization through out our respective deployments. We each wish to maintain control of our assignments. How can we make a set of resources that can be shared?  It can’t be done today.  Here is why not, and how to make it possible.

Simple example:  Nova in OpenStack Deployment A (OSDA) creates a VM using an image from Glance in OpenStackDeployment B. Under the rules of policy, the VM and the image must both reside in the same project. However, the endpoints are managed by two different Keystone servers.

At the Keystone to Keystone level, we need to provide a shared view of the project. Since every project is owned by a domain, the correct level of abstraction is for the two Keystone servers to have a common view of the domain.  In our example we create a Domain in OSDA named “SharedDom” and assume it assigns “54321CBE” as the domain id. Keystone for OSDB needs to have an identical domain. Lets loosen the rules on domain creation to allow that: create domain takes both name and accepts “54321CBE” as the domain’s identifier.

Once we have the common domain definition, we can create a project under “SharedDom” in OSDA and, again, put a copy into OSDB.

Let us also assume that the service catalog has been synchronized across the two Keystone images. While you might be tempted to just make each a region in one Keystone server, that surrenders the control of the cloud to the remote Keystone admin, and so will not be organizationally acceptable. So, while we should tag each service catalog as a region, and keep the region names generally unique, they still are managed by distinct peer Keystone servers.

The API call that creates a VM in one endpoint by fetching an image from another accesses both endpoints with a single token. It is impossible to do today. Which keystone is then used to allocate the token? If it is OSDA, then Glance would reject it. if OSDB, Nova would reject it.

For PKI tokens, we can extract the “Signing Data” from the token body and determine which certificate signed it. From the certificate we can determine which Keystone server signed the token. Keystone would then have to provide rules for which Keystone server was allowed to sign for a domain. By default, it would be only the authoritative Keystone server associated with the same deployment. However, Keystone from OSDA would allow Keystone from OSDB to sign for Tokens under “SharedDom” only.

If both keystone servers are using UUID tokens we could arrange for the synchronization of all tokens for the shared domain across keystone instances. Now each endpoint would still authenticate tokens against its own Keystone server. The rules would be the same as far as determining which Keystone could sign for which tokens, it would just be enforced at the Keystone server level. My biggest concern with this approach is that there are synchronization problems; it has a tendency toward race conditions. But it could be made to work as well.

The agreement can be set one direction.  If OSDA  is managing the domain, and OSDB accepts the agreement, then OSDA can sign for tokens for the domain that are used across both deployments endpoints.   Keystone for OSDA must explicitly add a rule that says that Keystone for OSDB can sign for tokens as well.

This setup works  on a gentleman’s agreement genteel understanding that only one of the two Keystone servers  actively manages the domain.  One Keystone server is delegated the responsibility of managing the domain.  The other one performs that delegation, and agrees to stay out of it.  If the shared domain is proving to be too much trouble, either Keystone server can disable the domain without the agreement of the other Keystone server.

As Mick Jagger sang “Hey, you!  Get off of my cloud!”