Copying files into a container at run time

There are three distinct things that have to happen between installing the keystone software and running a Keystone instance. The first if management of the configuration files. Second is the database migrations, and third is the keystone bootstrap of the data base values. When coding container images to run a keystone server, not only do you need to be aware of each of these stpes, you need to make sure you are performing them in such a way that you can run scale the the Keystone server horizontally, handle zero downtime upgrades, and handle token-validating key rotations. Federated identity adds an additional twist as you need to handle the addition of httpd config changes for new identity providers.

Let’s walk through this setup in detail.

Keystone was written assuming a Linux setup with configuration files in /etc. The keystone package owns the /etc/keystone directory. The primary configuration file is /etc/keystone/keystone.conf. This file is not checked in to the Keystone source repository, but is rather generated from defaults embedded in the source code. I generate the base config file using tox:

tox genconfig

While the RPMs and other package files do something similar, they then hold on the artifact, and will have the Keystone package own the file once installed.

When tox builds the sample configuration file, it ends up in the keystone git repository directory under etc/keystone.conf.sample. The values in this file direct all other changes.

The most important change to the keystone.conf file deals with the Database connection. The connect config is in the [database] section of the config file. The default value is

connection = <none>

This cannot be defaulted. This is the SQL Alchemy connection string used to find and authenticate to the Relational Database that stores all of the data for the Keystone instance. It looks like this:


The password field is used for authentication, and it must be clear text. This is, obviously, a bad pattern. Finding a way to fix this problem is beyond the scope of this article. For now, lets talk about mitigation.

In the Kubernetes world, passwords should be kept in secrets. Using a local podman, we can mimic this by passing in environment variables when we run the container, or by mounting a volume. The container needs to use these variables to update the config file prior to running. The script that runs your application, whether to run bootstrap or to run the keystone server itself, must update the config file. If the environment variable is passwed in on the command line, the script would look like this:

openstack-config  --set  /etc/keystone/keystone.conf database connection mysql+pymysql://keystone:$MYSQL_PASSWORD@$mariadb-keystone/keystone

This way, the user only has one variable they can affect: the password itself.

If you want to do the same thing using a file:

MYSQL_PASSWORD=$( cat /etc/keystone/mysql.pass )
openstack-config  --set  /etc/keystone/keystone.conf database connection mysql+pymysql://keystone:$MYSQL_PASSWORD@$mariadb-keystone/keystone

You could use both, to allow a command line override for testing:

if [ -z "$MYSQL_PASSWORD" ]
then MYSQL_PASSWORD=$( cat /etc/keystone/dbpass.txt )

This allows me to run the database initialization like this from the cli:

 podman run -it   --add-host keystone-mariadb:   --network maria-bridge   --env MYSQL_PASSWORD=my-secret-pw   localhost/keystoneconfig

But better yet, if I want to run it while mounting a file inside the container:

 podman run -it --mount type=bind,source=/tmp/pass.txt,destination=/etc/keystone/dbpass.txt  --add-host keystone-mariadb:   --network maria-bridge   localhost/keystoneconfig

This last option matches the way that Kubernetes would mount a secret as a file.

We can use this same method for handline the keys for fernet tokens. The following line runs the keystone server

 podman run -d  --mount type=bind,source=/tmp/pass.txt,destination=/etc/keystone/dbpass.txt      --mount type=bind,source=/etc/keystone/fernet-keys,destination=/etc/keystone/fernet-keys   --add-host keystone-mariadb:   --network maria-bridge   -it localhost/keystone

Here’s the modified version of the script that runs the web server and wsgi module. You can see where I commented out the explicit creation of the fernet key repository.

# Copied from
# Make sure we're not confused by old, incompletely-shutdown httpd
# context after restarting the container.  httpd won't start correctly
# if it thinks it is already running.
rm -rf /run/httpd/* /tmp/httpd*
keystone-manage bootstrap --bootstrap-password my-secret-password
if [ -z "$MYSQL_PASSWORD" ]
then MYSQL_PASSWORD=$( cat /etc/keystone/dbpass.txt )
openstack-config  --set  /etc/keystone/keystone.conf database connection mysql+pymysql://keystone:$MYSQL_PASSWORD@$MYSQL_HOST/keystone
#keystone-manage fernet_setup  --keystone-user keystone  --keystone-group keystone
exec /usr/sbin/apachectl -DFOREGROUND

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.