After calling for Keystone to migrate to HTTPD, several people asked me if I would show how this can be done. Here are the steps.
Update 2; See a more recent article here: http://andymc-stack.co.uk/2013/06/apache2-mod_wsgi-openstack-pt1-keystone/ which is based on the info below. The below article goes into more detail. This is a moving target, and I would recommend reading both articles.
Update: in testing keystone, you need to change the username from demo to admin as per the export lines near the bottom of this page.
A prerequisite is to set up NSS HTTPD SSL support. We can use it in conjunction with Keystone to provide encrypted access for authentication and authorization.
I created a file to enable wsgi:
/var/www/cgi-bin/keystone/keystone.py
import os
from keystone import config
from paste import deploy
from keystone.common import logging
from keystone.common import utils
from keystone.common import wsgi
LOG = logging.getLogger(__name__)
CONF = config.CONF
config_files = ['/opt/stack/keystone/etc/keystone.conf']
CONF(config_files=config_files)
conf = CONF.config_file[0]
#name = 'admin'
name = os.path.basename(__file__)
if CONF.debug:
CONF.log_opt_values(logging.getLogger(CONF.prog), logging.DEBUG)
options = deploy.appconfig('config:%s' % CONF.config_file[0])
application = deploy.loadapp('config:%s' % conf, name=name)
And I hardlinked it:
cd /var/www/cgi-bin/keystone/ ln keystone.py admin ln keystone.py main
This is based on a Fedora devstack deployment, which checks out the code in /opt/stack/keystone. I ran against master from git, which as of this writing should be the same as the next Essex RC. I ran python setup.py install with this, which gave me the installed egg in
/usr/lib/python2.7/site-packages/keystone-2012.2-py2.7.egg.
Since eggs deployed this way are part of the default Python path, it is usable by WSGI applications in Apache.
To run against the Fedora RPMs, the difference would be to read in the config file /etc/keystone/keystone.conf instead.
I’ve changed /etc/httpd/conf.d/keystone.conf to
WSGIScriptAlias /main /var/www/cgi-bin/keystone/main WSGIScriptAlias /admin /var/www/cgi-bin/keystone/adminNSSRequireSSL NSSRequireSSL
and restarted httpd. To test:
The most straightforward way to test Keystone is via the command line interface (CLI). When running via devstack, the normal approach is to source the environment file:
. openrc
which pulls in the localrc. This sets many values, including the URL to be used when the Keystone CLI talks to the server: OS_AUTH_URL. Typically this is set as some variation on the local hostname and the port 5000. It also sets the username to demo To test HTTPD, change this to:
export OS_AUTH_URL=https://$HOSTNAME/admin/v2.0 export OS_USERNAME=admin
However, running the command line client gave an error:
[ayoung@ayoungstack devstack]$ keystone user-list No handlers could be found for logger "keystoneclient.client" Invalid OpenStack Identity credentials
Looking in the access log, I saw the token request was succeeding (returned 200) but the second request failed with unauthorized (401).
One difference between Eventlet and the Fedora build of HTTPD is that each request in HTTPD is handled by a separate process. This is called prefork, and you can test a given install by running
[ayoung@ayoung keystone]$ httpd -l Compiled in modules: core.c prefork.c http_core.c mod_so.c
To see the built in modules. The default way that Keystone stores Tokens is by using a key value pair storage. You see this in the keystone.conf file.
[token] driver = keystone.token.backends.kvs.Token
By changing this to the SQL store, the tokens are now accessable by other processes, including the ones that will handle follow on requests to Keystone.
[token] driver = keystone.token.backends.sql.Token
And now the keystone CLI works:
[ayoung@ayoungstack devstack]$ keystone user-list +----------------------------------+---------+--------------------+--------+ | id | enabled | email | name | +----------------------------------+---------+--------------------+--------+ | 6d42897928974d469a4223302e6bae4a | True | admin@example.com | admin | | 72f2941c7e3e40b292d50bd8b1662899 | True | demo@example.com | demo | | 92556c2d36de4f46a0a9916d96d2e793 | True | glance@example.com | glance | | e452d5f649ed489c8b84b8192e781263 | True | nova@example.com | nova | +----------------------------------+---------+--------------------+--------+
I set up all the things and when testing with command line client as suggested within the article I have such error:
$ keystone user-list
No handlers could be found for logger “keystoneclient.client”
Authorization Failed: An unexpected error prevented the server from fulfilling your request. (OperationalError) attempt to write a readonly database u’INSERT INTO token (id, expires, extra) VALUES (?, ?, ?)’
My apache2 web server is running as root – could it be a reason? I checked owner of the keystone.db and it’s good, file permissions also looks good. The keystone service is run as ‘user’ user not root and if I use http mode everything works very well. I tried also to use suexec in apache to run cgi script under keystone user ‘user’ but it didn’t helped. Any clues?
I just saw the same thing as well, but with Devstack, with the error coming from Horizon.
HTTPD should not be running as root. If you run it with the init script/ssytemd, it should be running as the user ‘nobody’ or ‘apache’ depending on your setup. Mine runs as apache.
That said, it does sound like a permissions problem. Perhaps the issue is that the mod_wsgi call to the DB layer is coming from a user not trusted by MySQL.
OK I ran apache2 as ‘user’ user and the problem with read only DB disappeared and keystone works perfectly over ssl. Thanks.
But now I try to configure it with Swift all over ssl. Did you try it maybe? When I tried “swift stat – v” command I can see in logs that token was successfully retrieved from keystone over ssl but later swift isn’t able to talk with keystone properly over ssl via apache. In apache logs I can see that there is a request for resource with path “/var/www/v2.0” which is not found of course.
When I tried normal http mode (without apache in between) everything works well. I know it is not a subject of this article but maybe you have some experience in it?
I installed Openstack with Devstack.
But, in compute node, “fail” happens.
Repeated words are “Invalid OpenStack Identity credentials”.
I want to know Why.
==============================================
+ [[ 0 -ne 0 ]]
+ [[ http://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-uec.tar.gz =~ openvz ]]
+ KERNEL=
+ RAMDISK=
+ DISK_FORMAT=
+ CONTAINER_FORMAT=
+ UNPACK=
+ case “$IMAGE_FNAME” in
+ ‘[‘ cirros-0.3.0-x86_64-uec ‘!=’ cirros-0.3.0-x86_64-uec.tar.gz ‘]’
+ IMAGE_NAME=cirros-0.3.0-x86_64-uec
+ xdir=/home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec
+ rm -Rf /home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec
+ mkdir /home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec
+ tar -zxf /home/glory/devstack/files/cirros-0.3.0-x86_64-uec.tar.gz -C /home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec
++ for f in ‘”$xdir/”*-vmlinuz*’ ‘”$xdir/”aki-*/image’
++ ‘[‘ -f /home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec/cirros-0.3.0-x86_64-vmlinuz ‘]’
++ echo /home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec/cirros-0.3.0-x86_64-vmlinuz
++ break
++ true
+ KERNEL=/home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec/cirros-0.3.0-x86_64-vmlinuz
++ for f in ‘”$xdir/”*-initrd*’ ‘”$xdir/”ari-*/image’
++ ‘[‘ -f /home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec/cirros-0.3.0-x86_64-initrd ‘]’
++ echo /home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec/cirros-0.3.0-x86_64-initrd
++ break
++ true
+ RAMDISK=/home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec/cirros-0.3.0-x86_64-initrd
++ for f in ‘”$xdir/”*.img’ ‘”$xdir/”ami-*/image’
++ ‘[‘ -f /home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec/cirros-0.3.0-x86_64-blank.img ‘]’
++ echo /home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec/cirros-0.3.0-x86_64-blank.img
++ break
++ true
+ IMAGE=/home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec/cirros-0.3.0-x86_64-blank.img
+ [[ -z cirros-0.3.0-x86_64-uec ]]
+ ‘[‘ ” = bare ‘]’
+ KERNEL_ID=
+ RAMDISK_ID=
+ ‘[‘ -n /home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec/cirros-0.3.0-x86_64-vmlinuz ‘]’
++ glance –os-auth-token d769e9c1d8014781a90504473550aae7 –os-image-url http://129.254.173.128:9292 image-create –name cirros-0.3.0-x86_64-uec-kernel –public –container-format aki –disk-format aki
++ grep ‘ id ‘
++ get_field 2
++ read data
Request returned failure status.
Invalid OpenStack Identity credentials.
+ KERNEL_ID=
+ ‘[‘ -n /home/glory/devstack/files/images/cirros-0.3.0-x86_64-uec/cirros-0.3.0-x86_64-initrd ‘]’
++ grep ‘ id ‘
++ glance –os-auth-token d769e9c1d8014781a90504473550aae7 –os-image-url http://129.254.173.128:9292 image-create –name cirros-0.3.0-x86_64-uec-ramdisk –public –container-format ari –disk-format ari
++ get_field 2
++ read data
Request returned failure status.
Invalid OpenStack Identity credentials.
+ RAMDISK_ID=
+ glance –os-auth-token d769e9c1d8014781a90504473550aae7 –os-image-url http://129.254.173.128:9292 image-create –name cirros-0.3.0-x86_64-uec –public –container-format ami –disk-format ami
Request returned failure status.
Invalid OpenStack Identity credentials.
++ failed
++ local r=1
++ set +o xtrace
stack.sh failed: full log in /opt/stack/logs/stack.sh.log.2012-09-17-213800
hi i got the following error
No handlers could be found for logger “keystoneclient.client”
Authorization Failed: Unable to communicate with identity service: [Errno 111] Connection refused. (HTTP 400)
what may be its reason
Sounds like maybe you are talking on the wrong port. Could also be firewall. If you are going against default HTTP, it should be port 443, not 5000 or 35357
I want to know after these steps, is keystone api also moved to apache? I want to generate token use keystone-api ,and after these steps is the url ‘http://localhost:5000/admin/v2.0/tokens ?
the generate api http://docs.openstack.org/api/openstack-identity-service/2.0/content/POST_authenticate_v2.0_tokens_Token_Operations.html
I have solved it.
1. I checked out keystone source from GIT.
2. Copied keystone/httpd/wsgi-keystone to /etc/httpd/conf.d/wsgi-keystone
3. Copied keystone/httpd/keystone.py to /usr/local/www/wsgi-scripts/keystone.py
4. Inside /etc/httpd/conf.d/wsgi-keystone modified following lines :-
WSGIScriptAlias /keystone/main /usr/local/www/wsgi-scripts/keystone.py
WSGIScriptAlias /keystone/admin /usr/local/www/wsgi-scripts/keystone.py
So that we directly use the keystone.py (WSGI Application)
5. Restarted httpd.
6. The /etc/httpd/conf/httpd.conf is left unmodified
7. Try running curl -vs http://127.0.0.1/keystone/admin fails with error
im getting “Target WSGI script ‘/var/www/cgi-bin/keystone/main’ cannot be loaded as Python module”
while configuring federation for keystone any help
[info] [client 10.10.10.12] mod_wsgi (pid=25248, process=’keystone’, application=’10.1.193.250:5000|’): Loading WSGI script ‘/var/www/cgi-bin/keystone/main’.
[error] [client 10.10.10.12] mod_wsgi (pid=25248): Target WSGI script ‘/var/www/cgi-bin/keystone/main’ cannot be loaded as Python module.
[error] [client 10.10.10.12] mod_wsgi (pid=25248): Exception occurred processing WSGI script ‘/var/www/cgi-bin/keystone/main’.
[error] [client 10.10.10.12] Traceback (most recent call last):
[error] [client 10.10.10.12] File “/var/www/cgi-bin/keystone/main”, line 21, in
[error] [client 10.10.10.12] config.configure()
[error] [client 10.10.10.12] File “/usr/lib/python2.7/dist-packages/keystone/common/config.py”, line 694, in configure
[error] [client 10.10.10.12] help=’Do not monkey-patch threading system modules.’))
[error] [client 10.10.10.12] File “/usr/lib/python2.7/dist-packages/oslo/config/cfg.py”, line 1579, in __inner
[error] [client 10.10.10.12] result = f(self, *args, **kwargs)
[error] [client 10.10.10.12] File “/usr/lib/python2.7/dist-packages/oslo/config/cfg.py”, line 1731, in register_cli_opt
[error] [client 10.10.10.12] raise ArgsAlreadyParsedError(“cannot register CLI option”)
[error] [client 10.10.10.12] ArgsAlreadyParsedError: arguments already parsed: cannot register CLI option
PLease ask questions like these on the OpenStack developers mailing list
Hi , this is a very nice guide on how to configure the wsgi server for keystone. Would these configuration be valid for Ubuntu on Kilo ? Thanks.
Roughly, but Ubuntu puts things in different locations, names things slightly differently, and thus you need to make the Ubuntu specific changes. I don’t know what those are, but many people do…ask around.