Keystone and Cassandra: Parity with SQL

Look back at our Pushing Keystone over the Edge presentation from the OpenStack Summit. Many of the points we make are problems faced by any application trying to scale across multiple datacenters. Cassandra is a database designed to deal with this level of scale. So Cassandra may well be a better choice than MySQL or other RDBMS as a datastore to Keystone. What would it take to enable Cassandra support for Keystone?

Continue reading

Introduction to Ironic

“I can do any thing. I can’t do everything.”

The sheer number of projects and problem domains covered by OpenStack was overwhelming. I never learned several of the other projects under the big tent. One project that is getting relevant to my day job is Ironic, the bare metal provisioning service. Here are my notes from spelunking the code.

Continue reading

Using Puppet to setup PostgreSQL for Keystone on Fedora

Using Puppet to manage software configuration makes sense. Setting up PostgreSQL support for Keystone development and testing has been my excuse to learn it.

sudo yum install puppet
sudo puppet module install puppetlabs/postgresql

Then create a file /etc/puppet/site.pp

Put this line in it:

class { 'postgresql::server': }

postgresql::db{ 'keystone':
  user          => 'keystone',
  password      => 'keystone',
  grant         => 'all',
sudo  puppet apply --verbose /etc/puppet/site.pp

Confirm that postgresql is running:

systemctl status postgresql.service

Should get you

postgresql.service - PostgreSQL database server
	  Loaded: loaded (/usr/lib/systemd/system/postgresql.service; disabled)
	  Active: active (running) since Thu, 03 Jan 2013 13:26:46 -0500; 43min ago
	 Process: 17529 ExecStop=/usr/bin/pg_ctl stop -D ${PGDATA} -s -m fast (code=exited, status=0/SUCCESS)
	 Process: 17553 ExecStart=/usr/bin/pg_ctl start -D ${PGDATA} -s -o -p ${PGPORT} -w -t 300 (code=exited, status=0/SUCCESS)
	 Process: 17545 ExecStartPre=/usr/bin/postgresql-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)
	Main PID: 17556 (postgres)
	  CGroup: name=systemd:/system/postgresql.service
		  ├ 17556 /usr/bin/postgres -D /var/lib/pgsql/data -p 5432
		  ├ 17557 postgres: logger process   
		  ├ 17559 postgres: writer process   
		  ├ 17560 postgres: wal writer process   
		  ├ 17561 postgres: autovacuum launcher process   
		  â”” 17562 postgres: stats collector process

Test we can connect to the PostgreSQL command line tool:

psql -h localhost -U keystone keystone 
Password for user keystone: 
psql (9.1.7)
Type "help" for help.

keystone=> \d

To run the Keystone unit test test against the database, alter the file /opt/stack/keystone/tests/backend_sql.conf. Comment out the sqlite connection line, and uncomment the postgresql line.

#connection = sqlite://
#To Test MySQL:
#connection = mysql://root:keystone@localhost/keystone?charset=utf8
#To Test PostgreSQL:
connection = postgresql://keystone:keystone@localhost/keystone?client_encoding=utf8
idle_timeout = 200

Then you can run the unit tests with

./ -N test_sql_upgrade

If the tests fail (and they will), they will leave the database in an unusable state. You can drop the database and recreate with puppet:

sudo su postgres -c "dropdb keystone"
sudo  puppet apply --verbose /etc/puppet/site.pp

Speeding up SQLite based unit tests


If you write database driven applications, you probably have used SQLite at some point. Since it is a simple embedded database, it is a logical choice to use for unit tests that go to the database. However, SQLite performance on Ext4 (default Fedora File system) is lack-luster.
A cheap way to speed things up is to use a ramdisk as the backing store for the database.
Continue reading