Keystone and Eclipse PyDev

“Step through your code” –some of the best advice I ever got, from Code Complete.

I am a fan of Eclipse. Although I am conversant in VI and Fluent in emacs, I tend to write code in Eclipse. While the Python source code browsing is only mediocre when compared with the Java support, the integrated debugging is very powerful. Here is how I have set things up to work for Keystone.

Setting up the Workspace and Project

yum install eclipse-pydev

run devstack and get /opt/stack populated.

It will also populate the Python site-libs required by Keystone.

At a minimum, in /opt/stack/devstack/localrc have

ENABLED_SERVICES=g-api,g-reg,key,mysql,qpid

Glance is a very good test that Keystone is working, and it is simpler to work with than nova, since it is a smaller code base.

Fire up eclipse PyDev and change the workspace to /opt/stack

File->new->PyDev Project

Form Values:

  • Project Name: keystone
  • Grammar Version: 2.7
  • Interpreter default.
  • Add Project to the Python Path

To check that things are working, run the tests from inside Eclipse.  From the run menu (or the run icon on the toolbar) select “Run Configurations” or “Debug Configuration” and create a new one. Make the project “keystone” and for the “Main module” add /usr/bin/nosetest. Yes, there are other ways to do it, as PyDev knows about Unit tests, but this is an approach worth knowing, as it can keep your command line and PyDev know-how in sync.

If you see errors like Failure: ImportError (No module named memcache) … ERROR  That is due to the python packages not being installed.  Your best bet is to yum install them, but running the setup.py for Keystone will do it, too. Devstack should have gotten them by now, but development sometimes moves forward in unequal lurches.

 Debugging a test

To run just the ldap tests, create a debug configurating

nosetests test_backend_ldap

the command should still be /usr/bin/nosetests, and on the argument tab, add in test_backend_ldap

set a breakpoint in a test, for example test_authenticate in test_backend.py.

Run the debug command:

And you should get the error:

error: cannot switch to a different thread

On the Environment tab add in the environment variable to use standard threading, not the monkey patched version:

and rerun the debug, and breakpoints should now work.

One technique I have found to be very powerful is to run devstack, then kill the Keystone process, and then run Keystone in the Eclipse debugger instead.

./stack.sh
screen -x

Type ctrl-a 1 or whatever the screen is for Keystone for your particular run. CTRL-C to kill Keystone.

To debug the Keystone server in Eclipse, once again create a new Debug Configuration. The main module is ${workspace_loc:keystone/bin/keystone-all} which you can get by a right-click context menu from the pydev navigator on keystone/bin/keystone-all.

On the Arguments tab: –standard-threads. This deals with the Thread monkey-patching issue so you can get breakpoints.

3 thoughts on “Keystone and Eclipse PyDev

  1. Hi, this is a great post, I have tried to follow it but I have problems with breakpoints. I have tried following the steps from Openstack site and then I came across your post. I have also added ‘-standard-threads’ under the Arguments tab as advised but still no luck.

    All I did was download Eclipse IDE for Java EE Developers, installed PyDev for Eclipse and run devstack. I then killed the nova service via screen. I then go to nova-all > debug > Python run. I go to the horizon dashboard and trigger the request that would hit my breakpoint. I would be very thankful for any advise.

  2. I’m not certain exactly where nova does its monkey patching, but you probably need to find that spot and change the call such that it is not monkeypatching “Threads” as that is what breaks the breakpoint processing. Look through the mailing list and patch archives and you might find that Nova has a different way of disabling the monkey_patching of “Thread”

  3. Thanks, just thought that the standard-thread would work for other Openstack services. I have tried disabling the monkey_patching for nova but I must be missing a step somewhere. Thanks for your help.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.