Having worked on the FreeIPA GUI from inception through GA (well, RC3,  soon to be GA)  here’s what I’ve learned about the writing a web application using today’s technologies.

Javascript is your friend.

When you write a web application, you have no choice but to work with Javascript.  All but the most static of sites require at least some Javascript to function.  So, you have the choice of doing Javascript and your server side scripting/template language, or just Javascript.  You greatly simplify development if you stick to a single langue for user interaction. For people who think the language leaves much to be desired, I don’t fault you, but you have to work with it.

Javascript, the good parts

This book was instrumental in helping us set up decent coding standards and practices.  Javascript gives you plenty of rope to hang yourself.  This book gives you enough information to stay out of the noose.  Of course, it gets extra points for the obscure Princess Bride reference that you are not getting.

Use JSON for remoting to your server

JSON is the remoting mechanism we all wish we had back when XML took the world by storm.  It simplifies marshaling down to the basics:  string, array, dictionary.  It is light weight, requires little effort to parse, and is human readable.  JSON is valid Python and Javascript, which means that you have a very natural bridge between these two languages.  But I’d be very surprised at the any language that didn’t naturally convert a comparable subset of its syntax more cleanly to JSON than to any other general purpose marshaling tool.

Use CURL to talk to your server

Since the JSON marshaling is just another web technology, liberal usage of cURL or other HTTP based command line tools let you determine exactly what your RPC requests and responses should and will look like.  We use CURL to fetch sample data, to test new features, and bulk upload data.

Make it work without a server

Fetching JSON from the server can be mocked out by capturing static snapshots of the JSON.  Our JSON mechanism has everything go through one URL, with a method field that determines which RPC gets called.  We stubbed this out in development to fetch a comparable named static file from the test data directory, and then expanded the scheme a little bit to allow the unit tests to fetch different files on context.  But to do WebUI development, we do not need to talk back to a server, and that simplified things greatly.

Use Firebug

Can’t speak highly enough of this Firefox plugin.  Probably used mostly as a debugger.  However, its HTML introspection, Javascript console, listing of network calls and error reporting all make it possible to get work done.  We could not have done this project without it.

Java Script Lint

We have a simple configuration file for the  jsl command line utility. It tells us if we are straying from our coding standards.

Use QUnit for unit testing

QUnit is to Javascript as JUnit is to Java:  the helpful framework that lets you sleep easy at night, knowing that it is unlikely that you broke things with your last push.

Write the CLI first

All applications have a tendency to absorb business logic over time.  In FreeIPA, we had the advantage of a really strong CLI that was written first, and that is the preferred tool of administrators.  Thus, we could make the rule that it the WebUI can’t do anything that the CLI can’t do first, and we pretty much have stuck to it; the only exception is batch process, and even that can be done from the command line using curl.  It also gives us a path.  I see the WebUI as being for tasks that are done infrequently, that are complex enough that the user requires hand holding, and for bringing new people into the system.  With a good CLI, they have something that they can graduate to, to script, and to integrate with other systems.

Since IPA is a security product, I didn’t want to find out that the web UI had some flaw that turned it into a major security hole.  By using the same mechanism as the CLI, I feel confident that we have in no way weakened the security of the application.  We’ve managed to keep the window into the server as small and protected as possible.

Internationalize on the Server

FreeIPA is written in Python in a mod_wsgi application.  Since the browser already sends internationalization information to the web server, we let the server tell us what language to display, and what strings to use.  It does mean that we have to extract strings out of our ui, and explicitly put them into a server side component, but the end result is very easy to work with.

Use JQuery or a comparable toolkit

As I compare the features we have implemented with those provided by Dojo, I am struck by their similarity, and can’t help but wonder if we should have used it instead of JQuery.  I think that we made the right choice, though, because we really had to customize our application, and I think that for Dojo, we would have been fighting the framework, whereas with JQuery, it provided just enough framework to let us design the application the way our User Experience Team  designed it.  We did use JQuery UI, and probably benefited  somewhat from using ThemeRoller, but I’d have to say that by now we probably work around the JQuery UI code as much as we use it.  Not as a slight on JQuery UI, it is a great toolkit, but our look and feel is very customized and JQueryUI again provides a fixed way to do things like buttons.  With JQuery, we have to potential to use someone a table widget from somewhere else, or just build one our self.  It is the degree of brainshare that JQuery has that makes it so compelling.

Use The BBQ approach for navigation

You don’t want to submit forms, or have links make additional GET requests back to your server that will require the complete redrawing of the page.  The BBQ plugin makes a much cleaner interface by using the #url technology used to link to a different portion of the same page.  It takes some getting used to, and the entire state of your application can end up in your URL, but it handles Back button and history without doing additional HTTP requests.

Use Selenium for Functional Testing

This is an effort we are just starting.  Selenium allows you to record and playback interactions with the WebUI.  Knowing what I know now, I wish we had started with it sooner, as I think it would have given us an even better battery of pre-checkin tests, and allowed us to automate development of features that had tricky interactions with the server.

GIT

How did I ever survive without Git?  I’ll admit, there was a learning curve, but there are just so many benefits to a distributed revision control system that I just have to put a plug for it in here.  Really, this is not specific to WebUI development, but I think it is safe to say that all development benefits from the advance made in distributed version control.  If you are coding, use Git (I’m sure that  Mercurial or Bazaar are really good, too, just can’t speak from experience there.)

 

There is a lot happening out there.  It is entirley possible that there is some key technology we haven’t tripped over yet.  And, to be honest, I’m still not certain if it is necessary to follow all of the techniques in Javascript, The Good Parts; some of the other tools we use might mitigate the risks it enumerates, and we’ve come across at least on problem that would have been cleaner to solve using prototype inheritance, but couldn’t use it due to our object creation approach.

It also is certainly true that the more of a framework you build, the more you find yourself refactoring to account for things outside your initial design. Things that area easy to code as a one off are trickier to make happen if you have to keep from breaking a framework.  But the resulting code is certainly cleaner and more maintainable if you take the time to do it right.

There are many more lessons learned from this past half-year.  I’m glad I have had this chance to work along these lines.  I think it has proven a fast, light, and robust approach.

Special thanks to Endi Dewata, both for being such a great partner on the project and for proofreading this article.