Facelets Taglibs

These are my notes on how to reverse engineer what tags are doing in a JSF application. In this case, I am trying to figure out what are the classes behind the Configuration tags in the RHQ.  I am trying to figure out what is being done by the tag

onc:config

This tag is activated with the following value at the top of the page:

xmlns:onc=”http://jboss.org/on/component”

To Figure out what this tag means, I look in WEB-INF/web.xml.  The web.xml value

facelets.LIBRARIES

Lets me know where the file is that defines the acceptable tags I can add to an xhtml page for this component.

/WEB-INF/tags/on.component.taglib.xml

This taglib defines the values

tag-name config
component-type org.jboss.on.Config
renderer-type org.jboss.on.Config

Note that these are JSF component names, and not Java class names.  To resolve these to Java classes, we need to find the mappings.  The mapping files are defined in web.xml under the entry:

javax.faces.CONFIG_FILES

In particular, I found what I wanted in

/WEB-INF/jsf-components/configuration-components.xml,

The values I care about are:

component-type org.jboss.on.Config
component-class org.rhq.core.gui.configuration.ConfigUIComponent

and the renderer for Config and ConfigurationSet components

component-family rhq
renderer-type org.jboss.on.Config
renderer-class org.rhq.core.gui.configuration.ConfigRenderer

This render extends javax.faces.render.Renderer.  This is a Flyweight that parses and generates HTML.  It has two main methods: decode an encode. decode parses the request that comes in, encode injects html into the response that goes out.

Decode appears to be called only on a post.  Encode seems to be called even on a get.

Refactoring: Introduce Builder

As I move more and more towards Immutable objects, I find myself extracting the building logic out from the usage of an object on a regular basis.  The basic format of this pattern is

mutable object =>  immutable object + builder

There are two aspects to using an object: creating it, and reading its state. This pattern splits those two aspects up into different object, providing a thread safe and scalable approach to sharing objects without restoring to locking. Modifications of the object are reflected by creating a new object, and then swapping out the immutable

There are two ways to go about making the builder. If the object is created as part of user interface or networking code, it make sense to use a generic map object, and in the build method, confirm that all required properties are there.
so code like:

if (inputStringKey.equals(‘x’)) obj.setX(inputStringValue);

becomes

builder.set(inlutStringKey, inputStringValue)
obj = builder.build();

If, on the other hand, the original object tends to be modified by other code, it makes sense to split the object so that the setters are on the builder object.  Thus the above code becomes:

builder.setX( x);
obj = builder.build();

The immutable object shouldn’t have getters, you should use public properties for them, make those properties public and immutable themselves. In Java, this means tagging them as final, in C++ make them const.

It the immutable object is fetched from a data store, there are two approaches to updating that datastore.  The simplest and least flexible is to have the builder update the store as part of the build function.  This mixes functionality, and means that you can only update a single object per transaction, but it does increase consistency of viewing the object across the system.  The more common approach is to have a deliberate call to store the immutable object in the datastore.  Often, this is a transactional store, with multiple immutable objects sent in bulk as an atomic commit.

Here are the steps to perform this refactoring:

1. Append the word builder to the class name

2. Create a new internal class for holding the immutable state of the the object and give it the name of the original object.  Give  the builder and instance member field of the internal type.  The internal class should have a no-args constructor.

3.  Push all fields down to the new class by  moving the fields .

4. Add a method called build that returns the internal type.

5.  Replace all calls of the form getProperty()  with

x=  builder.build() ;

x.property;

It is important to use the local variable, and to share it amongst all the places in the code that reference the local object.  If you don’t you will end up with one instance per reference, probably not what you want.

6.  Remove the getProperty methods on the builder.

7.  Give the internal class a constructor that takes all of its fields as parameters.  It should throw a standard exception if one of the parameters is null.

8.  Change the build method so that it returns a new instance, that gets all of its fields set via the new constructor.  This instance should just use the fields in the internal instance of the (soon-to-be) immutable inner object.  You will have to provide dummy initializers for the newly immutable fields.  The build method should thrown an exception if any of the values are missing or invalid.  It is usually required that you return all errors, not just the first one.  Thus this exception requires a collection of other errors.  These can be exceptions, or some other custom type, depending on the needs of your application.

9.   For each setProperty method on the builder, provide a copy of the field in the buidler object.  Make the corresponding field in the inner object immutable, and change the build method to use the field in the builder instead.

10. When you have finished step 9, you should have an immutable inner object. Provide an appropriate copy constructor.

11. Remove the no-args constructor from the immutable object.

12. remove the default values for the immutable fields.

You can now create a new builder object based on a map.  The keys of the map should be the names of the fields of the builder.  The Build method pulls the elements out of the map and uses them as the parameters for the constructor. This approach showcases one of the great limitations of Java introspection:  Parameter names are lost after compilation.  We only have access to the types and the order of the parameter names.  Thus, maintaining this map would be error prone.

A more performant approach is to extend the builder object above with a setProperty(String, String) method that sets the values of the builder fields directly.

Any Java object can act as a map if you use introspection.  Thus, you could do what the Bean API does and munge the key into the form “setX” by changeing the case on the first letter of the key name, and then calling

this.getClass().getMethod()

You could also use  property introspection like this:

this.getClass().getProperty(key)

Since you are using introspection, even though you are inside the class that should have access to private members, Java treats you as an outsider.  You can either drop permissions on the fields at compile time by making them public, or do so at runtime using one of various hacks.

This is one case where it makes sense to make the member fields public.  There is no information hiding going on here.  There may actually be some client codethat is better off calliing

builder.property = x

Than

builder.setProperty(x)

In C++, we have fewer choices, as there is no way either at run time nor at compile time to provide an iteration through the fields of a class.  The best you can do is to create a map of functors.  The keys of the map are again the fields of the builder, the values are functions which set the fields.  You end up with a setProperty function on the builder that looks like:

void setProperty(String& key, Strin& value){

propsetmap[key](value);

}

although with logic to handle erroneous keys.

A builder is a short lived, single threaded, mutable object.  It is up to the calling code to provide enough data to populate the builder.  The builder pattern works nicely with  inversion of control frameworks.  Code that uses an object should not call the builder directly, but rather fetch the object from the framework, where as code else where builds the object and adds it to the container.

If your code has interspersed sets and gets of properties, it is going to be really difficult to introduce the builder.  Chances are you are going to want to do additional refactorings to separate the concerns in your code.

BProc 5.0 is in Git

Penguin Computing has graciously provided their version of BProc to the
project. Special Thanks goes to John Hawkes, the Director or Software
Engineering and Andreas Junge VP of Engineering at Penguin for making this
happen. In the next couple of weeks, we’ll work at getting some packages
made from the code and posted on the files portion of the Sourceforge
site.

The code is targeted at a RHEL 5.4 Kernel, due to development priorities
at Penguin. Expect to see a set of RPMs for the Kernel and the userland
codes for the unified process space code.

For now, we are going to focus on the kernel side of development. Our
goal is to get the necessary changes into the upstream Kernel. We will
begin work shortly on a version of BProc that targets the current
development Linux Kernel.

As such, the beoboot and related code is going to be left untouched in the
CVS tree.

The Git Repository is live here:

http://bproc.git.sourceforge.net/git/gitweb-index.cgi

Expect it to go through some changes in the future as we get it cleaned up
and ready for distribution.

I still have an emergency hold on all email traffic to this list due to
the high percentage of SPAM. I will pass through all message that come
from real human beings, and that are regarding the BProc code base.

The code is very different from the last released version of bproc 4.0. I
will post more details in the future about how to get a Kernel built, why
we made the decisions we did, and what the plan is for the future. I am
very excited about this project, and hope to get some community momentum.

Thanks Again to everyone who made this happen.

Highlander Syndrome in Package Management

Somewhere between systems work and application development lies the realm of package management. There are two main schools of thought in package management: inclusive or exclusive. If you are inclusive, you want everything inside a package management system, and everything should be inside one package management system. If you are exclusive, you want the system to provide little more than an operational environment, and you will manage your own applications thank-you-very-much.

One problem with the inclusive approach is, in the attempt to clean up old versions, you often end up with The Highlander Syndrome. There can be only one version of a library or binary installed on your system. The Exclusive approach is more end application focused. I may need to run a different version of Python than is provided by the system, and I don’t want to be locked in to using only the version installed system wide. In fact, I may require several different versions, and each of these require their own approach.

CPAN, Pear, and Maven have provide language specific approaches level APIs to resolving dependencies at the per application level. Maven is particualrly good at providing multiple versions of the API: I errs so far this way that often the same Jar file will exist multiple times in the maven repository, but under different paths.

There should be middle ground for the end user between all or nothing in package managemnt. As a system administrator, I don’t want users running “just any” software on their system, but as an end user I don’t want to be locked in to a specific version of a binary.

If the role of application maintainer is split from the role of system administrator, than the people that fill those two roles may have reason to use a different approach to package management. Since the app developer can’t be trusted, the sys admin doesn’t provide root access. With no root access, the app developer can’t deploy an RPM/Deb/MSI. The app developer doesn’t want the system administrator updating the packages that the app depends on just because there is a new bugfix/feature pack. So, the app developer doesn’t use the libraries provided by the distribution, but instead provides a limited set. Essentially, the system has two administrators, two sets of policy, and two mechanisms for applying that policy.

Each scripting language has its own package management system, but the binary languages tend to use the package management system provide by the operating system.  Most Scripting language programmers prefer to work inside their language of choice, so the Perl system is written in perl, the emacs system is written in LISP, the Python one in Python and so on. The Wikipedia article goes into depth on the subject, so I’ll refrain from rewritintg that here.

A Package management system is really a tuple.  The variables of that system are:

  • The binary format of the package
  • The database used to track the state of the system
  • The mechanism used to fetch packages
  • The conventions for file placement

There is some redundancy in this list.  A file in the package my also be considered a capability, as is the “good name” of the package.  A package contain empty sets for some of the items in this list. For example, an administrative package may only specify the code to be executed during install, but may not place any files on a file system. At the other extreme, a package may provide a set of files with no executable code to be run during the install process.

Of these items, it is the conventions that really prevent interoperability. This should come as no surprise:  It is always easier to write an adapter on top of an explicit interface than an implicit one. The Linux Standards Base helps, as does the standards guidelines posted by Debian, Red Hat, and other distribution providers. However, if you look at the amount of traffic on the mailing lists regarding “file X is in the wrong place for its type” you can understand why automating a cross package install is tricky.  Meta package management schemes attempt to mitigate the problem, but they can really only deal with thing that are in the right place.

Take the placement of 64 bit binaries.  For library files, Red Hat has provided a dual system:  put 32 bit libriares under /usr/lib and 64 bit libraries under /usr/lib64.  Debian puts them all into the same directory, and uses naming to keep them apart. In neither case, however, did they provide a place to make 32 and 64 bit binaries co-exist. How much easier would migration have been if we had /usr/bin32 and /usr/bin64, with a symlink from either into /usr/bin?

Thus we see a couple of the dimensions of the problem.  An application should have a good name:  web server, mail client,  and so on.  A system should support multiple things which provide this capability, a reasonable default, and customizability for more advanced users.The system should provide protection against applications with known security holes, but provide for the possibility of multiple implementations released at different points in time.

An interesting take on package management comes from OSGi.  It is a language specific package management approach, specifically for Java.  It takes advantage of portions of the the Java language to allow the deployment of multiple versions of the same package inside a since Process. When I mentioned this to some old time Linux sys admins, they blanched.  OSGi does not specify how to fetch the packages, much like RPM without YUM or DPKG  with out APT.  OSGi packages are installed into the application.  As such, they are much more like shared libraries, with specific code sections run on module load and unload.  Different OSGi container provide different sets of rules, but basically the packages must exist inside of a subset of directories in order to be available for activation.  I have heard an interesting idea that the JPackage/RPM approach and OSGi should ideally merge in the future.  To install a Jar into your OSGi container, you would have to install an RPM.

One additional issue on the Java/RPM front is Maven.  Both Maven and RPM want to run the entire build process from start to finish.  Both have the concept of a local Database of packages to resolve dependencies.  For long term Java/RPM peaceful coexistence, RPM is going to have to treat Maven as a first class citizen, the way that it does make.  Maven should provide a means to generate a spec file that has the absolute minimum in it to track dependencies, and to kick off an RPM build of the Maven artifacts.

Hacking the Palm Pre from Fedora

My work machine is a Fedora 11 (F11) X86_64 system. The palm development SDK is distributed as a series of .deb packages, specifically targeted at an 32 bit Ubuntu 8 system. While I have the advantage of having a 32bit Debian system at home, so I was able to run through the setup process for development, ideally I would be able to attach to and control the Pre from my work machine.

The first step is to download the .deb files onto the F11 machine. I actually only needed the novacom deb, which in my case is novacom_1.0.38_i386.deb. Deb files are accessable using ar (happy talk like a pirate day!).

In a new and empty directory, run
ar -vxf ~/novacom_1.0.38_i386.deb

And you will see the three contained files:

control.tar.gz data.tar.gz debian-binary

Extract the data file using tar

tar -zxf data.tar.g

This will add a usr directory with the binaries in

usr/local/bin/novacom{d}

Novacom is a two piece effort: a daemon an a client. First make sure you can run the daemon.

First, lets see the file type:

file novacomd

novacomd: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8, stripped

Let’s see what libraries it requires:

ldd novacomd
linux-gate.so.1 => (0x008f1000)
libpthread.so.0 => /lib/libpthread.so.0 (0x0078a000)
libusb-0.1.so.4 => not found
libc.so.6 => /lib/libc.so.6 (0x00110000)
/lib/ld-linux.so.2 (0x006ae000)

Note that the USB library is missing. I have it installed on my system, but only the 64 bit version. To get the 32 bit version, first, figure out what the 32 bit vversion is named.

yum search libusb

libusb.i586

And install

sudo yum install libusb.i586

F11 and the RHEL based approach for running 32bit apps on 64 makes this fairly easy to do. Unlike Debian based system which pretty much require you building a chroot if you are going to run a significant amount of 32 bit binaries, Red Hat based systems put 64 bit libraries into /usr/lib64 and 32 bit libraries int /usr/lib, so they don’t conflict. Now lddd shows we have everything:

ldd novacomd
linux-gate.so.1 => (0x00262000)
libpthread.so.0 => /lib/libpthread.so.0 (0x008a5000)
libusb-0.1.so.4 => /usr/lib/libusb-0.1.so.4 (0x00770000)
libc.so.6 => /lib/libc.so.6 (0x00263000)
/lib/ld-linux.so.2 (0x006ae00)

And we can now run this. Since it is talking straight to hardware, it insists on running as root:

./novacomd
[2009/9/22 11:40:48] novacomd version novacomd-62 starting…
[2009/9/22 11:40:48] novacomctl socket ready to accept
[2009/9/22 11:40:48] need to run as super user to access usb

so:

sudo ./novacomd
[2009/9/22 11:41:11] novacomd version novacomd-62 starting…
[2009/9/22 11:41:11] novacomctl socket ready to accept
[2009/9/22 11:41:11] sending rst
[2009/9/22 11:41:11] sending rst
[2009/9/22 11:41:11] sending rst
[2009/9/22 11:41:11] going online
[2009/9/22 11:41:11] novacom_register_device:188: dev ‘e851588c804e8caa722490a0314ce9782dd4d9a4’ via usb type castle-linux

No we turn our attention to the client piece.

file novacom
novacom: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.8, stripped
[ayoung@ayoung novacom]$ ldd novacom
linux-gate.so.1 =>  (0x00173000)
libc.so.6 => /lib/libc.so.6 (0x006d2000)
/lib/ld-linux.so.2 (0x006ae000)

So we are ready to run.  There is no novaterm in this deb.  Instead, you run novacom in terminal mode.  A little noted line in that I will make big here is:

./novacom $* -t open tty://0

Fun with Unicode

I’m on IRC with a Coworker in Israel. While my Hebrew is pretty poor, I have attempted to communicate short sayings with him like:

בוקר טוב

And of course

שלם

However, his response are getting garbled. I blame his IRC program (mIRC) since it is running Windows, but I’d like to confirm that this is our problem. Eventually I’ll sniff the packets on the wire, but what should I look for?

I took the word שלם and dumped it into open office, and then saved the file as straight text. A hex dump of it looks like this:

hexdump -C ~/Documents/shalom.txt
00000000 ef bb bf d7 a9 d7 9c d7 9d 0a |……….|

Just the letter ש looks like this

hexdump -C ~/Documents/shin.txt
00000000 ef bb bf d7 a9 0a |……|

A File with just the letter ‘a’ in it looks like this:

hexdump -C ~/Documents/a.txt
00000000 ef bb bf 61 0a |…a.|

To check the file type of these:

file ~/Documents/a.txt
/home/ayoung/Documents/a.txt: UTF-8 Unicode (with BOM) text

The value 61 is the ascii value for ‘a’, so we know where to look. Note that this file is one byte shorter than the other two. This aligns with UTF-8. The letter ‘a’ maps directly to the ascii value for ‘a’, where as the Hebrew characters us the ascii expansion scheme. I’ll translate this raw binary back to the Unicode value:

If shin dumps to d7 a9

D7a9=1101 0111 1010 1001

110 Means it falls into the range of U+0080-U+07FF And is translated from the Unicode value by the pattern 110yyyxx 10xxxxxx

So yyy= 101 and xx xxxxxx = 11 101001

Note the yyy. The max this portion can be is 7, so you left pad it with 0 to get 0000 0101

And the rest becomes 1110 1001. This is the hex value 0 5 E 9, Which is in the range U+0590 to U+05FF as expected.

Technology Overload

Here is the current list of technologies on my horizon, not all of which I am completely clueless:

  • JBossAS5
  • OSGi
  • Qpid/AMQP
  • JSF
  • Facelets
  • Seam
  • Git
  • Jopr/RHQ
  • Maven
  • EJB3/Hibernate
  • Portlets
  • Struts and Tiles…did this before but it has been a few years.
  • Jepp
  • JNA/JNAerator

Of course, there are also my completely unrelated side projects cpp-resolver and the bproc work, both of which go in completely differnt directions.    My brain hurts…but it is a good kind of hurt.  And yet, strangely, in my brain these all fit together into a single consistent whole.

cpp-resolver

I’ve finally created my own open source project.  I’ve taken the cpp resolver code and posted it on source forge.  Let the bug reports commence!

http://sourceforge.net/projects/cpp-resolver/

I’ll probably copy the articles describing it over there at some point as well.

Dogfooding

When I was contracting at Sun, someone sent out a memo using the phrase “eat our own dog food.”  Scott McNeally sent out a response to the effect that Sun did not make dog food, “We Fly our own airplanes.”

In the software world, the phrase has become so used that it has been verbed; If you work at a software company that uses its own products in house,  you are Dogfooding.

Since so much of my professional work has been on Red Hat Enterprise, I’ve tended to run Debian based systems for my desktop, to keep abreast of what is happening in both worlds.  At Red Hat, I’ve finally had reason to run Fedora as my desktop machine, and have been enjoying the experience.  The vast majority of the Software I run now is distributed by the company I work for, and is Open Source.  It really is an amazing feeling. I am now running Fedora 11 not only on my work laptop, but my wife gave me permission to blow away the Windows install on her eeepc and install it there as well.  Fedora makes a great Netbook OS.

However, one tenant of Software Development is to develop on the platform on which you are going to ship.  For Red Hat, that is RHEL5, and so I need access to a RHEL5 install, and in fact need both 32 and 64 bit, since the world has not completely moved to 64 bit yet.  I’ve used virtual machines in the past, but always from VMware.  Now I am running QEMU/KVM on my Laptop.  Like most things Linux-y, the command prompt method of controlling the virtual machine subsystem is a first class citizen:  I don’t need a visual console to start up a VM.  I realize this is old hat to some people, but It is new to me, and I am enjoying it.

That is the wonderful thing about the Open Source development model:  you very quickly take ownership of the software that is essential to you.  Whenever a user becomes a contributor, that software is no longer just something out there.  It has become personal.

Anyways. As I Fly the Red Hat virtualization airplane, I’ve learned a few things.  The GUI, Virtual Machine Manger, is great for getting over the learning curve.  The command line is virsh.  These tools are a  part of the libvirt project.  There is a command to start Qemu based vms directly, but his seems to bypass the libvirt infrastructure.  Rnning qemu-kvm allowed me to start a VM saved in /var/lib/libvirt/images, but was not able to talk to the kvm subsystem.  One thing that threw me was that connecting to the virtual shell and running the list command did not show my virtual machine;  by default, that only shows running virtual machines, and you need to add the –all option to see non-running VMs..important if you want to only run them occasionally as I do.  To connect to the system, run:

sudo virsh -c qemu:///system

There is also another URL qemu:///session that I am not yet familiar with.

Working with VMware, I was comfortable with the split of the info into a vmdk and vmx files for binary data and configuration information.  IN KVM/Qemu/libvirt land, the equivalent of the VMDK is  a boot image.  This seems right to me, in keeping with the fearless Unix way of not inventing a new technology if an existing one makes sense.  The Analogue of the vmx file is in /etc/libvirt/qemu/.

One thing I would like to get set up is bridged networking for my VMs to the corporate lan.  The base install takes the conservative view that the network should be confined to the local system.  I’ve seen some write ups to getting TAP interfaces set up to allow your virtual NICs to get packets to and from the physical NICs, but haven’t done that yet.  The configuration for the host local network can be viewed from the virtual machine manager, and it shows the range of DHCP address given out to the hosts.  It contains a wizard for adding new networks, but I am not sure if the VMware paradigm of a bridged network maps cleanly to the Linux view…I suspect not.  I see under advanced options when creating the VM that I can set the Network to bridged, but it doesn’t seem to find my DHCP server to PXE boot.  As an aside, I’d like to understand how this operates in an Ipv6 environment, as Uncle Sam is now dictating IPv6 for all new Software purchases.    So many things to learn!

Importing JBoss Application Server 5 code into Eclipse Take 2

I wasn’t super pleased with my results yesterday, so I kept plugging away.  Better results today.

  • Checked out sources.  This time, I made a backup copy.
  • ran mvn install in the jbossas directory
  • ran mvn eclipse:eclipse in the jbossas directory
  • imported the projects into eclipse.  Saw about 100 errors.
  • I had to exclude  a couple of things from the source paths by hand.

In two cases, I had errors that showed different versions of standard Java classes regarding CORBA and JAAS.  To get these to build correctly, I went to the Build Path popup and selected the Order and Export tab.  In both cases, the JRE directory was the last one listed.  I moved it to the top of the list.  I suspect that some scripting is in order to reorder these in all of the .classpath files.  However, once again, I have a project with no errors.

I notice that most of the projects refer to the jar files built from previous projects as opposed to depending on the projects themselves.  Everything has been installed in M2_REPO, and is fetched from there as well.  This is astep above installing in /thirdparty.