With the release of Fedora 13, I have a new target OS for software. In order to deal with the vagaries of installs, I have come to the pattern of creating one VM per target OS, which I get to the starting point, and then clone that for any actual work.
Author Archives: Adam Young
Spaghetti
To the Tune of “We’re Ready” by Boston.
Lyrics by Me, idea by Cousin Chris,
Spaghetti galore
Its filling up the void inside you
We’ll make up some more
The garlic bread will have to tide you
You’ll be cleaning the sauce off your clothing tonight
And you know that your pants will be feeling too tight
It builds cellulite
SPAGHETTI!
Gluttony
Who needs a reason
This pasta is great, what is the season–
-ing I feel like I’ve never have tasted before
Don’t worry about what you dropped on the floor
I’m ready for more
I’m ready for more
SPAGHETTI!
With bolognaise
Spaghetti now
Spaghetti for me to chow down
Steady now
Our kitchen will need a hose down
You’ll be cleaning the sauce off your clothing tonight
And you know that your pants will be feeling too tight
It builds cellulite
SPAGHETTI!
JPackage, Maven and buildr
XML is not a pretty programming language. I suspect this fact is at the heart of many peoples frustration with maven as a build system for Java projects. I know that it was one of the reasons the Candlepin team decided to use buildr in stead of Maven as the build and package system.
Found ‘${BUILDROOT}’ in installed files; aborting
If you get the above mentioned error while building an RPM, here’s what it means. rpmbuild has executed the script:
usr/lib/rpm/check-buildroot
This looks through the files that are set to be installed in your rpm to see if any of them contain the embedded path used to build or install them in the rpmbuild process.   For example, I have my ~/.rpmmacros file set to up with the following entry:
%_topdir %(echo $HOME)/rpmbuild
which means that I build in /home/ayoung/rpmbuild. Underneath this directory, I see, amongst other things, the subdirectory BUIOLDROOT. The current wisdom says that an RPM should use BUILDROOT as the target for any installs. This is the set of files that get packaged up for the final RPM. The files here then get checked to see if they have this path embedded in them. For example, when building rubygem libxml, I see:
+ /usr/lib/rpm/check-buildroot
Binary file /home/ayoung/rpmbuild/BUILDROOT/rubygem-libxml-ruby-1.1.4-1.young.x86_64/usr/lib/ruby/gems/1.8/gems/libxml-ruby-1.1.4/lib/libxml_ruby.so matches
Found ‘/home/ayoung/rpmbuild/BUILDROOT/rubygem-libxml-ruby-1.1.4-1.young.x86_64’ in installed files; aborting
There is a simple work around. in ~/.rpmmacros, add the line:
%__arch_install_post  /usr/lib/rpm/check-rpaths  /usr/lib/rpm/check-buildroot
Which is, think, a typo, but it shuts off the check. However, I wouldn’t advise doingthis.
In the case of libxml, the paths are there as an artifact of the build process. The .so carried the full path to the directory in which it was build as a way to link to the source files, for debuggins. I can see the path by running:
objdump –dwarf libxml_ruby.so
I don’t have a final solution, because I need to work through the gem install process, but the end effect will be
- Remove an non-essential files that have this path in them
- rewrite the path in the remaining files to be the correct location in the final installed location in the rpm
Update: Since this is a ruby binary RPM, the correct thing to do is to move the gem install into the %build stage and then copy it into ${BUILDROOT}. It currently happens in the %install stage. RPM is wise enogh to do much magic in the BUILD stage, such as producing the debuginfo rpm and so on.
Debugging Techniques
Always look at the top error message in the spew that comes out from the compiler. This is usually where the problem is triggered. Junk after that is often spurious, especially if the first problem is a syntax error.
Use a revision control system like Git or Mercurial, and make a checkpoint of your code when ever you have made a significant addition. This way, you don’t get into the situation where you had it working, you broke it, and you can’t get it running again. Git has a little bit of a learning curve, but it rocks. If you are comfortable posting the code where other people can see, you can push your changes to github, and then if you have a catastrophic machine failure, you don’t lose your work.
The binary search is your friend. If you know that the error is somewhere between lines 10 and 20, Comment out lines 15-20 and see if you still have the problem. If so, comment out lines 13 and 14, and so on. A binary search means that you can search 1000 lines of code in 10 comparisons, and save yourself a metric shit-ton of time. A metric shit-ton is an industry term.
Step through your code in a debugger, line by line, to make sure you know what it is actually doing, not what you think it is doing. Very illuminating. If you don’t have access to a debugger, make liberal use of trace statements. In gcc I often use:
#define TRACE() printf (“%s:%s:%d\n”,__FILE__,__FUNCTION__,__LINE__)
There is something comparable in most languages.
Always work from success. When I start a new C program, I start from
#include
int main(){ printf (“Hello, World.”);}
And compile and run between each minor change.
Don’t be afraid to ask for help. A second set of eyes will often quickly see what you missed. Even more so, when you explain the code to someone else, you often solve the problem yourself. This is known as “verbal debugging.”
Candlepin: Metaphor for an Entitlement System
The planning meeting was held in Massachusetts. When brainstorming project names, someone mentioned that most New England of activities: Candlepin Bowling. Thus, the project is named Candlepin.
When describing a project, especially something fairly abstract like an entitlement system, you can clarify communication by using a strong metaphor for the system. So, to explain entitlements, I am going to use a bowling alley as my metaphor.
One way to think of an entitlement is this:
An entitlement is contract that you can hook up your computer system to my content stream.
But for our metaphor I’m going to say:
An entitlement is kinda like getting a lane a bowling alley.
To which you say:
Huh?
Think about it. When you go bowling, you pay money, but you don’t get a good, and you don’t get a service. What you get is access to a resource for a limited time. Say a small company wants to do a team building activity:
We’re going bowling!
This company has 18 employees. So, we go over to Westgate Lanes (A nod to the local Candlepin Alley of my childhood. Indulge me) and we walk to the main desk. We’ve self organizaed ourselves into six teams of three people each. We get our shoes, and our group gets three lanes assigned to us. We go, and each team pairs up with another team, the two teams select a lane from the three available, and they bowl. After each game, the teams re-shuffle the match ups, switch lanes and play another game. When each team has played against all the other teams, we return our shoes and go home.
Here is how the analogy maps to entitlement management.
The Data Center is the Bowling Alley.
The lanes are the physical machines that the virtual machines will run on.
The company is still the company paying the bills.
The front desk is the assignment system where you buy slices of time on the machines of the data center.
The three lanes that our company is assigned has a communication network due to the fact that we all need to coordinate our games. This is the VPN and VLAN setup that lets you specify a cluster of machines can all work together.
The pin setter and the ball retrieval and the scoring projector are analogous to the resources required to run the programs.
The score card is the backing store for the database instance that your applications talk to.
We can extend the metaphor to a larger world, too. Say we have a bowling league that spans multiple towns and multiple bowling alleys. This league is composed of teams. The league sets the schedule, the games are played at the various alleys through out the district. At the end of the season, the lead team from our league actually plays against the lead team from another league.
This reflects the hierarchical structure of resource management. You can see that the bowling alley doesn’t really care about leagues except as a way to generate traffic through the alleys. From the Alley’s perspective, the league is just another customer, paying for lane time. Perhaps in some cases, the league pays for the time, in others, the individual teams do. Authority to use a specific lane may have to be cleared not only through the clerk at the desk of the alley, but through the league official that is managing a tournament. Just like if my company buys a chunk of virtual machines on a cloud somewhere, and then delegates them for internal usage.
Note that the metaphor works for internal clouds as well. At the Really Big Company (RBC) campus, they take their bowling so seriously that they have a series of lanes installed into a building on their campus. Now, the scheduling and resource management have been brought in house, but the rest of the rules still apply.
Lexington Green
When summoned by the horseman’s cry
from breaking fast and tending barns
The Yeoman farmers trained response
was to secure their ready arms
The nineteenth dawn that April bore
revealed a revolutionary scene
Captain Parker’s men had met
to learn the news on Lexington Green
But red clad soldiers marched all night
From landing ships on Cambridge shore
And through Lexington they’d pass
En route to capture Concord’s Store
Lieutenant William Sutherland
Called to the men across the sward
Commanding them “Disperse Ye Rebels
Ye Villains, most unruly mob.”
Outnumbered, Captain Parker’s men
were ordered to disperse once more
But chaos and uncertainty
lay beneath the fog of war
Who fired first? the tales suggest
Perhaps a sniper off the scene
but British shot and bayonets
Killed nine men on Lexington Green
Lieutenant Colonel Francis Smith
Arriving with three companies
Ordered drumbeats for the march
To Concord’s Bridge and history
In Concord town the Minutemen
Had learned of Lexington’s Melee
On Punkatasset over-watched
As Redcoats made their first foray
Searching houses, barns and fields
for weapons cached and powder kegs
but long since moved and all they found
were milk and barely, ham and eggs
When searching soldiers caught the site
Of mounds fresh piled in the fields
And ready spades we turned to dig
And three great cannon were revealed
But as they searched, the Minutemen
From Acton, Bedford, and Westford
Joined the Lincoln men above
The Concord rivers northern shore
A Regiment of militiamen
descended Punkatasset Ridge
assaulted Captain Parson’s Force
assigned to guard the Northern Bridge
The Regulars formed to volley fire
As if for warfare in the town
A warning shot rang out and then
the musket balls were raining down
They fled their post and headed south
to form with Regiments complete
and leaving off their fruitless search
the British Colonel called retreat
The local men who took the bridge
Had learned to shoot when they were Boys
Some had marched to Montreal
To fight the French and Iroquois
At Meriam’s Corner, and Brooks Hill
The local muskets took their toll
and passing through the ambuscade
at Bloody Angle Thirty fell
At last they came back to the field
where shot had shattered dawn serene
And Parker had then his revenge
Upon the sward of Lexington Green
Klezmer Bebop
One of the tricks of Bebop is to extend the scale by one note so that there are 8 distinct tones ot be played in a measure, which, in 4/4 time, is divided into 8 eighth notes. For a major scale, you add the #5. When you do this, the down beat hits one of the main notes of the chord.
Guice gets it
I’m currently working on Candlepin, a Java web service application that uses Google Guice for dependency injection. I’ve long suspected that I would like Guice. I’m not a big fan of annotations, but I’ll admit that there are something that you can’t do in Java using other approaches.  Guice makes appropriate use of annotations to make dependency injection work in a type safe an intelligent manner.
I had to break a dependency in order to make a class testable. Here’s the old interface:
@Inject
public JavascriptEnforcer(
DateSource dateSource,
RulesCurator rulesCurator,
ProductServiceAdapter prodAdapter );
The problem with this is that RulesCurator is a database access class, which means I need a DB for my unit test.  What the JavascriptEnforcer specifically needs is the product of the database query.
Here’s a piece of code from the body of the constructor hat I also want to remove.
ScriptEngineManager mgr = new ScriptEngineManager();
jsEngine = mgr.getEngineByName("JavaScript");
This links us to one creation strategy. I want to use a different one in my Unit test.
My first step was to create a new constructor, and have all of the code I want to extract in the old constructor. This would probably have been sufficient for the unit tests, although it might risk a class not found for the DB stuff. Here’s the new constructor interface:
public JavascriptEnforcer(
DateSource dateSource,
Reader rulesReader,
ProductServiceAdapter prodAdapter,
ScriptEngine jsEngine) ;
Here’s where Guice shined. I need two custom compnents here, one which fulfills the rulesReader dependency, the other which fulfils the jEngine. The jsEngine one is easier, and I’ll show that. First, create a custom Provider. A Provider is a factory. For the jsEngine, that factory just looks like this:
package org.fedoraproject.candlepin.guice;
import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
import com.google.inject.Provider;
public class ScriptEngineProvider implements Provider<ScriptEngine> {
public ScriptEngine get() {
return new ScriptEngineManager()
.getEngineByName("JavaScript");
}
}
Which we then register with our custom module:
class DefaultConfig extends AbstractModule {
@Override
public void configure() {
...
bind(ScriptEngine.class)
.toProvider(ScriptEngineProvider.class);
}
}
Now comes the interesting one. We want a Reader. But a Reader is a very common class, and we don’t want to force everything that needs a Reader to use the same creation strategy. Here is where Guice uses Annoations.
public class RulesReaderProvider implements Provider<Reader> {
private RulesCurator rulesCurator;
@Inject
public RulesReaderProvider(RulesCurator rulesCurator) {
super();
this.rulesCurator = rulesCurator;
}
public Reader get() {
return new StringReader(rulesCurator.getRules().getRules());
}
}
Note how the Provider itself is a component. This allows us to use another component, the RulesCurator, without creating a direct dependency between the two classes. Still, this does not distinguish one reader from another. That happens with another bind call.
public void configure() {
...
bind(ScriptEngine.class)
.toProvider(ScriptEngineProvider.class);
bind(Reader.class)
.annotatedWith(Names.named("RulesReader"))
.toProvider(RulesReaderProvider.class);
}
Then, the inject Annotation for our Enforcer looks like this:
@Inject
public JavascriptEnforcer(
DateSource dateSource,
@Named("RulesReader") Reader rulesReader,
[...]
ScriptEngine jsEngine)
The key here is that the @Named matches between the two components.
Could Maven use a single directory for archives.
Maven is too important a part of too many projects for most Java developers to ignore. However, some of the decisions made in building with Maven are suspect, mostly the blind download of binary files from a remote repository. While Maven gets more and more Open Source clean, there are still issues, and the biggest is building Maven itself. Both Debian and Fedora have fairly old versions of Maven, in the range of 2.0.7 as of this writing. Considering that the GA is 2.2.0 and There is work on 3.0, we risk a pretty serious divide in the Open Source Java world of we don’t keep up with Maven, and get a clean way to build it.