Base64 Encoding

I needed to initialize a file as part of a unit test. The file conatined binary data. My first thought was UUEncoding, which quickly got upgraded to Base64 encoding. To get it into a format that would then work in Java or C code I used the following line of BASH.

base64 < rui.pfx | sed -e ‘s!\\!\\\\!g’ -e ‘s!\”!\\\”!g’ -e ‘s!$!\\n\”!’ -e ‘s!^!+\”!’

It was for testing a decrypter from using a key generated by SSL.

It is important to double the slashes already existing in the file before you add your own.

Note, I added this as a way to remeber how to put binary Data into code, not as a recommendation for how to mock up for unit test.  The correct solution was to use:

InputStream inputStream = this.getClass().getResourceAsStream(
file.getName());

Cool it with the Ayn Rand comparisons

No, we are not entering the time of Atlas Shrugged.  No, Obama’s plan to deal with the economic melt down is not the same thing as the various acts passed during Atlas Shrugged.  No, we are no about to enter a world dominated by socialism.

If you feel the need to drop out of society and move to a valley in Colorado, please feel free to do so.  I will be understandably jealous.

Continue reading

contracting

After  the dot com bubble burst, I worked as an independent contractor in software for a few years. This is extremely common in the field. Some companies will hire independents outright, where as many others make you go through contracting firms to mitigate risk. (There was a lawsuit regarding compensation and stock options at Microsoft that caused this, worth googling and reading about). I set up an LLC that I used for a couple of the projects. I want to record a few of the lessons that I learned.

I leanred that when figuring negotiating price, I needed to  remember to factor in Health Insurance, which is much higher for a single payer than for a member of a large organization. I had to pay both sides of payroll tax (roughly 15%) but you then could deduct half of that as a business expense. The math gets a little tricky, but I would say that I was paying about 5% more in taxes than I would have as a salaried employee. There are tax benefits: you can expense much more than you can as a full time employee.  Many of these additional costs to me would have been norne by the comapny that issued me a W2 if I had been a Full Time Employee.

Regardless of the setup between me and the company, I  treated it just like I was  a company, and separated out the money I earned as an LLC from the money I paid to myself as an employee.  Except for the times I had to work through a body shop that issued  a W2, I had to deal with paying  both state and federal taxes. I didn’t pay income tax directly out of your monthly pay, so I held on to the tax money for an average of 1 month an a half longer, which I could invest. There is a final accounting done at the end of the year.  There are books on this. I had a good accountant.

One thing I learned to watch is the length of the contracts. Once a contract ends, if I didn’t have another lined up, I went  without work, but was still paying the mortgage. It was up to me to fill in my own schedule.  I spent a good deal of the time working on my own software, and a lot of time cleaning the apartment.

Initializing Dependencies in c++

Since the approach I described for initializing dependencies in C++ does not work in Visual Studio, and probably other C++ implementations as well, Shankar Unni suggested a more portable approach.  Granted it uses a Macro, but I think this one is justified.

#define DEPENDENCY_INITIALIZATION \
static bool initialization_function(); \
static bool is_module_initialized = initialization_function();\
static bool initialization_function()

This is composed of:

  1. A forward declaration of the initialization function.
  2. A static variable that will be initialized at load time (prior to calling main)
  3. The start of the definition for the initialization function.

This is mean to be called like this:

DEPENDENCY_INITIALIZATION
{
supply<A,1>::configure(destroy);
supply<B,1>::configure();
return true;
}

Where A and B are the classes to be registered for later initialization.

National Security

For years, our distance from other countries provided us with security.  We were an Island Nation.  Once our manifest destiny was completed, there was no significant threat left in our hemisphere, and we were too far from the nations of the old world for them to threaten us.  Even the attack on Pearl Harbor was far away:  Hawaii was not a state, just an island protectorate in the middle of the Pacific.  Since WWII, we have grown more and more used to the concept of global threat.  We were in a staring contest with Russia, half a world away.  The threat of ballistic missiles carrying nuclear warheads made the end of the world seem plausible, but it was abstract.  Mutually Assured Destruction was the word of the day:  If they try to take us out, they will take themselves out, and they are not going to do that.  We fought proxy wars and ran tank maneuvers at NTC and in Germany.

Continue reading

Playing Football after Graduation

West Point has started letting Grads that are recruited by Pro sports teams go be recruiters, and play pro ball.  This has caused much heated discussion, especially amongst my fellow alumni.  I initially wrote this as a response to an email discussion, but decided to sit on it for a while and ruminate.  This is really more of a collection of my thoughts at the time than a coherent essay.

It seems particularly stark in contrast to the classmates that are headed to Iraq. Would people feel as strongly about the matter if we were not sending people into harms way?

Also, is admissions the only reason that success in Army Sports, Football in particular, is important?

What about the rest of us that “did our time” and are now sitting out this conflict? Yeah, we played Army for our total of 8 years Active and Reserve. For many, it was a great experience that has lead to success later in life. Are we any less guilty of avoiding our Duty? How about the guy that “only” goes Signal Corps as opposed to going in to a combat Arms, or that goes Artillery to avoid Infantry? There always is a way that someone who goes less than the full HUAH can be said to be shirking.

Is it really doing our Country any good to be sending our Grads over to Iraq? I think most people would say that it is not cut and dried: some yes, some know, many I don’t knows. So why is it so important that these kids go to Iraq instead of playing Football? Is it really just a question of paying your dues?

Maybe the best way this kid can server his country is by being a kick-ass footballer, getting the name of West Point up in front of the country, and help to raise the awareness of civilians that we even have service academies. Maybe He’ll have a two year career, get cut, and end up back on Active Duty. Maybe he’ll be such a kick ass recruiter that he’ll fill the Army’s quota single handedly. Or maybe the Army wasted money in training him, and it was a mistake to send him to the NFL.

Is keeping a bunch of barely post adolescents isolated from the rest of civilization for four years the best way to prepare them for the officer corps? Does the Army get as much bang for it’s buck vie the Service Academies as it does Via ROTC? Sure West Point has produced it’s share of generals, but would those same people be great generals if they had gone ROTC? Would the opportunities in the Army be different if the Largest block of officers in the Army didn’t come from the same school? I have no idea if what we are doing makes sense or not. I know I gained a lot and gave up a lot by going to West Point. I’ll never know what I would have gained if I had gone another route.

Letting Cadets go professional  will allow the coaches to recruit players who, as Seniors in High School think they have a chance to play pro ball. Most College Football players want to go pro, but few are chosen. I suspect that a good portion of these players would make decent soldiers. So Army Football gets a better team, and good but the less-than-great ball players now have the chance of a career as an Army officer.

Many kids enter West Point as an option, and only develop the drive to be Army officers while being at West Point. I suspect that this is one of the most important roles that West Point plays in support of our Officer corps.

Interview Question for Distributed Computing

This is an updated version of my interview question,  made non-bproc like.

Using only the following API:

  • printf(…)
  • int get_node_count() // number of compute nodes attached to the head node.  0 means no other nodes
  • int get_current_node()// 0 for the head node, 1-n for the compute nodes.
  • int remote_fork(int node) // like fork, but returns an fd to the child/parent process
  • void send_long_sync(int fd, long value)//send and wait, blocks until receipt
  • long recv_long_sync(int fd)//block until value is available
  • long gettime()

Calculate the average clock skew on the cluster.    Return 0 on success, and -1 on any failures.

Assume that all nodes are up and running.  This is a c subset of c++.  Each of these functions throw an exception upon failure.  rfork has the same semantics as fork:  when it returns, there are two copies of the program running, just on separate machines.  The  next line of code to execute will be the line immediately after the fork on both machines.  However, the returned value is not a process ID, and the parent process  does not need to wait for the remote processs to finish:  the child process is automatically reaped by init.

Working around an NFS hang

While Sun might have wanted us to believe “The Network is The Computer” the truth is that we often only need the network for access to stuff a brief points in time, and can get away with doing our real work on our local machines.  One of the systems at work went down today and is still currently avaiable.  This machine was exported several directories which I and some of my co-workers have NFS mounted.  When it failed, basica utilities on my machine were no longer functioning.  I tried several simple commands: like clear, ls, and which.

The problem was that my PATH environment variable had the remote machine before local machines.  This was required by our build process so that things like make would resolve to the correct version, consistent  across all machines.  When you type a command in at the command line, the shell resolves the command by trying each directory specified by the PATH variable.  In this case, the very first directory was not only failing, but hanging.

One trick that helped to confirm the problem was using echo * on a directory to see what files were there.  Since the command echo is built in to the shell, it does not cause any path navigation.  To view /usr/bin you execute echo /usr/bin/*.

To work around the problem export PATH=/bin:/usr/bin:$PATH. With that, basic utilities are once again resolved on the local machine.  Once the NFS server comes back up, exit out of you shell and create a new one, or re-export your path once again.

Red Day, Green Day

I have started tracking stocks on Yahoo. Not that I am investing, but I find that looking a certain set of stocks helps explain the news. Specifically, I have a list that has two main types of stocks: Tech and Oil. Here is the list:

VMW 10:29AM ET 46.76 Up 0.38 Up 0.82% 251,554 17.93B Chart, Messages, Key Stats, More
YHOO 10:34AM ET 28.33 Up 0.24 Up 0.85% 4,203,632 37.88B Chart, Messages, Key Stats, More
AAPL 10:34AM ET 143.57 Up 3.32 Up 2.37% 8,182,315 126.18B Chart, Messages, Key Stats, More
JAVA 10:34AM ET 16.06 Up 0.15 Up 0.94% 1,501,868 14.46B Chart, Messages, Key Stats, More
DELL 10:34AM ET 19.84 Up 0.37 Up 1.91% 3,191,453 43.34B Chart, Messages, Key Stats, More
EMC 10:29AM ET 14.40 Up 0.07 Up 0.52% 4,446,405 30.26B Chart, Messages, Key Stats, More
IBM 10:29AM ET 114.99 Down 0.53 Down 0.46% 1,447,515 159.18B Chart, Messages, Key Stats, More
ORCL 10:34AM ET 19.79 Up 0.36 Up 1.85% 11,308,559 101.65B Chart, Messages, Key Stats, More
MSFT 10:34AM ET 28.30 Up 0.25 Up 0.89% 10,469,087 263.39B Chart, Messages, Key Stats, More
GOOG 10:33AM ET 449.14 Up 5.06 Up 1.14% 1,378,627 140.75B Chart, Messages, Key Stats, More
GGP 10:28AM ET 38.58 Down 0.13 Down 0.34% 727,075 9.41B Chart, Messages, Key Stats, More
RHT 10:29AM ET 18.31 Up 0.78 Up 4.45% 1,322,182 3.56B Chart, Messages, Key Stats, More
NOVL 10:34AM ET 6.24 Up 0.07 Up 1.13% 304,018 2.20B Chart, Messages, Key Stats, More
BP 10:29AM ET 60.73 Up 0.07 Up 0.12% 1,307,600 191.53B Chart, Messages, Key Stats, More
HAL 10:29AM ET 39.29 Up 0.19 Up 0.49% 2,895,156 34.58B Chart, Messages, Key Stats, More
SLB 10:29AM ET 86.40 Up 1.02 Up 1.19% 1,678,732 103.48B Chart, Messages, Key Stats, More
KBR 10:28AM ET 28.97 Up 0.59 Up 2.08% 140,700 4.92B Chart, Messages, Key Stats, More
XOM 10:29AM ET 86.95 Up 0.75 Up 0.87% 3,768,280 465.18B Chart, Messages, Key Stats, More
MT 10:29AM ET 79.93 Up 1.08 Up 1.37% 1,044,700 113.63B Chart, Messages, Key Stats, More
COP 10:29AM ET 76.21 Up 0.48 Up 0.63% 2,057,477 119.00B Chart, Messages, Key Stats, More
 

A couple of steel stocks are mixed in at the end, and KBR is Kellog Brown and Root, the company that Haliburton spun off and that fed me while I was deployed to Haiti all those years ago. GGP is General Growth Properties, and represents both real estate and the retail market. What I look at first and foremost is the overall color. I don’t care about any particular stock so much as if the tech sector is all red but the oil sector is green, or if the whole thing is one or the other.

I realize that all of this summary info is available on the headlines of the Wall Street Journal or on the finance pages, but there is something about being able to draw your own conclusion from the presentation of the data that makes it easier to comprehend quickly.

Another piece of information that leaps out from this display is if a given stock is behaving differently from its market. There is a difference between Oracle going up 5% on a tech green day and a tech red day.

Tufte would be proud .

More musings

Synagogue would be a hell of a lot more fun if more people heckled the Rabbi.

Ayn Rand seems to appeal to men in their mid twenties.

Martin Fowler is responsible for popularizing some of the most important ideas in software design.

There is an amazing power in large combinations of simple things. The universe is defined by a very small number of rules repeated a lot of times.

Even if we accept that there should be a categorical imperative, how can we determine which aspect of our action should be the generalizable part and which is the part that is adapted to our present situation?

Considering how much information is stored on me in computers, I still have to fill out my address an excessive number of times. Shouldn’t we be at the point that my address is be pre-filled in a web form?

High fiber cereal does not make a good dinner.

Wrestling and rock climbing a very complimentary sports. Both require a sense of balance in your body. Rock climbing develops hand grip strength that is a benefit to wrestling. Both require and develop a strong core.

Binary search is the primary tool of debugging anything.

The build-execute-change cycle is the key to productivity in software. Building high performance software usually kills you in the build step.

We wasted too much time on Drill and Ceremony at West Point. If Napoleon Bonaparte ever invades, the United States Corps of Cadets will be ready.

No drug can be more addictive than the sound of your own child’s laughter.