Musings

Don’t hit publish on the blog when you just want to save a draft.

Big Builds are Bad. Software should be developed and distributed in small packages. Linux is successful due to things like apt, yum, and yast.

Interface Specifications need to be more specific.  Just saying that something is a string is not really helpful if that something needs to conform to a pattern.

Programming and blogging requires sugar in the brain.

Interviews are tricky…on both sides of the table. Career fairs are worse.

C++ Has a lot of magic in it. Can we make type level programming more transparent?

Microsoft purchasing Yahoo would be good for Google, but bad for just about everyone else.

Being a Dad is really cool. Even when it sucks, it is great. Sometimes kids refuse to go to sleep. This leads to sleep deprivation, but also leads to really wonderful moments in rocking chair in the middle of the night.

Pool is a great Geek game. Lower left-hand English is neat.

Snowshoes are good off the trail. Not so good on the trail. If your going on the trail, take the cross country skis. Snowmobiles smell funny.

New Hampshire winter weather is still as brutal today as it was when I left the area in the early ’90s.

It is hard to sing a Jazzy version of Old MacDonald had a Farm.  It is harder to do after the tenth repetition while trying to get a child to fall asleep.
If you listen to Children’s CDs long enough, you will develop favorite children’s songs. I like the hippo song.

Is there really a difference between the Ethernet and SCSI protocols? I don’t know, but it would be fun to find out.

The compiler is your friend. Let it check your work for you.

Why write code on a white board if you have a computer available? Especially if you have an overhead projector?

Where do the local peregrine falcons sleep? Where would they be sleeping if we hadn’t built up the whole area?

If I could have a redo on which language to take as a Sophomore, I would probably would have liked to take Chinese. Russian and Arabic would also do. German was not a good choice for me.

If Bush Senior had insisted on pushing to Baghdad, it would have been my generation in this mess as opposed to the current set of junior officers. Instead of Haiti, I would have gone to Basra or something.

There are too many interesting topics in the world to pursue them all, or even a small fraction of them.

Every philosopher I’ve read, especially the ones I disagree with, ave said something that is valuable and true.

No matter how old you are, when you get together with your parents, you revert to teenager status.

This list should never see the light of day.

Layout of the oil situation in Iraq

Not sure if I should post this or not.  So many people are in a better position to really understand the situation than I am.  Really, what I want is for people to respond to this to expand, clarify, correct and elucidate me.

Maybe we won’t be in Iraq forever. Maybe the real yardstick we need for success is the amount of oil flowing out of the country. Iraq is producing a small fraction of it’s capacity. If it ramped up to several times it’s prewar amount, it could be the largest producer in the world. Considering the current cost of oil is $3.00/gal at the pumps in Massachusetts, It is possible that there could be a healthy dose of currency flowing into Iraq that is not coming out of the US Treasury. There are several factors at play:

Sunni-Shiite-Kurd political contention. If any one of these groups gains power over the majority of the Oil revenue, it will be significantly threatening to the other two.

OPEC. The nearest nations to Iraq are the ones with the most to lose from a large capacity, non-OPEC nation in their midst. Iraq is a founding member of OPEC, but it may not be perceived as an independent entity by the others, and more as a puppet of the US. Any move to substantially increase oil production will bring up the global supply, drop the price, and decrease profits across the board.

China and India: These two nations are growing fast, and demand for Oil from these nations has kept the cost of oil high. Both have a vested interest in increasing global supply to decrease price.

France: Much of Iraq’s pre-war debt was owned by France. The US invasion changed France’s relationship to Iraq from insider to outsider. They have their own problems with Muslims both at home and across the Mediterranean in Algeria.

Australia: Currently planning on pulling out of Iraq. Fewer troops to patrol means a higher possibility of successful insurgent attacks.

Russia: Russia is now a huge supplier of Oil to Europe. While Russia is not a part of OPEC, it still has a lot on the line in terms of Oil Revenue.

Turkey: Oil revenue to the Kurds means more money that can flow over the border to Kurd separatists in Turkey. However, stability on Turkey’s southern border will be beneficial to Turkey overall. Mosul, in Northern Iraq, is one of the most violent locations, with a major operation underway even as I type this. Turkey has performed operations across the border into Norther Iraq targeted at Kurdish fighters.

US Economy: The cost of maintaining troops in Iraq may prove too high. The US may have to withdraw before a critical mass of oil extraction infrastructure is in place. The US is anticipating the oil influx. Most of the companies that will benefit from Iraq oil are based in the US or Britain, and thus have a lot to gain. The US President and many in his administration are primarily oil-men, but with the election coming up, that may change.

Middle East Nations: Saudi Arabia and Iran both have a vested interest in supporting their allies in the Iraqi Government. If A Sunni Majority again emerged, it would be more closely allied with Saudi Arabia. A Shiite Majority would be allied with Iran. Either case would lead to increased domestic strife and a decrease in oil production. With a Sunni majority, Iran would have the “moral high-ground” of supporting Shiites against their oppressors. The reverse would be true with a Shiite majority and Saudi Arabia supporting their side. I realize that this is a vast oversimplification, but the a part of the broad trends.

The path forward needs to walk a tight rope of a balance of power that will lead to stability. Both sides need to believe that they have more to gain from getting oil set up than they will lose in relative power and status. But even if Iraq is able to produce a substantial quantity of oil, it won’t necessarily work wonders for Iraqi society. Typically, Oil is run by a few companies. It may be possible that an Iraqi Wildcatting culture will grow up, but that can’t happen until there is a long term education amongst Iraqis in oil extraction. Oil has several offshoot industries. So there should be growth of the competitive/cooperative spirit that is the product of a healthy capitalist system. But it will still be centered around a single industry with major fluctuations based on global events. If we look to Saudi Arabia, we see that oil wealth does not necessarily carry with it human rights.

Here is a dark scenario. Iraq will get enough of its oil infrastructure up and running to make US involvement essential, but not stabilize enough that we can pull out. We will have a force in Iraq in perpetuity, drawing fire and making enemies. That force will be essential to defending the Oil extraction facilitated in Iraq.

I don’t know what will really happen. I certainly don’t have the perspective to see that clearly. Just trying to layout the situation based on the limited info that I have.

Wrapping a boolean return value with an exception

I am currently working with a long block of code that uses && (logical and) to a long list of functions. The idea is to run all of the functions and short circuit if any of them fail. The problem is that there is no consistant mechanism for error reportin. If any of the functions fail, I have no way of knowing which one. This is my planned approach.

#include <iostream>
#include <stdexcept>
using namespace std;

bool invert(bool val){
return !val;
}

#define attempt( X ){ if (!X){ throw runtime_error(#X);}}

int main(){
cout << “OK” << endl;
try{
attempt(invert(true));
}catch(runtime_error& re){
cout << “runtime_error exception:” << re.what() <<endl;
}
return 0;
}

Ayn Rand and Robert Pirsig

A younger friend at work has recently completed reading Atlas Shrugged. He is fairly close to the age I was at when I got enthralled with Ayn Rand. Something about her writing appeals to the emerging man. “You can do anything. The world will try to hold you down, but you can create.” At the time of my transition from Army Officer to civilian programmer, nothing was more comforting to hear.

Continue reading

Why virtual, why not

Using a non-virtual method in C++ is syntactic sugar:  It doesn’t allow you to do anything you could not as easily do BY using a static member function and passing in the object instance as the first parameter.  If you don’t fully qualify the name of the function, the type checking of the first parameter will make sure you call the right version, or at least get a compilation error until you do fully qualify.The only reason you would need to use one is for template programming, where some classes have the method you wish to call as a virtual, and you want to be consistent across all classes passed to the template.

This is not to say that all non-virtual functions are bad, just an observation about what the language  provides.

Affiliate links

I keep getting comments from people along the lines of:  I just found you on Google search and added you to my Reader.  When I follow the links back to their web pages, they are marketing sites.  This is my blanket apology to all people who send such links:

I am more than willing to post your responses, and give you the links back you so desire.  But this blog has a minimum requirement for comments:  You have to show that you read the article, and then contribute something to a discussion on the topic at hand.  Note that just reading the article is not enough.  I am not going to make this a vehicle for spam.

If you are genuinely interested in the topic, by all means post, even if it is just to ask for more information.  I will certainly try to answer your question.  But again, don’t just fire off questions hoping to get the affiliate links.  I’m sorry, but that is just not how I am managing this site. The questions have to be something where the discussion would expand on the topic of the post.

I suspect that Google is smart enough at this point to identify blog sites and filter out links from them when calculating web page standings.

Integrated Development Environments

I understand now why so many people that have cut their teeth on C coding in Unix/Linux hate C++.  A coder can get her job done in C without needing significant tool support.  An editor, a build toolchain, and limited reverse engineering support a-la objdump and you are good to go.  Tag support makes navigation a lot easier, and suddenly, you have decent productivity.

When I started in the professional world, I learned C++ using Microsoft Visual Studio.  I had, prior to then, been a big fan of Borland for Pascal, and had even purchased a copy of Borland C++.  But, unlike the MS product, it did not then have an integrated debugger, and my learning curve was quite steep.  MSVC took me through the steps, provided decent navigation, and easy to use wizards for MS technology.  I did have the frustration of trying to work through libraries without having the source code, but this wasn’t the VC developers fault.

When I left the MS world for the Java world, the first thing I noticed was the lack of an IDE.  I was just so productive with all of the tools integrated, that it took me a while to get used to coding in EMACS. I had long since learned the value of stepping through code after I wrote it to make sure it was doing what I expected. Using gdb, while effective, was much slower than an integrated debugger.  It wasn’t until I got my hands on Intellij’s Idea that I felt a return to the productivity I had under MSVC.

When refactoring support hit in Eclipse, I was stunned at the ease of maintaining code.  I can’t say enough about this, and may try to recreate some of my earlier writings on Refactoring in order to show how valuable the approach can be.  Suffice to say, it is now a go-to tool for coding that I miss when forced to do without.

Now I am trying to figure out how best to do C++ development on Linux.  etags just does not handle context well enough, and CScope’s navigation is fairly clunky.  I’ve got a demo of Visual Slickedit, and the Eclipse platform with C/C++ support built in, but I have yet to find a smooth working environment.  My problem comes from dealing with an alien code base, tightly coupled dependencies and byzantine makefiles that I haven’t had time to spelunk.  I’ve not been able to get automated refactoring support to work and even base navigation is problematic.  As I work out the solution, I will  record my approaches here so that I will be able to recreate them in the future.

Oracle ODBC Setup on Ubuntu (32 on 64)

First of all, I am running a 64 bit Ubuntu 7.04 but I need a 32 Bit Oracle for the application I am using. I have a 32 bit chroot setup. The setup for this is beyond the scope of this article. This version of Ubuntu ships with unixodbc version 2.2.11-13 . There is a symbol, SQLGetPrivateProfileStringW, defined in later versions that the Oracle 11g driver requires. This symbol is not defined in unixodbc version 2.2.11-13. Thus, you have to use the 10.2 Oracle Drivers.

I Downloaded 3 files from The Oracle tech download page for 32bit Linux: the instantclient, SQL Plus, and ODBC packages. I unzipped these in my directory ~/apps/oracle32 Which now looks like this:

adyoung@adyoung-laptop$ pwd
/home/adyoung/apps/oracle32/instantclient_10_2
adyoung@adyoung-laptop$ ls
classes12.jar libocci.so.10.1 libsqora.so.10.1 ojdbc14.jar
genezi libociei.so ODBC_IC_Readme_Linux.html sqlplus
glogin.sql libocijdbc10.so ODBCRelnotesJA.htm
libclntsh.so.10.1 libsqlplusic.so ODBCRelnotesUS.htm
libnnz10.so libsqlplus.so odbc_update_ini.sh

I created an entry in /chroot/etc/odbcinst.ini:

[Oracle 10g ODBC driver]
Description = Oracle ODBC driver for Oracle 10g
Driver = /home/adyoung/apps/oracle32/instantclient_10_2/libsqora.so.10.1
Setup =
FileUsage =
CPTimeout =
CPReuse =

And another in /chroot/etc/odbc.ini

[EMO]
Application Attributes = T
Attributes = W
BatchAutocommitMode = IfAllSuccessful
BindAsFLOAT = F
CloseCursor = F
DisableDPM = F
DisableMTS = T
Driver = Oracle 10g ODBC driver
DSN = EMO
EXECSchemaOpt =
EXECSyntax = T
Failover = T
FailoverDelay = 10
FailoverRetryCount = 10
FetchBufferSize = 64000
ForceWCHAR = F
Lobs = T
Longs = T
MetadataIdDefault = F
QueryTimeout = T
ResultSets = T
ServerName = 10.10.15.15/DRORACLE
SQLGetData extensions = F
Translation DLL =
Translation Option = 0
DisableRULEHint = T
UserID = adyoung
StatementCache=F
CacheBufferSize=20

Once again, DSN and IP Address have been changed to protect the guilty. To test the datasource, run:

sudo dchroot -d LD_LIBRARY_PATH=/home/adyoung/apps/oracle32/instantclient_10_2 DataManagerII

To just test sqlplus connectivity, from inside the chroot, run:

./sqlplus adyoung/adyoung@10.10.15.15/DRORACLE

Note that using the instant client, no TNSNAMES.ORA file is required.

ODBC Setup on Ubuntu/Debian

For the base config tools:

sudo apt-get install unixodbc-bin unixodbc odbcinst1debian1

For Postgres

sudo apt-get install odbc-postgresql

Use the template file provided to setup the odbc driver entries:

sudo odbcinst -i -d -f /usr/share/psqlodbc/odbcinst.ini.template

And this sets up the sample DSNs. (all one line)

 sudo odbcinst -i -s -l  -n adyoung-pg -f /usr/share/doc/odbc-postgresql/examples/odbc.ini.template

Then modify /etc/odbc.ini to suit your DB.

For MSSQL Server and Sybase:

 sudo apt-get install tdsodbc

sudo odbcinst -i -d -f /usr/share/doc/freetds-dev/examples/odbcinst.ini

Unfortunately, this does not have a sample ODBC setup template.

The structure of a postgres database

Since my work at Penguin was primarily system work, I’ve not had to work with databases much for quite a while. Which is really a pity, since I love the elegance and simplicity of the interface, and the strength of the pre-and-post condition constraints. Anyone who has managed to push my Database hot button knows that my Database of choice is PostgreSQL. It is opensource to the core. The developers treated transactions processing as baseline, no a feature to be added later. It supports nested SQL statements (the S ins SQL is for “Structured”). I am happy to be plugging away at a Postgres Database again.

Part of my current project entails porting the backing store for an application from Oracle to Postgres. There is a nuke script that attempts to remove all objects from a database that have a certain prefix. I will change here to protect the guilty. Let’s pretend that my database is to hold overly emotional music references, and thus all of the tables, views, triggers, and so on are prefixed with ‘EMO’. The oracle script does this:

declare
sqlstmt varchar(200);
varowner varchar(50);
TYPE cur_type IS REF CURSOR;

objectcursor cur_type;

Begin
select USER into varowner from global_name;

open objectcursor for
select ‘drop ‘ || object_type || ‘ ‘ || owner || ‘.’ || object_name
|| decode(object_type,’TABLE’, ‘ cascade constraints’)
from all_objects
where owner = varowner
and (object_name like ‘EMO%’ or object_name like ‘%STAT%PROC’)
and object_type in (‘TABLE’, ‘VIEW’, ‘SEQUENCE’, ‘SYNONYM’,
‘FUNCTION’,’PROCEDURE’, ‘PACKAGE’);

Loop
fetch objectcursor into sqlstmt;
exit when objectcursor%NOTFOUND;
execute immediate sqlstmt;
End Loop;
Close objectcursor;
End;

But tring to port this directly to postgres gets an error right at the beginning:

ERROR: syntax error at or near “varchar” LINE 13: sqlstmt varchar(200);

So I’ll bring this whole thing into a function instead. At then end of my script, I will just drop the function. This has the advantage that I can check the format of my code without having to execute it each time.

When working in an an unfamiliar language (an I’ve never really done PG\PLSQL before) I apply a simple approach. First, I get a “Hello, World” type program working. In this case, I used an article from ONLamp.com and got that to run. Next I strip away everything that is context specific until I get a piece of code that is minimal but syntactically correct. It looks like this:

CREATE OR REPLACE FUNCTION dbemo_nuke_postgres ()
RETURNS integer AS $$

declare

BEGIN
return 1;

End;

$$ LANGUAGE plpgsql;

Note that I modified the prefix so that it won’t Hit the match for EMO% in the like statement I am going to need later. Not sure what would happen if I dropped my function as I ran it. Most likely it would work fine, but why chance it?

Now I start adding in parts of the original function piece by piece. First step is to put in the declares clause. The postgres porting guide says I don’t need a cursor. I quickly learn that this goes between the function declaration and the BEGIN statement:

CREATE OR REPLACE FUNCTION dbemo_nuke_postgres ()
RETURNS integer AS $$

declare
sqlstmt varchar(200);
varowner varchar(50);
objectcursor RECORD;

BEGIN
return 1;

End;

$$ LANGUAGE plpgsql;

This is saved in the file EMO_nuke_postgres_part.sql. I run this from the command prompt:

adyoung@adyoung-laptop$ psql -f EMO_nuke_postgres_part.sql
CREATE FUNCTION

And we’re off. Now to the Oracle specifics.

select USER into varowner from global_name;

This command seems to be asking for info regarding the current user.

select ‘drop ‘ || object_type || ‘ ‘ || owner || ‘.’ || object_name
|| decode(object_type,’TABLE’, ‘ cascade constraints’)
from all_objects
where owner = varowner
and (object_name like ‘VPX%’ or object_name like ‘%STAT%PROC’)
and object_type in (‘TABLE’, ‘VIEW’, ‘SEQUENCE’, ‘SYNONYM’,
‘FUNCTION’,’PROCEDURE’, ‘PACKAGE’)

This code is dynamically creating a drop statement for various objects in the schema. Where is this information stored in postgresql? Let’s take a look:

adyoung@adyoung-laptop$ psql
Welcome to psql 8.2.6, the PostgreSQL interactive terminal.

Type: \copyright for distribution terms
\h for help with SQL commands
\? for help with psql commands
\g or terminate with semicolon to execute query
\q to quit
adyoung=# SELECT * from pg_
pg_aggregate pg_group pg_settings pg_stats
pg_am pg_index pg_shadow pg_stat_sys_indexes
pg_amop pg_indexes pg_shdepend pg_stat_sys_tables
pg_amproc pg_inherits pg_shdescription pg_stat_user_indexes
pg_attrdef pg_language pg_stat_activity pg_stat_user_tables
pg_attribute pg_largeobject pg_stat_all_indexes pg_tables
pg_authid pg_listener pg_stat_all_tables pg_tablespace
pg_auth_members pg_locks pg_stat_database pg_temp_1.
pg_autovacuum pg_namespace pg_statio_all_indexes pg_timezone_abbrevs
pg_cast pg_opclass pg_statio_all_sequences pg_timezone_names
pg_catalog. pg_operator pg_statio_all_tables pg_toast.
pg_class pg_pltemplate pg_statio_sys_indexes pg_trigger
pg_constraint pg_prepared_statements pg_statio_sys_sequences pg_type
pg_conversion pg_prepared_xacts pg_statio_sys_tables pg_user
pg_cursors pg_proc pg_statio_user_indexes pg_views
pg_database pg_rewrite pg_statio_user_sequences
pg_depend pg_roles pg_statio_user_tables
pg_description pg_rules pg_statistic
adyoung=# SELECT * from pg_

I know that metadata is stored in the pg_.* family of objects, so I use statement completion to show me what is there. Nothing is obvious to me yet, so I poke around at pg_tables.

adyoung=# \d pg_tables;
View “pg_catalog.pg_tables”
Column | Type | Modifiers
————-+———+———–
schemaname | name |
tablename | name |
tableowner | name |
tablespace | name |
hasindexes | boolean |
hasrules | boolean |
hastriggers | boolean |
View definition:
SELECT n.nspname AS schemaname, c.relname AS tablename, pg_get_userbyid(c.relowner) AS tableowner, t.spcname AS “tablespace”, c.relhasindex AS hasindexes, c.relhasrules AS hasrules, c.reltriggers > 0 AS hastriggers
FROM pg_class c
LEFT JOIN pg_namespace n ON n.oid = c.relnamespace
LEFT JOIN pg_tablespace t ON t.oid = c.reltablespace
WHERE c.relkind = ‘r’::”char”;

Ah Cha! It is a view into a relation called pg_class that seems to hold all of the metadata. Let’s take a look at that:

adyoung=# \d pg_class
Table “pg_catalog.pg_class”
Column | Type | Modifiers
—————-+———–+———–
relname | name | not null
relnamespace | oid | not null
reltype | oid | not null
relowner | oid | not null
relam | oid | not null
relfilenode | oid | not null
reltablespace | oid | not null
relpages | integer | not null
reltuples | real | not null
reltoastrelid | oid | not null
reltoastidxid | oid | not null
relhasindex | boolean | not null
relisshared | boolean | not null
relkind | “char” | not null
relnatts | smallint | not null
relchecks | smallint | not null
reltriggers | smallint | not null
relukeys | smallint | not null
relfkeys | smallint | not null
relrefs | smallint | not null
relhasoids | boolean | not null
relhaspkey | boolean | not null
relhasrules | boolean | not null
relhassubclass | boolean | not null
relfrozenxid | xid | not null
relacl | aclitem[] |
reloptions | text[] |
Indexes:
“pg_class_oid_index” UNIQUE, btree (oid)
“pg_class_relname_nsp_index” UNIQUE, btree (relname, relnamespace)

Lots of good stuff here. OK, we now have enough raw information to generate some real knowledge. In Postgres, everything is a class. To find out if something is a table you select only those fields with the right type:

adyoung=# SELECT distinct (relkind) from pg_class ;
relkind
———
S
i
r
t
v

Funny that it isn’t type ‘t’ for table, but that seems to be of type toast:

adyoung=# SELECT relname from pg_class where relkind =’t’ limit (3);
relname
—————-
pg_toast_10757
pg_toast_10762
pg_toast_10767
(3 rows)
OK, our original SQL seems to be creating a statment to then execute to drop for each class. WHat collumns it is selecting. Here’s our where clause.:

where owner = varowner
and (object_name like ‘EMO%’ or object_name like ‘%STAT%PROC’)
and object_type in (‘TABLE’, ‘VIEW’, ‘SEQUENCE’, ‘SYNONYM’,
‘FUNCTION’,’PROCEDURE’, ‘PACKAGE’).

Let’s start with the like clause.

adyoung=# SELECT count(relname) from pg_class where relname like ‘vpx%’ or relname like ‘%STAT%PROC’;
count
——-
228

Looks good. But S, i, r, t v doesn’t seem to cover the array we want. I don’t see any functions. Let me chgeck the pg_proc:

adyoung=# \d pg_proc
Table “pg_catalog.pg_proc”
Column | Type | Modifiers
—————-+———–+———–
proname | name | not null
pronamespace | oid | not null
proowner | oid | not null
prolang | oid | not null
proisagg | boolean | not null
prosecdef | boolean | not null
proisstrict | boolean | not null
proretset | boolean | not null
provolatile | “char” | not null
pronargs | smallint | not null
prorettype | oid | not null
proargtypes | oidvector | not null
proallargtypes | oid[] |
proargmodes | “char”[] |
proargnames | text[] |
prosrc | text |
probin | bytea |
proacl | aclitem[] |
Indexes:
“pg_proc_oid_index” UNIQUE, btree (oid)
“pg_proc_proname_args_nsp_index” UNIQUE, btree (proname, proargtypes, pronamespace)

Not a view. OK:

adyoung=# SELECT proname from pg_proc where proname like ‘vcdb_nuke_postgres’ ;
proname
——————–
vcdb_nuke_postgres
vcdb_nuke_postgres
(2 rows)
We know that my stuff is in there.

More on this to come…