Transcribing Music using mpg123

A long time tradition for learning how to play like the great soloists is to learn their solos from recordings.  Ideally, you would listen to the recording slowed down.  However, on analog equipment, when you slow down the recording, you have to adjust the pitch back up, since you have effective dropped the frequency of the recorded notes.  With mp3 files, you have the advantage of having sampled the original recording into discrete chucks called frames.  Thus to slow down a recording, you can just tell your music player to double the number of times it plays each frame, and you have slowed down the playback without affecting the pitch.

Say you are a harmonica player and you want to learn John Popper’s solo at the beginning of Runaround.  You can do so by playing it back in mpg123.  Here’s the approach I would use.  Start by playing the song from the beginning:

mpg123 runaround.mp3

Now find the start of the solo.  To skip ahead, use the -k option to skip frames.  Try at about 1000.

mpg123 -k 1000 runaround.mp3

Now use a binary search:  If you hear the solo playing, go backwards, if you don’t go forwards.  At 1000 I hear it playing, so I go back to 500, then 250. At 250 I just hear the intro guitar, so I jump forward by 125 to 325 and so on until I figure out that the solo starts roughly at 270.  Now I slow down the playing by half:

mpg123 -h2 -k 1000 runaround.mp3

It is distorted as all hell, but I can make out the notes, and more importantly, match it on a piano, harmonica or saxophone.  I can slow it down even further with

mpg123 -h3 -k 1000 runaround.mp3

Which is as far as I would want to go.  Note that for older recordings, where the solo may be exclusively on the left or right track, you could add the -0 or -1 options to shut out more of the background noise.

As you progress, you want to move forward in the song.  Use the -v option to display frame information as you play.

Simple LDAP client code using deprecated API

This query returns all objects.

#include <stdio.h>

#define LDAP_DEPRECATED    1

#include <ldap.h>

const  char *base = “dc=application,dc=company,dc=com”;
char loginDN[] =”cn=Manager,dc=virtualcenter,dc=vmware,dc=int”;

#define TRACE()  printf(“%s:%d\n”, __FILE__,__LINE__)

/*TODO free memory*/
void handle_search_result(LDAP *ldap,  LDAPMessage *result){
char  *attr;
LDAPMessage *entry;
BerElement *ber;
char **values;

printf(“Entries =  %d\n”, ldap_count_entries( ldap, result ));

for (entry = ldap_first_entry(ldap, result);
entry != NULL;
entry = ldap_next_entry(ldap, result))
{
printf(“dn=(%s)\n”,ldap_get_dn( ldap, entry ));

for (attr = ldap_first_attribute(ldap, result, &ber);
attr != NULL;
attr = ldap_next_attribute(ldap, result, ber))
{
printf(“\t%s\n”,attr);

values = ldap_get_values(ldap, entry, attr);
int   i =0 ;
int len =  ldap_count_values(values);
for (i =0; i < len; ++i){
printf(“\t\t%s\n”,values[i]);
}
}
result = ldap_next_message(ldap, result);
}
}

int execute(){
printf(“Running\n”);
LDAP *ldap;
char *uri = “ldap://localhost”;
int rc = ldap_initialize(&ldap, uri);
if (LDAP_SUCCESS != rc){
printf(“error:%s\n”,  ldap_err2string(rc));
return rc;
}

rc =  ldap_bind_s(ldap, loginDN, “vmware”, LDAP_AUTH_SIMPLE );
if (LDAP_SUCCESS != rc) goto error;

char *attrs;
LDAPMessage *res = 0;
rc =  ldap_search_s (ldap,base,LDAP_SCOPE_CHILDREN
,”(objectclass=*)”,0,0,&res);
if (LDAP_SUCCESS != rc) goto error;

printf(“number of results = %d\n”,ldap_count_messages(ldap,res));
int attid = 0;
char * attr;

handle_search_result(ldap,  res);

goto cleanup;
error:
printf(“error:%s\n”,  ldap_err2string(rc));
cleanup:
ldap_unbind(ldap);
return rc;

return  0;
}

int main(){

return execute() ? -1 : 0;

}

OpenLDAP API is somewhat hostile

After a few days beating my head against OpenLDAP’s C API, the only thing I can think is that the developers don’t want you to use it.

All of the simple functions, the functions that are in all of the sample code out there, has been deprecated:

  1. ldap_bind and it’s ilk are gone with the exception of ldap_sasl_bind and ldap_sasl_bind_s.
  2. ldap_init, and ldap_open are gone, although ldap_initialize still exists.

What seems to be going on is an attempt to push people to use SASL: The (not so) Simple Authentication and Security Layer that abstracts away the differences between Kerberos, Radius, and other full fledged authentication and authorization mechanisms.

The problem here is that the most common  application for using LDAP is for authenticating users.  It doesn’t matter that you “shouldn’t” do this according to the established literature.  If you are stuck talking to Active DIrectory, LDAP is your primary tool.

The LDAP_Result api makes sense from an object oriented perspective, but man, is that a lot of function calls just to iterate through a list.  For example, to process the effect of calling ldap_sasl_bind you have to call:

  1. ldap_result
  2. ldap_first_message
  3. ldap_parse_result
  4. ldap_free for each value allocated by ldap_parse_result
  5. ldap_next_message
  6. ldap_freemsg

The problem here is they are trying to be objected, but stopping short of actually returning vtables.  I suspect I could just look at the LDAPMessage structure and get the same result.

I think the choice of SASL is somewhat suspect as well.  I would much prefer it if they used PAM.  If you are going to make use of a flexible authentication mechanism, go with the dominant one.

I admit that I am not an expert on LDAP things, and that perhaps I am missing something.  But trying to get the C equivalent of

ldapsearch -H “ldap://localhost” -LLL -x -b  ‘dc=application,dc=domain,dc=com’ ‘(objectclass=*)’

Should not take several days.

OVER keyword in Oracle

I’m looking at the SQL Scripts that create and modify the database for VPXD. Specifically, I am looking at the oracle version stats[123] scripts. They included a query that uses the “OVER” keyword and I have not found a decent description of what this keyword does. This is the query:

SELECT
st.counter_id AS COUNTER_ID
,v_time_rowid AS TIME_ID
,ST.STAT_VAL AS STAT_VAL
,DENSE_RANK()

OVER (PARTITION BY ST.COUNTER_ID ORDER BY SM.SAMPLE_TIME DESC) AS DENSE_RANK
FROM vpx_sample_time1 sm,vpx_stat_counter sc, vpx_stat_def sd, vpx_hist_stat1 st
WHERE sm.rollup_counter=to_number(to_char(v_start_time_id)||to_char(v_cnt))
AND st.time_id = sm.time_id
AND st.counter_id=SC.COUNTER_ID
AND SC.STAT_ID=SD.ID
AND ((SC.DEVICE_ID =v_null_device_id and v_stat_rollup_level<3) or (v_stat_rollup_level>=3))
AND SD.STAT_LEVEL<=v_stat_rollup_level
AND SD.ROLLUP_TYPE=3)
WHERE DENSE_RANK=1

;

The one description of the Over command I have found has come from the Microsoft website:

“Determines the partitioning and ordering of the rowset before the associated window function is applied”

OK, so it is a union of two types of functionality: partitioning and ordering. The queries I am looking at use both. Let’s start with partitioning:

PARTITION BY
Divides the result set into partitions. The window function is applied to each partition separately and computation restarts for each partition.

The window function in this place must somehow refer to DENSE_RANK. To me this looks like it is possibly trying to insert a subselect into the overall select. Here is how MSDN defines DENSE_RANK. I’d like to reiterate that these are Oracle functions I’m trying to understand, and Microsoft is doing all the work explaining them. When I tried executing this SQL Query in SQLPLUS via the Oracle 11 instant client, it balked:
SQL> 2 3 4 5 6 SQL> SP2-0734: unknown command beginning “OVER (PART…” – rest of line ignored.
SQL> SP2-0734: unknown command beginning “FROM vpx_…” – rest of line ignored. It parses fine when I do it as part of a plsql stored procedure.

Time to dig out the slightly larger guns: I picked up a copy of the SQL Cookbook by Anthony Moliaro from Our Friends in Sebastopol, CA. This covered both DENSE_RANK and the OVER keyword.

Over, it turns out, is one of a new set of functions added to SQL 2003 that the Standard calls Windowing functions. This means they operate on a subset of the results returned from the outside query. The best example I saw showed if you are selecting all employees from all departments, and next to each one you wanted to show how many employees were in that department, you could use count(*) with an over clause that specified PARTITION by department. This example is in the Cookbook.

DENSE_RANK () tells you how many rows come before the current row in some ordering scheme. The over clause is used to define that ordering scheme. If two rows have the same rank, they have equivalent values according to DENSE_RANK. For instance, if we have five people with five salaries, their DENSE_RANK of salaies will be 1 through five, highest to lowest. If the top two salaries were identical, top rank would return 1,1,2,3,4.

OK, so it looks like the original query wants to use DENSE_RANK on a subset of the returned rows of the query. The window will be defined by this portion of the clause: PARTITION BY ST.COUNTER_ID

And the order within this window will be determined by: ORDER BY SM.SAMPLE_TIME DESC

OK time to get serious about playing:

I’ve learend a little biut since the above frustrations trying to run the stored procedure. Here’s what I do now:

run sqlplus. For me this is

/home/adyoung/apps/oracle/instantclient_11_1/sqlplus adyoung/adyoung@10.17.195.159/DRORACLE

Now make it so anything I print to the screen is visible:

set serveroutput on

Load my sql file in and execute it.

get myfile.sql

/

OK, I have a development method. To print debugging output:
DBMS_OUTPUT.PUT_LINE(‘Text Here’);
And we can debug. In the time honored tradition of always moving from success, I comment everything out but a simple output statement. In this case, I know that the real code uses a variable for the rollup level, so my File (minus commented out section, looks like this:
create or replace procedure stattest
is
v_stat_rollup_level int;
begin
v_stat_rollup_level := 2;
dbms_output.put_line(‘Value: ‘ || TO_CHAR(v_stat_rollup_level));
end stattest;
Which executes using the above method. Now I start adding in parts of the SELECT.
SELECT st.counter_id AS COUNTER_ID FROM vpx_hist_stat1 st;
Of course, it hates this. When I load it into sqlplus I get:
Warning: Procedure created with compilation errors.
Of course, in order to see these compilation errors, I have to jump through a couple more hoops:
SQL> set arraysize 1
SQL> set MAXDATA 6000
SQL> show errors procedure stattest
LINE/COL ERROR
——– —————————————————————–
6/1 PLS-00428: an INTO clause is expected in this SELECT statement
So I add a variable called maxcounter in the declare section and modify my select:
SELECT
max(st.counter_id) into maxcounter
FROM
vpx_hist_stat1 st;
dbms_output.put_line(‘Maxcounter: ‘ || TO_CHAR(maxcounter));
And I have a procedure that executes and does something. On to the rest of the select. After several iterations of trial and error, I have the SQL query wokring as a cursor. I’ve now figured out even a better way to work. I’ve modified my file like below, and can now cat it into the sqlplus interpreter:
set arraysize 1
set MAXDATA 6000
set serveroutput on

create or replace procedure stattest
is
v_stat_rollup_level int := 2;
maxcounter int := 0;

CURSOR MYCURSOR is SELECT
st.counter_id as COUNTER_ID,
st.STAT_VAL as STAT_VAL
FROM vpx_sample_time1 sm,
vpx_stat_counter sc,
vpx_stat_def sd,
vpx_hist_stat1 st

WHERE
st.time_id = sm.time_id
AND st.counter_id=SC.COUNTER_ID
AND SC.STAT_ID=SD.ID
AND SD.ROLLUP_TYPE=3
AND SD.STAT_LEVEL<=v_stat_rollup_level;

begin

open MYCURSOR;

FOR recname in MYCURSOR
LOOP
dbms_output.put_line(‘Value: ‘ || TO_CHAR(v_stat_rollup_level));
END LOOP;

dbms_output.put_line(‘Maxcounter: ‘ || TO_CHAR(maxcounter));

end stattest;

/

show errors procedure stattest

.

OK, I am going to bail here.  This post has been half written for a while, and is not going to get completed anytime soon.  I think I’d vote “Over” as something that would be useful in the next rev of postgres, if only for simplifying porting from Oracle.

Dependency Licenses

The following script, when run on a RPM based system, tells you what licenses your application might trip by linking with other code.  You have to  set $MYEXE to som path to an executable.

for LIB in `ldd $MYEXE | sed -e ‘s!.*=>!!’ -e ‘s! (.*!!’` ; do rpmquery –whatprovides $LIB ; done | grep -v “any package” | sort -u | xargs rpmquery –qf %-30{NAME}:%{LICENSE}”\n

Here is a sample run for perl:

glibc                         :LGPL
perl                          :Artistic or GPL

Note that this is only for the linked files that are registered with RPM.

Code to Convert a MAC address to IPv6

The following code will convert an ethernet MAC address to the comparable  IPv6 link only address.

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <stdarg.h>

int main(int argc, char** argv){

int addrlen = strlen(“0000:0000:0000:0000:0000:0000:0000:”);
char* out = malloc(addrlen);
char * outorig = out;

memset(out, 0, addrlen);

char* addr =    “00:0c:29:20:4e:e3″;

if (argc > 1){
addr = argv[1];
}else{
fprintf(stderr,”usage %s macaddr\n”,argv[0]);
exit(-1);
}

int len = strlen(addr);

if (len > 18){
printf (“String too long\n”);
exit(-1);
}

int i ;
int col_count = 0;
unsigned char current = 0;
for (i = 0; i < len; ++i){
char c = addr[i];

if (0 == c){
break;
}else if (‘:’ == c){
switch( col_count ){
case 0:{
sprintf(out,”fe80::”);
out += strlen(out);

unsigned short c2 = ( current | 0×02 );
if (c2 == current){
c2 = current & 0xcf;
}
sprintf(out,”%02x”,c2);
out += strlen(out);
}
break;
case 2:{
sprintf(out,”%02xff:fe”,current);
out += strlen(out);
}
break;
default:
sprintf(out,”%02x”,current);
out += strlen(out);

if (col_count % 2){
sprintf(out,”:”);
out += strlen(out);
}
}
++col_count;
current = 0;
}else if ((c >= ‘a’) && (c <= ‘f’)){
current *=16;
current += ( 10 + c – ‘a’);
}else if ((c >= ‘A’) && (c <= ‘F’)){
current *=16;
current += ( 10 + c – ‘a’);
}else if ((c >= ’0′) && (c <= ’9′)){
current *=16;
current += ( c – ’0′);
}
}

sprintf(out,”%x”,current);
out += strlen(out);
printf(outorig);
return 0;

}