Rodsadai compiling again, server move giving familiar errors

This morning I tried adding the client-settings.wsdd file to the resources directory (not the server-settings.wsdd) and the rodsadai tests magically started working again (yay!)

The rest of the afternoon was spent setting up Ogsa-dai on it's new home, and having familiar cryptic errors thrown the first time I attempt to connect. Tomorrow I shall poke both more.

Jar hunt finished, maven repository updated, musical servers commencing

So, the jars have been included in the new repository (I had to spend some extra time generating some extra files that is necessary for remote repositories)

Right now, we are dealing with moving a server between VMs, at the end we are hoping to have an ubuntu globus server (the previous server was corrupted by attempts to install dual mysql databases).

Also, for some odd reason, the tests that I was so elated to have working yesterday... are now not working citing an "OGSA-DAI resource null is unknown" error. I am thinking it might be some sort of "the IP has changed" conflict... but at the same time the other OGSA-DAI examples are still working. Either way, I don't think I am going to be able to diagnose it today, and am out of ideas of what to try until we get the other VM set up to match this one.

File Transfer on the grid

Today, with the help of Dan we tried to recreate some functionality which the grid promises. It would be safe to say, we got pretty acceptable results. First we tried to transfer a file >100mb over the grid using just normal gridftp, it went through fine. The next step was to test the reliable file transfer. The way we decided to check the reliability was to power down one node while the file was transferring (to simulate a test case scenario). So Dan started the transfer, pulled the cord out for a minute and as soon as he plugged it back in, voila, it resumed as if nothing happened. Another thing, Dan informed me about was, that the timeout can be set by us as to how long we want to leave the file in the queue before it terminates the transfer. Right now it's set to 60 minutes, so we decided to increase the time and Dan will pull the cord before he leaves and plug it back in tomorrow morning. Now that would be a real extreme test of one facet of reliability. Another thing which we noticed was that upon powering back the node, it did not ask us to whether we want to resume or not, it just picked up automatically, which works for us because that ensures that as a user, once the transfer starts, you don't have to monitor whether the file reached or not and I am assuming that's the way PHINMS functions too.
Currently I am working on creating a test case document, to record all these cases and to outline what all are we going to achieve based on the requirements laid down by the project charter.

13 new jars

So... 13 new jars were added to the RODSAdai project in order to get the simple client test working (as opposed to just compiling). 2 of them were extra Ogsadai jars... and the other 11 were based in globus, and it took me all day to ferret them out.

I tried deploying the jars to our repository, but it seems that sourceforge doesn't like something because I kept getting 405 (method not allowed) errors, thus I have sent an email to Anurag to start the process with him since he set up the repository initially...

The other reason I sent him the jars and dependency info is because my main workstation is in the process of being cloned and I wanted to make an email-based backup.

Tomorrow will most likely be spent creating a working Ubuntu globus node, as my current one got corrupted.

Postgres Errors

The Postgres database on the staging node was failing to start the postmaster due to a recent IP address change. I updated the postgresql.conf file to reflect the recent IP change. Postgres started fine after the configuration file was modified.
Postgres version: 8.1
Original Postgres Error: autovacuum not started because of misconfiguration
Fix: Modified the /var/lib/pgsql/data/postgresql.conf


Tested 100+ MB file GridFTP and RFT file transfers with Anurag.

Going on a JAR hunt...

Today, I started beefing up the standard JUnit tests in RODSAdai... and remembered that a LOT more code is needed to run OGSA-DAI implementing clients than to just build it.

So, since the new test code runs an OGSA-DAI test... I am having a lot of attempts at "mvn compile", a test error or failure will result, so I check the log, find the class it needs, and make a reference to it, installing it in the maven repository if it was not in the global one, and repeating.

This will probably consume me tomorrow and maybe a bit of friday. I mean it when I say a lot of jars.

Spatial Series despite ambivalence

Today I put some thoughts down on creating a research-only node housed in places like developer's spare rooms... to be completely heuristic but also serve as a place to work on things when the CDC lab was inaccessible. I also found that the Ubuntu Globus node was corrupted for whatever reason (I think it was trying to get an independant MySQL server and VDT-installed globus to co-exist) so I started the process on creating a new node to rebuild and then transition-to.

I also crafted out how a spatial series could be loaded based on HL7 Table data and some of the constraints that would have to be defined (column order/naming, whether certain values were going to be parsed from the query, etc)

Tomorrow I am hoping I get some sample queries, test my spatial series processors, write a time series processor, and get some feedback on the constraints I have thought up (and how to make them flexible using properties)

Container issues on 1005

Globus node 1005 is failing to start the Globus container. I am currently troubleshooting the following error on node 1005:

: 55: ==: unexpected operator
.................................. ^[[31m*^[[39;49m
### 2008-06-24 10:09:56 vdt-control(do_init) enabling the init service 'globus-ws'
### 2008-06-24 10:09:56 vdt-control(do_init) starting globus-ws: /etc/init.d/globus-ws start
### 2008-06-24 10:09:56 vdt-control(system) /etc/init.d/globus-ws start
[: 55: ==: unexpected operator
WARNING: It seems like the container died directly
Please see $GLOBUS_LOCATION/var/container.log for more information
Starting Globus container. PID: 7047
### 2008-06-24 10:10:06 vdt-control(do_init) starting 'globus-ws' failed: 1024
### 2008-06-24 10:10:06 vdt-control(clean_up) all done

medLEE NLP Grid Service Progress

Dr. Albert Lai of Columbia CoE, on June 18, wrote...

Yes, we were able to develop a functioning wrapper service for MedLEE. We
ended up wrapping a batch job version of MedLEE instead of the client
server version.

We used Introduce, a piece of software developed by Shannon, which created
most of the stubs necessary to deploy software onto the grid. However,
since this used the Java core of Globus and not the C core currently being
used in the PHGrid efforts.

This leaves the current service unsecured. One of the steps we still need
to do is to somehow figure out how to integrate the certificates and all of
the authentication/authorization into the Java core.

There were a number of problems launching MedLEE from Java. However, I
have hacked together a (suboptimal) solution to make it run. There are
some other limitations of the version that we have put together, but it
seems to be sufficient for any demo needs. Right now, we package files and
transfer them to the server in a way that would limit the size of the data
transfer to available memory of the process as it is my understanding that
the files are serialized and then transferred. There are also some other
specific options that we currently do not support via the grid.

-Albert

Now we just need to work on getting the grid service deployed on the public health research grid.

Project Management update

We held the a project kickoff meeting for the CDC and Contractor resources who will be working on the Secure Reliable Grid Messaging PoC. We'll be meeting over the following few days to get together a set of detailed requirements and tasks based on the project charter. Following from this will be timelines and resources on the tasks so we can establish more fine grained expectations.

I met with Tom Brinks this afternoon and we started working on a work breakdown structure to define the functional areas within the project. We'll post a draft on the wiki as soon as something digital exists.