Code transfer

I finished moving all of the active source projects from sourceforge to google code to support our transitioning off the project.

The following subprojects have been successfully moved (GIPSEPoisonService, GIPSEService, GIPSEServiceInstaller, gmap-polygon, gridviewer were all moved prevously:

• GridMedlee: from sf to gc.


• PHGridLanding: from sf to gc.


• SecureSimpleTransfer: from sf to gc.


• gipse-dbimporter: from sf to gc.


• gipse-store: from sf to gc.


• gipse-poly-web: from sf to gc.


• loader-gmaps-poly: from sf to gc. (this includes the CSVs for settng up the gridviewer GIS tables)


• npds-gmaps: from sf to gc.


• npds-gmaps-web: from sf to gc.


• poicondai: from sf to gc.)


• schemas: from sf to gc. (this includes the schemas and example xml for the GIPSE services)

The sf projects will be left intact so as not to break any links, but all activity will be made on the google side from today onward.

More code migration

I just finished moving over two additional subprojects from our sourceforge repository to the newer google code repository.

From now on, please access gridviewer and gmap-polygon through google code. The sf version will stick around for a while to minimize inconvenience of repository switching, but eventually it will be replaced with just a pointer to the google code repository.

We're switching to google's site for a few minor reasons: 1) the issue tracking / wiki software is better. It lets you create pretty clean workflows through their tagging system; 2) the code review feature is useful; 3) their Subversion server is quite faster than sourceforge's; and 4) it's easy to switch, so we can switch back to sourceforge if it becomes better.

Note, we're not moving everything over to google code (yet) so there will be some time until all the subprojects come over.

Creating a new maven project: npdsamds

I have created a new maven project to hold the service end of the npds-to-amds code, and it is appropriately named npdsamds. It will essentially be a gridified version of the npds service that should be served up via a globus container (Thus requiring security and all that jazz).

Maven solves many worlds of hurt with its dependency management, and the repositories where you can store jars so that others may use them. It also makes other tools pale in comparison and seem much more annoying.

One of them is SVN, which for some reason is "old" on my copy of ubuntu and cannot be easily upgraded using the standard ubuntu tools. Thus, whenever I try and add new projects to the SVN repository or move files, I always have to create new folders and re-check-out the code because otherwise the version of subclipse I use with (you guessed it) eclipse touches the svn special secret hidden files and makes it so that lowly command-line subversion is eschewed from adding the files. I also have to do this every time that subclipse botches something.

The other tool is ant. Don't get me wrong, maven /uses/ ant. But maven is to ant as a chocolate cake is to baker's chocolate. The combination maven of which ant is a part, is much better than the bitter chunks of ant by itself. Ant offers all sorts of commands that rely on all sorts of duck-aligning and basically anyone who writes an ant script also has to write a very long doc about what each part of the script can do and how to invoke it. Maven, you can usually just tell someone "type 'mvn package'" and it will produce the thingy they need, and it is a LOT easier for the person who coded the stuff inside the maven package to set up the project that way.

Don't get me wrong, both tools are very powerful and useful in their own right. I just have a small personal bias in that maven hasn't made me curse it's name yet.
But, we might write a plugin that allows maven to build globus services. So, I'm sure I will be cursing it's name soon enough.

package changes and SVN

So, most of this morning was spent doing some data file massaging into CSVs, which were then loaded into the postgresql database.

It was sort of humbling, because my prior approaches were to build a big parser library and then spend another day or two debugging it. In this case, I just took the files, got a macro on them, and them told the database to import them. The result is I spent just 2/3rd of a day getting all the data I wanted in the right spot. The thing that will be bad is I'll have to do it all again (well, probably spend 1/3 of a day) when database updates need to be made, then again, I probably would have to tweak things in my loader scripts if they needed to be made, and it's still the same cost to the end user... and in fact probably less. Before it was download files, download code, adjust properties, package, build, run. Now it's download CSV, create table in database, import CSV into table in database. Yeah. It's easier now.

Otherwise, there are lists of relational data (all the zip3s in Nevada, all the zip5s in a given zip3, etc). Now I get to start building the code libraries that will serve it up as needed. Before I did that I had to do a package update... which involves changing one line in a lot of places.

Luckily eclipse handled the refactoring for me beautifully, and it terribly broke the subversion management of it all. Thus, I spent the rest of the day deleting out the appropriate files and committing the changes by hand.

Tomorrow is the implementation of the lists, and updating documentation for the loader and the list-libraries and, well, everything else.

Need almost coded.

So I got the PoiconDai-web piece coded... so now it is being gracious enough to put the chart above the table of data.

In Rodsadai-web, I hit a few snags service-enabling, so I will probably have to troubleshoot. Luckily, I took a snapshot of the working code and put it into a tag of SVN and just ported it back for a demo that is tomorrow.

I am really liking MVN and SVN.

Repositories and Builds and Tomcat, Oh My

Yesterday I was here for a while trying to get SVN to behave. I found that the "gov/cdc/phgrid/ogsadai/example" path was stuck in the root directory, and since I was trying to create a the same path inside the maven directory structure (under /main/src/java/, I think it was) it refused to add it there. Even after I removed the old gov structure. I was thinking about scouring the old repository, had Brian try a few things, and still hit roadblocks



Then I had the late arriving epiphany to just move the tree, and that worked, and it was good.



Then I gave Anurag a zip of my local repository files, so that he could create a public repository.



I also had a wonderful discussion with Jeremy about a RODS-OGSADAI interface. I had a bit of a paradigm shift, but in the meantime, I have a good idea for what sorts of use cases, test cases, and code I will need to develop. I have also taken up the task of finding out why Tomcat doesn't find the same security credentials that were find for command line operation, as that hurdle will need to be crossed eventually for happy JSP-based items pulling data from the grid.



Cheers, I hope everyone has a wonderful weekend!

SVN and Maven

I have checked in most of my code into the Sourceforge SVN, and I just got the note that we should be using the package gov.cdc.ncphi..... and not org.cdc...

I guess I get to learn about subversion directory changes next :).

Otherwise, I have hit a challenge on the maven-ifcation of the code. Maven prefers jar files stored in remote repositories somewhere. All the OGSA-DAI and Globus code is set up with a series of build files (especially globus considering it has a lot of C++ code in addition to Java code) but there is no maven repository. I experimented with moving jar files into a resource location, but I think I am just going to have to store the jar files in the local repository with the appropriate maven metadata. I am not sure of the best way to do that, whether there is some sort of tool, or whether I should try and get maven building the OGSA-DAI code.

I guess what I am looking for now is a resource for taking projects that already exist and getting it to a point where maven is now managing it, because a lot of the stuff that was handled by changing the classpath before now is going to be handled by formatting the myriad dependencies into an xml file.

I feel like in the end we will have the local repository and we will be able to publish it to our own repository so that other development groups can reference our files with the appropriate sets of Ogsa-Dai libraries and globus libraries.

I also need to start researching how to implement a globus WS extension and looking into the best ways to get data back and forth between RODS and Ogsa-Dai.