Globus Toolkit 5.0 released

Just retweeting that the GT5.0 is now released. Although a big development over 4.0.x and 4.2.x, this doesn't include all the cool stuff that Crux will bring.

Although the community has pledged support for the 4.x releases through (at least) the end of 2010, there is a recommendation to upgrade to 5.0.0 immediately.

Correcting Globus Handshake Errors

The Problem:
When Globus-WS is deployed to Tomcat 5.5.x within Windows, a Handshake error is thrown when secure Globus commands are issued. The secure commands run fine in a standalone container but fail when Globus is deployed to Tomcat.

Example:
C:\>counter-client -m conv -z none -s https://192.168.20.120:8443/wsrf/services/Se
cureCounterService

Error: ; nested exception is:
javax.xml.rpc.soap.SOAPFaultException: ; nested exception is:
org.globus.common.ChainedIOException: Authentication failed [Caused by:
Failure unspecified at GSS-API level [Caused by: Handshake failure]]

The Solution:
(Commands are based on JDK 1.5.x)

This is caused when the SSL client does not trust the CA that signed the certificate. The solution is to add the CA certificate as a trustedCA.


1. Create a Java Key Store:

keytool -genkey -alias servercert -keyalg RSA -dname "CN=Your_host_name, OU=yoursite.net, O=your_organization, L=city, ST=state C=country" -keypass changeit -keystore server.jks -storepass changeit

2. Create a PKCS12 Keystore:

keytool -genkey -alias globus -keystore globus.p12 -storetype pkcs12 -keyalg RSA -dname "CN=Your_host_name, OU=yoursite.net, O=your_organization, L=city, ST=state C=country" -keypass changeit -storepass changeit

3. Export your PKCS12 Keystore:
keytool -export -alias globus -file globus.cer -keystore globus.p12 -storetype pkcs12 -storepass changeit

4. Import your PKCS12 Ketstore file into you Java Keystore:

keytool -import -keystore server.jks -alias globus -file globus.cer -v -trustcacerts -noprompt -storepass changeit

5. Import the 3rd Party CA into your Java Keystore as a Trusted CA

keytool -import -keystore server.jks -alias globusCA -file c:\etc\grid-security\certificates\31f15ec4.0 -v -trustcacerts -noprompt -storepass changeit

6. Import the host certificate issued by the 3rd Party CA into your Java Keystore.

keytool -import -keystore server.jks -alias containercert -file c:\etc\grid-security\importcontainercert.pem -v -trustcacerts -noprompt -storepass changeit

Based on the proceedure above, your server.xml file should look like this:

className="org.globus.tomcat.coyote.net.HTTPSConnector"
port="8443" maxThreads="150"
minSpareThreads="25" maxSpareThreads="75"
autoFlush="true" disableUploadTimeout="true"
scheme="https" enableLookups="true"
acceptCount="10" debug="0"
protocolHandlerClassName="org.apache.coyote.http11.Http11Protocol"
socketFactory="org.globus.tomcat.catalina.net.BaseHTTPSServerSocketFactory"
keystoreFile="C:\apache-tomcat-5.5.27\conf\server.jks"
keystorePass="changeit"
cacertdir="c:\etc\grid-security\certificates"
encryption="true"/>

---------------------------------------------------------
The commands change slightly when using JDK 1.6.x.

1. Create a Java Key Store:

keytool -genkeypair -alias servercert -keyalg RSA -dname "CN=Your_host_name, OU=yoursite.net, O=your_organization, L=city, ST=state C=country" -keypass changeit -keystore server.jks -storepass changeit

2. Create a PKCS12 Keystore:

keytool -genkeypair -alias globus -keystore globus.p12 -storetype pkcs12 -keyalg RSA -dname "CN=Your_host_name, OU=yoursite.net, O=your_organization, L=city, ST=state C=country" -keypass changeit -storepass changeit

3. Export your PKCS12 Keystore.

keytool -exportcert -alias globus -file globus.cer -keystore globus.p12 -storetype pkcs12 -storepass changeit

4. Import your PKCS12 Ketstore file into you Java Keystore.

keytool -importcert -keystore server.jks -alias globus -file globus.cer -v -trustcacerts -noprompt -storepass changeit

5. Import the 3rd Party CA into your Java Keystore as a Trusted CA.

keytool -importcert -keystore server.jks -alias globusCA -file c:\etc\grid-security\certificates\31f15ec4.0 -v -trustcacerts -noprompt -storepass changeit

6. Import the host certificate issued by the 3rd Party CA into your Java Keystore.

keytool -importcert -keystore server.jks -alias containercert -file c:\etc\grid-security\importcontainercert.pem -v -trustcacerts -noprompt -storepass changeit

building! testing! Deploying?

So, I got a list of the jars (which can be found in the $GLOBUS_LOCATION$\lib directory after an introduce install) which grid-viewer now needs to run the GIPSE client:

addressing-1.0.jar
caGrid-metadata-security-1.3.jar
caGrid-ServiceSecurityProvider-client-1.3.jar
caGrid-ServiceSecurityProvider-stubs-1.3.jar
caGrid-ServiceSecurityProvider-common-1.3.jar
axis.jar (this has to be the version that is shipped with globus)
jaxrpc.jar
saaj.jar
cog-axis.jar
cog-jglobus.jar
commons-logging.jar
commons-discovery.jar
wss4j.jar
wsdl4j.jar
wsrf_core.jar
wsrf_core_stubs.jar
jce-jdk13.jar (and this one seems to have those pesky bouncycastle.org security libs)
puretls (*for secure access*)
cryptix32 (*for secure access*)
cryptix-asn1 (*for secure access*)

These are stored in my private repository and downloaded by maven because they are classified as "provided" in the POM (which means they are needed for compilation and testing, but will be provided in the classpath when the war is installed). Thus, I copied them all over to the tomcat/commons/lib directory.

Initial attempts seem to be resulting in NoClassDefFound errors. So it seems there may be a few jars needed still in tomcat alone... or that tomcat is not properly loading the jars in the common space (and that might be the case, I remember RODSA-DAI having issues running in tomcat becuse of library/classpath issues).

Either way, I am hoping to get it sorted out early tomorrow, have it returning data to the map. Then I want to get it tested on a secure-globus environment (in case some more security libraries are needed and to make sure that it plays nice with other secure clients like RODSA-DAI), and then I'll be ready to continue with the refactors.

Right now, the main refactorings have to do with shifting from a "state, zip3, zip5" paradigm to a "region" paradigm. This should allow for showing zip3s and zip5s and states all on the same map. A complimentary paradigm shift will be allowing multiple loads on one map (load one query with one set of pushpins, load another query with another set of pushpins... so you can click both sets of pushpins and compare data from two queries on the same map). All with more services and more realtime options.

Services and Deploys.

So today I did manage to get a secure service and a secure client up and deployed in my current environment, and while it is a bit of a pain, it shouldn't be insurmountable, just tedious. I look forward to seeing if Vaughn found a better way to adjust the service parameters as AMDS is probably going to be very dynamic.

Tomorrow will be adjusting some of the properties by hand (namely the security parameters and such) and trying to make a little website off the client (like I did with rodsadai) just so I can see what happens when I deploy things alongside rodsadai and what libraries have to change (with Felicia's help).

I know this is going to be rather heuristic. But I didn't learn Globus that well before, and It'll help me sew together all the little bits about how all these services are going to bump into each other, and help me get a better perpective about the appliance network we are going to be making.

Whether introduce helps...

So, I have Introduce on my development box, but whenever I try and run the tutorial, it blew up in the very first step with an odd error bubbled up from ant about some method not working properly.

I subscribed to the introduce user group mailing list, asked my question and posted my error, and the response was "You must have used Introduce in the past and are now trying to use a new version of introduce, since introduce modifies the Globus libraries when it runs, you will need a fresh copy of Globus to use a new version of introduce or you get library conflicts"

That is kind of a dealbreaker for me. I am trying to use Introduce for something other than it's intended purpose: to build regular grid services and not caGrid specific services... so it appears that some of the changes that Introduce makes for caGrid (like mutating a Globus installation) are more than I bargained for. Also, introduce is supposed to be something that simplifies the tedious process of populating all the nooks and crannies of a given grid service, and little is more tedious than scrubbing a Globus install.

Vaughn is currently working with a different Eclipse based tool, so another thing I am batting my head against is trying to make something that he, too, will be making: An AMDS service shell. Vaughn is a bit further ahead than me and has already made the Java interface and converted all of the XSDs into Java objects, so if he gets a Grid service that matches that interface with security, I'll probably use that and implement the interface for Poison Data while he implements it for RODS data.... and then start making the client that looks like quicksilver but in fact pulls AMDS data rather than NPDS data.

The other thing that Vaughn has been talking about is maintaining that firefox/apple/google model of easy to install-ness with easily pluggable extensions. Thus, we want to make sure that our service is easily installable or even included on our default grid node we hand out... which makes me start wondering about whether we even want multiple AMDS services or just one AMDS service with multiple return options. (IE, instead of having Poison-AMDS and Rods-AMDS as I am envisioning it, do we want to just have one AMDS that returns two sets of metadata, one for Poison and one for Rods, and then however many extra sets depending on how many other data services are networked behind the scenes, and these questions might have been answered already and I just missed it).

Either way, I am sort of a mass of ambivalences right now. Introduce probably won't work for me, so I am wondering whether I want to try and modify a service from scratch (using the interface that Vaughn already created), do I want to play with the system Vaughn is using (or watch over his shoulder and help) or do I want to start refactoring npdsgmaps and npdsgmaps-web to deal with AMDS style metadata and queries. Or do I just want to just focus on getting rodsadai and the old AMDS client running on the same jboss without tripping over each other since there will probably be multiple clients and they'll need to behave.

It's not unpleasant, but it is a bit frantic. Oh well, ambivalence means I have lots of ideas, and the next step is figuring out which are best or most appealing.

Cheers.

Minor deploy, and Introduce

So, the next step for Quicksilver is to make an AMDS-intermediary layer that is a secure Globus service. Thus, tonight I am going to be going over some of the GRID/Globus books again and tomorrow I will probably install introduce and run the tutorial.

The reason I want to read the grid books is so I can see what goes into creating a grid service so that I can see what the introduce client will do for me. Then, I can start better understanding the heuristics of making an intermediary layer for poison data showing up in AMDS web.

Oh yeah, I have to go over the AMDS schema as well.

The end result is that the viewer part of Quicksilver, will end up becoming more of a generic AMDS viewer. Thus, one picks (a) secure service(s) that return(s) AMDS data, fills out the form that it/they require(s) (based on retrieved metadata for selected services), compiles the results into some sort of mappable data, and displays it in the map... and one of those secure AMDS services will be poison data.

One of the concerns I had was that making the NPDS->AMDS converter secure would be completely heuristic... then I remembered that people have to be given logins to be able to access poison data, and it would be much easier if they just were able to access it if they had a node on the grid.

So yeah, big context switch over the next couple of days, expect flagellating expectations as I realize the can, can't, and how.

Also, I ported a bunch of the improvements I made to npdsgmaps-web (the viewer part of quicksilver) back to gmap-poly-web (a demonstration library made on top of Gmap-Polygon). That should be deployed to training tomorrow so the two apps act the same (and because the version currently on training doesn't use encoded polygons, and is therefore much slower and error prone).

Cheers!

VMWare Grid node Distro

The next version of the VMWare grid node appliance will not be a VDT installation of Globus. It has been converted to a native Globus installation.

This has been done for two reasons. The first reason is to reduce the overall disk space requirements for the VM appliance. The second reason is to distribute a model that closely resembles the NCPHI installation of Globus.

Please keep in mind that this change does not reduce grid functionality. The original VDT installation was intended to reduce the complexity of installing a grid node. If we are distributing a DVD with a fully installed grid node, it serves no purpose for it to be a VDT installation.

This change will allow the users to research issues based on a standard Globus installation. I believe this move will reduce configuration issues as new grid services come online.

VMWare Appliance Now available online

Dan made a VMWare Appliance of the PHGrid Globus node and it's available at http://phgrid.net/documents/node/v1/.

If you download both of the files here, you can run a SUSE installation of the Globus Toolkit. This will make configuration easier as all you have to do is make a few configuration changes(set the IP, hostname, user accounts, request a user and host cert, etc).

This will reduce the install time down even lower (5 minutes) and allow PHGrid nodes to run on Windows (through the free VMWare player).

This first release is still pretty large (4.5GB download, 8GB necessary to run) so it may take you a while to download (took me over 30 minutes to upload).