Dueling for Certificates.

So, It appears that the problem between Rodsadai-web and Amds-web that they are playing musical chairs for the client-certificate.

Felecia (who is developing AMDS-Web) has been helping me debug. We were able to replicate the issues outside of staging on my dev node (handy, since debugging these involves a lot of jboss-restarts). It appears that it is just a mutual exclusion issue: If you started AMDS-web first, it will be able to marshal the needed security credentials, and when Rodsadai-web tries to marshal the credentials, it will not be able-to and the connection to a remote globus server will fail. The behavior is the same in reverse. If the rodsadai-web client is called first, it works fine and then AMDS-web will throw a connection error.

Now, we are not sure if the failure to marshal is because of slightly different needed credentials that are being cached (apparently Rodsadai-web is using tramsaction layer security (TLS) with only privacy and AMDS-web is using TLS with privacy and integrity) or whether there is some sort of explicit locking behavior that prevents different applications from using the same set of proxies.

Either way, it reveals that we need to find a better way for clients to make secure connections to Globus services. Whether it involves some sort of Server-level pooling or having the apps load application-specific certificates, the current method would not allow client applications with slightly different needs to coexist on the same server (or worse, not even allow client applications with the exact same needs to coexist).

Thus, we plan to delve into the innards of globus security to see whether the certs need to change or how they are being marshalled. (Probably both).