So, today I spent some time prepping for the poicondai loader load of a lot of zip code polygons into a database.
But a lions share of the day was spent setting up the shiny new secure tomcat installation on the staging node. Also known as "boy, I love being able to configure different ports!"
So, the window for taking down the node and updating RODSAdai to call secure tomcat is tomorrow... but that didn't stop us from making sure we couldn't set up secure globus and make some ogsadai calls to it. That took a little bit of time mainly because the tarball I set up apparently broke or was not happy in it's new environment, so we had to build a new one from scratch.
Many thanks to Felicia and Dan for pretty much doing most of the footwork before me so all I had to do was go "get this, put that there, lemme check... yay!" Dan was awesome with the configuration and Felicia knew most of this stuff from having to deal with it before a couple of times.
Tomorrow is loading polygons, updating build styles, and starting the poicondai-web modifications.
Showing posts with label OGSA-DAI. Show all posts
Showing posts with label OGSA-DAI. Show all posts
Monday, November 17, 2008
Monday, October 6, 2008
Planning and such.
So, RODSAdai and Poicondai are documented, fleshed out, and there ready for people to consume and ask-questions about if people want to recreate them or install them in new places.
The next step is to essentially re-create the OGSA-DAI end of RODSAdai using CaBIG's Introduce framework. Introduce, for all intents and purposes, seems to be really really good at generating a full-out service skeleton and having the tools to package up everything into a .gar file (which is the deployable for a globus toolkit server).
Thus, instead of OGSA-DAI at a given location accepting a query, this will be parameters passed into a webservice. Whether those parameters will be a query or just some filter values will be a discussion for Jeremy (because if they are just filter values, it becomes that much easier to prevent against SQL Injection attacks by using a prepared statement). Also, it will be interesting to see how well parameterized the connection logic could be and whether it would be as universally installable as ogsadai (it probably would not be as versatile).
Otherwise, we haven't really investigated the Dallas slowness at this point, I will also have to ask Jeremy if that Pitt node has been set up or not. I have a few ideas for what to try, but we will see.
The next step is to essentially re-create the OGSA-DAI end of RODSAdai using CaBIG's Introduce framework. Introduce, for all intents and purposes, seems to be really really good at generating a full-out service skeleton and having the tools to package up everything into a .gar file (which is the deployable for a globus toolkit server).
Thus, instead of OGSA-DAI at a given location accepting a query, this will be parameters passed into a webservice. Whether those parameters will be a query or just some filter values will be a discussion for Jeremy (because if they are just filter values, it becomes that much easier to prevent against SQL Injection attacks by using a prepared statement). Also, it will be interesting to see how well parameterized the connection logic could be and whether it would be as universally installable as ogsadai (it probably would not be as versatile).
Otherwise, we haven't really investigated the Dallas slowness at this point, I will also have to ask Jeremy if that Pitt node has been set up or not. I have a few ideas for what to try, but we will see.
Saturday, September 13, 2008
RODSAdai, PoiConDai, and NPDS-ws
Today I started off by having a productive discussion with the folks who helped me with RODSAdai: Dr. Jeremy Espino from Pitt (who brings the RODS) and Mario Antonioletti and Alastair Grant from University of Edinburgh (who bring the OGSA-DAI).
We discussed future developments, concerns about security, scalability, and automating installation and configuration as much as possible. We also talked about our availability being reduced for the next phases... But mainly we talked about the fact that one of the things that got lost in the making of RODSAdai 1.0 was a quick and descriptive design document that explained what we were all trying to do and how. Something so that Alastair, Mario, or Jeremy would be able to go "oh, well to connect it that way you should try configuring it like this..." Furthermore it is much more likely that the whole Axis/Globus conflict would have been found earlier and mitigated sooner since Alastair was already familiar with that particular limitation. Thus, the first step for the next phase of RODSAdai is to take the ideal-state and current-state diagrams of RODSAdai that I made, and more information about what they mean, how they currently work, and how I hope to get them working to focus advice and let anyone search for red flags.
But I won't be able to do that until Thursday of next week, because right now I am in demodemodemo mode to get PoiConDai in some form of pretty graphical display so we can show that yes, we are pulling Poison Center call data into the CDC.
This was greatly moved forward this morning with the help of Dr. Espino, namely because he already wrote code that used AXIS to consume the NPDS webservice a couple of weeks ago and had already incorporated all of the JAXB libraries needed for parsing the "any" response returned by the web service. The codebase is called npds-ws-client, and I am going to use it because it is pretty and is along the lines of what I was going to do, only with the properties file and return serializing I was planning on doing already implemented (and thank goodness for it).
Let me pause here to say how much I love maven again. It has extensions for generating axis stubs and jaxb code, so all you need to do is have the WSDLs handy and you don't have to worry about constantly regenerating the parsing bits and movig them about manually. How cool is that!?
Meanwhile, my next step is to incorporate a few of those unit tests I posted about earlier, and then set up a web-client front end that I am going to call PoiConDai web for showing the data in an ever-snazzier JSP. I think I am going to get something really nifty ready by Tuesday Evening.
We discussed future developments, concerns about security, scalability, and automating installation and configuration as much as possible. We also talked about our availability being reduced for the next phases... But mainly we talked about the fact that one of the things that got lost in the making of RODSAdai 1.0 was a quick and descriptive design document that explained what we were all trying to do and how. Something so that Alastair, Mario, or Jeremy would be able to go "oh, well to connect it that way you should try configuring it like this..." Furthermore it is much more likely that the whole Axis/Globus conflict would have been found earlier and mitigated sooner since Alastair was already familiar with that particular limitation. Thus, the first step for the next phase of RODSAdai is to take the ideal-state and current-state diagrams of RODSAdai that I made, and more information about what they mean, how they currently work, and how I hope to get them working to focus advice and let anyone search for red flags.
But I won't be able to do that until Thursday of next week, because right now I am in demodemodemo mode to get PoiConDai in some form of pretty graphical display so we can show that yes, we are pulling Poison Center call data into the CDC.
This was greatly moved forward this morning with the help of Dr. Espino, namely because he already wrote code that used AXIS to consume the NPDS webservice a couple of weeks ago and had already incorporated all of the JAXB libraries needed for parsing the "any" response returned by the web service. The codebase is called npds-ws-client, and I am going to use it because it is pretty and is along the lines of what I was going to do, only with the properties file and return serializing I was planning on doing already implemented (and thank goodness for it).
Let me pause here to say how much I love maven again. It has extensions for generating axis stubs and jaxb code, so all you need to do is have the WSDLs handy and you don't have to worry about constantly regenerating the parsing bits and movig them about manually. How cool is that!?
Meanwhile, my next step is to incorporate a few of those unit tests I posted about earlier, and then set up a web-client front end that I am going to call PoiConDai web for showing the data in an ever-snazzier JSP. I think I am going to get something really nifty ready by Tuesday Evening.
Wednesday, August 13, 2008
Axis problem has functioning workaround
To address the axis mismatch problem (described here), we developed a separate RODSAdai download servlet that can be called by the RODS application to return either a TimeSeries or SpatialSeries object serialized over HTTP.
Jeremy has tested this in his Pittsburgh environment and it is working.
Mario also mentioned a tool called Jar Jar Links (http://code.google.com/p/jarjar/) that may be able to help. We'll investigate this once the time is found.
So good news is that we're no longer blocking on the custom Axis 1.2RC2 that Globus uses.
Jeremy has tested this in his Pittsburgh environment and it is working.
Mario also mentioned a tool called Jar Jar Links (http://code.google.com/p/jarjar/) that may be able to help. We'll investigate this once the time is found.
So good news is that we're no longer blocking on the custom Axis 1.2RC2 that Globus uses.
Tuesday, August 12, 2008
Axis mismatch is hurting RODSAdai
Dr. Espino discovered a pretty serious bug with how we are planning on using Globus that is preventing the RODSAdai code from running within the same war as RODS.
It comes down to Globus 4.0.5 using a customized version of Apache Axis 1.2RC2 (modified to specifically support Globus services). RODS (like many modern Java applications) also uses Axis. RODS specifically uses version 1.4 of Axis. The version of Axis used by Globus conflicts with the version of Axis used by RODS and is causing the RODSAdai code to fail.
The idea of RODSAdai is that it is a jar that can be included in RODS (or other Java applications) that calls out to OGSA-DAI services on the Public Health Research Grid. So it is beneficial to have the RODSAdai code be as drop in and easy to use as possible.
Jeremy and I had a call with the OGSA-DAI folks (Alistair and Mario) where they basically helped us work out that this is a limitation of Globus, not necessarily of OGSA-DAI. Mario is taking this question to the Globus community to ask how they typically deal with this situation.
In the meantime, Jeremy and I came up with a hack that should work for demo purposes until we get a proper solution from the Globus folks. We're running separate wars for RODS and RODSAdai. This will allow RODS to use the Axis 1.4 jar and RODSAdai to use the Axis 1.2RC2 jar that Globus/OGSA-DAI requires. The hack is that the two wars need to communicate with a HTTP based IPC. Not beautiful, but we'll see if it works.
I added a new servlet to RODSAdai to support this and Jeremy will test it out in his environment later today. Updates will be posted here.
We also found out from OGSA-DAI that they are working on a version that will work with Globus 4.2. Which should come in handy as Dan begins our 4.2 transition testing and planning.
It comes down to Globus 4.0.5 using a customized version of Apache Axis 1.2RC2 (modified to specifically support Globus services). RODS (like many modern Java applications) also uses Axis. RODS specifically uses version 1.4 of Axis. The version of Axis used by Globus conflicts with the version of Axis used by RODS and is causing the RODSAdai code to fail.
The idea of RODSAdai is that it is a jar that can be included in RODS (or other Java applications) that calls out to OGSA-DAI services on the Public Health Research Grid. So it is beneficial to have the RODSAdai code be as drop in and easy to use as possible.
Jeremy and I had a call with the OGSA-DAI folks (Alistair and Mario) where they basically helped us work out that this is a limitation of Globus, not necessarily of OGSA-DAI. Mario is taking this question to the Globus community to ask how they typically deal with this situation.
In the meantime, Jeremy and I came up with a hack that should work for demo purposes until we get a proper solution from the Globus folks. We're running separate wars for RODS and RODSAdai. This will allow RODS to use the Axis 1.4 jar and RODSAdai to use the Axis 1.2RC2 jar that Globus/OGSA-DAI requires. The hack is that the two wars need to communicate with a HTTP based IPC. Not beautiful, but we'll see if it works.
I added a new servlet to RODSAdai to support this and Jeremy will test it out in his environment later today. Updates will be posted here.
We also found out from OGSA-DAI that they are working on a version that will work with Globus 4.2. Which should come in handy as Dan begins our 4.2 transition testing and planning.
Monday, July 21, 2008
RODSAdai on Jboss Worked!
So, this morning I finished up installing JBoss-4.2.2, got it running and then transferred over the Rodsadai-war from the maven target directory into the JBoss deploy directory.
Then, I went to the server endpoint, and tried running my little test JSP. And it worked! As simple as that. It didn't throw errors, it didn't say anything about failed path building, it just ran and gave me the output I wanted when I first tried all this Thursday week before last. So now I know it works and can start being integrated with RODS.
The downside of this, is that it's yet another server. Fortunately, I talked to Jeremy and he already had RODS running in a few JBoss instances, so it will not be an obstruction of his coding plans to say "this needs to run in JBoss to work"... going "RODS needs to become a gridsphere portlet" would be much less palatable. It might require some musical ports if tomcat and jboss need to run on the same server, but honestly the secure OGSA-DAI we are running would be in the globus container and not in tomcat, and other tomcat applications can run on the tomcat within jboss.
That doesn't mean I didn't spend a lot of time today trying to get tomcat to work securely based on how JBoss' tomcat did it. It appears that JBoss' tomcat uses an entirely different realm, passing itself to the security engine that JBoss manages. I tried pulling jars from JBoss into standalone tomcat and adjusting the realm and started getting all sorts of "hey, you're trying to run me outside of JBoss type errors and library failures."
Maybe Globus already has some sort of realm tweak you can perform for tomcat that makes it point to the globus container and security manager. Maybe one can be designed. Then again, maybe saying "it needs to run in JBoss" won't be a fatal problem.
Then, I went to the server endpoint, and tried running my little test JSP. And it worked! As simple as that. It didn't throw errors, it didn't say anything about failed path building, it just ran and gave me the output I wanted when I first tried all this Thursday week before last. So now I know it works and can start being integrated with RODS.
The downside of this, is that it's yet another server. Fortunately, I talked to Jeremy and he already had RODS running in a few JBoss instances, so it will not be an obstruction of his coding plans to say "this needs to run in JBoss to work"... going "RODS needs to become a gridsphere portlet" would be much less palatable. It might require some musical ports if tomcat and jboss need to run on the same server, but honestly the secure OGSA-DAI we are running would be in the globus container and not in tomcat, and other tomcat applications can run on the tomcat within jboss.
That doesn't mean I didn't spend a lot of time today trying to get tomcat to work securely based on how JBoss' tomcat did it. It appears that JBoss' tomcat uses an entirely different realm, passing itself to the security engine that JBoss manages. I tried pulling jars from JBoss into standalone tomcat and adjusting the realm and started getting all sorts of "hey, you're trying to run me outside of JBoss type errors and library failures."
Maybe Globus already has some sort of realm tweak you can perform for tomcat that makes it point to the globus container and security manager. Maybe one can be designed. Then again, maybe saying "it needs to run in JBoss" won't be a fatal problem.
Tuesday, July 15, 2008
Portals, they have all that cert management, but will they wrap a JSP easily?
I had some more discussions with some of the folks at OGSADAI and Pitt, In addition to some reviews with Brian. After some more eyes and some more cryptic errors, it looks like a portal might be the quick solution for getting all of the security managed.
The benefits include the ability to easily set up authentication and authorization schemes on the user end... and tie them into the authentication and (eventually) authorization schemes in the globus grid. It would also tie in easily with applications like PURSe.
The drawback is that RODSAdai and RODS might have to be modified to deal with the portal... especially things like P-Grade which is more focused on workflow registry and execution.
Right now the frontrunner looks like Gridsphere, but we are also thinking of other things to try (attached debugging of the web project) to see if we can get secure tomcat clients working.
The benefits include the ability to easily set up authentication and authorization schemes on the user end... and tie them into the authentication and (eventually) authorization schemes in the globus grid. It would also tie in easily with applications like PURSe.
The drawback is that RODSAdai and RODS might have to be modified to deal with the portal... especially things like P-Grade which is more focused on workflow registry and execution.
Right now the frontrunner looks like Gridsphere, but we are also thinking of other things to try (attached debugging of the web project) to see if we can get secure tomcat clients working.
Thursday, July 10, 2008
RODSAdai Webapp not able to find certification path
Today was pretty much all day trying to hammer against tomcat and getting it to recognize the globus certificates for clients... and I have no idea where such things are controlled within tomcat, and the things I tried didn't fix the problem.
Thus, the "the secure client won't run through tomcat" problem is back, and I still am not sure how to tell tomcat "use these keystores that exist within globus when invoking this client which connects to a secure globus server". The error that is thrown is the familiar "unable to find certification path"
I really have two problems: The first is that I have no idea how to get tomcat to use a keystore other than the default java one for client programs. Ideally it should be made-to pick up the certificates and look up the proxies like the command line clients... something about the tomcat container prevents security policies from being overwritten in the same way they are within the command line clients.
The second problem is that there is tons of stuff written about how to enable tomcat as a secure ogsadai/wsrf server, and it is blocking out google searches of how to enable tomcat to run secure clients which access other globus/wsrf servers with things other than the default keystore.
I have tried several things which haven't worked:
1. Moving client-config.wsdd into the WEB-INF/classes/ directory of the web-app.
2. Modifying server.xml in [tomcat-root]/conf to use the globus certificates for secure access as described by this website: 'http://www.gridlab.org/WorkPackages/wp-5/guide/axis.html#install' and it was really more of an old way of setting up some sort of globus ws server with an old version of globus.
3. importing cog-tomcat.jar into the libs directory (of the webapp and the server), after finding that cog-tomcat seemed to hold several of the security handlers...
I am out of ideas at the moment... Dr. Jeremy Espino is working on setting up his globus/ogsadai environment and then he'll tinker with it too.
I figure there is a workaround to write clients that explicitely invoke the globus security elements... but I imagine I will have to pore over acres of ogsa-dai and wsrf source code to set such things up and any resulting code might be locked to very specific versions of ogsadai and/or globus.
I just feel like there is some "oh, you just set this property in the config file for the webapp" thing I don't know exists... I just have no idea where that is and my attempts to look for it keep getting flushed out by examples showing me how to enable https on port 8443.
Thus, the "the secure client won't run through tomcat" problem is back, and I still am not sure how to tell tomcat "use these keystores that exist within globus when invoking this client which connects to a secure globus server". The error that is thrown is the familiar "unable to find certification path"
I really have two problems: The first is that I have no idea how to get tomcat to use a keystore other than the default java one for client programs. Ideally it should be made-to pick up the certificates and look up the proxies like the command line clients... something about the tomcat container prevents security policies from being overwritten in the same way they are within the command line clients.
The second problem is that there is tons of stuff written about how to enable tomcat as a secure ogsadai/wsrf server, and it is blocking out google searches of how to enable tomcat to run secure clients which access other globus/wsrf servers with things other than the default keystore.
I have tried several things which haven't worked:
1. Moving client-config.wsdd into the WEB-INF/classes/ directory of the web-app.
2. Modifying server.xml in [tomcat-root]/conf to use the globus certificates for secure access as described by this website: 'http://www.gridlab.org/WorkPackages/wp-5/guide/axis.html#install' and it was really more of an old way of setting up some sort of globus ws server with an old version of globus.
3. importing cog-tomcat.jar into the libs directory (of the webapp and the server), after finding that cog-tomcat seemed to hold several of the security handlers...
I am out of ideas at the moment... Dr. Jeremy Espino is working on setting up his globus/ogsadai environment and then he'll tinker with it too.
I figure there is a workaround to write clients that explicitely invoke the globus security elements... but I imagine I will have to pore over acres of ogsa-dai and wsrf source code to set such things up and any resulting code might be locked to very specific versions of ogsadai and/or globus.
I just feel like there is some "oh, you just set this property in the config file for the webapp" thing I don't know exists... I just have no idea where that is and my attempts to look for it keep getting flushed out by examples showing me how to enable https on port 8443.
Tuesday, July 8, 2008
RODSAdai secure access from in the test context
Today I ran into the familiar syndrome... lovingly known as "But it works in the ogsadai directory".
So, secure access of the RODS database does work from the ogsadai directory... which has a setenv.sh file that sets the java classpath up with all sorts of jars and property files from globus and ogsadai.
However, when I try to make a secure call from a test context... I get the dreaded error of "unable to find valid certification path to requested target".. I am pretty sure this means that I need to include some more jars from the globus context, hopefully those jars will have something that imports the globus security infrastructure. If it does, then it might very well be included in the war file that eventually gets moved to tomcat and it will solve the "secure access doesn't work from tomcat" problem.
Many thanks to Dr. Jeremy for helping me get over my vacillations on what to try next. My next step will be to get the test for secure access running. Then I will start working on getting these clients called securely from within tomcat. Dr. Espino pointed out that everything should test and pass before moving on to other projects/discoveries... and if it's going to take some painful jar hunting to get it working, then so be it.
So, secure access of the RODS database does work from the ogsadai directory... which has a setenv.sh file that sets the java classpath up with all sorts of jars and property files from globus and ogsadai.
However, when I try to make a secure call from a test context... I get the dreaded error of "unable to find valid certification path to requested target".. I am pretty sure this means that I need to include some more jars from the globus context, hopefully those jars will have something that imports the globus security infrastructure. If it does, then it might very well be included in the war file that eventually gets moved to tomcat and it will solve the "secure access doesn't work from tomcat" problem.
Many thanks to Dr. Jeremy for helping me get over my vacillations on what to try next. My next step will be to get the test for secure access running. Then I will start working on getting these clients called securely from within tomcat. Dr. Espino pointed out that everything should test and pass before moving on to other projects/discoveries... and if it's going to take some painful jar hunting to get it working, then so be it.
Monday, July 7, 2008
Secure traffic for rodsadai
Today we got Secure traffic for rodsadai working. Dan helped with several of the hostname issues (many thanks to him for that) and I got the client factory to pass the secure client instead of the generic sql client.
Tomorrow, I will be adjusting all of the tests to take into account proper spatial and time series formatting, and probably do a bit of re-architecting to turn secure access into a separate test (this will most likely involve passing the query client into the processor as opposed to having the processor call its own).
That and documenting, putting in descriptive errors, that sort of thing.
Tomorrow, I will be adjusting all of the tests to take into account proper spatial and time series formatting, and probably do a bit of re-architecting to turn secure access into a separate test (this will most likely involve passing the query client into the processor as opposed to having the processor call its own).
That and documenting, putting in descriptive errors, that sort of thing.
Thursday, July 3, 2008
Lots of stuff done, lots of stuff to do.
Things completed:
- Confirmed that secure Ogsa-dai was working on the new 205 box. Many thanks to Dan for his help and awesomeness.
- Have pulled the current RODSAdai files over to the 205 box, and linked the 205 box's maven to the repository, showing that I can recreate all the dev code and have it building in about 5 minutes in a new environment... and that is so cool.
- Got the HL7_ADTS table on the 205 box.
Things to be done now:
- Implement the secureClient for RODSAdai
- Document
- Expand tests to deal with time and spatial series data.
- Work with Jeremy on integrating with RODS and getting the security to work with tomcat.
Cheers, Things seems to be rolling along nicely, have a good 4th of July weekend!
Unknown CA Error: Fix
Globus was recently reinstalled on the Ubuntu node. Instead of installing the certificates in the /etc/grid-security directory, I choose to use the non-root install and install the certificates in the $GLOBUS_LOCATION/etc directory.
I received the following error in the $GLOBUS_LOCATION/var/container.log file when I tried to start the Globus container:
Remote exception was ;nested exception is: org.globus.common.ChainedIOException: Authentication failed [Caused by:Failure unspecified at GSS-API level [Caused by: Unknown CA]]
I knew the certificates were good, but it seemed like Globus was unable to locate the installed certificates. After researching the issue, I came across Globus Bug# 4303.
Link: http://bugzilla.globus.org/globus/show_bug.cgi?format=multiple&id=4303
Based on the information in the Bug Report, I discovered that Globus was looking for the certificates in the default, /etc/grid-security/certificates directory. To correct the problem I created two symbolic links. The first was a link from $GLOBUS_LOCATION/etc to /etc/grid-security. The second was a link from $GLOBUS_LOCATION/etc/certificates to $GLOBUS_LOCATION/TRUSTED_CA.
After creating the links, the Globus container started up.
I received the following error in the $GLOBUS_LOCATION/var/container.log file when I tried to start the Globus container:
Remote exception was ;nested exception is: org.globus.common.ChainedIOException: Authentication failed [Caused by:Failure unspecified at GSS-API level [Caused by: Unknown CA]]
I knew the certificates were good, but it seemed like Globus was unable to locate the installed certificates. After researching the issue, I came across Globus Bug# 4303.
Link: http://bugzilla.globus.org/globus/show_bug.cgi?format=multiple&id=4303
Based on the information in the Bug Report, I discovered that Globus was looking for the certificates in the default, /etc/grid-security/certificates directory. To correct the problem I created two symbolic links. The first was a link from $GLOBUS_LOCATION/etc to /etc/grid-security. The second was a link from $GLOBUS_LOCATION/etc/certificates to $GLOBUS_LOCATION/TRUSTED_CA.
After creating the links, the Globus container started up.
Wednesday, July 2, 2008
Now time series is processing, and the ogsadai errors are gone
Yay for troubleshooting and debugging.
The cause for the OD errors was some transposed digits in the port number for Postgres.
Otherwise, I managed to code up the time-series and write a quick console test, and it passed!
Now, to move those tests into the automatic build testing, and then document it and work on secure data transport now that we have two securable nodes.
Also, I got to play with an OLPC (One Laptop Per Child) laptop today, it is really neat and there might be some cool grid computing aspects or data reporting to be made.
The cause for the OD errors was some transposed digits in the port number for Postgres.
Otherwise, I managed to code up the time-series and write a quick console test, and it passed!
Now, to move those tests into the automatic build testing, and then document it and work on secure data transport now that we have two securable nodes.
Also, I got to play with an OLPC (One Laptop Per Child) laptop today, it is really neat and there might be some cool grid computing aspects or data reporting to be made.
Monday, June 30, 2008
Rodsadai compiling again, server move giving familiar errors
This morning I tried adding the client-settings.wsdd file to the resources directory (not the server-settings.wsdd) and the rodsadai tests magically started working again (yay!)
The rest of the afternoon was spent setting up Ogsa-dai on it's new home, and having familiar cryptic errors thrown the first time I attempt to connect. Tomorrow I shall poke both more.
The rest of the afternoon was spent setting up Ogsa-dai on it's new home, and having familiar cryptic errors thrown the first time I attempt to connect. Tomorrow I shall poke both more.
Friday, June 27, 2008
Jar hunt finished, maven repository updated, musical servers commencing
So, the jars have been included in the new repository (I had to spend some extra time generating some extra files that is necessary for remote repositories)
Right now, we are dealing with moving a server between VMs, at the end we are hoping to have an ubuntu globus server (the previous server was corrupted by attempts to install dual mysql databases).
Also, for some odd reason, the tests that I was so elated to have working yesterday... are now not working citing an "OGSA-DAI resource null is unknown" error. I am thinking it might be some sort of "the IP has changed" conflict... but at the same time the other OGSA-DAI examples are still working. Either way, I don't think I am going to be able to diagnose it today, and am out of ideas of what to try until we get the other VM set up to match this one.
Right now, we are dealing with moving a server between VMs, at the end we are hoping to have an ubuntu globus server (the previous server was corrupted by attempts to install dual mysql databases).
Also, for some odd reason, the tests that I was so elated to have working yesterday... are now not working citing an "OGSA-DAI resource null is unknown" error. I am thinking it might be some sort of "the IP has changed" conflict... but at the same time the other OGSA-DAI examples are still working. Either way, I don't think I am going to be able to diagnose it today, and am out of ideas of what to try until we get the other VM set up to match this one.
Wednesday, June 25, 2008
Going on a JAR hunt...
Today, I started beefing up the standard JUnit tests in RODSAdai... and remembered that a LOT more code is needed to run OGSA-DAI implementing clients than to just build it.
So, since the new test code runs an OGSA-DAI test... I am having a lot of attempts at "mvn compile", a test error or failure will result, so I check the log, find the class it needs, and make a reference to it, installing it in the maven repository if it was not in the global one, and repeating.
This will probably consume me tomorrow and maybe a bit of friday. I mean it when I say a lot of jars.
So, since the new test code runs an OGSA-DAI test... I am having a lot of attempts at "mvn compile", a test error or failure will result, so I check the log, find the class it needs, and make a reference to it, installing it in the maven repository if it was not in the global one, and repeating.
This will probably consume me tomorrow and maybe a bit of friday. I mean it when I say a lot of jars.
Tuesday, June 10, 2008
RODSAdai
I spent a lot of today drawing out how I wanted the interfaces to look for the new RODS over OGSA-DAI application; RODSAdai. I also talked over some of the specifics and test cases with Brian (who reminded me of the beauty of factories).
I also mavened up an eclipse project, tinkered with it a bit and got some preliminary Java files set up, and checked in the stubs to the RODS-NCPHI test server.
Tomorrow, expansions, filling in the code stubs, lots and lots of changes, and maybe some test cases run.
I also mavened up an eclipse project, tinkered with it a bit and got some preliminary Java files set up, and checked in the stubs to the RODS-NCPHI test server.
Tomorrow, expansions, filling in the code stubs, lots and lots of changes, and maybe some test cases run.
Monday, June 9, 2008
JUnit and Use Cases
Today I spent some time researching how JUnit works (JUnit is the preferred testing framework of maven, which is nice, considering that whenever you build to deploy, it will run the test cases and let you know if something you adjusted or refactored broke).
The next thing I did was to start drawing out the application by the top-level use cases (the use cases will derive test cases which will derive code). The apps I build for the RODS interface will basically be an extension of OGSA-DAI's discovery and query algorythms. A server list stored in a properties file (which can later be upgraded to a database of some sort) will be used to pull up the appropriate server (by key) and the default Ogsa-Dai data resource to hit on that server... the query gets passed on, and then the specialized class structure gets returned.
Now I am thinking of the smaller cases and how all the classes will break out around each other. After that I'll start actually modifying OGSA-DAI example classes and writing wrapper classes and interfaces (and their test cases). Right now I just want to get it all on paper so I have a good reference for which peice I am working on and where it fits.
The next thing I did was to start drawing out the application by the top-level use cases (the use cases will derive test cases which will derive code). The apps I build for the RODS interface will basically be an extension of OGSA-DAI's discovery and query algorythms. A server list stored in a properties file (which can later be upgraded to a database of some sort) will be used to pull up the appropriate server (by key) and the default Ogsa-Dai data resource to hit on that server... the query gets passed on, and then the specialized class structure gets returned.
Now I am thinking of the smaller cases and how all the classes will break out around each other. After that I'll start actually modifying OGSA-DAI example classes and writing wrapper classes and interfaces (and their test cases). Right now I just want to get it all on paper so I have a good reference for which peice I am working on and where it fits.
Friday, June 6, 2008
Repositories and Builds and Tomcat, Oh My
Yesterday I was here for a while trying to get SVN to behave. I found that the "gov/cdc/phgrid/ogsadai/example" path was stuck in the root directory, and since I was trying to create a the same path inside the maven directory structure (under /main/src/java/, I think it was) it refused to add it there. Even after I removed the old gov structure. I was thinking about scouring the old repository, had Brian try a few things, and still hit roadblocks
Then I had the late arriving epiphany to just move the tree, and that worked, and it was good.
Then I gave Anurag a zip of my local repository files, so that he could create a public repository.
I also had a wonderful discussion with Jeremy about a RODS-OGSADAI interface. I had a bit of a paradigm shift, but in the meantime, I have a good idea for what sorts of use cases, test cases, and code I will need to develop. I have also taken up the task of finding out why Tomcat doesn't find the same security credentials that were find for command line operation, as that hurdle will need to be crossed eventually for happy JSP-based items pulling data from the grid.
Cheers, I hope everyone has a wonderful weekend!
Then I had the late arriving epiphany to just move the tree, and that worked, and it was good.
Then I gave Anurag a zip of my local repository files, so that he could create a public repository.
I also had a wonderful discussion with Jeremy about a RODS-OGSADAI interface. I had a bit of a paradigm shift, but in the meantime, I have a good idea for what sorts of use cases, test cases, and code I will need to develop. I have also taken up the task of finding out why Tomcat doesn't find the same security credentials that were find for command line operation, as that hurdle will need to be crossed eventually for happy JSP-based items pulling data from the grid.
Cheers, I hope everyone has a wonderful weekend!
Wednesday, June 4, 2008
Maven and it's lovely repository
So, I spent a good part of the day paring down how many libraries my example client code actually needed from globus and ogsadai respectively. The list dropped from 40 to about 8.
Then I spent a lot of time putting one of those 8 into the local maven repository in a way that maven seems to recognize... and adjusting the pom file to point to it as a dependency. I hit "mvn package" (ie. compile) and I got a long, thoughtful message about how the dependency wasn't found and how I can add a file with the helpful "mvn install:install-file" command. They also had a "mvn deploy" command for deploying code to a commonly owned repository... showing that deploying to a central and accessible location is
Then proceeded to add all 8 files in about 30 minutes, along with dependencies to the pom file, and then ran mvn compile successfully.
I am really beginning to like maven!
Tomorrow, I am going to put some of the interface and repository plans for the RODS<->OD interface to paper... perhaps start some of the code for simple things like resource discovery. Then, I will test to see whether the jar produced runs properly when put into the proper environment... then I will start reading up on where to put the JSP files. I will also make an eclipse project file and import it into eclipse.
There is also a meeting to discuss package layouts and plans and the like. I am looking forward to it.
Then I spent a lot of time putting one of those 8 into the local maven repository in a way that maven seems to recognize... and adjusting the pom file to point to it as a dependency. I hit "mvn package" (ie. compile) and I got a long, thoughtful message about how the dependency wasn't found and how I can add a file with the helpful "mvn install:install-file" command. They also had a "mvn deploy" command for deploying code to a commonly owned repository... showing that deploying to a central and accessible location is
Then proceeded to add all 8 files in about 30 minutes, along with dependencies to the pom file, and then ran mvn compile successfully.
I am really beginning to like maven!
Tomorrow, I am going to put some of the interface and repository plans for the RODS<->OD interface to paper... perhaps start some of the code for simple things like resource discovery. Then, I will test to see whether the jar produced runs properly when put into the proper environment... then I will start reading up on where to put the JSP files. I will also make an eclipse project file and import it into eclipse.
There is also a meeting to discuss package layouts and plans and the like. I am looking forward to it.
Subscribe to:
Posts (Atom)
Posts
