Service Philosphy

Last week I worked on merging the service and the core functionality of AMDS. The bulk of that effort was spent trying to get the GDTE tool to call the specific JaxBElements as specified by the AMDS XSD. At one point I thought I had this licked but after some initial testing I realized that there was some additional issues that would take even longer to resolve. So for now, I shelved trying to get the service to work in that fashion and just created manual marshal and unmarshal routines that would get the job done. I guess for the programmer that likes to just look at an interface and begin extending through raw guts, intuition and instinct, it forces them to read the README file before moving forward. I can talk about those type of programmers because I am guilty of the same. :-)

The entire code check in process took place over the weekend. I wanted to wrap my head around a directory structure for services going forward so that it can be easy for someone who wants to download the code and build. So through my thinking I came up with the idea that a service should have the following directory structure:

+--ServiceProjectName:

• +--Service
• +--Common
• +--Client
• +--lib
• --build.xml
• --README.txt
• --POM.xml

This structure follows my philosophy for grid service development which follows:

A grid service should have 3 components:

1. The core or common components - these components do all of the work for the service from making database calls, structuring inputs and outputs, etc.
2. Service Implementation -Since there are a couple of techniques for creating Globus services its interface should be implemented from the core components making the service implementation very thin. I have based my service implementation on GDTE which is a more generic service implementation than Introduce. However, if someone wanted to implement a service using Introduce then the common components are preserved in such a way that this is very easily done. Additionally, configuration file location and environment variables are accessible in locations as prescribed by the GTK4 development manuals.
3. Client Implementation - the client implementation is based on the service implementation. In my TODO list I am going to try and make this more generic and base the client off of the WSDL GDTE tool generates a client stub, I will use it as a base. I will transform it into something generic later.

Included in my service is a GUI for the service configuration. A snap shot is below:


The AMDS configuration is fairly complex and since this service will be moving to production, I wanted to make it as easy as possible to administer from a UI as oppose to just having a configuration file. From the UI it is easily seen that its design is intended to manage a suite of AMDS services at one node. This taps into my ideas about the service ecosystem: The service should be easy to extend/develop, to deploy/undeploy, management/configure, and include a default usable client. The intended audience for this targeted functionality is not just for developers but also for the non-technical person when using the service. These specific ideas follow the IPhone App-Store Model of application development, a topic to go into detail later.

Lastly, here are my TODOs for this week:

• Complete the generic configuration of the AMDS service.
• Build the default client to be used with the AMDS Service
• Begin to think about multi-node access for the client (The client should be able to aggregate multiple node endpoints running the AMDS service)
• Upload the updated service and remaining artifacts to source control, with a base level instruction set.
  

More to come...

Simplifying the service creation process

This is week 2 of me formally joining the Grid team in the NCPHI Lab. I am now focusing the majority of my efforts on specific Grid task but still contribute some of my time the PHINMS team as an advisor and architect. In the past 2 weeks of my efforts in the lab I have been trying to wrap my head around some of the challenges of the Grid regarding its implementation in the public health community. Additionally, I have closely examined the whole process of building and deploying a Globus service.

For the first week or so, I spent my time trying to get my development environment set up. This effort took more time than I expected due to the shift in paradigm of a pure production environment to a more R&D environment. None-the-less, I have prevailed and the result is that I am now up and running on 2 VMs. My personal preference is the Unbuntu VM. I also have a RedHat VM which Admin, Dan swears by. Since I always consider admins a programmer's best friend, I have to quietly defer. The Unbuntu environment is what I use heavily at home so there is an added layer of comfort here.

To get to the meat of this post, I have successfully built and deployed (in my environment) a Globus service! In week one I went through the GTK4 programmers manual examining the manual process in creating a service. Since I am a purest when it comes to programming, I figure, if I can not get any of the tools to work I can always just do it manually. After reviewing the book I poked around the Globus system. I downloaded the ws-core and deployed it to a Tomcat container. In my home environment this took no time. I then played with some of the packaged services. From there I was told about Introduce. This is the Globus Service Development Toolkit that Peter and Felica were using. In my attempts to get it set up it was a struggle. There are a lot of moving parts but I did have some mild success. After going through the generated service package there was a lot of items in it that I was not sure of its purpose. I chalked it up to not having enough knowledge about globus and caBig. Later I discussed some of my findings with Felica. She is very experienced with the Globus container and Introduce and she shared with me that there are a lot of caBig wrapped Globus jars used in the services that are generated as well as some redundancy regarding the jars used in the caBig default container. After further examination and discussions, I quickly arrived at the conclusion that deploying services using the Introduce tool may add a support burden that make approach critical mass as PHGrid gains wider adoption. Regardless, it did work, so at a minimum this was a milestone for me to have created a service. This was all in week 1.

Week 2
Immediate, I began to search for other solution that was simpler to understand, develop and manage. I then came across GDTE. The website was a little choppy but informative. A little background, I am an old school developer, from the days of C/C++ and Motif so the majority of my career I have used vi as my editing tool and have only recently, past 2-3 years, began to attach to an IDE, again I am a purest. :-) While I believe over time the IDE may erode my mind, it does make sense as the number of libs and methods/functions are too numerous to remember. So it does add some efficiency. All of that to say, I used mostly Netbeans in the past and I only used eclipse intermittently. GDTE uses eclipse so I had to put on my eclipse hat. It took me about 15 minutes to find the correct distro but then took me a few days to get it configured properly in my environment. To my credit there were some inconsistencies in my VM environment. However in the course of a couple of days, Dan Washington, he is part of the League of Superior Admins, came to my rescue and resolved my issues. So early Thursday I had a working eclipse environment. From there, I went to the GDTE site and followed their straight forward approach in developing a service. The end result is that their service generation is much cleaner and uses the pure Globus jars for its deployment. It does add 2 extra jars but these are Minor. Overall, the process was much simpler than Introduce and the output was much cleaner so for my efforts I am rolling with GDTE. Here is their link:

http://mage.uni-marburg.de/trac/gdt/wiki/WikiStart

Here are some hurdles I have over come via trial and error. Do not use the latest version of eclipse 3.4.x GDTE will not work with this version. I downloaded easyeclipse 1.3.1.1. here is their link:

http://www.easyeclipse.org/site/distributions/index.html

Download the version :EasyEclipse for Plug-ins and RCP Apps 1.3.1.1
This version will have all of the plug-ins you need. From there you can just follow the instructions from the GDTE folks. This group seems to be very active, I looked though their e-mail archives to get a feel. They are aware of the issues with 3.4 and are going to make fixes in their next distro.

Lastly, only installed the Service Generator and certifcate tool, I had issues with the ViGO (GDT Visual Grid Orchestrator (ViGO) Version 1.0 ) tool as there are some dependencies needed by eclipse. I got my service built, compiled and deployed successfully using this tool last night at 10:23pm. So having been up since 4 am I was a little tired in trying to figure it out. I will later.

Next, I plan to merge the back end code (I wrote in week 1) with my newly created service and then formally deploy the results to the NCPHI training node. Additionally, I will determine the "fitness" of the resulting GAR from GDTE. The fitness determination will let me know how the gar generation process should be augmented to increase the supportability in a PHGrid federation. I will post more about this later.

One last note, if you are familiar with annotation then you will love the GDTE tool. Their technique for Globus services development seems to be similar to WSIT/METRO (Sun's WS development). More to come.

A Hybrid Mesh Network: Path to a Distributed Secure Social Grid Network

After a rigorous weekend of coding and searching for code, I am quickly coming up with a model that can specifically express most of the ideas of a distributable secure social grid network. On Saturday morning I was reviewing some of my detailed thought experiments relating to “What would it take to create an infinitely scalable distributed social network”. Reading through the thoughts here it can be easily seen how that consideration is not only relevant but paramount from the perspective of supporting the expanding boundary conditions that a health care network will demand. So to that end, I began to review the types of physical networks that can support that type of environment and came up with the idea of basing my ideas on a Hybrid Mesh Network that will leverage two Networking sub-models: Fully Connected Topology (Point-to-Point) and Star Topology (Hub and Spoke). These topologies take into account three of the most important expressions of a Grid Node’s ability to connect and participate in the network which are:

• A Consumer Grid Node – this node is consuming services/applications, vocabularies and autonomously exchanging data with “N” number of other nodes
• A Producer Grid Node – this node is providing services/applications, vocabularies and autonomously exchanging data with “N” number of other nodes
• A Producer/Consumer Grid Node – this node is providing services/application and vocabularies to some nodes while consuming services/applications to other nodes and autonomously exchanging data with all.

In all three cases, the incorporated PKI for credential management and access control will need to encompass direct control and delegated security models. Considering deployment requirements now, will make a solution that much more adoptable by a broad range of organizations because it is conforming to some of their deployment best practices. For leisure reading I took a look at this document ,I think it provides a good guidance in regards to a Producer or the Producer/Consumer DMZ deployment best practices. Thoughts? Until next time…..

Enhancing the PHGrid Platform to realize the vision

Last Friday I had an opportunity to sit in on a presentation given by Tom Savel and Ken Hall, Grid Value Proposition. I believe I can contribute to the effort by utilizing some of my past and current related work. Just having published the final drafts of the PHGrid Architecture documents, I began to go through several blogs, documents, and the wiki to get a full pulse check of how PHGrid could be enhanced to implement the remaining infrastructure pieces described in the Architecture documents. In my thought experiment, I believe if one places a wrapper around the Globus container, ideally, it will make it that much more manageable by one who is non-technical, a significant advantage. I believe this is an important hurtle because having come from the world of PHINMS and seeing its growth over the seven years of my involvement, lowering the support burden and enhancing the configuration via a user interface (UI) was one of the main factors in growing its user base to over 700 active nodes and growing. What was found through that experience is that the following should be well defined:

• UI and Automation for certificate exchange, renewal, distribution and revocation mechanism – this should be based on an invitation model for added security.
• UI for Node-to-Node access control which will assist in fostering secure and reliable data exchange and service/application distributions
• UI and Automation for auto-discovery and monitoring
• Queue based data transactions (Similar to PHINMS)
• A Graphical Installer (Similar to installshield type installers)
• Mechanisms that provide auto updating and patches
• A well defined support plan from deployment to long term management.

I believe the above describes the bases for realizing the full visions expressed by Tom and Ken while incorporating all of the existing PHGrid efforts. Since my background cross-sections all of the above mentioned from a technical perspective, I will try and put some things together. I will post some initial results soon. Please share your thoughts. Until next time….