Successful GIPSEService test

Forgot to post that last week Ron Price and I successfully tested a deployment of the GIPSEService at the Denver DOH.

Ron, working with Art Davidson, set up a synthetic aggregate data set and then he deployed an instance of the GIPSEService (8/31 gipse spec from the SVN repository).

I was then able to submit a test query from the NCPHI Lab using lab credentials that was successfully processed by the GIPSEService and sent back a response document containing the relevant observation set. Also, I tested with inappropriate credentials from unauthorized locations and I was not able to access the service (as expected since Ron's security controls prevent access by unauthorized users or locations).

Easy upgrading GIPSEService to 20090831 gipse.xsd schema

In case you have customized the GIPSEService, you can easily update it to use the http://ncphi.phgrid.net:8080/schemas/gipse/20090831 namespace gipse.xsd by following the few steps below: (thanks to Peter for collecting the details)

1. Update the gipse.xsd within the ./schema directory with the latest version (available in the sourceforge project)


2. Replace all the occurrances of "20090630" with "20090831" within the following files:
   ./schema/GIPSEPoison/gipse.xsd (replaced entirely)
   ./schema/GIPSEPoison/GIPSEPoison.wsdl
   ./GIPSEQueryRequest-ForClient.xml
   ./introduce.xml
   ./introduce.xml.prev
   ./namespace2package.mappings
   


3. Run "ant clean"


4. Run "ant all"


5. Run "ant createDeploymentGar"


6. Undeploy your old service


7. Deploy your updated service (created in step 5)

That's it, you shoudln't need to modify any Java code.

GIPSEService update

I just checked in updates to the GIPSEService project to include support for the Age and Service Area (Disposition) stratifiers as well as the Data Source filter.

This has pretty major ramifications as the version of gipse.xsd changed from 6/30 to 8/31. This means that anyone using the source code for GIPSEService should redownload and rebuild their local gar so that the latest version of gipse.xsd is supported.

You can see the latest copy of the gipse.xsd in its sourceforge project.

Taha wanted us to try out google code instead of the usual sourceforge project, so you can grab the latest version of the GIPSEService project through google code's svn. (btw, so far google code is a lot cleaner, but I miss the ability to download a tarball through the browser)

I'll update the NCPHI lab node during Friday's usual deployment, so you have until then to update your clients to use the latest updates.

If we were actually in production (planned for after Sep 1), we would actually version the service and provide a support window where both versions were supported. We're only breaking interfaces because there are no production users yet.

If you don't update, you will see an exception like this:

[java] org.xml.sax.SAXException: Invalid element in gov.cdc.ncphi.phgrid.services.gipse.stubs.QueryMetadataRequestQuery - MetadataQuery
[java] at org.apache.axis.message.SOAPFaultBuilder.createFault(SOAPFaultBuilder.java:221)
[java] at org.apache.axis.message.SOAPFaultBuilder.endElement(SOAPFaultBuilder.java:128)
...

This is caused because the client expects a specific xml namespace to be used by gipse. Older clients will expect http://ncphi.phgrid.net:8080/schemas/gipse/20090630, while this latest version uses http://ncphi.phgrid.net:8080/schemas/gipse/20090831.

This should be a rather painless upgrade for any existing users. If you customized the service code then you'll struggle (but what did you expect, next time contribute your changes back so we can make sure your changes make it into future versions).

For anyone still using the amds* database structure, you will need to update your db. Use the gipse-store project to generate ddl for SQLServer or PostgreSQL. Then you can copy over all your data from amds_extract using a query like:

INSERT INTO gipse_store (date, indicator_id, value, data_source_id, zip5, state)
SELECT date, condition, count, source_oid, zip5, state FROM amds_extract

Updated GIPSEService (finally)

I updated the GIPSE BioSense service so that it uses the 6/30 GIPSE data structure draft.

The GIPSEService can read from any jdbc data source and is configurable by editing the db.gipse.table property to point to the view/table name that contains the GIPSE data.

Also note that if you are upgrading your service, but not your data store (still using amds_extract) you should set the following properties in your gipse.properties file:
db.gipse.table=amds_view
db.gipse.table.column.date=date
db.gipse.table.column.state=state
db.gipse.table.column.zip5=zip5
db.gipse.table.column.zip3=zip3
db.gipse.table.column.condition=condition
db.gipse.table.column.classifier=classifier
db.gipse.table.column.count=count

This service is built to share the BioSense aggregate counts using the required stratifiers (date, state/zip5/zip3, indicator, count) so it will not use age, service area / disposition nor facility even though the schema supports these optional stratifiers.

There is a lot of buzz around adding age, so the next version may end up supporting the age optional stratifier with a couple of age groups.

AMDS Service Beta updates

I worked with the CSC/NEDSS programmer to test out their usage of AMDSService. I left out some of the boilerplate jars, but after adding those they were able to configure, build and successfully test out the AMDSService running on their local node. So this is progress.

BioSense AMDS Service - Beta Release

The AMDS Service for BioSense is now in beta release. Please view details of how to download,configure, build, deploy and use on the service registry page.

You can also download the raw gar from sourceforge. But I recommend getting the source and building with your own configuration.

This service uses the updated 5/31 AMDS draft.

This service is specifically developed to share BioSense aggregate data over PHGrid, but can actually be used for any JDBC data source that wants to be shared using the AMDS spec.

This release is significantly different than the 4/30 alpha release. Specifically we're using Introduce 1.3 (big improvement over 1.2) for service development and configuration management and iBATIS for easy db access / ORM. This release is smaller than the alpha release in size and lines of code so theoretically it will be easier to use. Please let me know any comments. We'll be following the weekly build schedule with a target of July 8 for code freeze.

Update: Tom asked me to explain that we're using iBATIS rather than Hibernate. Both are decent JDBC/ORM frameworks. I chose iBATIS because it has a lighter footprint and I got it working in about 15 minutes. This doesn't mean we won't use Hibernate in the future, but just that we're using iBATIS for now.

Service Philosphy

Last week I worked on merging the service and the core functionality of AMDS. The bulk of that effort was spent trying to get the GDTE tool to call the specific JaxBElements as specified by the AMDS XSD. At one point I thought I had this licked but after some initial testing I realized that there was some additional issues that would take even longer to resolve. So for now, I shelved trying to get the service to work in that fashion and just created manual marshal and unmarshal routines that would get the job done. I guess for the programmer that likes to just look at an interface and begin extending through raw guts, intuition and instinct, it forces them to read the README file before moving forward. I can talk about those type of programmers because I am guilty of the same. :-)

The entire code check in process took place over the weekend. I wanted to wrap my head around a directory structure for services going forward so that it can be easy for someone who wants to download the code and build. So through my thinking I came up with the idea that a service should have the following directory structure:

+--ServiceProjectName:

• +--Service
• +--Common
• +--Client
• +--lib
• --build.xml
• --README.txt
• --POM.xml

This structure follows my philosophy for grid service development which follows:

A grid service should have 3 components:

1. The core or common components - these components do all of the work for the service from making database calls, structuring inputs and outputs, etc.
2. Service Implementation -Since there are a couple of techniques for creating Globus services its interface should be implemented from the core components making the service implementation very thin. I have based my service implementation on GDTE which is a more generic service implementation than Introduce. However, if someone wanted to implement a service using Introduce then the common components are preserved in such a way that this is very easily done. Additionally, configuration file location and environment variables are accessible in locations as prescribed by the GTK4 development manuals.
3. Client Implementation - the client implementation is based on the service implementation. In my TODO list I am going to try and make this more generic and base the client off of the WSDL GDTE tool generates a client stub, I will use it as a base. I will transform it into something generic later.

Included in my service is a GUI for the service configuration. A snap shot is below:


The AMDS configuration is fairly complex and since this service will be moving to production, I wanted to make it as easy as possible to administer from a UI as oppose to just having a configuration file. From the UI it is easily seen that its design is intended to manage a suite of AMDS services at one node. This taps into my ideas about the service ecosystem: The service should be easy to extend/develop, to deploy/undeploy, management/configure, and include a default usable client. The intended audience for this targeted functionality is not just for developers but also for the non-technical person when using the service. These specific ideas follow the IPhone App-Store Model of application development, a topic to go into detail later.

Lastly, here are my TODOs for this week:

• Complete the generic configuration of the AMDS service.
• Build the default client to be used with the AMDS Service
• Begin to think about multi-node access for the client (The client should be able to aggregate multiple node endpoints running the AMDS service)
• Upload the updated service and remaining artifacts to source control, with a base level instruction set.
  

More to come...

Simplifying the service creation process

This is week 2 of me formally joining the Grid team in the NCPHI Lab. I am now focusing the majority of my efforts on specific Grid task but still contribute some of my time the PHINMS team as an advisor and architect. In the past 2 weeks of my efforts in the lab I have been trying to wrap my head around some of the challenges of the Grid regarding its implementation in the public health community. Additionally, I have closely examined the whole process of building and deploying a Globus service.

For the first week or so, I spent my time trying to get my development environment set up. This effort took more time than I expected due to the shift in paradigm of a pure production environment to a more R&D environment. None-the-less, I have prevailed and the result is that I am now up and running on 2 VMs. My personal preference is the Unbuntu VM. I also have a RedHat VM which Admin, Dan swears by. Since I always consider admins a programmer's best friend, I have to quietly defer. The Unbuntu environment is what I use heavily at home so there is an added layer of comfort here.

To get to the meat of this post, I have successfully built and deployed (in my environment) a Globus service! In week one I went through the GTK4 programmers manual examining the manual process in creating a service. Since I am a purest when it comes to programming, I figure, if I can not get any of the tools to work I can always just do it manually. After reviewing the book I poked around the Globus system. I downloaded the ws-core and deployed it to a Tomcat container. In my home environment this took no time. I then played with some of the packaged services. From there I was told about Introduce. This is the Globus Service Development Toolkit that Peter and Felica were using. In my attempts to get it set up it was a struggle. There are a lot of moving parts but I did have some mild success. After going through the generated service package there was a lot of items in it that I was not sure of its purpose. I chalked it up to not having enough knowledge about globus and caBig. Later I discussed some of my findings with Felica. She is very experienced with the Globus container and Introduce and she shared with me that there are a lot of caBig wrapped Globus jars used in the services that are generated as well as some redundancy regarding the jars used in the caBig default container. After further examination and discussions, I quickly arrived at the conclusion that deploying services using the Introduce tool may add a support burden that make approach critical mass as PHGrid gains wider adoption. Regardless, it did work, so at a minimum this was a milestone for me to have created a service. This was all in week 1.

Week 2
Immediate, I began to search for other solution that was simpler to understand, develop and manage. I then came across GDTE. The website was a little choppy but informative. A little background, I am an old school developer, from the days of C/C++ and Motif so the majority of my career I have used vi as my editing tool and have only recently, past 2-3 years, began to attach to an IDE, again I am a purest. :-) While I believe over time the IDE may erode my mind, it does make sense as the number of libs and methods/functions are too numerous to remember. So it does add some efficiency. All of that to say, I used mostly Netbeans in the past and I only used eclipse intermittently. GDTE uses eclipse so I had to put on my eclipse hat. It took me about 15 minutes to find the correct distro but then took me a few days to get it configured properly in my environment. To my credit there were some inconsistencies in my VM environment. However in the course of a couple of days, Dan Washington, he is part of the League of Superior Admins, came to my rescue and resolved my issues. So early Thursday I had a working eclipse environment. From there, I went to the GDTE site and followed their straight forward approach in developing a service. The end result is that their service generation is much cleaner and uses the pure Globus jars for its deployment. It does add 2 extra jars but these are Minor. Overall, the process was much simpler than Introduce and the output was much cleaner so for my efforts I am rolling with GDTE. Here is their link:

http://mage.uni-marburg.de/trac/gdt/wiki/WikiStart

Here are some hurdles I have over come via trial and error. Do not use the latest version of eclipse 3.4.x GDTE will not work with this version. I downloaded easyeclipse 1.3.1.1. here is their link:

http://www.easyeclipse.org/site/distributions/index.html

Download the version :EasyEclipse for Plug-ins and RCP Apps 1.3.1.1
This version will have all of the plug-ins you need. From there you can just follow the instructions from the GDTE folks. This group seems to be very active, I looked though their e-mail archives to get a feel. They are aware of the issues with 3.4 and are going to make fixes in their next distro.

Lastly, only installed the Service Generator and certifcate tool, I had issues with the ViGO (GDT Visual Grid Orchestrator (ViGO) Version 1.0 ) tool as there are some dependencies needed by eclipse. I got my service built, compiled and deployed successfully using this tool last night at 10:23pm. So having been up since 4 am I was a little tired in trying to figure it out. I will later.

Next, I plan to merge the back end code (I wrote in week 1) with my newly created service and then formally deploy the results to the NCPHI training node. Additionally, I will determine the "fitness" of the resulting GAR from GDTE. The fitness determination will let me know how the gar generation process should be augmented to increase the supportability in a PHGrid federation. I will post more about this later.

One last note, if you are familiar with annotation then you will love the GDTE tool. Their technique for Globus services development seems to be similar to WSIT/METRO (Sun's WS development). More to come.

De-duplication Insight from the Field

After Monday's post Dave Heinbaugh, the tech manager for the Tarrant County APC, was kind enough to spend some time with me on the phone walking through some of the scenarios for de-duplication.

He basically broke it down that based on the statistical analysis, it's really not worth trying to account for a single patient who gets served by multiple hospitals or clinics. Based on Dave's experience near the data, this does not happen.

So, de-duplication is an issue for NHIN where they are trying to figure out if 5 different EMR/EHRs are the same person, but for AMDS it may not be as an immediate, data quality issue as feared.

This doesn't get rid of the question of de-duplication, but shifts it a bit from trying to reduce specific instances of patients recorded multiple times to trying to make sure that a data source is not reported multiple times (for example Tarrant County is a BioSense data source so if you queried BioSense-AMDS and Tarrant-AMDS services for Respiratory conditions and combined them together you would get double counts).

De-duplication

I've been thinking about the "De-duplication" grid service a bit lately and want to get my thoughts out here in case someone else has been mulling it over as well.

De-duplication is related to the De-identification/Re-identification problem of Anonymized data across multiple data sources (interesting MIT paper here). But in a grid environment the problem becomes tougher because the traditional, really-hard-to-do-solution of checking the various MPIs is even tougher because data sources 1) don't frequently have MPIs and 2) don't want to share them if they do.

Here's the scenario I use to talk about the de-duplication problem set:
The scenario is this:
3 hospitals in Atlanta with overlapping catchment areas. A federated query is sent out for SyndromeX for April 1, 2009. Hospital A returns a count of 10, Hospital B returns a count of 15 and Hospital C returns a count of 20. The combined result from the federated query shows a total of 45 counts of SyndromeX for April 1, 2009.

There exists a question of data quality as it's possible that the same patient visited multiple hospitals and had his SyndromeX recorded at multiple locations. In a federated/grid configuration this is even more probable as Hospitals may use data from multiple feeder sources (labs, small clinics, etc) that may report the same patient encounter multiple times.

Dr. Lenert had a good idea where each Hospital could send over a list of PII (name, address, dob, social) without the medical information for purposes of de-duping. This requires a trusted intermediary to receive these matches and send them on to each Hospital to respond back with number of matches. A hospital cannot send directly to each other hospital because it would reveal its patients to potential competitors.

So the de-duping scenario may change to something like this:
1) User submits query for SyndromeX on April 1, 2009 and selects de-duplication.
2) Trusted intermediary receives the query and passes it on to Hospitals A,B,C.
3) Hospital A returns count of 10, Hospital B returns count of 15, Hospital C returns count of 20. Interim total count 55 is not returned to user.
4) Trusted intermediary then requests patient id info from each hospital in a separate request.
5) Trusted intermediary compares the PII from each hospital (perhaps using a tool like febrl) and determines that there is overlap of patients and actual count. Hospital A shares 1 patient with Hospital B. Hospital A shares 2 patients with Hospital C. Hospital B shares 3 patients with Hospital C. Therefore there are 6 unique duplicates.
6) Total count of 39 is returned to the user (45-1-2-3).
Of course, this is going to require some pretty serious data use agreements with the hospitals and the trusted intermediary to control the storage of PII (i.e. don't persist any PII)

The alternative without the trusted intermediary:
1) User submits query for SyndromeX on April 1, 2009 and selects de-duplication.
2) Each hospital receives the query and list of nodes to which is was submitted.
3) Hospital A identifies a preliminary count of 10. To check for duplicates, it submits only the PII without any condition data and a query ID to Hospital B and C for comparison.
4) Hospital B compares the PII to its own results for query ID and returns the number of matches to Hospital A (duplicate count=1).
5) Hospital C compares the PII to its own results for query ID and returns the number of matches to Hospital A (duplicate count=2).
6) At the same time, Hospitals B&C are performing similar communications with the other nodes participating in the query.
7) Hospital B identifies a preliminary count of 15. To check for duplicates, it submits only the PII without any condition data and a query ID to Hospital A and C for comparison.
8) Hospital A compares the PII to its own results for query ID and returns the number of matches to Hospital B (duplicate count=1).
9) Hospital C compares the PII to its own results for query ID and returns the number of matches to Hospital B (duplicate count=3).
10) Hospital C identifies a preliminary count of 20. To check for duplicates, it submits only the PII without any condition data and a query ID to Hospital A and B for comparison.
11) Hospital A compares the PII to its own results for query ID and returns the number of matches to Hospital C (duplicate count=1).
12) Hospital B compares the PII to its own results for query ID and returns the number of matches to Hospital C (duplicate count=3).
13) Hospital A returns a count of 10 and notes that duplicates with B=1 and duplicates with C=2.
14) Hospital B returns a count of 15 and notes that duplicates with A=1 and duplicates with C=3.
15) Hospital C returns a count of 20 and notes that duplicates with A=2 and duplicates with B=3.
16) The user then feeds these results into another service that processes the results and determines that the actual count is 39 based on the following calculated relationships: (A|B)-1; (A|C)-2; (B|C)-3. These reductions are subtracted from the preliminary total of 45 to yield 39.

Approach #2, while not requiring a trusted intermediary requires a lot of coordination with the queried nodes and also involves the sharing of patient info with other partners that may not be desired. Also, due to set math will grow exponentially with the number of nodes involved in the query (for 3 nodes, the number of service calls is 9 [3 from user to nodes, then 6 among the nodes-using n!/{n-r}!]; but for 4 nodes, the number of service calls is 16; and for 5 nodes the number of service calls is 25). This is also suspect due to my deficient math skills and just some back of the napkin calculations.

Anyway, I wanted to get these two approaches to de-duping out there and will update with more as I keep thinking.

PoiConDai II, revenge of PoiConDai

So, Poicondai II (Which I think is going to be called "PoiCenDai" because they are just called poison centers not poison _control_ centers anymore) is in the works.

The new service that will be feeding PoiCenDai is going to be giving back a lot more complex information, namely count aggregations based on region and time. Dr. Jeremy Espino has recommended Apache CXF as the service handler (which is basically the next generation of Apache AXIS, which will allow for this new PoiCenDai to work with globus that much more seamlessly should we need to distribute its operations on the grid) and using JAXB as the object marshaller (At least I am anticipating this because the last poison center webservice returned the data as an XML "any" stream with a "schema" stream)

Thus, the PoiCenDai service is going to need to take a command, determine the output type, marshal it all out, and then take the results and filter them into the appropriate regional polygons which were also drummed up with the help of the Gmap Polygon libraries.

Thus it's becoming one of those little moments where it's awkward figuring out where to start, whether I delve in with the new technologies for objectifying the PoiCen data, or start planning ways to keep methods for stuffing PoiCenDai data into polygons simple and relatively agnostic to JSPs.

I think I am going to focus on making the stubs for PoiCenDai closer to the JSP side for a couple of reasons... First off, I have just been dealing with the JSP end of things so it will be an easy bridge, second is that I am not sure of the service endpoints and will need to fiddle with JAXB and CXF and it would be good to know where the resulting objects are going to go.

Of course this is all liable to change as assumptions will probably be made and then be proven wrong or not feasible. Another wacky day in coding big enterprise type things.

Updates to the AMDS draft

After Tom received some feedback from Farzad Mostashari, we changed the AMDS schema a bit to make it more usable.
Now Level 1 has:
Syndrome
Syndrome classifier
Age Group (0<2;2<5;5<18;18<54;54<65; 65+)
Patient State
Patient Zip (either zip3 or zip5)
Date Start
Date End (in at least 1 day increments)
Count

We removed denominator as that's not really necessary (since you can query for all syndromes).

Level 2 (not developed yet):
Gender
Race
Ethnicity
Detailed Geography

Also, Dr. Lenert created different stages of access support for AMDS:
Stage 0: Scheduled transfers of AMDS documents to a supernode/central database
Stage 1: AMDS data available using secure grid services
Stage 2: AMDS data de-duped via secure grid services
Stage 3: AMDS data de-duped and combined together from multiple nodes (although after Stage 1, results can be combined together)

AMDS-RODS service development

I added a draft set of data structures for AMDS requests to the wiki. This led to further defining the initial set of operations for an AMDS compliant web service.

These operations include:
QueryAMDSMetadata
QueryFullAMDS
QuerySpatialAMDS
QueryTimeAMDS
QueryTimeSpatialAMDS

Based on these two pages, I created a sourceforge tracker item that Felicia is working on.

We're working on the RODS version first for a few reasons, primarily Peter and Felicia have familiarity with the RODS database structure and we don't have data structures for any other biosurveillance databases. We'd like to start working on ESSENCE and BioSense data structures to create AMDS-ESSENCE and AMDS-BioSense soon. John is working with partners to plan out how which services we build next.

Aggregate MDS Service planning

Tom and John presented the FY09 plan to the NCPHI governance council last week. A rather decent size block of work that the NCPHI R&D lab is planning is the development of a set of summary data services and associated coordination services.

These services will be an extension of the RODSA-DAI work and will provide access to various biosurveillance systems hosted by partners out in the states. Each service will return a set of aggregate syndrome counts that will map to a new common data structure that for now we are calling the "Aggregate Minimum Data Set". The AMDS will use the AHIC/HITSP MDS and select the minimum useful fields for aggregate reporting. The work on developing a scientifically vetted AMDS is in progress and so far is involving the Centers of Excellence and CDC BioSense personnel.

So the idea is that there will be multiple implementations of the AMDS services for each participating biosurveillance system and then a set of coordination services that know how to run a federated query across the participating nodes and combine together the results from the query. This is basically what the PHDGInet and RODSA-DAIWeb demos shows as a proof of principle, but 2009 will bring this to a proper pilot by developing services for actual installed biosurveillance systems running at partner sites.

Here's a list of the initial set of services that will support the pilot:

• AMDSX-DAI (Where X is one service for each participating system, in 2009 probably 5 different implementations planned for RODSA, ESSENCE, BioSense and specific state systems)


• AMDSCoordinator.RunFederatedQuery


• AMDSCoordinator.QueryAvailableCoverageArea

Globus service testing using SoapUI

One of the tools I like to use to test out web services is SoapUI (because it is free, open source, 100% Java and other superlatives). So far this is useful for calling public (anonymous access) services, but for secure services it didn't work out.

From the gt-users mailing list, Joel Scheider emailed me to let me know:

Using soapUI, it is possible to pass a client-side SSL credential to a web Service, e.g., for GSI Transport (TLS) authentication, but it's necessary to first convert the public/private key into Java keystore format, as described in Appendix A of this document:

http://agnis.googlegroups.com/web/UsingSoapUIWithAgnis.pdf

Instead of creating a proxy certificate, this method uses the client certificate directly, so delegation is not supported, but TLS authentication still works.

soapUI also claims to support WS-Security, but I haven't personally tried using that feature yet.

Thought this may be helpful for anyone else who needs a quick way to call Globus services.

PHGrid Service Registry 0.1 (beta)

When the PHGrid collaborators met after the PHIN Conference, one of the ideas was to set up a place where all the PHGrid collaborators can share descriptions of services they developed, services they are planning or even services they want to develop.

I just created a very basic service registry on the wiki available at http://sites.google.com/site/phgrid/Home/service-registry that contains the services that have been developed or that I know of as of now.

This registry initially serves as a human readable repository of service information, but eventually we will extend this with a UDDI registry and Index services that provides a standards based search for services.

This is of course a tiny step, but it is a start. Now we just have to get each service provider add in details and links for their services. Peter took the first step and went ahead and created a page for the RODSA-DAI services and client. This can be used as an example for how other services can be documented.

medLEE grid service

I met some of the grid architects from Ohio State University who were also attending the HealthGrid 2008 conference. OSU is working on grid services involving Columbia's Natural Language Processor, medLEE. I scheduled a call for next week to talk with some of the OSU team working with medLEE.

Since this is one of the activities we're working on for the Center of Excellence proof of concept project, it will be helpful to see what Ohio State's perspective is.