Forgive me technologists on this blog, because I am going to abuse terms that hold specific meaning in your world. With that, another important question -- what do we mean when we say "Public Health Grid"?
To some, it's the specific technologies. To others, it's a conceptual technical architectures. To me, it's the business of public health.
For us to develop a technical framework that meets the needs of the public health community, we need to understand the public health community, and how it works.
In my mind the Public Health Community is already set as distributed social relationships and funding mechanisms. A gaping hole is the access to information and the supporting information framework to support that distribution. This is the potential value of grid technologies -- they are the framework that fits. We just need to align them to the reality (and perhaps work on realigning some of the details of the social and funding frameworks, without altering the mission).
So, as a non-techie, in my world, the Public Health Grid is:
1. The Social Network of Public Health. This includes public health departments, clinical partners, academics, and industry. Note: Epidemiologists are one very small -- albeit important -- part of the that social network.
2. The Funding Network. This is what pays for the Social Network to exist. One open question is it aligned to where it needs to be?
3. The Technical Network. What we are trying to build now IS realigning the legacy to better meet the Social Network.
Bear in mind, the numerical order of these is intentional. Work needs to be done in all three, but we are at a point where the potential of 3 can be explored, but at its maximum when 1 & 2 are understood.
Wednesday, June 4, 2008
Is this the real Public Health Use Case?
The HealthGrid conference has been very good for me to stew on a number of important questions spawned from my favorite: "so what?"
What struck me more than anything in listening to the various folks from the clinical, research, bioinformatics worlds (the primary attendees of this conference) is that the real problem they are trying to solve is to get access to data from trustworthy sources so that each can do their job. So, in the world of public health, would it follow that the use case that matters most can be generalized as "Data Access".
If we solve that -- and then use the public health programs (biosurveillance, reportable disease surveillance) as the jargon to describe that -- do we solve 80% of the problem?
What struck me more than anything in listening to the various folks from the clinical, research, bioinformatics worlds (the primary attendees of this conference) is that the real problem they are trying to solve is to get access to data from trustworthy sources so that each can do their job. So, in the world of public health, would it follow that the use case that matters most can be generalized as "Data Access".
If we solve that -- and then use the public health programs (biosurveillance, reportable disease surveillance) as the jargon to describe that -- do we solve 80% of the problem?
2008 HealthGrid Conference (Chicago)
Met twice with Ian Foster, Jonathan Silverstein and their Argonne Globus colleagues. Discussed public health use cases. We also had an excellent meeting with the following Globus team members:
It is obvious that working directly with the Globus team will give us a much higher chance of success in achieving our vision of a national public health grid.
- Ravi Madduri, closely involved in caBIG
- Raj Kettimuthu, GridFTP project lead
- Frank Siebenlist, security architect
It is obvious that working directly with the Globus team will give us a much higher chance of success in achieving our vision of a national public health grid.
Tuesday, June 3, 2008
SVN and Maven
I have checked in most of my code into the Sourceforge SVN, and I just got the note that we should be using the package gov.cdc.ncphi..... and not org.cdc...
I guess I get to learn about subversion directory changes next :).
Otherwise, I have hit a challenge on the maven-ifcation of the code. Maven prefers jar files stored in remote repositories somewhere. All the OGSA-DAI and Globus code is set up with a series of build files (especially globus considering it has a lot of C++ code in addition to Java code) but there is no maven repository. I experimented with moving jar files into a resource location, but I think I am just going to have to store the jar files in the local repository with the appropriate maven metadata. I am not sure of the best way to do that, whether there is some sort of tool, or whether I should try and get maven building the OGSA-DAI code.
I guess what I am looking for now is a resource for taking projects that already exist and getting it to a point where maven is now managing it, because a lot of the stuff that was handled by changing the classpath before now is going to be handled by formatting the myriad dependencies into an xml file.
I feel like in the end we will have the local repository and we will be able to publish it to our own repository so that other development groups can reference our files with the appropriate sets of Ogsa-Dai libraries and globus libraries.
I also need to start researching how to implement a globus WS extension and looking into the best ways to get data back and forth between RODS and Ogsa-Dai.
I guess I get to learn about subversion directory changes next :).
Otherwise, I have hit a challenge on the maven-ifcation of the code. Maven prefers jar files stored in remote repositories somewhere. All the OGSA-DAI and Globus code is set up with a series of build files (especially globus considering it has a lot of C++ code in addition to Java code) but there is no maven repository. I experimented with moving jar files into a resource location, but I think I am just going to have to store the jar files in the local repository with the appropriate maven metadata. I am not sure of the best way to do that, whether there is some sort of tool, or whether I should try and get maven building the OGSA-DAI code.
I guess what I am looking for now is a resource for taking projects that already exist and getting it to a point where maven is now managing it, because a lot of the stuff that was handled by changing the classpath before now is going to be handled by formatting the myriad dependencies into an xml file.
I feel like in the end we will have the local repository and we will be able to publish it to our own repository so that other development groups can reference our files with the appropriate sets of Ogsa-Dai libraries and globus libraries.
I also need to start researching how to implement a globus WS extension and looking into the best ways to get data back and forth between RODS and Ogsa-Dai.
medLEE grid service
I met some of the grid architects from Ohio State University who were also attending the HealthGrid 2008 conference. OSU is working on grid services involving Columbia's Natural Language Processor, medLEE. I scheduled a call for next week to talk with some of the OSU team working with medLEE.
Since this is one of the activities we're working on for the Center of Excellence proof of concept project, it will be helpful to see what Ohio State's perspective is.
Since this is one of the activities we're working on for the Center of Excellence proof of concept project, it will be helpful to see what Ohio State's perspective is.
Medicus Install on Lab 1002
The attempt to build Medicus is failing due to the following missing project targets:
I am currently researching the issue and searching for additional documentation.
- Target `install' does not exist in this project.
- Target `deployService' does not exist in this project.
- Target `deployResource' does not exist in this project.
- Target `exposeResource' does not exist in this project.
I am currently researching the issue and searching for additional documentation.
Monday, June 2, 2008
High-Yield Public Health-Related Grid Services-Food for Thought
Had a very productive meeting with my colleagues today- around defining a handful of very high-yield and valuable services to be added to the core Globus services - which might be included with each Grid node.
The conclusion was the development of 3 very elegant, distinct and lightweight services that would be able to perform the following - given successful node installation:
1. Alerting to all (or a subset) of the members of the grid community. To improve the granularity of the notification, this service could leverage some form of the existing PHIN Directory developed by CDC.
2. Instant Messaging / Grid Chat - for those who want rapid communication / collaboration functionality between members of the PH Grid community.
3. A vocabulary service to facilitate the rapid standardization of local data sets, to facilitate data sharing and integration. As the use case for this service is refined, it is our hope that it can remain a very streamlined and lightweight service.
Given the availability of these services, a Grid portal must be created as well, to provide a user interface for all nodes/users to turn on/off the activity of these services.
Of course, all these services would only be available to those with digital certificates. And yes, the scaling of the digital certificate issues must be addressed, regardless of the other issues brought up in this post.
Look forward to further discussion on this.
Also working with Maven
Having just read the post below mine, I see that Anurag and I are on the same track.
I have created a "ncphi-examples" project to store all the little scripts and pages I have created already, and I now have moved in one of the peices of my modified OGSA-DAI code and already have it throwing compilation errors. I will probably fire off a quick email to Anurag about how best to import the OGSA-DAI code... at this point I am leaning towards just including it in the source tree, so that it will compile it locally and bundle it all into one big OGSA-DAI jar along with NCPHI-Specific modifications. By the end I imagine this particular Maven project will have the OGSA-DAI code, references to the jar repositories needed for builds provided by the maven site, and all the web code too.
As I play with Maven more and more, I find it perfect for just forcing you to have a sensible, realistic build structure. It will make you put your code in one place, your resources in another place, and show you the beauty of unit tests. In the end you get the deployable Jars and Wars that just make life that much easier... and I get the impression that your work will just be that much more legitimate when you go "oh, just install this maven tool, sync to the SVN, and then run 'mvn package' and you should be able to verify the compilation."
If anything this initial build will be a wonderful starting package for any future major NCPHI Globus OGSA-DAI code collaborations, and that is fortuitous considering I am also looking into defining the interface between RODS and OGSA-DAI for a outbreak detection solution that would use remote database access. Some of the interesting use of the extensible functionality that I can think of adding include the ability to deploy OGSA-DAI resources on the fly after filling out a simple form.
Otherwise, my vacation was lovely and personally productive, and it seems that a lot of documentation was completed after memorial day, but this gives me new solid directions and a lot of excitement.
I have created a "ncphi-examples" project to store all the little scripts and pages I have created already, and I now have moved in one of the peices of my modified OGSA-DAI code and already have it throwing compilation errors. I will probably fire off a quick email to Anurag about how best to import the OGSA-DAI code... at this point I am leaning towards just including it in the source tree, so that it will compile it locally and bundle it all into one big OGSA-DAI jar along with NCPHI-Specific modifications. By the end I imagine this particular Maven project will have the OGSA-DAI code, references to the jar repositories needed for builds provided by the maven site, and all the web code too.
As I play with Maven more and more, I find it perfect for just forcing you to have a sensible, realistic build structure. It will make you put your code in one place, your resources in another place, and show you the beauty of unit tests. In the end you get the deployable Jars and Wars that just make life that much easier... and I get the impression that your work will just be that much more legitimate when you go "oh, just install this maven tool, sync to the SVN, and then run 'mvn package' and you should be able to verify the compilation."
If anything this initial build will be a wonderful starting package for any future major NCPHI Globus OGSA-DAI code collaborations, and that is fortuitous considering I am also looking into defining the interface between RODS and OGSA-DAI for a outbreak detection solution that would use remote database access. Some of the interesting use of the extensible functionality that I can think of adding include the ability to deploy OGSA-DAI resources on the fly after filling out a simple form.
Otherwise, my vacation was lovely and personally productive, and it seems that a lot of documentation was completed after memorial day, but this gives me new solid directions and a lot of excitement.
Starting with maven
Worked on a sample project to illustrate how we can move towards a structured project which can be compiled with maven. Hopefully this will help in standardizing the application a little. Still have to meet Brian and Peter to discuss and finalize the project structure as I have tried to make it as general as possible. Once its done, it should become as simple as dropping your jsp or java file in the designated folder and compiling the whole app. in one go without worrying about the dependencies. Also, have to decide on the local repository url where we can store the jars relevant to our projects.
For more information on Maven: http://en.wikipedia.org/wiki/Apache_Maven
NPCHI GRID Research: Value to DiSTRIBuTE
I presented to members of the DiSTRIBuTE initiative at the Markle Foundation in NYC. Key points that were made during the presentation:
Steps to join the PHI Research Grid:
Installation of Globus Toolkit (Software) on Linux Operating System (Installation on Operating System Virtual
Machine (e.g., VMWare)). Required time- 30-45 Minutes
NCPHI Digital Certificate installation
Opening specific ports on existing firewall (24 hours - 24 days - depending on existing local policy)
Validate connection via Grid FTP
(ta da!)
Advantages of the grid:
Non-centralization of data
Data can remain on local node
Grid FTP (multipoint)
Access to non-centralized Grid services:
All nodes have the capability to run distributed analytics (customized) on demand
e.g., decentralize and open a DiSTRIBuTE analytics service
Advantages in eventually leveraging Grid:
Implementation of Additional Use Cases
Extensibility & Flexibility
Can leverage the use of Intelligent Agents
Afford new degrees of redundancy
In DiSTRIBuTE connecting to overall public health grid
Can put DiSTRIBuTE services on overall grid
Can leverage other services and data on overall grid for DiSTRIBuTE
Can develop a “DiSTRIBuTE summary data processing service” on grid
Significantly accelerate the growth of the DiSTRIBuTE network
Subscribe to:
Posts (Atom)
Posts
