I spent most of my first week browsing the repositories and trying to get a grasp of the overall project objectives. And, of course, trying to learn the litany of acronyms (maybe we should start a post just to start documenting them for quick reference).
We did get a simple landing page put together. The team has built some terrific demonstration applications, so a summary page was needed to briefly describe their respective functionality and also launch to the actual demo. It is a simple html page with a little added javascript functionality. You can find it here.
I am thrilled to be working on this open source project that has enormous potential for public health informatics. There are really great individuals on the team and I am excited to be working with them.
Sunday, June 7, 2009
Friday, June 5, 2009
Service Changes
The spec for GIPSE has changed, and so has the client, since the last time I got grid viewer working. Thus, I have been doing a lot of refactoring, and planning more refactoring for some of the places we want to take grid viewer, and it's taking some time.
On the service side, all the names of the objects being returned have changed, and that means all the classes have to be changed and the ways they are loaded too. That also means the metadata has changed and needs to be set up.
Also, the grid service will now allow for cumulative loads... in that one can load more than one set of results onto the map.. which is a completely new paradigm that will have to be handled, in addition to the already shaky paradigms of multiple, variable regions. And the new idea of having services come from a central repository instead of a database (like a Wiki page or UDDI).
But, I think I have finally gotten it all mapped out in my mind, and have built the task lists, and have started the massive refactoring that will be needed.
The thing is, I know this will happen again (new service bits), so I need to keep in mind where all these changes occur and try to make them obvious and as isolated as possible. Then it is more likely that a change in the GIPSE structure can be propogated without a change in the grid view dynamics, and generally less changes the better.
Finally, I think I am going to introduce a simple "this is the data we got back from the service" page for the sake of debugging and sanity. It will help immensely to see what is coming back to figure out how the grid viewer is interpreting it and illuminating other options or assumptions.
On the service side, all the names of the objects being returned have changed, and that means all the classes have to be changed and the ways they are loaded too. That also means the metadata has changed and needs to be set up.
Also, the grid service will now allow for cumulative loads... in that one can load more than one set of results onto the map.. which is a completely new paradigm that will have to be handled, in addition to the already shaky paradigms of multiple, variable regions. And the new idea of having services come from a central repository instead of a database (like a Wiki page or UDDI).
But, I think I have finally gotten it all mapped out in my mind, and have built the task lists, and have started the massive refactoring that will be needed.
The thing is, I know this will happen again (new service bits), so I need to keep in mind where all these changes occur and try to make them obvious and as isolated as possible. Then it is more likely that a change in the GIPSE structure can be propogated without a change in the grid view dynamics, and generally less changes the better.
Finally, I think I am going to introduce a simple "this is the data we got back from the service" page for the sake of debugging and sanity. It will help immensely to see what is coming back to figure out how the grid viewer is interpreting it and illuminating other options or assumptions.
Wednesday, June 3, 2009
GIPSE Loader
I added the top 30 BioSense Sub-Syndromes (as determined by Roseanne and Jerry) to the amds-db project and tested a load using the past 40 days of sub-syndrome RT data. This is increasing the size of the test data set that we use for testing the GIPSE services.
PS- In case you haven't noticed from Tom's massive renaming of all the AMDS-related wiki pages, AMDS has been renamed to Geocoded Interoperable Population Summary Exchange (GIPSE) by the NCPHI Director. So whenever you see GIPSE think AMDS.
PS- In case you haven't noticed from Tom's massive renaming of all the AMDS-related wiki pages, AMDS has been renamed to Geocoded Interoperable Population Summary Exchange (GIPSE) by the NCPHI Director. So whenever you see GIPSE think AMDS.
old code running on new box
A lot of strange issues knocked down, some new ones cropped up.
Issues knocked down:
- Some of the m2eclipse issues: jar projects will find and eat other jar projects just fine, war projects still go "I can't copy this" and give errors that force you to go to the command line where it works just fine.
- Quicksilver, gmap-poly-web showing on my new box: Meaning I got all of the geodata and user data transferred to the new database and connecting okay. I found some rows that got omitted in the transfer, added them, and replaced the CSVs.
- gmap-polygon and gmap-polyweb version 1.0 have been updated to reflect their proper version in their pom files (before they were considered 1.1, not 1.0).
New issues:
- For some reason M2Eclipse will look up maven artifacts in brians version of eclipse, but not the one I installed.
- IE on my dev box is super-duper-secure. Meaning it doesn't like downloading little things like jQuery or google-map javascript IDEs (but I can look at it from other sources of javascript)
- The GIPSE spec, and resulting client, have changed a lot from prior loads. This means lots and lots of code needs to be updated in gridviewer not just to use all the peices, but to reflect all the metadata options and the like.
Issues knocked down:
- Some of the m2eclipse issues: jar projects will find and eat other jar projects just fine, war projects still go "I can't copy this" and give errors that force you to go to the command line where it works just fine.
- Quicksilver, gmap-poly-web showing on my new box: Meaning I got all of the geodata and user data transferred to the new database and connecting okay. I found some rows that got omitted in the transfer, added them, and replaced the CSVs.
- gmap-polygon and gmap-polyweb version 1.0 have been updated to reflect their proper version in their pom files (before they were considered 1.1, not 1.0).
New issues:
- For some reason M2Eclipse will look up maven artifacts in brians version of eclipse, but not the one I installed.
- IE on my dev box is super-duper-secure. Meaning it doesn't like downloading little things like jQuery or google-map javascript IDEs (but I can look at it from other sources of javascript)
- The GIPSE spec, and resulting client, have changed a lot from prior loads. This means lots and lots of code needs to be updated in gridviewer not just to use all the peices, but to reflect all the metadata options and the like.
Tuesday, June 2, 2009
new box, old code.
So, today has been a lot of "Try and get the code from the old system onto the new box"
And while it has not been particularly difficult, it has been rather tedious.
Most of the morning has been spent getting SQL Server Management Studio and figuring out that the data export for the CSV's of locational data from Postgres was done in some weird format (IE, dump from the terminal client into a text file) which caused all sorts of padding issues which needed to be repaired, and then updating the data back to the repository.
A lot of the afternoon was spent figuring out that there is a strange M2Eclipse behavior with our projects that keeps installed jars from being found (so, in the eclipse instance I would build gmap-polygon, install it, and then try to build gmap-poly-web, only to have it blow up because it couldn't find gmap-polygon.jar, even though it was installed... and when I did it from the command line, it worked just fine). So that is something I am going to have to debug because really, it is much easier to just right click and get the pretty gui to do it for you.
Another portion of the afternoon was spent fixing a vise-versa error where I thought that a branch was the spot for new code, and not the trunk... thus, I was moving recent code from branch locations back into the trunks (luckily, the eclipse/subclipse SVN browser works rather well for that sort of thing... but sourceforge's SVN is rather slow regardless)
By the end of today, I had some small issue that keeps me from being able to run any cool web-code on this new box:
Gmap-Poly-Web is not set to work with gmap-polygon version 1.1. I either need to upgrade gmap-poly-web to deal with the 1.1 version of gmap-polygon, or I need to download the 1.0 version of gmap-polygon and install it long enough to build gmap-poly-web (I will probably do the latter as gmap-poly-web functionality can be seen in both quicksilver and grid-viewer).
I don't have the username/password tables for poicondai on this box. Thus, I will need to find them and import them before I can log into poicondai.
Finally, AMDSCore has been completely replaced with a new service which will need to be integrated, the main deploy target is going to be having gridviewer working with this new service and thus providing more reliable data pulls.
This, if anything, has illuminated several points where we needed to update our "this is how you get/build/install this software" entries and shown us a few spots where we need to work to make things more automatic and user friendly if we want non-experienced programmers to have an easy time with it.
And while it has not been particularly difficult, it has been rather tedious.
Most of the morning has been spent getting SQL Server Management Studio and figuring out that the data export for the CSV's of locational data from Postgres was done in some weird format (IE, dump from the terminal client into a text file) which caused all sorts of padding issues which needed to be repaired, and then updating the data back to the repository.
A lot of the afternoon was spent figuring out that there is a strange M2Eclipse behavior with our projects that keeps installed jars from being found (so, in the eclipse instance I would build gmap-polygon, install it, and then try to build gmap-poly-web, only to have it blow up because it couldn't find gmap-polygon.jar, even though it was installed... and when I did it from the command line, it worked just fine). So that is something I am going to have to debug because really, it is much easier to just right click and get the pretty gui to do it for you.
Another portion of the afternoon was spent fixing a vise-versa error where I thought that a branch was the spot for new code, and not the trunk... thus, I was moving recent code from branch locations back into the trunks (luckily, the eclipse/subclipse SVN browser works rather well for that sort of thing... but sourceforge's SVN is rather slow regardless)
By the end of today, I had some small issue that keeps me from being able to run any cool web-code on this new box:
Gmap-Poly-Web is not set to work with gmap-polygon version 1.1. I either need to upgrade gmap-poly-web to deal with the 1.1 version of gmap-polygon, or I need to download the 1.0 version of gmap-polygon and install it long enough to build gmap-poly-web (I will probably do the latter as gmap-poly-web functionality can be seen in both quicksilver and grid-viewer).
I don't have the username/password tables for poicondai on this box. Thus, I will need to find them and import them before I can log into poicondai.
Finally, AMDSCore has been completely replaced with a new service which will need to be integrated, the main deploy target is going to be having gridviewer working with this new service and thus providing more reliable data pulls.
This, if anything, has illuminated several points where we needed to update our "this is how you get/build/install this software" entries and shown us a few spots where we need to work to make things more automatic and user friendly if we want non-experienced programmers to have an easy time with it.
Cancer Registry - Useful Information
As we look at ways that a PHGrid infrastructure could augment cancer registry activities - here are some very useful links:
NPCR-Advancing E-cancer Reporting and Registry Operations (AERRO) (previously MERP):
www.cdc.gov/cancer/npcr/informatics/merp. This site has updated Registry diagrams and use case documents that describe the cancer registration business.
North American Association of Central Cancer Registries (NAACCR) (umbrella organization to coordinate standards across the cancer registry community) http://www.naaccr.org/
Electronic Pathology Reporting standard developed by NAACCR that uses the HL7 2.3.1 standard (working on update to HL7 2.5.1) and HL7 Messaging WorkBench to validate the message content: http://www.naaccr.org/index.asp?Col_SectionKey=7&Col_ContentID=501
NAACCR transmission format that is used to transmit data from the hospital cancer registry and from the central cancer registry (state) to the national agencies can be found at: http://www.naaccr.org/index.asp?Col_SectionKey=7&Col_ContentID=133. On page 58 of this document is a table that lists all of the data elements with a cross mapping of the agency that requires collection.
You can find more information about all of the Registry Plus software tools at http://www.cdc.gov/cancer/npcr/tools/registryplus/. A link to additional information on Link Plus can be found at the same URL.
NPCR-Advancing E-cancer Reporting and Registry Operations (AERRO) (previously MERP):
www.cdc.gov/cancer/npcr/informatics/merp. This site has updated Registry diagrams and use case documents that describe the cancer registration business.
North American Association of Central Cancer Registries (NAACCR) (umbrella organization to coordinate standards across the cancer registry community) http://www.naaccr.org/
Electronic Pathology Reporting standard developed by NAACCR that uses the HL7 2.3.1 standard (working on update to HL7 2.5.1) and HL7 Messaging WorkBench to validate the message content: http://www.naaccr.org/index.asp?Col_SectionKey=7&Col_ContentID=501
NAACCR transmission format that is used to transmit data from the hospital cancer registry and from the central cancer registry (state) to the national agencies can be found at: http://www.naaccr.org/index.asp?Col_SectionKey=7&Col_ContentID=133. On page 58 of this document is a table that lists all of the data elements with a cross mapping of the agency that requires collection.
You can find more information about all of the Registry Plus software tools at http://www.cdc.gov/cancer/npcr/tools/registryplus/. A link to additional information on Link Plus can be found at the same URL.
Monday, June 1, 2009
AMDS Service Beta updates
I worked with the CSC/NEDSS programmer to test out their usage of AMDSService. I left out some of the boilerplate jars, but after adding those they were able to configure, build and successfully test out the AMDSService running on their local node. So this is progress.
New box!
So, a large portion of today was spent moving from the Linux development station I am used-to to a new Windows development station I am not-as-used-to.
Some of the things were easier to set up. The biggest help was being able to share other windows development boxes and being able to nab their already-downloaded copies of Java, eclipse, and Globus-ws core files. The setup for globus also seems to be a lot snappier in Windows (but at the same time, I am doing a much less involved install, and this isn't the first time I've done it).
Otherwise, I was able to build and run the client from the AMDS Service that was pulled over from another computer. Now I need to build and run the client from a fresh client downloaded from SVN and configured myself... so I anticipate a few "I am not sure what this property is" issues, but then it will be configuring the client, building the jars that grid viewer needs on my box (gmap-polygon) and figuring out what is needed between those steps.
Some of the things were easier to set up. The biggest help was being able to share other windows development boxes and being able to nab their already-downloaded copies of Java, eclipse, and Globus-ws core files. The setup for globus also seems to be a lot snappier in Windows (but at the same time, I am doing a much less involved install, and this isn't the first time I've done it).
Otherwise, I was able to build and run the client from the AMDS Service that was pulled over from another computer. Now I need to build and run the client from a fresh client downloaded from SVN and configured myself... so I anticipate a few "I am not sure what this property is" issues, but then it will be configuring the client, building the jars that grid viewer needs on my box (gmap-polygon) and figuring out what is needed between those steps.
NHIN Specification Factory
At the end of April, the NHIN released a detailed document outlining current NHIN specifications. The NHIN Specification Factory has been uploaded to the WIKI and can be downloaded at http://sites.google.com/site/phgrid/nhin-interoperability-1/SpecFactoryApprovedQ2Q3Tasking.ppt?attredirects=0 in the general section of NHIN Interoperability at http://sites.google.com/site/phgrid/nhin-interoperability-1 .
BioSense AMDS Service - Beta Release
The AMDS Service for BioSense is now in beta release. Please view details of how to download,configure, build, deploy and use on the service registry page.
You can also download the raw gar from sourceforge. But I recommend getting the source and building with your own configuration.
This service uses the updated 5/31 AMDS draft.
This service is specifically developed to share BioSense aggregate data over PHGrid, but can actually be used for any JDBC data source that wants to be shared using the AMDS spec.
This release is significantly different than the 4/30 alpha release. Specifically we're using Introduce 1.3 (big improvement over 1.2) for service development and configuration management and iBATIS for easy db access / ORM. This release is smaller than the alpha release in size and lines of code so theoretically it will be easier to use. Please let me know any comments. We'll be following the weekly build schedule with a target of July 8 for code freeze.
Update: Tom asked me to explain that we're using iBATIS rather than Hibernate. Both are decent JDBC/ORM frameworks. I chose iBATIS because it has a lighter footprint and I got it working in about 15 minutes. This doesn't mean we won't use Hibernate in the future, but just that we're using iBATIS for now.
You can also download the raw gar from sourceforge. But I recommend getting the source and building with your own configuration.
This service uses the updated 5/31 AMDS draft.
This service is specifically developed to share BioSense aggregate data over PHGrid, but can actually be used for any JDBC data source that wants to be shared using the AMDS spec.
This release is significantly different than the 4/30 alpha release. Specifically we're using Introduce 1.3 (big improvement over 1.2) for service development and configuration management and iBATIS for easy db access / ORM. This release is smaller than the alpha release in size and lines of code so theoretically it will be easier to use. Please let me know any comments. We'll be following the weekly build schedule with a target of July 8 for code freeze.
Update: Tom asked me to explain that we're using iBATIS rather than Hibernate. Both are decent JDBC/ORM frameworks. I chose iBATIS because it has a lighter footprint and I got it working in about 15 minutes. This doesn't mean we won't use Hibernate in the future, but just that we're using iBATIS for now.
Subscribe to:
Posts (Atom)
Posts
