NIGMS Invites Biologists to Join High-Throughput Structure Initiative
http://www.nih.gov/news/health/feb2009/nigms-12.htm
Saturday, February 28, 2009
Friday, February 27, 2009
Not only are there new calendar controls...
But now YOU get to play with them.
The new version of Quicksilver is on the staging node, Read about it and find the link to the app by scrolling through the QuickSilver Service Registry Entry in the PHGrid Wiki.
The service should no longer crash on the Java end... and if it does, I would very much appreciate a quick note about what you were doing to crash it. Otherwise, I suggest poking the service with something not Internet Explorer because Internet Explorer is still a bit testy about the large amounts of javascript processing and might throw an intermittent "Operation aborted" error.
Next week will be working on password-protection capabilities and cool new graphing software. I will also be trying to squish the IE bug and reduce load times. I am looking forward to it!
Cheers!
The new version of Quicksilver is on the staging node, Read about it and find the link to the app by scrolling through the QuickSilver Service Registry Entry in the PHGrid Wiki.
The service should no longer crash on the Java end... and if it does, I would very much appreciate a quick note about what you were doing to crash it. Otherwise, I suggest poking the service with something not Internet Explorer because Internet Explorer is still a bit testy about the large amounts of javascript processing and might throw an intermittent "Operation aborted" error.
Next week will be working on password-protection capabilities and cool new graphing software. I will also be trying to squish the IE bug and reduce load times. I am looking forward to it!
Cheers!
Thursday, February 26, 2009
We have calendar controls, we have clinical effects.
Tomorrow, Should everything go well, a much more functional version of Quicksilver should be deployed.
It will be able to select for dates and clinical effects, will display all the states from the national level, and will be able to zoom in on states (which will show a collection of zip3s) and on zip3s (which will show a collection of zip5s) and on zip5s.
There will be Google charts for all the regions that have a color (which means they have more than one instance) and I have not been able to get a nasty error thrown after doing a lot of odd and strange tests.
Next week will be the adding of cooler charts (with scroll and zoom and hopefully much more nifty statistical significance) and some security layer stuff so that it will be easier to password protect access to the page.
Cheers!
It will be able to select for dates and clinical effects, will display all the states from the national level, and will be able to zoom in on states (which will show a collection of zip3s) and on zip3s (which will show a collection of zip5s) and on zip5s.
There will be Google charts for all the regions that have a color (which means they have more than one instance) and I have not been able to get a nasty error thrown after doing a lot of odd and strange tests.
Next week will be the adding of cooler charts (with scroll and zoom and hopefully much more nifty statistical significance) and some security layer stuff so that it will be easier to password protect access to the page.
Cheers!
Grid Node Updates
In troubleshooting Felicia's issue, I noticed some links in the /etc/grid-security directory on Felicia's development node were pointing to missing files. That may have been the cause of the build errors. The links in question were:
/etc/grid-security/globus-host-ssl.conf
/etc/grid-security/globus-user-ssl.conf
/etc/grid-security/grid-security.conf
I copied the files back to the correct locations. I also corrected the permissions on the /etc/grid-security/grid-mapfile.
On The Staging Node:
I removed the last of the Health Grid certificates from the staging node and updated them with PHGrid host certificates. Peter verified OGSADAI was functional with the new certificates.
On the Windows node:
I installed the NCPHI/AMDSRODS service and verified it was running by listing the WSRF services remotely via port 8443. The available services listed under Windows NCPHI/AMDSRODS are:
amdsMetaDataQuery
amdsFullQuery
amdsTimeQuery
amdsSpatialQuery
amdsTimeSpatialQuery
getMultipleResourceProperties
getResourceProperty
queryResourceProperties
getServiceSecurityMetadata
I am also in the process of working on Felicia's Windows development node. That should be ready for her in the morning.
/etc/grid-security/globus-host-ssl.conf
/etc/grid-security/globus-user-ssl.conf
/etc/grid-security/grid-security.conf
I copied the files back to the correct locations. I also corrected the permissions on the /etc/grid-security/grid-mapfile.
On The Staging Node:
I removed the last of the Health Grid certificates from the staging node and updated them with PHGrid host certificates. Peter verified OGSADAI was functional with the new certificates.
On the Windows node:
I installed the NCPHI/AMDSRODS service and verified it was running by listing the WSRF services remotely via port 8443. The available services listed under Windows NCPHI/AMDSRODS are:
amdsMetaDataQuery
amdsFullQuery
amdsTimeQuery
amdsSpatialQuery
amdsTimeSpatialQuery
getMultipleResourceProperties
getResourceProperty
queryResourceProperties
getServiceSecurityMetadata
I am also in the process of working on Felicia's Windows development node. That should be ready for her in the morning.
GeoTwitter
Interesting post by Jim Tobias
GeoTwitter: http://geotwitter.org/
More information: http://ebiquity.umbc.edu/blogger/2007/03/08/twitter-google-maps-mashup/
Moritz Stefaner: Visual Thinker
This morning, I have discovered a visual thinker in Germany: Moritz Stefaner.
His material is really too incredible for words.
Well Formed Data Blog....: http://well-formed-data.net/
Relation Browser: http://der-mo.net/relationBrowser/
Examples: http://der-mo.net/relationBrowser/RB_v1.15_demo/generic%20springs/index.html
Well Formed Data Blog....: http://well-formed-data.net/
Gestalt: http://www.the-product.org/gestalt/
Open Processing Beta: http://www.openprocessing.org/
His material is really too incredible for words.
Well Formed Data Blog....: http://well-formed-data.net/
Relation Browser: http://der-mo.net/relationBrowser/
Examples: http://der-mo.net/relationBrowser/RB_v1.15_demo/generic%20springs/index.html
Well Formed Data Blog....: http://well-formed-data.net/
Gestalt: http://www.the-product.org/gestalt/
Open Processing Beta: http://www.openprocessing.org/
Wednesday, February 25, 2009
SSTS on Windows
Multiple Windows absolute and relative path names were added to the SSTS jndi-config.xml file, but the service continues to return the following error:
An error has occurred, error message = java.io.EOFException
This leads me to believe the service code needs to be modified in order to support Windows nodes.
An error has occurred, error message = java.io.EOFException
This leads me to believe the service code needs to be modified in order to support Windows nodes.
We now have zip5s. Next stop, Dates and Clinical Effects
Today I managed to get zip5s working on the demo node. I ran some pretty thorough tests and I don't think there are any bugs (which is sort of scary, I am sure I will find some tomorrow).
Tomorrow, I am going to pull over code for date selection and clinical effect selection from the old poicondai screen, make sure the values are going into the right places, and probably improve the clinical effect selector to allow for multiple selections (and maybe even use something from jQuery to allow for searching the set of possible clinical effects).
Friday, it should be posted to the web and able for people to play-with.. and I'll start drafting all the changes for making Gmaps-polygon and Quicksilver that much cooler, namely by adding more security and niftier graphs. All while testing Windows and SQL-Server compatibility.
Speaking of cool new graphs... check out some of the blog entries on the various statistical engines and AJAX visualization possibilities. I am probably going to pick one of the more AJAX (as opposed to flash or java-applet) friendly ones and tinker with it on Friday after the deploy. I would be looking at them more but I have been busy with just getting zip3's and zip5's to work without throwing big nasty errors. Many thanks to Brian Lee and William Duck for helping me look into the statistics engine pieces (and discussing the various benefits and drawbacks of each over email).
Also, a big thanks goes out to Dale, our grid-vmware-engineer as it were. He has been a real help and very considerate of our issues with the lab/demo environment, usually fixing problems within a day and working with us on planned outages and upgrades to minimize downtime and the need to restart things. He has been doing it for many, many months and I just realized I hadn't said anything about how hard he works to make sure we're good in terms of networks and Virtual Machines.
Tomorrow, I am going to pull over code for date selection and clinical effect selection from the old poicondai screen, make sure the values are going into the right places, and probably improve the clinical effect selector to allow for multiple selections (and maybe even use something from jQuery to allow for searching the set of possible clinical effects).
Friday, it should be posted to the web and able for people to play-with.. and I'll start drafting all the changes for making Gmaps-polygon and Quicksilver that much cooler, namely by adding more security and niftier graphs. All while testing Windows and SQL-Server compatibility.
Speaking of cool new graphs... check out some of the blog entries on the various statistical engines and AJAX visualization possibilities. I am probably going to pick one of the more AJAX (as opposed to flash or java-applet) friendly ones and tinker with it on Friday after the deploy. I would be looking at them more but I have been busy with just getting zip3's and zip5's to work without throwing big nasty errors. Many thanks to Brian Lee and William Duck for helping me look into the statistics engine pieces (and discussing the various benefits and drawbacks of each over email).
Also, a big thanks goes out to Dale, our grid-vmware-engineer as it were. He has been a real help and very considerate of our issues with the lab/demo environment, usually fixing problems within a day and working with us on planned outages and upgrades to minimize downtime and the need to restart things. He has been doing it for many, many months and I just realized I hadn't said anything about how hard he works to make sure we're good in terms of networks and Virtual Machines.
Wil Duck: Ideas regarding statistical api...
Take a look at the following api's I found for handling all the
statistical analysis for the charting tool and beyond:
statistical analysis for the charting tool and beyond:
1. http://www.rforge.net/JRI/, Basically this is Java/R interface that
allows you to run R inside java applications as a single thread. I know
that Barry has mentioned before that he likes R and would like to use it
in some capacity within BioSense. However I did not see anywhere in the
readme files about this api accepting URL calls. R is plenty robust
enough to handle any statistical analysis that could ever be requested
for syndromic surveillance purposes.
The following link is to JGR, a JAVA GUI for R that we could we have
some use for as well: http://jgr.markushelbig.org/JGR.html
2. http://www.goldb.org/statgen.html, StatGen is a Python api written
to parse log files and do statistical calculations for time series and
charting visualization purposes. However, when I looked at the class
and method details I did not see any reference to the ability to
calculate statistical significance of a point estimate using C2. Still
may be worth looking over for ideas though.
I will continue combing through sourceforge and other repositories for
ideas but these two popped out at me for different reasons. JRI seems
the way to go.
Wil
Tuesday, February 24, 2009
Bug smacking.
Continued smacking some bugs I was finding during my development tests... the more memorable one being crashing upon empty zip3 data sets.
I also started getting ready for zip5 collection, and continued discussions about the best way to get state or zip-code limited geocentric data back from the NPDS Service.
I have also started talking with Geographic Information Specialist named Julie from the Maryland Poison Center, and learned that some CDC NCPHI residents are also going to be around to learn more about Globus and the applications we are researching and piloting.
Otherwise, I am also running over, in the back of my mind, the upgrades of Quicksilver and NPDS-Gmaps. After Zip5s and User Interface controls are added, I feel like the next step is some subtle re-architecture of the apps so that they can deal with things like Clean error propagation and better value checking for security needs. It will also tie in nicely with implementation of the new statistical and charting methods so that the information returned by Poicondai can be highlighted with the benefit of actual statistical analysis and not just "hey, look, a graph."
I also started getting ready for zip5 collection, and continued discussions about the best way to get state or zip-code limited geocentric data back from the NPDS Service.
I have also started talking with Geographic Information Specialist named Julie from the Maryland Poison Center, and learned that some CDC NCPHI residents are also going to be around to learn more about Globus and the applications we are researching and piloting.
Otherwise, I am also running over, in the back of my mind, the upgrades of Quicksilver and NPDS-Gmaps. After Zip5s and User Interface controls are added, I feel like the next step is some subtle re-architecture of the apps so that they can deal with things like Clean error propagation and better value checking for security needs. It will also tie in nicely with implementation of the new statistical and charting methods so that the information returned by Poicondai can be highlighted with the benefit of actual statistical analysis and not just "hey, look, a graph."
Monday, February 23, 2009
Bugs fixed, but were breeding.
So, I fixed the Quicksilver bug about data replicating on the state-level on dev. Thus, after the next deploy, one will be able to load the states as many times as possible for a given date range and the data will stay the same.
Otherwise, I spent the rest of the day trying to find a fix for the "if I try and load California zip3's, the service crashes" problem... the root of the cause seems to be that I am generating a list of all the zipcodes in California and sending it to the Service, and the list is rather long, longer than the comfy limits of the service.
Luckily, I found a way around it, I can just send a request for the state and not all the zips in said state and it will still aggregate them by zip code.
The only problem is that I can't specify a state with the "main" aggregated national service. So I either have to pull back ALL the zip codes for the entire country (which would take a minute or two) or limited data from only the Colorado poison center. I have sent an email to the lovely poison people to see if there is something I am missing that will work for my needs.
But otherwise, yay. Less horrible crashes and strange behavior while we get the data sorted out.
The other cool thing is I had a chat with William Duck and he is pretty much a statistical Guru. He is helping me look for statistical and web-chart libraries that will help provide better and niftier information via-chart than the nifty, but not as functional google charts (or maybe even see if google charts can do more complicated stuff). That will be a massive help.
Otherwise, I spent the rest of the day trying to find a fix for the "if I try and load California zip3's, the service crashes" problem... the root of the cause seems to be that I am generating a list of all the zipcodes in California and sending it to the Service, and the list is rather long, longer than the comfy limits of the service.
Luckily, I found a way around it, I can just send a request for the state and not all the zips in said state and it will still aggregate them by zip code.
The only problem is that I can't specify a state with the "main" aggregated national service. So I either have to pull back ALL the zip codes for the entire country (which would take a minute or two) or limited data from only the Colorado poison center. I have sent an email to the lovely poison people to see if there is something I am missing that will work for my needs.
But otherwise, yay. Less horrible crashes and strange behavior while we get the data sorted out.
The other cool thing is I had a chat with William Duck and he is pretty much a statistical Guru. He is helping me look for statistical and web-chart libraries that will help provide better and niftier information via-chart than the nifty, but not as functional google charts (or maybe even see if google charts can do more complicated stuff). That will be a massive help.
Windows Node Errors
The SSTS service has been installed on the Windows node, but I don't think it is compatible with Windows in it's current stage of development. I think the path names need to be modified within the code in order for it to be compatible with Windows. Below is an example of the error message received while trying to connect to the SSTS hosted on a Windows node.
Current Error:
An error has occurred, error message = java.net.ConnectException: Connection timed out: connect
Here's an excerpt from the logs:
The bold portion of the log is the path that should be corrected in the code. Going forward, I think we should test code on both Linux and Windows.
Current Error:
An error has occurred, error message = java.net.ConnectException: Connection timed out: connect
Here's an excerpt from the logs:
Feb 23, 2009 2:48:26 PM org.apache.catalina.loader.WebappClassLoader validateJarFileINFO: validateJarFile(C:\Program Files\Apache Software Foundation\Tomcat 5.5\webapps\caGridTransfer\WEB-INF\lib\servlet.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class
Feb 23, 2009 2:48:28 PM org.apache.axis.utils.JavaUtils isAttachmentSupportedWARNING: Unable to find required classes (javax.activation.DataHandler and javax.mail.internet.MimeMultipart). Attachment support is disabled.
Feb 23, 2009 2:48:29 PM org.cagrid.transfer.service.TransferServiceImplINFO: Starting the TransferService ImplementationFeb 23, 2009 2:48:29 PM org.cagrid.transfer.service.TransferServiceImpl INFO: etc path directory = C:\Program Files\Apache Software Foundation\Tomcat 5.5\webapps\wsrf\WEB-INF\etc/cagrid_TransferService
Feb 23, 2009 2:48:29 PM org.cagrid.transfer.service.TransferServiceImpl
The bold portion of the log is the path that should be corrected in the code. Going forward, I think we should test code on both Linux and Windows.
Sunday, February 22, 2009
ArcGIS Server 9.3 Flex API; example app
Hello all.
The ESRI ArcGIS Server blog has an article about the Flex API for synchronizing
maps and data grids; The links below are the example application and the article.
If you mouse over a geographic unit...the items on the data table are linked and react to the area moused-over.....
ArcGIS Server 9.3 Flex API example application:
http://serverapps.esri.com/flex/sync_gr_dg/sync_gr_dg.html
ArcGIS Server Blog article:
http://blogs.esri.com/Dev/blogs/arcgisserver/archive/2009/02/19/Sychronizing-map-and-datagrid-interaction-with-the-ArcGIS-API-for-Flex.aspx
The ESRI ArcGIS Server blog has an article about the Flex API for synchronizing
maps and data grids; The links below are the example application and the article.
If you mouse over a geographic unit...the items on the data table are linked and react to the area moused-over.....
ArcGIS Server 9.3 Flex API example application:
http://serverapps.esri.com/flex/sync_gr_dg/sync_gr_dg.html
ArcGIS Server Blog article:
http://blogs.esri.com/Dev/blogs/arcgisserver/archive/2009/02/19/Sychronizing-map-and-datagrid-interaction-with-the-ArcGIS-API-for-Flex.aspx
Saturday, February 21, 2009
Friday, February 20, 2009
New stuff, and a bunch to do.
The updated demos for Gmap-Polygon and and Quicksilver are now on the web, read the Service Registry entries in the wiki to learn more.
In the meantime, there is still a bunch of stuff to be done on Quicksilver and a few things to tweak on Gmap-Polygon, so here comes the list.
There are probably a few things I am forgetting, and in addition to all these feature add-ons, I will be checking compatibility and configuration compatibility with different databases and the like.
Either way, I am looking forward to making things niftier and more robust. I hope you all have a good weekend!
Cheers,
Peter
In the meantime, there is still a bunch of stuff to be done on Quicksilver and a few things to tweak on Gmap-Polygon, so here comes the list.
- Gmap-Polygon
- Bugs
- Internet Explorer sometimes throws an operation aborted error. I am not sure what to do to fix this as it is sporadic, comes with no extra information, and is related to Javascript.
- Focus is maintained between different sessions, so someone on one computer has picked California, the other person logging in will see California.
- Internet Explorer sometimes throws an operation aborted error. I am not sure what to do to fix this as it is sporadic, comes with no extra information, and is related to Javascript.
- Speed
- Need to get smaller (less complicated, fewer vertices) zip3 polygons so that zip3s of states like California and Florida take less time to transfer and load
- Need to get smaller (less complicated, fewer vertices) zip3 polygons so that zip3s of states like California and Florida take less time to transfer and load
- Security
- Need to bubble-up errors and make it so that simple, non detailed error notifications are given in a production setting
- Need to check parameters for validity, warding off various injection attacks
- Need to bubble-up errors and make it so that simple, non detailed error notifications are given in a production setting
- Quicksilver
- Bugs
- Data is aggregating, not updating, in state counts. Need to make sure that time series replace in new polygons
- States with a lot of zip-codes are causing errors upon attempted zip3 loading, the string is overloading the service. Need to mitigate by seeing if ziplist length can be extended or by breaking up call into multiple lengths.
- State data is being loaded from the Colorado NPDS Service and not the global NPDS Service
- Data is aggregating, not updating, in state counts. Need to make sure that time series replace in new polygons
- Unimplemented features
- Zip5s need to be loaded and displayed
- The ability to select dates and conditions need to be added in accordance with the use case
- The graph-viewer needs to be updated to display a much more statistically useful graph (outliers flagged by different colors, possibly zoom-able)
- Zip5s need to be loaded and displayed
- Security
- Need to add security so that login/pass must be supplied before any map information is viewable.
- Need to double check values and propagate friendly/non-detailed error should values not be invalid.
- Need to add security so that login/pass must be supplied before any map information is viewable.
There are probably a few things I am forgetting, and in addition to all these feature add-ons, I will be checking compatibility and configuration compatibility with different databases and the like.
Either way, I am looking forward to making things niftier and more robust. I hope you all have a good weekend!
Cheers,
Peter
Thursday, February 19, 2009
Those thursday nights
Greetings all from the lab.
Tomorrow tends to be our "regular deploy day", which of course means I went "I am going to Get This Bug Fixed before I go home"
So, it's about 8:00PM, but I got that bug fixed, even if in testing I discovered about 3 other bugs.
But tomorrow, should the deploy go well, the zip3s in Quicksilver will behave that much better.
But woo-ey it was a doozy to get those zip3 loads behaving... and a bunch of the other bugs fixed over this last week. Unfortunately, it was a lot of stuff on the back end, so the front end probably won't change that much to the end user.
I guess that is just one of the facts of life sometimes, you can work for a week on a bunch of cool new improvements, and the user will never know it (But ooooh, do they notice it when something goes wacky).
Cheers
Tomorrow tends to be our "regular deploy day", which of course means I went "I am going to Get This Bug Fixed before I go home"
So, it's about 8:00PM, but I got that bug fixed, even if in testing I discovered about 3 other bugs.
But tomorrow, should the deploy go well, the zip3s in Quicksilver will behave that much better.
But woo-ey it was a doozy to get those zip3 loads behaving... and a bunch of the other bugs fixed over this last week. Unfortunately, it was a lot of stuff on the back end, so the front end probably won't change that much to the end user.
I guess that is just one of the facts of life sometimes, you can work for a week on a bunch of cool new improvements, and the user will never know it (But ooooh, do they notice it when something goes wacky).
Cheers
AMDS BioSense Extract
I loaded Dr. Tokar's synthetic BioSense data into the Lab's staging database. This gives us AMDS data for 2008 covering GA, SC, NC and IN for BioSense's 10 syndrome classifications.
I'll put the SQLServer DDL up somewhere as soon as we determine a proper way to share DDL.
Some may ask why we're using SQLServer when everything else we use is open source. The answer is because the CDC maintains a farm of SQLServer databases that are paid for at the enterprise level (i.e. "free" to the program). Everything we write will also be available for use in an OSS RDBMS (such as PostgreSQL or MySQL). So although we use SQLServer, nothing is hard wired to SQLServer.
I'll put the SQLServer DDL up somewhere as soon as we determine a proper way to share DDL.
Some may ask why we're using SQLServer when everything else we use is open source. The answer is because the CDC maintains a farm of SQLServer databases that are paid for at the enterprise level (i.e. "free" to the program). Everything we write will also be available for use in an OSS RDBMS (such as PostgreSQL or MySQL). So although we use SQLServer, nothing is hard wired to SQLServer.
Windows Node Progress
I installed Globus 4.0.8 on the Windows node to avoid the potential 4.0x and 4.2x compatibility issues. Eventually we will have to roll out an upgrade to 4.2x across the grid.
I was able to used the host certificates from staging to start the Windows container. The good thing is, Windows can use the certificates generated by the Linux nodes. Later we will have to solve the issue of generating correct subject names from Windows nodes. Until then, the work around is to initiate the user request on the Linux node and copy the certificates to Windows.
I also installed Apache Tomcat 5.5.27 and Apache Ant 1.7.1 in order to match the versions on staging. When I deployed Globus to Tomcat I received a couple minor errors.
Error 1:
C:\gt4>ant -f share\globus_wsrf_common\tomcat\tomcat.xml redeploySecureTomcat -D
tomcat.dir="C:\Program Files\Apache Software Foundation\Tomcat 5.5"
Unable to locate tools.jar. Expected to find it in C:\Program Files\Java\jre6\li
b\tools.jar
Buildfile: share\globus_wsrf_common\tomcat\tomcat.xml
The Fix: I copied tools.jar from the JDK directory to JRE directory it was looking for it in. This can also be fixed by changing the JAVA_HOME variable.
Error 2:
[copy] Copying 1 file to C:\Program Files\Apache Software Foundation\Tomcat
5.5\webapps\wsrf\WEB-INF
[copy] Warning: Could not find file C:\gt4\lib\xalan.jar to copy.
[copy] Copying 50 files to C:\Program Files\Apache Software Foundation\Tomc
at 5.5\webapps\wsrf\WEB-INF\lib
The Fix: I renamed xalan-2.6.jar to xalan.jar
This resulted in a successful deployment of the Globus container to Tomcat.
BUILD SUCCESSFUL
Total time: 1 second
The next test is to deploy one of Felicia's grid services to the Windows node and verify that it can be called remotely.
I was able to used the host certificates from staging to start the Windows container. The good thing is, Windows can use the certificates generated by the Linux nodes. Later we will have to solve the issue of generating correct subject names from Windows nodes. Until then, the work around is to initiate the user request on the Linux node and copy the certificates to Windows.
I also installed Apache Tomcat 5.5.27 and Apache Ant 1.7.1 in order to match the versions on staging. When I deployed Globus to Tomcat I received a couple minor errors.
Error 1:
C:\gt4>ant -f share\globus_wsrf_common\tomcat\tomcat.xml redeploySecureTomcat -D
tomcat.dir="C:\Program Files\Apache Software Foundation\Tomcat 5.5"
Unable to locate tools.jar. Expected to find it in C:\Program Files\Java\jre6\li
b\tools.jar
Buildfile: share\globus_wsrf_common\tomcat\tomcat.xml
The Fix: I copied tools.jar from the JDK directory to JRE directory it was looking for it in. This can also be fixed by changing the JAVA_HOME variable.
Error 2:
[copy] Copying 1 file to C:\Program Files\Apache Software Foundation\Tomcat
5.5\webapps\wsrf\WEB-INF
[copy] Warning: Could not find file C:\gt4\lib\xalan.jar to copy.
[copy] Copying 50 files to C:\Program Files\Apache Software Foundation\Tomc
at 5.5\webapps\wsrf\WEB-INF\lib
The Fix: I renamed xalan-2.6.jar to xalan.jar
This resulted in a successful deployment of the Globus container to Tomcat.
BUILD SUCCESSFUL
Total time: 1 second
The next test is to deploy one of Felicia's grid services to the Windows node and verify that it can be called remotely.
Wednesday, February 18, 2009
Oh, Internet Explorer... why do you not love GMap Polygon?
So, Internet Explorer seems to have a fickle relationship with GmapPolygon.
Sometimes, when it is loading a set of polygons, it will throw an error saying "The page at [address] cannot be loaded. Operation Aborted". This is a thoroughly unhelpful error, because it doesn't offer to show me the part of the code that broke or otherwise debug the issue. Furthermore, it's not re-create-able on demand. It tends to do it whenever it feels like it, for any given set of polygons... polygons which loaded fine not three seconds before.
Some of the blogs I checked (example) indicate that such errors arise because IE was processing some javascript, freaked out, and cancelled the page load... Usually because some spurious parent element is being referenced. The problem is that in the javascript I don't think any parent DOM elements are being referenced, just maps which are created inside the head script.
I'll admit, I am not that familiar with javascript, so nothing really stands out to me that makes me go "oh, that's what is making it blow up sometimes." So I invite anyone to look at the source of polygon pulls (I suggest poking the site with firefox) to see if anything stands out.
Otherwise, I imagine a lot of the problems involve the fact that a polygon is represented by a metric ton of text... especially polygons that border rivers and coastlines... such sweeping, beautiful crenulations that only nature can carve out are represented digitally by thousands of vertices. which all need to be pulled back from a database somewhere and then provided to google maps.
Meanwhile, I am attempting to make Gmap-Polygon get those back end database connections to be more robust and reliable, and improve caching so that the database doesn't even have to be hit that often... but we are also just looking at getting simpler polygons so that there is less data to pull, transfer, and store, which should have the biggest impact.
Sometimes, when it is loading a set of polygons, it will throw an error saying "The page at [address] cannot be loaded. Operation Aborted". This is a thoroughly unhelpful error, because it doesn't offer to show me the part of the code that broke or otherwise debug the issue. Furthermore, it's not re-create-able on demand. It tends to do it whenever it feels like it, for any given set of polygons... polygons which loaded fine not three seconds before.
Some of the blogs I checked (example) indicate that such errors arise because IE was processing some javascript, freaked out, and cancelled the page load... Usually because some spurious parent element is being referenced. The problem is that in the javascript I don't think any parent DOM elements are being referenced, just maps which are created inside the head script.
I'll admit, I am not that familiar with javascript, so nothing really stands out to me that makes me go "oh, that's what is making it blow up sometimes." So I invite anyone to look at the source of polygon pulls (I suggest poking the site with firefox) to see if anything stands out.
Otherwise, I imagine a lot of the problems involve the fact that a polygon is represented by a metric ton of text... especially polygons that border rivers and coastlines... such sweeping, beautiful crenulations that only nature can carve out are represented digitally by thousands of vertices. which all need to be pulled back from a database somewhere and then provided to google maps.
Meanwhile, I am attempting to make Gmap-Polygon get those back end database connections to be more robust and reliable, and improve caching so that the database doesn't even have to be hit that often... but we are also just looking at getting simpler polygons so that there is less data to pull, transfer, and store, which should have the biggest impact.
A non technology, non health blog that you may find interesting
Lately, I've been following reading a book and following a blog, each called 'Wikinomics: Exploring How Mass Collaboration Changes Everything' http://www.wikinomics.com/blog/. The points of view of the contributors to the book / blog are stunningly like those that contribute to this space. In fact, in the book, the authors make some bold proclamations that commercial interests that are closed, hierarchical, and essentially 'old-school' are now the 'losers' in the market. I have yet to find any consideration of how governments and / or health entities fare in these approaches. I wonder if we should consider ourselves a case study?
Tuesday, February 17, 2009
Lots of things, little and big
Sorry I haven't posted in a while, I have been dealing with a whole bunch of issues with Gmap-Polygon and the new Poicondai which has been dubbed "Quicksilver"
In short, there are bugs in both, gmap-polygon will be upgraded with more robust data persistence, we need to find ways to mitigate the issues that come with slightly different environments needing slightly different runtime variables (can we force them to be static? Is there a way to set them with an external config file?), and there is a lot more coding to be done, namely getting the zip3 graphs repaired and the zip5s displaying in Quicksilver... and also replacing the old style google graphs with niftier graphs that have cooler statistical analytics and coded outliers and the like.
I'll get the big list posted as soon as I find out a good way to get ordered list code posted into blogspot.
In short, there are bugs in both, gmap-polygon will be upgraded with more robust data persistence, we need to find ways to mitigate the issues that come with slightly different environments needing slightly different runtime variables (can we force them to be static? Is there a way to set them with an external config file?), and there is a lot more coding to be done, namely getting the zip3 graphs repaired and the zip5s displaying in Quicksilver... and also replacing the old style google graphs with niftier graphs that have cooler statistical analytics and coded outliers and the like.
I'll get the big list posted as soon as I find out a good way to get ordered list code posted into blogspot.
Fault Tolerence / Gold Build catalog
Thought Place Holder:
New node build level selection - We currently have a services registry on the PHGrid blog for the various services hosted throughout the grid. There should be an option WITHIN the grid that allows nodes to easily select and automatically deploy existing services for the purpose of fault tolerance or customization.
Functionality:
As new nodes come online, they are granted access to a services catalog that has the capability of automatically deploying and configuring services within the catalog. The model that I'm thinking about is similar to the auto provisioning capability of HP Data Center Automation (Formerly Opsware SAS) and Bladelogic Operations Manager's compliance capability.
Results:
1. This would drastically reduce the configuration time needed for new grid nodes.
2. It could open the door for location based service consumption.
3. Provide fault tolerance for existing services
4. Reduce rework at new grid sites
New node build level selection - We currently have a services registry on the PHGrid blog for the various services hosted throughout the grid. There should be an option WITHIN the grid that allows nodes to easily select and automatically deploy existing services for the purpose of fault tolerance or customization.
Functionality:
As new nodes come online, they are granted access to a services catalog that has the capability of automatically deploying and configuring services within the catalog. The model that I'm thinking about is similar to the auto provisioning capability of HP Data Center Automation (Formerly Opsware SAS) and Bladelogic Operations Manager's compliance capability.
Results:
1. This would drastically reduce the configuration time needed for new grid nodes.
2. It could open the door for location based service consumption.
3. Provide fault tolerance for existing services
4. Reduce rework at new grid sites
Monday, February 16, 2009
The reality of software development
The (London) Times had a front page recently on the juggernaut that is the NHS IT project. First there was a letter making the lunatic proposal that we commission our university computer departments to write the code 1. There's a nice response from Alan Pollard (British Computer Society) that makes the position clearer
Few IT projects fail because of technology. IT falls victim to overexpectation, unco-ordinated decision making, lack of clear objectives and relentless cost paring without a corresponding and realistic reduction in the desired outcome. All these are driven by those setting out the needs for planning, managing and approving the project. Today’s IT professional has to be far more than just a computer scientist. He or she has to be skilled in psychology, business matters, diplomacy, project management and politics.
Thursday, February 12, 2009
PHGrid/NHIN Interoperability
Dr. Savel and I were talking a bit today about how PHGrid and NHIN could interoperate.
NHIN works with two basic components: Gateways and Adapters. The Gateway is a common set of SOA infrastructure that is able to communicate with other Gateways within the NHIN. The Gateway is deployed within each participating network (e.g. CDC, NIH, DoD, VA, etc.) and uses an Adapter to translate NHIN traffic into the protocols necessary on the participating network.
Since the PHGrid set of web service protocols (WS-I, WS-Security, SOAP 1.1) are the same protocols used by NHIN, each PHGrid node is effectively an NHIN gateway. So any PHGrid node is able to communicate with any NHIN gateway (hosting services and calling services).
So what the PHIRG needs to test is security interoperability with NHIN. This can be accomplished by mapping an NHIN certificate to a PHGrid service and testing execution. We've done similar tests with Utah and caGrid networks, so this should be something worthwhile to test with ONC.
This has great potential as it would expand the reach of the NHIN into wherever PHGrid nodes are deployed (assuming the node security admins allow NHIN users to connect to their local node).
NHIN works with two basic components: Gateways and Adapters. The Gateway is a common set of SOA infrastructure that is able to communicate with other Gateways within the NHIN. The Gateway is deployed within each participating network (e.g. CDC, NIH, DoD, VA, etc.) and uses an Adapter to translate NHIN traffic into the protocols necessary on the participating network.
Since the PHGrid set of web service protocols (WS-I, WS-Security, SOAP 1.1) are the same protocols used by NHIN, each PHGrid node is effectively an NHIN gateway. So any PHGrid node is able to communicate with any NHIN gateway (hosting services and calling services).
So what the PHIRG needs to test is security interoperability with NHIN. This can be accomplished by mapping an NHIN certificate to a PHGrid service and testing execution. We've done similar tests with Utah and caGrid networks, so this should be something worthwhile to test with ONC.
This has great potential as it would expand the reach of the NHIN into wherever PHGrid nodes are deployed (assuming the node security admins allow NHIN users to connect to their local node).
Wednesday, February 11, 2009
Poison Use Cases
I thought it would be a good idea to document the use cases that Peter has been developing for PoiConDAI/PoiCenDAI. I did this because we've needed it for a long time and also because we're now actively engaging the AAPCC/BioSense epis on what functionality is required.
Documenting our use cases and revising them as necessary let us all build from a common picture.
View the use cases on the phgrid wiki.
Documenting our use cases and revising them as necessary let us all build from a common picture.
View the use cases on the phgrid wiki.
Tuesday, February 10, 2009
ALERT: Joint Statement from chairs of CCHIT, HITSP, and NeHC (AHIC Successor) released today!
2/10/2009
A Shared Roadmap and Vision for Health IT
John Halamka, Mark Leavitt, John Tooker
Introduction
Today’s economic crisis has highlighted our need for breakthrough improvements in the quality, safety and efficiency of healthcare. The nation’s business competitiveness is threatened by growing healthcare costs, while at the same time our citizens risk losing access to care because of unemployment and the decreasing affordability of coverage. Meanwhile, the quality variations and safety shortfalls in our care system have been well documented.
Health IT is not a panacea for all of these challenges, but it is a critical first step toward addressing many of them. Before we can restructure payment systems to reward quality, we need reliable, near real time data on outcomes. Before we can reward teamwork and collaboration that re-integrates care, we need applications that let clinicians communicate patient information instantly and securely. And in order to reverse the growing burden of chronic diseases, we need online connections that engage individuals in their care and motivate them to make healthier lifestyle choices.
Our current, paper-based health information process wastes hundreds of billions of dollars annually. Transforming this into a streamlined 21st century electronic system will require many components: a conversion to interoperable electronic health records (EHRs) at healthcare facilities, the adoption of online personal health records (PHRs) for individuals, health information organizations that support and connect these systems to allow information sharing, and finally a national health information network that allows instantaneous secure access – always with appropriate consent from the individual -- wherever and whenever their records are needed.
Where we stand today
There are hundreds of stakeholders in the development and adoption of interoperable healthcare information technology including consumers, providers, patients, payers, employers, researchers, government agencies, vendors, and standards development organizations. Over the past 20 years, these groups have worked together informally, but until recently there has not been a process to create a single list of priorities or a coordinated project plan. This fragmented approach in many ways mimics the fragmented healthcare delivery system within the US.
In 2004, the Office of the National Coordinator (ONC) within the Department of Health and Human Services (HHS) was established and charged with creating a single strategic plan for all these stakeholders to work together to harmonize healthcare data standards, create architectures for data exchange, document privacy principles, and certify compliant systems which adhere to best practices. Under ONC/HHS guidance, several groups have successfully implemented this work, leading to demonstrable progress in integrating some aspects of healthcare delivery.
An HHS advisory committee, the American Health Information Community (AHIC), prioritized needs and developed harmonized health IT standards for the country based on multi-stakeholder collaboration around a tool known as a “use case.” It produced 3 use cases in 2006, 4 use cases in 2007, 6 use cases in 2008, and a prioritized list of standards gaps to fill in 2009. The successor to AHIC, the National eHealth Collaborative, is a voluntary consensus standards body that extends the strengths of AHIC by enabling broader private sector and consumer representation. It will continue this work by developing and prioritizing initiatives to solve real implementation challenges in the field.
The Healthcare Information Technology Standards Panel (HITSP), a voluntary group of standards experts, received 13 use cases plus a privacy/security standardization request from AHIC. All of these use cases led to unambiguous interoperability specifications that were delivered within 9 months of receiving the request. The standards were chosen by consensus in an open transparent manner with many controversies resolved along the way. At this point, standards for personal health record exchange, laboratories, biosurveillance, medications, quality, emergency first responder access to clinical summary data, home health device monitoring, immunizations, genomic data, hospital to hospital transfers of records including imaging data, public health reporting and patient-provider secure messaging are finished. Consequently, standards are no longer a rate limiting step to data exchange in these cases.
The Certification Commission for Healthcare Information Technology (CCHIT) has certified over 160 electronic health record products based on detailed functional and standards conformance criteria. It has achieved broad industry recognition as the place to develop a roadmap for the features and interoperability requirements to include in the yearly revisions of health care IT products.
Using the harmonized standards, the Nationwide Health Information Network, a pilot initiative of HHS, demonstrated a successful architecture for pushing data between stakeholders, for query/response to pull data, and appropriate security protections. Many of these pilots have become production systems in their localities.
Working together, thousands of volunteer hours in these organizations have led to policy and technology frameworks that have been embraced by several live healthcare exchanges including those at the Social Security Administration, eHealth Connecticut, Keystone Health Information Exchange, Boston Medical Center Ambulatory EMR, Vermont Information Technology Leaders, Inc. (VITL), MA-Share (a statewide data exchange), and Beth Israel Deaconess Medical Center.
New Framework for Collaboration
While much has been accomplished, much remains to be done to accelerate adoption and interoperability of health IT. After an 18 month process involving hundreds of stakeholders, the National eHealth Collaborative (NeHC) was created to carry forward this work. NeHC is structured as a voluntary consensus standards body to bring together consumers, the public health community, health care professionals, government, and industry to accelerate health IT adoption by providing a credible and transparent forum to help establish priorities and leverage the value of both the public and private sectors. As a public private partnership, it is able to reach broadly into all sectors of health care, including health professionals, government agencies, health systems, academic medicine, patient advocates, major employers, non-profits, technology providers, and others.
This balancing of interests and expertise is critical to accelerating adoption and would be difficult to replicate in a purely public or purely private sector setting. Past competing interests and priorities within each sector have contributed to the historically low creation and adoption of compatible enabling technologies. By expanding the role of the private sector beyond what was available through a public-driven forum, NeHC can leverage industry resources and best practices—at the same time, assured public sector and consumer participation engenders activities that are transparent and supportive of high-quality, patient-centric coordinated care. The National eHealth Collaborative has refined and expanded the process for establishing priorities developed under AHIC. The National eHealth Collaborative’s goals for the prioritization process are to:
- Identify breakthrough strategies to increase interoperability by prioritizing stakeholder-initiated value cases for national action
- Provide broader stakeholder input into which value cases and interoperability initiatives are pursued
- Place more emphasis on the value proposition of each proposed set of interoperability initiatives.
Building on experiences with use cases, NeHC has developed the “value case,” a new tool for setting national priorities which describes the utility and projected benefits of an initiative addressing a specific obstacle to achieving interoperability. Value cases may focus on standards harmonization, but may also address other breakthrough strategies for driving interoperability, including model processes (such as a model of the “ideal” care coordination process); best practices (such as incorporation of ePrescribing into provider workflow or managing the communication of results out to the referring physician); and frameworks (such as a service oriented architecture for health information exchange). Each value case includes an assessment of the feasibility of implementing the proposed standard or other construct and the extent of stakeholder commitment required to ensure widespread adoption.
The processes and criteria to efficiently move the value case process forward begins with a national strategy and national call for submission of cases, both from government and the private sector. High level government participation plays a key role in guiding the value case process. As value cases are developed, NeHC will facilitate the appropriate action. If standards harmonization is required, HITSP will be consulted to develop use cases and recommend standards for adoption, or expert panels may be convened to address architectures, best practices, terminologies, or other issues. Once approved by the NeHC Board, outputs will be provided to CCHIT for potential incorporation into certification criteria and as a signal to developers for their product modifications.
Roadmap
Given the resources of the proposed stimulus package, our country is poised for great success in healthcare IT. As a nation, we will work together to ensure every patient has a secure, interoperable electronic health record. But what does this mean for patient care?
- We will improve the quality of care by coordinating handoffs between providers. No longer will you be asked to fill out the clipboard with the basics of who you are, what medications you take and your existing medical conditions.
- Medications will be checked for interactions as they are prescribed. Caregivers will be electronically notified of critical values in lab results and important results on x-rays.
- Patients will be able to access their medical records electronically, communicate with their doctors, and use home monitoring devices to coordinate care without a visit to the doctor’s office.
- Beyond these improvements in quality, safety, and convenience, the coordination of care will result in better value for our healthcare dollar by minimizing redundancy and waste.
The roadmap for standards harmonization, certification of healthcare IT products, and secure data sharing of medication, laboratory, and clinical summary information is clear. Completing this work is a journey and all our organizations, NeHC, HITSP and CCHIT, are unified to walk that road together.
The momentum created by the close collaboration of all these groups is based on trust, established working relationships and clearly defined roles/responsibilities. Together, they constitute a healthy ecosystem of organizations, each with clear accountability, transparency, and governance to ensure they are all aligned. We are committed to working together to meet the expectations of consumers and other healthcare stakeholders in the future.
Vision
The past four years have seen significant accomplishments, despite the limited funding made available. Beyond the complex mechanics of setting up these activities, what is probably more important has been the development of engagement and trust from stakeholders throughout the health care sector, something that can not be rushed. With the increased funding available in the economic stimulus legislation, we will build on the momentum, trust, and leadership that has already been painstakingly established.
Our vision is one of a 21st century health system in which all health information is electronic, delivered instantly and securely to individuals and their care providers when needed, and capable of analysis for constant improvement and research. With better information upon which to base decisions, the challenging process of health reform can successfully proceed – measuring quality, rewarding value, engaging individuals -- and lead the way to better health for all Americans.
------
John D. Halamka, MD, MS, is Chief Information Officer of Beth Israel Deaconess Medical Center, Chief Information Officer of Harvard Medical School, Chairman of the New England Health Electronic Data Interchange Network (NEHEN), CEO of MA-SHARE (the Regional Health Information Organization), Chair of the US Healthcare Information Technology Standards Panel (HITSP), and a practicing Emergency Physician.
Mark Leavitt, MD, PhD, is Chair of the Certification Commission for Healthcare Information Technology (CCHIT).
John Tooker, MD, MBA, FACP is the Executive Vice President and Chief Executive Officer of the American College of Physicians (ACP), Chair of the board for the National Committee for Quality Assurance (NCQA), and Chair of the board of the National eHealth Collaborative (NeHC).
SatScan to examine Poisonings in Texas
This paper describes how Dr. Ed Hsu used SatScan to examine poisonings by county in Texas:
http://www.ij-healthgeographics.com/content/3/1/25
I have spoken with Dr. Hsu at length over the phone and he was very kind and provided me with their SatScan parameter files for these analyses. I used these files to guide my work with Pertussis surveillance and the creation of SatScan cluster maps for Pertussis by county for the USA.
Jim
http://www.ij-healthgeographics.com/content/3/1/25
I have spoken with Dr. Hsu at length over the phone and he was very kind and provided me with their SatScan parameter files for these analyses. I used these files to guide my work with Pertussis surveillance and the creation of SatScan cluster maps for Pertussis by county for the USA.
Jim
Monday, February 9, 2009
Poison Visualization requirements
John and I met with Drs. Tokars, Rhodes and Hicks to go over some possible requirements for how to update the Poison Data Aggregation visualization (v1 available here).
The initial suggestion is to update the time series line graph to include a line representing the rolling 28 day average (14 days back and 14 days forward) as well as a star marking any data point that is deemed statistically significant.
This basically means that the current usage of google charts will get updated. Currently candidates are: jQuery, dojoX, dundas, rweb, flot
The initial suggestion is to update the time series line graph to include a line representing the rolling 28 day average (14 days back and 14 days forward) as well as a star marking any data point that is deemed statistically significant.
This basically means that the current usage of google charts will get updated. Currently candidates are: jQuery, dojoX, dundas, rweb, flot
PHGrid Architecture
We had a collaboration meeting with the PHIN-SRM (Secure Reliable Messaging) team and sponsors that was extremely productive. One of the immediate suggestions was to actually draw up some pictures of how we think PHGrid will look, what infrastructure is required and what core and application services will be necessary.
It's a daunting task, but should give us a good chance to update all the graphics on the wiki into something consolidated.
It's a daunting task, but should give us a good chance to update all the graphics on the wiki into something consolidated.
Friday, February 6, 2009
Grid appliances - Joseph Dell Email
Out of the University of Florida, is the Grid Appliance project at www.grid-appliance.org . I downloaded and had this up and running immediately. Although the security is not that of Globus, as soon as I started the VM session, it went out and connected to the grid fabric and was ready to transfer files or process jobs. As info… Joseph
Thanks, Joseph. Very very cool. It's nice that other folks are thinking as we are thinking...a little validating, to be sure.
Thursday, February 5, 2009
Following PHIN 2008 there was a discussion about the need to bring the service to the data. This is something I've been thinking a lot about and trying to plan accordingly for. Especially since the need to bring the service to the data keeps popping up again and again. Last week I called Wayne to discuss my work with the cloud/Nimbus and how Johns Hopkins and Utah might collaborate on a project that brings the service to the data. The document at the link below is an overview of one architecture that Wayne and I discussed.
http://sites.google.com/site/phgrid/Home/resource-documents/ConceptualOverviewofaPotentialArchitecturetoBringtheServicetotheData.doc?attredirects=0Spatial Networking.com
Imagine using a tool such as this for social network analysis in support of outbreak response: (also has a catchy soundtrack)
SPATIAL NETWORKING
http://www.spatialnetworking.com/
This link is similar, and allows opportunities for travel teams to have situational awareness of opportunities where
There travel may intersect: http://www.whereyougonnabe.com/
SPATIAL NETWORKING
http://www.spatialnetworking.com/
This link is similar, and allows opportunities for travel teams to have situational awareness of opportunities where
There travel may intersect: http://www.whereyougonnabe.com/
A picture of what to do...
I apologize for the cliche, but I want to draw out a picture with my words...
I see the efforts of the NCPHI Grid lab progressing towards sort of a city map of options and interests, and being that I live in Atlanta, it's (naturally) playing out as a wagon wheel with a big network of semi-clogged roads surrounded by a ring bypass with a light rail system that is trying it's best.
The way I see it, the suburbs and exurbs are like the user interfaces, and the downtown areas are the land of data.
The exurbs are like the user interfaces for people who don't really need all the data, just a summary... It would represent things like the "health channel" or widgets on the front page of "health.gov": Things that turn red if something bad is happening... or things that will send out an email if you need to be extra careful about washing your hands at a particular football stadium. The inner-city suburbs are more like maintenance interfaces where really hard-core statisticians go to conduct some hard-core informatics and data mining.
Right now, my impression of this city is that there are two or three really big skyscrapers (like Biosense databases) and a bunch of little warehouses (hospitals). But the problem is that all the roads between these things are gravel and the parking at the big skyscrapers is expensive. In the meantime all the secure trucks keep sinking and get stuck on rainy days, and no-one has really build up sets of nice subdivisions anywhere.
In the meantime, NCPHI is sort of building model homes in certain spots, paving over a few of the roads to make reach out on a few points (AMDS-GMAP/Rodsadai and NPDS-GMAPS/poicondai being suburbs on the north end) and building a train station at either end...
I feel like the end point we are trying to get to is filling in the map.
On the outside, lots of new UI's need to be developed, some really complex ones with lots of analysts, some very simple ones so that people can essentially Google their neighborhood to see if anything is amiss, lots of support to make such things automatically set up and install instead of being a 17 step rollout.
In the city, a lot of virtual and physical infrastructure needs to be set into place: connecting all the data silos and little warehouses together. Even having little services run around indexing and enriching (instead of this data point only having a zip code and an address, it now has a zip code, address, county, census track...) and backing up the data (so now if one service goes down it can be modified by other services), and also lots of different pathways to share the data, with large pipes for big file transfers and secure networks for identified patient data, and all the connections to get the data out to the UI suburbs.
That is sort of where I see thing should be, and I also don't think NCPHI can really build all of this, but can really just plot out the map and tell people "it would be really cool if you turned this field of nothing into a really cool alert application using this tech connecting to these services..." or going "make this but for this style of data coming from this aggregator".
So yeah, a little thought on how all this is set up.
I see the efforts of the NCPHI Grid lab progressing towards sort of a city map of options and interests, and being that I live in Atlanta, it's (naturally) playing out as a wagon wheel with a big network of semi-clogged roads surrounded by a ring bypass with a light rail system that is trying it's best.
The way I see it, the suburbs and exurbs are like the user interfaces, and the downtown areas are the land of data.
The exurbs are like the user interfaces for people who don't really need all the data, just a summary... It would represent things like the "health channel" or widgets on the front page of "health.gov": Things that turn red if something bad is happening... or things that will send out an email if you need to be extra careful about washing your hands at a particular football stadium. The inner-city suburbs are more like maintenance interfaces where really hard-core statisticians go to conduct some hard-core informatics and data mining.
Right now, my impression of this city is that there are two or three really big skyscrapers (like Biosense databases) and a bunch of little warehouses (hospitals). But the problem is that all the roads between these things are gravel and the parking at the big skyscrapers is expensive. In the meantime all the secure trucks keep sinking and get stuck on rainy days, and no-one has really build up sets of nice subdivisions anywhere.
In the meantime, NCPHI is sort of building model homes in certain spots, paving over a few of the roads to make reach out on a few points (AMDS-GMAP/Rodsadai and NPDS-GMAPS/poicondai being suburbs on the north end) and building a train station at either end...
I feel like the end point we are trying to get to is filling in the map.
On the outside, lots of new UI's need to be developed, some really complex ones with lots of analysts, some very simple ones so that people can essentially Google their neighborhood to see if anything is amiss, lots of support to make such things automatically set up and install instead of being a 17 step rollout.
In the city, a lot of virtual and physical infrastructure needs to be set into place: connecting all the data silos and little warehouses together. Even having little services run around indexing and enriching (instead of this data point only having a zip code and an address, it now has a zip code, address, county, census track...) and backing up the data (so now if one service goes down it can be modified by other services), and also lots of different pathways to share the data, with large pipes for big file transfers and secure networks for identified patient data, and all the connections to get the data out to the UI suburbs.
That is sort of where I see thing should be, and I also don't think NCPHI can really build all of this, but can really just plot out the map and tell people "it would be really cool if you turned this field of nothing into a really cool alert application using this tech connecting to these services..." or going "make this but for this style of data coming from this aggregator".
So yeah, a little thought on how all this is set up.
Wednesday, February 4, 2009
Unified Medical Language System (UMLS) Basics - Online Tutorial
If you are not familiar with the Unified Medical Language System (UMLS) - please read through the tutorial.
http://www.nlm.nih.gov/research/umls/online%20learning/index.htm
Tuesday, February 3, 2009
Well, we know it works in tomcat.
I have confirmed that the old version of poicondai (the CO centric version that you can see on the demo server now) will run in tomcat. Also, the lab engineer, Dale, has already whipped up a VM with SQL Server for testing, so that is awesome, and I should be able to test on it tomorrow.
Meanwhile, I have finished testing and creating an interface on top of the NPDS Axis service I had created, and now I have created a new project called npdsgmaps to essentially act as the mediator between the webapp, the gmap-polygon libraries, and the npds service. Thus it will make sure the time series of the polygons get populated appropriately, and that all the extra data for the web services gets passed in without the polygons having to worry about it. I am tentatively hoping I will get basic data in a rudimentary poicendai-map program (IE, a state and/or some zipcodes showing data) by tomorrow evening... but that is a lot of tentative and hope, I'll let you know where I am :D.
Cheers.
Meanwhile, I have finished testing and creating an interface on top of the NPDS Axis service I had created, and now I have created a new project called npdsgmaps to essentially act as the mediator between the webapp, the gmap-polygon libraries, and the npds service. Thus it will make sure the time series of the polygons get populated appropriately, and that all the extra data for the web services gets passed in without the polygons having to worry about it. I am tentatively hoping I will get basic data in a rudimentary poicendai-map program (IE, a state and/or some zipcodes showing data) by tomorrow evening... but that is a lot of tentative and hope, I'll let you know where I am :D.
Cheers.
DRN Demo preparation
Dan and I tested the manual steps for the DRN demo with Joel Spangler from Geisinger. We transferred up the user SAS program using Ross' naming conventions, Joel combined the user SAS program with the Geisinger template, Joel ran the SAS program, I copied down the output file and displayed it using Open Office's spreadsheet program. Then Dan and I ran through the same steps using the NCPHI node.
I also wrote a few simple bash shell scripts that will help the demo go faster when we're running within WebEx. These bash scripts will be replaced by Don Brown's perl scripts once they are ready.
Curiously, the SSTS client runs a lot more slowly from Linux nodes than from Dan's windows client setup. It's a significant time difference (60 seconds on Linux vs. 5 seconds on Windows) and we'll be investigating to determine what the difference is.
I also wrote a few simple bash shell scripts that will help the demo go faster when we're running within WebEx. These bash scripts will be replaced by Don Brown's perl scripts once they are ready.
Curiously, the SSTS client runs a lot more slowly from Linux nodes than from Dan's windows client setup. It's a significant time difference (60 seconds on Linux vs. 5 seconds on Windows) and we'll be investigating to determine what the difference is.
Monday, February 2, 2009
Poicendai, AXIS, CXF, and Service Mishaps. Oh My
Sorry if I have been a bit mum over the past couple of days, but I
have had my head buried on trying to get the NPDS Service to return
data, and after discovering some of the issues that were
(embarassingly) problems on my end... I got data back... so now I am
trying to parse it into objects that would be easy for GMAP Polygons
to understand... and I'm trying to leave wiggle room for Dr. Espino
who is working on a really net Mvn fueled CXF version of the parser,
which will be better than the AXIS version I have because it won't
conflict with globus.
have had my head buried on trying to get the NPDS Service to return
data, and after discovering some of the issues that were
(embarassingly) problems on my end... I got data back... so now I am
trying to parse it into objects that would be easy for GMAP Polygons
to understand... and I'm trying to leave wiggle room for Dr. Espino
who is working on a really net Mvn fueled CXF version of the parser,
which will be better than the AXIS version I have because it won't
conflict with globus.
That, and I have just had a bunch of fun new requirements: Poicendai
(and therefore gmap-polygon) will now need to run based on a different
SQL platform, so I get to migrate test that, and I need to make sure
it will run on standalone tomcat (I am pretty sure it will but I still
have to test it), and start researching authentication schemes.
That, and we need to find better ways to handle Friday deploys to make
sure everything works after deployment, and that people can test
deployments in case the code originator isn't present and able to run
the test.
So, it's going to be very busy here over the next week, because I was
planning/hoping to get an unembarassingly demonstrable version of
PoiCenDai out this week.
Defense Department sets up its own SourceForge
DoD has set up their own open source portal. This seems similar to what the biosurveillance folks have been thinking of configuring.
http://news.cnet.com/8301-13505_3-10154053-16.html
From the article linked above...
"Despite being based on SourceForge's technology, Forge.mil has one significant difference: security. As David Mihelcic, chief technology officer for the Defense Information Systems Agency, told Federal Computer Week, the Department of Defense's code repository has been 'upgraded to meet DOD security requirements,' with smart cards used to provide log-in credentials."
(comically forgemil is down right now so I can't verify with my own eyes)
http://news.cnet.com/8301-13505_3-10154053-16.html
From the article linked above...
"Despite being based on SourceForge's technology, Forge.mil has one significant difference: security. As David Mihelcic, chief technology officer for the Defense Information Systems Agency, told Federal Computer Week, the Department of Defense's code repository has been 'upgraded to meet DOD security requirements,' with smart cards used to provide log-in credentials."
(comically forgemil is down right now so I can't verify with my own eyes)
Subscribe to:
Posts (Atom)
Posts
