As part of the project planning that John Stinn is doing, I'm starting to draft up some architecture diagrams for how the AMDS services (AMDSS) will be built, accessed and deployed. The first in this series is a deployment diagram for CDC facing and Partner facing components. This is all fairly vanilla UML 2.x diagram notation-spec, wikipedia in case anyone is interested.
Please let me know your comments.
Friday, October 31, 2008
Thursday, October 30, 2008
We have polygons
We have polygons, and they are colored depending on the amount of calls within.
Tomorrow, I work on making popups depending on which polygon was clicked.
Tomorrow, I work on making popups depending on which polygon was clicked.
Wednesday, October 29, 2008
Oh those wacky geolocations.
So, yesterday I spent some time working up a nifty little display to make sure all the counties were showing up the way I needed them to.
I found out that most (about 45) of the counties in Colorado, actually showed up in Colorado... and some of them were showing up in other states.
Because I made a very naive assumption that County Names did not repeat in the US. And, like that assumption that birthdays don't cluster, it came as a nasty surprise. Thankfully, because I made sure to maintain the poicondai-loader project the way I did, it only took me the better part of an evening to change the loading scheme so that it supported county, state, and geolocation data.
The rest of today has been spent setting up some utility classes for pulling what I am calling PoiConCounts. I spent a lot of thought-chewing time figuring out the granularity of the objects for optimal caching and pulling (and many thanks to Brian for busting me out of the loop I was finding myself in) and just settled on "the thing that is going to be displayed". I have made this flexible, in case the way things need to be displayed or stored ever changes.
Tomorrow I will be trying to get that first county yank into a map with appropriate shading. If that goes well I will probably be ambivalent about whether to try and get the histogram-popup working or move on to zipcodes. But let's not fillet those fish yet...
I found out that most (about 45) of the counties in Colorado, actually showed up in Colorado... and some of them were showing up in other states.
Because I made a very naive assumption that County Names did not repeat in the US. And, like that assumption that birthdays don't cluster, it came as a nasty surprise. Thankfully, because I made sure to maintain the poicondai-loader project the way I did, it only took me the better part of an evening to change the loading scheme so that it supported county, state, and geolocation data.
The rest of today has been spent setting up some utility classes for pulling what I am calling PoiConCounts. I spent a lot of thought-chewing time figuring out the granularity of the objects for optimal caching and pulling (and many thanks to Brian for busting me out of the loop I was finding myself in) and just settled on "the thing that is going to be displayed". I have made this flexible, in case the way things need to be displayed or stored ever changes.
Tomorrow I will be trying to get that first county yank into a map with appropriate shading. If that goes well I will probably be ambivalent about whether to try and get the histogram-popup working or move on to zipcodes. But let's not fillet those fish yet...
Monday, October 27, 2008
Maps and Polygons and JDBC, oh my.
Today has been an odd day of sorts, but I got a lot done.
First, the big thing... I got the connectivity to the database working and drawing polygons. Now the next big thing is going to be having a poicondai-web page that sorts everything out by counties and tags the resulting polygons with pop-up counts.
Then, the next big thing is going to be implementing modular pop-ups that instead of just showing counts, show a Google-chart for the selected area with a histogram and what-not. But that might get over-ridden with the encoding of zip-codes.
Otherwise, I am in danger of hitting that ambivalence trap. I spent some time sprucing up the old poicondai chart-based application, and then found out it would have been ostensibly better to get the new map application working... but at the same time the sprucing-up I did is going to probably be pulled into the new poicondai-maps application... and there are lots of better ways I could have set up the database pulling, and I might just replace the whole JDBC framework with Hibernate... and there are lots of different things I could do and I seem to be focusing on the most visually nifty and code-based annoying.
Also, I have lots of little doubts. Despite many days of concept proofing I keep worrying about some functionality not being there and throwing me back to the drawing board (what if I cannot assign click handlers to polygons? Stuff like that).
But like most things... I find that the harder I work, the more luck I seem to have... and it's not like the loops and caches I would build aren't going to be useful if I have to move from a polygon-based map to a tile-based map.
So wish me luck... now I am just trying to get things on map attached to the numbers coming back from NPDS.
First, the big thing... I got the connectivity to the database working and drawing polygons. Now the next big thing is going to be having a poicondai-web page that sorts everything out by counties and tags the resulting polygons with pop-up counts.
Then, the next big thing is going to be implementing modular pop-ups that instead of just showing counts, show a Google-chart for the selected area with a histogram and what-not. But that might get over-ridden with the encoding of zip-codes.
Otherwise, I am in danger of hitting that ambivalence trap. I spent some time sprucing up the old poicondai chart-based application, and then found out it would have been ostensibly better to get the new map application working... but at the same time the sprucing-up I did is going to probably be pulled into the new poicondai-maps application... and there are lots of better ways I could have set up the database pulling, and I might just replace the whole JDBC framework with Hibernate... and there are lots of different things I could do and I seem to be focusing on the most visually nifty and code-based annoying.
Also, I have lots of little doubts. Despite many days of concept proofing I keep worrying about some functionality not being there and throwing me back to the drawing board (what if I cannot assign click handlers to polygons? Stuff like that).
But like most things... I find that the harder I work, the more luck I seem to have... and it's not like the loops and caches I would build aren't going to be useful if I have to move from a polygon-based map to a tile-based map.
So wish me luck... now I am just trying to get things on map attached to the numbers coming back from NPDS.
Friday, October 24, 2008
VMWare Appliance Now available online
Dan made a VMWare Appliance of the PHGrid Globus node and it's available at http://phgrid.net/documents/node/v1/.
If you download both of the files here, you can run a SUSE installation of the Globus Toolkit. This will make configuration easier as all you have to do is make a few configuration changes(set the IP, hostname, user accounts, request a user and host cert, etc).
This will reduce the install time down even lower (5 minutes) and allow PHGrid nodes to run on Windows (through the free VMWare player).
This first release is still pretty large (4.5GB download, 8GB necessary to run) so it may take you a while to download (took me over 30 minutes to upload).
If you download both of the files here, you can run a SUSE installation of the Globus Toolkit. This will make configuration easier as all you have to do is make a few configuration changes(set the IP, hostname, user accounts, request a user and host cert, etc).
This will reduce the install time down even lower (5 minutes) and allow PHGrid nodes to run on Windows (through the free VMWare player).
This first release is still pretty large (4.5GB download, 8GB necessary to run) so it may take you a while to download (took me over 30 minutes to upload).
The data, it is in the database.
So today, I managed to get the polygons for the ~3500 counties into a database.
I used maven (and found some places where maven was being very odd). I used SAX (and found some places where SAX was being very odd). I used eclipse (and.. well, you get the idea).
It was one of those weird coding days... where I was pressed on only by the faith I had that I could get it working and that my brain had set it all up in my mind. And thankfully that seems to be the case. I was going to try and just brute force REGEX/parse the XML string into the objects I needed without using any sort of XML parser (because I wasn't that familiar with SAX/DOM/Etc) but I found out that I couldn't really think of a way to do it... and that is a good thing because I'm sure whatever I would have turned out would have been a candidate for some of those "LOL at this horrible code" type websites.
Instead, I have a little SAX parser that is elegant, fast, and coupled to a little prepared-statement inserter that is also elegant and fast. And I should be able to use it relatively quickly for when I need to load zipcodes.
Now, the next big step ahead of me is to finalize the "pull these counties and then draw these counties" map for poicondai. Then I also have to do the same thing for zip codes. And then make everything look better.
Here's to demonstrations.
I used maven (and found some places where maven was being very odd). I used SAX (and found some places where SAX was being very odd). I used eclipse (and.. well, you get the idea).
It was one of those weird coding days... where I was pressed on only by the faith I had that I could get it working and that my brain had set it all up in my mind. And thankfully that seems to be the case. I was going to try and just brute force REGEX/parse the XML string into the objects I needed without using any sort of XML parser (because I wasn't that familiar with SAX/DOM/Etc) but I found out that I couldn't really think of a way to do it... and that is a good thing because I'm sure whatever I would have turned out would have been a candidate for some of those "LOL at this horrible code" type websites.
Instead, I have a little SAX parser that is elegant, fast, and coupled to a little prepared-statement inserter that is also elegant and fast. And I should be able to use it relatively quickly for when I need to load zipcodes.
Now, the next big step ahead of me is to finalize the "pull these counties and then draw these counties" map for poicondai. Then I also have to do the same thing for zip codes. And then make everything look better.
Here's to demonstrations.
Aggregate MDS Service planning
Tom and John presented the FY09 plan to the NCPHI governance council last week. A rather decent size block of work that the NCPHI R&D lab is planning is the development of a set of summary data services and associated coordination services.
These services will be an extension of the RODSA-DAI work and will provide access to various biosurveillance systems hosted by partners out in the states. Each service will return a set of aggregate syndrome counts that will map to a new common data structure that for now we are calling the "Aggregate Minimum Data Set". The AMDS will use the AHIC/HITSP MDS and select the minimum useful fields for aggregate reporting. The work on developing a scientifically vetted AMDS is in progress and so far is involving the Centers of Excellence and CDC BioSense personnel.
So the idea is that there will be multiple implementations of the AMDS services for each participating biosurveillance system and then a set of coordination services that know how to run a federated query across the participating nodes and combine together the results from the query. This is basically what the PHDGInet and RODSA-DAIWeb demos shows as a proof of principle, but 2009 will bring this to a proper pilot by developing services for actual installed biosurveillance systems running at partner sites.
Here's a list of the initial set of services that will support the pilot:
These services will be an extension of the RODSA-DAI work and will provide access to various biosurveillance systems hosted by partners out in the states. Each service will return a set of aggregate syndrome counts that will map to a new common data structure that for now we are calling the "Aggregate Minimum Data Set". The AMDS will use the AHIC/HITSP MDS and select the minimum useful fields for aggregate reporting. The work on developing a scientifically vetted AMDS is in progress and so far is involving the Centers of Excellence and CDC BioSense personnel.
So the idea is that there will be multiple implementations of the AMDS services for each participating biosurveillance system and then a set of coordination services that know how to run a federated query across the participating nodes and combine together the results from the query. This is basically what the PHDGInet and RODSA-DAIWeb demos shows as a proof of principle, but 2009 will bring this to a proper pilot by developing services for actual installed biosurveillance systems running at partner sites.
Here's a list of the initial set of services that will support the pilot:
- AMDSX-DAI (Where X is one service for each participating system, in 2009 probably 5 different implementations planned for RODSA, ESSENCE, BioSense and specific state systems)
- AMDSCoordinator.RunFederatedQuery
- AMDSCoordinator.QueryAvailableCoverageArea
Thursday, October 23, 2008
poicondai, and ESRI GIS presentation.
Poicondai is coming along nicely. It's not as far along as I would like, but then again there were a lot of things I discovered that needed to be done that I did not anticipate (building a mini-custom GIS database for the sake of drawing polygons with simple Google maps). Today I am building a little builder that will build the database I need, and I have proven that I can build the overlays I need in Google maps and they look sorta cool.
Otherwise, I have also been attending security meetings and I attended an ESRI-hosted GIS presentation where I learned the differences between Web Mapping Services (WMS) with support for Style Layer Descriptors (SLD), Web Coverages Services (WCS), and WFS (Web Feature Service) and the uber nifty WFS transaction abilities.
In short, all three sets of acronyms are something approved by the Open Geospatial Consortium (IE, they are accepted standards) for a client seeking geospatial data to better define what they need.
WMS pretty much allows for creating overlays and polygons over maps, and the added SLD's allow a client asking for data to say "I want these lines to be blue" or "I want these measurements made in metric and English and I want both returned".
WFS allows for more succinct geospatial objects like "This is a trail" or "this is a path in the woods" or "these are all the rest areas in the park". They are vector based and when transactional, allow the client to actually change the data in the hosting database (hence, if a trail is off, it can be corrected... after being locked, adjusted, and committed.).
WCS is a way of asking for area coverages using raster projections. Hence shaded areas and change diagrams (IE, how did Mount St. Helens look before and after the eruption... with pretty colors!)
ESRI makes server and client products that deal with all the different flavors and visualizations, and even support lots of open source and/or free visualization and server products.
The other cool thing is that KML pretty much supports data from all the methods and encompasses all the cool ways of serializing the GIS data. KML sort of sits between server strength and client strength values and serves as a major transport language.
So yeah, tomorrow I am hoping for polygon databases and displays.
Otherwise, I have also been attending security meetings and I attended an ESRI-hosted GIS presentation where I learned the differences between Web Mapping Services (WMS) with support for Style Layer Descriptors (SLD), Web Coverages Services (WCS), and WFS (Web Feature Service) and the uber nifty WFS transaction abilities.
In short, all three sets of acronyms are something approved by the Open Geospatial Consortium (IE, they are accepted standards) for a client seeking geospatial data to better define what they need.
WMS pretty much allows for creating overlays and polygons over maps, and the added SLD's allow a client asking for data to say "I want these lines to be blue" or "I want these measurements made in metric and English and I want both returned".
WFS allows for more succinct geospatial objects like "This is a trail" or "this is a path in the woods" or "these are all the rest areas in the park". They are vector based and when transactional, allow the client to actually change the data in the hosting database (hence, if a trail is off, it can be corrected... after being locked, adjusted, and committed.).
WCS is a way of asking for area coverages using raster projections. Hence shaded areas and change diagrams (IE, how did Mount St. Helens look before and after the eruption... with pretty colors!)
ESRI makes server and client products that deal with all the different flavors and visualizations, and even support lots of open source and/or free visualization and server products.
The other cool thing is that KML pretty much supports data from all the methods and encompasses all the cool ways of serializing the GIS data. KML sort of sits between server strength and client strength values and serves as a major transport language.
So yeah, tomorrow I am hoping for polygon databases and displays.
Wednesday, October 22, 2008
Globus service testing using SoapUI
One of the tools I like to use to test out web services is SoapUI (because it is free, open source, 100% Java and other superlatives). So far this is useful for calling public (anonymous access) services, but for secure services it didn't work out.
From the gt-users mailing list, Joel Scheider emailed me to let me know:
Thought this may be helpful for anyone else who needs a quick way to call Globus services.
From the gt-users mailing list, Joel Scheider emailed me to let me know:
Using soapUI, it is possible to pass a client-side SSL credential to a web Service, e.g., for GSI Transport (TLS) authentication, but it's necessary to first convert the public/private key into Java keystore format, as described in Appendix A of this document:
http://agnis.googlegroups.com/web/UsingSoapUIWithAgnis.pdf
Instead of creating a proxy certificate, this method uses the client certificate directly, so delegation is not supported, but TLS authentication still works.
soapUI also claims to support WS-Security, but I haven't personally tried using that feature yet.
Thought this may be helpful for anyone else who needs a quick way to call Globus services.
Tuesday, October 21, 2008
new techs on old demo
Today I managed to finish sprucing up the poicondai demo with better logging, and better handling of the new zipcode lists. I also started my research of the JQuery show/hide toggle so I could show/hide the raw zip-code list unless data was to be specified.
Tomorrow I hope to complete the show/hide, and get a prelim set of geographical outlays in poicondai with count bubbles. It will look a lot like rodsadai.
Otherwise, today we had a cool meeting on security, and we are going to doing a lot of data and application classifications and start laying out all of the different standards and policies... and just looking upon our data with more of a "security" eye. Thus, I am meeting a lot of new security focused folks who ask the important, retrospectively obvious questions about what services are present and what sorts of fields are used and whether they might be dangerous when boxed up and shipped somewhere to someone who hasn't seen them before.
Tomorrow I hope to complete the show/hide, and get a prelim set of geographical outlays in poicondai with count bubbles. It will look a lot like rodsadai.
Otherwise, today we had a cool meeting on security, and we are going to doing a lot of data and application classifications and start laying out all of the different standards and policies... and just looking upon our data with more of a "security" eye. Thus, I am meeting a lot of new security focused folks who ask the important, retrospectively obvious questions about what services are present and what sorts of fields are used and whether they might be dangerous when boxed up and shipped somewhere to someone who hasn't seen them before.
Subscribe to:
Posts (Atom)
Posts
