GIPSE Store update

I just updated the amds-db project so that it is renamed to gipse-store to update the database structure to support new stratifiers. Now the database structure supports new optional stratifiers of:
age
gender
service area (e.g. ED, Outpatient, Inpatient, etc.)
facility

in addition to the required stratifiers of:
indicator
region (zip5 and/or state)

Note, although the GIPSE store supports these new indicators, the BioSense GIPSE service will still only use the indicator and region stratifiers.

I'm making this change to support storage from the BioSense HIE feeds as well as adding in additional stratifiers that may be used by the CoEs and Regional Collaboratives.

Next up is the change to the AMDSService to rename to GIPSE.

This project has ddl and seed data for PostgreSQL (we use this for dev) and SQLServer (we use this for production). Should anyone require another RDBMS please let me know (or add them yourself).

GIPSE Loader

I added the top 30 BioSense Sub-Syndromes (as determined by Roseanne and Jerry) to the amds-db project and tested a load using the past 40 days of sub-syndrome RT data. This is increasing the size of the test data set that we use for testing the GIPSE services.

PS- In case you haven't noticed from Tom's massive renaming of all the AMDS-related wiki pages, AMDS has been renamed to Geocoded Interoperable Population Summary Exchange (GIPSE) by the NCPHI Director. So whenever you see GIPSE think AMDS.

AMDS Technical updates

I added a project called db-importer that can be used to import CSV (and eventually xml) AMDS extracts into an AMDS Store database based on the schemas we're using at CDC. This isn't the fastest or best importer, but it works on MS SQLServer and PostgreSQL and will be used to bring in daily report extracts from systems like BioSense.

I also talked a bit with Vaughn to transition over his projects. I'll be fixing up / re-writing AMDSCore and AMDSPoison over the next few weeks in time for the PHIN Conference until we find another Java developer.

Finally, we had a productive meeting with the PHIN Messaging steward and team about ways to directly collaborate in the solution development of PHGrid, PHINMS and PHIN-SRM. Nothing major was covered, but lots of good ideas that we'll follow up on in the near future.

AMDS Data Store

I checked in the first batch of database definitions for the AMDS Data Store to SVN. This is basically the DDL for all the tables, views, triggers, etc. that we're using for the AMDS Store that will be populated with, initially, synthetic BioSense data. This will help out anyone else who is looking to build their own aggregate store.

I'm also starting a list of conditions and classifiers that we'll use internall in the AMDS Store but will be useful to any others developing AMDS services.

Eventually this will have DDL for postgres, any other dbs we can create scripts for by request.

AMDS Store for BioSense data

So part of the BioSense program strategic plan is to share national data sources in a federated manner. In order to support this using grid and AMDS, we've started working with the BioSense Data Quality team (they run the analysis on the Data Mart) to generate some sample files that we can use to populate an AMDS data store with all the aggregate counts of BioSense syndromes by zip code and by day.

So far, we've got a sample file for all the zip codes covered by VA, DoD and Real-Time data. Of course all the counts are redacted as we don't deal with real data in the lab. But this will let me start working on an ETL process to get this into the AMDS extract data base that Vaughn is developing against.

So far it's only about 5MB/day of data and that's before I normalize out a lot of the content (we won't actually store syndrome names in the tables, etc. etc.) so this is still a good sign.

Also, it looks like Tom has started calling AMDS the Population Data Object (PDO) some times so we may be relabling soon. This actually makes more sense as we're not talking about a data set so much as we are a specification for sharing population data (as opposed to patient specific data that is NHIN's currency).