PHMap and GIPSEPoison

So, I went away for vacation last week and when I got back, a lot of stuff had changed, but I feel it will make for better progress.

First up, I am supporting Chris in making PHMap... which may combine some of my service wrangling code and geo-fetching code with Chris' web code. This is fortuitous because Chris has a much better hang of actual web technologies like CSS, Spring, Javascript, and AJAX... and the concepts that make those all work smoothly together like REST and JSON. Thus, the code should be relatively clean and followable and adjustable, which is much harder when you have things like Java Objects which spitting out Javascript/HTML strings (like in my code).

This also means some of my service-wrangling and geo-fetching logic will probably be broken out or otherwise added-to to better enable the paradigm Chris wants to support . So essentially, refactors are upon us, and it will pretty much be the good old process of splitting function A into functions B and C since Chris will only need functon B.... and updating libraries and setting up property-less jar files for easy downloads instead of complicated builds.

Part of me feels a bit sheepish about how all the code sort of ended up mashed together and a lot of MVC lines were blurred... but then I remember that grid-viewer is essentially an evolution of RODSA-dai, which was a teeny little mashup that would have been overkill to load up with separate controllers and lots of different views. Then, suddenly, we needed polygons, and we needed to interact with the NPDS service, then we needed lots more UI tweaks and the ability to hit different services with different metadata.... and I must have been doing something right because I was able to make those modifications without too many large refactors, it's just now I get to deal with someone else actually needing to use it and not just install it, and their always runs the chance that the two paradigms for how something should work will be a bit obtuse, and larger refactors will be needed to accomodate both of them.

The other thing I will be doing mainly is getting GIPSEPoison working. I implemented a client and several tests and found that it is indeed getting data, but the aggregation needs to be tweaked, and some of the zip3 translations need to be taken into account. I think for that I will just split out some of the code I made for npdsgmaps (so it doesn't have any axis components) and make a new library that can be used by both npdsgmaps and GIPSEPoison.

Otherwise, lots of changes, lots of work, lots of progress, one hopes.

Cheers,
Peter

New time series handling.

I have adjusted the time series handling of gmap-polygon and grid-viewer so that regional collections hold multiple time series lists (differentiated by server).

Right now, it seems that it is building, but when you open the flot plot, all the data is somehow being tagged with one date, and it is not immediately apparent where or how that data is being set like that. I think I might have run into this problem before and it might have to do with how Java Calendars increments... I hope to figure it out later today or tomorrow.

Otherwise, after that is finished tomorrow, I will implement the new caching structure we discussed with Brian, and start publishing arrays so that the drop-downs for indicators and classifiers can be dynamically populated (as discussed with Chris).

After that, I am going to start allowing for multiple polygon loading... thus you can do one search, then another, and click and compare graphs. I might also look into making histograms in addition to line charts (that way, one can see the different data brought back by the different servers)

more services on gridviewer

There are now some more services on Grid-Viewer, and there are some visual tweaks.

Unfortunately, the data from multiple servers is not combining because of some back-end restrictions that I am going to work on next week, some additional refactors include separating polygons from pinpoints (thus, coastal states won't have really dark shadows (the result of multiple pinpoints being drawn in the same spot)), and storing data from multiple states in the markers (thus, flotplots should be able to distinguish which data came from which service in future versions).

Also, caching (with the help of Brian), fun Javascript and layout changes (with the help of Chris), and new multi-plotting and regional overlays.

Cheers everyone, have a good weekend!

grid-viewer updated and returning data

Yay! Grid-viewer and the Test Biosense GIPSE service have united on the training node to have grid viewer returning data (most of it is in January right now).

I'm happy because it is connecting over a secure connection to get both metadata and data, and it's relatively speedy, and allows me to move on with future plans for grid viewer and how it will behave.

First, I will be working with Chris to get a second GIPSE service testing, which will allow me to finish the server-selection logic (IE, you cannot select servers if the data you are looking for is out of their range. You won't be able to scan the North Carolina AMDS source for results in California or try to scan a BioSense node for Poison-based indicators).

Then, I am going to finish separating the selection bits from the mapping bits... which will require some generification: Right now, because it's based off of quicksilver, the map is expecting a series of states, a series of zip3s in a state, or a series of zip5s in a zip3... while it is already flexible enough to allow only a handful of states as opposed to all of them, I hope to expand it to allow for collections of zip3 (that may cross state borders) or zip5s (that may cross zip3 borders) that don't necessarily have to be related. It should also be possible to select states and zip3's at the same time (but I am not sure if that would be terribly helpful or just confusing).

Finally, I am hoping to allow for cumulative data loads. Thus, you can run one query, and then run another query with different pinpoints, thus you can click and compare the data between the two queries.

In addition to all this and in the process, I am hoping to clean up some of how gmap-polygon behaves so that less data has to be stored in databases and things aren't repeated as often.

It's going to be a lot of work, but I think the end results will be rather nifty, and I'm just happy to have this version all wired together and working so I can move on.

Service Changes

The spec for GIPSE has changed, and so has the client, since the last time I got grid viewer working. Thus, I have been doing a lot of refactoring, and planning more refactoring for some of the places we want to take grid viewer, and it's taking some time.

On the service side, all the names of the objects being returned have changed, and that means all the classes have to be changed and the ways they are loaded too. That also means the metadata has changed and needs to be set up.

Also, the grid service will now allow for cumulative loads... in that one can load more than one set of results onto the map.. which is a completely new paradigm that will have to be handled, in addition to the already shaky paradigms of multiple, variable regions. And the new idea of having services come from a central repository instead of a database (like a Wiki page or UDDI).

But, I think I have finally gotten it all mapped out in my mind, and have built the task lists, and have started the massive refactoring that will be needed.

The thing is, I know this will happen again (new service bits), so I need to keep in mind where all these changes occur and try to make them obvious and as isolated as possible. Then it is more likely that a change in the GIPSE structure can be propogated without a change in the grid view dynamics, and generally less changes the better.

Finally, I think I am going to introduce a simple "this is the data we got back from the service" page for the sake of debugging and sanity. It will help immensely to see what is coming back to figure out how the grid viewer is interpreting it and illuminating other options or assumptions.