Oh look, Zip3s have multiple polygons in them.

I am beginning to think there is a new adage: New Polygon, new thing discovered about your system that doesn't support the new polygon.

So, ZIP3's do not form one big contiguous block of area in all cases. In some cases, they will form multiple polygons because some zipcode changed from 61299 to 61304 or whatever.

But, the polygon system I had set up was expecting only one polygon per region. Thus, I will need to do a bit of an overhaul (it actually won't be too bad, yay object oriented coding) but will need to massage a new list of zip3's (and probably zip5s and states) out of the KML files.

Then, hopefully, I'll have entire states full of zip3s, and then be able to select zip5s within zip3s, and then comes the PoiConDai refurb.

package changes and SVN

So, most of this morning was spent doing some data file massaging into CSVs, which were then loaded into the postgresql database.

It was sort of humbling, because my prior approaches were to build a big parser library and then spend another day or two debugging it. In this case, I just took the files, got a macro on them, and them told the database to import them. The result is I spent just 2/3rd of a day getting all the data I wanted in the right spot. The thing that will be bad is I'll have to do it all again (well, probably spend 1/3 of a day) when database updates need to be made, then again, I probably would have to tweak things in my loader scripts if they needed to be made, and it's still the same cost to the end user... and in fact probably less. Before it was download files, download code, adjust properties, package, build, run. Now it's download CSV, create table in database, import CSV into table in database. Yeah. It's easier now.

Otherwise, there are lists of relational data (all the zip3s in Nevada, all the zip5s in a given zip3, etc). Now I get to start building the code libraries that will serve it up as needed. Before I did that I had to do a package update... which involves changing one line in a lot of places.

Luckily eclipse handled the refactoring for me beautifully, and it terribly broke the subversion management of it all. Thus, I spent the rest of the day deleting out the appropriate files and committing the changes by hand.

Tomorrow is the implementation of the lists, and updating documentation for the loader and the list-libraries and, well, everything else.

Setting up the locational library.

Well, right before I went off for the holidays, I completed the part of the gmap polygon libraries that would easily provide javascript for an array of polygons.

Now, I am in the process of building lists so that any application that wants to provide locations will be able to get lists of states, zip3's in said states, and zip5s in said zip3s. That, and improving the graphic awareness to be able to center on given areas after they are selected.

This of course means massaging large KML lists, and discussions on what would be most useful to someone trying to build their own databases on their own systems. Right now it seems the best answer would be "Comma delimited files", considering every database has a way to very quickly ingest such files.

I also am happy to have gotten some examples of text massaging using macros, and am once again making sure to earmark some time in the new year for basic scripting niftiness, namely because there are just some times when creative uses of grep and regular expressions will do so much more with a file than trying to create a SAX parser.