"A new effort to cull and table numerical data from Web pages was among several features announced today. Google Squared that creates tables of numerical data culled from searches of websites. In the example given, a search for "small dogs" created a table on different breeds, including data on such things as the breeds' heights and weights, placed inside boxes. Once an initial table is created, users can click on individual entries to check the source and--if the number is erroneous--correct the numbers through new searches. Finally, they can save their customized table for future reference."
I wonder how featurs like the "Google Squared" can be leveraged to take tons of unstructured available health information (or semi-structured, for example H1N1 line-list excel sheets sent from health departments all over the country/world, which are supposedly structured) and present it in a structured way amenable to statistical analysis.
read more: http://www.technologyreview.com/blog/editors/23522/?nlid=2025
Friday, May 29, 2009
Tutorial from Ohio Supercomputing Center
Tutorial from Ohio Supercomputing Center:
https://www.osc.edu/cms/sip/
https://www.osc.edu/cms/sip/
New section added to AMDS project on Wiki
Added new section to wiki in the AMDS projct with information on IXF and related resources. IXF is the Indicator eXchange Format - which closely adheres to the concepts of the Standard Data & Metadata eXchange (SDMX) protocol.
--Tom
Thursday, May 28, 2009
Launch of Open Mobile Consortium
Open Mobile Consortium: http://www.open-mobile.org/
Just been launched (May 26, 2009)
"The OMC is an unprecedented collaboration amongst nine high-profile organizations to develop an interopable set of platforms of high-quality open source mobile tools for humanitarian and civil society work." learn more here http://www.open-mobile.org/news/open-mobile-consortium-launches-open-source-mobile-tools-health-and-humanitarian-work
Tuesday, May 26, 2009
This may or may not be of interest to the Grid community.
I received an email regarding Intel's Parallel Studio (see below).
There is more information here:
http://software.intel.com/en-us/intel-parallel-studio-home/
Learn more: www.intel.com/software/parallelstudio
Jim Tobias
Download eval versions of
Intel® Parallel Studio.
Try the new software tools for yourself and build robust Windows* applications for multicore. See why Intel Parallel Studio got high marks during beta from industry leaders and C/C++ Microsoft Visual Studio* developers worldwide.
• Intel® Parallel Composer: Speeds up software development by incorporating parallelism with a C/C++ compiler and comprehensive threaded libraries. By supporting a broad array of parallel programming models, a developer can find a match to the coding methods most appropriate for their application.
• Intel® Parallel Inspector: This proactive bug finder is a flexible tool that adds reliability, regardless of the choice of parallelism programming models. Unlike traditional debuggers, Intel Parallel Inspector detects hard-to-find threading errors in multithreaded C/C++ Windows* applications and does root-cause analysis for defects such as data races and deadlocks.
• Intel® Parallel Amplifier: Assists in fine-tuning parallel applications for optimal performance on multicore processors by helping find unexpected serialization that prevents scaling.
I received an email regarding Intel's Parallel Studio (see below).
There is more information here:
http://software.intel.com/en-us/intel-parallel-studio-home/
Learn more: www.intel.com/software/parallelstudio
Jim Tobias
Download eval versions of
Intel® Parallel Studio.
Try the new software tools for yourself and build robust Windows* applications for multicore. See why Intel Parallel Studio got high marks during beta from industry leaders and C/C++ Microsoft Visual Studio* developers worldwide.
• Intel® Parallel Composer: Speeds up software development by incorporating parallelism with a C/C++ compiler and comprehensive threaded libraries. By supporting a broad array of parallel programming models, a developer can find a match to the coding methods most appropriate for their application.
• Intel® Parallel Inspector: This proactive bug finder is a flexible tool that adds reliability, regardless of the choice of parallelism programming models. Unlike traditional debuggers, Intel Parallel Inspector detects hard-to-find threading errors in multithreaded C/C++ Windows* applications and does root-cause analysis for defects such as data races and deadlocks.
• Intel® Parallel Amplifier: Assists in fine-tuning parallel applications for optimal performance on multicore processors by helping find unexpected serialization that prevents scaling.
Helping Dan debug Windows Container bugs
I've been helping Dan with some of the Windows bugs since I'm using a windows container to redevelop the AMDS services for the beta release.
I came across this bug when I tried globus-start-container.bat:
I thought I could fix this by running grid-proxy-init.bat prior to starting the container. This let Globus start up without error, but whenever I tried running a secure service I got an exception like this:
To fix this, you need to start up globus and with the "-containerDesc" parameter specifying a valid security_descriptor.xml (like the one found in $GLOBUS_LOCATION/etc/globus_wsrf_core) that specifies the location of the container cert and the container key. So your command would look something like "globus-start-container.bat =containerDesc c:\foo\security_descriptor.xml"
PS- For anyone trying to use the AMDS alpha, for the sake of debugging please wait for the Beta release on 5/31. The 5/31 release will re-write and fix a lot of what may be annoying anyone trying to use the current gar.
I came across this bug when I tried globus-start-container.bat:
Failed to start container: Container failed to initialize [Caused by: Secure con
tainer requires valid credentials]
I thought I could fix this by running grid-proxy-init.bat prior to starting the container. This let Globus start up without error, but whenever I tried running a secure service I got an exception like this:
Error: ; nested exception is:
org.globus.common.ChainedIOException: Authentication failed [Caused by:
Operation unauthorized (Mechanism level: Authorization failed. Expected "/CN=hos
t/somehostname.cdc.gov" target but received "/O=PHGRID/OU=phgrid.net/OU=Globus Public
Health NCPHI/OU=phgrid.net/CN=someuser/CN=someuser")]
To fix this, you need to start up globus and with the "-containerDesc" parameter specifying a valid security_descriptor.xml (like the one found in $GLOBUS_LOCATION/etc/globus_wsrf_core) that specifies the location of the container cert and the container key. So your command would look something like "globus-start-container.bat =containerDesc c:\foo\security_descriptor.xml"
PS- For anyone trying to use the AMDS alpha, for the sake of debugging please wait for the Beta release on 5/31. The 5/31 release will re-write and fix a lot of what may be annoying anyone trying to use the current gar.
Troubleshooting Globus Windows Authorization Errors
We were able to get CSC to the point where they are able to start the container and access the CounterService locally. This means that they have a functional grid node.
The problem we are facing now is the ChainedIOException error that is thrown while accessing the CounterService on the NCPHI node. I provided CSC with a copy of our Server.xml in order to synchronize our configurations. I also had them update the global_security_descriptor.xml with the correct information that allowed them to start the container.
We are still researching the root cause of the problem.
The problem we are facing now is the ChainedIOException error that is thrown while accessing the CounterService on the NCPHI node. I provided CSC with a copy of our Server.xml in order to synchronize our configurations. I also had them update the global_security_descriptor.xml with the correct information that allowed them to start the container.
We are still researching the root cause of the problem.
Friday, May 22, 2009
AMDS Draft Schema Work
After some productive conversations with Bill and Ian from the University of Washington CoE, I've made some changes to the forthcoming May 31 draft of the AMDS schema. The changes are checked into SVN and can be viewed here (schema), here (metadata response example), here (request example), here (response example).
Please take a look and let me know your comments. This does a few things:
1) It flattens the xml structure as much as possible. Simplicity is good.
2) It allows for more fine-grained geographic regions to be selected. We now have wildcard (*) and parent qualifiers so you can stratify your result however is appropriate.
3) It adds additional, optional stratifiers of Age, Facility and Service Area (called "bucket" by BioSense). These optional stratifiers won't be supported by all AMDS services/providers/publishers but they are now in the schema to allow for it. DiSTRIBuTE supports these stratifiers.
4) Renames "condition" to "indicator" since AMDS can really be used to query and report any population indicator as definied by the AMDS provider.
I plan on using this schema for the service operations on the AMDS-BioSense beta service (due May 31).
One thing that is becoming apparent is that we will need a good registry / registry services to capture all the metadata for the indicator classifiers, age classifiers, service area classifiers, facility classifiers, etc. that are used by all of the AMDS services. I'm noting this now, but it won't be built for a while (for the typical reasons).
Finally, I'll add that this is a very DRAFT schema and will likely be updated again between now and PHIN (especially based on your comments). We will also change a lot once the science side of the house determines what standards we need to meet (Ken is working on this with the epidemiologists).
Please take a look and let me know your comments. This does a few things:
1) It flattens the xml structure as much as possible. Simplicity is good.
2) It allows for more fine-grained geographic regions to be selected. We now have wildcard (*) and parent qualifiers so you can stratify your result however is appropriate.
3) It adds additional, optional stratifiers of Age, Facility and Service Area (called "bucket" by BioSense). These optional stratifiers won't be supported by all AMDS services/providers/publishers but they are now in the schema to allow for it. DiSTRIBuTE supports these stratifiers.
4) Renames "condition" to "indicator" since AMDS can really be used to query and report any population indicator as definied by the AMDS provider.
I plan on using this schema for the service operations on the AMDS-BioSense beta service (due May 31).
One thing that is becoming apparent is that we will need a good registry / registry services to capture all the metadata for the indicator classifiers, age classifiers, service area classifiers, facility classifiers, etc. that are used by all of the AMDS services. I'm noting this now, but it won't be built for a while (for the typical reasons).
Finally, I'll add that this is a very DRAFT schema and will likely be updated again between now and PHIN (especially based on your comments). We will also change a lot once the science side of the house determines what standards we need to meet (Ken is working on this with the epidemiologists).
Thursday, May 21, 2009
Metadata niftiness
Grid viewer now has servers which are only selectable if the regions selected are chosen. Drop downs are now more based on metadata and not old quicksilver facets.
The next steps will be expanding the conditional handling, futhering the migration from a single-pane to two-panes with the top focusing on metadata and selections and the bottom focusing on the map and repeated queries.... and dealing with new servers and services and making sure all the metadata performs as expected (and probably a few paradigm shifts to allow changes in what 'expected' is).
The next steps will be expanding the conditional handling, futhering the migration from a single-pane to two-panes with the top focusing on metadata and selections and the bottom focusing on the map and repeated queries.... and dealing with new servers and services and making sure all the metadata performs as expected (and probably a few paradigm shifts to allow changes in what 'expected' is).
Thinking about Laboratories and PHGrid
After giving a presentation this afternoon to the PHIN Laboratory Messaging Community of Practice, I worked with Brian to diagram a Proof of Concept that would examine introducing a suite of laboratory services on the PHGrid infrastructure - leveraging the significant work that has taken place in the development of STARRS (Specimen Tracking and Results Reporting System) and LUNA.
Subscribe to:
Posts (Atom)
Posts
