One of our partner asked for better distribution of Denominator and flu data on our sample node's GIPSEService (https://ncphi.phgrid.net:8443/wsrf/services/gipse/GIPSEService).
So I generated some synthetic data for the 12 BioSense Denominator and flu indicators (EDVisits,EDVisits-LowTemp,EDVisits-HighTemp,EDVisits-UnknownTemp,ILI-b,ILI-b-LowTemp,ILI-b-HighTemp,ILI-b-UnknownTemp,ILI-n,ILI-n-Lowtemp,ILI-n-HighTemp,ILI-n-UnknownTemp) for Jan,Feb,March,April and June.
The values are all random but at least you can generate some ratios and percentages and such (previously we only had about 35 days spread out over the year).
So please check out gridviewer and the service API and let me know if it is helpful at all.
Showing posts with label testing. Show all posts
Showing posts with label testing. Show all posts
Friday, August 7, 2009
Wednesday, June 24, 2009
Chronicles of NHIN CONNECT, volume 1
For the past few weeks, I have been in the process of becoming an expert in the NHIN CONNECT project. This project contains both the NHIN connect GATEWAY and the NHIN Connect ADAPTER.
The NHIN Connect project is located at http://www.connectopensource.org and version 2.0 is the current release of the code in question (although 2.1 is supposed to be forthcoming in early July).
While I can now configure a gateway and an adapter in under 1 hour each, there are several 'gotchas' that are not addressed in the documentation. Using the "pre-configured binaries' option (versus install-from-scratch), here are the 'gotchas' so far:
==>Adapter and Connect need different machines. I've tested them on the same one and functionality is mediocre at best. The lab has these set-up nicely on 2 seperate but equal machines.
==>Java is set to allocate 1.2 gb of memory from the start. Don't try this on a machine with less than 1 gb of memory... you want at least 2.
==>OID Registration doesn't work as advertised. Fortunately I haven't needed my OID for internall connectivity but when I connect outside of the lab, I will need this.
==>c:\java is hard-coded as java location. Make sure you install to this location. If you don’t some of the NHIN services break with odd errors. The documentation reflects inaccurate pointers to the java locations. And with _some_ of the application hard-coded with this java location, better safe than sorry.
==>The NHIN documentation says in multiple places that port 9080 is the non-secure port and that 9081 is the secure port. DON'T BELIEVE IT! Port 8181 is the secure port.
Thus concludes volume 1 of the Chronicles of NHIN CONNECT. Stay tuned for updates as the PHGRID <--> NHIN CONNECT interoperability testing continues...
The NHIN Connect project is located at http://www.connectopensource.org and version 2.0 is the current release of the code in question (although 2.1 is supposed to be forthcoming in early July).
While I can now configure a gateway and an adapter in under 1 hour each, there are several 'gotchas' that are not addressed in the documentation. Using the "pre-configured binaries' option (versus install-from-scratch), here are the 'gotchas' so far:
==>Adapter and Connect need different machines. I've tested them on the same one and functionality is mediocre at best. The lab has these set-up nicely on 2 seperate but equal machines.
==>Java is set to allocate 1.2 gb of memory from the start. Don't try this on a machine with less than 1 gb of memory... you want at least 2.
==>OID Registration doesn't work as advertised. Fortunately I haven't needed my OID for internall connectivity but when I connect outside of the lab, I will need this.
==>c:\java is hard-coded as java location. Make sure you install to this location. If you don’t some of the NHIN services break with odd errors. The documentation reflects inaccurate pointers to the java locations. And with _some_ of the application hard-coded with this java location, better safe than sorry.
==>The NHIN documentation says in multiple places that port 9080 is the non-secure port and that 9081 is the secure port. DON'T BELIEVE IT! Port 8181 is the secure port.
Thus concludes volume 1 of the Chronicles of NHIN CONNECT. Stay tuned for updates as the PHGRID <--> NHIN CONNECT interoperability testing continues...
Friday, January 23, 2009
New Grid Node
I worked with Joe Terdiman on getting his DRN grid node installed. Globus has been installed and configured to the point where the container comes up with no errors. Monday at 12EST we will install and test the SST service.
Wednesday, July 16, 2008
Questions to GridFTP guys
After having a discussion today with Vaughn, these are some of the questions we agreed on to ask to GridFTP guys, pertaining to OCISO guidelines.
* What are they using for payload level encryption as in right now, we do double encryption i.e. we encrypt data first then place it in the PHINMS queue where there is transport level encryption. Taking the same scenario, we know that for transport level, encryption is achieved through SSL however what is used for encryption before that. Now that could be outside the scope of GriFTP however its worth talking to them as to whether they have been involved in similar initiative like that.
* At present, PHINMS works through Request / Acknowledgement. Now RFT helps us in achieve the similar funtionality in terms of the fact that we can track the status of transfer as in Active/Pending/retrying/completed. So in this case, if they can provide us with some transactional diagram, sequence diagram which can help us explain this in our documentation
* What version of SSL is being used.
* What metadata fields can be configured
* As far as tranmission integrity is concerned, GridFTP is built as an extension of FTP which uses TCP/IP protocol for data transmission. Since TCP uses a checksum computed over the whole packet to verify that the protocol header and the data in each received packet have not been corrupted, this ensures data integrity on the whole packet. We can reconfirm with them on that.
Next step would be to set up a call with GridFTP to discuss that.
* What are they using for payload level encryption as in right now, we do double encryption i.e. we encrypt data first then place it in the PHINMS queue where there is transport level encryption. Taking the same scenario, we know that for transport level, encryption is achieved through SSL however what is used for encryption before that. Now that could be outside the scope of GriFTP however its worth talking to them as to whether they have been involved in similar initiative like that.
* At present, PHINMS works through Request / Acknowledgement. Now RFT helps us in achieve the similar funtionality in terms of the fact that we can track the status of transfer as in Active/Pending/retrying/completed. So in this case, if they can provide us with some transactional diagram, sequence diagram which can help us explain this in our documentation
* What version of SSL is being used.
* What metadata fields can be configured
* As far as tranmission integrity is concerned, GridFTP is built as an extension of FTP which uses TCP/IP protocol for data transmission. Since TCP uses a checksum computed over the whole packet to verify that the protocol header and the data in each received packet have not been corrupted, this ensures data integrity on the whole packet. We can reconfirm with them on that.
Next step would be to set up a call with GridFTP to discuss that.
Tuesday, July 15, 2008
More on SRGM Test cases
After talking to Brian, we decided to define some kind of timeline we're looking at, in terms of the test cases defined in the test cases document for SRGM_PoC and also identifying tasks vs research items. So here it goes
* Evaluate the GridFTP functionality against OCISO guidelines and requirements (Req. No 01)
* Evaluate reliable GRIDftp and WS-RM against NIST guidelines and requirements (specifically, but not limited to, the FIPS 140-2 Cryptographic Module Validation Program; FIPS 200 Minimum Security Requirements for Federal Information and Information Systems) (Req No.02)
These research items need to be looked into by involving the GridFTP folks, maybe on a call but before that, I will get together with Vaughn and see how we can formulate specific questions which we would want to pose to them.
* Transfer file from partner node (node B) to CDC lab node (node A) using PHINMS issued digital certificates (Req. No. 04,05 & 06)
We will get hold of a PHINMS issued digital certificate and do a transfer and since the transfer is using SSL, we would not have to explicitly encrypt and decrypt the file.
* Test ability of Globus components to integrate into existing PHINMS infrastructure (transfer of program payloads through both PHINMS and Globus). (Req No.08)
To be further looked into
* Evaluate capabilities for end to end payload level encryption (Req. No 10)
Confirm if there is a switch to enable payload level encryption. This is different than TLS/SSL
* Evaluate capabilities for file payload encryption for strength and validity (Req. No 13)
If previous test is valid, then which algorithm is used to complete the encryption
* Evaluate capabilities for guaranteed once and only once delivery of file and the robustness of its duplicate file detection (Req No 11)
Confirm that a message Transport State is tracked by both Sender and Receiver (Request-Ack)
* Evaluate capabilities for reliable data exchange for once-and-only-once transport of payload data ( Req No 14)
Application Specific
* Evaluate reliable GridFTP for reliable messaging for reliability due to node availability ( Req No. 15)
Same as the test case where we brought down the node multiple no. of times to check automatic restart. Need to check whether we can control transfer of time sensitive material.
* Evaluate user interface projects for RFT management and operation (including, but not limited to, job starting/stopping/resuming, route management) ( Req. No 16)
To be further looked into.
I will also be updating the test case doc with these additions.
* Evaluate the GridFTP functionality against OCISO guidelines and requirements (Req. No 01)
* Evaluate reliable GRIDftp and WS-RM against NIST guidelines and requirements (specifically, but not limited to, the FIPS 140-2 Cryptographic Module Validation Program; FIPS 200 Minimum Security Requirements for Federal Information and Information Systems) (Req No.02)
These research items need to be looked into by involving the GridFTP folks, maybe on a call but before that, I will get together with Vaughn and see how we can formulate specific questions which we would want to pose to them.
* Transfer file from partner node (node B) to CDC lab node (node A) using PHINMS issued digital certificates (Req. No. 04,05 & 06)
We will get hold of a PHINMS issued digital certificate and do a transfer and since the transfer is using SSL, we would not have to explicitly encrypt and decrypt the file.
* Test ability of Globus components to integrate into existing PHINMS infrastructure (transfer of program payloads through both PHINMS and Globus). (Req No.08)
To be further looked into
* Evaluate capabilities for end to end payload level encryption (Req. No 10)
Confirm if there is a switch to enable payload level encryption. This is different than TLS/SSL
* Evaluate capabilities for file payload encryption for strength and validity (Req. No 13)
If previous test is valid, then which algorithm is used to complete the encryption
* Evaluate capabilities for guaranteed once and only once delivery of file and the robustness of its duplicate file detection (Req No 11)
Confirm that a message Transport State is tracked by both Sender and Receiver (Request-Ack)
* Evaluate capabilities for reliable data exchange for once-and-only-once transport of payload data ( Req No 14)
Application Specific
* Evaluate reliable GridFTP for reliable messaging for reliability due to node availability ( Req No. 15)
Same as the test case where we brought down the node multiple no. of times to check automatic restart. Need to check whether we can control transfer of time sensitive material.
* Evaluate user interface projects for RFT management and operation (including, but not limited to, job starting/stopping/resuming, route management) ( Req. No 16)
To be further looked into.
I will also be updating the test case doc with these additions.
Thursday, June 19, 2008
RODSAdai now working more like it is supposed to
So, today I moved a bunch of the argument building (so that you only need two arguments instead of 6) and query invocation logic into the rodsadai superstructure. Thus, things are working more as I envisioned them and as a whole, as opposed to being peices tested independantly of each other. And I did this despite a power outage in the lab that reset everything in the middle.
Tomorrow is moving the test into the automated structure, getting the secure peices working, and then I delve into translating a tuplerowset into a spatial or time series. I will also be meeting with Dr. Espino and it's nice to have some pretty good progress.
Tomorrow is moving the test into the automated structure, getting the secure peices working, and then I delve into translating a tuplerowset into a spatial or time series. I will also be meeting with Dr. Espino and it's nice to have some pretty good progress.
Wednesday, June 11, 2008
More code and a teensy unit test
After talking with Dr. Espino some more via IM, I showed him some of the code he entered and he updated the files for the time series and the spatial series.
I also created a factory for loading the test property updated a starting properties file and wrote a test-case that checks the properties file is loaded and that a simple test property shows up. Tomorrow I will load the property parsing and serverList building, and then it is onto the modifications of the GTSecureClient from Ogsadai.
I checked in all the new stuff, and for those of you looking for the rods-test directory... it is https://svn.rods.pitt.edu/repos/phgrid, but you will have to email Jeremy Espino for Access.
I also created a factory for loading the test property updated a starting properties file and wrote a test-case that checks the properties file is loaded and that a simple test property shows up. Tomorrow I will load the property parsing and serverList building, and then it is onto the modifications of the GTSecureClient from Ogsadai.
I checked in all the new stuff, and for those of you looking for the rods-test directory... it is https://svn.rods.pitt.edu/repos/phgrid, but you will have to email Jeremy Espino for Access.
Subscribe to:
Posts (Atom)
Posts
