Automated deployment of a virtual machine

At Utah...

Dr. Julio Facelli and Ron Price chatted about the need to bring the grid service to the data.
So, they pinged Argonne about an automated deployment of a virtual machine that when stood up is the grid service that the data needs.

Ravi Madduri <madduri@mcs.anl.gov> responded:

Ron, We have been playing around with similar idea in gRAVI development where we create a grid service wrapper around an executable and also contextualize a VM that can host the application and the service on demand. There is a set of ant tasks that can be used to deploy gRAVI services ( or any gar ) to the nimbus cloud. Here is the README for this tool. It is a functional prototype, but you need an account on the nimbus cloud.

See previous post (<-click here) for further details about the nimbus cloud.

Ron will update as he progresses along this line.

Nimbus - Interesting model for deploying in the cloud

Nimbus provides a free, open source infrastructure for remote deployment and management of virtual machines, allowing you to:

· Create compute clouds (make your own EC2 style service). For examples, see the science clouds page.
· Deploy "one-click" auto-configuring virtual clusters (see the cloud clusters page). They adapt on the fly into new network and security contexts so you can set them up once and run them over and over again, even across different clouds.
· Serve clients that are compatible with the Amazon EC2 service, see What is the EC2 frontend?
· Integrate VMs on a set of resources already configured to manage jobs (i.e., already using a batch scheduler like PBS). See What is the Workspace Pilot?
· Interface to Amazon EC2 resources, see What is the EC2 backend?
· Easily experiment with new remote protocols and backends, see What is the RM API?

Thanks, Ron

Setting up RODS-GDBC

I took a hint from RODSAdai and decided to spec out the application I am planning to build for all to view before I get too deep into building the thing. The idea is that if I have questions, I can point them to this "what I am trying to do" document with a breif skeleton of the application.

This way, when people ask "what are you trying to do" I can get them to look here.

I have also installed introduce... and I am now going through the tutorial project, tomorrow I am planning on getting a simple skeleton drafted and contacting Dr. Jeremy Espino about interface changes and the link. I am also not sure how easily introduce projects can be incorporated into Maven and I'm pretty sure the same nasty axis problem will arise since the globus axis will probably need to be used with the client.

World's largest computing grid lives to go live

Contrary to popular belief, the world as we know it didn't implode after the Large Hadron Collider was flipped on. Sure -- someone, somewhere is growing a ninth arm and trying desperately to land a cameo on Fringe, but the planet at large is still humming along just fine. Now, the world's most ginormous computing grid (the Worldwide LHC Computing Grid, or WLCG) has gone live, and the gurus behind it are celebrating the beginning of its momentous data challenge: to analyze and manage over 15 million gigabytes of data each year. The Grid combines the IT power of over 140 computer centers, 100,000 processors and the collaborative efforts of 33 countries. Unfortunately, there's no word on when the official WLCG-based Call of Duty 4 server will be green-lit for action, but we hear it's pretty high on the priorities list.

Read all about it here

Planning and such.

So, RODSAdai and Poicondai are documented, fleshed out, and there ready for people to consume and ask-questions about if people want to recreate them or install them in new places.

The next step is to essentially re-create the OGSA-DAI end of RODSAdai using CaBIG's Introduce framework. Introduce, for all intents and purposes, seems to be really really good at generating a full-out service skeleton and having the tools to package up everything into a .gar file (which is the deployable for a globus toolkit server).

Thus, instead of OGSA-DAI at a given location accepting a query, this will be parameters passed into a webservice. Whether those parameters will be a query or just some filter values will be a discussion for Jeremy (because if they are just filter values, it becomes that much easier to prevent against SQL Injection attacks by using a prepared statement). Also, it will be interesting to see how well parameterized the connection logic could be and whether it would be as universally installable as ogsadai (it probably would not be as versatile).

Otherwise, we haven't really investigated the Dallas slowness at this point, I will also have to ask Jeremy if that Pitt node has been set up or not. I have a few ideas for what to try, but we will see.

3rd Party Certificates within Globus

Verisign Certificates within Globus

grid-proxy-init ERROR: Couldn't verify the authenticity of the user's credential to generate a proxy from.

grid_proxy_init.c:1070:globus_gsi_cred_handle.c:globus_gsi_cred_verify_cert_chain:1733:
Error verifying credential: Failed to verify credential
globus_gsi_callback.c:globus_i_gsi_callback_create_proxy_callback:442:
Could not verify credential
globus_gsi_callback.c:globus_i_gsi_callback_cred_verify:681:
Can't get the local trusted CA certificate: Cannot find issuer certificate for local credential with subject: DATA-REMOVED-BY-DAN

GridFTP Error:

error: globus_ftp_client_state.c:globus_l_ftp_client_connection_error:4217:
the server responded with an error
530 530-globus_xio: Authentication Error
530-globus_gsi_callback_module: Could not verify credential
530-globus_gsi_callback_module: Error with signing policy
530-globus_gsi_callback_module: Error in OLD GAA code: CA policy violation:

Solution Summary:

The certificates were extracted using the Portecle application in PEM format. The entire certificate chain should be used in each hash file. Remove the private key data from the file.

The commands:

openssl x509 -issuer_hash -in [file_name].pem was used to determine the hash name the file should be named after.

openssl x509 -issuer -in [file_name].pem was used to determine the access_id_CA that should be used in the signing policy.

openssl x509 -subject -in [file_name].pem was used to determine the cond_subjects that should be used in the signing policy.

The step by step solution:

1. Export your certificate with Internet Explorer using the Personal Information Exchange PKCS12 option.

2. Check the, “Include all certificates and certificate paths” box. NOTE: This should be the only option checked.

3. Upload the exported certificates to the Globus node. (Root, Intermediate, and Private)

4. Use Portecle to view the exported certificates. Portcle is started using the following command: java -jar portecle.jar

5. Right click on the certificate, then use the PEM Encoded option to export private key and public key certificate within Portecle.

6. Remove the private key data from the PEM file that was created.

7. Create a hash name for the PEM file that was created using the following command:

openssl x509 -issuer_hash -in [file_name].pem was used to determine the hash name the file should be named after.
openssl x509 -issuer -in [file_name].pem was used to determine the access_id_CA that should be used in the signing policy.
openssl x509 -subject -in [file_name].pem was used to determine the cond_subjects that should be used in the signing policy.

openssl x509 -in yourfile.pem -noout -hash

8. Rename the file to the hash number displayed in the following format: hash.0

9. .Manually create a signing policy named (hash.signing_policy)

10. Copy the new files to /etc/grid-security/certificates

11. Create a duplicate copy of the hash.0 file for the Issuer_Hash and the hash. Example: awd2dq.0 7847a3s.0 ( There should be two hash files that contain the same certificate chain. Only the names are different)

12. Create signing policy files for each hash file based on Intermediate and Root certificates.

PHGrid Service Registry 0.1 (beta)

When the PHGrid collaborators met after the PHIN Conference, one of the ideas was to set up a place where all the PHGrid collaborators can share descriptions of services they developed, services they are planning or even services they want to develop.

I just created a very basic service registry on the wiki available at http://sites.google.com/site/phgrid/Home/service-registry that contains the services that have been developed or that I know of as of now.

This registry initially serves as a human readable repository of service information, but eventually we will extend this with a UDDI registry and Index services that provides a standards based search for services.

This is of course a tiny step, but it is a start. Now we just have to get each service provider add in details and links for their services. Peter took the first step and went ahead and created a page for the RODSA-DAI services and client. This can be used as an example for how other services can be documented.

DEMODEMODEMO(again)

So there is a big demo tomorrow, and I have been preparing for that by making sure that the RODSA-DAI and Poicondai demos have been polished and a few of their quirks have been ironed out. I also was working on making all the items a lot more portable.

Furthermore, I have been ironing out some issues with a prototype public demo, and doing a lot of documentation. And that documentation is linked-to from this shiny new wiki page that Brian set up and I have filled out: here.

Meanwhile, I have started to look into the caGrid CQL and DCQL, and that stuff looks interesting.

Globus grid services on Windows nodes

I started the installation of the COG 4.1.5 on Windows Vista. According to documentation located in Globus.org's FAQ section, Windows nodes should be able to access grid services using the COG Kit from www.cogkit.org.

The installation has been stalled due to insufficient access rights of my user account on the local Windows machine. The Windows Admin password need to be entered in order to complete the install. I will talk to the admin in the morning so I can continue the install.

News: cancer Biomedical Informatics Grid® (caBIG®)

The cancer Biomedical Informatics Grid® (caBIG®) initiative announces a milestone in its expansion of support alternatives for end-users, IT staff and senior decision makers implementing caBIG® tools and infrastructure. Three companies, 5AM Solutions, Inc., Ekagra Software Technologies and SRA Corporation, are the first to be licensed by the National Cancer Institute (NCI) to market their services as part of the caBIG® Support Service Providers Program. Other companies in addition to 5AM, Ekagra and SRA are currently engaged in the licensing process, and will be announced in the coming weeks.

caBIG® Support Service Providers are independent entities that are approved by NCI as meeting specific criteria for performance of support services related to caBIG® needs. There are four categories of support: Help Desk Support; Adaptation and Enhancement of caBIG® -Compatible Software Applications; Deployment Support for caBIG® Software Applications Deployment; and Documentation and Training Materials and Services. Services rendered by caBIG® Support Service Providers to their clients are established under separate business arrangements organized by and between the service provider and its clients.

Visit https://cabig.nci.nih.gov/esn/service_providers to learn more about caBIG® Support Service Providers or access those Support Service Providers that are currently licensed. If you are interested in applying to become a Service Provider, you can access past and future announcements for caBIG® Support Service Provider at this site as well.