OGSA-DAI 3.0 is alive, now to make it dance.

With the help of the wonderful people at U of Edinburgh, I got the SQLClient that had been written to successfully poke a database using the OGSA-DAI tools.

I have since spend the rest of the day getting another grid node set up locally so that I could start pursuing clients that access databases on other grid nodes. The "Hello World" of this app would be the ability to enter a query and have the results of two different grid nodes data resources concatenated (or processed in some other way that otherwise merges them) and returned.

I feel that the Killer App version that would result from such studies would be something similar to globus-url-copy: A command line tool (or web page, or java program) that would take a query, a series of nodes, and then run the queries on the data resources of those given nodes.

Right now I am in that phase where I have seen several possibilities but not gauged the limitations... and I am not quite sure what all the commands are or what they do or how far they can span. I am also not quite sure how GlobUS fits into the OGSA-DAI toolkit and what would be the best strategy for minimizing the amount of work that would need to be done by a grid-node installer to get a medical centers information as accessible, yet secure, as possible.

Added feedback and glossary to blog!

Ok... they may not be perfect questions... but any feedback is better than no feedback.  If you have ideas about changing/adding questions...just share them.  And yes, I was the one who already voted on the "Great Scott" option.... how could I not?

Dr. Tom

p.s. Could someone add a one-line post explaining what "SimpleCA" is?  Better yet, I added a glossary tab to the grid status spreadsheet...   let's see how that works.

tgs

Daily Lab / POC Activities

Daily Lab / POC Activities

Extramural:

• Configured a SimpleCA on lab 1001 to test OGSA-DAI installation. We are testing OGSA-DAI on an internal grid before we do a roll out on PHGRID.
• Analyzed hack attacks on lab servers and plugged security holes:
  

It is recommended that strong security measures be put in place to fend off hacker attacks like the examples listed below. The first two attacks were conducted before the server was protected by hardened security measures. The third attack was conducted after the server was hardened. I did a trace on the source of the hack and included the hacker information and method of attack below.

Hacker Info:
IP address: 203.144.221.26
Host server: 203-144-221-26.static.asianet.co.th
Network: TRUENET-TH
ISP/organization: True Internet Co., Ltd.
ISP/organization address: Internet Service Provider, Bangkok, Thailand.
Geographical location: Thailand
Email: abuse@trueinternet.co.th
Phone: +662 6411800
Fax: +662 6421557

Attack Method:
Attempted compromise the server using a dictionary hack on common system accounts and common user names. This attack was attempted hundreds of time by this hacker. The hacker was clearly using a to generate so many attacks in a short amount of time. Below is an excerpt of the attack.

Feb 28 15:53:29 gump sshd[5368]: Invalid user admin from 203.144.221.26
Feb 28 15:53:32 gump sshd[5370]: Invalid user guest from 203.144.221.26
Feb 28 15:53:35 gump sshd[5373]: Invalid user master from 203.144.221.26
Feb 28 15:53:56 gump sshd[5385]: Invalid user admin from 203.144.221.26
Feb 28 15:53:58 gump sshd[5387]: Invalid user admin from 203.144.221.26
Feb 28 15:54:01 gump sshd[5389]: Invalid user admin from 203.144.221.26
Feb 28 15:54:04 gump sshd[5392]: Invalid user admin from 203.144.221.26
Feb 28 15:54:19 gump sshd[5402]: Invalid user webmaster from 203.144.221.26
Feb 28 15:54:22 gump sshd[5404]: Invalid user username from 203.144.221.26
Feb 28 15:54:25 gump sshd[5406]: Invalid user user from 203.144.221.26
Feb 28 15:54:30 gump sshd[5410]: Invalid user admin from 203.144.221.26
Feb 28 15:54:44 gump sshd[5424]: Invalid user danny from 203.144.221.26
Feb 28 15:54:47 gump sshd[5426]: Invalid user alex from 203.144.221.26
Feb 28 15:54:50 gump sshd[5428]: Invalid user brett from 203.144.221.26

Hacker Info:
IP address: 202.63.185.230
Host server: 202-63-185-230.static.exatt.net
Network: EXATT
ISP/organization: Exatt Technologies Pvt. Ltd.
ISP/organization address: 510 Akruti Arcade,, Opp Wadia School,, J. P. Road., Andheri (W), Mumbai, Maharashtra, India., Internet Service Provider
Geographical location: India in
Name: IP-Admin NOC
Email: noc_mum@exatt.com
Phone: +91-022-5645-0200
Fax: +91-022-5691-9342

Attack Method:
Spoofing while attempting to compromise the server using a dictionary hack on common system accounts and common user names.

- POSSIBLE BREAKIN ATTEMPT!
Apr 2 11:02:43 gump sshd[4986]: Invalid user sara from 202.63.185.230
Apr 2 11:02:43 gump sshd[4986]: Address 202.63.185.230 maps to 202-63-185-230.static.exatt.net, but this does not map back to the address - POSSIBLE BREAKIN ATTEMPT!
Apr 2 11:02:53 gump sshd[4990]: Address 202.63.185.230 maps to 202-63-185-230.static.exatt.net, but this does not map back to the address - POSSIBLE BREAKIN ATTEMPT!
Apr 2 11:02:55 gump sshd[4992]: Address 202.63.185.230 maps to 202-63-185-230.static.exatt.net, but this does not map back to the address - POSSIBLE BREAKIN ATTEMPT!
Apr 2 11:02:57 gump sshd[4994]: Address 202.63.185.230 maps to 202-63-185-230.static.exatt.net, but this does not map back to the address - POSSIBLE BREAKIN ATTEMPT!
Apr 2 11:03:00 gump sshd[4996]: Invalid user ftpuser from 202.63.185.230
Apr 2 11:03:00 gump sshd[4996]: Address 202.63.185.230 maps to 202-63-185-230.static.exatt.net, but this does not map back to the address - POSSIBLE BREAKIN ATTEMPT!
Apr 2 11:03:02 gump sshd[4998]: Invalid user uid from 202.63.185.230
Apr 2 11:03:02 gump sshd[4998]: Address 202.63.185.230 maps to 202-63-185-230.static.exatt.net, but this does not map back to the address - POSSIBLE BREAKIN ATTEMPT!
Apr 2 11:03:04 gump sshd[5000]: Invalid user gid from 202.63.185.230
Apr 2 11:03:04 gump sshd[5000]: Address 202.63.185.230 maps to 202-63-185-230.static.exatt.net, but this does not map back to the address - POSSIBLE BREAKIN ATTEMPT!
Apr 2 11:03:06 gump sshd[5002]: Invalid user shell from 202.63.185.230
Apr 2 11:03:06 gump sshd[5002]: Address 202.63.185.230 maps to 202-63-185-230.static.exatt.net, but this does not map back to the address - POSSIBLE BREAKIN ATTEMPT!

Hacker Info:
IP address: 218.207.69.139
Host server: 218.207.69.139
Network: CMNET
ISP/organization: China Mobile Communications Corporation
ISP/organization address: Mobile Communications Network Operator in China, Internet Service Provider in China
Geographical location: China cn
Name: Jinxia Sun
Email: abuse@chinamobile.com
Phone: +86-10-66006688-1755
Fax: +86-10-66006012

Attack Method:
Attempted to breach the server via SSH, but the server has been modified to reject unauthorized users. The hacker tried to breach the server twice and moved on.

Apr 4 06:26:41 gump sshd[12345]: refused connect from ::ffff:218.207.69.139 (::ffff:218.207.69.139)
Apr 4 06:32:33 gump sshd[12390]: refused connect from ::ffff:218.207.69.139 (::ffff:218.207.69.139)

OGSA-DAI 3.0 is fibrillating

Today I spent the better part reading through the documentation configuration guide, and then reading through the client configuration guide. The end result is I had a small client java program written mostly for me, with a few modifications by me to adjust hostnames and the like.

The program is designed to send a small query out to the OGSA-DAI attached database, and then transform the results and perform a task that would enhance efficiency. The program as written in the documentation assumes a MySQL-based data resourse, so I adjusted the program to use the PostgreSQL-based data resource I had deployed earlier that morning.

When I run it, I get an error. I don't know what is causing the error since it seems to happen in another thread and the cause is cited as "unknown source". I am guessing I either borked the deploy to tomcat/globus, or I am heading to the wrong URL in the program. It's strange because the dai-manager.jsp is telling me the services are there... but when I try to access their WSDL's I get AXIS errors.

At this point I am at a loss for ideas, so I will reach out to some of the OGSA-DAI contacts about some help checking my configuration and start to request more information about how they set up the distributed query application seen running through the link Ken posted yesterday.

Cheers, Peter

Grid Node Status Document Moved...

Moved to the the upper left corner of the page - for fast and visible access.  Ok, that was a short and sweet update... go team!

OGSA-DAI 3.0 has a heartbeat...

Today I spent most of the morning and a bit of the afternoon finishing the installation (lots of builds) for OGSA-DAI on my main environment. I was hampered a bit by silly things like setting the classpath due to my occasional memory lapses with linux conventions (remember, it's 'source ./setenv.sh'... not just './setenv.sh') I then spent the second part of the day installing OGSA-DAI on another lab node. But they seem to be running well enough between the two nodes.

Tomorrow I expect to read through the documentation for creating clients that use OGSA-DAI, and then I will hopefully run a few across the two machines. I also think there will be some deployment of the OGSA-DAI code to other globus nodes, perhaps some of them on the external phgrid networks.

After that, accessing the client via a JSP and then that goal will have been complete. I imagine the next steps will involve using different databases at different globus nodes and seeing how well the data integrates and showing that we can do it.

The site IT team still hasn't installed ubuntu on my dev box, so that is on hold.

Cheers,
Peter

Daily Lab / POC Activities

Extramural:

• Updated Spreadsheet
• Installed and configured prerequisite OGSA-DAI software on lab 1003 and 1001 (I.E. Postgresql, Tomcat, Ant, Java)
• Installed Globus Web Services on lab 1001
• Contacted grid sites for node installation. See spreadsheet for details.

OGSA-DAI Demo

This is to inform the list that a demo has been put up which implements a basic and optimised version of the example query:

Select postal_code, count(patients) from global_db where diagnosis = 'X' order by count
where global_db

is a federation of two databases. In this case, the databases are the split version of the OpenMRS demo data with dummy post codes.

The demo is accessed through JSP pages which will return the results as an HTML page. The JSP page is based around an OGSA-DAI client program. This is to allow users with OGSA-DAI client libraries to access the demo at this stage.

The basic version of the example query operates without any optimisation to manage the amount of data transfered by the database resources and OGSA-DAI. The optimised version uses optimisations on the OGSA-DAI query plan to push operations to the databases to reduce the amount of data transfer needed but still retain the ability to do federated queries on multiple resources using OGSA-DAI and will allow different database schemas to be used. The demo is setup as JSP pages,

Basic:
http://test.ogsadai.org.uk:8080/dai/cdc-query.jsp
Optimised:
http://test.ogsadai.org.uk:8080/dai/cdc-s-query.jsp

This are available for use now. If the URLs do not work, it is possible the OGSA-DAI instance is down for updating or maintenance. An email will be sent when the URLs are removed from use permenantly.

If possible, could a more realistic data set be made available which can be integrated into the demo and used for further development.

Greetings from the new guy!

Greetings everyone, I am the new developer to the project, Peter M White.

I have been around for about a week and a half now and have spent my time familiarizing myself with all the different facets of

• The GlobUS grid toolkit (which contains all the bits and peices to communicate between grid nodes)
• The MonALISA distributed monitoring toolkit, which basically gives you many interfaces and ways to monitor the health/capabilities of various nodes on a grid.
• The Java GlobUS CoG toolkit (which is a series of Java GUI front ends and tools for the GlobUS library of commands... there is also a CoG for python.)
• The OGSA-DAI toolkit... which is described on the website as "[a] project [...] to develop middleware to assist with access and integration of data from separate sources via the grid." I will now characterize it as "distributed database processing using webservices" and proceed to make many, many apologies for the oversimplification.

When I first came to the this project I threw myself into building an environment and reading all the documentation of the above projects. Quickly finding that they were all open source and had carefully constructed areas for plugging in modules to extend functionality. They also sprouted off millions of ideas in my head for things to try, so I was at a complete loss as to where to start for a little bit.

Luckily, after some conversations with Ken and Dan, I have a few goals. The main goal right now is to get OGSA-DAI up and running to a point where I can pull up a jsp-page (or some other gui) with a google-esque text box and a run-button. When one enters a query and hits "run" it should run the query and report back the results. More specifically, OGSA-DAI should query its test database and go out to other OGSA-DAI services which will query their respective test databases and then all the results will get compiled into one big set and then that set should be displayed.

Here is a list of steps towards our main goal.

• Finish setting up the OGSA-DAI code on the node I am working on (which includes finishing a test database and running some test and monitoring commands)
• Install OGSA-DAI on other nodes in the grid, and set up similar test databases there.
• Run some distributed query commands, and debug them.
• Evaluate the currently coded JSP pages that come with OGSA-DAI for expandability and make a JSP front end for running a distributed query.

The other, more minor goal, is to see if it is possible to set up a GlobUS node (and eventually the CoG tools and the OGSA-DAI services) on an Ubuntu box. Right now, all this code has been set up on SuSE boxes. While SuSE is powerful and wonderful and Yast is really cool, it is my experience that Ubuntu is still more user-friendly. Thus, going "you can run [grid] on Ubuntu too" (or even "we tried to set [grid] up on Ubuntu and it spit fire and tacks before shutting down the system, so you should stick with SuSE") would be beneficial.

Right now I am waiting on an Ubuntu install from our hosts and when that is set up I shall start trying to set up a node and asking Dan (who has already set up so much and I am grateful for it) for help when I get stuck.

Otherwise, I am incredibly excited to be working on this project. I think there is a lot of cool stuff that can be done with grid technology, and we are also in a position to make a lot of contributions to the technologies involved.

Cheers,

Peter

Daily Lab / POC Activities

Extramural:

• Installed dependencies for Subversion 1.4.6 on lab 1003
• Worked on RLS configuration of Dallas and Tarrant
• Peter currently researching OGSA-DAI