"Seegrid will be due for a migration to confluence on the 1st of August. Any update on or after the 1st of August will NOT be migrated"

APAC Grid Geosciences Project

Project: Seismic Simulator Portal


Related pages (edit)
Home
Overview
Project Scenarios
Project Outputs
System Specification
Issue Tracking
Project Journal
APAC Technical Discussion
Team Meetings
Development Toolkits
External Links
APAC computational infrastructure project
Gridsphere Portal toolkit
Globus toolkit

STATUS: View the current status of this project at: SeismicSimulatorStatus
How to get Access to the Portal

Introduction

Seismic Simulator is a Fortran-based program that numerically simulates the sound propagation of different frequencies through rock and fluid structures of varying densities.

Currently Seismic Simulator must be run directly from the Unix shell command line and it requires modification of the Fortran source code to configure simulation parameters. This arrangement is workable for internal research and development but is impractical for commercial use.

The Seismic Simulator Portal project is an attempt to provide an easy-to-use web-based interface to the Seismic Simulator. The ultimate goal is to provide access to the simulator’s functionality by remote clients on a fee-chargeable basis.

Planning

This project had a timeline of approximately 3 months and had approximately 1 1/3 EFTs (4 months equiv.) involved (excluding time required to alter Seismic Simulator code by Scientist).

The project involved:

  • Requirements analysis – 1 week
  • Development of a prototype portal in php – 2 weeks
  • Learning Gridsphere and its architecture – 3 weeks
  • Detail the correct architecture for gridsphere portals
  • Implementing UI components in Gridsphere 1 month
  • Linking components with Gridportlets and QUT portal tools – 2 weeks
  • Linking components with SRB (Jargon API) – 1 week
  • Establishing test infrastructure – 2 weeks
  • Testing and deployment -1 weeks

Requirements

Design

Architecture

From hereon in the Seismic Simulator Portal will be referred to as the Portal.

The Portal was built in accordance to the JSR-168 specification in the Gridsphere portal container framework. The Portal used the Gridportlets to connect to various Grid-based technologies such as Globus and Myproxy server. The “portlet” based classes in the source directories contain user interface based code whereas the “services” based classes contain only code that provides the functionality and a service to the portlet.

Class Diagram for packages in the Seismic Simulator portal (user classes)

* sessionpackage.GIF:
sessionpackage.GIF

Job Submission

Uses the Gridportlets framework and packages, simply using the SubmissionService, CredentialManagerService and UserManagerService services, we can submit jobs to the grid. here is the code fragment that submits a job: Job Submission code fragments

Data and Result Service

Since the data for the Portal is to be staged and stored in SRB the SRB Jargon API compiled for GSI was used to communicate from the Portal to a SRB server. The data and results services uses the API to connect, transfer and query files. The portlet “Setup Data”, uses the API to upload data from a user to SRB. The “Results” portlet uses the API to query SRB on the location of result files and query them. No result files are “downloaded” via this API since the result files can be quite large and thus we do not want to download them through the web server, which is undesirable. A SRB client is used to download result datasets.

Discussion for further investigation (Note as a point to be followed up by APAC):

Since these datasets are quite large (>100MB), it is not advisable to download them through the webserver, which would require:

User request (http) - webserver request to SRB (SRB) - SRB response (SRB) - webserver response to user (http)

This requires the file to be saved temporarily on the webserver when received from the srb server and then sent via http to the user. Which isn’t desirable since the file could possibly be very large (>10Gb) and people hosting these portals wouldn't like that!

The way we're doing it allows the user to obtain the address of files and then use a separate client which is a work around for the time being. Perhaps a java applet would be best, served by the web server. This way a srb session could be established between the user and the srb server directly and then files could be transferred directly to the user, bypassing the web server. The applet could be embedded in a gridsphere portlet!

Implementation (Walkthrough)

Else otherwise stated, user interface components are created via the Gridsphere Beans which are essentially wrappers over generic java swing/awt components.

LOGIN

Essentially the Default login portlet that is provided by the Gridsphere framework. User’s are registered for the Portal by the root gridsphere user and each user is given the username and passwords that match their APAC username and credential passphrase – which secondly through Gridportlets, maps the user to any credentials that may exist for the user on the machine that the user is to authenticate to. If credentials exist and a proxy has already been obtained, the user can do all things with Portal; else the user may need to upload a certificate or a proxy obtained.

SESSION

This page consists of several portlets that manage user sessions. The hierarchy of sessions are as follows: - User can have multiple sessions - Sessions can have multiple jobs

The gridsphere framework is used to “register” user sessions via “Hibernate” and save their corresponding data into the database. Later we intend to “register” sessions via ebXML so that the session data will be available to other implementation technologies not just gridsphere.

The Session Service (SessionService.java) “saveSession” function creates a new database record for the session, “deleteSession” deletes that record and all data that corresponds to it and “saveSession” renames a session and commits the change to the database.

Edit PDC

This page consists of several portlets that allow the user to edit parameters that will either serve ass input parameters to the executing Code or to modify the codes functionality and execution path.

When the user selects “Apply Changes” the parameters are committed to the Gridsphere database record that was created for the Session.

The parameters entered are later at job submission time, extracted from the database and outputted to a parameter file that is uploaded to the executing Host. The Host will use these parameters as input values to the code and to vary the codes execution.

SETUP DATA

This page allows the user to upload datasets, delete and rename datasets on the server and set the files which are the inputted to the Code.

The “Upload data” portlet is used to upload datasets to the users SRB account. All files are uploaded to a default location known by the Portal. The “preview” portlet lists what files have been uploaded and by clicking the “Preview” button, the user can view the location of the file. The “Rename” and “Delete” portlets are used to rename or delete a dataset in SRB.

The final portlet on this page is the most important, it is where the user selects which files are to act as the various required input files for the execution code. This must be set else an error will be generated and the Code can not be executed.

JOB SUBMISSION

The Job Submission Portlet uses the Grid Portlets job submission framework which essentially is a wrapper over the Globus API (COG). Firstly the Portal checks whether the user has a current Grid proxy, if not the user is requested to get one, else the user can submit their job. Currently, the Portal is configured to use a single executing host system, however if more than 1 is available the user will be given a choice of available hosts to execute the job on. If the user wants to submit the job they simply need to press “SUBMIT” and the job will be submitted to globus using the Grid Portlets commands.

RESULTS

This page allows the user to query their available results. Results sets will only be available upon job completion. The user selects the job they would like to retrieve the results for and the portlet queries SRB (where the Code places the results after completion,) which returns a list of the results and prints out the standard output and error for the job in their allocated text areas. The user can obtain the location of a results set by selecting the dataset and clicking "Location". A preview of the results set can be obtained by once again selecting the file and clicking “Preview”, the portlet will then query the SRB (using random file access) and return a slice of the results set as a 2D array (image).

To download the full datasets a SRB client must be used, the user will use the location of the results set given by the portlet to obtain the results sets.

UPLOAD CERTIFICATE

This is a supporting page required so that the user can upload their certificate to a Myproxy Server (server detailed in the GridPortlets framework – Resources.xml file). The page displays a applet that uses the Myproxy api to generate a proxy certificate from the user’s certificates on their machine and sends the proxy to the Myproxy server. Once the proxy is received, the user can then authenticate to the Grid by obtaining a Grid Proxy.

GET GRID PROXY

User must obtain a Grid Proxy to use the grid. This is done by either having a certificate on the machine that the user is autheniticating to (which is highly unlikely) or by submitted a proxy certificate to a Myproxy server (see previous Portlet description.) The Portlet is used to query Globus for Grid access, the user enters their username and passphrase for their certificate and requests access. Globus will either grant access to the Grid for a pre-allocated amount of time or disallow access (error message generated). Once a proxy is granted the user can submit a job, job submission will not be allowed unless this step is not taken prior to job submission.

Portal Issues:

- All data is uploaded to SRB prior to job execution using the portal IF dataset size is <10Mb else a SRB client must be used – BUG: Stage in - All data is downloaded from SRB after job execution using a SRB client – BUG: Stage out - No standard output/error is accessible during Code execution, only available at code completion (BUG: how do we monitor these during execution?) - Code has a script wrapper that is used to fetch files from execution and transfer result files to SRB after execution. (BUG: how do we stage data properly from SRB?). Script is executed by Globus. The script uses Scommands and assumes the executing host will properly authenticate the user in order for them to use SRB and the Code. (BUG: Thus does user need accessible Mdas files?) - Integrate supporting portlets into Seismic Simulator workflow or customize the Grid Portal to only include portlets of relevance (ie: Get Grid proxy and Upload certificate)

System Requirements

Seismic Simulator Portal
  • Java 1.4.2, ANT
  • Gridsphere 2.0.3
  • Gridportlets 1.0.3
  • Tomcat 5.5.x
  • VDT (GT2.4)
  • Myproxy-server
  • SRB Jargon API compiled with GSI
  • Access to APAC-Trial Grid config directories ($TMP, $DATA, etc)

Seismic Simulator Code
  • Seismic Simulator code compiled for host system
  • Access to $TMP, $DATA
  • SRB SCommand Client
  • Host to be able to map user to user’s real name (SRB name)

NOTE: for the visualisation to run correctly, the following line needs to be added to the tomcat startup script - CATALINA_OPTS="-Djava.awt.headless=true" The linux distribution also requires libxp.so.6 part of the X11 depreciated libs (sorry), its widely available though and you can install it via yum:

yum install xorg-x11-deprecated-libs-devel.i386

-- RyanFraser - 11 Jul 2005

Project Notes

  • Installing Gridsphere
    • now that version 2.0.3 has been released must of bugs in the install process have been removed (I haven’t experienced any probs with the latest release, though I had many probs with various cvs releases, but the Gridsphere team responded quickly and fixed the bugs!)
    • this release is stable and is going to be used as a “benchmark” for all APAC grid developments in 2005

  • Installing Gridportlets
    • Follow the instructions provided
    • Be sure to copy ant.jar and ant-launcher.jar into your $CATALINA_HOME/lib/shared dir, else installation will not work!

Topic attachments
I Attachment Action Size Date Who Comment
project_specification.docdoc project_specification.doc manage 76.0 K 11 Jul 2005 - 16:22 RyanFraser  
sessionpackage.GIFGIF sessionpackage.GIF manage 22.0 K 15 Jul 2005 - 14:31 RyanFraser  
Topic revision: r7 - 15 Oct 2010, UnknownUser
 

Current license: All material on this collaboration platform is licensed under a Creative Commons Attribution 3.0 Australia Licence (CC BY 3.0).