"Seegrid will be due for a migration to confluence on the 1st of August. Any update on or after the 1st of August will NOT be migrated"

Tutorial on using Underworld with GOCAD model in VGL

Overview

Before we proceed with this hands-on tutorial, let us briefly look at what Underworld and GOCAD are and how they are used in VGL.

Underworld is a 3D-parallel geodynamic modelling framework capable of deriving viscous or viscoplastic thermal, chemical and thermo chemical models consistent with tectonic processes, such as mantle convection and lithospheric deformation over long time scales (For further information, please visit Underworld website i.e. https://underworldproject.org).

GOCAD (Geological Object Computer Aided Design) is an integrated and geologically oriented CAD software that gives you everything to construct a wide range of earth-model, for application in geology, geophysics and reservoir engineering (visit GOCAD website i.e. http://www.gocad.org for further details).

In VGL, you provide Underworld with GOCAD model’s data files, a processing script and submit them to the cloud for processing where thermal simulations are performed. Once the processing is done, you will get numerous light text and HDFS output files storing global simulation metrics. These output files can then be visualised in visualisation software such as Paraview, Visit or gLucifer. Alternatively, those output files can also be used directly for analysis in Matlab or in Python via PyTables.

This tutorial will provide a step-by-step guidance on how to perform Underworld’s thermal simulation in VGL with a GOCAD model and an example processing script. The objectives are to get you familiar with VGL and eventually perform your own Underworld’s thermal simulations in VGL.

Step 0 – Preparation

To run through this tutorial, you need an internet connection and a web browser installed on your computer.

This tutorial was successfully tested with the following web browser: Internet Explorer 9, Firefox 16.x and Google Chrome 23.0.1271.95.

This tutorial has been prepared for VGL 1.1 Release Candidate 2.

The following shows a list of VGL deployment environments:

Step 1 – Dataset Selection

The first step in using Underworld in VGL is to discover what models supported by Underworld are available. An Underworld’s model usually comes with a number of input data files. VGL enables those input data files to be captured and made available to a job for processing.

This tutorial will demonstrate how to run Underworld’s thermal simulation with GOCAD model.

  • Click the DISK Icon on the popup to have the file available for the simulation.
In this tutorial, we will capture all data files in GOCAD model regardless of whether we will be using them later in our Underworld example script. To capture a model’s data file, click on the floppy disc icon (see VGL User Guide for further details). As of VGL v1.1, only one file can be captured at a time and there isn't an option to download all files. This will be addressed in the next version of VGL.

Step 2 – Job Construction

Once the data files are captured, the next step is to build a job to process the captured data files by using the “Job Wizard” which can be accessed by using the ‘Submit Jobs’ link.

To construct and submit a job in VGL, you need to sign in with your OpenID credentials (see this link if you haven't got an OpenID account set up yet).

Select Job Series

The first step in creating a job is to assign it to a series. Let us proceed to create a new series and name it “UnderworldTutorial”. You can use the description provided below or provide your own series description.

Enter Job Details

Once the name and description of a series are provided, click “Next” to proceed and enter the job details. To perform Underworld’s thermal simulation, the user must select “Underworld” software from the Toolbox dropdown list.

This tutorial will use the Compute and Storage Providers from National Computing Infrastructure (NCI) in Canberra to run the job and to store its input data files and also execution results (see Compute and Storage Providers in VGL Guide for further details). Alternatively, you could also choose to use Compute and Storage resources provided by NeCTAR to perform previously mentioned tasks. Regardless of which Compute and Storage Providers you choose, the end results of your job execution will always be the same.

Note: From observations, it seems like the Cloud computing infrastructure provided by NeCTAR? performed better than the one provided by NCI at the time of preparing this tutorial. You're recommended to use NeCTAR? as your Compute and Storage providers in this tutorial.

Manage Job Input Files

VGL will show your previous captured data files after you entered job details. You should see the GOCAD model data files displayed on the “Input files” panel.

At this step, you can add more inputs to the job (see Part I of VGL User Guide for further details).

This tutorial does not require you to provide any additional data file.

Define Your Job Script

In VGL, the user can define an Underworld’s processing script from scratch, copy and paste it from somewhere or import the script from existing script template.

If the user chooses to define an Underworld’s processing script from scratch or copy and paste the script from somewhere to work on previously captured or added data files, he or she must remember the path to those data files.

For the purpose of this tutorial, we choose to work from existing script template. On “Define your job script” page, expand the “Underworld Examples” tree and double click on “Gocad Simulation” node to import the simulation example script into the script builder.

When prompted to provide additional parameters (see screenshot below), select the paths to the voxel set and CSV input files from respective dropdown lists plus optionally choosing and specifying lower boundary conditions. These values will be populated in the Underworld example script.

Once the dataset’s paths and the optional lower boundary conditions are provided, the following Underworld example script will be inserted into the script builder (the voxetFilename and keyFilename parameter values show paths to the data files and the lowerBoundaryFlux argument variable shows the value you previously provided):

import subprocess, csv, math, os, sys, urllib, glob;
import logging

def cloudUpload(inFilePath, cloudKey):
    cloudBucket = os.environ["STORAGE_BUCKET"]
    cloudDir = os.environ["STORAGE_BASE_KEY_PATH"]
    queryPath = (cloudBucket + "/" + cloudDir + "/" + cloudKey).replace("//", "/")
    retcode = subprocess.call(["cloud", "upload", cloudKey, inFilePath, "--set-acl=public-read"])
    print "cloudUpload: " + inFilePath + " to " + queryPath + " returned " + str(retcode)

#Build the default list of arguments
argList = ["uwGocadGAMaterialsModelGenerator.sh", "--voxetFilename=Cooper_Basin_3D_Map_geology.vo", "--materialsPropName=Geology", "--keyFilename=KeyToVoxSet.csv", "--run"]

#Add optional arguments
lowerBoundaryFlux = "7"
lowerBoundaryTemp = ""
if (len(lowerBoundaryFlux) > 0):
    argList.append("--useLowerFluxBC")
    argList.append("--lowerBoundaryFlux=" + lowerBoundaryFlux)

if (len(lowerBoundaryTemp) > 0):
    argList.append("--lowerBoundaryTemp=" + lowerBoundaryTemp)

#Begin processing
print argList
retcode = subprocess.call(argList)
print "result: " + str(retcode)

#Upload results directory
dirName = "output_GAGocadGeothermal/"
print "Fetching output files"
outputFiles =  glob.glob(dirName + "*")
print outputFiles
for invFile in outputFiles:
    cloudUpload(invFile, invFile.replace(dirName, ""))
print "Done"

Once the above script template is loaded into the builder, you can then proceed to modify the code and submit it to the cloud for execution. For the purpose of this tutorial, we will leave the above script as it is and move on to the next section.

Review Job before Submission

The following review page will be shown before your job submission. It gives you opportunity to add additional input files to your job and examine input files to be submitted for processing.

As this tutorial does not require any additional input file, click “Submit Job” to proceed.

If the job submission is successful, you will be redirected to “Monitor Jobs” page where you can then monitor the status of the submitted job and view/download the job’s input and output files. VGL will display an error message if it fails to submit the job to the cloud for execution.

Step 3 – Job Monitoring

At this point in time, you should have submitted a job named “UnderworldJob1” for processing in the cloud.

A job belongs to a job series. To monitor the status of your submitted job, you must first select a series from the “Series List” pane. In our case, select a series named “UnderworldTutorial”. If you have a large number of series records created, use the “Query” button to search for that series.

Once the “UnderworldTutorial” series is selected, all jobs belonging to that series will be listed on the “Jobs of selected series” pane. If this is the first time you run through this tutorial, you should be seeing only one job.

At any point in time, a job can be in one of the following four states:
No. Status Description
1. Saved A job will be in this state if it hasn’t been submitted for processing yet or it was cancelled by the user shortly after its submission. You can edit, submit or delete a “Saved” job but not cancel or duplicate it.
2. Pending A job will be in this state if it has already been successfully submitted to the cloud for processing and is waiting for a compute resource to process it. You can only cancel and duplicate a “Pending” job.
3. Active A job will be in this state if it is being processed by the compute resource. Like a “Pending” job, you can only cancel and duplicate an “Active” job.
4. Done A job will be in this state if it has completed its execution. A completed job does not guarantee the job is successfully executed. VGL v1.1 doesn’t provide a straight forward way to indicate a job execution success or failure. The only way to figure out this is to look at the files it generated.
The number of files generated by an active or completed job is different depending on which toolbox you select to process your job in. Every successfully executed job will have a file called “vegl.sh.log” generated. This file keeps track of the job execution log and it can be used to troubleshoot why a job failed in its execution. You can only delete or duplicate a “Done” job.
In this tutorial, we are only interested in our previously submitted job named “UnderworldJob1”. To update its job status, use the “Refresh” button. A job normally (provided you don’t cancel it during its execution) goes through the following lifecycle: Saved -> Pending/Active -> Done.

The following screenshot shows our “UnderworldJob1” job is in “Active” state:

Depending on the size of your input dataset, the computational logic and processing load at NCI or NeCTAR, the above job will take approximately 3-5 minutes to finish on NeCTAR.

When the “UnderworldJob1” job has completed its execution, its status will be changed to “Done”. As a completed job does not guarantee the job is successfully executed, a good indication of a successful job execution is to look at the output files a completed job generated.

The following screenshot demonstrates that the “UnderworldJob1” job has successfully completed and executed (you should be getting the same output files from your own job submission):

To further confirm a job was successfully executed, you can examine the contents of “vegl.sh.log” file by using the “Logs” tab in VGL. You should be seeing the following log if you job successfully executed. Feel free to examine the contents shown in Environment and Python sub-tabs.