"Seegrid will be due for a migration to confluence on the 1st of August. Any update on or after the 1st of August will NOT be migrated"
You are here: SEEGrid>Siss Web>ErddapSetup (09 Jun 2011, DerrickWong)EditAttach

ERDDAP Setup

Prerequiste:

  • Sun Java 1.5
  • Apache Tomcat 5.5 or above (Latest version of Tomcat 6.0 recommended)
  • THREDDS (Installation guide can be found here)
    • Though not a prerequiste, ERDDAP (in Coverage context) is used to compliment THREDDS.
  • Unix users would require X - Display and X - Server to be installed.
  • Unix users would require to include the following JVM startup parameters:
    • JAVA_OPTS="-Djava.awt.headless=true -Xmx768m -XX:MaxPermSize=192m"
    • The following configuration file is located based on the Operating System installed:
      • Debian: /etc/default/tomcat6
      • Centos: /etc/sysconfig/tomcat6

ERDDAP

  • ERDDAP (Environmental Research Division's Data Access Program) aggregates scientific data from diverse local and remote sources and offers a simple, consistent way to download subsets of the data in common file formats and make graphs and maps.
  • Solutions ERDDAP provides which are relevant to our Coverage systems:
    • ERDDAP lets you make requests in a standardized way, regardless of the data source's request protocol.
      • Gridded data is available via the DAP hyperslab protocol and WMS.
      • Tabular data is available via the DAP constraint protocol.
    • ERDDAP also provides Data Access Forms (web pages) which helps create the DAP requests
    • ERDDAP lets you choose the file format for the results (e.g netcdf, ascii, dods, html).

Setup.

  • On Linux, Mac, and Windows, download erddapContent.zip and unzip it into ${CATALINA_HOME}, creating ${CATALINA_HOME}/content/erddap.
  • Read the comments in ${CATALINA_HOME}/content/erddap/setup.xml and make the requested changes. setup.xml is the file with all of the settings which specify how your ERDDAP behaves.
  • Read the comments in Working with the datasets.xml File, then modify the XML in ${CATALINA_HOME}/content/erddap/datasets.xml to specify all of the datasets you want your ERDDAP to serve.
  • Next, download erddap.war into ${CATALINA_HOME}/webapps. By default, tomcat should auto deploy the war. (if not, check Tomcat installation guide)
  • Configure the fonts for images in setup.xml, e.g:
         <fontFamily>SansSerif</fontFamily> .
         
  • On Linux, ensure that file permissions are given to the tomcat installation. e.g chown -R tomcat6:adm ${CATALINA_HOME}/content/erddap*

Deploying ERDDAP using Context Path in Tomcat (Optional).

  • Instead of putting the war file in ${CATALINA_HOME}/webapps, you can deploy it using a context path. Note: This is currently only supported for Version 1.32, and the source for this is attached below (context.zip).
  • Unzip erddap.war into a preferred location (e.g /usr/local/erddap or c:\projects\erddap). For the remaining steps, we will reference this path as ${ERDDAP}
  • Copy erddap-tomcat.xml to ${ERDDAP}.
  • Open erddap-tomcat.xml: Edit the file and insert the following:
    <!-- Please expand $ERDDAP_HOME - remember $ERDDAP_HOME is not a real variable -->
    <Context path="/erddap" docBase="$ERDDAP_HOME/current">
    </Context>
    
    
  • You can either copy the modified source file and compile it on your own, or use the pre-compiled class.
  • Compiling the project
    • Download context.zip and copy EDStatic.java to ${ERDDAP}\WEB-INF\classes\gov\noaa\pfel\erddap\util.
    • Use the following command line (in ${ERDDAP}/WEB-INF)
    • javac -cp ./classes;../../../lib/servlet-api.jar;lib/activation.jar;lib/axis.jar;lib/commons-discovery-0.2.jar;lib/itext-1.3.1.jar;lib/janino.jar;lib/jaxrpc.jar;lib/joda-time-1.6.jar;lib/joid.jar;lib/mail.jar;lib/netcdfAll-latest.jar;lib/postgresql-8.3-604.jdbc3.jar;lib/saaj.jar;lib/slf4j-jdk14.jar;lib/tsik.jar;lib/wsdl4j-1.5.1.jar;lib/servlet.jar ./classes/gov/noaa/pfel/erddap/util/EDStatic.java
            
    • On success, EDStatic.class would be created (check timestamp)
  • Using pre-compiled class
    • Download context.zip and copy EDStatic.class to ${ERDDAP}\WEB-INF\classes\gov\noaa\pfel\erddap\util.
  • ERDDAP conventionally assumes that the content folder is in the ${CATALINA_HOME}\content, deploying ERDDAP outside of ${CATALINA_HOME}\webapps would require you to specify the location for content directory. This is done in ${ERDDAP}/WEB-INF/web.xml:
  •    <context-param>
          <description>Erddap Content Directory</description>
         <param-name>erddapContentDirectory</param-name>
              <param-value>$ERDDAP_CONTENT_PATH</param-value>
       </context-param>
      
  • Note: $ERDDAP_CONTENT_PATH is not a real variable, replace it with path of content directory.

Specifying Custom Location For Content Directory.

  • SKIP this step if you have deployed ERDDAP using context path.
  • In ${CATALINA_HOME}/conf/context.xml, set the system property erddapContentDirectory to point to the preferred directory.
  • e.g. erddapContentDirectory=/usr/local/erddap/

Configuring mod_proxy_ajp with Tomcat 6 for ERDDAP (for Linux, tomcat running in apache)

  • Ensure that the AJP 1.3 Connector is enabled on port 8009: vim $CATALINA_HOME/conf/server.xml
         <!-- Define an AJP 1.3 Connector on port 8009 -->
         <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" />
         
  • Locate configuration directory: ls -l /etc/apache2/sites-enabled/000-default
     lrwxrwxrwx 1 root root 26 2009-02-17 15:36 /etc/apache2/sites-enabled/000-default -> ../sites-available/default 
  • Edit the configuration file: vim /etc/apache2/sites-available/default
    • After , insert/edit the following:
            ProxyPreserveHost On
            ProxyRequests Off
            <Proxy *>
               Order deny,allow
               Allow from all
            </Proxy>
      
            #Tomcat Manager
            ProxyPass /manager ajp://localhost:8009/manager
            ProxyPassReverse /manager ajp://localhost:8009/manager
      
            #THREDDS
            ProxyPass /thredds ajp://localhost:8009/thredds
            ProxyPassReverse /thredds ajp://localhost:8009/thredds
      
            #ERDDAP 
            ProxyPass /erddap ajp://localhost:8009/erddap 
            ProxyPassReverse /erddap ajp://localhost:8009/erddap
      

  • Restart Apache2 and Tomcat 6:
    • /etc/init.d/apache2 restart
    • /etc/init.d/tomcat6 restart

  • To test that your have configured the AJP Proxy correctly, try to access the tomcat manager via your internet browser (e.g. http://apacsrv6/manager/html)

Configuring datasets.xml for ERDDAP.

  • Location: <ERDDAP Installation>/<content folder>/datasets.xml (location may vary for tomcat and another web container deployment) .
  • The following steps details how to register datasets from BoM, C3DMM, Geophysics and Satellite (via THREDDS OPeNDAP):
    • For Unix installs, the following tools require X - Display and X - Server to be installed and running.
    • Windows-based users who tunnel/SSH into these Unix setups would require Xming to be installed.
    • GenerateDatasetsXml - The following command line tool generates a rough draft of the dataset XML for almost any type of datasets.
      • Located in <ERDDAP Installation>/current/WEB-INF/
      • After running the batch file, answer the following questions shown on the command prompt.
      • The EDDType option used for these datasets is EDDGridFromDap.
      • Enter URL of dataset. Note: URL must be a DAP connection, e.g. http://apacsrv6/thredds/dodsC/sat/b62.nc
      • ReloadEveryNMinutes: 1440
      • Results are shown on the screen, put on the clipboard and put in <ERDDAP Installation>/<content log folder>/log.txt.
      • example of logs:
        <dataset type="EDDGridFromDap" datasetID="satalite" active="true">
            <sourceUrl>http://localhost:8082/thredds/dodsC/sat/b62.nc</sourceUrl>
            <reloadEveryNMinutes>1440</reloadEveryNMinutes>
            <altitudeMetersPerSourceUnit>1</altitudeMetersPerSourceUnit>
            <!-- sourceAttributes>
                <att name="Conventions">CF-1.0</att>
                <att name="Source_Software">ESRI ArcGIS</att>
            </sourceAttributes -->
            <addAttributes>
                <att name="cdm_data_type">Grid</att>
                <att name="Conventions">CF-1.0, COARDS, Unidata Dataset Discovery v1.0</att>
                <att name="infoUrl">http://localhost:8082/thredds/dodsC/sat/b62.nc.html</att>
                <att name="institution">LOCALHOST:8082</att>
                <att name="license">[standard]</att>
                <att name="Metadata_Conventions">CF-1.0, COARDS, Unidata Dataset Discovery v1.0</att>
                <att name="standard_name_vocabulary">CF-12</att>
                <att name="summary">LOCALHOST:8082 data from http://localhost:8082/thredds/dodsC/sat/b62.nc.das .</att>
                <att name="title">Sat b62</att>
            </addAttributes>
            <axisVariable>
                <sourceName>y</sourceName>
                <destinationName>y</destinationName>
                <!-- sourceAttributes>
                    <att name="long_name">y coordinate of projection</att>
                    <att name="standard_name">projection_y_coordinate</att>
                    <att name="units">Meter</att>
                </sourceAttributes -->
                <addAttributes>
                    <att name="ioos_category">Location</att>
                </addAttributes>
            </axisVariable>
            <axisVariable>
                <sourceName>x</sourceName>
                <destinationName>x</destinationName>
                <!-- sourceAttributes>
                    <att name="long_name">x coordinate of projection</att>
                    <att name="standard_name">projection_x_coordinate</att>
                    <att name="units">Meter</att>
                </sourceAttributes -->
                <addAttributes>
                    <att name="ioos_category">Location</att>
                </addAttributes>
            </axisVariable>
            <dataVariable>
                <sourceName>LS7_ETM_OTH_P51_GLS2000_102_070_20000716_b62</sourceName>
                <destinationName>LS7_ETM_OTH_P51_GLS2000_102_070_20000716_b62</destinationName>
                <!-- sourceAttributes>
                    <att name="coordinates">x y</att>
                    <att name="esri_pe_string">PROJCS[\&quot;WGS_1984_UTM_Zone_53N\&quot;,GEOGCS[\&quot;GCS_WGS_1984\&quot;,DATUM[\&quot;D_WGS_1984\&quot;,SPHEROID[\&quot;WGS_1984\&quot;,6378137.0,298.257223563]],PRIMEM[\&quot;Greenwich\&quot;,0.0],UNIT[\&quot;Degree\&quot;,0.0174532925199433]],PROJECTION[\&quot;Transverse_Mercator\&quot;],PARAMETER[\&quot;False_Easting\&quot;,500000.0],PARAMETER[\&quot;False_Northing\&quot;,0.0],PARAMETER[\&quot;Central_Meridian\&quot;,135.0],PARAMETER[\&quot;Scale_Factor\&quot;,0.9996],PARAMETER[\&quot;Latitude_Of_Origin\&quot;,0.0],UNIT[\&quot;Meter\&quot;,1.0]]</att>
                    <att name="grid_mapping">transverse_mercator</att>
                    <att name="long_name">LS7_ETM_OTH_P51_GLS2000_102_070_20000716_b62</att>
                    <att name="missing_value">0</att>
                    <att name="units">Meter</att>
                </sourceAttributes -->
                <addAttributes>
                    <att name="ioos_category">Unknown</att>
                </addAttributes>
            </dataVariable>
        </dataset>
                 
      • Copy the generated xml (everything encompassed by the <dataset> param) paste into <ERDDAP Installation>/<content folder>/datasets.xml, making changes when necessary.
      • You probably would want to change the datasetID attribute to something more readable. This ID is used in DasDds batch command and for users to identify datasets with.
    • DasDds is a command line program that you can use after you have created a first attempt at the XML for a new dataset in datasets.xml.
      • Located in <ERDDAP Installation>/WEB-INF/
      • Specify the datasetID you have previously entered into dataset.xml
      • On Linux, ensure that file permissions are given to the tomcat installation. e.g chown -R tomcat6:adm ${CATALINA_HOME}/<content folder>/<cache folder>/*
      • Restart Tomcat once this is done.
  • Detail documentation on how to configure dataset.xml can be found here.

Batch Configuration for ERDDAP (Optional).

  • When configuring multiple datasets in ERDDAP, the batch configuration utility can be used. Note: This utility was written for Version 1.28, and the source for this is attached below (Batch.zip).
  • The following steps details how this can be achieved:
    • Download the these files (attached above) and put them into the following locations:
      • BatchGenerateDatasetXml.bat
      • BatchGenerateDatasetXml
        • Put this in <ERDDAP Installation>/WEB-INF/
        • Batch files for generating the dataset skeleton. Edit the batch file to change where you want to redirect the output (currently BatchGenerateDatasets.xml)
        • e.g
          java -classpath ./classes/gov/noaa/pfel/erddap;./classes;../../../lib/servlet-api.jar;lib/activation.jar;lib/axis.jar;lib/commons-discovery-0.2.jar;lib/itext-1.3.1.jar;lib/janino.jar;lib/jaxrpc.jar;lib/joda-time-1.6.jar;lib/joid.jar;lib/mail.jar;lib/netcdfAll-latest.jar;lib/postgresql-8.3-604.jdbc3.jar;lib/saaj.jar;lib/slf4j-jdk14.jar;lib/tsik.jar;lib/wsdl4j-1.5.1.jar   gov.noaa.pfel.erddap.BatchGenerator generate > BatchGenerateDatasets.xml
          
      • BatchGenerateDatasets.xml
        • Example of generated file after calling BatchGenerateDatasetXml
      • batch.xml
        • e.g of what the end user needs to configure to use the batch upload.
        • <?xml version="1.0" encoding="ISO-8859-1"?>
          
          <datasets>
             <dataset>
                <id>onshore_only_Bouguer_geodetic_fixed2</id>
                <eddType>EDDGridFromDAP</eddType>
                <resourceURL>http://apacsrv6/thredds/dodsC/ga/onshore_only_Bouguer_geodetic_fixed2.nc</resourceURL>
                <reload>1440</reload>
             </dataset>
             <dataset>
                <id>bhdemsub_latlong</id>
                <eddType>EDDGridFromDAP</eddType>
                <resourceURL>http://apacsrv6/thredds/dodsC/c3dmm/bhdemsub_latlong.nc</resourceURL>
                <reload>1440</reload>
             </dataset>
          </datasets>
          
      • BatchGenerator.java
        • Put this file in \erddap\WEB-INF\classes\gov\noaa\pfel\erddap\
        • Java class written to handle this batch commands
      • EDDGridFromDap.java
        • Java class modified to accomodate the BatchGenerator class.
        • Put this file in \erddap\WEB-INF\classes\gov\noaa\pfel\erddap\dataset
    • If you want to compile these classes yourself, you can use the following command line (WEB-INF folder):
    • javac -cp ./classes;../../../lib/servlet-api.jar;lib/activation.jar;lib/axis.jar;lib/commons-discovery-0.2.jar;lib/itext-1.3.1.jar;lib/janino.jar;lib/jaxrpc.jar;lib/joda-time-1.6.jar;lib/joid.jar;lib/mail.jar;lib/netcdfAll-latest.jar;lib/postgresql-8.3-604.jdbc3.jar;lib/saaj.jar;lib/slf4j-jdk14.jar;lib/tsik.jar;lib/wsdl4j-1.5.1.jar ./classes/gov/noaa/pfel/erddap/BatchGenerator.java
            
    • Results are shown on the screen, put on the clipboard and put in /content/erddaplogs/log.txt

Debugging

  • The logs can be found at:
    • $CATALINA_HOME/logs
    • $CATALINA_HOME/content/erddaplogs/log.txt

Related links:

Tested on:

  • apacsrv6, cgsrv3

Acronyms

  • ERDDAP - Environmental Research Division's Data Access Program
  • OPeNDAP - Open Source Project for Network Data Access Protocol A network protocol for transmitting data across the internet. Though appropriate for many kinds of data, the system was designed with scientific data in mind.
  • THREDDS - Thematic Realtime Environmental Data Distributed Services A project of Unidata to create ways to establish useful collections of earth science data.
Topic attachments
I Attachment Action Size Date Who Comment
Batch.zipzip Batch.zip manage 128.2 K 31 Jan 2011 - 15:28 DerrickWong Batch Configuration Utility
context.zipzip context.zip manage 51.2 K 09 Jun 2011 - 18:03 DerrickWong Files used for deploying ERDDAP using context path
Topic revision: r15 - 09 Jun 2011, DerrickWong
 

Current license: All material on this collaboration platform is licensed under a Creative Commons Attribution 3.0 Australia Licence (CC BY 3.0).