Master Page

Overview

The Master page serves as the main entry point to all of the WebSubmit modules available at your site.  It serves to organize this information and present it  to the users.  All WebSubmit modules can be accessed via hyperlinks from this page.  The master page forms the main frame of the WebSubmit home page server by the WebSubmit HTTPS server.

The appearance of the master page is governed by the master page database ($wsRootDir/lib/master.db) and  its auxiliary description file ($wsRootDir/lib/master.txt).  Fle master.db specifies what modules are available at your site and organizes them hierarchically on the master page.  File master.txt contains descriptions attached to entries in the master page.  The HTML for the master page is generated automatically from the database files when file $wsRootDir/bin/master.cgi is invoked.

The master database can be built starting from the prototypes provided with the system (master.db.example and master.txt.example).   Rename these files without the extension .example to use them.  The Configuration Manager  provides an easy way to add entries to the master database.

Module Organization
Modules are grouped into two basic classes: (1) Generic modules, applicable to all systems in the cluster, and (2) System-specific modules, used for running jobs or applications on a single system in the cluster.   Generic modules can be contained under a single heading on the master page.  System-specific modules can be collected under a heading for the system in question.  Within the collection of modules for a specific system, these may be further organization into subcategories.  Records in  the master database describe either containers or scripts.  Containers contain either other containers or scripts; scripts represent the module scripts to be executed.  Containers  permit  complicated sets of modules to be organized in a hierarchical way, using only a flat database file as specification.

The following example, based on master.db.example from the distribution, will better serve to illustrate these points.  Suppose that your site has three systems in its WebSubmit cluster:  LoadLeveler System A, NQS System A, and LSF System A.  These imaginatively named systems exemplify the three batch queuing systems presently supported in WebSubmit.  You choose to define a collection of modules for each of these systems, coincidentally exemplifying the full set of modules presently available with the distribution, and you wish to use the full set of distributed generic (all remote systems) modules as well.  A possible master page layout for this situation is given below.  All containers (and also the title) are given in dark magenta, and all scripts are indicated as hyperlinks.  Comments from master.txt are in black.
 

WebSubmit Master Page

Generic Tasks

Generic commands can be executed on any WebSubmit cluster system. 
  • Command Execution 

  • Generalized command facility for running commands on any WebSubmit cluster system. Commands can be executed and their output captured for display. 
     
  • Simple File Editor 

  • A simple file editor to allow users to make changes to ASCII text files stored on any WebSubmit cluster system. 
     
  • File Transfer 

  • File transfer interface to move single files between the client (local) system and any WebSubmit cluster system. 

LoadLeveler System A

The ex.org LLA system is a ??? with ??? nodes. The front end is methane.ex.org. Jobs are scheduled with the LoadLeveler (LL) batch queue control system.
  • Generic Job Submission 

  • A generic interface for submitting LL jobs to methane.ex.org.
     
  • MPI Job Submission 

  • An interface for submitting LL jobs parallelized with the Message Passing Interface (MPI) library to methane.ex.org. 
     
  • Gaussian Job Submission 

  • An interface for submitting LL Gaussian jobs to methane.ex.org. Gaussian is a first-principles quantum chemistry package for simple LCAO calculations on atoms and molecules. 
     
  • Queue Status Report 

  • An interface for checking and controlling the status of jobs currently in the queue on methane.ex.org. 

NQS System A

The ex.org NQSA system is a ??? CPU ???. Jobs are scheduled with the Network Queuing System (NQS) batch queue control system.
  • Generic Job Submission 

  • A generic interface for submitting NQS jobs to ethane.ex.org.
     
  • Gaussian Job Submission 

  • An interface for submitting NQS Gaussian jobs to ethane.ex.org. Gaussian is a first-principles quantum chemistry package for simple LCAO calculations on atoms and molecules. 
     
  • Queue Status Report 

  • An interface for checking and controlling the status of jobs currently in the queue on ethane.ex.org. 

LSF System A

The ex.org LSFA system is a ??? with ??? nodes. Jobs are scheduled with the Load Sharing Facility (LSF) batch queue control system.
  • Generic Job Submission 

  • A generic interface for submitting LSF jobs to propane.ex.org.
     
  • MPI Job Submission 

  • An interface for submitting LSF jobs parallelized with the Message Passing Interface (MPI) library to propane.ex.org. 
     
  • Queue Status Report 

  • An interface for checking and controlling the status of jobs currently in the queue on propane.ex.org. 
Sample Master Page: This is a sample master page.  The database used to create this page is given in the following section.
 

Module Types and Script Hyperlinks
In an effort to accommodate future plans for using  modules generated from specifications as well as ones hand-written in some CGI scripting language, a build attribute has been associated with script records.  Modules can be of one of two types: (1) hand-written by the developer (build = MANUAL), and (2) generated automatically from a specification in a hypothetical WebSubmit module specification language (build = AUTO).  The master database indicates whether a script is AUTO or MANUAL, in order to allow for different forms of hyperlinks on the master page for the two kinds of module.  (The AUTO type of module is not yet implemented.  JEK)

We also support a third option:  (3) pre-existing CGI scripts that don't use WebSubmit's tools to ensure that they are being accessed by a properly authorized user (build = WRAP).  (Note: This option is broken in this release.  JEK)

Database Configuration and Manipulation

General Database Structure
The master database follows the same basic structure used for other WebSubmit databases.  It contains an attribute specification list and a set of records structured according to that list.  The following attributes are required in master.db. Each record within the database is then a colon-delimited collection of the data that matches these attributes.  A * is used to indicate that the data is not applicable for a particular attribute (e.g., build for a CONTAINER).

It is possible to attach a descriptive paragraph to each container and script in the master page.  This is done by providing a master.txt file that is essentially a database mapping nodeNames to a text paragraph.

Constructing a Master Database
The basic procedure for construcing a master database can be summarized as follows:

  1. Construct lists of all modules and hosts that will be used at the site/
  2. For each module:
  3. Decide on a framework of containers to hold the modules in a logical, organized fashion
  4. For each container:
  5. Determine the appropriate nodePath for each module or container based on the organization of containers.
  6. Once all containers and modules have been specified, determine how you wish them to be laid out on the master page, and then determine the corresponding index entries in the database
  7. Build a collection of database records that correspond to the modules and containers you have outlined.
Sample Database
The following is the database that would give the example master page shown above.  Notice that the actual machine names of the frontends for LoadLeveler System A, NQS System A, and LSF System A are methane.ex..org, ethane.ex.org, and propane.ex.org.
::DB_ATTRIBUTES:: key:nodeName auto moduleName bqs host  
 title visible state type build index defaultMode
 sTitle nodePath

generic          : *           : *   : *               
: Generic Tasks            : ON  : ON  : CONTAINER : *      : 1  : *     
: *              : generic

command          : genExecCmd  : *   : *               
: Command Execution        : ON  : ON  : SCRIPT    : MANUAL : 1  : *     
: Commands       : generic/command

fileEdit         : genFileEdit : *   : *               
: Simple File Editor       : ON  : ON  : SCRIPT    : MANUAL : 2  : *     
: File Editor    : generic/fileEdit

fileTrans        : genFileTrans: *   : *               
: File Transfer            : ON  : ON  : SCRIPT    : MANUAL : 3  : *     
: File Transfer  : generic/fileTrans

LLA              : *           : *   : *               
: LoadLeveler System A     : ON  : ON  : CONTAINER : *      : 2  : *     
:                : LLA

methaneJob       : llJob       : LL  : methane.ex.org  
: Generic Job Submission   : ON  : ON  : SCRIPT    : MANUAL : 1  : BASIC 
: Generic Jobs   : LLA/methaneJob

methaneMPI       : llMPI       : LL  : methane.ex.org  
: MPI Job Submission       : ON  : ON  : SCRIPT    : MANUAL : 2  : BASIC 
: MPI Jobs       : LLA/methaneMPI

methaneGaussian  : llGaussian  : LL  : methane.ex.org  
: Gaussian Job Submission  : ON  : ON  : SCRIPT    : MANUAL : 3  : BASIC 
: Gaussian Jobs  : LLA/methaneGaussian

methaneQstat     : llQstat     : LL  : methane.ex.org  
: Queue Status Report      : ON  : ON  : SCRIPT    : MANUAL : 4  : *     
: Queue Status   : LLA/methaneQstat

NQSA             : *           : *   : *               
: NQS System A             : ON  : ON  : CONTAINER : *      : 3  : *     
: *              : NQSA

ethaneJob        : nqsJob      : NQS : ethane.ex.org   
: Generic Job Submission   : ON  : ON  : SCRIPT    : MANUAL : 1  : BASIC
: Generic Jobs   : NQSA/ethaneJob

ethaneGaussian   : nqsGaussian : NQS : ethane.ex.org   
: Gaussian Job Submission  : ON  : ON  : SCRIPT    : MANUAL : 2  : BASIC 
: Gaussian Jobs  : NQSA/ethaneGaussian

ethaneQstat      : nqsQstat    : NQS : ethane.ex.org   
: Queue Status Report      : ON  : ON  : SCRIPT    : MANUAL : 3  : *     
: Queue Status   : NQSA/ethaneQstat

LSFA             : *           : *   : *               
: LSF System A             : ON  : ON  : CONTAINER : *      : 4  : *     
: *              : LSFA

propaneJob       : lsfJob      : LSF : propane.ex.org  
: Generic Job Submission   : ON  : ON  : SCRIPT    : MANUAL : 1  : BASIC 
: Generic Jobs   : LSFA/propaneJob

propaneMPI       : lsfMPI      : LSF : propane.ex.org  
: MPI Job Submission       : ON  : ON  : SCRIPT    : MANUAL : 2  : BASIC 
: MPI Jobs       : LSFA/propaneMPI

propaneQstat     : lsfQstat    : LSF : propane.ex.org  
: Queue Status Report      : ON  : ON  : SCRIPT    : MANUAL : 3  : *     
: Queue Status   : LSFA/propaneQstat
The corresponding master.txt file follows:
::KEY:: generic
Generic commands can be executed on any WebSubmit cluster system.  

::KEY:: command
Generalized command facility for running commands on any WebSubmit
cluster 
system.  Commands can be executed and their output captured for 
display. 

::KEY:: fileEdit
A simple file editor to allow users to make changes to ASCII
text files stored on any WebSubmit cluster system.

::KEY:: fileTrans
File transfer interface to move single files between the client 
(local) system and any WebSubmit cluster system.

::KEY:: LLA
The ex.org LLA system is a ??? with ??? nodes.  
The front end is methane.ex.org.
Jobs are scheduled with the LoadLeveler (LL) batch queue control 
system.

::KEY:: methaneJob
A generic interface for submitting LL jobs to methane.ex.org.

::KEY:: methaneMPI
An interface for submitting LL jobs parallelized with the Message
Passing Interface (MPI) library to methane.ex.org.

::KEY:: methaneGaussian
An interface for submitting LL Gaussian jobs to methane.ex.org.  
Gaussian is a first-principles quantum chemistry package for
simple LCAO calculations on atoms and molecules.

::KEY:: methaneQstat
An interface for checking and controlling the status of jobs 
currently in the queue on methane.ex.org.

::KEY:: NQSA
The ex.org NQSA system is a ??? CPU ???.  
Jobs are scheduled with the Network Queuing System (NQS) batch
queue control system.

::KEY:: ethaneJob
A generic interface for submitting NQS jobs to ethane.ex.org.

::KEY:: ethaneGaussian
An interface for submitting NQS Gaussian jobs to ethane.ex.org.
Gaussian is a first-principles quantum chemistry package for
simple LCAO calculations on atoms and molecules.

::KEY:: ethaneQstat
An interface for checking and controlling the status of jobs 
currently in the queue on ethane.ex.org.

::KEY:: LSFA
The ex.org LSFA system is a ??? with ??? nodes.
Jobs are scheduled with the Load Sharing Facility (LSF)
batch queue control system.

::KEY:: propaneJob
A generic interface for submitting LSF jobs to propane.ex.org.

::KEY:: propaneMPI
An interface for submitting LSF jobs parallelized with the 
Message Passing Interface (MPI) library to propane.ex.org.

::KEY:: propaneQstat
An interface for checking and controlling the status of jobs 
currently in the queue on propane.ex.org.