WebSubmit Help Desk


Table of Contents

Overview

Setup and Configuration

WebSubmit Home Page

Page Layout
Link Bar
Comments and Reports

Usage
Overview
Comments about Form Elements
Session Management

System-Wide Application Interfaces

General System Commands
File Transfer
SImple File Editor

Load Leveler Application Interfaces
General Job Submission
MPI Job Submission
Gaussian 94 Job Submission
SP2 Queuing System Control

SGI Origin 2000 Application Interfaces
General NQS Job Submission
Queuing System Control


Overview

WebSubmit is intended to serve as a universal user interface to networked high-performance computing resources.  Its primary function is to allow users to submit job requests, execute simple commands, and control job execution.  A WebSubmit server is used as an intermediary between the client's browser and one of a group of valid remote computer systems.  The main WebSubmit page contains links to a group of application modules, each of which performs a specific function aimed at a specific remote computer system.  For example, a link on the WebSubmit home page might point to a module to interface to the IBM SP2 queuing system.  Each application module is currently implemented as a simple HTML form, although future versions of WebSubmit might provide more flexibility.
 
 At the present time, the WebSubmit cluster at NIST encompasses the IBM SP2 (danube.nist.gov) and the SGI Origin 2000 (arno.nist.gov); interfaces are planned for the Pentium cluster (rapid.nist.gov) and the CRAY (granta.nist.gov).  Access to systems outside of the WebSubmit cluster using WebSubmit is not provided for.
 
 

Setup and Configuration

Registration

WebSubmit uses an authentication system to restrict system access to valid users.  Authentication between the user and the WebSubmit server is accomplished using a special digitally-signed certificate, issued by a valid WebSubmit certificate authority.  In order to use the system, users must register themselves with this authority and receive a certificate to gain access.  The procedure for obtaining a certificate is as follows:
  1. Register the WebSubmit certificate authority with your browser software
  2. Submit a certificate request
  3. Once a certificate has been generated, load it into your browser software
Each of these tasks can be accomplished through a central Certificate Manager maintained on the WebSubmit server.  If you plan to use WebSubmit from a variety of different machines that access different certificate databases, then you must either export your certificate to those machines or obtain additional certificates.  At present, WebSubmit only supports the use of Netscape Communicator v3.0+.  If you have further questions about certificates and registering with the system, please contact the WebSubmit administrator.
 

Remote System Configuration

WebSubmit can be used to access a select group of machines within the WebSubmit cluster.  In order to access these machines properly, a configuration file must exist for each user on each machine they wish to utilize.  This configuration file allows the WebSubmit server to execute processes on that system in the user's name.  Execution is performed using secure protocols (Secure SHell and Secure CoPy) to minimize the risk of invalid users gaining access to your account.  SSH and SCP can be utilized by the WebSubmit server provided the single file .shosts exists in the user's home directory on the desired system.  This file should contain the following line:

      websubmit.nist.gov    nobody

Its access permissions should be set so that this file is readable and writable only by the user.  On UNIX systems, this corresponds to permissions of 600 (-rw-------).  If this file is not present on the target computer system, WebSubmit will raise an error indicating that there was a permission problem.
 
 

WebSubmit Home Page

Users access all modules on the WebSubmit system through a main home page.  The home page is laid out into three regions (HTML frames).  Along the left border of the screen, there is a collection of HTML links that act as a menu bar (the Link Bar).  Each element of the menu either connects to related information or to configuration features of WebSubmit.  Along the foot of the page there is a link that allows user to contact the WebSubmit adminstrator regarding problems with the system.  The central frame (Master Page) contains links to the WebSubmit application modules.
 

Link Bar

The link bar (located along the left edge of the WebSubmit screen) contains links to various useful WebSubmit help, information, and configuration functions.

Master Page

The central frame contains links to the WebSubmit application modules.  These are organized into classes, each of which contains modules relevant to a single target compute engine).  Within each class, the individual application modules are further organized into functionally-related elements. For example, if a series of modules is used to interact with a queing system on a specific machine, then each will appear under a heading related to the queing system.
 

Comments and Reports

At the foot of the master page (and all WebSubmit pages), there is a link that allows users to send mail to the WebSubmit administrator.  Mail should be sent to the WebSubmit administrator to report problems with the software, to request changing the user's status (active or inactive), or to present general inquiries.
 

Usage

Overview
WebSubmit was designed to be easy to use.  Users should be familiar with the type of forms interfaces to application modules.  A simple interaction with an application interface would proceed as follows:
  1. Go to the WebSubmit master (home) page
  2. Click the link to the desired application module
  3. Fill out the form for the application module with all necessary information
  4. Submit the form by clicking on the relevant action buttons.
Directions for specific application modules are contained below.  The behavior of the form elements within each module should be transferrable across modules, so usage of one module should aid in utilizing another.
 

Comments about Form Elements

Relative vs. Absolute Pathnames
Some form elements request file or directory names.  In most cases, these names can be entered using either relative or absolute pathnames.  Absolute pathnames begin with a  / (e.g., /some/directory/name), whereas relative paths do not (e.g., some/relative/path).  If not explicitly stated, relative pathnames will be interpreted as relative to the user's home directory on the relevant system.  For example, if a user specifies a directory name as foo/bar, and their home directory on the relevant system is /home/user1, then the directory name will be interpreted by WebSubmit as /home/user1/foo/bar unless otherwise stated.
 

Session Management

Some of the WebSubmit modules (e.g., General Job Submission) provide the user with the possibility of saving their form input so that ti can be recalled at a later date.  Each group of form data is considered as a session that can be saved in the Session Library for that particular module.  A more detailed discussion of Session Management and its use in WebSubmit can be found in the Session Management section of the Reference Guide.

 

System-Wide Application Interfaces

Individual application interfaces perform specific functions on remote systems.  They are either general (i.e., are useful for all systems), or they are specific to a given system.  For example, the General Command Interface can be used on any system, but the Load Leveler interface is only useful for the IBM SP2 (or other machines using Load Leveler as a queuing system).  To enter the interface for a specific application, simply click on the link in the master page that points to this application.   The following sections cover all system-wide application interfaces.
 

General System Commands

The general command interface allows users to execute (UNIX) commands on remote systems and view the output from these commands.   Commands are executed on a specified Remote Host (selected from a pull-down menu) within a specified working directory.   Output from and command(s) executed is returned in a different browser window.  Three slightly different modes of interaction are available:
  1. Single user-specified commands
  2. Single preselected commands
  3. Groups of commands
Relative pathnames for working directories are interpreted as relative to the user's home directory on the selected remote system.  Absolute pathnames can also be entered, although the user may not have permission to execute arbitrary commands in directories not within their directory hierarchy.

Single User-Specified Commands
Under the section Simple System Commands, there is a command line where users can enter and execute arbitrary commands.  Simple type the command to execute and then click the Submit button to run.  The command will execute as if it had been typed from the shell in the specified working directory on the selected remote host.

Single Preselected Commands
Under the section Simple System Commands, there is a series of preselected commands that are commonly used on UNIX systems.  For each of these commands, there is a brief description, a pull-down menu of a few options, and a text field where users enter arguments to the command.  As with single user-specified commands, each of these commands is executed in the specified working directory on the selected remote host.  In order to use one of these commands, select the desired options and arguments for the command, then press the button in the column labeled Command.  Pressing the Help button in a given command row will bring up the UNIX man page for that command on the selected remote system.

Groups of Commands
Under the section General Command Facility, there is a text window where several individual commands can be entered.  This is similar in spirit to a simple command shell.
 
 

File Transfer Interface

File transfers occur between the browser (client) system and the specified remote system.  The basic process for transferring file(s) is as follows:
  1. Select the remote system from the pull-down menu
  2. Specify the file(s) to be transferred
  3. Select the appropriate options for the transfer
  4. Select the proper archive options if necessary
  5. Click the arrow corresponding to the direction of file transfer
The current interface allows you to perform several types of file transfers between the client system (your local computer) and a target remote system: The type of transfer that is desired determines how the files are selected and the configuration of any file transfer or configuration options.

Modes of File Transfer

Single File Transfer
     If you wish to transfer a single file between the client and remote systems, you simply need to select the name of the file you wish to transfer.  On the client system, you can browse your filesystem by clicking the Browse... button, or just select an individual file.  At present, there is no mechanism for browsing a remote file system, so the file must be manually selected.  If you are transferring a file from the Remote to Client, then no client filename needs to be specified; you will be prompted for the name of the local file to which the remote file will be copied.  If you are transferring a file from the Client to Remote, then the remote filename can be specified in two ways:  (1) as a remote directory, to which the file will be copied or (2) as the new filename on the remote system.

Download a Group of Files or a Directory Hierarchy from Remote to Client
     Groups of files or entire directory hierarchies can be transferred from the remote host to the client system, but they must be transferred as an archive.  Four archive formats are currently supported: zip, gzipped tar, compressed tar and binhex.  In order to transfer a group of remote files or a directory structure, check the box for Remote-to-Client and select the format for the transferred data.  The file for the remote system must either be a wildcard expression for a group of files (i.e.,the expression contains * or ?) or a directory name.

Upload an Archive from Client to Remote
     Client archives can be transferred and unpacked on the remote system, provided they already exist as archives on the client system.  Simply specify the name of the client archive file, check the box Unpack Client-to-Remote, and choose the format of the file.  The remote file should specify the name of the directory in which the archive will be unpacked (default = remote home directory).  If the specified remote file does not exist, a directory is created with this name and the archive is unpacked in this location.
 

Options

Archive Options

Simple File Editor

The file editor interface is a simple tool that users can use to edit ASCII files on remote systems.  It is not a full-featured text editor like vi or emacs, and it is not intended to replace eitehr of these utilities.  It does, however, provide a means for making quick changes to straight ASCII files.  Binary files or documents created with word-processing utilities like Wordperfect cannot be edited.  Using the editor is very simple:
  1. Select the remote host where the file is located
  2. Select the name of the file to edit
  3. Load the file for editing
  4. Save the file once changes are complete (if desired)
The output of any commands (i.e., Load or Save) will be presented to the user at the top of the editor interface, just below the WebSubmit user ID and user name.  This indicates whether the action was successful or not.

Remote Host
Choose the remote host where the file is located using this pull-down menu.  Only machines within the WebSubmit hierarchy are available as remote hosts.

Remote File
Choose the name of the remote file to be edited by entering its name in this text field.  Pathnames, if not absolute, are interpreted as relative to the user's home directory on the specified remote host.  No browsing facility exists, so users must be careful to select existing files.

Action

Overwrite when saving
Determines whether the file in the text area will overwrite an existing file on the remote system.  For example, assume a  Remote File testfile is loaded, edited, then saved.  If the No box is checked next to overwrite, then changes to this file will not be saved to testfile on the remote system (since it already exists).
 
 

Load Leveler Application Interfaces

A group of interfaces is used to interact with the Load Leveler queuing system on the IBM SP2.  These provide facilities for job submission and for the control of jobs once they have been submitted.
 

General Job Submission

This module is used to submit general parallel jobs to Load Leveler.  It is ideal if the user has a compiled code that they wish to run in parallel on the SP2.  It allows for the specification of the working directory, executable name and arguments, data file names, and a subset of Load Leveler keywords.  The only required element on the form is the name of the executable code; all other elements have default values if they are left blank, although these defaults may not coincide with the desired values for the given job.  The general method for using this module is to fill out the form with information related to the desired job, and then to click the Submit Job button.  Session management functions can be performed using the Session Manager at the bottom of the form.  Specific fields within the form are discussed below.
   
 

MPI Job Submission

This module is used to submit parallel jobs that utilize the Message Passing Interface (MPI) for communication.  These jobs are executed within the Parallel Operating Environment (POE).  This module is similar to the Generic Job Submission module, with two important distinctions.  First, the executable code you specify will be passed as an argument to poe (along with any arguments to your code itself); general executable scripts are not allowed.  Second, the job_type is assumed to be parallel.  A description of specific elements follows.
 
BASIC mode elements: ADVANCED mode elements:  

Gaussian 94

This module is used to submit parallel jobs for the Gaussian 94 quantum-chemistry package.   There are basic and advanced modes for the form that allow the user to control details of the job submission to different degrees.  As with the general job submission module, a job is submitted by filling out the form and clicking the Submit Job button.

Required Information (BASIC and ADVANCED modes)
The only information required by WebSubmit for the successful submission of a job is a working directory and a Gaussian input file.  The directives file must exist either on the remote computing system (where the job is run), or on the user's browser system.  The user must specify one input file (either local or remote); if two files are specified, an error is returned.

Optional Information (ADVANCED mode only)
The following information can be specified if the advanced mode of the form is being used.  If no values are specified, the default values will be used by WebSubmit.  
 

SP2 Queuing System Control

This module is used to examine the current queue status on your SP2 system, and to control your jobs that are currently within the queue.  When you have jobs queued in the system, a table will be shown with details about each job.  If you click the checkbox to the right of the job and then execute one of the Load Leveler Actions given below, this action will be taken on the selected jobs.
 
  • Jobs Currently Queued

  • The jobs that are currently in the queue for the current WebSubmit user.  This table presents information about each job and allows you to select the job (by clicking the checkbox) for further action.  Any selected job can be killed, held, have its scheduling priority changed, and have a detailed listing about the job displayed.  If you select multiple job and then choose an action to be performed, it will be performed on each job selected.

    SGI Origin 2000 Application Interfaces

    General NQS Job Submission

    This module is used for submitting general batch jobs to the Network Queuing System (NQS) operating on the SGI Origin 2000.  The only element that is required in this form is the Executable Script.  As with other WebSubmit modules, simply fill out the form and click the Submit Job button to execute your job.

    Queuing System Control

    This module provides users with the ability to examine and control their jobs in the NQS queue.  Details about the meaning of display elements can be found in the  NQS Documentation or in the qstat manual pages.
     
  • Jobs Currently Queued

  • The jobs that are currently in the queue for the current WebSubmit user.  This table presents information about each job and allows you to select the job (by clicking the checkbox) for further action.  Any selected job can be killed or have a detailed listing about it displayed.  If you select multiple job and then choose an action to be performed, it will be performed on each job selected.