Queuing Systems and WebSubmit Configuration
WebSubmit was developed as an interface to batch computing systems
on high-performance machines. Packages have been developed to deal
with Load Leveler, NQS, and LSF. Template
interfaces come with the distribution that should allow any site to configure
modules for each of these systems. However, in order to use each
set of modules, a few small system-dependent modifications need to be made
in some of the files that come with the distribution. The modifications
needed for each of the packages is detailed below.
General Package Information
The three relevant packages for queuing systems are wsLoadLeveler.tcl,
wsNQS,tcl, and wsLSF.tcl.
They are all located in $wsRootDir/lib.
Within each of these files, there is some system-dependent information
that needs to be changed in the primary namespace definition (at the top
of the file). System-dependent information is stored primarily in
two Tcl arrays: PATHS (for
information about paths to commands and files) and CONFIG
(for information about the system configuration). The remained of
the information in these packages should not rely on the details of your
system and hence need not be changed. PATHS
and CONFIG are associative arrays
that use a hostname / keyword combination to specify information.
The basic syntax for setting the values of these array elements is as follows:
set PATHS($hostName,$keyword)
value
When specifying information for your site, you should change the hostnames
and the values of the array elements, but not the keywords.
WebSubmit Manager
The WebSubmit GUI Manager provides a simple way to maintain all of the
packages in the distribution. Code in the distribution is tagged
so that system-dependent lines appear highlighted within the Manager GUI.
This makes editing easier. If you modify the package code within
the Manager, and wish lines to be tagged that you have added, simply start
the line with two semicolons (;;).
The next time you bring up this package in the Manager, the tagged lines
will be highlighted. More details about the manager can be found
in the relevant section of this manual.
Load Leveler Package: wsLoadLeveler.tcl
In order to set up the Load Leveler package, all that needs to be set are
the variables in the PATHS array. The following paths need to be
set within this array for each Load Leveler host connected to WebSubmit:
-
adminFileName: Absolute path to
LoadL_admin on $hostName
-
localAdminFileName: Absolute path
to locally-cahced copy of admin file
-
Commands: Absolute paths for the following commands should be set:
-
llsubmit
-
llcancel
-
llq
-
llstatus
-
llclass
-
llhold
-
llprio
-
jmsum
-
poe (If using the MPI module)
The Load Leveler admin file is parsed to dynamically determine the class
information for the specified host. In order to reduce overhead,
this file is not continually parsed. A local copy is maintained and
updated on a periodic basis. $wsRootDir/lib/LL
is provided as a repository directory for this information. Nesting
admin files within their own host directories at this level is probable
advisable. Hence, if your host is called some.host.name,
create $wsRootDir/lib/LL/some.host.name
as the directory to hold the admin file for this host.
NQS Package: wsNQS.tcl
The NQS package requires both path and configuration information to be
set. The path information points to the absolute paths for the relevant
NQS commands; configuration information specifies some information about
time, memory, and CPU limits on the host in question.
PATHS
The following variables should be modified for each host in PATHS.
In each case, provide the absolute paths to the relevant command.
CONFIG
The CONFIG array is set to contain information about the resource limits
on a given host. The following resource limits should be configured
for each NQS host connected to WebSubmit:
-
hostList: List all NQS hosts (no
hostname is provided as part of the keyword)
-
maxProcessors: The maximum number
of processors that can be used by a single user
-
maxMemory: The maximum amount of
memory that can be used (in Gb)
-
maxCPUTime: The maximum allowable
CPU time (in seconds)
LSF Package: wsLSF.tcl
The LSF package is similar to that for NQS in that both PATHS and CONFIG
need to be modified for your systems. As with the other packages,
information should be provided for each relevant host.
PATHS
The following variables should be modified for each host in PATHS.
In each case, provide the absolute paths to the relevant command.
-
bsub
-
bqueues
-
bjobs
-
bkill
-
bpeek
-
bstop
-
bresume
-
jmsum
CONFIG
The CONFIG array is set to contain information about the resource limits
on a given host. The following resource limits should be configured
for each NQS host connected to WebSubmit:
-
hostList: List all LSF hosts (no
hostname is provided as part of the keyword)
-
maxProcessors: The maximum number
of processors that can be used by a single user
Adding Queuing System Modules
Template files for job submission and job queue control are provided with
the WebSubmit distribution for Load Leveler, NQS, and LSF. The template
modules are contained in $wsRootDir/modules/templates/(LoadLeveler,NQS,LSF)/*.
In order to add one of these modules to your system for host $hostName,
follow this algorithm:
-
Create directory $hostName
in $wsRootDir/modules
-
Copy the relevant template directories into $wsRootDir/modules/$hostName.
For example, to add NQS templates, copy the nqsJob
and queueStat directories under
$wsRootDir/modules/templates/NQS
into $wsRootDir/modules/$hostName.
-
If you wish to rename the scripts, make sure to follow the naming convention:
-
If module CGI scripts reside in directory $moduleName,
then the module CGI script should be named $moduleName.cgi,
and its associated processing script (if any) should be called ${moduleName}submit.cgi
-
Add the relevant entries to master.db
so that the scripts are accessible
-
Update the relevant package (wsLoadLeveler.tcl,
wsNQS.tcl, or wsLSF.tcl)
so that the package contains the needed information for $hostName.
-
Make sure all files and directories under $hostName
have permissions 755.
-
Update auth.db so that it contains
the login information for users on $hostName