This Utility performs grid enabled file transfers between resources that use GSI authentication. It also allows the user to manage the filespace on remote resources.
An image displaying the ftp interface can be found here
The interface can display the file listing of any readable directory on the remote machine. Directories and files are clearly marked and listed separately from one another. A full unix file description (complete file name, file size, date last modified) is displayed when the file/directory is selected. The file listing can ordered by the filename extensions. The user can navigate easily through the file space with mouse clicks. Standard unix file space management is also provided. One can create new directories, delete files and rename existing files. File transfers can be done in several ways: individual file transfers, entire directory transfers, and multiple file tranfers. In each of these cases the user can choose to rename the selected files or directories.
Our code that enables the file tranfers is based on the Java CoG. It uses the GridFTPClient class. New instances of this class are created each time the ftp interface page is loaded in the browser window. In the authetication process the portal loops though the user's active credentials to determine which type of certificate is accepted by the remote host. While file listings on a given machine succeed for multiple certificates (assuming proper grid-mapfile entries), for successful file transfers the certificate used to autheticate to the two grid machines must be signed by the same authority. Also, the GridFTP daemons running on remote supercomputers must be configured to support third party file transfers for this software to work.
GRAMThe GRAM section of the portal enables the user to submit a job to a remote resource though the web interface. The Globus Resource Allocation Manager (GRAM) is the lowest level of Globus resource management architecture. GRAM allows the user to run jobs remotely, providing an API for submitting, monitoring, and terminating your job.
An image displaying the GRAM job submission interface can be found here
This section contains a thorough interface for preparation of a GRAM job submission. Through the portal interface one can submit jobs to interactive or batch systems. A complete list of the batch queues on the remote supercomputer is provided. In the near future, this list will provide load and scheduling information pertinent to the queues. One is also able to run MPI jobs, multiple jobs and to set the environment variables required by his/her job. An option for selecting the active certificate to be used for authetication is provided. Alternatively, the portal can loop though the user's active credentials and find one that is accepted by the remote host.
For interactive jobs, one can monitor the progress of a submitted job via an interactive java applet or if preferred one can also get a plain html display. The status of the job is updating continuously until completion. The user also has the ability to cancel a job that has not been completed.The portal registers each job submitted into a database. This enables the user to view details of an archived gram job (the complete RSL script, the job output, the status of the job, the job environment, etc.) and to stage it back into the interface if desired.
We are using the GramJob, GramAttributes and GassServer Java CoG classes. The GramAttributes object represents the full RSL script and the GASS server is used to communicate the standard input, output and error of an interactive job.
This section enables users to run demo cactus simulations, i.e., the user is not required to have their own cactus executables or parameter files. Instead the executable is staged from a single repository maintained by portal administrators. The gatekeeper on the target host is directed to the address of the executable on the repository (for example, modi4.ncsa.uiuc.edu/jobmanager is told to execute http://nexus.ncsa.uiuc.edu:8080/cactus/cactus_wave). The parameter file is transferred using GridFTP from the same repository to the target host.
This repository presently holds WaveToy executables compiled for sgi and linux platforms. In the demo section one launches the cactus simulation via GRAM and the demo attributes are recorded in the portal database. The user can see a history of submitted cactus demos on the local demos page. Finer details of the job submission can be viewed in the Utilities/GRAM section.
Once the demo simulation is started the job output can be viewed in an inteactive java applet. The present iteration number is displayed in plain html by reloading the view page. While the simulation is in progress an HTTP interface providing further information about the running Cactus simulation (included Cactus Thorns, parameter file, time to completion, downloadable output files, etc.). From the HTTP interface one can choose to cancel the simulation. If the demo simulation is completed the user can launch the File Advertiser to restart the same HTTP interface. A listing of output files in the data directory is gathered via GridFTP and displayed to the users. One can launch a simple visualization tool to plot the generated data files.
An image displaying the demo view page can be seen on this page.
In order to launch any visualization tools through the portal the user is required to have an X environment. Additionaly, prior to launching a visualisation tool one needs to run xhost +<remote_host> on the local client machine.
Cactus ConfigurationThis section allows the user to perform standard cvs checkouts (no grid security, yet) of the cactus code. The portal runs cvs checkout as a GRAM job. The user can checkout Cactus Flesh, entire arrangements or individual thorns.
The portal keeps track of cvs logins and passwords, keeping the user from having to reenter these for each cvs checkout. The user simply enters them once into a setup interface before initiating any cvs commands.
Furthermore, the portal allows users to configure personal cvs repositories and their contents for use in the portal. Portal administrators perform this task for main group cvs repositories such as cvs.cactuscode.org:/cactus. This interface is displayed here.
This section also enables a user to duplicate a configuration that resides on one machine on a second resource. The method that we use to accomplish this is to perform the same sequence of cvs checkouts on the remote machine. This not an exact copy as it pulls out the present content from cvs which can differ from the earlier checkouts.
Look here for the view page of a cactus configuration built within the portal.
In the future we want the portal to:
The Cactus Simulations section is an extension of the Utilities/GRAM interface. Furthermore, it contains a special achive with both: 1) simulations submitted by the user inside portal sessions, and 2) external simulations that other group members have announced to the portal. Upon selection of an individual simulation from the archive the portal displays a detailed information page with current dynamic values and file advertiser/HTTP interfaces.
The Cactus simulation interface separates the cactus specific elements from the standard GRAM arguments, facilitating job submission. For an interactive simulation one can use the java applet to follow the progress of the job, see the current iteration number, the HTTP port where the simulation is running, etc. Upon completion, the simulation is recorded into the archive. The user is able to stage the properties of a recorded simulation back into the job submission interface.
A Cactus simulation can be set up to communicate to the portal via the XMLRPC mechanism whether it is launch inside the portal or on the command line. It communicates information designated at run time such as the HTTP port where the simulation is running, the directory where the data files are stored, the current iteration number, and the time left to completion. Cactus simulations can also invoke a deannounce method to remove themselves from the portal database.
The Archive page of the simulations section displays internally and externally submitted simulations as mentioned above. The external announced simulation are divided into two sections: the simulations that announced to the portal in the last 24 hours, and previously announced simulations. The display of the present day's simulations is a helpful collaborative tool, as other group members can readily monitor active simulations.
Cactus ParametersThis utility is really a general text file importer/exporter; A complete listing of the remote file system similar to the one in the Utilities/FTP interface is displayed.
The portal keeps a permanent collection of parameter files for each user. One can import a text file from a grid resource into this collection on the portal, where it can be edited and then exported to the same or to a different grid resource. There are numerous functions implemented to allow the user to manipulate the file in the local repository. One can modify the file in the portal database, rename it, create a new local file, delete entries from the database, attach a short description of a parameter file and then export it to the desired location on a remote host. The user's local collection of parameter files is maintained across portal sessions.
An image displaying the parameter file interface can be found here
The Visualisation section provides a a general framework for starting up applications for the graphical display of data sets. By pressing a button one can launch any visualisation program present on the remote host. The user accomplishes this by constructing the command line to be executed on on the remote machine.
To help the user build the command line the interface contains a file listing interface and a text area of visualization notes. The file listing interface displays any readable directories or files on the remote machine. The user can find present visualization programs and data files. A full unix file description (complete file name, file size, date last modified) is displayed when the file/directory is selected. The file listing can ordered by the filename extensions. One is also able to keep "Visualization Notes" in a text area on the viz page. The default visualization notes contain a list with the location of visualization tools (Vision2.0, xgraph, gnuplot, etc.) on the various supercomputers. This interface assumes an X environment for the remote display. An example diplay of Vision2.0 is seen here
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}