Difference between revisions of "Broadband Platform Utilities"

From SCECpedia
Jump to navigationJump to search
 
Line 33: Line 33:
 
   -g GMPE_GROUP_NAME, --gmpe-group=GMPE_GROUP_NAME
 
   -g GMPE_GROUP_NAME, --gmpe-group=GMPE_GROUP_NAME
 
                         GMPE group: ['nga-west2', 'nga-west1', 'cena group 1']
 
                         GMPE group: ['nga-west2', 'nga-west1', 'cena group 1']
 +
 +
An example on Gordon:
 +
 +
  ./bbp_hpcc_validation.py -c GP -e Landers --no-hypo-rand -d /oasis/scratch/ux454496/temp_project/bbp_home/part_a/Landers -n 50 --email maechlin@usc.edu
  
 
Fabio also pointed at the setup_bbp_env.sh script. This will configure the compute node run-time environment. This needs to be edited away from HPC to Gordon. He made special reference to the $TMP_DIR/BBP_DATA_DIR definition at two places in this script. These are used at HPC to point to a on-compute node disk. Once results are written there, they are moved to another spot for storage. We should be able to point this to solid-state disk on Gordon, rather than compute node disk storage on HPC.
 
Fabio also pointed at the setup_bbp_env.sh script. This will configure the compute node run-time environment. This needs to be edited away from HPC to Gordon. He made special reference to the $TMP_DIR/BBP_DATA_DIR definition at two places in this script. These are used at HPC to point to a on-compute node disk. Once results are written there, they are moved to another spot for storage. We should be able to point this to solid-state disk on Gordon, rather than compute node disk storage on HPC.

Latest revision as of 18:39, 28 June 2015

Notes configuring BBP utilities to run on SDSC Gordon.

Motivation

SDSC Gordon has solid state disks. These may speed up BBP processing, because BBP uses many small files. Initial tests, running the UnitTests and Acceptance Tests suggest a speedup of 1.8x, which is enough it make this worthwhile investigating further.

Approach

We have information how long it takes to run BBP on SCEC HPC cluster. We will reconfigure to run on Gordon, and will compare time-to-solution using Gordon against use of HPC.

Guidance

Fabio pointed at bbp_hpcc_validation.py - This script will create a series of PBS files, that contain appropriate PBS parameters (e.g. number of nodes), and path names. These should be changed to point to directories on Gordon.

[ux454496@gordon-ln2 batch]$ ./bbp_hpcc_validation.py -h
Usage: bbp_hpcc_validation.py [options]
Options:
 -h, --help            show this help message and exit
 -c CODEBASE, --codebase=CODEBASE
                       Codebase for the simulation: ['gp', 'ucsb', 'sdsu',
                       'exsim', 'csm']
 -e EVENT, --event=EVENT
                       Validation event (should be configured in BBP)
 -d SIMDIR, --dir=SIMDIR
                       Simulation directory
 --skip-rupgen         Skip the rupture generator, run only 1 simulation
 --hypo-rand           Enables hypocenter randomization
 --no-hypo-rand        Disables hypocenter randomization
 -n NUMSIM, --num-simulations=NUMSIM
                       Number of simulations to run
 --email=EMAIL         Email for job notifications
 --new-nodes           Schedule the job in the new HPCC nodes
 --save-tmpdata        Save the contents of the tmpdata directory
 --only-rup            Only runs the rupture generator
 -g GMPE_GROUP_NAME, --gmpe-group=GMPE_GROUP_NAME
                       GMPE group: ['nga-west2', 'nga-west1', 'cena group 1']

An example on Gordon:

 ./bbp_hpcc_validation.py -c GP -e Landers --no-hypo-rand -d /oasis/scratch/ux454496/temp_project/bbp_home/part_a/Landers -n 50 --email maechlin@usc.edu

Fabio also pointed at the setup_bbp_env.sh script. This will configure the compute node run-time environment. This needs to be edited away from HPC to Gordon. He made special reference to the $TMP_DIR/BBP_DATA_DIR definition at two places in this script. These are used at HPC to point to a on-compute node disk. Once results are written there, they are moved to another spot for storage. We should be able to point this to solid-state disk on Gordon, rather than compute node disk storage on HPC.

SDSC Gordon

Related Entries