SCEC BBP Phase 1 Evaluation
August 2, 2013
Review Panel: Douglas S. Dreger (Chair, UC Berkeley), Gregory C. Beroza (Stanford), Steven M. Day (SDSU), Christine A. Goulet (UC Berkeley), Thomas H. Jordan (USC), Paul A. Spudich (USGS), and Jonathan P. Stewart (UCLA)
Executive Summary
The first validation phase of the SCEC Broadband Platform (BBP) was evaluated for the suitability of simulated pseudo-spectral acceleration (PSA) for use in engineering applications. The focus at this stage investigates the centering of the simulation methods by comparing the mean ground motion estimates. Future phases will address the dispersion of estimates or in terms of standard deviations (often referred to as “sigmas”). Five simulation methods were evaluated, and there were two parts to the validation.
In the first, Part A, the methods were studied using the bias of simulation results with respect to observations for seven specific events. Bias is defined as the natural logarithm of the ratio of the observed PSA to that for each simulation method. The term goodness-of-fit (GOF) is used interchangeably to represent this bias in the current project. A suite of 63 periods were used to define the PSA from 0.01 to 10s. For each event 40 stations providing good azimuthal coverage of the source and providing coverage within 200 km (1 to 193 km) of closest rupture distance range were used in the validation. Part A examined the mean bias for the events, individually and collectively. Assessment criteria included (1) performance relative to thresholds of 0.5 and 0.35 natural log units, (2) a measure of distance-period dependence of the bias, and (3) performance of the methods compared to published ground motion prediction equations (GMPEs). The results of this analysis show that three methods, EXSIM, G&P and SDSU, are suitable for broadband simulation of PSA from 0.01 to 3 seconds period over the entire distance range.
The second validation test, Part B, considered a comparison with published GMPEs of simulated PSA at two distances, 20 and 50 km for MW6.2 and 6.6 strike-slip, and reverse-slip scenarios. The Part B acceptance criterion permits simulation means to deviate only up to a preset amount from the mean of the NGS-West 2 GMPEs (the permissible deviation being scaled by the maximum positive and negative deviations of the GMPEs from their mean). While all methods satisfied the evaluation criteria for at least one of the cases, only EXSIM, G&P and SDSU satisfy the acceptance criteria for all of the cases.
Based on the Part A and Part B validation tests the EXSIM, G&P and SDSU methods are considered suitable for broadband simulation of median PSA from 0.01 to 3 seconds period within the validation magnitude range; they are suitable up to MW8 for the purposes of assessing relative effects of changes in source geometry, rupture direction, presence of secondary slip on splays, hanging wall effects, etc. We note that for periods above 1 second there is increased bias relative to recordings, and above 3 seconds period there are significant deviations from GMPEs. Further analysis will be required to understand the source of this additional bias, but possible contributors include the specifics of the moment-area scaling relationships used in source generation for the simulations and possible over excitation of surface waves from the employed 1D velocity models. In addition, above 3 seconds period the uncertainties in the GMPE estimates themselves may be comparable to the differences between the simulations and GMPEs. Future work should investigate systematic differences in simulated motions from 1D vs. 3D velocity structures.
It should be recognized that when the simulations are applied at magnitudes beyond the validation range, the results have additional epistemic uncertainty that has not been formally investigated. While formal validations of the methods are not currently possible for magnitudes greater than 8, their use at higher magnitude will require better understanding of epistemic uncertainties related to various model inputs including e.g., scaling and effects of stress parameters, subfault stress drop, parameterization of slip velocity function, and the degree of occurrence of super shear rupture velocity.
The concept and implementation of the BBP was to have well defined, version controlled methodologies and common post-processing and analysis tools. This approach successfully provides a framework for the evaluation of simulated ground motions for engineering applications that use PSA. All of the five methods currently implemented on the BBP should continue to be refined and improved to provide a variety of options for users, and to help to characterize epistemic uncertainty. Future development of the BBP should also evaluate the ability of simulations to represent observed PSA sigma values, and should include additional metrics to evaluate the suitability of simulated time histories for engineering applications.
Complete Report
Click to download full report (8.19MB PDF).
Appendix A. Project overview document (Goulet, 2013)
Appendix B. Excel spreadsheet for Tables 3.1.2 and 3.1.3
Appendix C. Excel spreadsheet and Matlab program for interpretation of distance (Table 3.2.1)
Appendix D. Excel spreadsheet for combined mean bias, mean absolute bias metric and comparison with GMPE
Appendix E. Modeler self-assessment documentation
Additional References
Recent Advances in Broadband Ground Motions Simulation Methods and Their Validation, Seismological Research Letters Focus Issue, anticipated publication date January/February 2015. Articles include:
- The SCEC Broadband Platform validation exercise for pseudo-spectral acceleration: methodology for code validation in the context of seismic hazard analyses, by Christine A. Goulet, Norman A. Abrahamson, Paul G. Somerville, Kathryn E. Wooddell
- Validation of the SCEC Broadband Platform V14.3 Simulation Methods Using Pseudo Spectral Acceleration, by Douglas Dreger, Gregory Beroza, Steven Day, Christine Goulet, Thomas Jordan, Paul Spudich, and Jonathan Stewart
- SCEC Broadband Platform: System Architecture and Software Implementation, by Philip J. Maechling, Fabio Silva, Scott Callaghan, Thomas H. Jordan
- The Composite Source Model for Broadband Simulations of Strong Ground Motions, by John G. Anderson
- Implementation and validation of EXSIM (a stochastic finite-fault ground-motion simulation algorithm) on the SCEC broadband platform, by Gail M. Atkinson and Karen Assatourians
- Refinements to the Graves and Pitarka (2010) Broadband Ground Motion Simulation Method, by Robert Graves and Arben Pitarka
- The SDSU Broadband Ground Motion Generation Module BBtoolbox Version 1.5, by Kim Olsen and Rumi Takedatsu
- UCSB Method for Broadband Ground Motion from Kinematic Simulations of Earthquakes, by Jorge Crempien and Ralph J. Archuleta
