ASCR Monthly Computing News Report - February 2008

The monthly survey of computing news of interest to ASCR is compiled by Jon Bashor (JBashor@lbl.gov) with news provided by ASCR Program Managers and Argonne, Fermi, Lawrence Berkeley, Lawrence Livermore, Los Alamos, Oak Ridge, Pacific Northwest and Sandia National labs. Contact information and links to additional information, where available, are included with each article.

In this issue...


Improved LLNL Seismic Modeling Capabilities Used to Model Recent Earthquake
A team of applied mathematicians and seismologists led by Anders Petersson at Lawrence Livermore National Laboratory have created an earthquake simulation model as part of the Serpentine Wave Propagation project. The overall goal of the project is to look at the propagation of waves in nature, whether they be seismic, electromagnetic, or sound waves, all of which are governed by essentially the same mathematical equations. The team has spent several years developing new embedded boundary numerical techniques and encapsulating that mathematics research into an open source software simulation package called WPP.
The latest release of WPP was announced in late December 2007 and adds local mesh refinement and viscoelastic modeling to WPP’s suite of seismic wave propagation capabilities. The mesh refinement significantly enhances the computational efficiency by allowing the mesh size to follow the velocity structure of the earth. The viscoelastic capability adds advanced modeling of dissipative materials, allowing for a close to constant quality factor in a wide frequency band. This version of WPP was recently used to model the October 2007, magnitude 5.5 Alum Rock earthquake, which was the largest San Francisco Bay Area earthquake since 1989.
Contact: Lori Diachin, diachin2@llnl.gov
SciDAC Team’s New Method for Accelerating Molecular Dynamics Simulations
The main challenge in the SciDAC project on Hierarchical Petascale Simulation Framework for Stress Corrosion Cracking is developing a computational methodology that can simultaneously treat the vast range of scales in time (picoseconds to seconds and beyond) and length (angstroms to millimeters) necessary for accurately simulating the technologically critical process of stress corrosion cracking. As part of this multi-institution project (involving University of Southern California, Harvard, Purdue, California State University at Northridge, and Los Alamos and Lawrence Livermore national laboratories), researchers at Los Alamos National Laboratory are developing a method for accelerating molecular dynamics simulations at the solid-liquid interface.
In the parallel-replica dynamics method, time is parallelized to achieve longer simulations for infrequent-event processes, such as the diffusion of atoms on a surface, or, as is relevant to this project, the activated processes that advance a stress-loaded crack tip. Because stress corrosion cracking often involves a liquid phase in contact with the crack tip, the parallel-replica dynamics method is being extended so that it can be used to accelerate the dynamics at a solid-liquid interface. Initial results look promising for obtaining significant parallel speedup in time for this much more complex system, which heretofore was limited to time scales accessible to direct molecular dynamics.
Contact: Art Voter, afv@lanl.gov
Major New Capabilities Added to ROSE Performance Optimization Software
Recent work on ROSE, led by Dan Quinlan at LLNL, has added support for Fortran 2003, completing the long-term goal of providing uniform handling of all major languages used in DOE applications (including Fortran 2003, Fortran 90/95, Fortran 77, Fortran 66, C (C89 and C99), and C++). The Fortran 2003 work leveraged the Open Fortran Parser (OFP) developed at LANL, and has been supported through collaborations with LANL and Rice University. This new capability allows ROSE to provide custom analysis, source-to-source translation, domain-specific optimization and general optimization capabilities to a broader class of DOE applications than was previously possible. Additional major work on ROSE involves the development of new source-to-source capabilities that permit program analysis and transformation for automatically rewriting existing applications to take advantage of new architectures. In particular, ROSE now supports OpenMP as a target for high-level parallel transformations in applications to increase the overlap of computation and communication, shared and distributed memory parallel program analysis, alias analysis, global program analysis, and many specific optimizations and specialized loop optimizations that are available via library calls.
In addition, ROSE has recently added support for building Eclipse-based tools and is also connected to the Open Analysis framework (in collaboration with Colorado State and Cornell). Such capabilities are critical for the next generation of source-based tools that understand DOE application source code and can automatically manipulate and generate source code. ROSE is freely available and is distributed worldwide. ROSE is currently a major component of the PERI SciDAC institute and has been used for extensive work with DOE applications including MADNESS (ORNL), S3D (SNL), ALE3d (LLNL), KULL (LLNL), Ares (LLNL), and Chroma (UIUC). ROSE is also used for course work at Vienna Institute of Technology, Carnegie Mellon University, and University of Texas at San Antonio, as part of graduate classes ranging from program analysis to cyber-security.
Contact: Lori Diachin, diachin2@llnl.gov
New Release of ANL's Cobalt for HPC System Software Research
Argonne researchers will deploy a major new release this month of Cobalt, an open source platform for HPC system software research. Cobalt is a small but highly flexible package whose agility makes it an ideal research platform. Developed at Argonne, Cobalt enables rapid reconfiguration of components, permitting exploration of many interlinked system management issues. Cobalt also makes porting and adapting code to new platforms and system models relatively easy.
The most dramatic feature of the new release is Cobalt’s increased scalability — up to 500 teraflop/s on the IBM Blue Gene/P. This feature enables the Leadership Computing Facility applications team at Argonne to use the largest BG/P system effectively — with 40,960 quad-core compute nodes (163,840 processors) — for acceptance testing. The new Cobalt software system also includes a simulation component that minimizes system outages through offline testing. In addition, the new release includes some support for the Portable Operating System Interface (POSIX) resource management specifications.
Cobalt has been adopted by several major high-performance computing facilities since 2005, including NCAR, MIT, and Japan’s National Institute of Advanced Industrial Science and Technology.
Contact: Gail Pieper, pieper@mcs.anl.gov
LBNL Teams Best Paper Award - Again
Work by five Berkeley Lab researchers won the best paper award for the applications track at the annual IEEE International Parallel and Distributed Processing Symposium (IPDPS), which will take place in Miami this April. Samuel Williams, Jonathan Carter, Lenny Oliker, John Shalf and Kathy Yelick authored the paper "Lattice Boltzmann Simulation Optimization on Leading Multicore Platforms."  Carter, Oliker and Shalf also authored a paper that won the same best paper award at the 2007 symposium. The conference received 410 submissions for four technical tracks: algorithms, applications, architectures, and software.
Contact: Ucilia Wang, uwang@lbl.gov
Catamount Supports Cray XT4 Quad Core Opterons at Sandia
The XT4 Catamount Lightweight Kernel Risk Mitigation project reached a major milestone in January 2008, when Catamount, the lightweight kernel operating system for Cray XT3 systems, successfully booted on Quad Core Opteron chips used in the Cray XT4. During February, 10 different applications (including GTC, LSMS, POP, S3D, and VH1) have successfully been tested on the four quad-core Opterons available to Sandia. The intent of these initial runs was to verify functionality. Comparison runs with Compute Node Linux will be done in March.
Contact: Sue Kelly, smkelly@sandia.gov
HPC Resources Used to Show How Electrons Stimulate Acid-Base Reactions
Study results published in the February 15 issue of Science provided important information about the potential for researchers to precisely control chemistry in systems ranging from biology to energy technology. Researchers from the Pacific Northwest National Laboratory, the Heriot-Watt University in Edinburgh, UK, and Johns Hopkins University found that, in contrast to widely familiar acid-base behavior in solution, electrons spurred the chemical reaction between hydrogen chloride and ammonia when they reacted to form ammonium chloride. Their results revealed that supplying or removing an extra electron can make the reaction go from acid and base to neutral molecule or back again. Aided by computer programs (primarily the MOLPRO code) developed for high performance supercomputers at the DOE’s Environmental Molecular Sciences Laboratory (http://www.emsl.pnl.gov), the researchers developed information about the nature of chemical bonding and structure and correlated experimental with theoretical results. For more information see http://www.pnl.gov/news/release.asp?id=298.
NSF Grant Will Aid Supernova Research
A team led by National Center for Computational Sciences (NCCS) astrophysicist Bronson Messer will use a National Science Foundation (NSF) grant to prepare its supernova simulation tool for the era of petascale computing. The team, a collaboration among researchers from the University of Tennessee (UT), North Carolina State University, and Florida Atlantic University, has received an $800,000 NSF grant given through the agency's PetaApps program. The researchers will use the funding to upgrade the Chimera application for next-generation supercomputers that are orders of magnitude faster than today's most powerful systems. In particular, the program seeks important applications to take advantage of petascale supercomputers..
The researchers are using Chimera to work out details of the core-collapse supernova, the death of a star at least 10 times as massive as the sun. In particular, they are investigating how the shockwave created when the star's core collapses eventually blows most of the star into space. They have made important contributions to the field using terascale supercomputers, such as the first explanation for a pulsar's spin that matches observations from actual pulsars, which the group reported in Nature last year.
Contact: Jayson Hines, hinesjb@ornl.gov


ACM Honors Berkeley’s Vern Paxson for Network Measurement Research
The Association for Computing Machinery (ACM) has recognized Vern Paxson, a member of Berkeley Lab's Advanced Computing for Science Department, with the 2007 Grace Murray Hopper Award for his research on how to measure Internet behavior. His innovative techniques are used to assess new communications concepts, improve network performance, and prevent network intrusion. They provide both the research community and Internet operators with the tools to improve the operation of this increasingly diverse, decentralized communications infrastructure. Paxson is also a senior scientist with the International Computer Science Institute (ICSI) Center for Internet Research in Berkeley and an associate professor at UC Berkeley The award carries a $35,000 prize, and funding is provided by Google, Inc.
Paxson's research on Internet measurement brought the scientific process to the measurement of the Internet's behavior and the conditions under which it operates, raising the practice of Internet measurement to a higher level. As a result, the research community is able to evaluate new ideas and technologies and identify problems and priorities that are needed for increased efficiency. In addition, Internet operators are able to alleviate traffic congestion, detect attacks, and improve communications reliability.
LBNL Mathematician James Sethian Elected to National Academy of Engineering
James Sethian, head of the LBNL Mathematics Group and a professor of mathematics at UC Berkeley, has been elected to the National Academy of Engineering. Sethian was one of 65 new members and nine foreign associates whose election was announced on Feb. 8. Sethian was honored "for the development of efficient methods of tracking moving interfaces." Sethian's research has led to the development of level set methods, which are numerical techniques that can follow the evolution of interfaces, as well as a host of other techniques to track interfaces in various settings. These interfaces can develop sharp corners, break apart and merge together. The techniques have a wide range of applications, including problems in fluid mechanics, combustion, manufacturing of computer chips, computer animation, image processing, structure of snowflakes and the shape of soap bubbles.
Contact: Jon Bashor, jbashor@lbl.gov
Bair to Lead CELS Strategic Planning, Beckman New ANL LCF Project Director
Effective February 4th, Ray Bair, who established the Argonne Leadership Computing Facility (ALCF) in 2006 and initially acted as its director, joined Argonne's Computing, Environment and Life Science (CELS) directorate to lead the strategic planning effort at CELS. In his new position, Bair will assist with the development of the CELS components of Argonne's strategic plan and lab-wide computational science planning and program development.
Pete Beckman has been appointed project director of the ALCF. He is an internationally recognized expert in advanced computing, with an outstanding track record in managing major efforts both at national laboratories and in industry, who brings more than a decade of experience in large-scale computing and project management to this position. Beckman will be responsible for bringing the 500 teraflop/s BG/P through acceptance testing and into early science, bringing the 100 teraflop/s BG/P into INCITE production, upgrading the storage and I/O systems and transitioning the ALCF organization into steady-state operations. He will also recruit additional staff and users to the ALCF and chart a path for the future. For more information, Contact Ray Bair (rbair@anlmail.anl.gov ) or Pete Beckman (beckman@mcs.anl.gov)
Sandians Elected to SIAM/SIAG-SC Leadership
Michael Heroux was elected Chair and Cynthia Phillips the Program Director of the SIAM special interest group on Supercomputing (SIAG-SC). In addition to sponsoring the SIAM Conference on Parallel Processing for Scientific Computing, SIAG-SC promotes the discussion and dissemination of supercomputing research and development within the broader SIAM computing.
Contact: Scott Collis, sscoll@sandia.gov


New Vis Hardware Online Soon at ORNL
The NCCS is in the process of replacing its aging visualization cluster, a move that will dramatically boost the amount of data users can analyze at the center, the speed at which they can analyze it, and the quality of visualizations they can view. The Hawk system has been responsible both for data analysis and for powering the center's 240-square-foot EVEREST Powerwall, but it is nearing five years old, ancient for a leadership computing system. In the coming months it will be replaced by three separate clusters: Lens, a large-memory 32-node system that will be dedicated to data analysis; Everest, a separate cluster that will be dedicated to operating the 35-million-pixel Powerwall; and Orb, a state-of-the-art Lustre file system dedicated to serving data to the large Powerwall.
Contact: Jayson Hines, hinesjb@ornl.gov
NERSC, Swiss Computing Center Formalize Program to Exchange Expertise
The Swiss National Computing Centre (CSCS) and the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory have signed a memorandum of understanding for a staff exchange program between the two centers. The agreement gives more formal structure to already existing ties between the two centers. Berkeley Lab Associate Director for Computing Sciences Horst Simon is a member of the CSCS advisory board. Both centers also share a common technological focus, having selected Cray XT supercomputers as their primary systems after thorough reviews of various systems. Last year, a group from CSCS visited NERSC for a series of discussions about systems and facilities. Howard Walter, who oversees NERSC’s computational systems, paid a return visit in January 2008, sharing NERSC’s expertise in designing and building energy efficient computing facilities.
Contact: Jon Bashor, jbashor@lbl.gov


ORNL Researchers Team Up with the NFL for a Carbon-Neutral Super Bowl
Is it possible to have a carbon-neutral Super Bowl?  When National Football League Environmental Program Coordinator Jack Groh wanted to find out, he called Oak Ridge National Laboratory (ORNL) and teamed up with scientists David L. Greene and John Tomlinson to calculate how much carbon the sporting event is responsible for. Among the carbon-producing activities they assessed were lighting and heating the venue; NFL-organized fan events including exhibits, interviews, and games; and NFL-arranged transportation, such as buses and limos, to and from the University of Phoenix Stadium. The scientists concluded Super Bowl XLII would be responsible for the emission of about a million pounds of carbon.
NBC Nightly News came to ORNL on Jan. 28 to cover the story, interviewing Greene at the National Center for Computational Sciences (NCCS). Using the center's EVEREST Powerwall, a 30-foot-wide, 8-foot-tall display that can show 35 million pixels of information, visualization expert Sean Ahern demonstrated nighttime illumination in the Phoenix area and population distributions around the country associated with the Super Bowl's carbon footprint. Carbon's role in the world's climate is a key research focus of the NCCS, where this year more than 15 percent of computer resources will be dedicated to climate studies.
Contact: Jayson Hines, hinesjb@ornl.gov
Argonne's LCF Offers INCITE and BG/P Workshop Series
The Argonne Leadership Computing Facility (ALCF) is hosting several workshops in March at Argonne National Laboratory. The ALCF workshop series offers an opportunity to help accelerate usage of ALCF resources as part of the INCITE program and to interact with both ALCF and IBM staff.
An "INCITE Getting Started" workshop will be held on March 4-5. The workshop will provide INCITE program awardees with information on ALCF services and resources, technical details on the IBM Blue Gene/P architecture, as well as hands-on assistance in porting and tuning of their applications on the IBM BG/P. Attendance is restricted to awardees of new and renewed INCITE projects. For further details about the INCITE Getting Started workshop, contact Chel Lancaster (lancastr@alcf.anl.gov).
In addition, the ALCF and Blue Gene Consortium are jointly hosting a "Blue Gene/P Porting and Tuning" workshop for Blue Gene Consortium members on March 6. It is an abbreviated version of the two-day INCITE workshop. For more information about the Blue Gene/P Porting and Tuning workshop, contact Ed Jedlicka (jedlicka@mcs.anl.gov).
Annual NCCS Users Meeting, Cray XT Workshop Coming Soon to ORNL
At the National Center for Computational Sciences (NCCS) Users Meeting, to be held April 17–18, 2008, researchers will gather with NCCS staff and vendors to discuss challenges and solutions in areas such as porting and scaling of applications on the XT system. Each project new to the Department of Energy’s INCITE (Innovative and Novel Computational Impact on Theory and Experiment) program will be invited to give a 10-minute presentation on its upcoming work, while renewing INCITE projects will review their work at an evening poster session. NCCS staff will also give presentations on resources, capabilities and services available to the center’s users.
This year’s meeting will be preceded by a three-day “Introduction to Cray XT” workshop, held April 14–16 and sponsored jointly by the NCCS and the National Institute for Computational Sciences, home to the NSF’s newest supercomputer. Staff from the NCCS, ORNL’s Joint Institute for Computational Sciences, Cray, and chipmaker AMD will discuss XT issues and researchers attending the meeting will participate in hands-on sessions with the Cray XT system.
For more information, or to register, visit www.nccs.gov
Sandia Researcher to Lecture at International Summer School
Sandia National Laboratories researcher Rich Lehoucq has been invited to lecture SIAG/LA-SIMUMAT International Summer School on Numerical Linear Algebra (see http://www.simumat.es/SIAGLA2008). The school is the first in a series of periodic International Summer Schools on Numerical Linear Algebra (ISSNLA) organized by the SIAM Activity Group on Linear Algebra.
Contact: Rich Lehoucq, rblehou@sandia.gov
Juan Meza Participates in MLK Symposium at Michigan
Juan Meza, head of the High Performance Computing Department at LBNL, spoke at the Marjorie Lee Browne colloquium at the University of Michigan in January, taking part in a Martin Luther King Symposium. The colloquium honored Browne, the first African-American woman to earn a Ph.D. in mathematics from the University of Michigan, in 1950.
Meza, in a talk titled "I Want to Be a (Computational) Mathematician," gave his personal account of how he became interested in mathematics as a college undergrad and the research projects he has undertaken since then. The title is a reference to a book by Paul Halmos about his life as a mathematician and the history of the discipline from the 1930s to 1980s. In his talk, Meza spoke about the rise of the field of computational mathematics and its contributions to science. He gave examples of his recent work on developing new algorithms and using them in massively parallel computers to solve challenging problems.
Contact: Juan Meza, JCMeza@lbl.gov
Congressman Congratulates ORNL Scientists for IPCC, Nobel Effort
Congressman Zach Wamp recently returned home to his district and paid a visit to Oak Ridge National Laboratory, where he recognized five ORNL researchers for their contributions to climate science. Tom Wilbanks, David Greene, David Erickson, Paul Hanson, and Virginia Dale were each recognized by Wamp in a speech at the National Center for Computational Sciences. Their work, performed on an NCCS supercomputer dubbed Cheetah, was responsible for roughly one-third of the data submitted by the U.S., climate community to the U.N.'s Intergovernmental Panel on Climate Change, which shared the Nobel Prize with former Vice President Al Gore.
Contact: Jayson Hines, hinesjb@ornl.gov
Berkeley Lab's David Bailey Gives Pelz Memorial Talk at Rutgers
David Bailey, chief technologist of LBNL's Computational Research Division, gave a Richard B. Pelz Memorial Lecture at Rutgers University in New Jersey that highlighted research into boosting the ability of high-performance computers to improve their numeric calculations. Bailey's talk in January included a discussion about a paper he co-authored with Pelz on using 64-digit arithmetic for investigating vortex roll-up. Pelz, a professor of mechanical and aerospace engineering at Rutgers, specialized in computational fluid dynamics. Bailey's talk is part of an ongoing memorial project in honor of Pelz, who died in 2002.
Upcoming Workshop on Lustre File System at ORNL
The NCCS and Sun Microsystems will host a one-day workshop on April 16, 2008, focused on helping application scientists get the most from the Lustre File System. The workshop will be presented by Oleg Drokin and Wang Di, two file system engineers who are also Lustre developers with Sun. A morning session will present several case studies describing how application IO performance can be improved including ongoing enhancements to IO middleware libraries like MPI-IO and HDF5. The afternoon will have an open discussion about application IO with Lustre. This will offer users an opportunity to describe their applications to Lustre engineers and methods for improving performance. This also provides a forum for users to engage with Lustre developers to discuss future application IO needs and how Lustre can better address user requirements. For more information, or to register, visit www.nccs.gov.