ASCR Monthly Computing News Report - September 2010



Berkeley Lab and Princeton Scientists Watch Stars Explode in 3D
Researchers from Princeton University and the Lawrence Berkeley National Laboratory have found a new way to make computer simulations of supernovae exploding in three dimensions. The new simulations are based on the idea that the collapsing star itself is not sphere-like, but distinctly asymmetrical and affected by a host of instabilities in the volatile mix surrounding its core. Writing in the Sept. 1 issue of the Astrophysical Journal, Jason Nordhaus and Adam Burrows of Princeton University, and the Lawrence Berkeley National Laboratory’s Ann Almgren and John Bell report that the new simulations are beginning to match the massive blow-outs astronomers have witnessed when gigantic stars die.
supernovae3DThe team performed these 3D simulations with approximately 4 million computer processor hours on the National Energy Research Scientific Computing Center’s (NERSC) Cray XT4 “Franklin” system. The simulations were run using a sophisticated computer code called CASTRO, the development of which was led by Almgren and Bell of Berkeley Lab’s Center for Computational Sciences and Engineering. The work is being done as part of the Computational Astrophysics Consortium supported by DOE’s SciDAC program.

The research was featured in the Sept. 24 issue of Time magazine
Link to the article
Read the Berkeley Lab news article
Interpreting IR Stretching Band of Water Improves Understanding of Hydrogen Bonding
In an INCITE research endeavor, University of California, Davis researchers have interpreted the complex shape of the infrared (IR) stretching band of neat, heavy water, using first principles molecular dynamics and ab-initio electronic structure calculations. Their interpretation of the IR stretching band of water is providing a better understanding of hydrogen bonding, which is critical to understanding hydrogen as an alternative fuel.
The researchers carried out calculations using the Qbox code on Intrepid, the IBM Blue Gene/P supercomputer at the Argonne Leadership Computing Facility (ALCF). The researchers showed that intermolecular dipolar correlations play a key role in determining the shape and width of the band, and that these correlations are long-ranged, extending to the second coordination shell. Both hydrogen-bonded and non-hydrogen-bonded molecules contribute to the IR stretching band over the entire frequency range, with no distinctive peak or shoulder associated with each species. Within a molecular orbital picture, the researchers identified specific features of the band arising from correlations of electronic contributions to the IR activity. Additional spectroscopic investigations carried out by the UCD team, in collaboration with researchers at Lawrence Livermore National Laboratory (LLNL) and MIT include the study of x-ray absorption spectra.
Contact: Giulia Galli (
SciDAC Institute Describes New Matrix Coloring Algorithm in SIAM Journal
Researchers at the CSCAPES (Combinatorial Scientific Computing and Petascale Simulations) SciDAC institute at Ohio State University, Purdue University and Sandia National Laboratories have developed scalable parallel algorithms for graph coloring of sparse matrices. Their work was recently published in SIAM J. Sci. Comp. 32:4. The algorithms provide an efficient way to compute Jacobians using either finite differences or automatic differentiation, where the data is distributed among processors. This is an important step in parallel nonlinear solvers. An implementation is available in the Zoltan and Trilinos algorithms toolkits.
Contact: Erik Boman,
Scalability on Blue Gene/P Improves Significantly Using New FMO Code
One thousand water molecules can now be treated at the MP2 level in under ten minutes. MP2 is an electron-correlated method that recovers the so-called “dispersion” interaction, a quantum effect important for hydrogen bonding — in turn, important for studying biological systems at the molecular level.
The GAMESS performance improvement benefits an Ames Laboratory INCITE project currently under way to predict bulk properties of water systems that require high fidelity. This molecular-scale challenge is of critical importance to national scientific issues such as global warming and the environment. The research is targeting very large systems whose sizes can only be addressed with massive computational resources such as Intrepid.
Contact: Graham Fletcher,
Supercomputers Assist Cleanup of Decades-Old Nuclear Waste
The Hanford Site in Washington—which produced fuel slugs for nuclear weapons, acted as a waste storage facility for nearly five decades, and was one of three primary locations for the Manhattan Project—is among the most contaminated nuclear waste grounds in the country. A research team led by Peter C. Lichtner of Los Alamos National Laboratory is using the Oak Ridge Leadership Computing Facility’s (OLCF’s) Jaguar supercomputer, located at Oak Ridge National Laboratory (ORNL), to build a three-dimensional model of an underground uranium waste plume at the Hanford Site’s 300 Area. A better understanding of the underground migration properties of uranium, which has infiltrated the Columbia River, may aid stakeholders in weighing options for contaminant remediation.
“The results could apply to other sites along the Columbia River that are contaminated too. And what we learn from this site we should be able to apply to other sites as well, not only at Hanford, but also around the country - at Oak Ridge and other areas dealing with contamination,” said Lichtner. The team performed massively parallel simulations of depleted uranium flow through soil using PFLOTRAN, a code developed under a SciDAC-2 project, which aims to advance computing at the petascale - or a quadrillion calculations per second. The code has been run on more than 130,000 processors of ORNL’s Jaguar Cray XT5 supercomputer to describe the flow of fluid through porous media, in this case the movement of soluble depleted uranium through a soil mixture of sand, gravel and fine-grained silts. The team simulated one year in only 11 hours by using more than 4,000 processors.


Berkeley Lab Mathematicians Chorin, Sethian Win Prestigious International Prizes
Berkeley Lab’s Alexandre Chorin and James Sethian won prestigious prizes from the International Council for Industrial and Applied Mathematics (ICIAM) for groundbreaking work in applied math, with impacts ranging from fluid mechanics and aerodynamics to medical imaging and semiconductor manufacturing. The ICIAM is composed of many of the national and international associations of professional mathematicians concerned with applications.
Chorin won the 2011 Lagrange Prize from ICIAM for his groundbreaking work in applied math. A member of the Berkeley Lab Mathematics Group and professor of Mathematics at UC Berkeley, Chorin was honored for “his fundamental and original contributions to applied mathematics, fluid mechanics, statistical mechanics, and turbulence modeling.” The Lagrange Prize provides international recognition to mathematicians who have made an exceptional contribution to applied mathematics throughout their career. Read more »
Sethian, leader of the Berkeley Lab Math Group, won the 2011 Pioneer Prize from ICIAM. Sethian, who is also a professor of mathematics at UC Berkeley, was honored “for his fundamental methods and algorithms that have had a large impact in imaging and shape recovery in medicine, geophysics and tomography, and drop dynamics in inkjets.” The Pioneer Prize recognizes pioneering work introducing applied mathematical methods and scientific computing techniques to an industrial problem area or a new scientific field. Read more at this link.
Read the LBNL news release
Argonne’s Todd Munson Named Senior Fellow of the Computation Institute
Todd Munson, a computational scientist in Argonne’s Mathematics and Computer Science Division, has been named a senior fellow of the University of Chicago/Argonne Computation Institute. Munson is a key part of the MCS Division’s optimization group. He has a leadership role in the Toolkit for Advanced Optimization (TAO) project; he is actively involved in a study of uncertainty quantification for equilibrium model forecasts for climate and smart grid technologies; and he is investigating border tax adjustments and their impacts on carbon markets. Munson (with his colleagues) also has been addressing so-called mixed-integer nonlinear programs, which are among the most challenging computational optimization problems faced by researchers; such problems arise, for example, in designing the thermal insulation system for the Large Hadron Collider. Munson joined Argonne in 2000 after receiving his Ph.D. in computer science from the University of Wisconsin at Madison. He was appointed a CI Fellow in 2005.
LBNL’s Michael Wehner Joins Congressional Briefing on Extreme Weather
Michael Wehner, a member of LBNL’s Computational Research Division who researches extreme weather conditions resulting from global climate change, was one of four panelists providing input during a Congressional briefing on “Extreme Weather in a Warming World.” The Select Committee on Energy Independence and Global Warming, chaired by Rep. Edward J. Markey (D-Mass.), held the briefing on Thursday, Sept. 23, in the Rayburn House Office Building in Washington, D.C. The most prominent speaker was Husain Haqqani, Pakistan’s Ambassador to the United States, who discussed the historic floods that have displaced millions of his countrymen.
Wehner spoke of his research and said that “Changes in the magnitude and frequency of extreme weather associated with changes in the average climate are likely the most serious consequence of human-induced global warming.” Among the likely effects are more frequent heat waves, intense precipitation, more severe droughts, and more frequent and severe tropical cyclones.
To learn more about the briefing and view a video of the session, click this link
Horst Simon Is Named Deputy Director of Berkeley Lab
Associate Laboratory Director for Computing Sciences Horst Simon, an internationally recognized expert in computer science and applied mathematics, has been named Deputy Director of Berkeley Lab. “Horst is a strong leader who has helped to lead a tremendously productive program in high performance computing that is world-class,” said Berkeley Lab Director Paul Alivisatos. “As Deputy Director he’ll help me lead major scientific initiatives, oversee strategic research investments, and maintain the intellectual vitality of Berkeley Lab.” Read more ».
Kathy Yelick has been named the new Associate Laboratory Director for Computing Sciences and will continue as NERSC Division Director.  Read more ».
Juan Meza will serve as Acting Director of the Computational Research Division while a search is conducted for a permanent director.
LBNL’s Yelick Delivers Keynote at International Conference on Parallel Processing
Kathy Yelick, director of the NERSC Division and recently appointed Berkeley Lab’s Associate Laboratory Director for Computing Sciences, delivered the Sept. 16 keynote address at the 39th International Conference on Parallel Processing (ICPP). Held Sept. 13-16 in San Diego, ICPP is sponsored by the International Association of Computing and Communication (IACC) and provides a forum for engineers and scientists in academia, industry, and government to present their latest research findings in any aspects of parallel and distributed computing. Yelick spoke on “Paving the Road to Exascale Computing.” Also at ICPP 2010, Khaled Ibrahim and Erich Strohmaier of Berkeley Lab’s Computational Research Division gave a presentation on “Characterizing the Relation between Apex-Map Synthetic Probes and Reuse Distance Distributions.”
Contact: Jon Bashor,
PNNL’s Pak Chung Wong to Chair IEEE VisWeek Conference in 2012
Pak Chung Wong of PNNL will be chairing the IEEE VisWeek Conference for 2012 to be held in the Pacific Northwest. IEEE VisWeek is the premier forum for visualization advances for academia, government, and industry. This event brings together researchers and practitioners with a shared interest in tools, techniques, and technology. Wong has led and performed ASCR-supported research and development on information technology and scientific computation at PNNL for over a decade. His research interests include visual analytics, predictive analytics, visualization, privacy and security, and social computing.


Say Hello to ADIOS 1.2 for Improved I/O on Supercomputers
Last week a team including the Oak Ridge Leadership Computing Facility’s Scott Klasky released ADIOS 1.2, the latest incarnation of one of computational science’s most effective I/O tools. So far ADIOS has helped researchers make huge strides in fusion, astrophysics, and combustion. The new version features some interesting improvements to help researchers in taking full advantage of leading supercomputing platforms.
For instance, users can now use the API directly to interactively construct new variables during run time. ADIOS also features a custom I/O method that writes data to subfiles and aggregates it into larger pieces for maximum performance on leadership-class systems. And now users who run on large systems can switch from running on P-processors and writing to P-files—or one file or M-files, transparently. Version 1.2 also features further support for self-describing data in the output. For example, users can automatically retrieve the average value, minimum, maximum, and standard deviation for all arrays at negligible computational cost. And finally, version 1.2 features some new asynchronous transport methods, allowing even faster I/O. “The focus for this release is broader compatibility and user convenience. The introduction of the API calls to replace the XML file addresses longstanding requests from a small but vocal part of our user community,” said team member Jay Lofstead. For more information on ADIOS and/or to download the source, please visit the following link:
OLCF Provides Fast, Easy Information through New Website
The Oak Ridge Leadership Computing Facility (OLCF), home to the world’s most powerful scientific supercomputer, has launched a new website geared to making the user experience as fast and easy as possible. Located at, the site has been designed to maximize value and ease of navigation for users of the Cray XT Jaguar supercomputer and other OLCF resources. It offers improved search, faster page loads, and a comprehensive knowledge base. “The rapidly growing volume of information available to our users is itself a serious challenge,” noted Ashley Barker, head of the facility’s User Assistance and Outreach Group. “In response, we have created both a searchable knowledge base of user support information and a browsable user guide for Jaguar, both of which will be updated regularly. In addition, we are encouraging users to help us by providing feedback through the website. We are confident this approach will make it easier for users to get the information they need and help them get the greatest possible scientific discovery with the least effort.”
Argonne Postdoc Applies Optimization to Energy Management of Buildings
Victor Zavala, a Director’s Postdoctoral Fellow in the Mathematics and Computer Science Division at Argonne National Laboratory, is working with BuildingIQ, a cutting-edge building energy management company, to develop next-generation energy management systems for buildings. The work leverages Zavala’s expertise in optimization modeling and algorithms to develop next-generation energy management systems for buildings. These systems will exploit building dynamic models, weather forecasts, occupant sensors, meeting room scheduling, and price structures to optimize the building conditions (e.g., temperature set-points, human comfort, CO2 emissions) in real time. The aim is to shift the energy demands in space and time around the building to save energy and costs. The initial focus is Argonne’s new Theory and Computing Sciences (TCS) building, which hosts around 700 occupants and several energy-intensive high-performance computing facilities that require large amounts of cooling throughout the year.


ALCF to Host Early Science Program Kick-Off Workshop
AThe Argonne Leadership Computing Facility (ALCF) is hosting an Early Science Program Kick-Off Workshop on October 18-19 at Argonne National Laboratory. The by-invitation-only workshop represents the first gathering of key players in this exciting endeavor to build the future of computational science. At the workshop:
  • IBM representatives will present a first look at the next-generation Blue Gene hardware and software, including compilers and messaging.
  • ALCF staff will provide hands-on help for those new to Blue Gene to facilitate the development efforts key to the ESP.
  • ESP project teams will map out and discuss plans for their first six months of development.
  • IBM, ALCF, and project representatives will present performance measurements.
  • The next-generation Blue Gene Tools, Libraries, and Programming Models project will present their efforts and participate in detailed planning sessions with IBM.
Contact: David Martin or
Chel Heinzel
LBNL’s Hank Childs Gives VisIt Tutorial to U.S. Army Research Lab
Hank Childs, a member of Berkeley Lab’s Visualization Group, gave a tutorial on VisIt, an open source visualization and analysis tool designed for processing large datasets, to researchers at the U.S. Army Research Laboratory in Aberdeen, MD, on Sept. 20. This marks the fourth tutorial Childs has presented in 2010. VisIt was originally developed at LLNL and won an R&D 100 Award in 2005. When DOE established the Visualization and Analytics Center for Enabling Technologies (VACET) led by LBNL in 2006, the center joined the VisIt development effort, making further extensions for use on the large, complex datasets emerging from the SciDAC program.