ASCR Monthly Computing News Report - June 2009

In this issue...


2009 SciDAC Conference Draws More Than 400 Registered Participants

Between June 14–18, about 400 of the leading experts in computational science convened in San Diego for the Scientific Discovery through Advanced Computing (SciDAC) 2009 conference for four days of invited talks, technical presentations and posters, and a special evening devoted to scientific visualization. Kicking off the conference program on Monday, June 15, was a keynote address by Dr. Ray Orbach, the former DOE Under Secretary of Science, who was a strong advocate for scientific computing during his tenure. The conference program featured more than 50 talks by scientists from DOE national labs and other research institutions in the U.S., Japan and Germany, as well as a joint plenary session and poster session with the concurrent Department of Defense Users Group Conference.

On Monday, June 15, 29 scientific visualizations vied to win one of 10 OASCR Awards, chosen by popular vote (see next item).

Full details of the 2009 SciDAC Conference can be found at: https://hpcrd.lbl.gov/scidac09/index.html

Second Annual OASCRs Awards Honor Top Scientific Visualizations

Ten scientific visualizations were chosen to receive OASCR awards (named for DOE’s Office of Advanced Scientific Computing Research) at the Electronic Poster and Visualization Night session of the 2009 SciDAC Conference. The conference was held June 14–18 in San Diego, Calif. The top 10 visualizations were chosen by popular vote from a total of 29 submissions. As the visualizations played on large screens around the room, about 200 attendees marked their ballots. The winning submissions received gold-colored statuettes sponsored by Data Direct Networks.

Members of two SciDAC-funded projects — the Visualization and Analytics Center for Enabling Technologies (VACET) and the Institute for Ultrascale Visualization (IUSV) — won five of the ten awards, with three going to VACET and two to IUSV.

The wining visualizations (in alphabetical order) are
  • The Big One, a simulated 7.8 earthquake in Southern California, produced by Amit Chourasia, Kim Olsen, Steven Day, Luis Dalguer, Yifeng Cui, Jing Zhu, David Okaya, Phil Maechling and Thomas H. Jordan.
  • Five Years of the Breaking Waves Simulation, a compilation showing the evolution of the “Breaking Waves” simulation, produced by Douglas Dommermuth, Thomas O’Shea, Paul Adams and Randall Hand.
  • GEOS-5 Seasonal CO2 Flux, the Goddard Earth Observing System Model v. 5 showing seasonal CO2 buildup and reduction in North America, produced by Jamison Daniel (VACET) and David Erickson.
  • ImageVis 3D, a new volume-rendering program developed by the NIH/NCRR Center for Integrative Biomedical Computing, produced by Jens Kruger and Tom Fogal (VACET).
  • Impact of a Copper Bullet on Six Layers of Harness Satin Weave Kevlar Fabric, produced by Eric Fahrenthold, Moss Chimek, Kwon Joong Son, April Bohannan, Randall Hand and Kevin George.
  • A Lifted Ethylene-Air Jet Flame Stabilized by the Interaction between a Fuel Jet and the Surrounding Preheated Air, produced by Jacqueline H. Chen, Kwan-Liu Ma (IUSV), Hongfeng Yu, Ray W. Grout, Chaoli Wang, Chun Sang Yoo, Edward Richardson and Ramanan Sankaran.
  • Simulation of Non-Newtonian Suspensions: Shear Thinning Case, which shows how suspensions such as concrete or paint react as strain is applied, produced by William George, Nicos Martys, Steven Satterfield, John Hagedorn, Marc Olano and Judith Terrill.
  • Simulation of the Gravitationally Confined Detonation Model of Type Ia Supernovae for Ignition at Multiple Points, produced by Brad Gallagher, George Jordan, Dean Townsley, Robert Fisher, Nathan Hearn, Jim Truan and Don Lamb.
  • Turbulent Flow of Coolant in an Advanced Recycling Nuclear Reactor, produced by Hank Childs (VACET), Paul Fischer, Aleks Obabko, Dave Pointer and Andrew Siegel.
  • Visualization of Electron-Scale Turbulence in Strongly Shaped Fusion Plasma, produced by Chris Ho, Chad Jones, Kwan-Liu Ma (IUSV), and Stephane Ethier.

The Electronic Visualization and Poster Night was organized by Hank Childs of Lawrence Berkeley National Laboratory (LBNL) and Valerio Pascucci of the University of Utah, both members of VACET.

Visualization Highlight: Climate Visualization Team Wins OASCR Award for ORNL

“GEOS-5 Seasonal CO2 Flux,” a climate visualization by two scientific computing researchers at Oak Ridge National Laboratory (ORNL), has received an OASCR (Office of Advanced Scientific Computing Research) at the annual SciDAC 09 conference in San Francisco this week. Jamison R. Daniel of the Scientific Computing Group at the National Center for Computational Sciences and David J. Erickson III of the Computational Earth Sciences Group in the Computer Science and Mathematics Division received the award for their visualization, which describes the seasonal flux of CO2 in the atmosphere over North America.

“The visualization illustrates the seasonal atmospheric CO2 boundary fluxes in the NASA GEOS-5 climate simulation,” Daniel explained. The simulations, which are currently running on the ORNL’s Jaguar, will make it possible to resolve down to regional scale the predictions and projections of climate change, using global models that will contribute to the 2011 United Nations Intergovernmental Panel on Climate Change (IPCC) 5th Assessment Report. Work on the visualization was completed in December 2008 and was submitted to SciDAC in May.

Visualization Highlight: Argonne Research Team Wins OASCR Award

Members of Argonne National Laboratory’s SHARP team won an OASCR at the SciDAC 2009 conference for the movie “Turbulent Flow of Coolant in an Advanced Recycling Nuclear Reactor.” The SHARP project is a collaboration between the Nuclear Engineering (NE) and the Mathematics and Computer Science (MCS) divisions that is developing high-accuracy simulation tools for reactors. Both the visualization and the runs for the winning entry were done in the Argonne Leadership Computing Facility. The visualization was performed by Hank Childs (then at Lawrence Livermore National Laboratory, now at LBNL) on Eureka, one of the world’s largest graphics processing units, which provides more than 111 teraflops and more than 3.2 terabytes of RAM. The simulations were a joint effort involving Aleks Obabko and David Pointer (NE) and Paul Fischer (MCS) and were carried out on the IBM Blue Gene/P Intrepid, one of the world’s fastest computers. The SHARP team’s effort, led by Andrew Siegel of MCS, is supported by a grant of 7.5 million hours on the Intrepid, from the U.S. Department of Energy’s Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program, to conduct detailed numerical experiments of thermal striping in sodium-cooled fast reactors. The project’s results will help scientists better understand the physics of jet mixing in reactor vessels, leading to more optimal designs for future facilities.



Rice University Wins PLDI Distinguished Paper Award

Rice University researchers received the Distinguished Paper Award on June 18, 2009 at the ACM SIGPLAN 2009 Conference on Programming Language Design and Implementation (PLDI) held in Dublin, Ireland for their paper entitled "Binary Analysis for Measurement and Attribution of Program Performance". This paper describes the approach used by the HPCToolkit performance tools to perform call path profiling of fully-optimized applications.. At runtime, HPCToolkit uses on-the-fly analysis of an executable's machine code to determine how to unwind the call stack to attribute costs to the full calling contexts in which they are incurred. To help correlate measurements to an application's source code in a useful fashion, HPCToolkit analyzes an application's machine code and symbol information to recover information about loop nests and inlined code.

This work is part of a broader effort to develop performance tools that enable precise measurement and analysis of executions of hybrid parallel codes. A forthcoming paper in SC09 entitled "Diagnosing Performance Bottlenecks in Emerging Petascale Applications" describes how this work applies to codes running on the DOE Leadership computing platforms.

HPCToolkit is being developed with support from the DOE Office of Science and is a principal focus of Rice University's research as part of the SciDAC Center for Scalable Application Development Software and the SciDAC Performance Engineering Research Institute.

Contact: John Mellor-Crummey, johnmc@cs.rice.edu
OLCF’s Jaguar Models Tomorrow’s Transistors

A simulation of electrical current moving through a futuristic electronic transistor has been modeled atom-by-atom in less than 15 minutes by Purdue University researchers. The work demonstrates that future electronic devices can be quickly simulated on advanced computers, opening the door to new nanoscale semiconductor components that are more powerful and use less energy. The simulation was run on the Oak Ridge Leadership Computing Facility’s (OLCF’s) Jaguar supercomputer, the world’s second fastest and one of just two computers capable of petascale performance. The modeling of the transistor ran on more than 147,000 computer processors simultaneously, according to Gerhard Klimeck, professor of electrical and computer engineering and director for the National Science Foundation-funded Network for Computational Nanotechnology.

“If this had run on a single-processor computer, it would have taken us 3.3 years to complete,” Klimeck said. “This is the first time we’ve been able to do an atomic-level simulation of a transistor within the realm of engineering instead of as a once-in-a-lifetime computer run.”

Jaguar ranks second on the semiannual Top500.org list of supercomputers and was built using 182,000 AMD Opteron processors.

For more information see this link: http://news.uns.purdue.edu/x/2009a/090617KlimeckJaguar.html

LBNL’s Wehner Contributes to Major Report Describing Climate Change Impacts on U.S.

Two researchers at Lawrence Berkeley National Laboratory, Michael Wehner and Evan Mills, contributed to the analysis of the effects of climate change on all regions of the United States, described in a major report released June 16 by the multi-agency U.S. Global Change Research Program. “Global Climate Change Impacts in the United States” covers such effects as changes in rainfall patterns, drought, wildfire, Atlantic hurricanes, and effects on food production, fish stocks and other wildlife, energy, agriculture, water supplies, and coastal communities.

Wehner, who is a climate researcher in the Scientific Computing Group of Berkeley Lab’s Computational Research Division, developed projections of future climate change for the report chapters covering global and national impacts of climate change. One of Wehner’s research interests is extreme weather conditions resulting from climate change. The report addresses nine zones of the United States (Southwest, Northwest, Great Plains, Midwest, Southeast, Northeast, Alaska, U.S. islands, and coasts), and describes potential climate change effects in each.

Read more at this link
NERSC Helps Discover Cosmic Transients

An innovative new sky survey, called the Palomar Transient Factory (PTF), will utilize the unique tools and services offered by the Department of Energy’s (DOE) National Energy Research Scientific Computing Center (NERSC) at LBNL to expose relatively rare and fleeting cosmic events, like supernovae and gamma ray bursts. In fact, during the commissioning phase alone, the survey has already uncovered more than 40 supernovae, or stellar explosions, and astronomers expect to discover thousands more each year. Such events occur about once a century in our own Milky Way galaxy and are visible for only a few months.

“This survey is a trailblazer in many ways,” says Shrinivas Kulkarni, who is professor of astronomy and planetary science at the California Institute of Technology (Caltech), director of Caltech Optical Observatories, and principal investigator of PTF. “It is the first project dedicated solely to finding transient events, and as part of this mission we’ve worked with NERSC to develop an automated system that will sift through terabytes of astronomical data every night to find interesting events, and have secured time on some of the world’s most powerful ground-based telescopes to conduct immediate follow up observations as events are identified.”

Peter Nugent, a staff scientist in LBNL’s Computational Research Division and the NERSC Analytics Group, is also the Real-Time Transient Detection Lead for the PTF project. Every night the PTF camera — a 100-megapixel machine mounted on the 48-inch Samuel Oschin Telescope at Palomar Observatory in Southern California — will automatically snap pictures of the sky, then send those images to NERSC for archiving via high-speed networks provided by the DOE’s Energy Sciences Network (ESnet) and the National Science Foundation’s High Performance Wireless Research and Education Network (HPWREN).

Read more at this link
LLNL, Sandia Researchers Present at Scientific Computations Meeting in Bulgaria

Researchers from Lawrence Livermore and Sandia national laboratories presented their research at the 7th International Conference on Large-Scale Scientific Computations, held in Sozopol, Bulgaria, June 4–8, 2009. This meeting was sponsored by the Institute for Parallel Processing at the Bulgarian Academy of Sciences in celebration of the 60th birthday of Richard Ewing.

LLNL researcher Panayot Vassilevski gave an invited presentation on his ASCR-funded applied math resesarch “Operator-Dependent Discretization Spaces by Constrained Energy Minimization.” Dr. Vassilevski also organized a special session on discretizations and fast solution techniques for large-scale physics applications with collaborator Ludmil Zikatanov (Penn State) whose speakers included Rob Falgout (LLNL), Clemens Hofreither (Johannes Kepler University), and Ivan Graham (University of Bath).

Sandians P. Bochev, D. Ridzal and C. Baker also participated in the conference. Bochev presented a talk on compatible least-squares methods for the Stokes equations; Ridzal spoke about robust algorithms via an optimal control reformulation; and Baker discussed optimization and eigenvalue problems. Bochev and Ridzal served as members of the Scientific Committee of the conference and organized a special session on “Unconventional Uses of Optimization in Scientific Computing.” In addition, Bochev and T. Manteuffel (CU Boulder) organized a special session on “Least-Squares Finite Element Methods.” Approximately 120 people attended the conference from the EU, USA and other countries.

For more information, select this link
Article by LBNL, ORNL Researchers to Appear in Journal of Computational Physics

The article “Numerical simulation of non-viscous liquid pinch off using a coupled level set-boundary integral method,” by Maria Garzon and James Sethian of LBNL and Leonard Gray of ORNL, has been accepted by the Journal of Computational Physics. Liquid drop formation has been widely studied for many years due to its fascinating nature and its importance in various technical and industrial fields, such as inkjet printing, sprays and electrosprays, e.g., for mass spectrometry applications or the painting of automobiles. There are two significant challenges in the numerical modeling of these free boundary problems: the domain can undergo topological changes, and typically the evolution of the free surface boundary condition must be obtained from the solution of a differential equation posed on the moving boundary.

In this work, the researchers have further developed advanced Level Set techniques for solving this type of moving boundary problem, and have applied them to simulate the breakup of an inviscid fluid column (Rayleigh-Taylor instability). The fluid model is potential flow, and the free surface boundary condition is the Bernoulli equation for the balance of surface tension forces. The potential flow solution is obtained from a 3D axisymmetric Galerkin boundary integral calculation, and the surface velocities are computed using a new fast gradient algorithm. The Level Set method is employed for the difficult tasks of solving the Bernoulli equation and advancing the fluid free surface in time. The computational techniques are capable of following the fluid evolution through pinch off, and as well through the secondary pinch-off events that follow the initial separation of the fluid. The numerical results are in excellent agreement with known scaling laws at breakup.

Professor Maria Garzon is also at the University of Oviedo, Oviedo Spain. The Level Set (LBNL) and boundary integral (ORNL) projects have been, for many years, major components of the OASCR funded applied mathematics research at these laboratories.

New Capabilities Developed to Simulate Blood Flow through Inferior Vena Cava Filters

LLNL researchers Michael Singer and Bill Henshaw, along with Santa Clara Medical Center doctor Stephen Wang, have developed new simulation capabilities to study the hemodynamics of the TrapEase vena cava filter to identify areas of stagnant and/or recirculating flow that may have an effect on intrafilter thrombosis. The computational fluid dynamics simulation uses the ASCR-funded Overture software to model steady state flow for various thrombi shapes and sizes for unoccluded and partially occluded filters.

The modeling effort offers advantages over in vitro techniques, specifically improved resolution and easy adaptation for new filter designs, thrombus morphologies, and flow parameters. The results of the study agreed with those of previous bench experiments and are supported by clinical studies. The article was recently published in the Journal of Vascular and Interventional Radiology.

More information can be found at this link


Sandia’s Tammy Kolda Receives ACM Honor

Tammy Kolda has been elected a Senior Member of the Association for Computing Machinery (ACM). Kolda is a Principal Member of Technical Staff in Computer Sciences and Information Systems Center at Sandia National Laboratories. Kolda’s work in global optimization methods is supported by ASCR’s Applied Math Research program.

LBNL Fellow Receives Outstanding Dissertation Award

Maciej Haranczyk, a Seaborg Fellow in the Scientific Computing Group since September 2008, received the 2008 Most Outstanding Doctoral Dissertation Award in the Department of Chemistry during a ceremony at the University of Gdansk on June 15, 2009. The award was decided by a three-party committee, the members of which were affiliated with the Polish Chemical Society, University Gdansk, and Technical University of Gdansk. Haranczyk finished his doctoral thesis in summer 2008 under the supervision of Professor Maciej Gutowski. The title of his thesis was “Algorithms and Software Tools for Exploration of Chemical Spaces, Charge Distributions, and Solvation Effects. Applications thereof to DNA Fragments.”

Diachin, Meza Named to Panel on Digitization and Communications Science

Lori Freitag Diachin of the Center for Applied Scientific Computing at Lawrence Livermore National Laboratory, and Juan C. Meza, head of the High Performance Computing Research Department at Lawrence Berkeley National Laboratory, have been named to the National Academies Panel on Digitization and Communications Science for two-year terms. The panel will conduct a series of meetings to review an area of research as determined by the National Research Council.

Argonne’s Rusty Lusk Gives Keynote Address at University of Notre Dame Workshop

Ewing “Rusty” Lusk, Director of the Mathematics and Computer Science Division and Distinguished Fellow at Argonne National Laboratory, gave the keynote address at the Center for Research Computing Workshop on Scientific Computing, April 30. In his talk, titled “Programming Models for High-Performance Computing,” Lusk provided historical context about high-performance computing, discussed the current state, and explored possible new future directions. Lusk’s talk opened the workshop, which is an annual event designed for University of Notre Dame graduate students, postdoctoral students, and faculty who use the Center facilities.

More about the workshop is available at this link
LBNL’s Andrew Canning Gives Invited Talk at SCINT 2009 in Korea

Andrew Canning of LBNL’s Scientific Computing Group gave an invited talk on Wednesday, June 10, at the 10th International Conference on Inorganic Scintillators and Their Applications in Jeju, Korea. His topic, “First Principles Studies and Predictions for Ce and Eu Activated Scintillators,” is part of a larger project, “High-throughput discovery of improved scintillation materials,” which aims to synthesize and characterize new scintillation materials in microcrystal form and select candidates for crystal growth.

Contact: Andrew Canning, acanning@lbl.gov
Juan Meza Elected Treasurer of the Mathematical Programming Society

Juan Meza, head of LBNL’s High Performance Computing Research Department, has been elected to a three-year term as treasurer of the Mathematical Programming Society (MPS). The MPS is an international organization dedicated to the promotion and the maintenance of high professional standards in the subject of mathematical programming. It publishes the journals Mathematical Programming A and B, the MPS/SIAM Series on Optimization and the newsletter Optima. Every three years the Society sponsors the International Symposium on Mathematical Programming. Every other year it supports the Conference on Integer Programming and Combinatorial Optimization (IPCO).

Strohmaier and Yelick Give Presentations at ISC’09

Erich Strohmaier, leader of the Future Technologies Group in LBNL’s Computational Research Division, and NERSC Director Kathy Yelick were among the presenters at the 2009 International Supercomputing Conference (ISC’09) held June 23–26 in Hamburg, Germany. Over 1,500 attendees from over 45 countries attended.

Strohmaier was on the ISC’09 Scientific Program Committee and presented highlights of the 33rd TOP500 List, as well as a scientific session on “Generalized Utility Metrics for Supercomputers.” Yelick gave a talk on “Multicore/Manycore: What Can We Expect from the Software?” Yelick and Strohmaier also participated in a “Hot Seat Session” in which leading HPC vendors were grilled with tough questions.


SciDAC’s VACET Team Demonstrates Tools for Analyzing Massive Datasets

As computational scientists are confronted with increasingly massive datasets from supercomputing simulations and experiments, one of the biggest challenges is having the right tools to gain scientific insight from the data. A team of DOE researchers recently ran a series of experiments to determine whether VisIt, a leading scientific visualization application, is up to the challenge. Running on some of the world’s most powerful supercomputers, VisIt achieved unprecedented levels of performance in these highly parallel environments, tackling data sets far larger than scientists are currently producing.

The team ran VisIt using 8,000 to 32,000 processing cores to tackle datasets ranging from 500 billion to 2 trillion zones, or grid points. The project was a collaboration among leading visualization researchers from Lawrence Berkeley National Laboratory, Lawrence Livermore National Laboratory, and Oak Ridge National Laboratory.

Specifically, the team verified that VisIt could take advantage of the growing number of cores powering the world’s most advanced supercomputers, using them to tackle unprecedentedlylarge problems. Scientists confronted with massive datasets rely on data analysis and visualization software such as VisIt to “get the science out of the data,” as one researcher said. VisIt, a parallel visualization and analysis tool that won an R&D 100 award in 2005, was developed at LLNL for the National Nuclear Security Administration.

Read more at this link
ESnet Rolls Out IPv6 Network Management System

Although it has been a network protocol standard for more than 10 years, IPv6 (Internet Protocol version 6) has only been minimally implemented by the networking community. But that could change now that the U.S. Department of Energy’s Energy Sciences Network (ESnet) has deployed a production IPv6 management system across its entire network. By transitioning its network management system to IPv6, ESnet will both broaden the acceptance of IPv6 and gain hands-on experience in using the protocol to manage a national network.

Developed under the auspices of the Internet Engineering Task Force IPv6 working group, IPv6 was designed to dramatically increase the number of available Internet addresses, making it easier to assign addresses and route traffic across networks. Although available on nearly every computer system, IPv6 has only been deployed on less than 1 percent of the Internet-enabled machines around the world. ESnet will use the IPv6-based network monitoring system to continuously monitor the performance of its network. Typically, the monitoring system “polls” all of the network devices, such as routers and interfaces, at 30-second intervals. If a problem is detected, network operation staff are automatically alerted to the situation so it can be resolved.

Read more at this link
Sandia’s Tramonto Code Scales to 16K Cores on Jaguar

Tramonto is a real space matrix based code that solves the integral equations of certain implemented density functional theories. Fluids density functional theory (F-DFT) approaches have been used to study a wide range of physical systems. Some examples are: fluids at interfaces, surface forces, colloidal fluids, wetting, porous media, capillary condensation, interfacial phase transitions, nucleation phenomena, freezing, self-assembly, lipid bilayers, ion channel proteins, solvation of surfaces and molecules. The characteristic particle size in F-DFT models ranges from atoms (e.g., Argon) to colloidal particles, proteins, or cells. Thus these F-DFT approaches provide a multiscale framework for studying the physics of many complex fluid systems. Under the support of ASCR’s Applied Math Research program and with computer time from ASCR’s INCITE program, Mike Heroux, Laura Frink and Andy Salinger have developed the theory and algorithms that have enabled the scalable Tramonto solutions from 1,024 cores to 16,384 cores on the OLCF Jaguar system that advance the study of the behavior of anti-microbial peptides and other complex fluid environments.


Tenth ACTS Workshop Will Be Held August 18-21 at LBNL

The tenth annual Workshop on the DOE Advanced CompuTational Software (ACTS) Collection will be held August 18–21, 2009 at LBNL. This year’s workshop will be held in collaboration with the Berkeley CSE program at LBNL and the Center for Information Technology Research in the Interest of Society (CITRIS) at the University of California. The theme is “Leveraging the Development of Computational Science and Engineering Software through Sustainable High Performance Tools.”

The workshop will include a range of tutorials on the tools currently available in the ACTS Collection, discussion sessions aimed to solve specific computational needs of the workshop participants, and hands-on practice using NERSC’s state-of-the-art computers. The workshop is open to computational scientists from industry, academia, and national labs. Registration fees are fully sponsored by the DOE Office of Science. In addition, DOE will sponsor travel expenses and lodging for a limited number of graduate students and postdoctoral fellows.

For more information, go to this link
ORNL Hosts Lustre Workshop

Users, engineers, and developers converged on Oak Ridge National Laboratory May 19–20 for part two of the Lustre Scalability Workshop. Sponsored by ORNL, Sun Microsystems, and Cray, Inc., the workshop brought together representatives from the world’s largest Lustre deployments to identify key scalability issues and develop a roadmap for the future, namely bandwidth in the terabytes per second range and the manageability of exabytes of storage by 2015. In all, the conference hosted 30 attendees and was held at ORNL’s Joint Institute for Computational Sciences.

“The workshop provided a great opportunity for us to involve our most demanding users in setting the direction we will take with Lustre over the next several years. This is an important part of our ongoing commitment to meet the most demanding I/O and storage needs of the HPC community” said Peter Bojanic, director of Sun’s Lustre Group.