ASCR Monthly Computing News Report - March 2011

In this issue:


Argonne Researcher Misun Min Recognized for Top Paper

A paper by Argonne National Laboratory assistant computer scientist Misun Min and her collaborator Taehun Lee at the City College of City University of New York, has been recognized as one of the “Top 25 Hottest Articles” in the Journal of Computational Physics from Oct. to Dec. 2010.

The paper, titled “A spectral-element discontinuous Galerkin lattice Boltzmann method for nearly incompressible flow,” presents a new method that involves decoupling the numerical scheme into collision and streaming steps. The new method provides increased numerical stability over traditional schemes at high Reynolds numbers (up to 9500) characteristic of turbulent flows. Understanding such flows is important in numerous computational fluid dynamics applications, including geoscience, engineering, and hydrodynamics.

Full reference: Misun Min and Taehun Lee, A spectral-element discontinuous Galerkin lattice Boltzmann method for nearly incompressible flows, Journal of Computational Physics 230(1), 245-259, 2011.

Invisible Milky Way Satellite Uncovered with Help from NERSC

Astronomers predict that large spiral galaxies, like our Milky Way, have hundreds of satellite galaxies orbiting around them. While a few satellites like the Magellanic Clouds are visible, many other galaxies are too dim to see. Scientists suspect that these faint satellite galaxies are primarily comprised of mysterious “dark matter,” which makes up 85 percent of all matter in the universe and so far remains undetected.

Using supercomputers at the National Energy Research Scientific Computing Center (NERSC), Sukanya Chakrabarti has developed a mathematical method to uncover these “dark” satellites. When she applied this method to our own Milky Way galaxy, Chakrabarti discovered that a faint satellite might be lurking on the opposite side of the galaxy from Earth, approximately 300,000 light-years from the galactic center.

“Our approach has broad implications for many fields of physics and astronomy—for the indirect detection of dark matter as well as dark-matter dominated dwarf galaxies, planetary dynamics, and for galaxy evolution dominated by satellite impacts,” says Chakrabarti, who presented these findings at the American Astronomical Society meeting in Seattle. This result came from Chakrabarti’s postdoctoral research at the University of California, Berkeley. She is currently an assistant professor of physics at Florida Atlantic University.

PNNL Researchers Implement Bioinformatics Technique to Improve Peptide Identification

Scientists at Pacific Northwest National Laboratory (PNNL) have developed a pattern-matching algorithm that improves the accuracy of peptide identification by between 50 and 150 percent, compared with standard approaches. The key to the method was deconstructing the pattern-matching problem using principles of statistical physics, which mathematically connects the behavior of individual atoms to large groups of molecules that can be observed and measured. The new method allows researchers to compare unknown peptide samples with both a peptide database of “ideal” samples and a library of experimental peptide samples. The discovery is being published in the Journal of Proteome Research.

Contact: Bill Cannon,
Researchers Use Jaguar to Simulate the Next Big Step in Igniting ITER

University of California–Irvine researcher Zhihong Lin is using the Jaguar supercomputer at Oak Ridge National Laboratory (ORNL) to study fusion reactions, which produce helium from hydrogen and release energy in the process, in hopes of igniting ITER, an experimental fusion reactor in southern France. In 2010, the team led by Lin published three articles in research journals devoted to plasma physics.

Lin played a key role in developing the gyrokinetic toroidal code (GTC) to calculate plasma particles’ motion using supercomputers in hopes of painting a more complete picture of turbulence in a fusion plasma. Lin’s team plans to use GTC with another gyrokinetic code, GYRO, for its simulations. The team spent 2010 upgrading GTC and GYRO and making sure they would be able to simulate charged particles interacting with magnetohydrodynamic (MHD) waves. In 2011, the team is calculating the evolution of charged particles travelling in a doughnut-shaped reactor, called a tokamak, and observing how and where MHD waves transport energetic particles. By 2012 Lin expects the model to be fully developed—simulating all turbulent interactions between the particles in a fusion reaction.

LBNL Experts Edit, Contribute to Book on Performance Tuning

David Bailey and Sam Williams of Lawrence Berkeley National Laboratory’s (LBNL’s) Computational Research Division (CRD) recently co-edited a book called Performance Tuning of Scientific Applications, which presents current research in performance analysis from some of the most notable experts in the field. Robert Lucas, Director of Computational Sciences at the Information Sciences Institute and principal investigator (PI) for the SciDAC Performance Engineering Research Institute, was also an editor for the book. Performance analysis has grown into a full-fledged, sophisticated field of empirical science. Describing useful research in modern performance science and engineering, this book helps real-world users of parallel computer systems to better understand both the performance vagaries arising in scientific applications and the practical means for improving performance.

In addition to Bailey and Williams, other Berkeley Lab contributors to the book include Associate Lab Director for Computing Sciences Kathy Yelick, as well as Mark Adams, John Bell, Vincent Beckner, Jonathan Carter, Khaled Ibrahim, Juan Meza, John Shalf, Hongzhang Shang, Erich Strohmaier, Lenoid Oliker, Lin-Wang Wang, Harvey Wasserman and Zhengji Zhao.

Simulating Tomorrow’s Accelerators at Near the Speed of Light

Realizing the promise of laser-plasma accelerators crucially depends on being able to simulate their operation in three-dimensional detail. Until now such simulations have challenged or exceeded even the capabilities of supercomputers.

A team of researchers led by Jean-Luc Vay of Berkeley Lab’s Accelerator and Fusion Research Division (AFRD) has borrowed a page from Einstein to perfect a revolutionary new method for calculating what happens when a laser pulse plows through a plasma in an accelerator like BELLA, the Berkeley Lab Laser Accelerator now being built at the U.S. Department of Energy’s Lawrence Berkeley National Laboratory. Using their “boosted-frame” method and NERSC supercomputers, Vay’s team has achieved full 3D simulations of a BELLA stage in just a few hours of supercomputer time, calculations that would have been beyond the state of the art just two years ago.

Predicting Nuclear Structure and Nuclear Reactions

With the promise of providing a robust and precise model of nuclear structure and reactions based on the underlying theory of strong interactions, quantum chromodynamics represents a “holy grail” for physics with many practical applications. Predictions of the structures and reactions of nuclei are important for basic science and the future of the nation’s energy and security needs.

A comprehensive description of all nuclei (stable and unstable) and their reactions requires investigating rare and exotic isotopes with unusual proton-to-neutron ratios that are difficult to produce and study experimentally because of their short lifetimes. Led by James Vary at Iowa State University, researchers are performing state-of-the-art simulations on the IBM Blue Gene/P at the Argonne Leadership Computing Facility (ALCF) and Jaguar at Oak Ridge National Laboratory (ORNL) to provide needed predictions of the structures and reactions of nuclei, where direct experiment is not possible or is subject to large uncertainties. Such calculations are relevant to nuclear energy, nuclear security, and nuclear astrophysics applications, where rare nuclei lie at the heart of nucleosynthesis and energy generation in stars.

A major focus area of the project is a high-quality microscopic description of the nuclear fission process. To achieve this goal, researchers have developed a new nuclear energy density functional, “UNEDF1,” that greatly improves results for fission. They expanded the experimental data set in the optimization of UNEDF1 to include excitation energies of superdeformed fission isomers in the actinide region. Further improvement of the nuclear energy density functional requires ab initio simulations of nuclear systems that are beyond experimental reach. For this purpose, ab initio simulations of “neutron drops” with 4 through 42 neutrons in external fields have been performed on Jaguar at ORNL. These neutron drop results will be incorporated in the data set to further optimize the energy density functional.

Contact: James Vary,
Jaguar Users Benefiting from ALCC Computing Hours for Energy-Mission Research

DOE’s Office of Advanced Scientific Computing Research (ASCR) has awarded more than 270 million processor hours to researchers using the Oak Ridge Leadership Computing Facility (OLCF). Through the ASCR Leadership Computing Challenge (ALCC), allocations were awarded to five projects with an emphasis on high-risk, high-impact research related to DOE’s energy mission. They are already supporting important efforts such as advancing the Administration’s clean-energy agenda and understanding the Earth’s climate.

“The processor hours from ALCC allow projects to run full scenarios instead of only getting a glimpse of part of them, which really helps scientists running simulations,” said Doug Kothe, director of the Consortium for Advanced Simulation of Light Water Reactors, a project that was allocated 30 million hours through ALCC. ORNL is leading this multi-institutional energy innovation hub. Nuclear energy provides an enormous opportunity for the United States to provide carbon-free energy, said Kothe. The project will allow engineers to simulate a currently operating reactor with the goal of validating the virtual model created by the consortium.

The ALCC program offers awards for one-year projects, and researchers can submit applications year-round. The awards go to projects with the potential to advance the DOE missions, such as how to strengthen our energy security, improve environmental quality, and develop economic vitality through public–private partnerships.


Argonne’s Ian Foster to Receive IEEE Computer Society’s Tsutomu Kanai Award

Ian Foster, an Argonne Distinguished Fellow and director of the Argonne/University of Chicago Computation Institute, has been named the recipient of the IEEE Computer Society’s 2011 Tsutomu Kanai Award. The award recognizes major contributions to state-of-the-art distributed computing systems and their applications.

Foster is considered one of the world’s foremost researchers in distributed, parallel, and data-intensive computing technologies. He spearheaded the development of the Globus Toolkit, considered the de facto standard in grid computing. He also is a PI in the DOE SciDAC “Center for Enabling Distributed Petascale Science” and in the DOE ASCR project “Scaling the Earth System Grid to Petascale Data.”

Foster is a fellow of the American Association for the Advancement of Science, Association for Computing Machinery, and British Computer Society. His other awards include the Global Information Infrastructure Next Generation award, the British Computer Society’s Lovelace Medal, R&D Magazine’s Innovator of the Year, and honorary doctorates from the University of Canterbury, New Zealand and CINVESTAV, Mexico.

LBNL’s Kathy Yelick Co-Authors NRC Report on Computer Performance

Kathy Yelick, Associate Laboratory Director for Computing Sciences at Berkeley Lab and director of the lab’s NERSC Division, was a panelist in a March 22 discussion of The Future of Computer Performance: Game Over or Next Level?” a new report by the National Research Council. Yelick is one of 13 members of the committee that authored the report and the only member representing a DOE lab. The symposium, held in Washington, D.C., began with a briefing and discussion of the recently released report and subsequent panel sessions explored issues raised by the report and opportunities and challenges for sustaining growth in computing performance. Yelick, who is a recognized leader in programming languages, was a member of the panel discussing “Parallelism and Innovative Programming Models, Algorithms, and Languages.”

The symposium was organized by the NRC’s Computer Science and Telecommunications Board.

LBNL’s Bruce Bargmeyer Honored for Contributions to Technology Standards

Bruce Bargmeyer, leader of the Metadata, Semantics, and Ecoinformatics Group in the Advanced Computing for Science Department at LBNL, has been recognized with the InterNational Committee for Information Technology Standards (INCITS) Merit Award for 2011.

In notifying Bargmeyer of the award, Lynn Barra, director of ITI Standards Operations for the INCITS Secretariat, wrote “Without question, you have earned this recognition and INCITS would like to recognize your more than 15 years of participation and contributions to INCITS/DM32.8 Technical Committee and ISO/IEC JTC 1/SC32. Your leadership and broad knowledge of important issues and dedicated efforts are what lead to the successful development and advancement of standards.” The award was presented Monday, April 4, at an INCITS symposium in Hillsboro, Oregon.

PNNL Researcher Named Guest Editor of Publication Special Issue

PNNL scientist Pak Chung Wong will serve as a guest editor of a special issue of IEEE Computer Graphics & Applications in 2012. He will join Professor Han-Wei Shen of The Ohio State University and Professor Valerio Pascucci of the University of Utah who have also been selected to guest-edit the publication. All three currently serve as principal investigators on ASCR-funded projects. The special issue will focus on Extreme Scale Visual Analytics.

Contact: Pak Chung Wong,
Berkeley Lab’s Juan Meza Named One of Most Influential Hispanics in Technology

The editors of Hispanic Engineer & Information Technology magazine (HE&IT) have selected Juan Meza, interim head of the Lawrence Berkeley National Laboratory’s Computational Research Division, as one of the “Top 200 Most Influential Hispanics in Technology.” The winners were recognized for their excellence in work, strong commitment to their communities and leadership as role models. All the “Top 200” will be featured in the April 2011 issue of HE&IT magazine and have been invited to attend the Hispanic 200 Leadership Summit in Orlando, Fla. this summer. HE&IT magazine is distributed to engineering students at Hispanic-serving institutions, and to Hispanic professionals, high-level government and industry policy makers and executives across the country.


ORNL Retires Venerable Early Jaguar System

ORNL lost a computing powerhouse with the March retirement of the Jaguar XT4 supercomputer. Once the second-most powerful supercomputer in the world, the Cray machine was later eclipsed by its XT5 sibling—also named Jaguar. It was decommissioned March 8.

The system went into service in February 2007, with 68 cabinets delivering a peak performance of 65 trillion calculations a second, or 65 teraflops. The peak performance jumped to 119 teraflops the following month when the XT4 was connected to the existing XT3 incarnation of Jaguar, placing that combined machine at number two on the Top500 List of the world’s most powerful supercomputers. The system was upgraded once again in spring 2008, with 31,000 processing cores delivering up to 263 teraflops.

During its tenure the XT4 delivered more than 830 million processor hours to cutting-edge research in a wide variety of fields, including climate, combustion, fusion, chemistry, materials science, and astrophysics. “It had an amazingly long life for a cutting-edge platform,” noted OLCF Science Director Bronson Messer. “The incremental upgrade path had only been tried a few times, but it worked out great. We always got a more capable machine than we started out with, and the user experience barely changed. That’s all a computational scientist can ask for.”


LBNL Contributes to Success of Sold-Out Tapia Diversity in Computing Conference

For the first time in its 10-year history, the Richard Tapia Celebration of Diversity in Computing Conference registered a record-setting 526 attendees, prompting organizers to close registration a week before the April 3–5 conference began in San Francisco. Of 526 who registered, about 75 percent are students, more than 50 percent are women, and more than 70 percent are from other under-represented groups, including African Americans, Native Americans, Hispanics, or Latinos. “We would love to have more attendees, but our hotel facilities simply can’t handle any more people,” said 2011 Tapia General Chair David Patterson, a professor of computer science at the University of California at Berkeley, faculty researcher in Berkeley Lab’s Computational Research Division, and former president of the ACM, the society sponsoring the Tapia conference.

Tapia 2011 is the fifth time that Berkeley Lab has been a supporter, contributing $5,000 to fund scholarships for students to attend the conference. Other LBNL contributions include:

Conference Organizing Committee
  • Jon Bashor is the PR chair (for the third time).
  • Tony Drummond is the Student Poster chair. In the past, Tony has been a speaker and organizer of the Doctoral Consortium.
Conference Program
  • Elizabeth Bautista was part of a Tuesday panel session on Opportunities at DOE Labs. The session was organized by Debbie McCoy of ORNL and also included Rebecca Hartman-Baker and Judith Hill of ORNL, Tammy Kolda of Sandia/California, Yashema Mack of the University of Tennessee, and Celeste Matarazzo of Lawrence Livermore National Laboratory (LLNL).
  • Deb Agarwal, Elizabeth Bautista, Orianna Demasi, Juan Meza, Sean Peisert, and Dani Ushizima hosted tables during Tuesday’s “Lunch with Interesting Bay Area People.”

On the day following the conference, about 20 attendees visited Berkeley Lab to learn more about careers in computing sciences and computing research at Berkeley Lab, hearing presentations from staff and touring research facilities. The tour and the Lab’s participation in the Tapia conference itself are already yielding concrete results. Juan Meza, Acting Director of the Computational Research Division, reports that several participants have requested summer internships or postdoc positions.

ALCF Hosts Gordon Bell Preparation Workshop

Each year, the ACM Gordon Bell Prize is awarded in recognition of outstanding achievement in high-performance computing. The award is administered by the Association for Computing Machinery (ACM), with financial support for the stipend (currently $10,000) provided by Gordon Bell, a pioneer in high-performance and parallel computing. It tracks progress of parallel computing over time, with particular emphasis on rewarding innovation in applying high-performance computing to applications in science. Prizes are awarded for peak performance as well as special achievements in scalability and time-to-solution on important science and engineering problems and low price/performance.

To assist users in pursuit of a Gordon Bell Prize, the Argonne Leadership Computing Facility (ALCF) conducted a Gordon Bell Preparation Workshop on March 16–18. At the workshop, Leopold Grinberg, Brown University; ALCF’s Principal Performance Applications Engineer, Vitali Morozov; and members of the ALCF’s Data Analytics and Visualization team worked on readying their submission: a first-of-its-kind, multiscale simulation in the domain of patient-specific intracranial arterial tree blood flow.

Candidates for the Gordon Bell Prize must be submitted by April 15th.

Contact: Vitali Morozov,

OLCF Spring Training Workshop Arms Users with Supercomputing Skills

The OLCF’s 2011 Spring Training event was held March 7–11 at the center. The workshop had courses for novices and supercomputing veterans alike. Sessions during the early part of the week focused on basic supercomputing skills as part of the High-Performance Computing Crash Course. Monday was devoted to using Linux, and Tuesday focused on the Message-Passing Interface, or MPI, two essential skills for using large parallel processing systems. The OLCF’s Rebecca Hartman-Baker and Arnold Tharrington taught the sessions.

Wednesday and Thursday focused on the Cray XT5 machine particularly, starting with an intermediate course then moving to advanced subjects on Thursday. The last day of the event, Friday, culminated with the annual OLCF Users’ Meeting. The OLCF group leaders and the User Council spoke about ORNL supercomputing over the past year, and ORNL scientific liaisons spoke and fielded questions from users.

Not all attendees were physically present, though. Users unable to attend in person were able to use WebEx, an online conferencing tool, so they could still take part in seminars. Whitten said that virtual users outnumbered those present some days, and WebEx users averaged 25 per day.

Japanese University Delegation Visits Berkeley Lab to Discuss Potential Collaborations

Representatives from Japan’s Tsukuba University’s Center for Computational Sciences (CCS) visited Berkeley Lab to meet with researchers from CRD and NERSC and explore areas of potential collaboration. The university is located 30 miles northeast of Tokyo in an area known as Tsukuba Science City, home to Japan’s national research facilities encompassing such fields as science, industry, agriculture and forestry, environment and space development. CCS has been involved in parallel computing research since 1977 and has historically deployed custom-designed systems tailored to scientific applications.

Berkeley Lab Hosts Albany High Students on Job Shadow Day

As part of Albany High School’s annual Job Shadow Day, 19 juniors from the Bay Area high school spent several hours shadowing Berkeley Lab scientists, mathematicians, engineers, technicians and communications staff. Based on their choice of potential career paths, students were matched with mentors at the main Lab facility, NERSC in Oakland and the Potter Street biosciences research center.

“I had a really great time exploring the various jobs that are done in a world-class laboratory. The environment was great, and the scientists that showed us around were great,” said Edward Gong, who spent the morning at NERSC. “We learned a lot about different aspects of computer science as well as electrical engineering and sciences. The tutorials were engaging, and interactive; everyone got a chance to give insight on situations. Overall, the experience is great.”