Berkeley Lab

Taking Accelerator Simulations to the Exascale

The Department of Energy’s Exascale Computing Project (ECP) has announced support for 15 critical research applications for next-generation supercomputers, and ATAP will lead one of them: “Exascale Modeling of Advanced Particle Accelerators,” headed by Dr. Jean-Luc Vay of ATAP’s BELLA Center and Center for Beam Physics.

Jean-Luc Vay
Particle accelerators are a vital part of the DOE-supported infrastructure of discovery science and university research, as well as private-sector applications, and have a broad range of benefits to industry, security, energy, the environment and medicine.

This project supports the practical and economic design of smaller, less-expensive plasma-based accelerators. Turning plasma accelerators from a promising technology into a mainstream scientific tool depends critically on high-performance, high-fidelity modeling of complex processes that develop over a wide range of space and time scales. Lawrence Livermore National Laboratory and the SLAC National Accelerator Laboratory will also participate in the project.

“An opportunity to help lead the way to exascale applications”
— ATAP Director Wim Leemans

“Accelerator modeling is not only crucial to future progress with BELLA and other accelerators — it is an opportunity to help lead the way to exascale applications,” says ATAP Director Dr. Wim Leemans, noting that transforming science through exascale computing is one of Director Witherell’s strategic priorities for the Laboratory.

With 50 to 100 times the performance of today’s typical supercomputers, “exascale computing will be able to accomplish in minutes to hours what presently would take days to weeks,” adds Vay. This will enable accelerator designers to perform far more-detailed and higher-fidelity simulations and to examine more-complex phenomena.

The ten-year challenge taken up by the proposal is the modeling of a chain of up to a hundred plasma acceleration stages in less than a week, and ideally less than a day.

The recent report by the Accelerator R&D Subpanel of HEPAP, the High Energy Physics Advisory Panel, observed that “advancing the capabilities of accelerator simulation codes to capitalize on the drive toward exascale computing would have large benefits in improving accelerator design and performance.” Coupled to algorithmic advances, such as the Lorentz boosted frame approach, adaptive mesh refinement, scalable spectral electromagnetic solvers, and numerical Cherenkov instability mitigation methods, it will “enable reaching the ultimate goal of realtime virtual prototyping of entire accelerators” — the detailed and accurate “end to end” modeling that has long been a dream of accelerator simulation.

“This accelerator modeling project embodies the new paradigm of combining experimental and computational methods to advance a critical technology,” said James Symons, LBNL’s Associate Laboratory Director for Physical Sciences. “Realizing the potential of plasma-driven accelerators will impact fields ranging from health care to manufacturing to basic research.”

Supercomputer simulations of plasma-based accelerators typically use a “moving window” to restrict the simulation area to a region of interest that encompasses the laser beam and the portion of wakefield that accelerates the electron beam. In this small-scale simulation meant to illustrate the physics, a laser beam (red and blue disks) propagating through an under-dense plasma displaces electrons, creating a wake that supports very high electric fields (pale blue and yellow), that can accelerate an electron beam (white) to high energy in a short distance. While the simulation box is spatially much smaller, the number of time steps that are required to simulate the crossing of the laser through the plasma is still very large, typically over a million. Exascale computers, 50-100x more powerful than today’s typical supercomputers, will be game changers for what we can feasibly model.

ECP Work Will Be Next Stage of an Ongoing Effort

“We’ve spent years preparing to take advantage of this opportunity,” Leemans observes. Modeling has long been recognized as a key to designing advanced particle accelerators. ATAP has been among the leaders in it, often working together with LBNL’s Computing Research Division and National Energy Research Supercomputing Center (NERSC) as well as colleagues at other laboratories.

Vay coordinates the Berkeley Lab Accelerator Simulation Toolkit (BLAST) effort and the emergent multi-laboratory Consortium for Advanced Modeling of Particle Accelerators (CAMPA). Vay also leads the NERSC Exascale Science Applications Program (NESAP) project on Advanced Modeling of Particle Accelerators. NESAP was launched in 2014 to prepare for NERSC’s newest supercomputer.

Exascale To Be a Big Part of Lab’s Future, and Vice Versa

Of the 15 fully funded ECP proposals, Berkeley Lab will lead two and participate in four others. An additional seven proposals received seed funding; Berkeley Lab will lead three and participate in two others.

“These awards reflect our extensive experience and expertise in computational science across a wide range of disciplines, including accelerator design, subsurface flows, cosmology, combustion, chemistry,” said Kathy Yelick, Associate Laboratory Director for Computing Sciences. “Our applied mathematics and computer science expertise will be needed to develop applications tailored to exascale systems.”

Click here to learn more about this and other Berkeley Lab ECP projects.

“The Incredible Shrinking Particle Accelerator” by Kathy Kincade of NERSC puts accelerator simulation into overall context of advanced LBNL computing efforts, including visualization.