Scientific Achievement

During relativistic magnetic reconnection, a fundamental process in plasmas in which oppositely directed magnetic field lines break and reconnect, releasing large amounts of energy that heat and accelerate charged particles. This process is often modeled using large-scale particle-in-cell (PIC) simulations on the largest supercomputers. However, these methods are computationally expensive, limiting their feasibility for large-scale, three-dimensional studies.

For the first time, a team of researchers from the Advanced Modeling Program in the Accelerator Technology & Applied Physics (ATAP) Division and the Center for Computational Science and Engineering at the Department of Energy’s Lawrence Berkeley National Laboratory (Berkeley Lab), in collaboration with Advanced Micro Devices, Inc. have applied mesh refinement (MR)—a technique for resolving key parts of a model with successively finer and finer meshes—to 2D relativistic magnetic reconnection simulations to model the disparity in spatial scales in reconnection efficiently.

ATAP first adapted MR for electromagnetic PIC simulations to model particle beams and accelerators. In this research, the team extended MR to a new class of applications, like magnetic reconnection. This posed new challenges due to high particle flux across the coarse-fine interface and required an approach to capture the complex interaction of charged particles with the electromagnetic fields in these systems.

They found that simulations with an MR ratio of eight agreed with the high-resolution baseline simulation without MR, utilizing only 36% of the macroparticles and 71% of the computing node hours required for the baseline.

Significance and Impact

Relativistic magnetic reconnection is frequently used to explain particle energization, which leads to high-energy emissions across various plasma systems. These systems include solar flares and extreme astrophysical phenomena such as pulsars, active galactic nuclei, gamma-ray bursts, black hole jets, laboratory astrophysics, and magnetic fusion devices. Investigating the plasma kinetic effects that underpin particle energization is essential for understanding high-energy emissions from these systems and is very challenging to model computationally.

This work demonstrates the feasibility of using MR to simulate magnetic reconnection accurately. It can be applied to 3D systems where more significant memory savings are anticipated, enabling future computationally efficient 3D reconnection studies. Furthermore, the MR strategies presented in this research have implications that extend beyond relativistic reconnection. They may also benefit non-relativistic magnetic reconnection (e.g., solar flares) and other high-energy systems with significant disparities in spatial scales.

Research Details

To ensure that MR does not introduce numerical artifacts, the team employed the Pseudo-Spectral Analytical Time-Domain (PSATD) Maxwell solver, which mitigates numerical dispersion. A new feature for absorbing electromagnetic waves at the coarse-fine interface was implemented to prevent non-physical reflections and charge accumulation. They then examined the impact of these algorithms on the accuracy of MR simulations conducted on graphics processing units (GPUs) using the award-winning PIC code WarpX by comparing particle spectra and current-sheet evolution with those obtained from high-resolution uniform grid simulations.

Role of absorbing boundary layers in MR for EM-PIC

Schematic to illustrate the static mesh refinement algorithm, with a parent grid at level 0 (L0) and a refined region at level 1 (L1). The refined region involves three patches: a fine patch, a coarse patch, and an auxiliary patch. The fine and auxiliary patches have L1 resolution, while the coarse patch has the same resolution as the level below it, i.e., the parent grid. Maxwell’s equations are solved on the fine and coarse patches of L1; these regions are terminated by absorbing layers indicated by the green bands surrounding these patches.

Absorbing layers are added at the coarse-fine interface to prevent numerical artifacts caused by sudden changes in grid resolution. Previously, perfectly matched layers (PML) were employed as the default method to dampen the electromagnetic fields at the coarse-fine boundary of the refined region. In this method, the standard components of the electromagnetic fields are attenuated. Since the PML approach is designed to absorb electromagnetic waves in a vacuum, it performs effectively in MR simulations of plasma-based particle accelerators. However, for scenarios such as reconnection, where the plasma density and current density at the coarse-fine interface are high, it was found that a simpler absorbing boundary condition—which dampens all components—was more effective in mitigating numerical artifacts at the coarse-fine boundary.

Advantages of the PSATD solver

Traditionally, finite-difference time-domain (FDTD) solvers are employed to solve Maxwell’s equations; however, they are susceptible to numerical dispersion, meaning waves propagate at a speed dependent on their wavelength in vacuum, an unphysical effect resulting from the finite difference approximations. In contrast, PSATD is an ultrahigh-order, unconditionally stable spectral method, which is particularly advantageous in avoiding numerical dispersion. In MR simulations of magnetic reconnection, researchers discovered that near the conclusion of reconnection, the FDTD method produced spurious structures with larger wavelengths in the coarse regions compared to the fine regions. This phenomenon is attributed to a mismatch in numerical dispersion between the coarse and fine grids. Conversely, the nearly dispersionless PSATD method does not generate these spurious structures. By integrating PSATD with absorbing boundaries, the MR algorithm effectively reproduced reconnection characteristics such as current sheet evolution and particle spectra.

Summary and future work

This study demonstrates that two-dimensional magnetic reconnection simulations with 37% mesh refinement and a refinement ratio of eight achieve a 1.4× performance improvement on GPU nodes while requiring only half the number of GPUs compared to a uniformly high-resolution grid. The reduced memory footprint of the MR approach improves computational efficiency and paves the way for more tractable 3D reconnection studies. The researchers aim to extend the current static MR implementation to support dynamic refinement regions that adapt to the current sheet’s evolving topology, enabling more efficient simulations of the complex reconnection phenomena.

Contact: Revathi Jambunathan

Researchers: Revathi Jambunathan, Henry Jones, Lizzette Corrales, Hannah Klion, Andrew Myers, Weiqun Zhang, and Jean-Luc Vay (Berkeley Lab); and Michael E. Rowan (Advanced Micro Devices, Inc.)

Funding: This work was partially supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research (ASCR), Exascale Computing Project, and Simulation and Analysis of Reacting Flow, funded by the Applied Mathematics Program in ASCR.

Publication: R. Jambunathan, H. Jones, L. Corrales, H. Klion, M. E. Rowan, A. Myers, W. Zhang, and J.-L. Vay. “Application of mesh refinement to relativistic magnetic reconnection,” Phys. Plasmas 32, 013905 (2025). https://doi.org/10.1063/5.0233583

For more information on ATAP News articles, contact caw@lbl.gov.