Director’s Corner
SCIENCE FEATURES
• Novel RF Controls for QIS
• Fiber Lasers Poised to Advance LPAs
NEWS IN BRIEF
• Deputy Secretary Turk Visits BELLA
• ICFA Features AMP Authors
• ATAP Leadership in Snowmass
ICYMI
• Watching TaTe2 in HiRES
• AMReX and Exascale Computing
HONORS AND AWARDS
• APS Fellowship for Prestemon
• Chad Mitchell Director’s Award
WORKPLACE LIFE
• Stay Healthy & Well—Get Outside
• Food Bank Challenge
• Lactation Rooms
IDEA
• Writing For and About People with Disabilities
PUBLICATIONS
SAFETY: THE BOTTOM LINE
• Safety Week Awards
• Safety Week Photostory
• “ISM 101” Available Online
• Unwanted Holiday Guests
• Preventing Slips, Trips, & Falls
The advanced hardware that is foundational to accelerators is highlighted by our two research features this month. Parlaying our accelerator controls and instrumentation expertise into advances in quantum information systems, a team of ATAP Division physicists and engineers at Berkeley Lab has demonstrated the feasibility of low-cost and high-performance radio frequency modules for qubit controls at room temperature. Meanwhile, our laser researchers continue to advance in their efforts to coherently combine pulses from fiber lasers, with the goal of producing high-energy, high-repetition-rate beams for future laser-plasma accelerators (LPAs). This is a frontrunning candidate for kBELLA, the proposed next-generation laser that will enable practical applications of LPAs. We had an opportunity to brief Deputy Secretary of Energy David Turk on these matters during his recent visit to Berkeley Lab.
Theory and computational modeling have always gone hand in hand in ATAP. Our Accelerator Modeling Program had the honor of authoring a paper and co-authoring another in the latest issue of the ICFA Beam Dynamics Newsletter.
Helping plan the future of high-energy physics, our staff are participating extensively in the Snowmass process, now taking place.
The Berkeley Lab Director’s Award for Scientific Achievement went to Chad Mitchell of the Accelerator Modeling Program, and Soren Prestemon, Deputy Division Director for Technology and head of our superconducting-magnet programs, joined the ranks of our Fellows of the American Physical Society.
As the year comes to a close, I would like to thank everyone for continuing to make progress under the difficult circumstances of the pandemic. I wish you a safe, restful, healthy, and happy holiday break, and look forward to another year of exciting research and development in 2022.
HOW A NOVEL RADIO FREQUENCY CONTROL SYSTEM ENHANCES QUANTUM COMPUTERS
—Researchers build an open source room-temperature control system for superconducting quantum processors
By Monica Hernandez, Quantum Communications Lead, Computing Research Division
RF mixing modules for electronic controls of superconducting quantum processors. (Credit: Gang Huang and Yilun Xu/Berkeley Lab)
A team of ATAP Division physicists and engineers at Berkeley Lab have successfully demonstrated the feasibility of low-cost and high-performance radio frequency modules for qubit controls at room temperature. They built a series of compact radio frequency (RF) modules that mix signals to improve the reliability of control systems for superconducting quantum processors. Their tests proved that using modular design methods reduces the cost and size of traditional RF control systems while still delivering superior or comparable performance levels to those commercially available.
Their research, featured as noteworthy in the Review of Scientific Instruments and selected as a Scilight by the American Institute of Physics, is open source and has been adopted by other quantum information science (QIS) groups. The team expects the RF modules’ compact design is suitable for adaptation to the other qubit technologies as well. The research was conducted at the Advanced Quantum Testbed (AQT) at Berkeley Lab, a collaborative research program funded by the U.S. Department of Energy’s Office of Science.
“This is an outstanding example of how expertise built up for one area of science, such as instrumentation and control of particle accelerators, can have benefits reaching across disciplines,” said ATAP Division Director Cameron Geddes.
A Question of Scale
Despite significant advances in building processors with more qubits, which will ultimately be needed to demonstrate a quantum advantage over classical computers, quantum computers continue to be noisy and error-prone. Each additional qubit introduces new layers of complexity and possibilities for electrical failure, especially at room temperature. This growth in complexity and computing power requires a rethinking of certain core control elements.
Traditional RF control systems use analog circuits to control superconducting qubits, but they can become bulky and overwhelmingly complex, thus serving as a potential point of failure and increasing the costs for hardware control. AQT researchers Gang Huang and Yilun Xu from Berkeley Lab’s Accelerator Technology and Applied Physics Division (ATAP) demonstrated a new way to control qubits that is already enhancing other quantum computing projects at the testbed’s user program. The team substituted the larger, more costly traditional RF control systems for one built at Berkeley Lab, which uses smaller interactive mixing modules.
A key aspect of this modular system is delivering high-resolution, low-noise RF signals needed to manipulate and measure the superconducting qubit at room temperature. To do so, it’s important to shift the qubit manipulation and measurement signal frequency between the electronics baseband and the quantum system.
“The new module exhibits low-noise, high-reliability operation and is now becoming our laboratory standard for microwave frequency modulation/demodulation across many different experimental configurations in AQT,” Huang explained.
Using the team’s low-noise RF mixing module to shift the bandwidth with a limited intermediate frequency between the electronics baseband and the quantum system intrinsic band allows researchers to utilize less noisy converters for better performance and at a lower cost.
Huang and Xu said that while their system was designed for superconducting systems, it could be expanded to other quantum information science platforms. “In general, the architecture of RF mixing can be expanded to higher frequencies,” they noted. “Therefore, if we replace some electronic components with appropriate frequency, this kind of compact design should be able to adapt to the other qubit platforms, i.e., semiconductor qubit systems.”
Researchers also designed electromagnetic interference shielding to eliminate undesired perturbations, which reduce signal integrity and limit overall performance. This shielding aims to prevent the signal from leaking out and interfering with surrounding electronics – a common problem for quantum computers.
Left: AQT cryogenic dilution refrigerator. Right: Two kinds of RF mixing modules: the up and the down converter. (Credit: Gang Huang and Yilun Xu/Berkeley Lab)
Open Source, Open Hardware
With the release of a control system that is open source, the team hopes that the broader community uses and contributes to the repository, improving the hardware. By replacing a few electronic components with appropriate frequency, this kind of compact design may adapt to a variety of quantum computing facilities.
“This is one of our first efforts to develop an open source control system for superconducting quantum processors,” explained Huang. “We will continue to optimize the physical size and cost of the module and further integrate with our FPGA-based controller to improve the extensibility of the qubit control system.”
Looking ahead, the researchers are already building on these efforts to create new possibilities in quantum computing and offer a new technology to control qubits.
“Such integration and optimization will help room-temperature-based control systems keep pace with advancements in the complexity of quantum processors,” noted Xu.
FIBER LASERS POISED TO ADVANCE LASER-PLASMA ACCELERATORS
—Gordon and Betty Moore foundation grant is latest impetus to program in high-average-power lasers
By William Schulz
The next phase in the development of laser-plasma particle accelerators (LPAs) — potentially game-changing tools for research and practical applications — is underway in ATAP. A new approach to high-power lasers — combining the pulses from many fast-acting but lower-energy optical fiber lasers — will energize these super-compact accelerators.
Berkeley Lab researchers have zeroed in on the limitations of today’s LPAs and believe that high-repetition-rate systems using optical fiber lasers are the path to increased stability and performance as well as higher pulse rate.
ATAP Division research scientist Tong Zhou conducting fiber laser combination experiments. An ongoing multi-institutional project to coherently combine the output of fast-pulsing but low-energy fiber lasers could be the secret to having both high energy and high repetition rate — key to the next steps in laser-plasma accelerators. (Credit: Marilyn Sargent/Berkeley Lab)
Cameron Geddes, ATAP Division Director, said, “Based on recent technological breakthroughs in fiber lasers and laser-plasma accelerators, it’s time to bring these together—to develop a next generation of compact and precision-controlled accelerators that can bring new capabilities to a wide range of applications.”
LPAs: Small is the new big
LPAs, in which ATAP’s Berkeley Lab Laser Accelerator (BELLA) Center is among the leaders, are a radically compact approach to particle acceleration, notable for achieving particle energies in centimeters that would require tens of meters with conventional technologies.
Conventional accelerators use microwaves in resonant metallic cavities to impart high energies to beams of subatomic particles. This mature technology, which can take several circular or linear forms, makes accelerators powerful engines of scientific discovery, in addition to numerous practical applications in medicine, industrial processing, and national security. Many of them are, however, large and costly.
LPAs offer an alternative way to accelerate and boost the energies of the particles. Rather than using microwaves, an intense beam of laser light fired through a gas will generate a plasma wave that charged particles can ride like a surfer.
Radically smaller than present-day means of achieving the same beam energy, LPAs would be attractive in many applications, ranging from biomedical treatment to free-electron-lasers research centers to nuclear nonproliferation. Ultimately they might even be the basis for a new generation of colliders, orders of magnitude smaller than today’s, for high-energy physics.
LPAs have been successfully demonstrated (BELLA Center holds the record, having accelerated electrons to an energy of 7.8 billion electron-volts in just 20 cm), but they require high laser power. A laser like the BELLA Petawatt produces output greater than to the entire output of the world’s electrical grid for an extremely brief instant, focused into a pulse the diameter of a human hair. However, it can only muster a pulse every second or so. Useful applications will require high laser power delivered in much more frequent pulses. That’s where the new fiber laser project comes in.
Laser teamwork means powerful pulses
Fiber lasers (based on optical fibers that are like those familiar from telecommunications and computer networking, but designed for optimal laser emission) are fast, but small. Each optical fiber provides a channel no wider than a human hair, and can only emit so much power. The project now getting underway—building upon several years of groundwork at Berkeley Lab, the University of Michigan, and Lawrence Livermore National Laboratory—will further develop a scheme called “coherent beam combining.” The goal is pulses energetic enough to drive an LPA, but delivered a thousand times a second.
The new project is led by ATAP Division researcher Tong Zhou. Berkeley Lab team members working on fiber laser development also include Russell Wilcox, Qiang Du, Thorsten Stezelberger, and Jeroen van Tilborg. Almantas Galvanauskas and his students at the University of Michigan and Leily Kiani at Lawrence Livermore National Laboratory also play important roles in the program.
The overall effort, continuing to build upon several years of progress, involves spatial, temporal, and spectral combining in a way that preserves “coherence” (a distinctive quality of laser beams, necessary for LPAs). It aims to bundle the relatively low-powered pulses from many fibers into 30-50 femtosecond long, 200-millijoule pulses with peak power much greater than one terawatt. This would be the highest energy and peak power ever obtained from a fiber laser, and more than sufficient for demonstrations of laser-plasma acceleration.
“Their power consumption would be improved compared to conventional lasers, and their ability to dissipate heat is excellent, addressing other challenges in building high-power lasers,” Zhou said.
Combining laser beams so that they truly resemble one powerful beam is challenging. The next steps being taken by the project that recently commenced, headed by Tong Zhou and supported in part by the Gordon and Betty Moore Foundation, will build upon existing work on spatial combining, as well as amplification in doped fibers. Spectral beam combining (the subject of Zhou’s prestigious Early Career Research Program award from the Department of Energy’s Office of High Energy Physics) and temporal stacking are other ongoing aspects of the overall effort to produce a high-power kilohertz beam from fiber lasers. (Credit: Russell Wilcox, Tong Zhou, Cameron Geddes (Berkeley Lab); Almantas Galvanauskas (University of Michigan).
The long-term goal is a collider for high-energy physics. For those purposes, an LPA would need laser energy on the order of 10 joules in short pulses (30 to 100 femtoseconds each), with a repetition rate greater than 10,000 pulses per second — specifications far beyond existing laser technology. Fiber lasers are a promising candidate for solving this problem, and could in the meanwhile power the many spinoff applications of LPAs.
Power isn’t the only important thing in a system that has to deliver a hair-thin beam into a capillary with an inside diameter just a few times larger than that. Measurement and active feedback for precision control of such attributes as pointing angle and position are the subjects of complementary work at BELLA Center. Machine learning is emerging as an important control technique, as highlighted in a recent article by Dan Wang of the Berkeley Accelerator Controls and Instrumentation (BACI) Program. Wang is pioneering new techniques important to both particle accelerators and fiber lasers, as part of a program with Qiang Du and others.
“We want to not only build a laser system that sets power and energy records, but also state-of-the-art controls, then use it to realize the first high-average-power, high-repetition-rate, laser-driven accelerator in the world,” Geddes said.
Such a system, coherently combining ultrashort pulses from many fiber lasers at a kilohertz repetition rate, is a frontrunner for the laser technology of kBELLA, the proposed next generation of the Berkeley Lab Laser Accelerator (BELLA) Center’s LPA drivers, as well as a broad variety of other applications ranging from accelerators to material processing.
A newly awarded $2.4 million grant from the Gordon and Betty Moore Foundation, by way of the Berkeley Lab Foundation, will aid the coherent-combining work. The Gordon and Betty Moore Foundation fosters path-breaking scientific discovery, environmental conservation, patient care improvements and preservation of the special character of the Bay Area. Additional support is provided by the U.S. Department of Energy, Office of High Energy Physics Early Career Research Program and Accelerator Stewardship Program.
NEWS IN BRIEF
Deputy Secretary of Energy David Turk Visits BELLA Center
David Turk, second-ranking official at the Department of Energy, toured ATAP’s Berkeley Lab Laser Accelerator (BELLA) Center during his November 3, 2021 visit to Berkeley Lab. ATAP staff briefed him on the BELLA Center’s work to create a new generation of particle accelerators that are much more compact than conventional ones by using ultra-intense lasers to drive plasma waves. Topics included kBELLA, a proposed next-generation laser facility, and coherent beam combining, a candidate for one of kBELLA’s key enabling technologies. kBELLA would help make laser-plasma accelerators useful for applications.
Photography by Thor Swift/Berkeley Lab.
L-R: Visiting the BELLA Center were Berkeley Lab Director Michael Witherell; Environment, Safety, and Health Division liaison to the Physical Sciences Directorate Herb Toor; ATAP researcher Tong Zhou; Senior Advisor for Nuclear Security Captain Sean M. Muth, USN; Special Assistant to the DOE Deputy Secretary Sandie Raines; Associate Laboratory Director for Physical Sciences Natalie Roe; DOE Deputy Secretary David Turk; and ATAP Division Director Cameron Geddes. They are looking at a photo of Laboratory founder Ernest O. Lawrence at the 37-Inch Cyclotron, part of a display of historical photographs in this area depicting our accelerator heritage.
“It was an honor to showcase our facilities and explain how the Division’s research benefits society,” said ATAP Director Cameron Geddes.
Supplying high average power at the eponymous repetition rate of a thousand pulses per second, or a kilohertz (compared to present-day high-power lasers like the BELLA Petawatt, which fire a pulse per second), kBELLA will be able to drive the next generation of laser- plasma accelerators. This will open up their many applications, ranging from biomedical treatment to light sources for fundamental research and national security. Such lasers will also enable new industrial processes that create and measure the properties of materials needed for advanced manufacturing, energy storage, and carbon management.
ATAP Director Cameron Geddes (right) shows Deputy Secretary Turk around the BELLA laser control room.
In the BELLA Petawatt amplifier bay, ATAP Director Cameron Geddes (center) shows Deputy Secretary Turk (right) and Special Assistant Sandie Raines (left) one of the plasma capillaries in which laser-plasma accelerators can achieve energies in centimeters that typically require hundreds of meters in conventional accelerators. At right, the plasma capillary in use,
creating a plasma that can guide ultra-intense lasers, enabling plasma acceleration.
ATAP Research Scientist Tong Zhou (r.) briefs Deputy Secretary Turk on coherent beam combining — a technique, being developed at BELLA Center together with colleagues from Lawrence Livermore National Laboratory and the University of Michigan, to assemble a high repetition rate, high average power laser beam from the fast-pulsing but low-powered beams of numerous fiber lasers. Such a laser is a frontrunner for kBELLA, a proposed shovel-ready project that could power the next generation of ultracompact laser-plasma particle accelerators.
To learn more…
See more of the Berkeley Lab endeavors that Deputy Secretary Turk visited.
Visit the kBELLA pages on the BELLA Center website.
AMP Researchers Featured in ICFA Beam Dynamics Newsletter
AMP ICFA authors
The latest issue of the ICFA Beam Dynamics Newsletter has a thematic focus on advanced accelerator modeling. Of the six articles, one has a lead author (and two co-authors) from ATAP’s Accelerator Modeling Program (AMP); another has three AMP co-authors.
AMP head Jean-Luc Vay was lead author of “Modeling of advanced accelerator concepts.” Co-authors included AMP’s Axel Huebl and Remi Lehe.
Huebl and AMP colleagues Chad Mitchell, Ji Qiang, and Rob Ryne were among the co-authors of David Sagan (Cornell) et al., “Simulations of future particle accelerators: issues and mitigations.”
The newsletters, which appear a few times a year, are published by the International Committee on Future Accelerators Beam Dynamics Panel. ICFA was created in 1976 by the Particle and Fields Division of the International Union of Pure and Applied Physics (IUPAP) to promote international collaboration towards construction and use of accelerators for high energy physics. The newsletter series began in 1984 and appears in IOPScience’s Journal of Instrumentation. Yunhai Cai of SLAC was editor for this issue.
ATAP Leadership in Snowmass
ATAP researchers have several leadership roles in the Snowmass process
The “Snowmass process,” the community-input phase of long-term strategic planning in US high-energy physics, has extensive ATAP technical participation, with many ATAP and other Berkeley Lab co-authors of Letters of Interest and whitepapers. As the meetings themselves got underway beginning September 24, ongoing leadership roles came to the fore.
Stephen Gourlay, former director of the division and now an active affiliate in retirement, is one of three co-conveners for the overall Accelerator Frontier. ATAP Director Cameron Geddes is a co-convener of the Advanced Accelerator Concepts topical group on the Accelerator Frontier.
GianLuca Sabbi of the Superconducting Magnet Program is a co-convener of the Magnets subgroup in the Accelerator Technology topical group, and Ji Qiang of the Accelerator Modeling Program is a co-convener for Theoretical Calculations and Simulation on the Computational Frontier.
Bringing early-career researchers into the planning process is an important part of Snowmass. BELLA Center’s Marlene Turner represents early-career researchers in the Multi-TeV Colliders topical group, and recently gave a talk on increasing the physics reach of future particle colliders by decreasing their energy consumption by implementing energy recovery methods, which are made practical by the high gradients of plasma based accelerators. Tianhuan Luo of the Berkeley Accelerator Controls and Instrumentation Program is on the Early Career Survey Team, and Axel Huebl of the Accelerator Modeling Program is Early Career Manager.
Liaisons from one topic to another are a key aspect of communication throughout the large, complex Snowmass process. Accelerator Modeling Program head Jean-Luc Vay serves as the liaison from the Computational Frontier to the Accelerator Frontier, and BELLA Center Deputy Director for Experiments is liaison from Community Engagement to the Accelerator Frontier. Berkeley Accelerator Controls and Instrumentation (BACI) Program head Derun Li is one of the representatives of the Accelerator Frontier in a new Muon Collider forum that combines the interests of the Accelerator, Energy, and Theory Frontiers.
The Snowmass process (named after its origins with summer studies in Snowmass, Colorado) is organized by the American Physical Society’s Division of Particles and Fields approximately every eight years. Delayed by the pandemic but persevering in virtual fashion, the process restarted in earnest September 24 and is now holding meetings that are expected to result in Topical Group reports early in 2022, then Frontier Reports, and finally, in autumn 2022, a summary report.
The resulting summary of the community’s views on the key questions of high-energy physics, and the technology possibilities for addressing them, will be taken into consideration by “P5” — the Particle Physics Project Prioritization Panel. P5 is a temporary subcommittee (reconstituted as needed on an approximately decadal basis) of the DOE’s High Energy Physics Advisory Panel, a standing committee that provides input to both the DOE and the National Science Foundation.
ICYMI
Recent highlights, in case you missed it…
A Tantalizing Look at Tantalum Telluride in HiRES
Illustration of the optically pumped (with near-infrared femtosecond pulses), and ultrafast electron diffraction probed experiment.
The High Repetition Rate Electron Scattering facility, developed by ATAP’s Berkeley Accelerator Controls and Instrumentation (BACI) Program and located at the Advanced Light Source (ALS), was used to perform the first-ever ultrafast electron diffraction studies of optical melting of tantalum ditelluride (TaTe2). Tantalum ditelluride is one of a class of materials called transition-metal dichalcogenides, studied increasingly over the last several years, that have interesting and potentially useful properties when formed into atomically thin monolayers.
The achievement, summarized in a Research Highlight by Berkeley Lab’s Molecular Foundry, also helps validate the scientific usefulness of this one-of-a-kind instrument, enabled by research that began with advanced photoinjectors for light sources. ATAP’s Daniele Filippetto, principal investigator of the HiRES UED beamline, is corresponding author of the paper in the Nature journal Communications Physics describing the experiment.
To learn more…
“Seeing Atoms and Molecules in Motion with an Electron ‘Eye,’ a Berkeley Lab news release, described HiRES and the technologies that made it possible.
Khalid Siddiqui and Daniel Durham are the two lead authors of the Communications Physics paper. Khalid explains his research in this video from the 2020 Berkeley Lab Research Slam competition. Now with Aarhus University in Denmark, he performed this work as a postdoctoral scholar in Berkeley Lab’s Materials Sciences Division. Daniel is a UC-Berkeley graduate student, and he continues to work at HiRES, performing ultrafast experiments on novel materials.
How AMReX is Influencing the Exascale Landscape
— A Q&A with Andrew Myers, core development team member
Editor’s Note: The WarpX “code” (scientific computing program) from ATAP’s Accelerator Modeling Program has become invaluable for advanced accelerator simulations. This interview, from an article by Kathy Kincade of Berkeley Lab Computing Sciences, describes how the AMReX software framework for high-performance computing co-evolved with WarpX.
Andrew Myers
While the hardware components of exascale are key, so too are the software packages and frameworks that will support the demanding scientific applications that run on these next-generation computing systems, and beyond.
Since first coming onto the exascale scene in recent years, AMReX has blossomed into one of the key software ecosystems for many Exascale Computing Project (ECP) efforts, from WarpX and MFiX-Exa to ExaWind, ExaStar, and ExaSky, among others.
AMReX — an open-source framework for performing block-structured adaptive mesh refinement calculations — is the result of a collaboration between Lawrence Berkeley National Laboratory (Berkeley Lab), the National Renewable Energy Laboratory, and Argonne National Laboratory, with development centered at Berkeley Lab. All three labs are part of the Exascale Computing Project’s (ECP) Block-Structured AMR Co-Design Center.
In this Q&A, Andrew Myers — a computer systems engineer in Berkeley Lab’s Center for Computational Sciences and Engineering and a member of the AMReX core development team — looks at how this unique HPC software framework has influenced and continues to influence, a broad spectrum of scientific applications, both within ECP and outside the ECP program.
How does the AMReX code help researchers at the exascale level, and what makes it unique among adaptive mesh refinement codes?
AMReX is a framework for performing block-structured adaptive mesh refinement calculations. It provides a set of multi-level, distributed data containers for mesh and particle data and handles things like parallel communication, GPU offloading, and inter-level operations so that applications don’t have to. It also has support for complex geometry through embedded boundaries and a variety of linear solvers.
I think a strength of AMReX is our readiness to adapt and to add functionality to support the needs of applications. We are always interested in growing our user base by supporting cool new application codes, and if there is functionality that doesn’t exist right now that would be generally useful, we are interested in helping you build it.
What applications is it most widely used for at present, and why?
AMReX (and its predecessor, BoxLib) have a long history with reacting flows in both the low-Mach-number and highly-compressible limits, and it supports a lot of research along those lines in combustion applications and in astrophysics. We also have a lot of experience with particle-mesh techniques in various contexts, going back to the Nyx code, which uses particles to model parcels of dark matter in an expanding background. Aside from those staples, there is a growing body of other application areas both inside and outside of ECP, ranging from fluctuating hydrodynamics to multi-phase flow problems, electromagnetics for microelectronic circuit design, wind farm modeling, and more.
What ECP projects currently utilize AMReX? Are some of these projects also part of NERSC’s Exascale Science Applications Program (NESAP)?
There are currently codes in six ECP application projects that “fully” use AMReX: Nyx in the ExaSky project, Castro in the ExaStar project, WarpX, MFiX-Exa, AMR-Wind in the ExaWind project, and PeleLM and PeleC in the combustion project. There is also a seventh project that has an additive manufacturing code called Truchas-AM, that we partially support; they only use the linear solvers in AMReX. Of these, WarpX is also a part of NESAP, and it has benefited greatly from the collaboration, particularly in regards to a load-balancing project led by former NERSC postdoc Michael Rowan.
With the advent of Perlmutter and the first exascale systems, how is AMReX adapting to these new GPU-dominant platforms?
At the start of the project, we knew GPUs were important, but we also figured that one of the eventual exascale machines would be a many-core CPU architecture, kind of like Cori KNL or the ARM-based Fugaku machine in Japan. Events unfolded differently, and we ended up needing to accelerate our GPU efforts to prepare for machines like Summit, Perlmutter, and Frontier. Much of the core AMReX framework has been redesigned to work well on GPU-based machines.
A big part of this process was the decision to move away from Fortran towards a pure C++ codebase. We resisted this at first – most of us were big Fortran fans, and in many ways, it is an ideal language for technical computing. But the tools for adding GPU support to a complex framework like AMReX, in our opinion, were better in C++, and with modern compilers that support the __restrict__ keyword the performance edge that Fortran enjoyed for CPU execution was eliminated, so we made the transition.
Compared to the work needed to prepare for Summit, gearing up for Perlmutter and Frontier has been relatively pain-free. Part of that has certainly been due to the expertise and computing——— resources made available to us through NERSC and the NESAP program.
How has AMReX influenced the development of WarpX?
AMReX and WarpX have really grown up alongside each other. We have a tightly-coupled development model (I personally split my time 50-50 between both codes), where the needs of WarpX drive AMReX development, and things we develop for WarpX that we think will be useful for other applications end up getting migrated to AMReX. For example, there is a function parser that was developed for WarpX by Weiqun Zhang, another AMReX developer, that performs the run-time evaluation of mathematical expressions written in plain text by WarpX users on both CPUs and GPUs, without any need to recompile either WarpX or AMReX. This is really hard to do well, especially with GPUs (my advice on this, by the way, was “don’t do that” — I guess I was wrong!) This was recently migrated from WarpX to AMReX so that other application codes can take advantage of it.
What are some tangible benefits that have come out of the push to exascale?
I think an under-rated success story of the ECP has been the way the technical advances made to the DOE’s scientific software stack have had spin-off benefits for smaller-scale scientific computing. The exascale-driven push towards GPU computing has benefited AMReX users who have no intention of ever running on a large fraction of Perlmutter or Summit. A number of academic groups contact us wanting to re-factor their existing simulation codes to use AMReX, both for its ability to support adaptive mesh refinement and for its performance portability across architectures. More locally, this past summer two undergraduate interns, Amanda Harkleroad and Emily Bogle, working with recent UC Berkeley graduate Victor Zendejas Lopez, helped build an AMReX-based code to model the growth of cancer cells. Due to improvements made to AMReX as part of the ECP, they can write code once and it will run on both commodity CPU and GPU hardware. I think that’s pretty cool.
HONORS AND AWARDS
Soren Prestemon Elected as APS Fellow
Soren Prestemon. Credit: Berkeley Lab/Paul Mueller
Soren Prestemon, ATAP’s Deputy Division Director for Technology, has been elected a Fellow of the American Physical Society.
He was nominated by the APS Division of Physics of Beams “for multiple, significant contributions to the research and development of high performance resistive, permanent magnet, and superconducting magnet systems for science applications from light-sources to high-energy physics.”
Prestemon heads our Superconducting Magnet Program. He also serves as Director of the multi-institutional U.S. Magnet Development Program and of the ATAP-Engineering Division Berkeley Center for Magnet Technology.
“I appreciate Soren’s leadership over many years in the Laboratory and community, establishing advanced magnets as a key driver of future HEP capabilities and also of the FES long range plan for fusion,” said ATAP Division Director Cameron Geddes.
Prestemon is the 30th member of ATAP and its predecessor organizations to receive this honor.
He is one of five Berkeley Lab researchers elected to APS Fellowship this year. Wanli Yang of the Advanced Light Source was also honored, and Naomi Ginsberg, Lane Martin, and Kristin Persson were nominated through UC-Berkeley. The honor recognizes exceptional contributions to the physics enterprise.
Chad Mitchell Receives Berkeley Lab Director’s Award
Chad Mitchell
Chad Mitchell, a staff scientist in ATAP’s Accelerator Modeling Program, was presented the Berkeley Lab Director’s Award in the category of Scientific Achievement in a virtual ceremony November 18, 2021.
He leads a team that developed novel beam dynamics methods to significantly extend the reach of future intensity-frontier particle accelerators for high energy physics discovery science, as well as for practical applications of high-intensity (multi-megawatt) proton beams.
A Zoom recording of the awards ceremony is available online (Berkeley Lab login credentials required); Mitchell is presented with the award by Natalie Roe, Associate Laboratory Director for Physical Sciences, beginning at 54:36.
WORKPLACE LIFE
Get Outside! And Other Tips to Stay Healthy and Well
Working late? Why not make time for a sunset. (Photo courtesy Asmita Patel)
The Lab’s Bay Area location includes an unparalleled abundance and variety of outdoor recreation opportunities. Even spending some time in your front yard or at a neighborhood park can be great for your physical health and your state of mind.
Suggestions for this and much more can be found on a new website, “Healthy and Well at LBNL.” Three Berkeley Lab organizations — the Health Services Group, the Human Resources Division, and the Inclusion, Diversity, Equity, and Accountability (IDEA) Office — have joined forces to bring us tips, links, and pointers to resources on physical well-being, work-life balance, and much more.
A Season for Sharing through the Food Bank Challenge
Sharing our good fortune
With Federal pandemic benefits expired and the cost of living on the rise, food banks in our area have seen a remarkable rise in already-high demand. We can help.
Through Friday, January 7, Berkeley Lab is holding the 2021 Winter Food Bank Challenge, benefitting the Alameda County Community Food Bank and the Food Bank of Contra Costa and Solano Counties. To inspire a little friendly competition, the Lab is organized into six teams. The Physical Sciences Area, which includes ATAP, is teamed up with the Projects and Infrastructure Modernization Division.
You can help the less fortunate by donating to either or both of these food banks through our team’s links by January 7, or by volunteering at those food banks and reporting your time. Visit the 2021 Winter Food Bank Challenge website for more information. Let’s beat last year’s $135,000 total
Lactation Accommodations
A room is available on request in B71
A room is available on request in B71
Berkeley Lab recently rolled out a new policy on lactation accommodations. It was drafted by Human Resources and improved based on feedback from the Lab’s Women Scientists and Engineers Council (WSEC) policy subcommittee.
A number of dedicated lactation rooms at the Lab can be reserved through Google Calendar. All these official lactation rooms are equipped with a hospital-grade pump. For more information (and a site with a great deal of other helpful information for parents), visit https://myfamilyberkeleylab.lbl.gov/expectant-nursing-mothers.
In Building 71, we will make arrangements for an office as an unofficial lactation room. To obtain a key and directions, please contact ATAP Division Administrator Wes Tabler or Outreach/IDEA coordinator Ina Reichel. We will provide door hanger signs to show that the assigned office is in use for this purpose.
INCLUSION, DIVERSITY, EQUITY AND ACCOUNTABILITY (IDEA)
Our communication choices can help make the 60 million adult Americans who report having one or more disabiities feel respected and included. (Illustration courtesy CDC)
Our choice of words, and of the ideas we express with them, can either uplift others or cause hurt and reinforce stereotypes. Disability, a widespread aspect of the human condition, is no exception to this principle. The Centers for Disease Control and Prevention have developed guidelines for understanding, dignity, and respect when Communicating With and About People with Disabilities.
Key takeaways
• People-first language is the best place to start when talking to or about a person with a disability.
• If you are unsure, ask the person how they would like to be described.
• It is important to remember that preferences can vary.
PUBLICATIONS AND PRESENTATIONS
Advanced Light Source Accelerator Physics
M. Ehrlichman, T. Hellert, S.C. Leemann, G. Penn, C. Steier, C. Sun, M. Venturini, and D. Wang, “Three-dipole kicker injection scheme for the Advanced Light Source upgrade accumulator ring,” accepted by Physical Review Accelerators and Beams, preprint online 29 November 2021.
Accelerator Modeling Program
Olga Shapoval, Remi Lehe, Maxence Thévenet, Edoardo Zoni, Yinjian Zhao, and Jean-Luc Vay, “Overcoming timestep limitations in boosted-frame particle-in-cell simulations of plasma-based acceleration”, Physical Review E 104, 055311 (29 November 2021), https://doi.org/10.1103/PhysRevE.104.055311
D. A. Bizzozero, J. Qiang (LBNL); L. Ge, Z. Li, C. Ng, L. Xiao (SLAC), “Multi-objective optimization with an integrated electromagnetics and beam dynamics workflow,” Nuclear Instruments & Methods in Physics Research A 1020, 165844 (21 December 2021), https://doi.org/10.1016/j.nima.2021.165844
J.-L. Vay, A. Huebl, R.Lehe (LBNL); N.M. Cook (RadiaSoft LLC); R.J. England (SLAC); U. Niedermayer (TU Darmstadt); P. Piot (Northern Illinois University and ANL); F. Tsung (UCLA); D. Winklehner (MIT), “Modeling of Advanced Accelerator Concepts,” ICFA Beam Dynamics Newsletter, Journal of Instrumentation 16, T10003 (11 October 2021), https://doi.org/10.1088/1748-0221/16/10/T10003
D. Sagan (Cornell University); M. Berz (Michigan State University); N.M. Cook (RadiaSoft LLC); Y. Hao (BNL); G. Hoffstaetter (Cornell University); A. Huebl (LBNL); C.-K. Huang (LANL); M.H. Langston (Reservoir Labs Inc.), C.E. Mayes (SLAC); C.E. Mitchell (LBNL); C.-K. Ng (SLAC); J. Qiang, R.D. Ryne (LBNL); A. Scheinker (LANL); E. Stern (FNAL); J.-L. Vay (LBNL); D. Winklehner (MIT); H. Zhang (TJNAF); “Simulations of Future Particle Accelerators: Issues and Mitigations”, ICFA Beam Dynamics Newsletter, Journal of Instrumentation 16, T10002 (11 October 2021), https://doi.org/10.1088/1748-0221/16/10/T10002
A. Myers, A. Almgren, L. D. Amorim, J. Bell (LBNL); L. Fedeli (CEA Saclay); L. Ge (SLAC and LBNL); K. Gott (LBNL); D.P. Grote (LLNL); M. Hogan (SLAC); A. Huebl, R. Jambunathan, R. Lehe (LBNL); C. Ng (SLAC); M. Rowan, O. Shapoval (LBNL); M. Thévenet (DESY); J.-L. Vay (LBNL); H. Vincenti (CEA Saclay); E. Yang (LBNL); N. Zaim (CEA Saclay); W. Zhang, Y. Zhao and E. Zoni (LBNL), “Porting WarpX to GPU-accelerated platforms”, Parallel Computing 108, 102833 (14 September 2021) https://doi.org/10.1016/j.parco.2021.102833
Lipeng Wan (ORNL); Axel Huebl, Junmin Gu (LBNL); Franz Poeschel, CAUSUS);
Ana Gainaru, Ruonan Wang, Jieyang Chen (ORNL); Xin Liang (Missouri University of Science and Technology); Dmitry Ganyushin (ORNL); Todd Munson, Ian Foster (ANL); Jean-Luc Vay (LBNL); Norbert Podhorszki (ORNL); Kesheng Wu (LBNL); Scott Klasky (ORNL), “Improving I/O Performance for Exascale Applications through Online Data Layout Reorganization,” IEEE Transactions on Parallel & Distributed Systems 33, 878-890 (April 2022), early access (December 2021), https://doi.ieeecomputersociety.org/10.1109/TPDS.2021.3100784
L. Fedeli, A. Sainte-Marie, N. Zaim (CEA Saclay); M. Thévenet (LBNL, now DESY); J.L. Vay, A. Myers (LBNL); F. Quéré, and H. Vincenti (CEA Saclay), “Probing strong-field QED with Doppler-boosted petawatt-class lasers”, Physical Review Letters 127, 114801 (10 September 2021,) https://doi.org/10.1103/PhysRevLett.127.114801
Invited talks without publication opportunity
A. Huebl, “Computational Frontier Status”, 2021 AF1 Community Meeting, Nov. 22-23, 2021.
J. Qiang, “Envelope instabilities and their mitigation in high intensity hadron beams” 64th ICFA beam dynamics workshop on High Intensity and High Brightness Hadron Beams, Fermilab, Chicago, Oct. 3-8, 2021.
A. Huebl, “openPMD – Scientific, Community Meta-Data Standard”, Workshop on SAXS@XFELs and HI & HE laser driven matter, Nov 4-5, Dresden, Germany.
BELLA Center
D.P. Grote, A. Friedman, C.G.R. Geddes, R. Lehe, C. Benedetti, T. M. Ostermayr, H-E Tsai, J.-L. Vay, C.B. Schroeder, E. Esarey, “Reduced bandwidth Compton photons from a laser-plasma accelerator using tailored plasma channels,” Physics of Plasmas (accepted).
A. Zingale, N. Czapla, D.M. Nasir (The Ohio State University); S.K. Barber, J.H. Bin, A.J. Gonsalves, F. Isono, J. van Tilborg, S. Steinke, K. Nakamura (LBNL); G.E. Cochran (LLNL); J. Purcell (The Ohio State University); W.P. Leemans (LBNL, now at DESY); C.G.R. Geddes, C.B. Schroeder, E. Esarey (LBNL); D.W. Schumacher (The Ohio State University), “Emittance preserving thin film plasma mirrors for GeV scale laser plasma accelerators,” Physical Review Accelerators and Beams 24, 121301 (2 December 2021); https://doi.org/10.1103/PhysRevAccelBeams.24.121301
SAFETY: THE BOTTOM LINE
Safety Week Awards Honor Top Performers
The success of Physical Sciences Area Safety Week is built on an all-hands effort, but each year, some stand out for exceptional dedication and the quality of their contributions.To promote ongoing safety-culture awareness and maintain the momentum of Safety Week, we honor an outstanding individual worker and the top-performing self-assessment team for outstanding safety and organization effort.
Individual Award: Wes Tabler
Wes Tabler
“Most helpful to others”
ATAP Division Administrator Wes Tabler received the individual award, honoring the person most helpful to others in Safety Week efforts. He went the extra mile in contributions throughout the entire Safety Week cycle, from helping make logistics arrangements and assignments beforehand, through ensuring good stewardship of tagged property during the event, to helping with follow-up on the matters that required additional planning and resources.
Team Award: HL-LHC AUP
Team honors go to the assessment team for the High-Luminosity LHC Accelerator Upgrade Project for their performance in work-area cleanup and assessment and their commitment to workplace safety. Their task involved Building 77A, where a wide variety of magnet structure and assembly and cable winding tasks are underway. They were selected for this honor by division management, after final walkthroughs and reports from all the teams. Team members included Ahmet Pekedis, Al Baskys, Andy Lin, Carlos Perez, Elizabeth Lee, Josh Herrera, Katherine Ray, Matt Reynolds, Juan Rodriguez, Mike Naus, and Mike Solis, led by Dan Cheng and Ian Pong.
Team members included Ahmet Pekedis, Al Baskys, Andy Lin, Carlos Perez, Elizabeth Lee, Josh Herrera, Katherine Ray, Matt Reynolds, Juan Rodriguez, Mike Naus, and Mike Solis, led by Dan Cheng and Ian Pong.
These honors were the result of pleasantly difficult decisions in assessing the excellent contributions of so many people across the Division. Let’s maintain the momentum on Safety Week’s themes of clean and well-organized workspaces, diligent chemical stewardship, and up-to-date training, and have another year of doing great science safely.
Safety Week Photostory
The Physical Sciences Area held Safety Week September 27 – October 1, 2021. The popular tradition began as Safety Day, an ATAP initiative spearheaded by Deputy Division Director Asmita Patel and Safety Coordinator Pat Thomas. The Engineering Division soon joined with us, and the Physics Division and the ALS-U Project followed suit.
To see more highlights, visit our Safety Week photostory.
Integrated Safety Management and the Hidden Dangers of Quotidian Risks
The circle of safety
“ISM 101,” a 20-minute online course, was developed by the Engineering Division to help us all understand the core principles of Integrated Safety Management (ISM) and how its five steps help ensure our own safety — and the safety of those around us.
The course emphasizes the normalized risks that hide beneath a cloak of familiarity and complacency in routine work, and how the principles of ISM apply to everything we do. It is a good refresher for everyone and especially recommended for those new to the Lab.
Putting Out the Unwelcome Mat for Four- and Six-Legged Holiday Guests
At this time of year, mice and rats often try to come indoors. Rodents can leave messes and spread diseases… and they are notorious for gnawing on electrical wiring. Ants, too, can invade over the holidays; interior and upper-storey offices are not safe. Here are some tips on how to be a less inviting holiday host as we leave the Lab for the year-end shutdown.
• Keep nonperishable snack foods and earthquake supplies in a closed container they cannot easily gnaw though, not loose in a desk drawer or backpack.
• Remove any food items at the end of each day, and be sure that your under-desk trash can (if you use one) is free of food scraps.
• Don’t leave perishables in breakroom refrigerators during the break. Bacteria and mold won’t be taking a holiday!
• Wipe down desktops, tables, and counter tops after food has been consumed. Mere scraps and crumbs by our standards can be a feast for something the size of a mouse, never mind an ant.
• Be sure that the last of the trash bags go to outside dumpsters before a building is left unoccupied over the break.
If you do see rodents or other pests, contact your Building Manager or the Work Request Center to request Pest Control services.
Do not attempt to clean up dead rodents or droppings. Contact your custodians. They are trained in how to clean up safely.
Let’s Think On Our Feet This Winter
Don’t let a happy holiday slip away! Movie courtesy Horst Simon, emeritus LBNL Deputy Director for Research, and Sponsor, Berkeley Lab Safety Culture Work Group. Poster by Lucky Cortez, UCSF, formerly of the ATAP Operations Team.
As we approach the longest night of the year here in the Northern Hemisphere, with rain often in the picture, leaves on the sidewalks and stairways, and holiday decorations almost within easy reach, this is a crucial time for slip, trip, and fall prevention both at the Lab and at home.
Let’s walk (and drive and bike) mindfully, which includes paying attention to the physical world around us rather than the virtual world in handheld devices.
Use handrails on stairways and steep walkways, and clean up slippery spills before others come upon them unawares.
Using ladders properly is important at any time of year. For more information on safe use of ladders and stepstools, contact Alyssa Brand, ext. 7246.
Further tips to keep easily prevented accidents from spoiling the festivities are available from the National Safety Council and the US Fire Administration.
Here’s wishing you and yours a happy holiday season and a safe return in the new year!
Follow us on social media
