Advanced Accelerator Simulation
The Fermilab Computing Division's Advanced Accelerator Simulation project is part of the ComPASS effort under the SciDAC-2 program. SciDAC-2 is a Department of Energy sponsored research and development initiative to bring advanced computing resources to bear on important problems in science. The primary objective of the ComPASS project within SciDAC-2 is to establish a comprehensive petascale simulation environment needed to solve the most challenging problems in 21st century accelerator science and technology.
Particle Beams and Accelerators
A particle accelerator is a complex apparatus used to create, capture, and accelerate large ensembles of particles with coordinates that move in close proximity. Such particle ensembles are called beams. Particle accelerators have helped enable some of the most remarkable scientific discoveries of the 20th century. They have also led to substantial advances in applied science and technology, many of which greatly benefit society. Accelerator applications include High Energy Particle Physics, where particle beams are utilized to study the fundamental properties of matter, space, and time, and materials and biological sciences. Accelerators applications under development address issues that include environmental and energy concerns through projects such as the accelerator transmutation of waste.
Beam Physics
Beam Physics studies the behavior of particle beams in accelerators, and provides the necessary tools to design and optimize their performance. To first order, Beam Physics is very similar to optics, since the active components of an accelerator are electromagnetic structures which act as lenses to guide the particle beams. Although the mathematical framework for modeling such a system is very elaborate, the modeling process is not very computationally intensive. This picture changes dramatically when the forces created by the beam itself are taken into account. An example of such force is the repulsive force due to the fact that a large number of particles of the same charge are confined at a small region of space at the same time. With modern accelerators using beams of tens of trillions of particles, modeling these multi-particle effects is very important and very computationally intense.
Click here for a movie of a Tevatron Beam-Beam interaction!
High Performance Computing
To accurately model multi-particle effects in a self-consistent way, it is necessary to utilize the power of parallel computing. We run our modeling codes on parallel clusters located at Fermilab and at the National Energy Research Scientific Computing Center (NERSC). Synergia is a parallel three dimensional modeling code for multi-particle effects developed at Fermilab through the Advanced Accelerator Modeling project. BeamBeam3d is a parallel 3D modeling code developed at LBNL tailored for studying beam-beam collisions that has been enhanced at Fermilab for modeling the Tevatron. The figure below shows the simulation speed of BeamBeam3d on various parallel clusters.
The pcac1 cluster is a local linux cluster consisting of 48 dual-CPU 2.4 GHz Xeon® compute nodes with a myrinet® low-latency switch fabric interconnect. The pcac2 cluster is a newer local linux cluster containing 20 dual-CPU 3.2 GHz Xeon® compute nodes also using a myrinet interconnect. Seaborg (now retired) was a parallel cluster at NERSC based on 0.375 GHz IBM Power3 processors. Bassi is a newer cluster at NERSC based on 1.9 GHz Power5 processors. Franklin is the new Cray XT4 cluster consisting of 9660 dual-processor 2.6 GHz Opterons. BG/L is a 1024 node cluster of dual-CPU 0.7 GHz IBM PowerPC processors at the Argonne Leadership Computing Facility.
Synergia simulation performance versus number of processors is shown below.
