3D Potential Through Laser Annihilation

Today we recognize the laser is reality beyond science fiction, used in targeted surgeries, precision manufacturing and in the exploration of materials at the nanoscale. Yet, harnessing the once-fabled destructive capabilities of certain lasers is proving invaluable on the path toward scientific discovery.

The x-ray electron-free laser (XFEL) is the perfect example of new technology and old perceptions converging on that narrow boundary between science and science fiction. Firing pulses of a trillion x-ray photons at molecular-sized samples in time scales on the order of million-billionths of a second (femtoseconds), researchers are aiming for the Holy Grail of ultra-fast X-ray Science – single-particle 3D imaging with atomic resolution.

Understanding the effects that these ultra-intense x-ray pulses will have on their potential targets will take the team work of Argonne National Laboratory’s Advanced Photon Source (APS) and the Argonne Leadership Computing Facility (ALCF), both of which are U.S. Department of Energy (DOE) Office of Science User Facilities.

But first, many atoms and molecules will have to meet with a sci-fi appropriate demise. And the ability to capture and decipher the innumerable quantum interactions that will occur on ridiculously small time scales will require the calculating power of ALCF’s IBM Blue Gene/Q supercomputer, Mira, running an advanced hybrid simulation program to sort it all out.

“There is a dream that you can get an atomic-resolution 3D structure of proteins and other substances that are important in understanding biological mechanisms without crystallization by simply throwing an ultra intense burst of x-rays onto the sample and interpreting the diffraction patterns they create,” says Linda Young, director of APS’s X-ray Science Division (XSD).

Diffraction patterns are created when x-ray photons collide with the electrons of a target sample—a specific atom or enzyme molecule, for instance—and scatter. These scatterings are captured as images by photon detectors inside the machine.

From the dizzying cascade of lines and flourishes that constitute these patterns, there emerges the information necessary to detect the electron locations of the sample before it was irradiated, and thus work backward to discern its 3D structure.

“This information can be used to create a ‘movie’ that can replay the whole annihilation process during and after the XFEL pulse at a rate of 10^18 frames per second. With this movie, we can observe the impact of the pulse on a sample and provide an accurate interpretation of the data recorded in diffraction patterns,” explains Phay Ho, an assistant physicist with APS.

The photon/electron collisions create infinite and simultaneous quantum reactions, where electrons emerge and disappear and new particles propagate, all of them creating those frantic lines etched on the detectors. To read between the lines, quite literally, Young and Ho work closely with computational scientists at the ALCF to optimize their method within a molecular simulation program called LAMMPS.

The team uses a hybrid code employing both molecular dynamics (MD) and Monte Carlo (MC) algorithms. Where the MD tracks the time evolution for all the particles in the system, MC incorporates detailed information from quantum mechanics to simulate the interactions between the electrons and the XFEL pulses. “So MC takes all of the complicated quantum mechanics and recasts it in a simpler way,” says ALCF assistant computational scientist, Chris Knight.

Unlike typical molecular simulations, the XFEL studies are computationally more intensive. The blast from the intense x-ray pulse produces more than a 10-fold increase in the number of particles, which are generated on relatively faster timescales. And the processes which occur during and after the bombardment lead to rapid expansion of the system size—or the simulated playing field on which this all takes place.

But rather than try to calculate every electronic structure and excited particle during a simulation, the MC algorithm uses a pre-computed database to update and track the electronic configuration of every particle interacting with an x-ray pulse. Even with a computational cost significantly smaller than fully quantum mechanical simulations, some unique computational challenges remain before the team can exploit the full potential of the hybrid method.

Owing to the modular nature of LAMMPS and the close collaboration between ALCF and APS, improvements to the code have increased efficiency nearly three-fold, reducing the time it takes to run the program on the time and space scales required for such rapid expansion. This speedup was achieved with improved multi-threaded parallelism, more efficiently balanced workloads across many processors, and optimized I/O. A key result is that the time spent in MC was reduced from 60 to less than 10 percent of the hybrid simulation’s runtime, making it nearly as fast as normal MD simulations.

“These simulation programs record a large amount of information relating to the electronic structure of the material, as well as positions, velocity and constituents with a time resolution on the order of attoseconds (one-thousandth of a femtosecond),” says Ho. The continued collaboration with ALCF to make these simulations faster will enable simulations of larger, more realistic systems for the millions and billions of timesteps required to model the types of large-scale processes studied at APS.

The team continues to tweak the hybrid code as well as pulse rates by studying Argon clusters composed of 20 thousand to 2 million particles, and nano-diamond materials composed of 1-100 million particles, with an end goal of mapping the electron pathways created by XFEL bursts.

According to Young, small bursts produce very high-resolution scattering patterns, while large bursts create radiation damage, causing smeared patterns and lower resolution.

All of the work with the XFEL was performed at the Linac Coherent Light Source (LCLS) at Stanford University’s SLAC National Accelerator Laboratory, which provides a billion-fold more peak intensity than any other x-ray source in the world.
While the APS’s own synchrotron is a powerful source for high-energy x-ray beams, the APS will not conduct single-shot single-particle imaging studies, though the question of radiation damage still will apply. Because synchrotron pulses are longer, computational efforts will have to propagate what happens during a short burst, like those of the XFEL, to longer time scales.

“This code will have to be very powerful to be able to do that,” says Young. “Not only do you have to follow what happens on a nanoscale, you have to follow it out to mesoscale. So it’s even more complicated at the APS, but the codes developed here will be very relevant.”

Argonne National Laboratory seeks solutions to pressing national problems in science and technology. The nation's first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance America's scientific leadership and prepare the nation for a better future. With employees from more than 60 nations, Argonne is managed by UChicago Argonne, LLC for the U.S. Department of Energy's Office of Science. DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.

Contact Information
Brian Grabowski
Communications Lead
bgrabowski@anl.gov
Phone: 630-252-1232