Scientists Stress Test OpenAI Models to Accelerate Scientific Discovery Across U.S. National Labs
From simulating diffraction patterns to optimizing X-ray data workflows, Brookhaven scientists joined 1,600 peers nationwide to test OpenAI’s latest AI models, probing how these tools could supercharge science across disciplines and pave the way for deeper research collaborations.
In February, the U.S. Department of Energy (DOE), in cooperation with OpenAI, an American artificial intelligence research company, launched the first-of-its-kind 1,000 Scientist AI Jam Session. As one of nine host sites throughout the DOE national laboratory complex, DOE's Brookhaven National Laboratory saw more than 120 scientists challenge and evaluate the capabilities of OpenAI's latest step-based logical reasoning AI model built for complex problem solving. More than 1,600 scientists across the DOE complex participated in the event. Such partnerships between the DOE labs and industry are increasingly important for maintaining the nation's technological edge, especially in the globally competitive AI research area.
"One of our country's greatest assets – and an envy of the world – is the Department of Energy's network of national laboratories, which for decades have driven breakthroughs in science and technology, strengthened national security, and fueled American prosperity," said U.S. Secretary of Energy Chris Wright who attended the event at DOE's Oak Ridge National Laboratory. "Like the Manhattan Project, which brought together the world's best scientists and engineers for a patriotic effort that changed the world, AI development is a race that the United States must win. Today's collaboration of America's national labs and technology companies is an important step in our efforts to secure America's future."
Brookhaven's Computing and Data Sciences (CDS) Directorate
From Brookhaven's Computing and Data Sciences (CDS) Directorate training room, researchers representing the Lab's many mission-critical research areas interacted with OpenAI's most advanced generative pre-trained Transformer, or GPT, models, including the o3-mini-high and o1 pro, to ascertain their viability as useful "research assistants." OpenAI's models, such as the popularly used ChatGPT, are large language models (LLMs) trained on massive datasets to understand human language nuance. This enables them to be responsive to various subjects, generating text or other content depending on user prompts. The "o" series models used for the AI Jam were essentially "stress tested" on their multi-step problem-solving and code-generation abilities, especially when applied to diverse science.
"Products like ChatGPT, general-purpose LLMs as a service, have significantly changed the perception of and access to state-of-the-art AI methods for people who are not AI experts," said Carlos X. Soto, the AI theory and security group lead at CDS, who participated in the AI Jam. "They have also lowered the barrier to entry of applying AI to scientific problems, which is part of what the AI Jam Session was about. Several domain scientists at Brookhaven Lab and elsewhere have already been applying these tools in their work, but the Jam Session offered a guided approach to doing so with the latest models, particularly the latest simulated reasoning models, which are expected to be more useful for scientists."
National Synchrotron Light Source II (NSLS-II)
Scientists from the National Synchrotron Light Source II (NSLS-II), a DOE Office of Science user facility at Brookhaven Lab, used the AI Jam Session to explore how the models could help solve a variety of complex problems, especially in ways that would save time, optimize data analysis, and analyze detailed simulations. Their projects spanned biological sample imaging, spectroscopy data analysis, workflows to facilitate multimodal research, experiment automation, and even calculations for potential accelerator upgrade designs.
