Last month took place one of the most important conferences for the High Energy Physics (HEP) community in Computing: the so-called CHEP 2023, standing for Computing on High Energy Physics and Nuclear Physics — yes, simple! 🙂
As a Computer Engineer working at CERN, it is a major event: it is the opportunity to see the trend of the latest technologies in our field. Nevertheless, although I was fully aware of the current popularity of ChatGPT, I was not expecting to find any talks on the subject. But I was totally wrong, indeed there were a couple!
I found them very appealing, so in this article, I would like to depict the main take home-messages of such talks. ChatGPT is not only reshaping our daily tasks but also major research areas such as the HEP one.
Let’s explore what is coming!
The HEP community refers to the global network of scientists, researchers, engineers, technicians, and institutions involved in the field of High Energy Physics. This community is dedicated to the study of the fundamental constituents of matter, the forces that govern their interactions, and the exploration of the fundamental laws of the universe.
CHEP is a series of conferences that focus on the use of computing, software, and data management in the field of HEP — and Nuclear Physics too.
Actually, CHEP is quite old conference. The first one took place in 1985, and since then, it has been organized biennially. Overall, CHEP conferences play a crucial role in driving advancements in computing and data management.
CHEP serves as a platform for knowledge exchange, collaboration, and the exploration of new computing techniques. That is why I was actually that surprised: if something appears at CHEP, it is likely to be an incoming trend! And in this last CHEP 2023, we had two plenary sessions about ChatGPT in HEP.
Ready?
The first plenary session about ChatGPT came very early in the schedule by David Dean from the Jefferson Lab. Titled Evolution and Revolutions in Computing: Science at the Frontier, David provided a vast overview of the latest revolutions in computing. And fear not, ChatGPT was one of them!
He concretely targeted the question around if ChatGPT can do physics, and the message was clear: it is a mind-blowing tool that can pass physics exams too, but there is a major flaw that may stop ChatGPT to be incorporated as a tool in the near future: Model Hallucinations.
Model Hallucinations
Despite the model’s capabilities to retrieve human-like responses, there are moments where it maintains a tendency to make up facts, to double down on incorrect information, and to perform tasks incorrectly. Those incorrect responses are known as hallucinations.
In fact, giving incorrect answers is not the problem itself. The main issue is that ChatGPT often exhibits these tendencies in a convincing and authoritative manner. Hallucinations are sometimes even present in the form of highly detailed information, giving a wrong sense of accuracy to the reader, and increasing the risk of overreliance. And that is definitely a problem in the research community.
In order to use ChatGPT as a trustful helper tool, hallucinations need to be controlled. Currently, ChatGPT will try to provide an answer to any of the given queries, even if it has not enough information about the target topic.
There should be nothing bad about ChatGPT admitting it is not able to provide an accurate response to a given prompt and it would make the tool more suitable in accurate environments such as in the HEP research.
The second plenary session touching on ChatGPT was entitled Radically different futures for HEP enabled by AI/ML, given by Kyle Crammer from the University of Wisconsin-Madison.
This second talk was more optimistic about the introduction of ChatGPT as a valuable asset in the HEP toolkit. In fact, Kyle referenced another talk from Christian Weber from the Brookhaven National Laboratory in which he presented real use cases of ChatGPT as a coding assistant, especially to migrate and convert code to new platforms. In fact, ChatGPT already implements a Python interpreter for coding purposes.
Each experiment in the HEP community has its own coding templates, i.e., although coding in Python, scientists must stick to some classes or style conventions. One of the use cases presented was to fine-tune ChatGPT to write analysis code based on the experiment template.
Attracted by this use-case, I tried to generate a template for an analysis on my current experiment, the CMS Experiment at CERN, Switzerland, and ChatGPT perfectly generated a first template. And I was simply using the web interface, imagine how powerful it could be after fine-tuning it with relevant data.
According to the presentation, even if sometimes the analysis was not accurate enough, it allowed for generating a first template or backbone for the analysis. This idea was explored to provide faster onboarding for new experiment members and build prototypes faster, among other use-cases.
We cannot deny that Large Language Models (LLMs) such as ChatGPT are changing our way to search for information, build applications, and even coding.
As with any advancement in technology, I think it is reasonable to evaluate any new tool in order to leverage its benefits and apply them to our main fields. These two plenary sessions are just two examples of this evaluation process in a big research community such as the HEP one.
While some evaluations may discard ChatGPT for the time being as a research helper, others may allow the incorporation of such tools in concrete and delimited domains. In any case, I believe it is important to not fear AI and continue to evolve with it, analyzing its benefits, knowing how to optimize its performance for your target domain, and much more importantly, being aware of the flaws to keep the critical spirit always alert!
