Sprecher
Beschreibung
In this talk, we discuss a deep learning model approach that uses messy data to learn the mapping between experimental parameters -> electron spectra. Many laser facilities, e.g. ZEUS at University of Michigan, have pre-existing operational procedures that produce "real-world” datasets where data are recorded manually and with assumptions and omissions. These do not necessarily provide clean and structured data to enable machine learning and often, the first step is to "clean" these data. However, data cleaning is often a bespoke procedure that can be cumbersome.
We circumvent (a significant part of the) data cleaning step by using open-source pre-trained Large Language Models in our deep learning model pipeline. The LLM can provide embeddings which effectively translate natural language to a semantically informed numerical representation. To map the numerical representation of the experimental parameters to observed electron spectra, we train convolutional neural networks. This results in a deep learning model that is a combination of a pre-trained LLM that feeds into an untrained convolutional neural network. Early training results are promising and suggest that this approach can be an effective method by which to work with experimental parameters that are challenging to represent numerically in a structured fashion. Because we put the pre-trained LLM directly into the model pipeline rather than calling a web-API, we can also utilize gradients acquired from automatic differentiation to perform sensitivity analysis as well as gradient-based optimization.
