SOFIE (System for Optimized Fast Inference code Emit) is CERN's inference engine inside ROOT / TMVA. You hand it a trained ML model, and instead of shipping a heavy runtime, it emits the exact standalone C++ needed to run that specific model, depending only on BLAS for the matrix math. That makes it small and predictable enough to drop into the tight environments of LHC experiments, including the trigger systems that decide, in real time, which collisions are worth keeping.
Why code generation, and not just PyTorch
Inference in high-energy physics has an unusual constraint profile: high throughput, low latency, strict memory limits, and a hard requirement for long-term maintainability with as few dependencies as possible. General frameworks carry runtimes that are awkward to deploy and maintain in that setting. SOFIE's answer is to write the code rather than interpret a graph at runtime. On CPU, recent work pushes SOFIE past ONNX Runtime on several real LHC models, through optimized kernels for common ops, better memory reuse, and improved dynamic-tensor support, while staying extremely lightweight.
The heterogeneous part (my GSoC work)
The High-Luminosity LHC will lean much harder on co-processors for anything that parallelizes well, and ML inference is a prime candidate. But heterogeneous hardware brings its own pain: non-uniform memory formats, diverse inference configurations, and costly data movement between host and device. My contribution was making SOFIE's generated code run across these backends without rewrites. SOFIE can now generate C++ that manages its memory through alpaka data buffers, so the same generated model runs on multiple accelerator backends with minimal modification and stays architecture-agnostic. Because the emitted code no longer bakes in a device, it slots into existing HEP workflows far more easily.
That left one portability leak. SOFIE relies on BLAS for optimized matrix operations, but each vendor ships its own BLAS with limited portability, which quietly breaks the write-once-run-anywhere promise. So we added sofieBLAS, a thin abstraction layer that exposes a single unified BLAS interface and selects the right backend at runtime. You keep vendor-optimized performance without hard-coding the vendor into the generated code. Alongside this I wrote parallel GPU kernels for the common ML operations SOFIE needs.
How it performs
We benchmarked SOFIE's generated code on both CPU and GPU against PyTorch and ONNX Runtime, on models that HEP actually runs in production: ATLAS GN2 (jet-flavour tagging), CMS ParticleNet, and diffusion models for fast detector simulation. The full numbers, on both CPU and GPU, are in the slides.
The CHEP 2026 slides
All 25 slides, scroll straight through them here.
