Analysis Systems
The goal of the Analysis Systems focus area is to develop sustainable analysis tools to extend the physics reach of the HL-LHC experiments by creating greater functionality, reducing time-to-insight, lowering the barriers for smaller teams, and streamlining analysis preservation, reproducibility, and reuse.
Focus Area Strategies:
- Establish declarative specifications for analysis tasks and workflows that will enable the technical development of analysis systems to be decoupled from the user- facing semantics of physics analysis.
- Leverage and align with developments from industry and the broader scientific software community to enhance sustainability of the analysis systems.
- Develop high-throughput, low-latency systems for analysis for HEP.
- Integrate analysis capture and reuse as first class concepts and capabilities into the analysis systems.
Contact us: as-team@iris-hep.org
AS Projects
AS Presentations
- 10 Aug 2020 - "Lectures on Statistics", Kyle Cranmer, 15th joint Fermilab-CERN Hadron Collider Summer School
- 17 Jul 2020 - "Graphs, Trees, and Sets: structured data in physics", Kyle Cranmer, ICML Workshop on Graph Representation Learning
- 17 Jul 2020 - "Likelihood-based models for Simulation-based Inference", Kyle Cranmer, ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models
- 10 Jun 2020 - "Reusable Workflows, active learning, and simulation-based inference", Kyle Cranmer, UCI Symposium on Reproducibility in Machine Learning
- 8 May 2020 - "hep_tables: An Introduction", Gordon Watts, Coffea Developers Meeting
- 7 May 2020 - "How machine learning can help us get the most out of our highest fidelity physical models", Kyle Cranmer, MIT Physics Department Colloquium
- 17 Mar 2020 - "Future Analysis Systems", Kyle Cranmer, Joint US ATLAS - US CMS Meeting on Facility R&D
- 16 Mar 2020 - "Reusable workflows in particle physics", Kyle Cranmer, Workshop on Accelerating Scientific Discovery through Advanced and Automated Workflows
- 5 Mar 2020 - "Artifical Intelligence for High-Energy Physics", Mark Neubauer, Artificial Intelligence for Nuclear Physics Workshop
- 5 Mar 2020 - "Machine Learning and Physics", Kyle Cranmer, Special QU-PCD Colloquium at DESY (CANCELED due to COVID19)
- 3 Mar 2020 - "Machine Learning for Effective Field Theories", Kyle Cranmer, PREFIT20: PRecision Effective FIeld Theory School
- 27 Feb 2020 - "Rethinking final analysis stages (poster)", Alexander Held, IRIS-HEP Poster Session
- 27 Feb 2020 - "Boost-Histogram for Analysis Systems (poster)", Henry Schreiner, IRIS-HEP Poster Session
- 27 Feb 2020 - "Functional Analysis Description Language (FuncADL)", Mason Proffitt, IRIS-HEP Poster Session
- 27 Feb 2020 - "pyhf: A Pure Python Statistical Fitting Library with Tensors and Autograd", Matthew Feickert, IRIS-HEP Postdoc Presentations
[expand]
- 27 Feb 2020 - "pyhf: Pure Python Implementation of HistFactory", Matthew Feickert, IRIS-HEP Poster Session 2020
- 17 Feb 2020 - "Mining for Dark Matter substructure: Learning from lenses without a likelihood", Johann Brehmer, Dark Matter Working Group seminar
- 5 Feb 2020 - "The frontier of simulation-based inference", Johann Brehmer, Workshop on Machine Learning at the LHC
- 28 Jan 2020 - "Python, Numpy, and Pandas", Henry Schreiner, Princeton Research Data Management Workshop 2020
- 28 Jan 2020 - "HEPData and IRIS-HEP", Kyle Cranmer, HEPData Advisory Board
- 6 Jan 2020 - "Normalizing flows and the likelihood ratio trick in particle physics", Johann Brehmer, Deep learning seminar
- 14 Dec 2019 - "Mining gold: Improving simulation-based inference with latent information (poster)", Johann Brehmer, NeurIPS 2019 workshop on Machine Learning and the Physical Sciences
- 5 Dec 2019 - "High-Performance Python: GPUs", Henry Schreiner, Princeton Research Computing Fall Mini-courses and Workshops
- 29 Nov 2019 - "uproot Tutorial", Mason Proffitt, Software Carpentry at CERN
- 25 Nov 2019 - "An introduction to pyhf and HistFactory likelihoods", Matthew Feickert, LHCb Statistics Working Group
- 21 Nov 2019 - "Analysis Tools in the 2020's and 2030's", Gordon Watts, Latin American Workshop on Software and Computing (S&C) Challenges in High-Energy Particle Physics (HEP)
- 20 Nov 2019 - "High-Performance Python: CPUs", Henry Schreiner, Princeton Research Computing Fall Mini-courses and Workshops
- 19 Nov 2019 - "Flows three ways", Kyle Cranmer, Deep Learning for Physics Seminar Series at Princeton Center for Theoretical Physics
- 7 Nov 2019 - "Constraining effective field theories with machine learning", Alexander Held, 24th International Conference on Computing in High Energy & Nuclear Physics
- 7 Nov 2019 - "Recent developments in histogram libraries", Henry Schreiner, CHEP 2019
- 7 Nov 2019 - "HEP Data Query Challenges", Mason Proffitt, CHEP 2019
- 7 Nov 2019 - "Likelihood preservation and statistical reproduction of searches for new physics", Matthew Feickert, CHEP 2019 Conference
- 7 Nov 2019 - "Aligning the MATHUSLA Detector Test Stand with Tensor Flow", Gordon Watts, CHEP 2019
- 5 Nov 2019 - "pyhf: a pure Python implementation of HistFactory with tensors and autograd (poster)", Matthew Feickert, CHEP 2019 Conference
- 5 Nov 2019 - "A Functional Declarative Analysis Language in Python (poster)", Emma Torro, CHEP 2019
- 5 Nov 2019 - "Using Analysis Declarative Languages for the HL-LHC", Gordon Watts, CHEP 2019
- 2 Nov 2019 - "Analysis workflows", Gordon Watts, CHEP 2019
- 29 Oct 2019 - "Harmonizing statistics tools - ideas", Alexander Held, ATLAS Statistics Committee Meeting
- 18 Oct 2019 - "pyhf: pure-Python implementation of HistFactory", Matthew Feickert, PyHEP 2019 Workshop
- 17 Oct 2019 - "Boost-Histogram: Hands-on", Henry Schreiner, PyHEP 2019
- 17 Oct 2019 - "Python Histogramming Packages", Henry Schreiner, PyHEP 2019
- 17 Oct 2019 - "Python 3.8: What's new", Henry Schreiner, PyHEP 2019
- 17 Oct 2019 - "Awkward 1.0", Jim Pivarski, PyHEP Workshop
- 17 Oct 2019 - "Simulation-based inference, interpretability, and experimental design", Kyle Cranmer, Workshop on Interpretable Learning in Physical Sciences Part of the Long Program Machine Learning for Physics and the Physics of Learning
- 16 Oct 2019 - "Lightning Talk: A Living HEP Analysis", Gordon Watts, PyHEP
- 5 Oct 2019 - "Particle Physics in the context of Data Science", Kyle Cranmer, The 6th IEEE International Conference on Data Science and Advanced Analytics
- 30 Sep 2019 - "Run 3/Run 4 Perspectives - Event Delivery Impacts?", Gordon Watts, HSF & ATLAS Joint Event Delivery Workshop
- 30 Sep 2019 - "What does the Revolution in Artificial Intelligence Mean for Physics?", Kyle Cranmer, Joint PITT-CMU Physics Department Colloquium
- 27 Sep 2019 - "Simulation-based inference, causality, and active learning", Kyle Cranmer, AI and the Scientific Method, ETH, Zurich
- 26 Sep 2019 - "Declarative programming: A paradigm shift in data analysis in preparation for the HL-LHC", Gordon Watts, eScience2019
- 14 Sep 2019 - "Jagged, ragged, awkward arrays", Jim Pivarski, Strange Loop 2019
- 13 Sep 2019 - "Histogramming and more", Henry Schreiner, 2019 IRIS-HEP Institute Retreat
- 13 Sep 2019 - "func-adl to C++/xAOD backend", Gordon Watts, IRIS-HEP Institute Retreat
- 12 Sep 2019 - "Boost Histogram Roadmap", Henry Schreiner, 2019 IRIS-HEP Institute Retreat
- 12 Sep 2019 - "Prototype declarative analysis interface using uproot and awkward-array", Mason Proffitt, IRIS-HEP Institute Retreat
- 12 Sep 2019 - "pyhf Roadmap: 2019 into 2020", Matthew Feickert, 2019 IRIS-HEP Institute Retreat
- 12 Sep 2019 - "Analysis Systems team meetings/process efficacy", Ben Galwesky, IRIS-HEP Institute Retreat
- 12 Sep 2019 - "Milestones - We don't need roads...", Gordon Watts, IRIS-HEP Institute Retreat
- 10 Sep 2019 - "The IRIS-HEP Blueprint Concepts and Process", Mark Neubauer, Blueprint Meeting on Fast Machine Learning and Inference
- 9 Sep 2019 - "The interplay between physically motivated simulations and machine learning", Kyle Cranmer, Machine Learning for Physics and the Physics of Learning Long Program at IPAM
- 31 Jul 2019 - "Overview and Future directions for ML in particle and astro physics", Kyle Cranmer, Hammers & Nails 2019
- 29 Jul 2019 - "IRIS-HEP Tutorial: Fast columnar data analysis with data science tools", Jim Pivarski, Division of Particles and Fields (DPF) of the American Physical Society (APS)
- 23 Jul 2019 - "Scientific Python Ecosystem; Columnar Data Analysis; Accelerating Python", Jim Pivarski, Third Computational and Data Science for High Energy Physics (CoDaS-HEP) School
- 22 Jul 2019 - "IRIS-HEP View", Gordon Watts, Computing infrastructures for future data analysis
- 10 Jul 2019 - "Motivation and requirements for awkward 1.0", Jim Pivarski, Analysis Systems Biweekly Meeting
- 9 Jul 2019 - "pyhf: a pure Python statistical fitting library for High Energy Physics with tensors and autograd", Matthew Feickert, 18th annual Scientific Computing with Python conference (SciPy 2019)
- 27 Jun 2019 - "Analysis in Run 4", Gordon Watts, ATLAS Software and Computing Week
- 24 Jun 2019 - "Future areas of focus for ML in particle physics", Kyle Cranmer, ATLAS Software and Computing Week
- 24 Jun 2019 - "Delivery of columnar data to analysis systems", Marc Weinberg, ATLAS Software & Computing Week #62
- 21 Jun 2019 - "IRIS-HEP Blueprint Concepts and Process", Mark Neubauer, Blueprint Meeting on Analysis Systems on Scalable Platforms
- 21 Jun 2019 - "Analysis Systems Perspectives and Goals", Kyle Cranmer, Analysis Systems R&D on Scalable Platforms Blueprint meeting
- 19 Jun 2019 - "SCAILFIN: Madminer deployment using REANA", Irina Espejo, Analysis Systems Topical Meeting
- 19 Jun 2019 - "Template Fits: HistFitter / TRexFitter", Alexander Held, Analysis Systems Topical Meeting
- 19 Jun 2019 - "AmpGen & Particle/DecayLanguage", Henry Schreiner, IRIS-HEP Analysis Systems Topical Workshop
- 19 Jun 2019 - "Histograms", Henry Schreiner, IRIS-HEP Analysis Systems Topical Workshop
- 19 Jun 2019 - "SCAILFIN: Reproducible Open Benchmarks", Heiko Mueller, Analysis Systems Topical Meeting
- 19 Jun 2019 - "Update on awkward-array, uproot, and related projects", Jim Pivarski, Analysis Systems Topical Workshop
- 19 Jun 2019 - "ServiceX", Ben Galwesky, Analysis Systems Topical Workshop
- 19 Jun 2019 - "MadMiner Update", Johann Brehmer, Analysis Systems Topical Meeting
- 19 Jun 2019 - "SCAILFIN: Reproducible Open Benchmarks", Sebastian Macaluso, Analysis Systems Topical Meeting
- 19 Jun 2019 - "Functional/Declarative Selection Languages", Gordon Watts, Analysis Systems Topical Workshop
- 19 Jun 2019 - "Reinterpretation Roadmap", Kyle Cranmer, Analysis Systems Topical Meeting
- 18 Jun 2019 - "Uproot: accessing ROOT data in the scientific Python ecosystem", Jim Pivarski, 3rd CMS Machine Learning Workshop
- 14 Jun 2019 - "Advances in Deep Learning motivated by Physics Problems", Kyle Cranmer, Theoretical Physics for Deep Learning
- 10 Jun 2019 - "Numpy, Pandas, PyROOT, and Uproot", Jim Pivarski, U.S. ATLAS Software Training at Argonne National Lab
- 6 Jun 2019 - "Constraining effective field theories with machine learning", Johann Brehmer, INFN Padova seminar
- 4 Jun 2019 - "Deep Learning for Higgs Boson Identification and Searches for New Physics at the Large Hadron Collider", Mark Neubauer, Blue Waters Symposium for Petascale Science and Beyond
- 29 May 2019 - "Columnar Analysis Tools HATS", Jim Pivarski, LPC HATS: Hands-on Training for CMS
- 29 May 2019 - "The Primacy of Experiment", Kyle Cranmer, The Universe Speaks in Numbers
- 28 May 2019 - "Scientific Python and Uproot HATS", Jim Pivarski, LPC HATS: Hands-on Training for CMS
- 22 May 2019 - "Pattern matching for decay trees", Jim Pivarski, IRIS-HEP Topical Meetings
- 9 May 2019 - "Summary of the 'Analysis Description Languages for the LHC' workshop", Jim Pivarski, LPC Physics Forum
- 8 May 2019 - "IRIS-HEP: A new software institute to prepare for the data from the High Luminosity Large Hadron Collider in the exabyte era", Mason Proffitt, Northwest Data Science Summit
- 8 May 2019 - "Programming languages and particle physics", Jim Pivarski, Fermilab Colloquium
- 7 May 2019 - "Thinking about Analysis Languages and Recent Progress", Gordon Watts, Analysis Description Languages
- 6 May 2019 - "How to build your own language (hands-on demo)", Jim Pivarski, Analysis Description Languages Workshop
- 1 May 2019 - "Future areas of focus for ML in particle physics", Kyle Cranmer, Gotham City Physics X ML
- 18 Apr 2019 - "Constraining effective field theories with machine learning", Johann Brehmer, Higgs and Effective Field Theory 2019
- 15 Apr 2019 - "boost-histogram and hist", Henry Schreiner, IRIS-HEP Topical Meeting
- 15 Apr 2019 - "Aghast", Jim Pivarski, IRIS-HEP Topical Meetings
- 15 Apr 2019 - "Future areas of focus for ML in particle physics", Kyle Cranmer, 3rd IML Machine Learning Workshop
- 8 Apr 2019 - "High-Performance Python and Interoperability with Compiled Code", Jim Pivarski, Princeton PICSciE mini-courses
- 5 Apr 2019 - "Scalable Cyberinfrastructure for Artificial Intelligence and Likelihood-Free Inference", Mark Neubauer, NSF Large Facilities Workshop
- 1 Apr 2019 - "PyROOT, uproot, and awkward-arrays", Jim Pivarski, Software Carpentry at Fermilab
- 21 Mar 2019 - "Conda: a complete reproducible ROOT environment in under 5 minutes", Henry Schreiner, 2019 Joint HSF/OSG/WLCG Workshop
- 18 Mar 2019 - "'Mining gold' from simulators to improve likelihood-free inference", Johann Brehmer, Likelihood-free inference workshop
- 18 Mar 2019 - "Overview of Likelihood-Free Inference for Physics", Kyle Cranmer, Likelihood-Free Inference Workshop
- 14 Mar 2019 - "Nested data structures in array and SIMD frameworks", Jim Pivarski, ACAT 2019
- 14 Mar 2019 - "Keynote: Constraining effective field theories with machine learning", Johann Brehmer, International Workshop on Advanced Computing and Analysis Techniques in Physics Research
- 14 Mar 2019 - "Beyond the Roadmap: HL-LHC HEP Software", Gordon Watts, ACAT 2019
- 11 Mar 2019 - "Aligning the MATHUSA Test Stand Detector: Using Tensorflow", Gordon Watts, ACAT 2019
- 28 Feb 2019 - "Bringing together simulations, physics insight, and machine learning to constrain new physics", Johann Brehmer, Dark universe seminar
- 25 Feb 2019 - "The C# LINQ Analysis Language", Gordon Watts, IRIS-HEP Topical Meeting on Analysis Description Languages
- 20 Feb 2019 - "IRIS-HEP and ATLAS", Gordon Watts, US ATLAS IB Meeting
- 13 Feb 2019 - "LINQ To ROOT", Gordon Watts, 1st DAWG Technology and Innovation Survey (HSF)
- 6 Feb 2019 - "IRIS-HEP Steering Board Meeting #1", Gordon Watts, IRIS-HEP Steering Board Meeting
- 6 Feb 2019 - "IRIS-HEP Analysis Systems", Kyle Cranmer, IRIS-HEP Steering Board Meeting
- 14 Jan 2019 - "Meticulous measurements with matrix elements and machine learning", Johann Brehmer, ITS/CHEP joint seminar
- 11 Jan 2019 - "Improving inference with matrix elements and machine learning", Johann Brehmer, HK IAS Program on High Energy Physics
- 29 Oct 2018 - "pyhf: a pure Python implementation of HistFactory with tensors and autograd", Matthew Feickert, DIANA/HEP Meeting
- 19 Oct 2018 - "Design Roadmap for Future Collaborations", Mark Neubauer, Deep Learning for Multimessenger Astrophysics Real-time Discovery at Scale
- 20 Sep 2018 - "Learning to constrain new physics", Johann Brehmer, IPPP seminar
- 13 Sep 2018 - "Learning to constrain new physics", Johann Brehmer, Pheno & Vino Seminar
- 27 Aug 2018 - "Learning to constrain new physics", Johann Brehmer, Elementary particle seminar
- 23 Jul 2018 - "Machine Learning to Probe a BSM Higgs Sector", Johann Brehmer, Higgs Hunting
- 27 Jun 2018 - "Constraining Effective Theories with Machine Learning", Johann Brehmer, Theory seminar
[/expand]
AS Publications
- Flows for simultaneous manifold learning and density estimation, J. Brehmer and K. Cranmer, arXiv 2003.13913 (30 Mar 2020) [1 citation].
- Equivariant flow-based sampling for lattice gauge theory, G. Kanwar, M. Albergo, D. Boyda, K. Cranmer, D. Hackett et. al., arXiv 2003.06413 (13 Mar 2020) [2 citations].
- Compact Representation of Uncertainty in Hierarchical Clustering, C. Greenberg, S. Macaluso, N. Monath, J. Lee, P. Flaherty et. al., arXiv 2002.11661 (26 Feb 2020).
- Mining gold from implicit models to improve likelihood-free inference, Proceedings of the National Academy of Sciences; DOI:10.1073/pnas.1915980117 (20 Feb 2020) [23 citations].
- Set2Graph: Learning Graphs From Sets, H. Serviansky, N. Segol, J. Shlomi, K. Cranmer, E. Gross et. al., arXiv 2002.08772 (20 Feb 2020) [1 citation].
- Normalizing Flows on Tori and Spheres, D. Rezende, G. Papamakarios, S. Racanière, M. Albergo, G. Kanwar et. al., arXiv 2002.02428 (06 Feb 2020) [2 citations].
- Mining for Dark Matter Substructure: Inferring subhalo population properties from strong lenses with machine learning, The Astrophysical Journal, Volume 886, Number 1; DOI:10.3847/1538-4357/ab4c41 (19 Nov 2019) [9 citations].
- The frontier of simulation-based inference, K. Cranmer, J. Brehmer and G. Louppe, arXiv 1911.01429 (Submitted to National Academy of Sciences) (04 Nov 2019) [2 citations].
- Extending RECAST for Truth-Level Reinterpretations, A. Schuy, L. Heinrich, K. Cranmer and S. Hsu, arXiv 1910.10289 (Submitted to DPF2019) (22 Oct 2019).
- Hamiltonian Graph Networks with ODE Integrators, A. Sanchez-Gonzalez, V. Bapst, K. Cranmer and P. Battaglia, arXiv 1909.12790 (27 Sep 2019) [1 citation].
- Benchmarking simplified template cross sections in $WH$ production, J. Brehmer, S. Dawson, S. Homiller, F. Kling and T. Plehn, JHEP 11 034 (2019) (19 Aug 2019) [7 citations].
- RECAST framework reinterpretation of an ATLAS Dark Matter Search constraining a model of a dark Higgs boson decaying to two b-quarks, ATL-PHYS-PUB-2019-032 (12 Aug 2019).
- Reproducing searches for new physics with the ATLAS experiment through publication of full statistical likelihoods, ATL-PHYS-PUB-2019-029 (05 Aug 2019).
- MadMiner: Machine learning-based inference for particle physics, J. Brehmer, F. Kling, I. Espejo and K. Cranmer, Comput.Softw.Big Sci. 4 3 (2020) (24 Jul 2019) [7 citations].
- Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale, A. Baydin, L. Shao, W. Bhimji, L. Heinrich, L. Meadows et. al., arXiv 1907.03382 (07 Jul 2019) [2 citations].
[expand]
- A hybrid deep learning approach to vertexing, R. Fang, H. Schreiner, M. Sokoloff, C. Weisser and M. Williams, arXiv 1906.08306 (Submitted to ACAT 2019) (19 Jun 2019).
- Effective LHC measurements with matrix elements and machine learning, J. Brehmer, K. Cranmer, I. Espejo, F. Kling, G. Louppe et. al., arXiv 1906.01578 (04 Jun 2019) [4 citations].
- FPGA-accelerated machine learning inference as a service for particle physics computing, J. Duarte, P. Harris, S. Hauck, B. Holzman, S. Hsu et. al., Comput.Softw.Big Sci. 3 13 (2019) (18 Apr 2019) [1 citation].
- Machine learning and the physical sciences, G. Carleo, I. Cirac, K. Cranmer, L. Daudet, M. Schuld et. al., Rev.Mod.Phys. 91 045002 (2019) (25 Mar 2019) [37 citations].
- Open is not enough, X. Chen, S. Dallmeier-Tiessen, R. Dasler, S. Feger, P. Fokianos et. al., Nature Phys. 15 (2019) (15 Nov 2018) [8 citations].
- Analysis Preservation and Systematic Reinterpretation within the ATLAS experiment, K. Cranmer and L. Heinrich, J.Phys.Conf.Ser. 1085 042011 (2018) (18 Oct 2018) [1 citation].
- Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model, A. Baydin, L. Heinrich, W. Bhimji, L. Shao, S. Naderiparizi et. al., arXiv 1807.07706 (20 Jul 2018) [3 citations].
- Machine Learning in High Energy Physics Community White Paper, K. Albertsson, P. Altoe, D. Anderson, J. Anderson, M. Andrews et. al., J.Phys.Conf.Ser. 1085 022008 (2018) (08 Jul 2018) [47 citations].
- Adversarial Variational Optimization of Non-Differentiable Simulators, G. Louppe, J. Hermans and K. Cranmer, arXiv 1707.07113 (22 Jul 2017) [10 citations].
- Yadage and Packtivity - analysis preservation using parametrized workflows, K. Cranmer and L. Heinrich, J.Phys.Conf.Ser. 898 102019 (2017) (06 Jun 2017) [9 citations].
- HEPData: a repository for high energy physics data, E. Maguire, L. Heinrich and G. Watt, J.Phys.Conf.Ser. 898 102006 (2017) (18 Apr 2017) [44 citations].
- QCD-Aware Recursive Neural Networks for Jet Physics, G. Louppe, K. Cho, C. Becot and K. Cranmer, JHEP 01 057 (2019) (02 Feb 2017) [93 citations].
[/expand]
Join us
We collaborate with groups around the world on code, data, and more. See our project pages for more.