The Analysis Grand Challenge
The “Analysis Grand Challenge” organized by IRIS-HEP includes the binned analysis, reinterpretation and end-to-end optimization of physics analysis use cases. It also includes the development of the required cyber infrastructure to execute them in order to demonstrate technologies envisioned for HL-LHC. To enable these use cases and more, the expected capabilities include:
- New user interfaces: complementary services that present the analyst with a notebook-based interface. Example software: Jupyter.
- Data access: services that provide quick access to the experiment’s official data sets, often allowing simple derivations and local caching for efficient access. Example software and services: Rucio, ServiceX, SkyHook, RNTuple.
- Event selection: systems/frameworks allowing analysts to process entire datasets, select desired events, and calculate derived quantities. Example software and services: Coffea, awkward-array, awkward-dask, func_adl, RDataFrame.
- Histogramming and summary statistics: closely tied to the event selection, histogramming tools provide physicists with the ability to summarize the observed quantities in a dataset. Example software and services: Coffea, func_adl, cabinetry, hist.
- Statistical model building and fitting: tools that translate specifications for event selection, summary statistics, and histogramming quantities into statistical models, leveraging the capabilities above, and perform fits and statistical analysis with the resulting models. Example software and services: cabinetry, pyhf, FuncX+pyhf fitting service
- Reinterpretation / analysis preservation: standards for capturing the entire analysis workflow, and services to reuse the workflow which enables reinterpretation. Example software and services: REANA, RECAST.
The Analysis Grand Challenge is being conducted during 2021‒2023, leaving enough time for tuning software tools and services developed as a part of the IRIS-HEP ecosystem before the start-up of the HL-LHC.
AGC repositories and related resources
AGC has a dedicated webpage for documentation: https://agc.readthedocs.io/en/latest/
Recent accomplishments and plans
Recent accomplishments:
- Demonstrate ServiceX -> coffea -> cabinetry -> pyhf differentiable programming roadmap: see more A.Held contribution
- Execute IRIS-HEP AGC tools soft-launch event - AGC Tools 2021 Workshop
- Execute second part of IRIS-HEP AGC tools soft-launch event - AGC Tools 2022 Workshop
- Developed an analysis example based on CMS Opendata used for Analysis Grand Challenge demonstration
- Presented first performance measurements with an AGC implementation at ACAT 2022
Future plans for 2023:
- Improve experiment-related coffea-casa setups (e.g. improve experiment specific data access and other features)
- Performance tests of ServiceX integrated in Coffea-Casa analysis facility at University Nebraska-Lincoln and ATLAS analysis facility instance at the University of Chicago
- Benchmark performance of prototype of other system components for Analysis Grand Challenge
- Improve and increase complexity of developed analysis example used for next round of demonstration (the first example based on CMS Open Data was shown at the AGC Tools 2022 Workshop)
- Incorporate ML into Analysis Grand Challenge analysis workflow and execute on analysis facilities
- Organise IRIS-HEP AGC community workshop at University Wisconsin-Madison - AGC 2023 Workshop
- Prepare for Analysis Grand Challenge execution event (September 2023)
Videos and tutorials
AGC Workshop 2022
- IRIS-HEP AGC Tools 2022 Workshop - Alex Held, Oksana Shadura - Youtube video at Analysis Grand Challenge Tools workshop 2022
- “Foundation libraries (uproot, awkward, hist, mplhep)” - Mason Proffitt - Youtube video at Analysis Grand Challenge Tools workshop 2022
- “Queries with func_adl and data delivery with ServiceX” - Gordon Watts - Youtube video at Analysis Grand Challenge Tools workshop 2022
- “Columnar analysis with coffea” - Mat Adamec - Youtube video at Analysis Grand Challenge Tools workshop 2022
- “Statistical Inference: pyhf and cabinetry” - Matthew Feickert - Youtube video at Analysis Grand Challenge Tools workshop 2022
- “From data delivery to statistical inference with CMS Open Data” - Alexander Held - Youtube video at Analysis Grand Challenge Tools workshop 2022
- “Scale-out with coffea: coffea-casa analysis facility” - Carl Lundstedt - Youtube video at Analysis Grand Challenge Tools workshop 2022
- “Data management with Skyhook” - Jayjeet Chakrabort - Youtube video at Analysis Grand Challenge Tools workshop 2022
- “Analysis user experience with Python HEP data science tools in ATLAS” - Matthew Feickert - Youtube video at Analysis Grand Challenge Tools workshop 2022
- “Analysis user experience with Python HEP data science tools in LHCb” - Nathan Allen Grieser - Youtube video at Analysis Grand Challenge Tools workshop 2022
- “Analysis user experience with Python HEP data science tools in CMS” - Lindsey Gray - Youtube video at Analysis Grand Challenge Tools workshop 2022
AGC Workshop 2021
- “Introduction to AGC Tools workshop” - Oksana Shadura, Alexander Held - Youtube video at Analysis Grand Challenge Tools workshop 2021
- “Data handling: uproot, awkward & vector” - Mason Proffitt - Youtube video at Analysis Grand Challenge Tools workshop 2021
- “Histogramming & visualization: hist & mplhep “ - Andrzej Novak - Youtube video at Analysis Grand Challenge Tools workshop 2021
- “Columnar analysis with coffea” - Lindsey Gray, Nick Smith - Youtube video at Analysis Grand Challenge Tools workshop 2021
- “Queries with func_adl” - Mason Proffitt - Youtube video at Analysis Grand Challenge Tools workshop 2021
- “Data Delivery with ServiceX” - Kyungeon Choi - Youtube video at Analysis Grand Challenge Tools workshop 2021
- “From data delivery to statistical inference: ServiceX, coffea, cabinetry & pyhf” - Alexander Held - Youtube video at Analysis Grand Challenge Tools workshop 2021
- “Data management with Skyhook” - Carlos Maltzahn, Jayjeet Chakraborty - Youtube video at Analysis Grand Challenge Tools workshop 2021
- “Scale-out with coffea: coffea-casa” - Oksana Shadura - Youtube video at Analysis Grand Challenge Tools workshop 2021
Fellows
Team
Presentations
- 24 Oct 2024 - "Tuning the CMS Coffea-casa facility for 200 Gbps Challenge", Oksana Shadura, Conference on Computing in High Energy and Nuclear Physics (CHEP 2024)
- 21 Oct 2024 - "The 200 Gbps Challenge: Imagining HL-LHC analysis facilities", Alexander Held, CHEP 2024
- 4 Sep 2024 - "AGC & IDAP / 200 Gbps", Oksana Shadura, IRIS-HEP Institute Retreat 2024
- 2 Sep 2024 - "Facilities R&D HSF highlights", Oksana Shadura, The 8th Asian Tier Center Forum
- 18 Jun 2024 - "Analysis Grand Challenge", Oksana Shadura, Analysis Facilities Workshop
- 11 Jun 2024 - "Coffea-casa and 200 Gbps challenge - experience with Kubernetes", Oksana Shadura, 2024 All-Hands Workshop of the U.S. CMS Software and Computing Operations Program
- 9 Jun 2024 - "The 200 Gbps Challenge at Nebraska", Oksana Shadura, US CMS Analysis Facility Meeting
- 3 Jun 2024 - "The ATLAS 200 Ggps Challenge", Gordon Watts, ATLAS Analysis Model Group at the ATLAS S&C Week (Internal)
- 8 May 2024 - "Notes about AF users UX feedback", Oksana Shadura, Common Analysis Tools (CAT) general meeting
- 3 Apr 2024 - "Evolution of Analysis Techniques and Statistical Treatment", Alexander Held, APS April Meeting 2024
- 19 Mar 2024 - "View from HSF - HSF AF White Paper Overview", Oksana Shadura, CMS Spring 2024 Offline and Computing Week
- 15 Mar 2024 - "Introduction - IRIS-HEP Data Analysis Pipeline (IDAP)", Oksana Shadura, IRIS-HEP Data Analysis Pipeline (IDAP) meeting
- 5 Mar 2024 - "Analysis Grand Challenge (AGC)", Oksana Shadura, US CMS Analysis Facility Meeting
- 1 Mar 2024 - "AGC and new coffea 2023/4", Alexander Held, IRIS-HEP / AGC Demo Day 4
- 10 Jan 2024 - "AGC Deep Dive", Oksana Shadura, NSF / IRIS-HEP Meeting (January 2024)
- 5 Dec 2023 - "Updates on Coffea-Casa AF", Oksana Shadura, Common Analysis Tools (CAT) general meeting
- 4 Oct 2023 - "The AGC with ATLAS Data", Gordon Watts, The ATLAS Software And Computing Week #76 (internal)
- 14 Sep 2023 - "AGC Team End-to-End Demo", Alexander Held, IRIS-HEP AGC Demonstration 2023
- 12 Sep 2023 - "Future Analysis Facilities R&D", Oksana Shadura, IRIS-HEP Institute Retreat
- 12 Sep 2023 - "Current Plans for AGC", Oksana Shadura, IRIS-HEP Institute Retreat
- 11 Sep 2023 - "Focus Area - Analysis Grand Challenge", Oksana Shadura, IRIS-HEP Institute Retreat
- 27 Jul 2023 - "Analysis Grand Challenge & Coffea-Casa analysis facility as a test environment for packages and services", Oksana Shadura, PyHEP.dev 2023 - "Python in HEP" Developer's Workshop
- 26 Jul 2023 - "Statistical models, analysis workflows & automatic differentiation", Alexander Held, PyHEP.dev 2023
- 24 Jul 2023 - "Analysis Grand Challenge Demo - Hands-on Demo Session", Oksana Shadura, Computational HEP Traineeship Summer School
- 24 Jul 2023 - "Analysis Grand Challenge Demo", Alexander Held, Computational HEP Traineeship Summer School
- 11 Jul 2023 - "Analysis Grand Challenge", Oksana Shadura, IRIS-HEP / Ops Program Analysis Grand Challenge Planning
- 24 May 2023 - "Coffea-casa analysis facility", Oksana Shadura, Common Analysis Tools (CAT) general meeting (CMS internal)
- 9 May 2023 - "Coffea-Casa - Building composable analysis facilities for the HL-LHC", Oksana Shadura, 26th International Conference on Computing in High Energy & Nuclear Physics
- 9 May 2023 - "Physics analysis for the HL-LHC: concepts and pipelines in practice with the Analysis Grand Challenge", Alexander Held, CHEP 2023
- 8 May 2023 - "Machine Learning for Columnar High Energy Physics Analysis", Elliott Kauffman, CHEP 2023
- 5 May 2023 - "Analysis Grand Challenge workshop closing", Alexander Held, IRIS-HEP Analysis Grand Challenge workshop 2023
- 3 May 2023 - "IRIS-HEP AGC Workshop 2023", Tal van Daalen, ServiceX User Experience
- 3 May 2023 - "IRIS-HEP AGC Workshop 2023", Gordon Watts, ServiceX for ATLAS
- 3 May 2023 - "User Experience for ML", Elliott Kauffman, IRIS-HEP AGC Workshop 2023
- 3 May 2023 - "Analysis Grand Challenge workshop introduction", Alexander Held, IRIS-HEP Analysis Grand Challenge workshop 2023
- 30 Mar 2023 - "Analysis Grand Challenge Demonstrator", Gordon Watts, ATLAS Software & Computing Plenary (internal)
- 23 Mar 2023 - "Coffea-casa analysis facility", Oksana Shadura, International Symposium on Grids & Clouds (ISGC) 2023 in conjunction with HEPiX Spring 2023 Workshop
- 23 Mar 2023 - "Physics analysis workflows and pipelines for the HL-LHC", Alexander Held, International Symposium on Grids & Clouds (ISGC) 2023
- 14 Mar 2023 - "Analysis Grand Challenge updates", Alexander Held, IRIS-HEP / Ops Program Analysis Grand Challenge Planning
- 24 Feb 2023 - "Integrating MLflow in AGC workflow", Elliott Kauffman, IRIS-HEP / AGC Demo Day
- 24 Jan 2023 - "Analysis Grand Challenge updates", Alexander Held, IRIS-HEP / Ops Program Analysis Grand Challenge Planning
- 16 Dec 2022 - "IRIS-HEP / AGC Demo Day #1", Tal van Daalen, ServiceX: ROOT files from uproot transformer
- 16 Dec 2022 - "Integrating AGC pipeline at BNL facility", Matthew Feickert, IRIS-HEP / AGC Demo Day #1
- 16 Dec 2022 - "First steps using inference server at coffea-casa facility", Elliott Kauffman, IRIS-HEP / AGC Demo Day
- 15 Nov 2022 - "Analysis Grand Challenge updates", Oksana Shadura, IRIS-HEP / Ops Program Analysis Grand Challenge Planning
- 3 Nov 2022 - "IRIS-HEP AGC update", Alexander Held, HSF Analysis Facilities Forum
- 25 Oct 2022 - "First performance measurements with the Analysis Grand Challenge", Oksana Shadura, ACAT 2022
- 12 Oct 2022 - "AGC - Perspective from focus areas and projects", Oksana Shadura, IRIS-HEP Institute Retreat
- 12 Oct 2022 - "AGC overview, status and plans", Alexander Held, IRIS-HEP Institute Retreat
- 4 Oct 2022 - "Analysis Grand Challenge updates", Oksana Shadura, IRIS-HEP / Ops Program Analysis Grand Challenge Planning
- 15 Sep 2022 - "End-to-end physics analysis with Open Data: the Analysis Grand Challenge", Alexander Held, PyHEP 2022 (virtual) Workshop
- 23 Aug 2022 - "Analysis Grand Challenge updates", Alexander Held, IRIS-HEP / Ops Program Analysis Grand Challenge Planning
- 16 Jul 2022 - "Analysis Facilities", Oksana Shadura, Seattle Snowmass Summer Meeting 2022
- 12 Jul 2022 - "Report from Analysis Ecosystems II Workshop", Oksana Shadura, Software & Computing Round Table (2022)
- 9 Jul 2022 - "The IRIS-HEP Analysis Grand Challenge", Alexander Held, ICHEP 2022
- 14 Jun 2022 - "Analysis Grand Challenge updates", Alexander Held, IRIS-HEP / Ops Program Analysis Grand Challenge Planning
- 13 Jun 2022 - "IRIS-HEP Analysis Grand Challenge: Status & plans", Alexander Held, ATLAS Software & Computing Week
- 25 May 2022 - "Analysis Facilities - Summary", Oksana Shadura, Analysis Ecosystem Workshop II
- 25 Apr 2022 - "IRIS-HEP Analysis Grand Challenge Tools Workshop", Oksana Shadura, RIS-HEP AGC Tools 2022 Workshop
- 25 Apr 2022 - "From data delivery to statistical inference with CMS Open Data", Alexander Held, IRIS-HEP AGC Tools 2022 Workshop
- 8 Apr 2022 - "CompF4 Analysis Facility - Discussion & Priorities", Oksana Shadura, Snowmass CompF4 Topical Group Workshop
- 5 Apr 2022 - "Analysis Grand Challenge updates", Oksana Shadura, IRIS-HEP / Ops Program Analysis Grand Challenge Planning
- 1 Apr 2022 - "Report about HSF Analysis Facilities Forum Kick-off meeting", Oksana Shadura, CMS Spring 2022 O&C Week
- 25 Mar 2022 - "AFs in the context of the IRIS-HEP AGC", Alexander Held, Analysis Facilities Forum Kick-off Meeting
- 1 Mar 2022 - "Analysis Grand Challenge updates", Alexander Held, IRIS-HEP / Ops Program Analysis Grand Challenge Planning
- 28 Jan 2022 - "Analysis Grand Challenge updates", Oksana Shadura, IRIS-HEP / Ops Program Analysis Grand Challenge Planning
- 16 Dec 2021 - "Analysis Grand Challenge updates", Oksana Shadura, IRIS-HEP Executive Board / Ops Program Grand Challenge Discussion
- 30 Nov 2021 - "Analysis Grand Challenge updates", Alexander Held, Steering Board Meeting
- 17 Nov 2021 - "Deep Dive - Analysis Grand Challenge", Oksana Shadura, NSF / IRIS-HEP Meeting (November 2021)
- 3 Nov 2021 - "From data delivery to statistical inference: ServiceX, coffea, cabinetry & pyhf", Alexander Held, IRIS-HEP AGC Tools 2021 Workshop
- 2 Nov 2021 - "Analysis Grand Challenge", Oksana Shadura, SwiftHep/ExcaliburHep workshop
Publications
- Physics analysis for the HL-LHC: Concepts and pipelines in practice with the Analysis Grand Challenge, A. Held, E. Kauffman, O. Shadura and A. Wightman, EPJ Web Conf. 295 06016 (2024) (05 Jan 2024) [1 citation].
- Machine Learning for Columnar High Energy Physics Analysis, E. Kauffman, A. Held and O. Shadura, EPJ Web Conf. 295 08011 (2024) (03 Jan 2024).
- Coffea-Casa: Building composable analysis facilities for the HL-LHC, S. Albin, G. Attebury, K. Bloom, B. Bockelman, C. Lundstedt, O. Shadura and J. Thiltges, EPJ Web Conf. 295 07009 (2024) (30 Nov 2023).
- First performance measurements with the Analysis Grand Challenge, Oksana Shadura, Alexander Held, arXiv:2304.05214 [hep-ex] (Submitted to ACAT 2022) (12 Apr 2023).
- The IRIS-HEP Analysis Grand Challenge, A. Held and O. Shadura, Unknown (26 Nov 2022) [4 citations].