Data Organization, Management and Access (DOMA)
The HL-LHC era will provide enormous challenges in the area of Data Organization, Management and Access (DOMA). The LHC will provide a significantly increased number of events and increased event complexity, both of which will drive much larger data sizes - with no changes in how the LHC community functions, the total increase in data volume may be a factor of 30.
Given the LHC experiments are, combined, managing nearly an exabyte of data, such a significant increase in volume is unmanageable. New mechanisms and techniques are necessary to more efficiently manage storage resources; the DOMA area in IRIS-HEP is working on the R&D necessary to affect such change.
It is not only data volumes that are potentially disruptive to the HL-LHC physics program; the extraordinarily large number of events (potentially 150 billion similated and recorded events per year per experiment) presents a challenge in data management for users. Along with the analysis systems team within IRIS, DOMA is working on improved techniques for delivering events to users.
Contact us: email@example.com
- 24 Jun 2019 - Delivery of columnar data to analysis systems, Marc Weinberg (University of Chicago), ATLAS Software & Computing Week #62
- 19 Jun 2019 - ServiceX, Ben Galwesky (National Center for Supercomputing Applications), Analysis Systems Topical Workshop
- 12 Jun 2019 - XRootD and HTTP performance studies, Edgar Fajardo (University of California, San Diego), XrootD Workshop@CC-IN2P3
- 04 Jun 2019 - Proposal for Sharing of Data on Data Access, Diego Davila (University of California, San Diego), DOMA / ACCESS Meeting
- 28 May 2019 - WLCG DOMA TPC Working Group, Brian Bockelman (Morgridge Institute), US CMS Tier-2 Facilities May 2019 Meeting
- 24 Apr 2019 - Skyhook: Programmable Object Storage for Analysis, Jeff LeFevre (University of California, Santa Cruz), IRIS-HEP Topical Meetings
- 20 Mar 2019 - Data Access in DOMA, Frank Wuerthwein (University of California, San Diego), HOW2019 (Joint HSF/OSG/WLCG Workshop)
- 20 Mar 2019 - Data Access at HPC: ATLAS/CMS Perspective, Frank Wuerthwein (University of California, San Diego), HOW2019 (Joint HSF/OSG/WLCG Workshop)
- 20 Mar 2019 - WLCG DOMA TPC Updates, Brian Bockelman (Morgridge Institute), 2019 Joint HSF/OSG/WLCG Workshop (HOW2019)
- 06 Mar 2019 - Scientific Data Lifecycle: Perspectives from an LHC Physicist, Shawn McKee (University of Michigan-Ann Arbor), MAGIC Meeting
- 26 Feb 2019 - Skyhook: programmable storage for databases, Jeff LeFevre (University of California, Santa Cruz), Vault'19
- 06 Feb 2019 - IRIS-HEP DOMA, Brian Bockelman (Morgridge Institute), IRIS-HEP Steering Board Meeting
- 02 Oct 2018 - Current production use of caching for CMS in Southern California, Edgar Fajardo (University of California, San Diego), DOMA / ACCESS Meeting
- 16 Nov 2017 - SkyhookDB - Leveraging object storage toward database elasticity in the cloud, Jeff LeFevre (University of California, Santa Cruz), DOMA Workshop 2017 (Flatiron Institute)
- 11 Jun 2019 - OSG Data Federation, Derek Weitzel (University of Nebraska - Lincoln), XRootD Workshop at CC-IN2P3
We collaborate with groups around the world on code, data, and more. See our project pages for more.