IRIS-HEP Fellow: Jayjeet Chakraborty
Fellowship dates: Jun - Sep 2020
Home Institution: National Institute Of Technology, Durgapur
Reproducible, large-scale SkyhookDM experimentsSkyhookDM injects programmable data management and data storage capabilities directly in the storage layer of distributed object databases such as Ceph. SkyhookDM utilizes and extends the Ceph distributed object storage platform with customized C++ object classes that enable database operations such as SELECT, PROJECT, AGGREGATE to be offloaded directly into the object storage layer, allowing applications to efficiently query multi-dimensional arrays. Compiling ceph along with Skyhook and running benchmark tests consists of a number of steps and can become irreproducible at times. The aim of this project is to implement a reproducible workflow with Popper to automate large-scale tests on different cloud infrastructure like GCP, Cloudlab and Kubernetes clusters and benchmark SkyhookDM at the 10's of terabyte scale over the various supported data formats.
More information: My project proposal
Carlos Maltzahn (UC Santa Cruz)
Ivo Jimenez (UC Santa Cruz)
Jeff LeFevre (UC Santa Cruz)