HNSciCloud Overview
•
0 likes•223 views
The document discusses the Helix Nebula Science Cloud (HNSciCloud) project. Key points: - HNSciCloud is a joint procurement by 10 research institutions to provide a common cloud platform for the European research community. - It aims to increase computing and storage capabilities for scientific data analysis through a hybrid cloud model combining procurers' data centers, commercial cloud providers, and high-speed networks. - The project has progressed through different phases including requirements analysis, prototype development, and pilots with two commercial providers to test cloud services for high-energy physics and other research workflows.
Report
Share
HNSciCloud Overview
- 2. 08/12/2016 2
- 3. Helix Nebula – The Science Cloud with Grant Agreement 687614 is a Pre-Commercial Procurement Action funded by H2020 Framework Programme HNSciCloud Overview M-PIL-3.2, 14th June 2018 João Fernandes CERN IT department 01/02/2018
- 4. Helix Nebula Science Cloud • To provide a common cloud platform for the European research community • Via a collective effort of 10 procurer Institutions forming the Buyers Group 08/12/2016 4 Expressing the need to increase the analysis capability and capacity offered to their users to keep pace with the growth in scientific data
- 5. Helix Nebula Science Cloud Joint Pre-Commercial Procurement 5 Procurers: CERN, CNRS, DESY, EMBL-EBI, ESRF, IFAE, INFN, KIT, STFC, SURFSara Experts: Trust-IT & EGI.eu Resulting IaaS services support use-cases from many research communities Deployed in a hybrid cloud mode: • procurers data centres • commercial cloud service providers • GEANT network and EduGAIN Federated Identity Management. Total procurement budget >5.3M€ Co-funded via H2020 Grant Agreement 687614
- 6. 6 The Hybrid Cloud Model Brings together • research organisations, • data providers, • publicly funded e- infrastructures, • commercial cloud service providers Procurement and governance approaches suitable for the dynamic cloud market In-house
- 7. Challenges Innovative IaaS cloud services integrated with procurers in-house resources to support a range of scientific workloads Compute and Storage Support a range of virtual machine and container configurations including HPCaaS, working with datasets in the petabyte range, accessible transparently Network Connectivity and Federated Identity Management Provide high-end network capacity via GEANT for the whole platform with common federated identity and access management AAI activities have been described as a ‘pilot’ use-case in a AARC2 project deliverable: https://aarc-project.eu/wp- content/uploads/2018/06/DSA1.1-v1.1FINAL.pdf Service Payment Models Explore a range of purchasing options to determine those most appropriate for the scientific application workloads, including vouchers or other means of easy integration in the organisations procurement models 7
- 8. HNSciCloud project phases Preparation •Analysis of requirements, current market offers and relevant standards •Build stakeholder group •Develop tender material Implementation and sharing Jan’16 Dec’18 Each step is competitive - only contractors that successfully complete the previous step can bid in the next 8 4 Designs 3 Prototypes 2 Pilots Call-off Feb’17 Call-off Dec’17 Tender Jul’16 Phases of the tender are defined by the Horizon 2020 Pre-Commercial Procurement financial instrument
- 9. Pilot Phase Timeline 26/05/2017 9 Dec’18Feb’18 3.5k350TB 5k/500TB 10k/1PB 40Gbps Scalability Testing End User Access We are here 2 Pilots Call-off Dec’17 M-PIL-3.2 CERN 13th- 14th June
- 10. What HNSciCloud is not Not a zero cost research grant Contractual Relationship: Important to understand how to integrate commercial cloud services into scientific activities Not a one-off test A predictable production-quality contribution to the scientific programme of the organisations involved Not just a technical evaluation Legal: Where is the data stored? Under what jurisdiction? What are the contractual terms and conditions? Commercial: what are the commercialisation plans after the project? How much will it cost to use them in production? What purchase models are available? Not a walk in the park It took 2.5 years of intense collaboration for procurers and companies to get here From common requirements specification for workloads of multiple scientific disciplines to tender launch, design and prototype phases Required significant effort by the procuring organisations to follow the process, provide all the necessary material and performing tests on the developed services. 10
- 11. T-Systems IaaS based on OTC RHEA IaaS provided by Exoscale Cloud Providers 11
- 12. HEP Use-Case Deployments Overview 26/05/2017 WLCG Experiments ALICE, ATLAS, CMS and LHCb CERN Batch Service (IT Dep.) Container Federation (IT Dep.) Live demo during lunch Interactive Analysis for End Users (IT & EP Deps.) Live demo during lunch Deep Learning for Fast Detector Simulation using GPUs (IT Dep. - CERN openlab)
- 13. WLCG Procurers 26/05/2017 8/10 procurers are actively supporting the LHC programme Operating sites in WLCG Hybrid Model bringing together WLCG sites and commercial cloud providers data centres linked via GEANT or/and NRENs More WLCG procuring organisations can participate via the early adopter programme (see Andrea’s slides): https://www.hnscicloud.eu/the-hnscicloud-adopter-group
- 14. European Open Science Cloud HNSciCloud promoted as a working example of an Open Science Cloud by the EC High Level Expert Group