10.10.28
Invited Speaker
Grand Challenges in Data-Intensive Discovery Conference
San Diego Supercomputer Center, UC San Diego
Title: High Performance Cyberinfrastructure Enables Data-Driven Science in the Globally Networked World
La Jolla, CA
21st Century e-Knowledge Requires a High Performance e-Infrastructure
11.12.09
Keynote Presentation
40-year anniversary Celebration of SARA
Title: 21st Century e-Knowledge Requires a High Performance e-Infrastructure
Amsterdam, Netherlands
UC Capabilities Supporting High-Performance Collaboration and Data-Intensive ...
07.10.22
University of California Council of Research
UC Irvine
Title: UC Capabilities Supporting High-Performance Collaboration and Data-Intensive Sciences
Irvine, CA
Experiences in Application Specific Supercomputer Design - Reasons, Challenge...
The document discusses challenges faced with application specific supercomputer design. It provides an example of QPACE, a supercomputer designed for quantum chromodynamics (QCD) computations. Key challenges discussed include data ordering issues when using InfiniBand networking that could cause computations to use invalid data if ordering of writes to memory was not enforced. Ensuring proper data ordering is important to avoid software consuming data before it is valid.
High Performance Computing - Challenges on the Road to Exascale Computing
The document discusses challenges in achieving exascale computing capabilities by 2018. It outlines how standard technology scaling will not be enough, and compromises will need to be made. These include reduced node performance, lower network bandwidth and fewer pins. Blue Gene architecture is presented as an example of a balanced system that achieves high performance through optimized interconnects and packaging density. A thought experiment proposes integrating significant solid state storage at each node to create an "active storage" machine based on Blue Gene architecture.
The document summarizes the 19th ACM HPDC conference and VTDC workshop held in 2010. It provides an overview of the accepted papers and talks, including the topics covered, presenters and their affiliations, and high-level discussions. Key areas included distributed storage systems, virtualization technologies, data-intensive computing, workflows, and cloud/grid resources.
"Accelerating Deep Learning Using Altera FPGAs," a Presentation from Intel
For the full video of this presentation, please visit:
http://www.embedded-vision.com/platinum-members/altera/embedded-vision-training/videos/pages/may-2016-embedded-vision-summit
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Bill Jenkins, Senior Product Specialist for High Level Design Tools at Intel, presents the "Accelerating Deep Learning Using Altera FPGAs" tutorial at the May 2016 Embedded Vision Summit.
While large strides have recently been made in the development of high-performance systems for neural networks based on multi-core technology, significant challenges in power, cost and, performance scaling remain. Field-programmable gate arrays (FPGAs) are a natural choice for implementing neural networks because they can combine computing, logic, and memory resources in a single device. Intel's Programmable Solutions Group has developed a scalable convolutional neural network reference design for deep learning systems using the OpenCL programming language built with our SDK for OpenCL. The design performance is being benchmarked using several popular CNN benchmarks: CIFAR-10, ImageNet and KITTI.
Building the CNN with OpenCL kernels allows true scaling of the design from smaller to larger devices and from one device generation to the next. New designs can be sized using different numbers of kernels at each layer. Performance scaling from one generation to the next also benefits from architectural advancements, such as floating-point engines and frequency scaling. Thus, you achieve greater than linear performance and performance per watt scaling with each new series of devices.
Petascale Analytics - The World of Big Data Requires Big Analytics
The document discusses big data and analytics technologies. It describes how new technologies like Hadoop and MapReduce enable processing of extremely large datasets. It also discusses future technologies like exascale computing and storage class memory that will be needed to manage increasing data volumes and support real-time analytics.
Coupling Australia’s Researchers to the Global Innovation Economy
08.10.02
First Lecture in the
Australian American Leadership Dialogue Scholar Tour
University of Adelaide
Title: Coupling Australia’s Researchers to the Global Innovation Economy
Adelaide, Australia
Coupling Australia’s Researchers to the Global Innovation Economy
This document summarizes Dr. Larry Smarr's presentation on linking Australian researchers to the global innovation economy through high-performance networking. Some key points:
- Australia has established a 1 Gbps dedicated connection between the University of Melbourne and UC San Diego to better connect Australian researchers globally.
- Dr. Smarr is visiting Australian universities to launch the next phase of this project - linking major research universities and CSIRO to each other and innovation centers worldwide with AARNet's new 10 Gbps network.
- This unprecedented bandwidth will allow Australian researchers to join emerging global collaborative research efforts on issues critical to Australia's future.
Exploring emerging technologies in the HPC co-design space
This document discusses emerging technologies for high performance computing (HPC), focusing on heterogeneous computing and non-volatile memory. It provides an overview of HPC architectures past and present, highlighting the trend toward more heterogeneous systems using GPUs and other accelerators. The document discusses challenges for applications to adapt to these changing architectures. It also explores potential future technologies like 3D memory and discusses the Department of Energy's efforts in codesign centers to facilitate collaboration between application developers and emerging hardware.
Coupling Australia’s Researchers to the Global Innovation Economy
08.10.17
Ninth Lecture in the
Australian American Leadership Dialogue Scholar Tour
University of Sydney
Title: Coupling Australia’s Researchers to the Global Innovation Economy
Sydney, Australia
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility
In this deck from the Swiss HPC Conference, Mark Wilkinson presents: 40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility.
"DiRAC is the integrated supercomputing facility for theoretical modeling and HPC-based research in particle physics, and astrophysics, cosmology, and nuclear physics, all areas in which the UK is world-leading. DiRAC provides a variety of compute resources, matching machine architecture to the algorithm design and requirements of the research problems to be solved. As a single federated Facility, DiRAC allows more effective and efficient use of computing resources, supporting the delivery of the science programs across the STFC research communities. It provides a common training and consultation framework and, crucially, provides critical mass and a coordinating structure for both small- and large-scale cross-discipline science projects, the technical support needed to run and develop a distributed HPC service, and a pool of expertise to support knowledge transfer and industrial partnership projects. The on-going development and sharing of best-practice for the delivery of productive, national HPC services with DiRAC enables STFC researchers to produce world-leading science across the entire STFC science theory program."
Watch the video: https://wp.me/p3RLHQ-k94
Learn more: https://dirac.ac.uk/
and
http://hpcadvisorycouncil.com/events/2019/swiss-workshop/agenda.php
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Early Benchmarking Results for Neuromorphic Computing
This document summarizes early benchmarking results for neuromorphic computing using Intel's Loihi chip. It finds that Loihi provides orders of magnitude gains over CPUs and GPUs for certain workloads that are directly trained on the chip or use novel bio-inspired algorithms. These include online learning, adaptive control, event-based vision and tactile sensing, constraint satisfaction problems, and nearest neighbor search. Larger networks and problems tend to provide greater performance gains with Loihi.
In this deck from the HPC User Forum at Argonne, Andrew Siegel from Argonne presents: ECP Application Development.
"The Exascale Computing Project is accelerating delivery of a capable exascale computing ecosystem for breakthroughs in scientific discovery, energy assurance, economic competitiveness, and national security. ECP is chartered with accelerating delivery of a capable exascale computing ecosystem to provide breakthrough modeling and simulation solutions to address the most critical challenges in scientific discovery, energy assurance, economic competitiveness, and national security. This role goes far beyond the limited scope of a physical computing system. ECP’s work encompasses the development of an entire exascale ecosystem: applications, system software, hardware technologies and architectures, along with critical workforce development."
Watch the video: https://wp.me/p3RLHQ-kSL
Learn more: https://www.exascaleproject.org
and
http://hpcuserforum.com
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Metacomputer Architecture of the Global LambdaGrid: How Personal Light Paths ...
08.05.15
Departments of Computer Science / Physics and Astronomy
University of Missouri@Columbia
Title: Metacomputer Architecture of the Global LambdaGrid: How Personal Light Paths are Transforming e-Science
Columbia, MO
Qualcomm is researching on-device artificial intelligence to power personal assistants and other applications through its Qualcomm AI Research division, with a focus on developing efficient neural networks, on-device learning techniques, and personalized models that can operate on mobile devices while protecting privacy. Qualcomm has been optimizing AI software and hardware including its Snapdragon processors for years to enable advanced on-device AI capabilities. The document discusses Qualcomm's research in areas like computer vision, natural language processing, reinforcement learning, and distributed learning to advance on-device intelligent assistants and other applications.
Calit2 has formed two divisional councils to provide leadership and strategic direction. It has also developed multiple communication channels like brochures and websites. Two new Calit2 buildings at UC San Diego and UC Irvine will provide major new laboratories linked by dedicated optical networks for over 1000 researchers working across disciplines like nanotechnology, biomedicine, and digital arts. Calit2 is also working on applications of high-speed networks for areas like telemedicine, disaster response, and digital cinema.
Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analys...
06.04.26
Invited Talk
CONNECT Board Meeting
Title: Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analysis (CAMERA)
La Jolla, CA
CLOUD COMPUTING: AN ALTERNATIVE PLATFORM FOR SCIENTIFIC COMPUTINGDavid Ramirez
After an overview of its fundamental technologies, Grid Computing is presented as the platform of choice for scientific High Performance Computing (HPC). The latest offerings in Cloud Computing (CC) would enable it to become a basis for creating easy to deploy, on-demand and widely accessible grids, putting HPC within the reach of most scientific and research communities. A case study framework is proposed for future development.
High Performance Cyberinfrastructure Discovery Tools for Data Intensive ResearchLarry Smarr
This document discusses how high-performance cyberinfrastructure and dedicated 10Gbps networks enable new levels of discovery for data-intensive research. It provides examples of how universities are using these resources for projects in fields like cosmology, ocean observing, and microbial analysis. Specifically, it describes how networks like National LambdaRail provide connectivity between campuses and commercial clouds, and how tools like OptIPortals allow researchers to remotely analyze large datasets in real-time using visualization techniques.
The document discusses big data in astronomy and the LineA-DEXL case. It provides an outline and introduction to big data in science and hypothesis-driven research. It discusses data management techniques like data partitioning and parallel workflow processing. It then provides details on the Laboratorio Nacional de Computacao Cientifica (LNCC) and its role in supporting computational modeling and bioinformatics. It discusses astronomy surveys that generate large amounts of data like the Dark Energy Survey and challenges of data from the Large Synoptic Survey Telescope. Finally, it discusses the need for data infrastructure, metadata management, and distributed data management to support scientific research involving big data.
21st Century e-Knowledge Requires a High Performance e-InfrastructureLarry Smarr
11.12.09
Keynote Presentation
40-year anniversary Celebration of SARA
Title: 21st Century e-Knowledge Requires a High Performance e-Infrastructure
Amsterdam, Netherlands
UC Capabilities Supporting High-Performance Collaboration and Data-Intensive ...Larry Smarr
07.10.22
University of California Council of Research
UC Irvine
Title: UC Capabilities Supporting High-Performance Collaboration and Data-Intensive Sciences
Irvine, CA
Experiences in Application Specific Supercomputer Design - Reasons, Challenge...Heiko Joerg Schick
The document discusses challenges faced with application specific supercomputer design. It provides an example of QPACE, a supercomputer designed for quantum chromodynamics (QCD) computations. Key challenges discussed include data ordering issues when using InfiniBand networking that could cause computations to use invalid data if ordering of writes to memory was not enforced. Ensuring proper data ordering is important to avoid software consuming data before it is valid.
High Performance Computing - Challenges on the Road to Exascale ComputingHeiko Joerg Schick
The document discusses challenges in achieving exascale computing capabilities by 2018. It outlines how standard technology scaling will not be enough, and compromises will need to be made. These include reduced node performance, lower network bandwidth and fewer pins. Blue Gene architecture is presented as an example of a balanced system that achieves high performance through optimized interconnects and packaging density. A thought experiment proposes integrating significant solid state storage at each node to create an "active storage" machine based on Blue Gene architecture.
The document summarizes the 19th ACM HPDC conference and VTDC workshop held in 2010. It provides an overview of the accepted papers and talks, including the topics covered, presenters and their affiliations, and high-level discussions. Key areas included distributed storage systems, virtualization technologies, data-intensive computing, workflows, and cloud/grid resources.
For the full video of this presentation, please visit:
http://www.embedded-vision.com/platinum-members/altera/embedded-vision-training/videos/pages/may-2016-embedded-vision-summit
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Bill Jenkins, Senior Product Specialist for High Level Design Tools at Intel, presents the "Accelerating Deep Learning Using Altera FPGAs" tutorial at the May 2016 Embedded Vision Summit.
While large strides have recently been made in the development of high-performance systems for neural networks based on multi-core technology, significant challenges in power, cost and, performance scaling remain. Field-programmable gate arrays (FPGAs) are a natural choice for implementing neural networks because they can combine computing, logic, and memory resources in a single device. Intel's Programmable Solutions Group has developed a scalable convolutional neural network reference design for deep learning systems using the OpenCL programming language built with our SDK for OpenCL. The design performance is being benchmarked using several popular CNN benchmarks: CIFAR-10, ImageNet and KITTI.
Building the CNN with OpenCL kernels allows true scaling of the design from smaller to larger devices and from one device generation to the next. New designs can be sized using different numbers of kernels at each layer. Performance scaling from one generation to the next also benefits from architectural advancements, such as floating-point engines and frequency scaling. Thus, you achieve greater than linear performance and performance per watt scaling with each new series of devices.
Petascale Analytics - The World of Big Data Requires Big AnalyticsHeiko Joerg Schick
The document discusses big data and analytics technologies. It describes how new technologies like Hadoop and MapReduce enable processing of extremely large datasets. It also discusses future technologies like exascale computing and storage class memory that will be needed to manage increasing data volumes and support real-time analytics.
Coupling Australia’s Researchers to the Global Innovation EconomyLarry Smarr
08.10.02
First Lecture in the
Australian American Leadership Dialogue Scholar Tour
University of Adelaide
Title: Coupling Australia’s Researchers to the Global Innovation Economy
Adelaide, Australia
Coupling Australia’s Researchers to the Global Innovation EconomyLarry Smarr
This document summarizes Dr. Larry Smarr's presentation on linking Australian researchers to the global innovation economy through high-performance networking. Some key points:
- Australia has established a 1 Gbps dedicated connection between the University of Melbourne and UC San Diego to better connect Australian researchers globally.
- Dr. Smarr is visiting Australian universities to launch the next phase of this project - linking major research universities and CSIRO to each other and innovation centers worldwide with AARNet's new 10 Gbps network.
- This unprecedented bandwidth will allow Australian researchers to join emerging global collaborative research efforts on issues critical to Australia's future.
Exploring emerging technologies in the HPC co-design spacejsvetter
This document discusses emerging technologies for high performance computing (HPC), focusing on heterogeneous computing and non-volatile memory. It provides an overview of HPC architectures past and present, highlighting the trend toward more heterogeneous systems using GPUs and other accelerators. The document discusses challenges for applications to adapt to these changing architectures. It also explores potential future technologies like 3D memory and discusses the Department of Energy's efforts in codesign centers to facilitate collaboration between application developers and emerging hardware.
Coupling Australia’s Researchers to the Global Innovation EconomyLarry Smarr
08.10.17
Ninth Lecture in the
Australian American Leadership Dialogue Scholar Tour
University of Sydney
Title: Coupling Australia’s Researchers to the Global Innovation Economy
Sydney, Australia
40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facilityinside-BigData.com
In this deck from the Swiss HPC Conference, Mark Wilkinson presents: 40 Powers of 10 - Simulating the Universe with the DiRAC HPC Facility.
"DiRAC is the integrated supercomputing facility for theoretical modeling and HPC-based research in particle physics, and astrophysics, cosmology, and nuclear physics, all areas in which the UK is world-leading. DiRAC provides a variety of compute resources, matching machine architecture to the algorithm design and requirements of the research problems to be solved. As a single federated Facility, DiRAC allows more effective and efficient use of computing resources, supporting the delivery of the science programs across the STFC research communities. It provides a common training and consultation framework and, crucially, provides critical mass and a coordinating structure for both small- and large-scale cross-discipline science projects, the technical support needed to run and develop a distributed HPC service, and a pool of expertise to support knowledge transfer and industrial partnership projects. The on-going development and sharing of best-practice for the delivery of productive, national HPC services with DiRAC enables STFC researchers to produce world-leading science across the entire STFC science theory program."
Watch the video: https://wp.me/p3RLHQ-k94
Learn more: https://dirac.ac.uk/
and
http://hpcadvisorycouncil.com/events/2019/swiss-workshop/agenda.php
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Early Benchmarking Results for Neuromorphic ComputingDESMOND YUEN
This document summarizes early benchmarking results for neuromorphic computing using Intel's Loihi chip. It finds that Loihi provides orders of magnitude gains over CPUs and GPUs for certain workloads that are directly trained on the chip or use novel bio-inspired algorithms. These include online learning, adaptive control, event-based vision and tactile sensing, constraint satisfaction problems, and nearest neighbor search. Larger networks and problems tend to provide greater performance gains with Loihi.
In this deck from the HPC User Forum at Argonne, Andrew Siegel from Argonne presents: ECP Application Development.
"The Exascale Computing Project is accelerating delivery of a capable exascale computing ecosystem for breakthroughs in scientific discovery, energy assurance, economic competitiveness, and national security. ECP is chartered with accelerating delivery of a capable exascale computing ecosystem to provide breakthrough modeling and simulation solutions to address the most critical challenges in scientific discovery, energy assurance, economic competitiveness, and national security. This role goes far beyond the limited scope of a physical computing system. ECP’s work encompasses the development of an entire exascale ecosystem: applications, system software, hardware technologies and architectures, along with critical workforce development."
Watch the video: https://wp.me/p3RLHQ-kSL
Learn more: https://www.exascaleproject.org
and
http://hpcuserforum.com
Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter
Metacomputer Architecture of the Global LambdaGrid: How Personal Light Paths ...Larry Smarr
08.05.15
Departments of Computer Science / Physics and Astronomy
University of Missouri@Columbia
Title: Metacomputer Architecture of the Global LambdaGrid: How Personal Light Paths are Transforming e-Science
Columbia, MO
Qualcomm is researching on-device artificial intelligence to power personal assistants and other applications through its Qualcomm AI Research division, with a focus on developing efficient neural networks, on-device learning techniques, and personalized models that can operate on mobile devices while protecting privacy. Qualcomm has been optimizing AI software and hardware including its Snapdragon processors for years to enable advanced on-device AI capabilities. The document discusses Qualcomm's research in areas like computer vision, natural language processing, reinforcement learning, and distributed learning to advance on-device intelligent assistants and other applications.
Calit2 has formed two divisional councils to provide leadership and strategic direction. It has also developed multiple communication channels like brochures and websites. Two new Calit2 buildings at UC San Diego and UC Irvine will provide major new laboratories linked by dedicated optical networks for over 1000 researchers working across disciplines like nanotechnology, biomedicine, and digital arts. Calit2 is also working on applications of high-speed networks for areas like telemedicine, disaster response, and digital cinema.
Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analys...Larry Smarr
06.04.26
Invited Talk
CONNECT Board Meeting
Title: Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analysis (CAMERA)
La Jolla, CA
Coupling Australia’s Researchers to the Global Innovation EconomyLarry Smarr
08.10.08
Fourth Lecture in the
Australian American Leadership Dialogue Scholar Tour
Swinburne University
Title: Coupling Australia’s Researchers to the Global Innovation Economy
Hawthorn, Australia
Calit2:Facilitating the Digital HumanitiesLarry Smarr
Calit2 facilitates collaborations between researchers in engineering, arts and humanities through interdisciplinary centers, conferences, exhibitions and an undergraduate research program. It provides advanced visualization technologies for digital humanities projects in areas such as digital archaeology, cultural heritage preservation, and art analysis. Calit2 also supports the development of new software tools and techniques for digital humanities research.
Limiting Global Climatic Disruption by Revolutionary Change in the Global Ene...Larry Smarr
10.06.08
Keynote Opening Talk
Xconomy Forum: The Rise of Smart Energy
Title: Limiting Global Climatic Disruption by Revolutionary Change in the Global Energy System
La Jolla, CA
The Growing Interdependence of the Internet and Climate ChangeLarry Smarr
10.04.30
Distinguished Lecture
Scientific Computing and Imaging (SCI) Institute
University of Utah
Title: The Growing Interdependence of the Internet and Climate Change
Salt Lake City, UT
The Importance of Large-Scale Computer Science Research EffortsLarry Smarr
05.10.20
Talk at Public Seminar on Large-Scale NSF Research Efforts for the Future Computer Museum
Title: The Importance of Large-Scale Computer Science Research Efforts
Mountain View, CA
Assay Lab Within Your Body: Biometrics and BiomesLarry Smarr
This document summarizes a lecture about analyzing the human microbiome and its relationship to human health. It discusses how the human body contains 100 trillion microbial cells that contain 100 times as many genes as human DNA. Recent advances now allow sequencing these microbial genomes and analyzing massive datasets to map the dynamics of the immune-microbial system and its connection to disease states. A key focus is generating high-resolution time series data of the gut microbiome and immune variables from large cohorts to understand how they influence conditions like inflammatory bowel disease. There is potential to design gut microbes as sensors of disease states by programming them to detect specific conditions.
Applying Photonics to User Needs: The Application ChallengeLarry Smarr
05.02.28
Invited Talk to the 4th Annual On*VECTOR International Photonics Workshop
Sponsored by NTT Network Innovation Laboratories
Title: Applying Photonics to User Needs: The Application Challenge
University of California, San Diego
Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analys...Larry Smarr
06.07.31
Invited Talk
CONNECT Investment Community Meeting
Calit2@UCSD
Title: Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analysis (CAMERA)
La Jolla, CA
Science and Cyberinfrastructure in the Data-Dominated EraLarry Smarr
10.02.22
Invited talk
Symposium #1610, How Computational Science Is Tackling the Grand Challenges Facing Science and Society
Title: Science and Cyberinfrastructure in the Data-Dominated Era
San Diego, CA
Making Sense of Information Through Planetary Scale ComputingLarry Smarr
Larry Smarr discusses how planetary-scale computing and high-speed networks enable data-intensive research through optical portals. This infrastructure allows remote visualization and analysis of large datasets across multiple sites in real-time. Examples include viewing microbial genomes, cosmological simulations, and remote instrument control. The infrastructure also aims to reduce carbon emissions through more efficient computing.
How to Terminate the GLIF by Building a Campus Big Data Freeway SystemLarry Smarr
12.10.11
Keynote Lecture
12th Annual Global LambdaGrid Workshop
Title: How to Terminate the GLIF by Building a Campus Big Data Freeway System
Chicago, IL
End-to-end Optical Fiber Cyberinfrastructure for Data-Intensive Research: Imp...Larry Smarr
10.10.13
Featured Speaker EDUCAUSE 2010
Anaheim Convention Center
Title: End-to-end Optical Fiber Cyberinfrastructure for Data-Intensive Research: Implications for Your Campus
Anaheim, CA
High Performance Cyberinfrastructure Required for Data Intensive Scientific R...Larry Smarr
11.06.08
Invited Presentation
National Science Foundation Advisory Committee on Cyberinfrastructure
Title: High Performance Cyberinfrastructure Required for Data Intensive Scientific Research
Arlington, VA
High Performance Cyberinfrastructure Enabling Data-Driven Science in the Biom...Larry Smarr
11.04.06
Joint Presentation
UCSD School of Medicine Research Council
Larry Smarr, Calit2 & Phil Papadopoulos, SDSC/Calit2
Title: High Performance Cyberinfrastructure Enabling Data-Driven Science in the Biomedical Sciences
Using Photonics to Prototype the Research Campus Infrastructure of the Future...Larry Smarr
08.02.21
Presentation
Philip Papadopoulos, Larry Smarr, Joseph Ford, Shaya Fainman, and Brian Dunne
University of California, San Diego
Title: Using Photonics to Prototype the Research Campus Infrastructure of the Future: The UCSD Quartzite Project
La Jolla, CA
06.07.26
Invited Talk
Cyberinfrastructure for Humanities, Arts, and Social Sciences, A Summer Institute, SDSC
Title: The OptIPuter and Its Applications
La Jolla, CA
Riding the Light: How Dedicated Optical Circuits are Enabling New ScienceLarry Smarr
The document discusses how dedicated optical circuits are enabling new science through high-bandwidth networks. It provides examples of several projects using dedicated optical networks, such as the OptIPuter project, to enable interactive analysis of large datasets through terabit network connections between supercomputing centers. The document concludes by discussing future ocean observatory networks that will use undersea fiber optics to enable remote interactive imaging and sensing.
Project StarGate An End-to-End 10Gbps HPC to User Cyberinfrastructure ANL * C...Larry Smarr
09.11.03
Report to the
Dept. of Energy Advanced Scientific Computing Advisory Committee
Title: Project StarGate An End-to-End 10Gbps HPC to User Cyberinfrastructure ANL * Calit2 * LBNL * NICS * ORNL * SDSC
Oak Ridge, TN
A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Int...Larry Smarr
11.12.12
Seminar Presentation
Princeton Institute for Computational Science and Engineering (PICSciE)
Princeton University
Title: A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research
Princeton, NJ
The document summarizes plans for two new Calit2 buildings at UC San Diego and UC Irvine that will provide laboratories for research in areas like nanotechnology, biomedical engineering, computer chips, and more. The buildings will be linked via high-speed optical networks and will support over 1000 researchers. Key aspects include ultra high-speed networking capabilities up to 10 gigabits per second, advanced visualization resources, and proposals to extend this infrastructure to enable new collaborative research projects.
High Performance Cyberinfrastructure is Needed to Enable Data-Intensive Scien...Larry Smarr
11.03.28
Remote Luncheon Presentation from Calit2@UCSD
National Science Board
Expert Panel Discussion on Data Policies
National Science Foundation
Title: High Performance Cyberinfrastructure is Needed to Enable Data-Intensive Science and Engineering
Arlington, Virginia
Preparing Your Campus for Data Intensive ResearchersLarry Smarr
The document discusses preparing university campuses for data-intensive researchers through high-performance cyberinfrastructure like the OptIPuter project. It describes how dedicated lightpaths can provide researchers with local scalable computing and storage through "OptIPortals" connected to global data repositories at speeds far exceeding normal internet. Several universities have deployed this to open new frontiers in research across diverse disciplines from science to humanities.
SDVIs and In-Situ Visualization on TACC's StampedeIntel® Software
Speaker: Paul Navrátil, Texas Advanced Computing Center (TACC)
The design emphasis for supercomputing systems has moved from raw performance to performance-per-watt, and as a result, supercomputing architectures are converging on processors with wide vector units and many processing cores per chip. Such processors are capable of performant image rendering purely in software. This improved capability is fortuitous, since the prevailing homogeneous system designs lack dedicated, hardware-accelerated rendering subsystems for use in data visualization. Reliance on this “software-defined” rendering capability will grow in importance since, due to growing data sizes, visualizations must be performed on the same machine where the data is produced. Further, as data sizes outgrow disk I/O capacity, visualization will be increasingly incorporated into the simulation code itself (in situ visualization).
This talk presents recent work in high-fidelity visualization using the OSPRay ray tracing framework on TACC’s local and remote visualization systems. We present work using OSPRay within ParaView Catalyst in situ framework from Kitware, including capitalizing on opportunities to reduce data costs migrating through VTK filters for visualization. We highlight the performance opportunities and advantages of Intel® Advanced Vector Extensions 512, the memory system improvements possible with Intel® Xeon Phi™ processor multi-channel DRAM (MCDRAM) and the Intel® Omni-Path Architecture interconnect.
Why Researchers are Using Advanced NetworksLarry Smarr
07.07.03
Remote Talk from Calit2 to:
Building KAREN Communities for Collaboration Forum
KIWI Advanced Research and Education Network
University of Auckland, Auckland City, New Zealand
Title: Why Researchers are Using Advanced Networks
La Jolla, CA
This document provides an overview of recent advances in artificial intelligence and machine learning, including convolutional neural networks, generative adversarial networks, reinforcement learning techniques, and applications in healthcare, autonomous vehicles, robotics, and more. It also highlights Nvidia's work in these areas through their GPUs, deep learning platforms, and research.
Larry Smarr - Making Sense of Information Through Planetary Scale ComputingDiamond Exchange
"Brave New World" DiamondExchange
February 28 - March 3, 2009
Date: Sunday, March 1, 2009
Presenter: Larry Smarr
Presentation: Making Sense of Information Through Planetary Scale Computing
Introduction to Software Defined Visualization (SDVis)Intel® Software
This document provides an overview of Intel's Software Defined Visualization (SDVis) initiative and updates on its current status. SDVis aims to enable scalable, flexible visualization that can run on a variety of systems from laptops to large clusters. It utilizes several open source libraries developed by Intel including Embree for ray tracing, OSPRay as a rendering engine, and OpenSWR for rasterization. The document discusses how SDVis addresses challenges of large-scale, high performance visualization. It provides examples of scientific visualization projects using SDVis and performance comparisons of Embree and OSPRay to GPU-based solutions. In addition, the document outlines several active integrations of SDVis technologies in visualization software including ParaView and
Similar to High Performance Cyberinfrastructure Enables Data-Driven Science in the Globally Networked World (20)
The Rise of Supernetwork Data Intensive ComputingLarry Smarr
Invited Remote Lecture to SC21
The International Conference for High Performance Computing, Networking, Storage, and Analysis
St. Louis, Missouri
November 18, 2021
My Remembrances of Mike Norman Over The Last 45 YearsLarry Smarr
Mike Norman has been a leader in computational astrophysics for over 45 years. Some of his influential work includes:
- Cosmic jet simulations in the early 1980s which helped explain phenomena from galactic centers.
- Pioneering the use of adaptive mesh refinement in the 1990s to achieve dynamic load balancing on supercomputers.
- Massive cosmology simulations in the late 2000s with over 100 trillion particles using thousands of processors across multiple supercomputing sites, producing petabytes of data.
- Developing end-to-end workflows in the 2000s to couple supercomputers, high-speed networks, and large visualization systems to enable real-time analysis of extremely large astrophysics simulations.
Metagenics How Do I Quantify My Body and Try to Improve its Health? June 18 2019Larry Smarr
Larry Smarr discusses quantifying his body and health over time through extensive self-tracking. He measures various biomarkers through regular blood tests and analyzes his gut microbiome by sequencing stool samples. This revealed issues like chronic inflammation and an unhealthy microbiome. Smarr then took steps like a restricted eating window and increasing plant diversity in his diet, which reversed metabolic syndrome issues and correlated with shifts in his microbiome ecology. His goal is to continue precisely measuring factors like toxins, hormones, gut permeability and food/supplement impacts to further optimize his health.
Panel: Reaching More Minority Serving InstitutionsLarry Smarr
This document discusses engaging more minority serving institutions (MSIs) in cyberinfrastructure development through regional networks. It provides data showing the importance of MSIs like historically black colleges and universities (HBCUs) in educating underrepresented minority students in STEM fields. Regional networks can help equalize opportunities by assisting MSIs in overcoming barriers to resources through training, networking infrastructure support, and helping institutions obtain necessary staffing and funding. Strategies mentioned include collaborating with MSIs on grants and addressing issues identified in surveys like lack of vision for data use beyond compliance. The goal is to broaden participation in STEAM fields by leveraging the success MSIs have shown in supporting underrepresented students.
Global Network Advancement Group - Next Generation Network-Integrated SystemsLarry Smarr
This document summarizes a presentation on global petascale to exascale workflows for data intensive sciences. It discusses a partnership convened by the GNA-G Data Intensive Sciences Working Group with the mission of meeting challenges faced by data-intensive science programs. Cornerstone concepts that will be demonstrated include integrated network and site resource management, model-driven frameworks for resource orchestration, end-to-end monitoring with machine learning-optimized data transfers, and integrating Qualcomm's GradientGraph with network services to optimize applications and science workflows.
Wireless FasterData and Distributed Open Compute Opportunities and (some) Us...Larry Smarr
This document discusses opportunities for ESnet to support wireless edge computing through developing a strategy around self-guided field laboratories (SGFL). It outlines several potential science use cases that could benefit from wireless and distributed computing capabilities, both in the short term through technologies like 5G, LoRa and Starlink, and longer term through the vision of automated SGFL. The document proposes some initial ideas for deploying and testing wireless edge computing technologies through existing projects to help enable the SGFL vision and further scientific opportunities. It emphasizes that exploring these emerging areas could help drive new science possibilities if done at a reasonable scale.
Join educators from the US and worldwide at this year’s conference, themed “Strategies for Proficiency & Acquisition,” to learn from top experts in world language teaching.
Delegation Inheritance in Odoo 17 and Its Use CasesCeline George
There are 3 types of inheritance in odoo Classical, Extension, and Delegation. Delegation inheritance is used to sink other models to our custom model. And there is no change in the views. This slide will discuss delegation inheritance and its use cases in odoo 17.
How to Create Sequence Numbers in Odoo 17Celine George
Sequence numbers are mainly used to identify or differentiate each record in a module. Sequences are customizable and can be configured in a specific pattern such as suffix, prefix or a particular numbering scheme. This slide will show how to create sequence numbers in odoo 17.
Understanding and Interpreting Teachers’ TPACK for Teaching Multimodalities i...Neny Isharyanti
Presented as a plenary session in iTELL 2024 in Salatiga on 4 July 2024.
The plenary focuses on understanding and intepreting relevant TPACK competence for teachers to be adept in teaching multimodality in the digital age. It juxtaposes the results of research on multimodality with its contextual implementation in the teaching of English subject in the Indonesian Emancipated Curriculum.
Principles of Roods Approach!!!!!!!.pptxibtesaam huma
Principles of Rood’s Approach
Treatment technique used in physiotherapy for neurological patients which aids them to recover and improve quality of life
Facilitatory techniques
Inhibitory techniques
How to Store Data on the Odoo 17 WebsiteCeline George
Here we are going to discuss how to store data in Odoo 17 Website.
It includes defining a model with few fields in it. Add demo data into the model using data directory. Also using a controller, pass the values into the template while rendering it and display the values in the website.
How to Install Theme in the Odoo 17 ERPCeline George
With Odoo, we can select from a wide selection of attractive themes. Many excellent ones are free to use, while some require payment. Putting an Odoo theme in the Odoo module directory on our server, downloading the theme, and then installing it is a simple process.
The membership Module in the Odoo 17 ERPCeline George
Some business organizations give membership to their customers to ensure the long term relationship with those customers. If the customer is a member of the business then they get special offers and other benefits. The membership module in odoo 17 is helpful to manage everything related to the membership of multiple customers.
Lecture_Notes_Unit4_Chapter_8_9_10_RDBMS for the students affiliated by alaga...Murugan Solaiyappan
Title: Relational Database Management System Concepts(RDBMS)
Description:
Welcome to the comprehensive guide on Relational Database Management System (RDBMS) concepts, tailored for final year B.Sc. Computer Science students affiliated with Alagappa University. This document covers fundamental principles and advanced topics in RDBMS, offering a structured approach to understanding databases in the context of modern computing. PDF content is prepared from the text book Learn Oracle 8I by JOSE A RAMALHO.
Key Topics Covered:
Main Topic : DATA INTEGRITY, CREATING AND MAINTAINING A TABLE AND INDEX
Sub-Topic :
Data Integrity,Types of Integrity, Integrity Constraints, Primary Key, Foreign key, unique key, self referential integrity,
creating and maintain a table, Modifying a table, alter a table, Deleting a table
Create an Index, Alter Index, Drop Index, Function based index, obtaining information about index, Difference between ROWID and ROWNUM
Target Audience:
Final year B.Sc. Computer Science students at Alagappa University seeking a solid foundation in RDBMS principles for academic and practical applications.
About the Author:
Dr. S. Murugan is Associate Professor at Alagappa Government Arts College, Karaikudi. With 23 years of teaching experience in the field of Computer Science, Dr. S. Murugan has a passion for simplifying complex concepts in database management.
Disclaimer:
This document is intended for educational purposes only. The content presented here reflects the author’s understanding in the field of RDBMS as of 2024.
Feedback and Contact Information:
Your feedback is valuable! For any queries or suggestions, please contact muruganjit@agacollege.in
The Jewish Trinity : Sabbath,Shekinah and Sanctuary 4.pdfJackieSparrow3
we may assume that God created the cosmos to be his great temple, in which he rested after his creative work. Nevertheless, his special revelatory presence did not fill the entire earth yet, since it was his intention that his human vice-regent, whom he installed in the garden sanctuary, would extend worldwide the boundaries of that sanctuary and of God’s presence. Adam, of course, disobeyed this mandate, so that humanity no longer enjoyed God’s presence in the little localized garden. Consequently, the entire earth became infected with sin and idolatry in a way it had not been previously before the fall, while yet in its still imperfect newly created state. Therefore, the various expressions about God being unable to inhabit earthly structures are best understood, at least in part, by realizing that the old order and sanctuary have been tainted with sin and must be cleansed and recreated before God’s Shekinah presence, formerly limited to heaven and the holy of holies, can dwell universally throughout creation
AI Risk Management: ISO/IEC 42001, the EU AI Act, and ISO/IEC 23894PECB
As artificial intelligence continues to evolve, understanding the complexities and regulations regarding AI risk management is more crucial than ever.
Amongst others, the webinar covers:
• ISO/IEC 42001 standard, which provides guidelines for establishing, implementing, maintaining, and continually improving AI management systems within organizations
• insights into the European Union's landmark legislative proposal aimed at regulating AI
• framework and methodologies prescribed by ISO/IEC 23894 for identifying, assessing, and mitigating risks associated with AI systems
Presenters:
Miriama Podskubova - Attorney at Law
Miriama is a seasoned lawyer with over a decade of experience. She specializes in commercial law, focusing on transactions, venture capital investments, IT, digital law, and cybersecurity, areas she was drawn to through her legal practice. Alongside preparing contract and project documentation, she ensures the correct interpretation and application of European legal regulations in these fields. Beyond client projects, she frequently speaks at conferences on cybersecurity, online privacy protection, and the increasingly pertinent topic of AI regulation. As a registered advocate of Slovak bar, certified data privacy professional in the European Union (CIPP/e) and a member of the international association ELA, she helps both tech-focused startups and entrepreneurs, as well as international chains, to properly set up their business operations.
Callum Wright - Founder and Lead Consultant Founder and Lead Consultant
Callum Wright is a seasoned cybersecurity, privacy and AI governance expert. With over a decade of experience, he has dedicated his career to protecting digital assets, ensuring data privacy, and establishing ethical AI governance frameworks. His diverse background includes significant roles in security architecture, AI governance, risk consulting, and privacy management across various industries, thorough testing, and successful implementation, he has consistently delivered exceptional results.
Throughout his career, he has taken on multifaceted roles, from leading technical project management teams to owning solutions that drive operational excellence. His conscientious and proactive approach is unwavering, whether he is working independently or collaboratively within a team. His ability to connect with colleagues on a personal level underscores his commitment to fostering a harmonious and productive workplace environment.
Date: June 26, 2024
Tags: ISO/IEC 42001, Artificial Intelligence, EU AI Act, ISO/IEC 23894
-------------------------------------------------------------------------------
Find out more about ISO training and certification services
Training: ISO/IEC 42001 Artificial Intelligence Management System - EN | PECB
Webinars: https://pecb.com/webinars
Article: https://pecb.com/article
-------------------------------------------------------------------------------
How to Configure Time Off Types in Odoo 17Celine George
Now we can take look into how to configure time off types in odoo 17 through this slide. Time-off types are used to grant or request different types of leave. Only then the authorities will have a clear view or a clear understanding of what kind of leave the employee is taking.
Split Shifts From Gantt View in the Odoo 17Celine George
Odoo allows users to split long shifts into multiple segments directly from the Gantt view.Each segment retains details of the original shift, such as employee assignment, start time, end time, and specific tasks or descriptions.
High Performance Cyberinfrastructure Enables Data-Driven Science in the Globally Networked World
1. ―High Performance Cyberinfrastructure Enables
Data-Driven Science in
the Globally Networked World‖
Invited Speaker
Grand Challenges in Data-Intensive Discovery Conference
San Diego Supercomputer Center, UC San Diego
La Jolla, CA
October 28, 2010
Dr. Larry Smarr
Director, California Institute for Telecommunications and Information Technology
Harry E. Gruber Professor, Dept. of Computer Science and Engineering
Jacobs School of Engineering, UCSD
Follow me on Twitter: lsmarr
2. Abstract
Today we are living in a data-dominated world where distributed scientific instruments,
as well as supercomputers, generate terabytes to petabytes of data. It was in response to
this challenge that the NSF funded the OptIPuter project to research how user-controlled
10Gbps dedicated lightpaths (or ―lambdas‖) could provide direct access to global data
repositories, scientific instruments, and computational resources from ―OptIPortals,‖ PC
clusters which provide scalable visualization, computing, and storage in the user's
campus laboratory. The use of dedicated lightpaths over fiber optic cables enables
individual researchers to experience ―clear channel‖ 10,000 megabits/sec, 100-1000
times faster than over today’s shared Internet—a critical capability for data-intensive
science. The seven-year OptIPuter computer science research project is now over, but it
stimulated a national and global build-out of dedicated fiber optic networks. U.S.
universities now have access to high bandwidth lambdas through the National
LambdaRail, Internet2's WaveCo, and the Global Lambda Integrated Facility. A few
pioneering campuses are now building on-campus lightpaths to connect the data-
intensive researchers, data generators, and vast storage systems to each other on
campus, as well as to the national network campus gateways. I will give examples of the
application use of this emerging high performance cyberinfrastructure in genomics,
ocean observatories, radio astronomy, and cosmology.
3. Academic Research ―OptIPlatform‖ Cyberinfrastructure:
A 10Gbps ―End-to-End‖ Lightpath Cloud
HD/4k Video Cams
HD/4k Telepresence
Instruments
End User HPC
OptIPortal
10G
Lightpaths
National LambdaRail
Campus
Optical Switch
Data Repositories & Clusters HD/4k Video Images
4. The OptIPuter Project: Creating High Resolution Portals
Over Dedicated Optical Channels to Global Science Data
Scalable
Adaptive
Graphics
Environment
(SAGE)
Picture
Source:
Mark
Ellisman,
David Lee,
Jason Leigh
Calit2 (UCSD, UCI), SDSC, and UIC Leads—Larry Smarr PI
Univ. Partners: NCSA, USC, SDSU, NW, TA&M, UvA, SARA, KISTI, AIST
Industry: IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent
5. On-Line Resources
Help You Build Your Own OptIPortal
www.optiputer.net
http://wiki.optiputer.net/optiportal
www.evl.uic.edu/cavern/sage/
http://vis.ucsd.edu/~cglx/
OptIPortals Are Built
From Commodity PC Clusters and LCDs
To Create a 10Gbps Scalable Termination Device
6. Nearly Seamless AESOP OptIPortal
46‖ NEC Ultra-Narrow Bezel 720p LCD Monitors
Source: Tom DeFanti, Calit2@UCSD;
7. 3D Stereo Head Tracked OptIPortal:
NexCAVE
Array of JVC HDTV 3D LCD Screens
KAUST NexCAVE = 22.5MPixels
www.calit2.net/newsroom/article.php?id=1584
Source: Tom DeFanti, Calit2@UCSD
8. Project StarGate Goals:
Combining Supercomputers and Supernetworks
• Create an ―End-to-End‖
10Gbps Workflow
• Explore Use of OptIPortals as
OptIPortal@SDSC
Petascale Supercomputer
―Scalable Workstations‖
• Exploit Dynamic 10Gbps
Circuits on ESnet
• Connect Hardware Resources
at ORNL, ANL, SDSC
• Show that Data Need Not be
Trapped by the Network Rick Wagner Mike Norman
―Event Horizon‖
Source: Michael Norman, SDSC, UCSD
• ANL * Calit2 * LBNL * NICS * ORNL * SDSC
9. Using Supernetworks to Couple End User’s OptIPortal
to Remote Supercomputers and Visualization Servers
Source: Mike Norman,
Rick Wagner, SDSC Argonne NL
DOE Eureka
100 Dual Quad Core Xeon Servers
200 NVIDIA Quadro FX GPUs in 50
Quadro Plex S4 1U enclosures
3.2 TB RAM rendering
ESnet
SDSC 10 Gb/s fiber optic network
NICS
visualization
ORNL
Calit2/SDSC OptIPortal1
20 30‖ (2560 x 1600 pixel) LCD panels
NSF TeraGrid Kraken simulation
Cray XT5
10 NVIDIA Quadro FX 4600 graphics
8,256 Compute Nodes
cards > 80 megapixels
99,072 Compute Cores
10 Gb/s network throughout
129 TB RAM
*ANL * Calit2 * LBNL * NICS * ORNL * SDSC
10. National-Scale Interactive Remote Rendering
of Large Datasets
SDSC ESnet ALCF
Science Data Network (SDN)
> 10 Gb/s Fiber Optic Network
Dynamic VLANs Configured
Using OSCARS
Rendering
Visualization Eureka
OptIPortal (40M pixels LCDs) 100 Dual Quad Core Xeon Servers
10 NVIDIA FX 4600 Cards 200 NVIDIA FX GPUs
10 Gb/s Network Throughout 3.2 TB RAM
Interactive Remote Rendering
Real-Time Volume Rendering Streamed from ANL to SDSC
Last Year Last Week
High-Resolution (4K+, 15+ FPS)—But: Now Driven by a Simple Web GUI
• Command-Line Driven •Rotate, Pan, Zoom
• Fixed Color Maps, Transfer Functions •GUI Works from Most Browsers
• Slow Exploration of Data • Manipulate Colors and Opacity
• Fast Renderer Response Time
Source: Rick Wagner, SDSC
11. NSF OOI is a $400M Program
-OOI CI is $34M Part of This
30-40 Software Engineers
Housed at Calit2@UCSD
Source: Matthew Arrott, Calit2 Program Manager for OOI CI
12. OOI CI
is Built on NLR/I2 Optical Infrastructure
Physical Network Implementation
Source: John Orcutt,
Matthew Arrott, SIO/Calit2
13. California and Washington Universities Are Testing
a 10Gbps Connected Commercial Data Cloud
• Amazon Experiment for Big Data
– Only Available Through CENIC & Pacific NW
GigaPOP
– Private 10Gbps Peering Paths
– Includes Amazon EC2 Computing & S3 Storage
Services
• Early Experiments Underway
– Robert Grossman, Open Cloud Consortium
– Phil Papadopoulos, Calit2/SDSC Rocks
14. Open Cloud OptIPuter Testbed--Manage and Compute
Large Datasets Over 10Gbps Lambdas
CENIC NLR C-Wave Dragon
Open Source SW
Hadoop
• 9 Racks MREN Sector/Sphere
• 500 Nodes Nebula
• 1000+ Cores Thrift, GPB
• 10+ Gb/s Now Eucalyptus
• Upgrading Portions to Benchmarks
100 Gb/s in 2010/2011
14
Source: Robert Grossman, UChicago
15. Ocean Modeling HPC In the Cloud:
Tropical Pacific SST (2 Month Ave 2002)
MIT GCM 1/3 Degree Horizontal Resolution, 51 Levels, Forced by NCEP2.
Grid is 564x168x51, Model State is T,S,U,V,W and Sea Surface Height
Run on EC2 HPC Instance. In Collaboration with OOI CI/Calit2
Source: B. Cornuelle, N. Martinez, C.Papadopoulos COMPAS, SIO
16. Run Timings of Tropical Pacific:
Local SIO ATLAS Cluster and Amazon EC2 Cloud
ATLAS ATLAS ATLAS EC2 HPC EC2 HPC
Ethernet Myrinet, Myrinet Ethernet Ethernet
NFS NFS Local Disk 1 Node Local Disk
Wall Time* 4711 2986 2983 14428 2379
User Time* 3833 2953 2933 1909 1590
System 798 17 19 2764 750
Time* *All times in Seconds
Atlas: 128 Node Cluster @ SIO COMPAS. Myrinet 10G, 8GB/node, ~3yrs old
EC2: HPC Computing Instance, 2.93GHz Nehalem, 24GB/Node, 10GbE
Compilers: Ethernet – GNU FORTRAN with OpenMPI
Myrinet – PGI FORTRAN with MPICH1
Single Node EC2 was Oversubscribed, 48 Process. All Other Parallel
Instances used 6 Physical Nodes, 8 Cores/Node. Model Code has been
Ported to Run on ATLAS, Triton (@SDSC) and in EC2.
Source: B. Cornuelle, N. Martinez, C.Papadopoulos COMPAS, SIO
17. Using Condor and Amazon EC2 on
Adaptive Poisson-Boltzmann Solver (APBS)
• APBS Rocks Roll (NBCR) + EC2 Roll
+ Condor Roll = Amazon VM
• Cluster extension into Amazon using Condor
Local
Running in Amazon Cloud
Cluster EC2 Cloud
NBCR NBCR
VM VM
NBCR
VM
APBS + EC2 + Condor
Source: Phil Papadopoulos,
SDSC/Calit2
18. Moving into the Clouds:
Rocks and EC2
• We Can Build Physical Hosting Clusters & Multiple,
Isolated Virtual Clusters:
– Can I Use Rocks to Author ―Images‖ Compatible with EC2?
(We Use Xen, They Use Xen)
– Can I Automatically Integrate EC2 Virtual Machines into
My Local Cluster (Cluster Extension)
– Submit Locally
– My Own Private + Public Cloud
• What This Will Mean
– All your Existing Software Runs Seamlessly
Among Local and Remote Nodes
– User Home Directories Can Be Mounted
– Queue Systems Work
– Unmodified MPI Works
Source: Phil Papadopoulos, SDSC/Calit2
19. ―Blueprint for the Digital University‖--Report of the
UCSD Research Cyberinfrastructure Design Team
• Focus on Data-Intensive Cyberinfrastructure
April 2009
No Data
Bottlenecks
--Design for
Gigabit/s
Data Flows
http://research.ucsd.edu/documents/rcidt/RCIDTReportFinal2009.pdf
20. Current UCSD Optical Core:
Bridging End-Users to CENIC L1, L2, L3 Services
Quartzite Communications
To 10GigE cluster
node interfaces
Core Year 3
Enpoints:
Quartzite Wavelength
>= 60 endpoints at 10 GigE
Core
Selective
.....
Switch
>= 32 Packet switched Lucent To 10GigE cluster
node interfaces and
>= 32 Switched wavelengths other switches
To cluster nodes
.....
>= 300 Connected endpoints
Glimmerglass
To cluster nodes
.....
Production
GigE Switch with
OOO
Dual 10GigE Upliks
Switch
To cluster nodes
Approximately 0.5 TBit/s
32 10GigE
.....
Arrive at the ―Optical‖ GigE Switch with
Force10 Dual 10GigE Upliks
Center of Campus.
...
GigE Switch with
Switching is a Hybrid of:
To Packet Switch CalREN-HPR
Research
Dual 10GigE Upliks
Packet, Lambda, Circuit --
other
nodes
Cloud
GigE
OOO and Packet Switches
10GigE
Campus Research
4 GigE
4 pair fiber
Cloud
Juniper T320
Source: Phil Papadopoulos, SDSC/Calit2
(Quartzite PI, OptIPuter co-PI)
Quartzite Network MRI #CNS-0421555;
OptIPuter #ANI-0225642
21. UCSD Campus Investment in Fiber Enables
Consolidation of Energy Efficient Computing & Storage
WAN 10Gb:
N x 10Gb CENIC, NLR, I2
Gordon –
HPD System
Cluster Condo
DataOasis
(Central) Storage
Triton – Petascale
Data Analysis
Scientific
Instruments
Digital Data Campus Lab OptIPortal
Collections Cluster Tile Display Wall
Source: Philip Papadopoulos, SDSC/Calit2
22. UCSD Planned Optical Networked
Biomedical Researchers and Instruments
• Connects at 10 Gbps :
CryoElectron
Microscopy Facility – Microarrays
San Diego – Genome Sequencers
Supercomputer – Mass Spectrometry
Center
– Light and Electron
Microscopes
– Whole Body Imagers
– Computing
– Storage
Cellular & Molecular
Medicine East
Calit2@UCSD
Bioengineering
Radiology
Imaging Lab
National
Center for
Microscopy
& Imaging Center for
Molecular Genetics
Pharmaceutical
Sciences Building Cellular & Molecular
Biomedical Research Medicine West
23. Moving to a Shared Campus Data Storage
and Analysis Resource: Triton Resource @ SDSC
Triton
Resource
Large Memory Shared Resource
PSDAF Cluster
• 256/512 GB/sys • 24 GB/Node
• 9TB Total • 6TB Total
• 128 GB/sec • 256 GB/sec
• ~ 9 TF • ~ 20 TF
x256
x28
UCSD Research Labs
Large Scale Storage
• 2 PB
• 40 – 80 GB/sec
• 3000 – 6000 disks
• Phase 0: 1/3 TB, 8GB/s
Campus Research
Network
Source: Philip Papadopoulos, SDSC/Calit2
24. Calit2 Microbial Metagenomics Cluster-
Next Generation Optically Linked Science Data Server
Source: Phil Papadopoulos, SDSC, Calit2
512 Processors
~200TB
~5 Teraflops Sun
1GbE X4500
~ 200 Terabytes Storage and Storage
10GbE
Switched 10GbE
/ Routed
Core
25. Calit2 CAMERA Automatic Overflows
into SDSC Triton
@ SDSC
Triton Resource
@ CALIT2
Transparently CAMERA -
Sends Jobs to Managed
Submit Portal Job Submit
on Triton Portal (VM)
10Gbps
Direct
Mount
CAMERA ==
DATA No Data
Staging
26. Prototyping Next Generation User Access and Large
Data Analysis-Between Calit2 and U Washington
Photo Credit: Alan Decker Feb. 29, 2008
Ginger
Armbrust’s
Diatoms:
Micrographs,
Chromosomes,
Genetic
Assembly
iHDTV: 1500 Mbits/sec Calit2 to
UW Research Channel Over NLR
27. Rapid Evolution of 10GbE Port Prices
Makes Campus-Scale 10Gbps CI Affordable
• Port Pricing is Falling
• Density is Rising – Dramatically
• Cost of 10GbE Approaching Cluster HPC Interconnects
$80K/port
Chiaro
(60 Max)
$ 5K
Force 10
(40 max) ~$1000
(300+ Max)
$ 500
Arista $ 400
48 ports Arista
48 ports
2005 2007 2009 2010
Source: Philip Papadopoulos, SDSC/Calit2
28. 10G Switched Data Analysis Resource:
Data Oasis (RFP Responses Due 10/29/2010)
OptIPuter RCN
Colo
CalRe
32 n
Triton
20
24
32
2
Trestles 12 Existing
40
Storage
Oasis Procurement (RFP)
Dash
8 • Phase0: > 8GB/s sustained, today 1500 –
• RFP for Phase1: > 40 GB/sec for Lustre 2000 TB
> 40
• Nodes must be able to function as Lustre GB/s
OSS (Linux) or NFS (Solaris)
100 • Connectivity to Network is 2 x 10GbE/Node
Gordon • Likely Reserve dollars for inexpensive
replica servers
Source: Philip Papadopoulos, SDSC/Calit2