Larry Smarr discusses how planetary-scale computing and high-speed networks enable data-intensive research through optical portals. This infrastructure allows remote visualization and analysis of large datasets across multiple sites in real-time. Examples include viewing microbial genomes, cosmological simulations, and remote instrument control. The infrastructure also aims to reduce carbon emissions through more efficient computing.
Project StarGate An End-to-End 10Gbps HPC to User Cyberinfrastructure ANL * C...
09.11.03
Report to the
Dept. of Energy Advanced Scientific Computing Advisory Committee
Title: Project StarGate An End-to-End 10Gbps HPC to User Cyberinfrastructure ANL * Calit2 * LBNL * NICS * ORNL * SDSC
Oak Ridge, TN
The Pacific Research Platform: A Regional-Scale Big Data Analytics Cyberinfra...
The document discusses the Pacific Research Platform (PRP), a regional big data cyberinfrastructure connecting researchers across California universities. PRP provides high-speed networks and data transfer nodes to enable sharing of large datasets for projects like medical imaging, cryo-electron microscopy, and machine learning. Recent grants are expanding PRP to add GPUs and non-von Neumann processors to support these computationally intensive applications.
Genomics at the Speed of Light: Understanding the Living Ocean
06.07.17-19
Invited Talk
The Gordon and Betty Moore Foundation 2nd Annual Marine Microbiology Investigator Symposium The Golden Gate Club, The Presidio of San Francisco
Title: Genomics at the Speed of Light: Understanding the Living Ocean
San Francisco, CA
Positioning University of California Information Technology for the Future: S...
05.02.15
Invited Talk
The Vice Chancellor of Research and Chief Information Officer Summit
“Information Technology Enabling Research at the University of California”
Title: Positioning University of California Information Technology for the Future: State, National, and International IT Infrastructure Trends and Directions
Oakland, CA
The document describes the Strongly Coupled LambdaCloud project at Calit2. It discusses two new buildings at Calit2 that house over 1000 researchers working on nanotech, biotech, chips, VR and other areas. It also describes the OptIPuter network that connects these researchers via 10Gbps lightpaths, enabling collaborative data-intensive research worldwide. The network includes 50 OptIPortals connected to resources like supercomputers and satellite imagery.
08.05.27
UCSD CSE 91 - Perspectives in Computer Science (Spring 2008)
Calit2@UCSD
Title: Calit2 - CSE's Living Laboratory for Applications
La Jolla, CA
Calit2 was created in 2000 as a collaboration between UC San Diego and UC Irvine to advance telecommunications and information technologies. It received $100 million from the state to build new facilities. Calit2 brings together over 1000 researchers from various disciplines to work on projects involving nanotechnology, biomedicine, computing and more. It has received over $350 million in federal grants and partners with many private companies. Calit2 operates major research facilities and builds virtual reality systems to enable new forms of collaboration. It aims to help California remain a global innovation leader through technological advancement.
08.01.26
Foundational Talk
Virtual World and Immersive Environments
NASA Ames
Title: Remote Telepresence for Exploring Virtual Worlds
Mountain View, CA
The Jump to Light Speed - Data Intensive Earth Sciences are Leading the Way t...
05.06.14
Keynote to the 15th Federation of Earth Science Information Partners Assembly Meeting: Linking Data and Information to Decision Makers
Title: The Jump to Light Speed - Data Intensive Earth Sciences are Leading the Way to the International LambdaGrid
San Diego, CA
The document discusses the emerging LambdaCloud optical network infrastructure that enables collaborative data-intensive research across institutions. Key points:
- The LambdaCloud provides 10Gbps lightpaths connecting over 50 OptiPortals worldwide, allowing shared access to high-performance computing resources, data repositories, and research instruments.
- Projects include the OptIPlaner for global collaborations, the GreenLight project to measure energy costs of cloud computing for specific research communities, and plans for the Triton supercomputer at UCSD to connect biomedical researchers and instruments.
- Early testbeds show the ability to sort over a billion records across multiple sites sustaining over 5Gbps of network throughput.
The Academic and R&D Sectors' Current and Future Broadband and Fiber Access N...
05.02.23
Invited Access Grid Talk
MSCMC FORUM Series
Examining the National Vision for Global Peace and Prosperity
Title: The Academic and R&D Sectors' Current and Future Broadband and Fiber Access Needs for US Global Competitiveness
Arlington, VA
Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analys...
06.07.31
Invited Talk
CONNECT Investment Community Meeting
Calit2@UCSD
Title: Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analysis (CAMERA)
La Jolla, CA
07.07.03
Remote Talk from Calit2 to:
Building KAREN Communities for Collaboration Forum
KIWI Advanced Research and Education Network
University of Auckland, Auckland City, New Zealand
Title: Why Researchers are Using Advanced Networks
La Jolla, CA
Riding the Light: How Dedicated Optical Circuits are Enabling New Science
The document discusses how dedicated optical circuits are enabling new science through high-bandwidth networks. It provides examples of several projects using dedicated optical networks, such as the OptIPuter project, to enable interactive analysis of large datasets through terabit network connections between supercomputing centers. The document concludes by discussing future ocean observatory networks that will use undersea fiber optics to enable remote interactive imaging and sensing.
High Performance Cyberinfrastructure is Needed to Enable Data-Intensive Scien...
11.03.28
Remote Luncheon Presentation from Calit2@UCSD
National Science Board
Expert Panel Discussion on Data Policies
National Science Foundation
Title: High Performance Cyberinfrastructure is Needed to Enable Data-Intensive Science and Engineering
Arlington, Virginia
The document discusses plans to establish a high-bandwidth optical network connection between the California Institute for Telecommunications and Information Technology (Calit2) in the United States and the Center for Scientific Research and Higher Education of Ensenada (CICESE) in Mexico. It describes several visits and collaborations between the institutions over recent years to develop the connection. The goal is to integrate CICESE into Calit2's global OptIPuter network to enable bandwidth-intensive international research collaborations over dedicated optical lambdas.
How Global-Scale Personal Lightwaves are Transforming Scientific Research
07.03.22
Distinguished Lecturer
Technology for a Changing World Series
Baskin School of Engineering, UCSC
Title: How Global-Scale Personal Lighwaves are Transforming Scientific Research
Santa Cruz, CA
The document summarizes plans for two new Calit2 buildings at UC San Diego and UC Irvine that will provide laboratories for research in areas like nanotechnology, biomedical engineering, computer chips, and more. The buildings will be linked via high-speed optical networks and will support over 1000 researchers. Key aspects include ultra high-speed networking capabilities up to 10 gigabits per second, advanced visualization resources, and proposals to extend this infrastructure to enable new collaborative research projects.
Invited Remote Lecture to SC21
The International Conference for High Performance Computing, Networking, Storage, and Analysis
St. Louis, Missouri
November 18, 2021
Calit2: Experiments in Living in the Virtual/Physical WorldLarry Smarr
10.12.15
Invited Talk
"Cultivating Networked Centers of Excellence" CineGrid International Workshop 2010
Title: Calit2: Experiments in Living in the Virtual/Physical World
A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Int...Larry Smarr
11.12.12
Seminar Presentation
Princeton Institute for Computational Science and Engineering (PICSciE)
Princeton University
Title: A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research
Princeton, NJ
Project StarGate An End-to-End 10Gbps HPC to User Cyberinfrastructure ANL * C...Larry Smarr
09.11.03
Report to the
Dept. of Energy Advanced Scientific Computing Advisory Committee
Title: Project StarGate An End-to-End 10Gbps HPC to User Cyberinfrastructure ANL * Calit2 * LBNL * NICS * ORNL * SDSC
Oak Ridge, TN
The Pacific Research Platform: A Regional-Scale Big Data Analytics Cyberinfra...Larry Smarr
The document discusses the Pacific Research Platform (PRP), a regional big data cyberinfrastructure connecting researchers across California universities. PRP provides high-speed networks and data transfer nodes to enable sharing of large datasets for projects like medical imaging, cryo-electron microscopy, and machine learning. Recent grants are expanding PRP to add GPUs and non-von Neumann processors to support these computationally intensive applications.
Genomics at the Speed of Light: Understanding the Living OceanLarry Smarr
06.07.17-19
Invited Talk
The Gordon and Betty Moore Foundation 2nd Annual Marine Microbiology Investigator Symposium The Golden Gate Club, The Presidio of San Francisco
Title: Genomics at the Speed of Light: Understanding the Living Ocean
San Francisco, CA
Positioning University of California Information Technology for the Future: S...Larry Smarr
05.02.15
Invited Talk
The Vice Chancellor of Research and Chief Information Officer Summit
“Information Technology Enabling Research at the University of California”
Title: Positioning University of California Information Technology for the Future: State, National, and International IT Infrastructure Trends and Directions
Oakland, CA
The document describes the Strongly Coupled LambdaCloud project at Calit2. It discusses two new buildings at Calit2 that house over 1000 researchers working on nanotech, biotech, chips, VR and other areas. It also describes the OptIPuter network that connects these researchers via 10Gbps lightpaths, enabling collaborative data-intensive research worldwide. The network includes 50 OptIPortals connected to resources like supercomputers and satellite imagery.
Calit2 - CSE's Living Laboratory for ApplicationsLarry Smarr
08.05.27
UCSD CSE 91 - Perspectives in Computer Science (Spring 2008)
Calit2@UCSD
Title: Calit2 - CSE's Living Laboratory for Applications
La Jolla, CA
Calit2 was created in 2000 as a collaboration between UC San Diego and UC Irvine to advance telecommunications and information technologies. It received $100 million from the state to build new facilities. Calit2 brings together over 1000 researchers from various disciplines to work on projects involving nanotechnology, biomedicine, computing and more. It has received over $350 million in federal grants and partners with many private companies. Calit2 operates major research facilities and builds virtual reality systems to enable new forms of collaboration. It aims to help California remain a global innovation leader through technological advancement.
Remote Telepresence for Exploring Virtual WorldsLarry Smarr
08.01.26
Foundational Talk
Virtual World and Immersive Environments
NASA Ames
Title: Remote Telepresence for Exploring Virtual Worlds
Mountain View, CA
The Jump to Light Speed - Data Intensive Earth Sciences are Leading the Way t...Larry Smarr
05.06.14
Keynote to the 15th Federation of Earth Science Information Partners Assembly Meeting: Linking Data and Information to Decision Makers
Title: The Jump to Light Speed - Data Intensive Earth Sciences are Leading the Way to the International LambdaGrid
San Diego, CA
The document discusses the emerging LambdaCloud optical network infrastructure that enables collaborative data-intensive research across institutions. Key points:
- The LambdaCloud provides 10Gbps lightpaths connecting over 50 OptiPortals worldwide, allowing shared access to high-performance computing resources, data repositories, and research instruments.
- Projects include the OptIPlaner for global collaborations, the GreenLight project to measure energy costs of cloud computing for specific research communities, and plans for the Triton supercomputer at UCSD to connect biomedical researchers and instruments.
- Early testbeds show the ability to sort over a billion records across multiple sites sustaining over 5Gbps of network throughput.
The Academic and R&D Sectors' Current and Future Broadband and Fiber Access N...Larry Smarr
05.02.23
Invited Access Grid Talk
MSCMC FORUM Series
Examining the National Vision for Global Peace and Prosperity
Title: The Academic and R&D Sectors' Current and Future Broadband and Fiber Access Needs for US Global Competitiveness
Arlington, VA
Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analys...Larry Smarr
06.07.31
Invited Talk
CONNECT Investment Community Meeting
Calit2@UCSD
Title: Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analysis (CAMERA)
La Jolla, CA
Why Researchers are Using Advanced NetworksLarry Smarr
07.07.03
Remote Talk from Calit2 to:
Building KAREN Communities for Collaboration Forum
KIWI Advanced Research and Education Network
University of Auckland, Auckland City, New Zealand
Title: Why Researchers are Using Advanced Networks
La Jolla, CA
Riding the Light: How Dedicated Optical Circuits are Enabling New ScienceLarry Smarr
The document discusses how dedicated optical circuits are enabling new science through high-bandwidth networks. It provides examples of several projects using dedicated optical networks, such as the OptIPuter project, to enable interactive analysis of large datasets through terabit network connections between supercomputing centers. The document concludes by discussing future ocean observatory networks that will use undersea fiber optics to enable remote interactive imaging and sensing.
High Performance Cyberinfrastructure is Needed to Enable Data-Intensive Scien...Larry Smarr
11.03.28
Remote Luncheon Presentation from Calit2@UCSD
National Science Board
Expert Panel Discussion on Data Policies
National Science Foundation
Title: High Performance Cyberinfrastructure is Needed to Enable Data-Intensive Science and Engineering
Arlington, Virginia
Bringing Mexico Into the Global LambdaGridLarry Smarr
The document discusses plans to establish a high-bandwidth optical network connection between the California Institute for Telecommunications and Information Technology (Calit2) in the United States and the Center for Scientific Research and Higher Education of Ensenada (CICESE) in Mexico. It describes several visits and collaborations between the institutions over recent years to develop the connection. The goal is to integrate CICESE into Calit2's global OptIPuter network to enable bandwidth-intensive international research collaborations over dedicated optical lambdas.
How Global-Scale Personal Lightwaves are Transforming Scientific ResearchLarry Smarr
07.03.22
Distinguished Lecturer
Technology for a Changing World Series
Baskin School of Engineering, UCSC
Title: How Global-Scale Personal Lighwaves are Transforming Scientific Research
Santa Cruz, CA
The document summarizes plans for two new Calit2 buildings at UC San Diego and UC Irvine that will provide laboratories for research in areas like nanotechnology, biomedical engineering, computer chips, and more. The buildings will be linked via high-speed optical networks and will support over 1000 researchers. Key aspects include ultra high-speed networking capabilities up to 10 gigabits per second, advanced visualization resources, and proposals to extend this infrastructure to enable new collaborative research projects.
Similar to Making Sense of Information Through Planetary Scale Computing (20)
The Rise of Supernetwork Data Intensive ComputingLarry Smarr
Invited Remote Lecture to SC21
The International Conference for High Performance Computing, Networking, Storage, and Analysis
St. Louis, Missouri
November 18, 2021
My Remembrances of Mike Norman Over The Last 45 YearsLarry Smarr
Mike Norman has been a leader in computational astrophysics for over 45 years. Some of his influential work includes:
- Cosmic jet simulations in the early 1980s which helped explain phenomena from galactic centers.
- Pioneering the use of adaptive mesh refinement in the 1990s to achieve dynamic load balancing on supercomputers.
- Massive cosmology simulations in the late 2000s with over 100 trillion particles using thousands of processors across multiple supercomputing sites, producing petabytes of data.
- Developing end-to-end workflows in the 2000s to couple supercomputers, high-speed networks, and large visualization systems to enable real-time analysis of extremely large astrophysics simulations.
Metagenics How Do I Quantify My Body and Try to Improve its Health? June 18 2019Larry Smarr
Larry Smarr discusses quantifying his body and health over time through extensive self-tracking. He measures various biomarkers through regular blood tests and analyzes his gut microbiome by sequencing stool samples. This revealed issues like chronic inflammation and an unhealthy microbiome. Smarr then took steps like a restricted eating window and increasing plant diversity in his diet, which reversed metabolic syndrome issues and correlated with shifts in his microbiome ecology. His goal is to continue precisely measuring factors like toxins, hormones, gut permeability and food/supplement impacts to further optimize his health.
Panel: Reaching More Minority Serving InstitutionsLarry Smarr
This document discusses engaging more minority serving institutions (MSIs) in cyberinfrastructure development through regional networks. It provides data showing the importance of MSIs like historically black colleges and universities (HBCUs) in educating underrepresented minority students in STEM fields. Regional networks can help equalize opportunities by assisting MSIs in overcoming barriers to resources through training, networking infrastructure support, and helping institutions obtain necessary staffing and funding. Strategies mentioned include collaborating with MSIs on grants and addressing issues identified in surveys like lack of vision for data use beyond compliance. The goal is to broaden participation in STEAM fields by leveraging the success MSIs have shown in supporting underrepresented students.
Global Network Advancement Group - Next Generation Network-Integrated SystemsLarry Smarr
This document summarizes a presentation on global petascale to exascale workflows for data intensive sciences. It discusses a partnership convened by the GNA-G Data Intensive Sciences Working Group with the mission of meeting challenges faced by data-intensive science programs. Cornerstone concepts that will be demonstrated include integrated network and site resource management, model-driven frameworks for resource orchestration, end-to-end monitoring with machine learning-optimized data transfers, and integrating Qualcomm's GradientGraph with network services to optimize applications and science workflows.
Wireless FasterData and Distributed Open Compute Opportunities and (some) Us...Larry Smarr
This document discusses opportunities for ESnet to support wireless edge computing through developing a strategy around self-guided field laboratories (SGFL). It outlines several potential science use cases that could benefit from wireless and distributed computing capabilities, both in the short term through technologies like 5G, LoRa and Starlink, and longer term through the vision of automated SGFL. The document proposes some initial ideas for deploying and testing wireless edge computing technologies through existing projects to help enable the SGFL vision and further scientific opportunities. It emphasizes that exploring these emerging areas could help drive new science possibilities if done at a reasonable scale.
Blockchain technology is transforming industries and reshaping the way we conduct business, manage data, and secure transactions. Whether you're new to blockchain or looking to deepen your knowledge, our guidebook, "Blockchain for Dummies", is your ultimate resource.
論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...Toru Tamaki
Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, Volker Tresp, Philip Torr "A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models" arXiv2023
https://arxiv.org/abs/2307.12980
Best Programming Language for Civil EngineersAwais Yaseen
The integration of programming into civil engineering is transforming the industry. We can design complex infrastructure projects and analyse large datasets. Imagine revolutionizing the way we build our cities and infrastructure, all by the power of coding. Programming skills are no longer just a bonus—they’re a game changer in this era.
Technology is revolutionizing civil engineering by integrating advanced tools and techniques. Programming allows for the automation of repetitive tasks, enhancing the accuracy of designs, simulations, and analyses. With the advent of artificial intelligence and machine learning, engineers can now predict structural behaviors under various conditions, optimize material usage, and improve project planning.
How Social Media Hackers Help You to See Your Wife's Message.pdfHackersList
In the modern digital era, social media platforms have become integral to our daily lives. These platforms, including Facebook, Instagram, WhatsApp, and Snapchat, offer countless ways to connect, share, and communicate.
INDIAN AIR FORCE FIGHTER PLANES LIST.pdfjackson110191
These fighter aircraft have uses outside of traditional combat situations. They are essential in defending India's territorial integrity, averting dangers, and delivering aid to those in need during natural calamities. Additionally, the IAF improves its interoperability and fortifies international military alliances by working together and conducting joint exercises with other air forces.
How RPA Help in the Transportation and Logistics Industry.pptxSynapseIndia
Revolutionize your transportation processes with our cutting-edge RPA software. Automate repetitive tasks, reduce costs, and enhance efficiency in the logistics sector with our advanced solutions.
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Chris Swan
Have you noticed the OpenSSF Scorecard badges on the official Dart and Flutter repos? It's Google's way of showing that they care about security. Practices such as pinning dependencies, branch protection, required reviews, continuous integration tests etc. are measured to provide a score and accompanying badge.
You can do the same for your projects, and this presentation will show you how, with an emphasis on the unique challenges that come up when working with Dart and Flutter.
The session will provide a walkthrough of the steps involved in securing a first repository, and then what it takes to repeat that process across an organization with multiple repos. It will also look at the ongoing maintenance involved once scorecards have been implemented, and how aspects of that maintenance can be better automated to minimize toil.
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc
Six months into 2024, and it is clear the privacy ecosystem takes no days off!! Regulators continue to implement and enforce new regulations, businesses strive to meet requirements, and technology advances like AI have privacy professionals scratching their heads about managing risk.
What can we learn about the first six months of data privacy trends and events in 2024? How should this inform your privacy program management for the rest of the year?
Join TrustArc, Goodwin, and Snyk privacy experts as they discuss the changes we’ve seen in the first half of 2024 and gain insight into the concrete, actionable steps you can take to up-level your privacy program in the second half of the year.
This webinar will review:
- Key changes to privacy regulations in 2024
- Key themes in privacy and data governance in 2024
- How to maximize your privacy program in the second half of 2024
Sustainability requires ingenuity and stewardship. Did you know Pigging Solutions pigging systems help you achieve your sustainable manufacturing goals AND provide rapid return on investment.
How? Our systems recover over 99% of product in transfer piping. Recovering trapped product from transfer lines that would otherwise become flush-waste, means you can increase batch yields and eliminate flush waste. From raw materials to finished product, if you can pump it, we can pig it.
Comparison Table of DiskWarrior Alternatives.pdfAndrey Yasko
To help you choose the best DiskWarrior alternative, we've compiled a comparison table summarizing the features, pros, cons, and pricing of six alternatives.
Implementations of Fused Deposition Modeling in real worldEmerging Tech
The presentation showcases the diverse real-world applications of Fused Deposition Modeling (FDM) across multiple industries:
1. **Manufacturing**: FDM is utilized in manufacturing for rapid prototyping, creating custom tools and fixtures, and producing functional end-use parts. Companies leverage its cost-effectiveness and flexibility to streamline production processes.
2. **Medical**: In the medical field, FDM is used to create patient-specific anatomical models, surgical guides, and prosthetics. Its ability to produce precise and biocompatible parts supports advancements in personalized healthcare solutions.
3. **Education**: FDM plays a crucial role in education by enabling students to learn about design and engineering through hands-on 3D printing projects. It promotes innovation and practical skill development in STEM disciplines.
4. **Science**: Researchers use FDM to prototype equipment for scientific experiments, build custom laboratory tools, and create models for visualization and testing purposes. It facilitates rapid iteration and customization in scientific endeavors.
5. **Automotive**: Automotive manufacturers employ FDM for prototyping vehicle components, tooling for assembly lines, and customized parts. It speeds up the design validation process and enhances efficiency in automotive engineering.
6. **Consumer Electronics**: FDM is utilized in consumer electronics for designing and prototyping product enclosures, casings, and internal components. It enables rapid iteration and customization to meet evolving consumer demands.
7. **Robotics**: Robotics engineers leverage FDM to prototype robot parts, create lightweight and durable components, and customize robot designs for specific applications. It supports innovation and optimization in robotic systems.
8. **Aerospace**: In aerospace, FDM is used to manufacture lightweight parts, complex geometries, and prototypes of aircraft components. It contributes to cost reduction, faster production cycles, and weight savings in aerospace engineering.
9. **Architecture**: Architects utilize FDM for creating detailed architectural models, prototypes of building components, and intricate designs. It aids in visualizing concepts, testing structural integrity, and communicating design ideas effectively.
Each industry example demonstrates how FDM enhances innovation, accelerates product development, and addresses specific challenges through advanced manufacturing capabilities.
Kief Morris rethinks the infrastructure code delivery lifecycle, advocating for a shift towards composable infrastructure systems. We should shift to designing around deployable components rather than code modules, use more useful levels of abstraction, and drive design and deployment from applications rather than bottom-up, monolithic architecture and delivery.
The DealBook is our annual overview of the Ukrainian tech investment industry. This edition comprehensively covers the full year 2023 and the first deals of 2024.
Making Sense of Information Through Planetary Scale Computing
1. Making Sense of Information Through Planetary Scale Computing Invited Presentation to the Diamond Exchange—Brave New World Monterey, CA March 1, 2009 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD
2. Data Mining a Decade Ago - NCSA Industrial Partner Projects Caterpillar Effluent Quality Control Smart Selling Warranty Claims Analysis Customer Value Analysis Ford Product Compatibility Harshness, Noise, Vibration Marketing Sears Transaction Management Boeing Post-Flight Diagnostics Allstate Medical Claims Financial Impact May Be Greater Than $30 Million Slide from NCSA 1998
3. JP Morgan Hero Risk Management Calculation Using NCSA Supercomputer Extended JPM's Risk Management Capabilities After Southeast Asia Meltdown Two Week Period in January 1998 NCSA and SGI Doubled Memory in a Week Hundreds of Market Scenarios Simulated HPC Strategic Business Analysis Calculations Used 128-Processor SGI Origin NCSA, Strategic Vendor (SGI), Industrial Partner (JPM) Existing Relationships Facilitated Quick Startup Win-Win-Win Result Andrew Abrahams, Jeff Saltz, JP Morgan Slide from NCSA 1998
4. NCSA / Allstate NT Cluster Data Refinery Terabyte “ Smart Bucket” Source: Allstate & Tilt Thompkins, NCSA CompaqNT Server External Networks CompaqNT Server 1000 Gigabytes of Allstate Claims Data Data Mine on Cleaned Gigabyte Samples NCSA 1998 Visualization Stations Parallel Compute Cluster
5. Academic Research “OptIPlatform” Cyberinfrastructure: A 10,000 Mbps (10Gbps) Lightpath Cloud National LambdaRail Campus Optical Switch Data Repositories & Clusters HPC HD/4k Video Images HD/4k Video Cams End User OptIPortal 10G Lightpath HD/4k Telepresence Instruments
6. Two New Calit2 Buildings Provide Laboratories for “Living in the Future” “ Convergence” Laboratory Facilities Nanotech, BioMEMS, Chips, Radio, Photonics Virtual Reality, Digital Cinema, HDTV, Gaming Over 1000 Researchers in Two Buildings Linked via Dedicated Optical Networks UC San Diego www.calit2.net Over 400 Federal Grants, 200 Companies
7. The Calit2 OptIPortals at UCSD and UCI Are Now a 2 Gbit/s HD Collaboratory Calit2@ UCSD wall NASA Ames Visit Feb. 29, 2008 UCSD cluster: 15 x Quad core Dell XPS with Dual nVIDIA 5600s UCI cluster: 25 x Dual Core Apple G5 Calit2@ UCI wall
9. The Shared Internet is Fine for Email and Web - But It is Not Adequate for Data-Intensive Research Measured Bandwidth from User Computer to Stanford Gigabit Server in Megabits/sec http://netspeed.stanford.edu/ Computers In: Australia Canada Czech Rep. India Japan Korea Mexico Moorea Netherlands Poland Taiwan United States Data Intensive Sciences Require Fast Predictable Bandwidth UCSD Source: Larry Smarr and Friends Stanford Server Limit 100-1000x Normal Internet! Time to Move a Terabyte 10 Days 12 Minutes “ Broadband Internet”
10. Dedicated Optical Fiber Channels Makes High Performance Cyberinfrastructure Possible (WDM) WDM Enables 10Gbps Shared Internet on One Lambda and a Personal 10Gbps Lambda on the Same Fiber!
11. Dedicated 10Gbps Lightpaths Tie Together State and Regional Fiber Infrastructure NLR 40 x 10Gb Wavelengths Expanding with Darkstrand to 80 Interconnects Two Dozen State and Regional Optical Networks Internet2 Dynamic Circuit Network Is Now Available
12. The OptIPuter Creates an OptIPlanet Collaboratory: Enabling Data-Intensive e-Research www.evl.uic.edu/cavern/sage “ OptIPlanet: The OptIPuter Global Collaboratory” – Special Section of Future Generations Computer Systems, Volume 25, Issue 2, February 2009 Calit2 (UCSD, UCI), SDSC, and UIC Leads—Larry Smarr PI Univ. Partners: NCSA, USC, SDSU, NW, TA&M, UvA, SARA, KISTI, AIST Industry: IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent
14. The Rapid Growth in Scalable Visualization ORNL 35Mpixel EVEREST 2004 1999 LLNL 20 Mpixel Wall NCSA 4 MPixel NSF Alliance PowerWall TACC 307 Mpixel Stallion NSF TeraGrid 1997 1999 2004 2005 Calit2@UCI 200 Mpixel HiPerWall NSF MRI EVL 100 Mpixel LambdaVision NSF MRI 2008 A Decade of NSF Investment Two Orders of Magnitude Growth!
15. My OptIPortal TM – Affordable Termination Device for the OptIPuter Global Backplane 20 Dual CPU Nodes, 20 24” Monitors, ~$50,000 1/4 Teraflop, 5 Terabyte Storage, 45 Mega Pixels--Nice PC! Scalable Adaptive Graphics Environment ( SAGE) Jason Leigh, EVL-UIC Source: Phil Papadopoulos SDSC, Calit2
16. Visual Analytics--Use of Tiled Display Wall OptIPortal to Interactively View Microbial Genome (5 Million Bases) Acidobacteria bacterium Ellin345 Soil Bacterium 5.6 Mb; ~5000 Genes Source: Raj Singh, UCSD
17. Use of Tiled Display Wall OptIPortal to Interactively View Microbial Genome Source: Raj Singh, UCSD
18. Use of Tiled Display Wall OptIPortal to Interactively View Microbial Genome Source: Raj Singh, UCSD
19. OptIPortals Scale to 1/3 Billion Pixels Enabling Viewing of Very Large Images or Many Simultaneous Images Spitzer Space Telescope (Infrared) Source: Falko Kuester, Calit2@UCSD NASA Earth Satellite Images Bushfires October 2007 San Diego
20. Calit2/EVL Varrier -- 60 Screen Panorama OptIPortal Dan Sandin, Greg Dawe, Tom Peterka, Tom DeFanti, Jason Leigh, Jinghua Ge, Javier Girado, Bob Kooima, Todd Margolis, Lance Long, Alan Verlo, Maxine Brown, Jurgen Schulze, Qian Liu, Ian Kaufman, Bryan Glogowski Mars Rendered at 46,000 x 23,000 pixels 360 Degree Mars Landscape Rover Spirit at McMurdo 2006 16384 by 4096 pixels Photo: Amy Bennion
21. Calit2 3D Immersive StarCAVE OptIPortal: Enables Exploration of High Resolution Simulations Cluster with 30 Nvidia 5600 cards-60 GB Texture Memory Source: Tom DeFanti, Greg Dawe, Calit2 Connected at 50 Gb/s to Quartzite 30 HD Projectors! 15 Meyer Sound Speakers + Subwoofer Passive Polarization-- Optimized the Polarization Separation and Minimized Attenuation
22. Calit2 VirtuLab- Our Visual Skunkworks Source: Tom DeFanti, Calit2 Autostereo 4k VTC 3D TV 4k on OptIPortal
24. Pattern Recognition Out of Massive Amounts of Cultural Data Software Studies Initiative, Calti2@UCSD Interface Designs for Cultural Analytics Research Environment Jeremy Douglass (top) & Lev Manovich (bottom) Second Annual Meeting of the Humanities, Arts, Science, and Technology Advanced Collaboratory (HASTAC II) UC Irvine May 23, 2008 [email_address] 200 Mpixel HIPerWall
25. Interactive Analysis of Time Evolving Cubes of Data: Cosmological Supercomputer Simulations Two 64K Images From a Cosmological Simulation of Galaxy Cluster Formation Mike Norman, SDSC October 10, 2008 log of gas temperature log of gas density
26. The New Science of Metagenomics “ The emerging field of metagenomics, where the DNA of entire communities of microbes is studied simultaneously, presents the greatest opportunity -- perhaps since the invention of the microscope – to revolutionize understanding of the microbial world.” – National Research Council March 27, 2007 NRC Report: Metagenomic data should be made publicly available in international archives as rapidly as possible.
27. Calit2 Microbial Metagenomics Cluster- Next Generation Optically Linked Science Data Server 512 Processors ~5 Teraflops ~ 200 Terabytes Storage 1GbE and 10GbE Switched / Routed Core ~200TB Sun X4500 Storage 10GbE Source: Phil Papadopoulos, SDSC, Calit2
28. CAMERA’s Global Microbial Metagenomics CyberCommunity Nearly 2500 Registered Users From 55 Countries
29. OptIPuter Persistent Infrastructure Enables Calit2 and U Washington CAMERA Collaboratory Ginger Armbrust’s Diatoms: Micrographs, Chromosomes, Genetic Assembly Photo Credit: Alan Decker Feb. 29, 2008 iHDTV: 1500 Mbits/sec Calit2 to UW Research Channel Over NLR
30. Telepresence Meeting Using Digital Cinema 4k Streams Lays Technical Basis for Global Digital Cinema Sony NTT SGI Streaming 4k with JPEG 2000 Compression ½ Gbit/sec 100 Times the Resolution of YouTube! Calit2@UCSD Auditorium 4k = 4000x2000 Pixels = 4xHD Keio University President Anzai UCSD Chancellor Fox
31. Rendering Supercomputer Data at Digital Cinema Resolution Source: Donna Cox, Robert Patterson, Bob Wilhelmson, NCSA
32. Cisco CWave for CineGrid: A New Cyberinfrastructure for High Resolution Media Streaming* Equinix 818 W. 7th St. Los Angeles PacificWave 1000 Denny Way (Westin Bldg.) Seattle Level3 1360 Kifer Rd. Sunnyvale StarLight Northwestern Univ Chicago Calit2 San Diego McLean CENIC Wave Cisco Has Built 10 GigE Waves on CENIC, PW, & NLR and Installed Large 6506 Switches for Access Points in San Diego, Los Angeles, Sunnyvale, Seattle, Chicago and McLean for CineGrid Members Some of These Points are also GLIF GOLEs Source: John (JJ) Jamison, Cisco May 2007 * 2007 CWave core PoP 10GE waves on NLR and CENIC (LA to SD)
33. Open Cloud OptIPuter Testbed--Manage and Compute Large Datasets Over 10Gbps Lambdas HW Phase 1 (2008) 4 racks 120 Nodes 480 Cores 10+ Gb/s WAN Open Source SW Hadoop Sector/Sphere Thrift, GPB Eucalyptus B enchmarks Phase 2 (2009) will add additional racks to current sites and increase number of sites Source: Robert Grossman, UIC NLR C-Wave MREN CENIC Dragon
34. Terasort on Open Cloud Testbed Sorting 10 Billion Records (1.2 TB) at 4 Sites (120 Nodes) Sustaining >5 Gbps--Only 5% Distance Penalty
37. Just in Time OptIPlanet Collaboratory: Live Session with NASA Ames from Calit2 Source: Falko Kuester, Calit2; Michael Sims, NASA View from NASA Ames Lunar Science Institute Mountain View, CA Virtual Handshake HD compressed 6:1 From Start to This Image in Less Than 2 Weeks! Visit Yesterday by JPL’s Firouz Naderi Feb 19, 2009
38. Remote Control of Scientific Instruments: Live Session with JPL and Mars Rover from Calit2 Source: Falko Kuester, Calit2; Michael Sims, NASA September 17, 2008
39. EVL’s SAGE OptIPortal VisualCasting Multi-Site OptIPuter Collaboratory CENIC CalREN-XD Workshop Sept. 15, 2008 EVL-UI Chicago U Michigan Streaming 4k Source: Jason Leigh, Luc Renambot, EVL, UI Chicago At Supercomputing 2008 Austin, Texas November, 2008 SC08 Bandwidth Challenge Entry Requires 10 Gbps Lightpath to Each Site Total Aggregate VisualCasting Bandwidth for Nov. 18, 2008 Sustained 10,000-20,000 Mbps!
40. U Michigan Virtual Space Interaction Testbed (VISIT) Instrumenting OptIPortals for Social Science Research Using Cameras Embedded in the Seams of Tiled Displays and Computer Vision Techniques, we can Understand how People Interact with OptIPortals Classify Attention, Expression, Gaze Initial Implementation Based on Attention Interaction Design Toolkit (J. Lee, MIT) Close to Producing Usable Eye/Nose Tracking Data using OpenCV Source: Erik Hofer, UMich, School of Information Leading U.S. Researchers on the Social Aspects of Collaboration
42. The Planet is Already Committed to a Dangerous Level of Warming Temperature Threshold Range that Initiates the Climate-Tipping V. Ramanathan and Y. Feng, Scripps Institution of Oceanography, UCSD September 23, 2008 www.pnas.orgcgidoi10.1073pnas.0803838105 Additional Warming over 1750 Level 90% of the Additional 1.6 Degree Warming Will Occur in the 21 st Century
43. The IPCC Recommends a 25-40% Reduction Below 1990 Levels by 2020 On September 27, 2006, Governor Schwarzenegger signed California the Global Warming Solutions Act of 2006 Assembly Bill 32 (AB32) Requires Reduction of GHG by 2020 to 1990 Levels 15% Reduction from 2008 Levels 4 Tons of CO 2 -equiv. for Every Person in California The European Union Requires Reduction of GHG by 2020 to 20% Below 1990 Levels (12/12/2008) Australia has Pledged to Cut by 2020 its GHG Emissions 5% from 2000 Levels via the World's Broadest Cap &Trade Scheme (12/15/08) [~5% Below 1990 Levels] Neither the U.S. or Canada has an Official Target Yet President Elect Obama Has Endorsed the AB32 2020 Goal
44. ICT is a Critical Element in Achieving Countries Greenhouse Gas Emission Reduction Targets Applications of ICT could enable emissions reductions of 7.8 Gt CO 2 e in 2020, or 15% of business as usual emissions. But it must keep its own growing footprint in check and overcome a number of hurdles if it expects to deliver on this potential. www.smart2020.org
45. The Global ICT Carbon Footprint Roughly the Same as the Aviation Industry Today www.smart2020.org
47. Electricity Usage by U.S. Data Centers: Emission Reductions are Underway Source: Silicon Valley Leadership Group Report July 29, 2008 https://microsite.accenture.com/svlgreport/Documents/pdf/SVLG_Report.pdf
48. The UCSD GreenLight Project: Instrumenting the Energy Cost of Computational Science Focus on 5 Communities with At-Scale Computing Needs: Metagenomics Ocean Observing Microscopy Bioinformatics Digital Media Measure, Monitor, & Web Publish Real-Time Sensor Outputs Instrument Eight Racks of Compute, Storage, Routers Outputs Available Via Service-oriented Architectures Allow Researchers Anywhere To Study Computing Energy Cost Develop Middleware that Automates Optimal Choice of Compute/RAM Power Strategies for Desired Greenness Partnering With Minority-Serving Institutions Cyberinfrastructure Empowerment Coalition Source: Tom DeFanti, Calit2; GreenLight PI
49. Application of ICT Can Lead to a 5-Fold Greater Decrease in GHGs Than its Own Carbon Footprint Major Opportunities for the United States* Smart Electrical Grids Smart Transportation Systems Smart Buildings Virtual Meetings * Smart 2020 United States Report Addendum www.smart2020.org While the sector plans to significantly step up the energy efficiency of its products and services, ICT’s largest influence will be by enabling energy efficiencies in other sectors, an opportunity that could deliver carbon savings five times larger than the total emissions from the entire ICT sector in 2020. --Smart 2020 Report
51. UCSD is Installing Zero Carbon Emission Solar and Fuel Cell DC Electricity Generators San Diego’s Point Loma Wastewater Treatment Plant Produces Waste Methane UCSD 2.8 Megawatt Fuel Cell Power Plant Uses Methane 2 Megawatts of Solar Power Cells Being Installed Available Late 2009
52. Launch of ZEVnet Fleet of Wireless Cars-- First Calit2 Testbed for Intelligent Transportation April 18, 2002 Irvine, CA www.zevnet.org
53. Reducing Traffic Congestion: Calit2 California Peer-to -Peer Wireless Traffic Report Citizen to Citizen Accident Reports Real-Time Freeway Speeds “ Leave Now” Paging Services San Diego (866) 500 0977 LA & OC (888) 9 CALIT2 Bay Area (888) 4 CALIT2 http://traffic.calit2.net Source: Ganz Chockalingam, Calit2 20,000+ Users > 1000 Calls Per Day
54. Using High Definition to Link the Calit2 Buildings: Living Greener June 2, 2008 LifeSize System
55. UCSD is Becoming a “Living Laboratory of the Green Future www.gogreentube.com/watch.php?v=NDc4OTQ1
61. “ It Will Be the Biggest Single Peacetime Project Humankind Will Have Ever Undertaken”
Editor's Notes
This is a production cluster with it’s own Force10 e1200 switch. It is connected to quartzite and is labeled as the “CAMERA Force10 E1200”. We built CAMERA this way because of technology deployed successfully in Quartzite