06.01.13
Invited Talk
Department of Computer Science
Donald Bren School of Information and Computer Sciences
Title: Metacomputer Architecture of the Global LambdaGrid
Irvine, CA
Calit2: a View Into the Future of the Wired and Unwired InternetLarry Smarr
06.01.23
Invited Talk to the National Research Council's Computer Science and Telecommunications Board
Title: Calit2: a View Into the Future of the Wired and Unwired Internet
La Jolla, CA
Cloud Camp Milan 2K9 Telecom Italia: Where P2P?Gabriele Bozzi
1. The document discusses the potential for peer-to-peer (P2P) computing as an alternative or complement to the traditional client-server model, especially in the context of cloud computing.
2. It notes challenges with P2P such as lack of centralized control and potential for freeloading, but also advantages like harnessing unused resources.
3. Emerging technologies like autonomic and cognitive networking aim to address P2P challenges by enabling self-configuration and optimization of distributed resources.
1. The document discusses the potential for peer-to-peer (P2P) computing as an alternative or complement to the traditional client-server model, especially in the context of cloud computing.
2. P2P systems offer access to distributed resources but lack centralized control, which makes it difficult to ensure reliability, performance, and security.
3. Autonomic and cognitive approaches may help address issues with P2P by enabling self-configuration, healing, optimization and protection of distributed resources.
4. Future networking approaches like DirecNet envision high-speed mobile mesh networks that could further enable wide-scale distributed computing architectures.
The document discusses future services and developments for the Janet network. It covers several areas including automation, layer 2 and 3 VPNs, virtualized services, optimizing large data transfers, and working with the GÉANT project. perfSONAR is highlighted as a useful tool for measuring network performance when investigating throughput issues or evaluating new deployments.
This document proposes a new platform called CAAP (Cluster As Application Platform) that allows distributed applications to run on wireless device clusters. The key aspects are:
1. Replacing wired links between high-performance computing nodes with wireless D2D (device-to-device) links between mobile devices.
2. Allowing user-programmable networking beyond traditional protocols by programming RDMA verbs.
3. Pooled mobile device clusters can join multiple clusters simultaneously, running multiple applications and instances on shared resources. This opens new opportunities for distributed and edge computing applications.
Project StarGate An End-to-End 10Gbps HPC to User Cyberinfrastructure ANL * C...Larry Smarr
09.11.03
Report to the
Dept. of Energy Advanced Scientific Computing Advisory Committee
Title: Project StarGate An End-to-End 10Gbps HPC to User Cyberinfrastructure ANL * Calit2 * LBNL * NICS * ORNL * SDSC
Oak Ridge, TN
Preparing Your Campus for Data Intensive ResearchersLarry Smarr
The document discusses preparing university campuses for data-intensive researchers through high-performance cyberinfrastructure like the OptIPuter project. It describes how dedicated lightpaths can provide researchers with local scalable computing and storage through "OptIPortals" connected to global data repositories at speeds far exceeding normal internet. Several universities have deployed this to open new frontiers in research across diverse disciplines from science to humanities.
Opportunities for Advanced Technology in TelecommunicationsLarry Smarr
06.12.07
Invited Talk
37th IEEE Semiconductor Interface Specialists Conference
Catamaran Resort Hotel
Title: Opportunities for Advanced Technology in Telecommunications
San Diego, CA
Wimax Emulator to Enhance Media and Video Qualityijceronline
International Journal of Computational Engineering Research (IJCER) is dedicated to protecting personal information and will make every reasonable effort to handle collected information appropriately. All information collected, as well as related requests, will be handled as carefully and efficiently as possible in accordance with IJCER standards for integrity and objectivity.
The document provides information about an ITIC committee briefing at the Marshall Space Flight Center on November 29, 2012. It discusses the membership and activities of the ITIC committee. It also summarizes presentations and topics discussed at the briefing, including SPoRT weather modeling activities, mobile applications, high performance networking, and opportunities for IT innovation on the International Space Station.
Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analys...Larry Smarr
06.07.31
Invited Talk
CONNECT Investment Community Meeting
Calit2@UCSD
Title: Cyberinfrastructure for Advanced Marine Microbial Ecology Research and Analysis (CAMERA)
La Jolla, CA
OptIPuter-A High Performance SOA LambdaGrid Enabling Scientific ApplicationsLarry Smarr
07.03.21
IEEE Computer Society Tsutomu Kanai Award Keynote
At the Joint Meeting of the: 8th International Symposium on Autonomous Decentralized Systems
2nd International Workshop on Ad Hoc, Sensor and P2P Networks
11th IEEE International Workshop on Future Trends of Distributed Computing Systems
Title: OptIPuter-A High Performance SOA LambdaGrid Enabling Scientific Applications
Sedona, AZ
Coupling Australia’s Researchers to the Global Innovation EconomyLarry Smarr
The document summarizes Dr. Larry Smarr's lecture on connecting Australian researchers to the global innovation economy through high-performance networks. It discusses projects that established dedicated 1Gbps and 10Gbps connections between Australian universities and research centers and international partners. This infrastructure will allow Australian researchers to collaborate globally on issues like climate change, health care, and more. The goal is for Australia to have connectivity on par with the best in the world to attract top researchers and partners.
Global Research Platforms: Past, Present, FutureLarry Smarr
The document discusses the benefits of exercise for mental health. Regular physical activity can help reduce anxiety and depression and improve mood and cognitive functioning. Exercise boosts blood flow, releases endorphins, and promotes changes in the brain which help regulate emotions and stress levels.
High Performance Cyberinfrastructure is Needed to Enable Data-Intensive Scien...Larry Smarr
11.03.28
Remote Luncheon Presentation from Calit2@UCSD
National Science Board
Expert Panel Discussion on Data Policies
National Science Foundation
Title: High Performance Cyberinfrastructure is Needed to Enable Data-Intensive Science and Engineering
Arlington, Virginia
Calit2 - CSE's Living Laboratory for ApplicationsLarry Smarr
08.05.27
UCSD CSE 91 - Perspectives in Computer Science (Spring 2008)
Calit2@UCSD
Title: Calit2 - CSE's Living Laboratory for Applications
La Jolla, CA
A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Int...Larry Smarr
11.12.12
Seminar Presentation
Princeton Institute for Computational Science and Engineering (PICSciE)
Princeton University
Title: A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research
Princeton, NJ
Coupling Australia’s Researchers to the Global Innovation EconomyLarry Smarr
08.10.17
Ninth Lecture in the
Australian American Leadership Dialogue Scholar Tour
University of Sydney
Title: Coupling Australia’s Researchers to the Global Innovation Economy
Sydney, Australia
A High-Performance Campus-Scale Cyberinfrastructure for Effectively Bridging ...Larry Smarr
10.04.07
Presentation by Larry Smarr to the NSF Campus Bridging Workshop
University Place Conference Center
Title: A High-Performance Campus-Scale Cyberinfrastructure for Effectively Bridging End-User Laboratories to Data-Intensive Sources
Indianapolis, IN
Calit2 has formed two divisional councils to provide leadership and strategic direction. It has also developed multiple communication channels like brochures and websites. Two new Calit2 buildings at UC San Diego and UC Irvine will provide major new laboratories linked by dedicated optical networks for over 1000 researchers working across disciplines like nanotechnology, biomedicine, and digital arts. Calit2 is also working on applications of high-speed networks for areas like telemedicine, disaster response, and digital cinema.
The Importance of Large-Scale Computer Science Research EffortsLarry Smarr
05.10.20
Talk at Public Seminar on Large-Scale NSF Research Efforts for the Future Computer Museum
Title: The Importance of Large-Scale Computer Science Research Efforts
Mountain View, CA
The document discusses the growing carbon footprint of information and communication technologies (ICT) and efforts to make cyberinfrastructure more energy efficient and environmentally sustainable. Specifically, it mentions that (1) ICT energy usage is growing rapidly and accounts for 2% of global greenhouse gas emissions, (2) universities are working on initiatives like the GreenLight project to reduce ICT energy usage through techniques like dynamic power management, and (3) further research is needed to develop more energy-efficient computing technologies, data center designs, and videoconferencing solutions to reduce the need for travel.
Harnessing the Power of Data From Our Bodies – What I Have Learned by Measuri...Larry Smarr
10.10.05
Invited Talk
Diamond Management & Technology Consultants Meeting, “Taming the Torrent: Harnessing the Power of Data”
Title: Harnessing the Power of Data From Our Bodies – What I Have Learned by Measuring Myself
Kohler, WI
Overview of Photonics Research at Calit2: Scaling from Nanometers to the EarthLarry Smarr
10.03.26
UCSD-NICT Joint Symposium on Innovative Lightwave,
Millimeter-Wave and THz Technologies
for Future Sustainable Network
Title: Overview of Photonics Research at Calit2: Scaling from Nanometers to the Earth
La Jolla, CA
Assay Lab Within Your Body: Biometrics and BiomesLarry Smarr
This document summarizes a lecture about analyzing the human microbiome and its relationship to human health. It discusses how the human body contains 100 trillion microbial cells that contain 100 times as many genes as human DNA. Recent advances now allow sequencing these microbial genomes and analyzing massive datasets to map the dynamics of the immune-microbial system and its connection to disease states. A key focus is generating high-resolution time series data of the gut microbiome and immune variables from large cohorts to understand how they influence conditions like inflammatory bowel disease. There is potential to design gut microbes as sensors of disease states by programming them to detect specific conditions.
06.12.13
Panelist
Panel on Issues, Challenges, and Future Directions of Multimedia Research
IEEE International Symposium on Multimedia (ISM 2006)
Title: Towards GigaPixel Displays
La Jolla, CA
Positioning University of California Information Technology for the Future: S...Larry Smarr
05.02.15
Invited Talk
The Vice Chancellor of Research and Chief Information Officer Summit
“Information Technology Enabling Research at the University of California”
Title: Positioning University of California Information Technology for the Future: State, National, and International IT Infrastructure Trends and Directions
Oakland, CA
Science and Cyberinfrastructure in the Data-Dominated EraLarry Smarr
10.02.22
Invited talk
Symposium #1610, How Computational Science Is Tackling the Grand Challenges Facing Science and Society
Title: Science and Cyberinfrastructure in the Data-Dominated Era
San Diego, CA
The Academic and R&D Sectors' Current and Future Broadband and Fiber Access N...Larry Smarr
05.02.23
Invited Access Grid Talk
MSCMC FORUM Series
Examining the National Vision for Global Peace and Prosperity
Title: The Academic and R&D Sectors' Current and Future Broadband and Fiber Access Needs for US Global Competitiveness
Arlington, VA
The document summarizes plans for two new Calit2 buildings at UC San Diego and UC Irvine that will provide laboratories for research in areas like nanotechnology, biomedical engineering, computer chips, and more. The buildings will be linked via high-speed optical networks and will support over 1000 researchers. Key aspects include ultra high-speed networking capabilities up to 10 gigabits per second, advanced visualization resources, and proposals to extend this infrastructure to enable new collaborative research projects.
The Jump to Light Speed - Data Intensive Earth Sciences are Leading the Way t...Larry Smarr
05.06.14
Keynote to the 15th Federation of Earth Science Information Partners Assembly Meeting: Linking Data and Information to Decision Makers
Title: The Jump to Light Speed - Data Intensive Earth Sciences are Leading the Way to the International LambdaGrid
San Diego, CA
LambdaGrids--Earth and Planetary Sciences Driving High Performance Networks a...Larry Smarr
05.02.04
Invited Talk to the NASA Jet Propulsion Laboratory
Title: LambdaGrids--Earth and Planetary Sciences Driving High Performance Networks and High Resolution Visualizations
Pasadena, CA
Calit2-a Persistent UCSD/UCI Framework for CollaborationLarry Smarr
05.02.16
Invited Talk
Sun Microsystems Global Education and Research
Conference 2005
Title: Calit2-a Persistent UCSD/UCI Framework for Collaboration
San Francisco, CA
This document summarizes a presentation on machine learning and fluid network planes. It begins with an agenda and introduction to fluid network planes and instances. It then discusses the role of machine learning in fluid network planes, including applications such as optimization, virtual network embedding problems, run-time operations, and intent-based closed-loop automation. Recent research is presented on machine learning-based YouTube QoE estimation using real 4G/5G network traces to predict video quality and inform control actions. Results are shown comparing 4G and 5G networks in terms of radio parameters, stalling events, handovers, and video resolutions under different mobility conditions.
Grid computing has evolved over two generations to address the needs of utilizing widely distributed computing resources effectively. The first generation involved projects in the 1990s that linked supercomputing sites, allowing high-performance applications to leverage computational resources across multiple sites. This included projects like FAFNER, which distributed integer factorization computations via a web interface, and I-WAY, which scheduled jobs across 17 US sites connected by a high-performance network. The second generation focused on developing the necessary infrastructure for grid computing to function on a global scale, addressing issues like heterogeneity, scalability, and adaptability. This required core services for administration, communication, information, and naming across distributed systems.
Similar to Metacomputer Architecture of the Global LambdaGrid (20)
The Rise of Supernetwork Data Intensive ComputingLarry Smarr
Invited Remote Lecture to SC21
The International Conference for High Performance Computing, Networking, Storage, and Analysis
St. Louis, Missouri
November 18, 2021
My Remembrances of Mike Norman Over The Last 45 YearsLarry Smarr
Mike Norman has been a leader in computational astrophysics for over 45 years. Some of his influential work includes:
- Cosmic jet simulations in the early 1980s which helped explain phenomena from galactic centers.
- Pioneering the use of adaptive mesh refinement in the 1990s to achieve dynamic load balancing on supercomputers.
- Massive cosmology simulations in the late 2000s with over 100 trillion particles using thousands of processors across multiple supercomputing sites, producing petabytes of data.
- Developing end-to-end workflows in the 2000s to couple supercomputers, high-speed networks, and large visualization systems to enable real-time analysis of extremely large astrophysics simulations.
Metagenics How Do I Quantify My Body and Try to Improve its Health? June 18 2019Larry Smarr
Larry Smarr discusses quantifying his body and health over time through extensive self-tracking. He measures various biomarkers through regular blood tests and analyzes his gut microbiome by sequencing stool samples. This revealed issues like chronic inflammation and an unhealthy microbiome. Smarr then took steps like a restricted eating window and increasing plant diversity in his diet, which reversed metabolic syndrome issues and correlated with shifts in his microbiome ecology. His goal is to continue precisely measuring factors like toxins, hormones, gut permeability and food/supplement impacts to further optimize his health.
Panel: Reaching More Minority Serving InstitutionsLarry Smarr
This document discusses engaging more minority serving institutions (MSIs) in cyberinfrastructure development through regional networks. It provides data showing the importance of MSIs like historically black colleges and universities (HBCUs) in educating underrepresented minority students in STEM fields. Regional networks can help equalize opportunities by assisting MSIs in overcoming barriers to resources through training, networking infrastructure support, and helping institutions obtain necessary staffing and funding. Strategies mentioned include collaborating with MSIs on grants and addressing issues identified in surveys like lack of vision for data use beyond compliance. The goal is to broaden participation in STEAM fields by leveraging the success MSIs have shown in supporting underrepresented students.
Global Network Advancement Group - Next Generation Network-Integrated SystemsLarry Smarr
This document summarizes a presentation on global petascale to exascale workflows for data intensive sciences. It discusses a partnership convened by the GNA-G Data Intensive Sciences Working Group with the mission of meeting challenges faced by data-intensive science programs. Cornerstone concepts that will be demonstrated include integrated network and site resource management, model-driven frameworks for resource orchestration, end-to-end monitoring with machine learning-optimized data transfers, and integrating Qualcomm's GradientGraph with network services to optimize applications and science workflows.
Wireless FasterData and Distributed Open Compute Opportunities and (some) Us...Larry Smarr
This document discusses opportunities for ESnet to support wireless edge computing through developing a strategy around self-guided field laboratories (SGFL). It outlines several potential science use cases that could benefit from wireless and distributed computing capabilities, both in the short term through technologies like 5G, LoRa and Starlink, and longer term through the vision of automated SGFL. The document proposes some initial ideas for deploying and testing wireless edge computing technologies through existing projects to help enable the SGFL vision and further scientific opportunities. It emphasizes that exploring these emerging areas could help drive new science possibilities if done at a reasonable scale.
Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Em...Erasmo Purificato
Slide of the tutorial entitled "Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Emerging Trends" held at UMAP'24: 32nd ACM Conference on User Modeling, Adaptation and Personalization (July 1, 2024 | Cagliari, Italy)
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfNeo4j
Presented at Gartner Data & Analytics, London Maty 2024. BT Group has used the Neo4j Graph Database to enable impressive digital transformation programs over the last 6 years. By re-imagining their operational support systems to adopt self-serve and data lead principles they have substantially reduced the number of applications and complexity of their operations. The result has been a substantial reduction in risk and costs while improving time to value, innovation, and process automation. Join this session to hear their story, the lessons they learned along the way and how their future innovation plans include the exploration of uses of EKG + Generative AI.
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxSynapseIndia
Your comprehensive guide to RPA in healthcare for 2024. Explore the benefits, use cases, and emerging trends of robotic process automation. Understand the challenges and prepare for the future of healthcare automation
UiPath Community Day Kraków: Devs4Devs ConferenceUiPathCommunity
We are honored to launch and host this event for our UiPath Polish Community, with the help of our partners - Proservartner!
We certainly hope we have managed to spike your interest in the subjects to be presented and the incredible networking opportunities at hand, too!
Check out our proposed agenda below 👇👇
08:30 ☕ Welcome coffee (30')
09:00 Opening note/ Intro to UiPath Community (10')
Cristina Vidu, Global Manager, Marketing Community @UiPath
Dawid Kot, Digital Transformation Lead @Proservartner
09:10 Cloud migration - Proservartner & DOVISTA case study (30')
Marcin Drozdowski, Automation CoE Manager @DOVISTA
Pawel Kamiński, RPA developer @DOVISTA
Mikolaj Zielinski, UiPath MVP, Senior Solutions Engineer @Proservartner
09:40 From bottlenecks to breakthroughs: Citizen Development in action (25')
Pawel Poplawski, Director, Improvement and Automation @McCormick & Company
Michał Cieślak, Senior Manager, Automation Programs @McCormick & Company
10:05 Next-level bots: API integration in UiPath Studio (30')
Mikolaj Zielinski, UiPath MVP, Senior Solutions Engineer @Proservartner
10:35 ☕ Coffee Break (15')
10:50 Document Understanding with my RPA Companion (45')
Ewa Gruszka, Enterprise Sales Specialist, AI & ML @UiPath
11:35 Power up your Robots: GenAI and GPT in REFramework (45')
Krzysztof Karaszewski, Global RPA Product Manager
12:20 🍕 Lunch Break (1hr)
13:20 From Concept to Quality: UiPath Test Suite for AI-powered Knowledge Bots (30')
Kamil Miśko, UiPath MVP, Senior RPA Developer @Zurich Insurance
13:50 Communications Mining - focus on AI capabilities (30')
Thomasz Wierzbicki, Business Analyst @Office Samurai
14:20 Polish MVP panel: Insights on MVP award achievements and career profiling
Measuring the Impact of Network Latency at TwitterScyllaDB
Widya Salim and Victor Ma will outline the causal impact analysis, framework, and key learnings used to quantify the impact of reducing Twitter's network latency.
論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...Toru Tamaki
Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, Volker Tresp, Philip Torr "A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models" arXiv2023
https://arxiv.org/abs/2307.12980
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc
Six months into 2024, and it is clear the privacy ecosystem takes no days off!! Regulators continue to implement and enforce new regulations, businesses strive to meet requirements, and technology advances like AI have privacy professionals scratching their heads about managing risk.
What can we learn about the first six months of data privacy trends and events in 2024? How should this inform your privacy program management for the rest of the year?
Join TrustArc, Goodwin, and Snyk privacy experts as they discuss the changes we’ve seen in the first half of 2024 and gain insight into the concrete, actionable steps you can take to up-level your privacy program in the second half of the year.
This webinar will review:
- Key changes to privacy regulations in 2024
- Key themes in privacy and data governance in 2024
- How to maximize your privacy program in the second half of 2024
Blockchain technology is transforming industries and reshaping the way we conduct business, manage data, and secure transactions. Whether you're new to blockchain or looking to deepen your knowledge, our guidebook, "Blockchain for Dummies", is your ultimate resource.
Comparison Table of DiskWarrior Alternatives.pdfAndrey Yasko
To help you choose the best DiskWarrior alternative, we've compiled a comparison table summarizing the features, pros, cons, and pricing of six alternatives.
How RPA Help in the Transportation and Logistics Industry.pptxSynapseIndia
Revolutionize your transportation processes with our cutting-edge RPA software. Automate repetitive tasks, reduce costs, and enhance efficiency in the logistics sector with our advanced solutions.
Mitigating the Impact of State Management in Cloud Stream Processing SystemsScyllaDB
Stream processing is a crucial component of modern data infrastructure, but constructing an efficient and scalable stream processing system can be challenging. Decoupling compute and storage architecture has emerged as an effective solution to these challenges, but it can introduce high latency issues, especially when dealing with complex continuous queries that necessitate managing extra-large internal states.
In this talk, we focus on addressing the high latency issues associated with S3 storage in stream processing systems that employ a decoupled compute and storage architecture. We delve into the root causes of latency in this context and explore various techniques to minimize the impact of S3 latency on stream processing performance. Our proposed approach is to implement a tiered storage mechanism that leverages a blend of high-performance and low-cost storage tiers to reduce data movement between the compute and storage layers while maintaining efficient processing.
Throughout the talk, we will present experimental results that demonstrate the effectiveness of our approach in mitigating the impact of S3 latency on stream processing. By the end of the talk, attendees will have gained insights into how to optimize their stream processing systems for reduced latency and improved cost-efficiency.
Advanced Techniques for Cyber Security Analysis and Anomaly DetectionBert Blevins
Cybersecurity is a major concern in today's connected digital world. Threats to organizations are constantly evolving and have the potential to compromise sensitive information, disrupt operations, and lead to significant financial losses. Traditional cybersecurity techniques often fall short against modern attackers. Therefore, advanced techniques for cyber security analysis and anomaly detection are essential for protecting digital assets. This blog explores these cutting-edge methods, providing a comprehensive overview of their application and importance.
Sustainability requires ingenuity and stewardship. Did you know Pigging Solutions pigging systems help you achieve your sustainable manufacturing goals AND provide rapid return on investment.
How? Our systems recover over 99% of product in transfer piping. Recovering trapped product from transfer lines that would otherwise become flush-waste, means you can increase batch yields and eliminate flush waste. From raw materials to finished product, if you can pump it, we can pig it.
Metacomputer Architecture of the Global LambdaGrid
1. “ Metacomputer Architecture of the Global LambdaGrid " Invited Talk Department of Computer Science Donald Bren School of Information and Computer Sciences University of California, Irvine January 13, 2006 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD
2. Abstract I will describe my research in metacomputer architecture, a term I coined in 1988, in which one builds virtual ensembles of computers, storage, networks, and visualization devices into an integrated system. Working with a set of colleagues, I have driven development in this field through national and international workshops and conferences, including SIGGRAPH, Supercomputing, and iGrid. Although the vision has remained constant over nearly two decades, it is only the recent availability of dedicated optical paths, or lambdas, that has enabled the vision to be realized. These lambdas enable the Grid program to be completed, in that they add the network elements to the compute and storage elements which can be discovered, reserved, and integrated by the Grid middleware to form global LambdaGrids. I will describe my current research in the four grants in which I am PI or co-PI, OptIPuter, Quartzite, LOOKING, and CAMERA, which both develop the computer science of LambdaGrids, but also couple intimately to the application drivers in biomedical imaging, ocean observatories, and marine microbial metagenomics.
3. Metacomputer: Four Eras The Early Days (1985-1995) The Emergence of the Grid (1995-2000) From Grid to LambdaGrid (2000-2005) Community Adoption of LambdaGrid (2005-2006)
5. The First Metacomputer: NSFnet and the Six NSF Supercomputers NCSA NSFNET 56 Kb/s Backbone (1986-8) PSC NCAR CTC JVNC SDSC
6. NCSA Telnet--“Hide the Cray” One of the Inspirations for the Metacomputer NCSA Telnet Provides Interactive Access From Macintosh or PC Computer To Telnet Hosts on TCP/IP Networks Allows for Simultaneous Connections To Numerous Computers on The Net Standard File Transfer Server (FTP) Lets You Transfer Files to and from Remote Machines and Other Users John Kogut Simulating Quantum Chromodynamics He Uses a Mac—The Mac Uses the Cray Source: Larry Smarr 1985
7. From Metacomputer to TeraGrid and OptIPuter: 15 Years of Development TeraGrid PI OptIPuter PI 1992 “ Metacomputer” Coined by Smarr in 1988
8. “ What we really have to do is eliminate distance between individuals who want to interact with other people and with other computers.” ― Larry Smarr, Director, NCSA Long-Term Goal: Dedicated Fiber Optic Infrastructure Using Analog Communications to Prototype the Digital Future “ We’re using satellite technology…to demo what It might be like to have high-speed fiber-optic links between advanced computers in two different geographic locations.” ― Al Gore, Senator Chair, US Senate Subcommittee on Science, Technology and Space Illinois Boston SIGGRAPH 1989
9. NCSA Web Server Traffic Increase Led to NCSA Creating the First Parallel Web Server 1993 1995 1994 Peak was 4 Million Hits per Week! Data Source: Software Development Group, NCSA, Graph: Larry Smarr
11. I-WAY Prototyped the National Metacomputer -- Supercomputing ‘95 I-WAY Project 60 National & Grand Challenge Computing Applications I-Way Featured: IP over ATM with an OC-3 (155Mbps) Backbone Large-Scale Immersive Displays I-Soft Programming Environment Led Directly to Globus UIC http://archive.ncsa.uiuc.edu/General/Training/SC95/GII.HPCC.html CitySpace Cellular Semiotics Source: Larry Smarr, Rick Stevens, Tom DeFanti
12. The NCSA Alliance Research Agenda- Create a National Scale Metacomputer The Alliance will strive to make computing routinely parallel, distributed, collaborative, and immersive. --Larry Smarr, CACM Guest Editor Source: Special Issue of Comm. ACM 1997
13. From Metacomputing to the Grid Ian Foster, Carl Kesselman (Eds), Morgan Kaufmann, 1999 22 chapters by expert authors including: Andrew Chien, Jack Dongarra, Tom DeFanti, Andrew Grimshaw, Roch Guerin, Ken Kennedy, Paul Messina, Cliff Neuman, Jon Postel, Larry Smarr, Rick Stevens, and many others http://www.mkp.com/grids “ A source book for the history of the future” -- Vint Cerf Meeting Held at Argonne Sept 1997
14. Exploring the Limits of Scalability The Metacomputer as a Megacomputer Napster Meets Entropia Distributed Computing and Storage Combined Assume Ten Million PCs in Five Years Average Speed Ten Gigaflop Average Free Storage 100 GB Planetary Computer Capacity 100,000 TeraFLOP Speed 1 Million TeraByte Storage 1000 TeraFLOPs is Roughly a Human Brain-Second Morovec-Intelligent Robots and Mind Transferral Kurzweil-The Age of Spiritual Machines Joy-Humans an Endangered Species? Vinge-Singularity Source: Larry Smarr Megacomputer Panel SC2000 Conference
16. Challenge: Average Throughput of NASA Data Products to End User is < 50 Mbps Tested October 2005 http://ensight.eos.nasa.gov/Missions/icesat/index.shtml Internet2 Backbone is 10,000 Mbps! Throughput is < 0.5% to End User
17. Each Optical Fiber Can Now Carry Many Parallel Line Paths or “Lambdas” ( WDM) Source: Steve Wallach, Chiaro Networks “ Lambdas”
18. States are Acquiring Their Own Dark Fiber Networks -- Illinois’s I-WIRE and Indiana’s I-LIGHT Source: Larry Smarr, Rick Stevens, Tom DeFanti, Charlie Catlett Today Two Dozen State and Regional Optical Networks 1999
19. From “Supercomputer–Centric” to “Supernetwork-Centric” Cyberinfrastructure Megabit/s Gigabit/s Terabit/s Network Data Source: Timothy Lance, President, NYSERNet 32x10Gb “Lambdas” 1 GFLOP Cray2 60 TFLOP Altix Bandwidth of NYSERNet Research Network Backbones T1 Optical WAN Research Bandwidth Has Grown Much Faster Than Supercomputer Speed! Computing Speed (GFLOPS)
20. The OptIPuter Project – Creating a LambdaGrid “Web” for Gigabyte Data Objects NSF Large Information Technology Research Proposal Calit2 (UCSD, UCI) and UIC Lead Campuses—Larry Smarr PI Partnering Campuses: USC, SDSU, NW, TA&M, UvA, SARA, NASA Industrial Partners IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent $13.5 Million Over Five Years Linking Global Scale Science Projects to User’s Linux Clusters NIH Biomedical Informatics NSF EarthScope and ORION Research Network
21. What is the OptIPuter? Applications Drivers Interactive Analysis of Large Data Sets OptIPuter Nodes Scalable PC Clusters with Graphics Cards IP over Lambda Connectivity Predictable Backplane Open Source LambdaGrid Middleware Network is Reservable Data Retrieval and Mining Lambda Attached Data Servers High Defn. Vis., Collab. SW High Performance Collaboratory See Nov 2003 Communications of the ACM for Articles on OptIPuter Technologies www.optiputer.net
22. End User Device Tiled Wall Driven by OptIPuter Graphics Cluster Source: Mark Ellisman, OptIPuter co-PI
23. Campuses Must Provide Fiber Infrastructure to End-User Laboratories & Large Rotating Data Stores SIO Ocean Supercomputer IBM Storage Cluster 2 Ten Gbps Campus Lambda Raceway Streaming Microscope Source: Phil Papadopoulos, SDSC, Calit2 UCSD Campus LambdaStore Architecture Global LambdaGrid
24. OptIPuter@UCI is Up and Working Created 09-27-2005 by Garrett Hildebrand Modified 11-03-2005 by Jessica Yu 10 GE SPDS Catalyst 3750 in CSI ONS 15540 WDM at UCI campus MPOE (CPL) 10 GE DWDM Network Line Engineering Gateway Building, Catalyst 3750 in 3 rd floor IDF MDF Catalyst 6500 w/ firewall, 1 st floor closet Wave-2 : layer-2 GE. UCSD address space 137.110.247.210-222/28 Floor 2 Catalyst 6500 Floor 3 Catalyst 6500 Floor 4 Catalyst 6500 Wave-1 : UCSD address space 137.110.247.242-246 NACS-reserved for testing ESMF Catalyst 3750 in NACS Machine Room (Optiputer) Viz Lab Wave 1 1GE Wave 2 1GE Kim-Jitter Measurements This Week! Calit2 Building UCInet HIPerWall Los Angeles 1 GE DWDM Network Line Tustin CENIC Calren POP UCSD Optiputer Network
25. OptIPuter Software Architecture--a Service-Oriented Architecture Integrating Lambdas Into the Grid GTP XCP UDT LambdaStream CEP RBUDP Globus XIO GRAM GSI DVC Configuration Distributed Virtual Computer (DVC) API DVC Runtime Library Distributed Applications/ Web Services Telescience Vol-a-Tile SAGE JuxtaView Visualization Data Services LambdaRAM DVC Services DVC Core Services DVC Job Scheduling DVC Communication Resource Identify/Acquire Namespace Management Security Management High Speed Communication Storage Services IP Lambdas Discovery and Control PIN/PDC RobuStore
26. Special issue of Communications of the ACM (CACM): Blueprint for the Future of High-Performance Networking Introduction Maxine Brown (guest editor) TransLight: A Global-scale LambdaGrid for e-Science Tom DeFanti, Cees de Laat, Joe Mambretti, Kees Neggers, Bill St. Arnaud Transport Protocols for High Performance Aaron Falk, Ted Faber, Joseph Bannister, Andrew Chien, Bob Grossman, Jason Leigh Data Integration in a Bandwidth-Rich World Ian Foster, Robert Grossman The OptIPuter Larry Smarr, Andrew Chien, Tom DeFanti, Jason Leigh, Philip Papadopoulos Data-Intensive e-Science Frontier Research Harvey Newman, Mark Ellisman, John Orcutt Source: Special Issue of Comm. ACM 2003
27. NSF is Launching a New Cyberinfrastructure Initiative www.ctwatch.org “ Research is being stalled by ‘information overload,’ Mr. Bement said, because data from digital instruments are piling up far faster than researchers can study. In particular, he said, campus networks need to be improved. High-speed data lines crossing the nation are the equivalent of six-lane superhighways, he said. But networks at colleges and universities are not so capable. “Those massive conduits are reduced to two-lane roads at most college and university campuses,” he said. Improving cyberinfrastructure, he said, “will transform the capabilities of campus-based scientists.” -- Arden Bement, the director of the National Science Foundation
28. The Optical Core of the UCSD Campus-Scale Testbed -- Evaluating Packet Routing versus Lambda Switching Goals by 2007: >= 50 endpoints at 10 GigE >= 32 Packet switched >= 32 Switched wavelengths >= 300 Connected endpoints Approximately 0.5 TBit/s Arrive at the “Optical” Center of Campus Switching will be a Hybrid Combination of: Packet, Lambda, Circuit -- OOO and Packet Switches Already in Place Funded by NSF MRI Grant Lucent Glimmerglass Chiaro Networks
29. “ Access Grid” Was Developed by the Alliance for Multi-site Collaboration Access Grid Talk with 35 Locations on 5 Continents— SC Global Keynote Supercomputing ‘04 Problems Are Video Quality of Service and IP Multicasting
30. Multiple HD Streams Over Lambdas Will Radically Transform Global Collaboration U. Washington JGN II Workshop Osaka, Japan Jan 2005 Prof. Osaka Prof. Aoyama Prof. Smarr Source: U Washington Research Channel Telepresence Using Uncompressed 1.5 Gbps HDTV Streaming Over IP on Fiber Optics-- 75x Home Cable “HDTV” Bandwidth!
31. Partnering with NASA to Combine Telepresence with Remote Interactive Analysis of Data Over National LambdaRail HDTV Over Lambda OptIPuter Visualized Data SIO/UCSD NASA Goddard www.calit2.net/articles/article.php?id=660 August 8, 2005
32. September 26-30, 2005 Calit2 @ University of California, San Diego California Institute for Telecommunications and Information Technology The Global Lambda Integrated Facility (GLIF) Creates MetaComputers on the Scale of Planet Earth T H E G L O B A L L A M B D A I N T E G R A T E D F A C I L I T Y Maxine Brown, Tom DeFanti, Co-Chairs www.igrid2005.org 21 Countries Driving 50 Demonstrations 1 or 10Gbps to Calit2@UCSD Building Sept 2005-- A Wide Variety of Applications i Grid 2005
33. First Trans-Pacific Super High Definition Telepresence Meeting in New Calit2 Digital Cinema Auditorium Lays Technical Basis for Global Digital Cinema Sony NTT SGI Keio University President Anzai UCSD Chancellor Fox
34. The OptIPuter Enabled Collaboratory: Remote Researchers Jointly Exploring Complex Data OptIPuter will Connect The Calit2@UCI 200M-Pixel Wall to The Calit2@UCSD 100M-Pixel Display With Shared Fast Deep Storage “ SunScreen” Run by Sun Opteron Cluster UCI UCSD
36. Adding Web & Grid Services to Optical Channels to Provide Real Time Control of Ocean Observatories Goal: Prototype Cyberinfrastructure for NSF’s Ocean Research Interactive Observatory Networks (ORION) LOOKING NSF ITR with PIs: John Orcutt & Larry Smarr - UCSD John Delaney & Ed Lazowska –UW Mark Abbott – OSU Collaborators at: MBARI, WHOI, NCSA, UIC, CalPoly, UVic, CANARIE, Microsoft, NEPTUNE-Canarie LOOKING: ( L aboratory for the O cean O bservatory K nowledge In tegration G rid) LOOKING is Driven By NEPTUNE CI Requirements http://lookingtosea.ucsd.edu/ Making Management of Gigabit Flows Routine
37. First Remote Interactive High Definition Video Exploration of Deep Sea Vents Source John Delaney & Deborah Kelley, UWash Canadian-U.S. Collaboration
40. The Sargasso Sea Experiment The Power of Environmental Metagenomics Yielded a Total of Over 1 billion Base Pairs of Non-Redundant Sequence Displayed the Gene Content, Diversity, & Relative Abundance of the Organisms Sequences from at Least 1800 Genomic Species, including 148 Previously Unknown Identified over 1.2 Million Unknown Genes MODIS-Aqua satellite image of ocean chlorophyll in the Sargasso Sea grid about the BATS site from 22 February 2003 J. Craig Venter, et al. Science 2 April 2004: Vol. 304. pp. 66 - 74
41. Evolution is the Principle of Biological Systems: Most of Evolutionary Time Was in the Microbial World Source: Carl Woese, et al You Are Here Much of Genome Work Has Occurred in Animals
42. Calit2 Intends to Jump Beyond Traditional Web-Accessible Databases Data Backend (DB, Files) W E B PORTAL (pre-filtered, queries metadata) Response Request + many others Source: Phil Papadopoulos, SDSC, Calit2 BIRN PDB NCBI Genbank
43. Data Servers Must Become Lambda Connected to Allow for Directly Optical Connection to End User Clusters Traditional User Response Request Source: Phil Papadopoulos, SDSC, Calit2 + Web Services Flat File Server Farm W E B PORTAL Dedicated Compute Farm (1000 CPUs) TeraGrid: Cyberinfrastructure Backplane (scheduled activities, e.g. all by all comparison) (10000s of CPUs) Web (other service) Local Cluster Local Environment Direct Access Lambda Cnxns OptIPuter Cluster Cloud Data- Base Farm (0.3PB) 10 GigE Fabric
45. Calit2/SDSC Proposal to Create a UC Cyberinfrastructure of OptIPuter “On-Ramps” to TeraGrid Resources UC San Francisco UC San Diego UC Riverside UC Irvine UC Davis UC Berkeley UC Santa Cruz UC Santa Barbara UC Los Angeles UC Merced OptIPuter + CalREN-XD + TeraGrid = “OptiGrid” Source: Fran Berman, SDSC , Larry Smarr, Calit2 Creating a Critical Mass of End Users on a Secure LambdaGrid