Near Exascale Computing in the Cloud

The document discusses challenges in achieving exascale computing capabilities by 2018. It outlines how standard technology scaling will not be enough, and compromises will need to be made. These include reduced node performance, lower network bandwidth and fewer pins. Blue Gene architecture is presented as an example of a balanced system that achieves high performance through optimized interconnects and packaging density. A thought experiment proposes integrating significant solid state storage at each node to create an "active storage" machine based on Blue Gene architecture.

The document describes NCAR's wind forecasting system for Xcel Energy. Key points: - NCAR operates a probabilistic wind power prediction system for Xcel Energy using ensemble forecasts from its WRF model and an Analog Ensemble technique. - The system provides day-ahead and short-term wind power forecasts for effective grid integration and energy trading. - Accurate wind power predictions provide economic benefits of $1.9 million per 1% improvement and reduced 238,136 tons of CO2 emissions in 2011 due to avoided generation.

Cloudgene is an open-source platform that provides a graphical web interface to simplify the execution of MapReduce programs for genomic data analysis in public and private clouds. It allows users to integrate different MapReduce programs through a plugin interface, import and export data from various sources, and connect programs together in a pipeline. Cloudgene handles setting up clusters in public clouds and installing programs and data, making it easier for scientists to perform MapReduce analysis without having to manage the underlying infrastructure.

The document discusses challenges faced with application specific supercomputer design. It provides an example of QPACE, a supercomputer designed for quantum chromodynamics (QCD) computations. Key challenges discussed include data ordering issues when using InfiniBand networking that could cause computations to use invalid data if ordering of writes to memory was not enforced. Ensuring proper data ordering is important to avoid software consuming data before it is valid.

In this slidecast, Jason Stowe from Cycle Computing describes the company's recent record-breaking Petascale CycleCloud HPC production run. "For this big workload, a 156,314-core CycleCloud behemoth spanning 8 AWS regions, totaling 1.21 petaFLOPS (RPeak, not RMax) of aggregate compute power, to simulate 205,000 materials, crunched 264 compute years in only 18 hours. Thanks to Cycle's software and Amazon's Spot Instances, a supercomputing environment worth $68M if you had bought it, ran 2.3 Million hours of material science, approximately 264 compute-years, of simulation in only 18 hours, cost only $33,000, or $0.16 per molecule." Learn more: http://blog.cyclecomputing.com/2013/11/back-to-the-future-121-petaflopsrpeak-156000-core-cyclecloud-hpc-runs-264-years-of-materials-science.html Watch the video presentation: http://wp.me/p3RLHQ-aO9

This document summarizes the detection of radio emissions from fireballs (very bright meteors) using the Long Wavelength Array radio telescope. A search of over 11,000 hours of all-sky radio images found 49 long-duration radio transients. Ten of these transients correlated spatially and temporally with fireballs detected by an optical meteor monitoring network. This provides evidence that fireballs emit previously undiscovered low frequency radio pulses. Further analysis found characteristics inconsistent with expected radio reflections from meteor trails, suggesting a non-thermal radio emission mechanism from the fireballs. This identifies a new class of natural radio transients and provides a new probe to study meteor physics.

In this video from GTC 2018, Peter Dueben from ECMWF presents: How to Prepare Weather and Climate Models for Future HPC Hardware. "Learn how one of the leading institutes for global weather predictions, the European Centre for Medium-Range Weather Forecasts (ECMWF), is preparing for exascale supercomputing and the efficient use of future HPC computing hardware. I will name the main reasons why it is difficult to design efficient weather and climate models and provide an overview on the ongoing community effort to achieve the best possible model performance on existing and future HPC architectures. I will present the EU H2020 projects ESCAPE and ESiWACE and discuss recent approaches to increase computing performance in weather and climate modeling such as the use of reduced numerical precision and deep learning." Watch the video: https://wp.me/p3RLHQ-ixu Learn more: https://www.ecmwf.int/ and https://www.nvidia.com/en-us/gtc/ Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

This document summarizes a low-cost and scalable system for visualizing electricity consumption. The system uses inexpensive current sensors connected to data collection units that transmit power usage data to a Google App Engine cloud server. This allows visualization apps to retrieve the data and monitor consumption across many sensors over time. The system was demonstrated at the SC11 conference with data from 82 sensors across Japan and the US being visualized.

05.07.29 Briefing to the CalREN-XD Subcommittee CENIC Board Title: The OptIPuter as a Prototype for CalREN-XD San Diego, CA

The document describes a Kinect pipeline that uses hardware acceleration to perform background removal and body part classification directly on the FPGA. It explores using a random forest algorithm called Forest Fire, which has an efficient FPGA implementation, to classify pixels into player/background and body part classes. Two approaches are evaluated: a one-stage Forest Fire model, and a two-stage approach where the first stage classifies player/background and the second stage further classifies body parts. The hardware implementations achieve faster speeds and lower power compared to a CPU baseline.

1) BurstCube is a CubeSat mission that will use four CsI scintillators read out by silicon photomultipliers (SiPMs) to detect gamma rays between 10 keV and 1 MeV and observe astrophysical counterparts to gravitational waves. 2) The study characterized SiPMs to qualify them for use in space missions like BurstCube by measuring their sensitivity, dark current, and pulse height in a dark box setup with an LED pulser. 3) Preliminary characterization of the SiPM's dark current and response to LED pulses was performed and the data was analyzed to understand the device's performance and background noise levels. Further optical tests and integration with scintillators is planned.

This document discusses using various sensors in mobile devices for creative purposes beyond their typical uses. It describes how sensors like accelerometers, gyroscopes, ambient light sensors, and magnetometers can be accessed through JavaScript to build applications like a Theremin music instrument controlled by ambient light levels or tracking real-world device movement in 3D. It also covers experimental uses of beacons and connecting devices to visualize juggling. The document encourages developers to think creatively about how sensor data could enable new types of applications on mobile platforms.

This document discusses clouds at CERN. It provides background on CERN, including that it was founded in 1954 by 12 European states for "Science for Peace" and now has 20 member states. It notes CERN has around 2300 staff, 1000 other paid personnel, and over 11,000 users. The document discusses challenges in scaling IT infrastructure with fixed staff and budgets. It outlines CERN's approach of moving to cloud models using open source tools. The status provides details on OpenStack deployments at CERN and experiments. It outlines next steps such as moving to new OpenStack releases and using cells to scale capacity.

This document provides a tutorial about a seismic sensor network. It discusses: 1) The special demands of seismic and acoustic applications including large-scale deployment, challenged networks, and remote monitoring requirements. 2) An overview of the software and hardware used in the network including the CDCCs, Q330 data loggers, Duiker data collection software, and DTS remote management software. 3) How to assemble a seismic node in 30 minutes by connecting sensors, data loggers, and wireless nodes together and reprogramming the nodes.

This document provides an overview of 3D Elevation Program (3DEP) lidar data. It defines key lidar terminology and describes the various products that can be derived from lidar point clouds, including digital surface models (DSMs), digital terrain models (DTMs), intensity images, and canopy height models. It explains the quality levels and availability of 3DEP lidar data across the United States. The document also briefly discusses the latest lidar technologies, such as multispectral, single-photon, and Geiger-mode lidar.

Talk at the EarthCube End-User Domain Workshop for Rock Deformation and Mineral Physics Research. By Mark Rivers, University of Chicago

This document provides an overview of distributed data management for the Large Hadron Collider (LHC) experiments at CERN. It discusses the worldwide computing grid that is used to store, process, and analyze the immense volumes of data produced by the LHC experiments each year. The grid consists of Tier 0, 1, and 2 computing centers around the world. It has enabled scientists from many collaborating institutions to work together on data from the LHC experiments.

10.02.22 Invited talk Symposium #1610, How Computational Science Is Tackling the Grand Challenges Facing Science and Society Title: Science and Cyberinfrastructure in the Data-Dominated Era San Diego, CA

The document summarizes Dr. Larry Smarr's presentation on the Pacific Research Platform (PRP) and its role in working toward a national research platform. It describes how PRP has connected research teams and devices across multiple UC campuses for over 15 years. It also details PRP's innovations like Flash I/O Network Appliances (FIONAs) and use of Kubernetes to manage distributed resources. Finally, it outlines opportunities to further integrate PRP with the Open Science Grid and expand the platform internationally through partnerships.

11.12.12 Seminar Presentation Princeton Institute for Computational Science and Engineering (PICSciE) Princeton University Title: A Campus-Scale High Performance Cyberinfrastructure is Required for Data-Intensive Research Princeton, NJ

In this deck from DataTech19, Debbie Bard from NERSC presents: Supercomputing and the scientist: How HPC and large-scale data analytics are transforming experimental science. "Debbie Bard leads the Data Science Engagement Group NERSC. NERSC is the mission supercomputing center for the USA Department of Energy, and supports over 7000 scientists and 700 projects with supercomputing needs. A native of the UK, her career spans research in particle physics, cosmology and computing on both sides of the Atlantic. She obtained her PhD at Edinburgh University, and has worked at Imperial College London as well as the Stanford Linear Accelerator Center (SLAC) in the USA, before joining the Data Department at NERSC, where she focuses on data-intensive computing and research, including supercomputing for experimental science and machine learning at scale." Watch the video: https://wp.me/p3RLHQ-kLV Sign up for our insideHPC Newsletter: http://insidehpc.com/newsletter

Talk delivered at Free and Open Source Software for Geo North America 2019 (FOSS4GNA) Large scale solar arrays or farms have been installed globally faster than can be reliably tracked by interested stakeholders. We have built a deep learning model with Sentinel 2 satellite imagery that allows us to create accurate, timely global maps of solar farms.

NVIDIA is working on tackling climate change through the development of digital twins of Earth using AI and high performance computing. They are collaborating with various partners on initiatives like Destination Earth, which envisions an interactive digital twin platform for modeling and simulation. NVIDIA technologies like Omniverse, AI, and upcoming CPUs like Grace could help make a fully realized digital twin a reality. This would allow researchers to better understand climate systems and explore different scenarios to help mitigate and adapt to climate change.

Drone-based monitoring and 3D modeling of the National Arboretum in Canberra is allowing for detailed phenotyping of tree growth over time. Drones equipped with RGB and multispectral cameras capture aerial images that are processed using software like Pix4D to generate orthomosaic images, digital elevation models, 3D point clouds, and tree metrics like height and area. The data is helping researchers monitor changes in the young research forest over several years. Advanced visualization tools are being developed to better explore the large, complex datasets.

05.02.23 Invited Access Grid Talk MSCMC FORUM Series Examining the National Vision for Global Peace and Prosperity Title: The Academic and R&D Sectors' Current and Future Broadband and Fiber Access Needs for US Global Competitiveness Arlington, VA

The document discusses the need for a new generation of cyberinfrastructure to support interactive global earth observation. It outlines several prototyping projects that are building examples of systems enabling real-time control of remote instruments, remote data access and analysis. These projects are driving the development of an emerging cyber-architecture using web and grid services to link distributed data repositories and simulations.

05.06.14 Keynote to the 15th Federation of Earth Science Information Partners Assembly Meeting: Linking Data and Information to Decision Makers Title: The Jump to Light Speed - Data Intensive Earth Sciences are Leading the Way to the International LambdaGrid San Diego, CA