OpenStack is an open source cloud computing platform that can manage large networks of virtual machines and physical servers. It uses a distributed architecture with components like Nova (compute), Swift (object storage), Cinder (block storage), and Quantum (networking). OpenStack has been successful due to its scalability, support for multiple hypervisors including Hyper-V, and compatibility with popular programming languages like Python. While OpenStack is best suited for large public and private clouds, its complex installation and lack of unified deployment tools can present challenges, especially for small to mid-sized clouds.
Nowadays there is significant diversity in Infrastructure
as a Service (IaaS) clouds. The differences span from
virtualization technology and hypervisors, through storage
and network configuration, to the cloud management
APIs. These differences make migration of a VM (or
a set of VMs) from a private cloud into a public cloud,
or between different public clouds, complicated or even
impractical for many use-cases.
HVX is a virtualization platform that enables complete
abstraction of underlying cloud infrastructure from the
application virtual machines. HVX allows deployment
of existing VMs into the cloud without any modifications,
mobility between the clouds and easy duplication
of the entire deployment.
HVX can be deployed on almost any existing IaaS
cloud. Each instance of the HVX deployment packs in
a nested hypervisor, virtual hardware, network and storage
configuration.
Combined with image store and management APIs,
the HVX can be used for the creation of a virtual cloud
that utilizes existing cloud provider infrastructure as the
hardware rather than using physical servers, switches and
storage.
An Evaluation of OpenStack Deployment Frameworksshane_gibson
Symantec evaluated several OpenStack deployment frameworks to test provisioning OpenStack clusters from bare metal. They tested Fuel Web, MaaS/JuJu, Crowbar, Foreman, and Rackspace Private Cloud. Crowbar had the fastest time to deploy a full OpenStack cluster and met most of Symantec's requirements. The evaluation provided feedback to vendors on improving automation, resiliency, and managing complex configurations when deploying OpenStack at scale.
Puppet and Nano Server provide an amazing mix when it comes to automated cloud deployments. This slide deck is from my session at PuppetCamp NYC and Boston.
This document discusses integrating OpenStack with Hyper-V. It outlines goals of establishing an OpenStack/Hyper-V development community and reintegrating Hyper-V code for the Folsom release. It describes successes in Folsom including restored and improved Hyper-V functionality. It provides details on using Hyper-V Server 2012 with OpenStack and key contributions to the Folsom integration. Finally, it discusses plans for the Grizzly release and resources for learning more.
Presentation of OpenStack survey to Internet Research Lab at National Taiwan University, Taiwan. OpenStack framework and architecture overview. (ppt slide for download.) Materials collected from various resources, not originally produced by the author.
Briefly explained Nova, Swift, Glance, Keystone, and Quantum.
The document provides an agenda and overview of a session on hacking Apache CloudStack. The agenda includes introductions, a session on introducing CloudStack, and a hands-on session with DevCloud. The overview discusses what CloudStack is, how it works as an orchestration platform for IAAS clouds, its architecture and core components, and how users can consume and manage resources through it.
CAPS: What's best for deploying and managing OpenStack? Chef vs. Ansible vs. ...Daniel Krook
Presentation at the OpenStack Summit in Tokyo, Japan on October 29, 2015.
http://sched.co/49vI
This talk will cover the pros and cons of four different OpenStack deployment mechanisms. Puppet, Chef, Ansible, and Salt for OpenStack all claim to make it much easier to configure and maintain hundreds of OpenStack deployment resources. With the advent of large-scale, highly available OpenStack deployments spread across multiple global regions, the choice of which deployment methodology to use has become more and more relevant.
Beyond the initial day-one deployment, when it comes to the day-two and beyond questions of updating and upgrading existing OpenStack deployments, it becomes all the more important choose the right tool.
Come join the Bluebox and IBM team to discuss the pros and cons of these approaches. We look at each of these four tools in depth, explore their design and function, and determine which scores higher than others to address your particular deployment needs.
Daniel Krook - Senior Software Engineer, Cloud and Open Source Technologies, IBM
Paul Czarkowski - Cloud Engineer at Blue Box, an IBM company
Daniel Krook - Senior Software Engineer, Cloud and Open Source Technologies, IBM
This document compares the CloudStack and OpenStack platforms in terms of architecture, installation, administration, security, high availability, and other factors. CloudStack has a more monolithic controller architecture compared to OpenStack's fragmented architecture. CloudStack is easier to install with fewer parts and repositories, while OpenStack installation is more difficult with many choices and configuration required. Both provide self-service user interfaces, quotas, and networking features like floating IPs and VLAN support. The document concludes that CloudStack is more mature and refined for enterprise adoption due to its simpler installation, user-friendly GUI, and documentation, while OpenStack requires more development customization but has deeper community support.
CloudStack vs OpenStack vs Eucalyptus: IaaS Private Cloud Brief Comparisonbizalgo
This document compares the architectures, installation processes, administration tools, security features, and high availability capabilities of CloudStack, Eucalyptus, and OpenStack. CloudStack has a monolithic controller architecture and the easiest installation process. Eucalyptus closely mimics AWS but has a more difficult multi-component installation. OpenStack is the most fragmented with many interdependent pieces and a challenging installation. All three provide basic security through VLANs and firewalls, with Eucalyptus and OpenStack adding additional authentication. High availability varies by platform, with CloudStack using a load-balanced controller, Eucalyptus relying on component failover, and OpenStack's Swift storage using replication across its ring topology.
CloudStack and OpenStack both provide platform for managing and deploying virtual infrastructure. CloudStack UI is easier to use and more user friendly, while OpenStack UI is simpler but based on Django framework. CloudStack uses monolithic controller architecture with datacenter model, while OpenStack is more fragmented with shared nothing architecture. CloudStack networking supports basic, advanced, flat and VLAN modes. OpenStack uses security groups and supports flat, DHCP and VLAN modes. CloudStack storage is primary and secondary, while OpenStack uses Cinder for block storage and Swift for object storage. CloudStack deployment is easier while OpenStack typically requires tools like Puppet or Chef.
DevCloud - Setup and Demo on Apache CloudStack buildacloud
Hands-on Hacking Session by Amogh Vasekar
1. Demo of CloudStack using DevCloud
2. How we got there -
A) Building CloudStack from scratch
B) Deploying databases
C) Configuring your own DevCloud using Marvin
dodai is a Cluster as a Service (CaaS) tool comprised of dodai-deploy and dodai-compute. dodai-deploy is a software management tool for distributed environments that provides templates for deploying software like OpenStack, Hadoop, and SGE. It is fast, scalable, and easy to use. dodai-compute is a bare metal version of Nova that provides the same interface for operating physical machines and separates networks logically with OpenFlow switches. The presentation demonstrated installing an OpenStack Folsom cloud on EC2 using dodai's new "Install as a Service" functionality.
Openstack is an open source cloud computing platform that consists of several independent components that work together to provide infrastructure as a service capabilities. It allows users to provision compute, storage, and networking resources on demand in a self-service manner similar to public cloud providers like AWS. Some key components include Nova for compute, Glance for images, Swift for object storage, Cinder for block storage, Neutron for networking, and Keystone for identity services. Openstack can be used to build public, private, or hybrid clouds and supports a variety of use cases and workloads.
CloudStack is an open-source cloud computing platform that provides infrastructure as a service. It allows management of multiple hypervisors from a single interface and scales to support thousands of servers across locations. Performance testing showed CloudStack can deploy over 10,000 VMs with reasonable response times even under heavy load. While competitors like OpenStack and Eucalyptus are also free, CloudStack offers easier installation and supports more hypervisors out of the box. Case studies found companies like ASG have successfully leveraged CloudStack for custom private cloud solutions.
This document provides an overview of OpenStack, including what it is, the main OpenStack services, and how to perform single node and multinode installations using DevStack and PackStack. OpenStack is an open-source cloud computing platform that provides infrastructure as a service. It consists of interrelated components to control hardware resources like processing, storage, and networking. The document describes the main OpenStack services like Dashboard, Compute, Networking, Object Storage, Block Storage, Image Service, Telemetry, and Orchestration. It then covers how to do a single node installation on Ubuntu, the networking requirements for multinode, and how to install and configure DevStack and PackStack for OpenStack deployment.
This talk covered the OpenStack basics that VMware Administrators need to be aware of to be successful in their deployments. We also had the Tesora team join us on stage to discuss the importance of Database-as-a-Service with the Trove project!
OpenStack Summit Vancouver: Lessons learned on upgradesFrédéric Lepied
Deploying OpenStack in production at any scale, upgrade support is one of the requirements to have a successful deployment. Without upgrade management, adeployment will have bugs and security issues from day 1. Also in longer term, it will miss the latest features that OpenStack offers.
OpenNebula TechDay Boston 2015 - installing and basic usageOpenNebula Project
This document provides an overview of installing and using OpenNebula. It discusses planning an OpenNebula environment including repositories, backends, and physical resources. It then covers installing OpenNebula on two nodes, configuring passwordless SSH, and starting services. The document demonstrates adding hosts, images, networks, templates, and instantiating VMs through the CLI and Sunstone interface. It also covers groups, permissions, contextualization, and the different views in OpenNebula including the administrator, VDC administrator, and cloud user perspectives.
This document provides an overview of OpenStack, an open-source cloud computing platform. It discusses OpenStack's components, development process using Gerrit, and integration testing. It also covers using Windows as a guest operating system in OpenStack with Hyper-V, including Windows Cloud-Init tools and supported Windows versions. Key OpenStack components that support Windows like Nova, Neutron, Cinder, and Manila are summarized.
OpenStack is an open-source cloud computing platform that was launched in 2010. It provides services for managing compute, storage, and networking resources in a data center. The core OpenStack services are Nova (compute), Swift (object storage), and Glance (image service). OpenStack uses a RESTful API and supports many hypervisors, operating systems, and storage backends. It allows for public and private cloud deployments and supports common industry standards.
This document provides an overview of Apache CloudStack, an open source cloud computing platform. It describes CloudStack's key characteristics including on-demand self-service, broad network access, resource pooling, rapid elasticity, and API access. It outlines CloudStack's support for different cloud service models including SaaS, PaaS, and IaaS and discusses its hypervisor support, zone, pod, and cluster architecture. The document also summarizes CloudStack's management server, high availability features, networking, security groups, and usage accounting capabilities.
The document provides a technical overview of the CLIMB OpenStack cloud including hardware, software, and configuration details. The key components are IBM servers and storage, xCAT for provisioning, SaltStack for configuration management, OpenStack for cloud services, and IBM Spectrum Scale (formerly GPFS) for parallel file storage. Spectrum Scale is integrated with OpenStack components like Cinder, Glance, and Swift to provide scalable block and object storage.
OpenStack Day Seattle 2015, Enterprise Track
Title: Windows Containers, Hyper-V and OpenStack
Speakers: Ben Armstrong, Microsoft & Alessandro Pilotti, Cloubase Solutions
XenServer Virtualization In Cloud EnvironmentsTim Mackey
= As presented at the CloudStack Silicon Valley Meetup in September 2015. =
XenServer is a virtualization platform which has been deployed in a variety of industries and to support a multitude of workloads. In this session we discuss some of the components which make it valuable not just for traditional server and desktop virtualization, but also within "the cloud". This includes discussion of VM density, network scalability, containers (such as Docker) and GPU virtualization. We end with coverage of how XenServer templates are represented within Apache CloudStack.
The Future of SDN in CloudStack by Chiradeep Vittalbuildacloud
The core of CloudStack networking has always been software-defined. As the networking industry evolves to a software-defined future, CloudStack will have to evolve with it.
The presentation will examine the present state of SDN in CloudStack, look at some industry directions and attempt to predict the evolution of CloudStack with those trends.
Bio
Chiradeep Vittal is a Distinguished Engineer in the Converged Infrastructure Group at Citrix where he has technology leadership responsibilities around Citrix Cloud Platform, Citrix Lifecycle Manager and Citrix Workspace Pod. He is also a Project Management Committee member of the Apache CloudStack Project. At cloud.com (acquired by Citrix), he was a founding engineer, often tasked with the thorny details of virtualized networking and storage. Prior to cloud.com, he worked at several Silicon Valley startups in various architectural roles.
Chiradeep has a B.Tech in Computer Science from IIT, Bombay and a M.Sc from the University of Alberta. He has spoken / presented at several conferences, including CloudStack Collab, LISA, OSCON, ONS, SDN Summit and LinuxCon. His twitter handle is @chiradeep and occasionally blogs at http://cloudierthanthou.wordpress.com
Using apache camel for microservices and integration then deploying and managing on Docker and Kubernetes. When we need to make changes to our app, we can use Fabric8 continuous delivery built on top of Kubernetes and OpenShift.
Windows Server 2016 introduces several new features including software defined storage, Hyper-V improvements, automation enhancements, and Nano Server. Storage Spaces Direct enables hyper-converged infrastructure, Storage Replica allows for volume replication, and Resilient File System provides improved integrity and performance. Hyper-V gains features like cluster rolling upgrades and shielded VMs. Automation is focused on a graphical authoring experience and Linux/Azure support. Nano Server offers a headless deployment optimized for cloud workloads.
My talk at ScaleConf 2017 in Cape Town on some tips and tactics for scaling WordPress, with reference to WordPress.com and the container-based VIP Go platform.
Video of my talk is here: https://www.youtube.com/watch?v=cs0DcY80spw
Why Kubernetes as a container orchestrator is a right choice for running spar...DataWorks Summit
Building and deploying an analytic service on Cloud is a challenge. A bigger challenge is to maintain the service. In a world where users are gravitating towards a model where cluster instances are to be provisioned on the fly, in order for these to be used for analytics or other purposes, and then to have these cluster instances shut down when the jobs get done, the relevance of containers and container orchestration is more important than ever.
Container orchestrators like Kubernetes can be used to deploy and distribute modules quickly, easily, and reliably. The intent of this talk is to share the experience of building such a service and deploying it on a Kubernetes cluster. In this talk, we will discuss all the requirements which an enterprise grade Hadoop/Spark cluster running on containers bring in for a container orchestrator.
This talk will cover in details how Kubernetes orchestrator can be used to meet all our needs of resource management, scheduling, networking, and network isolation, volume management, etc. We will discuss how we have replaced our home grown container orchestrator with Kubernetes which used to manage the container lifecycle and manage resources in accordance to our requirements. We will also discuss the feature list as container orchestrator which is helping us deploy and patch 1000s of containers and also a list which we believe need improvement or can be enhanced in a container orchestrator.
Speaker
Rachit Arora, SSE, IBM
This document provides an overview and lessons on installing and configuring the Hyper-V role in Microsoft Virtual Academy. It discusses installing the Hyper-V server role, managing Hyper-V, configuring Hyper-V settings, and Hyper-V host storage and networking. The lessons cover topics like Hyper-V architecture, storage and networking considerations, using Hyper-V Manager and PowerShell, resource pools, Storage Spaces, SMB 3.0, and network teaming.
CloudStack is an open-source cloud computing platform that provides infrastructure as a service. It allows users to provision resources such as virtual servers and networking on demand through a self-service web portal. CloudStack can manage tens of thousands of servers across multiple geographically distributed datacenters and supports multiple hypervisors including XenServer, KVM, and vSphere. It provides high availability, scalability, and automation of infrastructure management.
- ASP.NET 5 is the next generation of Microsoft's web framework that aims to address limitations of the current ASP.NET stack such as limited hosting possibilities and dependency on IIS.
- It features a modular architecture based on OWIN and Katana that decouples the application from the server and allows hosting on non-IIS platforms like Linux.
- Key improvements include cross-platform support, a more developer-friendly experience with features like no-compile debugging, and an emphasis on performance and light weight deployment through tools like the K command line.
ONUG Tutorial: Bridges and Tunnels Drive Through OpenStack Networkingmarkmcclain
This document summarizes OpenStack networking (Neutron) and discusses its key components and architecture. It describes how Neutron provides network abstraction and virtualization through pluggable backend drivers. It also outlines some common Neutron features like security groups and highlights new capabilities in the Juno release like IPv6 support and distributed virtual routing. The document concludes by looking ahead to further networking developments in OpenStack.
Docker is a tool that allows users to package applications into containers to run on Linux servers. Containers provide isolation and resource sharing benefits compared to virtual machines. Docker simplifies deployment of containers by adding images, repositories and version control. Popular components include Dockerfiles to build images, Docker Hub for sharing images, and Docker Compose for defining multi-container apps. Docker has gained widespread adoption due to reducing complexity of managing containers across development and operations teams.
Presented at the CloudStack Silicon Valley User Group in September 2015 at Nuage Networks. Discussed impact of containers, emerging software defined networking platforms, NFV, IPv6 and performance.
This document provides an overview of OpenStack, an open-source cloud computing platform. It discusses the evolution of infrastructure from traditional to virtualized and cloud-based systems. It then describes OpenStack's architecture and core components for compute, storage, networking, identity management, and more. The document also outlines how OpenStack can be used to deploy private clouds and manage virtual infrastructure and applications. It discusses different administration roles for managing applications on the cloud versus managing the cloud platform itself.
Do you think that Nova, Cinder, Heat, Ceilometer, and Neutron are all references to global warming and looming apocalypse? For all those who come to the OpenStack community and wonder what all the fuss is about, this quick introduction will answer your many questions. It includes a short history of the largest Open Source project in history and will touch on
the basic OpenStack components, so you will be prepared the next time someone mentions Keystone, Nova and Swift in the same sentence.
This session was presented by Beth Cohen at the OpenStack meetup on Feb 19th, 2014 in Boston. Beth works for Verizon developing cool Cloud based products that she can't talk about without a strict NDA. She is a technical leader with over 25 years of experience architecting leading-edge system infrastructures and managing complex projects in the telecom, manufacturing, financial services, government, and technology industries. She has been involved in building some of the world's largest OpenStack architectures and has way too much fun at OpenStack Summits!
Rami Sayar - Node microservices with DockerWeb à Québec
The document discusses converting a monolithic Node.js application into microservices and deploying them using Docker. It begins by defining microservices and their benefits. It then describes converting a sample pizza ordering application into independent microservices for handling messages, serving the frontend, and providing an API. Next, it covers patterns for networking microservices, including using an API gateway. It concludes by demonstrating how to deploy the microservices to Docker containers and use an orchestration tool like Kubernetes to manage them.
Strategies for migrating workloads from VMware to OpenStackAlessandro Pilotti
As OpenStack adoption increases, companies are looking at ways to migrate workloads from traditional virtualization solutions like VMware or System Center to cloud infrastructures based on OpenStack.
There are multiple angles to this problem to consider:
Pets vs Cattle: when is OpenStack the right choice and when not
Lift and shift: when is worth to use automated migration tools like Coriolis to move virtual machines
App migration: when to migrate individual applications and their data to a new platform (e.g. a PaaS)
Re-develop: when to adopt modern paradigms like microservices, containers etc and redevelop applications
There's no one-size-fits-all answer, but knowing what your options are, you can do the right choice for each use case. Costs also have a big impact, for example it can be worth to lift-and-shift and use the saved money to re-architect the next app generation.
We will also do a fully automated lift-and-shift and DRaaS live demo using the Coriolis project.
Hyper-C is OpenStack on Windows Server 2016, based on Nano Server, Hyper-V, Storage Spaces Direct (S2D) and Open vSwitch for Windows. Bare metal deployment features Cloudbase Solutions Juju charms and MAAS.
The Microsoft cloud ecosystem evolved considerably in recent years to interoperate with a wide range of open source technologies, including hardware (Open Compute), cloud software platforms (OpenStack), networking (Open vSwitch, OpenDaylight) and orchestration (Juju, Heat).
During this session we will show how to deploy in no time an entire OpenStack cloud based on Microsoft Hyper-V using MaaS and Juju. Networking is going to be based on Open vSwitch, which brings OVSDB and VXLAN to Hyper-V, allowing full interoperability with KVM and other hypervisors.
To conclude, we are going to orchestrate with Juju on top of our OpenStack cloud some of the most common Microsoft workloads, including Active Directory, IIS, SQL Server, SharePoint and Exchange, side by side with open source applications.
Here's an FreeRDP based HTML5 client to connect to the Hyper-V console from any client.
We are developing this feature mainly for integration with the OpenStack dashboard, but it can be used in any scenario requiring remote Hyper-V guest management.
A demo webcast is available here: http://bit.ly/VgwMvs
This webcast will show you how to properly configure and deploy Memcached and Solr on Windows, including all the required Drupal integration. The webcast includes also instructions on proper configuration of your Drupal cron tasks for Solr indexing in conjunction with Windows Task Scheduler.
PHP is a first class citizen on IIS. A lot has been done in order to make sure that PHP can work well and fast on Windows. We will start by installing PHP with PHP Manager and discussing all the options including what thread safe vs non thread safe and VC6 vs VC9 means. Next we'll take a look at how to optimize the FastCGI IIS extension that Microsoft and Zend developed to provide a secure and performant environment for PHP applications. The last part of this webcast will show how to build PHP with Profile Guided Optimizations (PGO), a technique that can provide a significant performance boost in a wide range of applications.
When it comes to Drupal management and deployment, Drush is a must. This webcasts is about installing and configuring Drush on Windows, in order to manage local and remote Drupal sites on Windows and Linux.
This webcast covers the theoretical introduction to Web Farms and how to build Drupal Web Farms with IIS. Don't miss the second part of the webcast (also part of this series) where a full demo on creating Drupal Web Farms with 4 virtual machines will be presented. If you are already familiar with Web Farms, Application Request Router, Web Farm Framework you can skip to part 2. Otherwise, this webcast is highly recommended and propaedeutic to grasp all the basic knowledge that you might need later.
This is a webinar done with Acquia introdcing how well you can run Drupal on Windows with a demo about WebPI and Drush.
Here's a link to the recorded webinar:
http://tinyurl.com/6rfz3px
Building modern web sites with ASP .Net Web API, WebSockets and RSignalAlessandro Pilotti
My session at ITCamp.ro 2012:
Web site development is an ever changing landscape. Thanks to the latest web browser technologies it's possible to create highly responsive single page applications, requiring a new approach to design and development on the server side. During this session we'll see ho to use .Net technologies to get the best out of the new Web API, WebSockets and the excellent RSignal framework.
Scaling Connections in PostgreSQL Postgres Bangalore(PGBLR) Meetup-2 - MydbopsMydbops
This presentation, delivered at the Postgres Bangalore (PGBLR) Meetup-2 on June 29th, 2024, dives deep into connection pooling for PostgreSQL databases. Aakash M, a PostgreSQL Tech Lead at Mydbops, explores the challenges of managing numerous connections and explains how connection pooling optimizes performance and resource utilization.
Key Takeaways:
* Understand why connection pooling is essential for high-traffic applications
* Explore various connection poolers available for PostgreSQL, including pgbouncer
* Learn the configuration options and functionalities of pgbouncer
* Discover best practices for monitoring and troubleshooting connection pooling setups
* Gain insights into real-world use cases and considerations for production environments
This presentation is ideal for:
* Database administrators (DBAs)
* Developers working with PostgreSQL
* DevOps engineers
* Anyone interested in optimizing PostgreSQL performance
Contact info@mydbops.com for PostgreSQL Managed, Consulting and Remote DBA Services
The Rise of Supernetwork Data Intensive ComputingLarry Smarr
Invited Remote Lecture to SC21
The International Conference for High Performance Computing, Networking, Storage, and Analysis
St. Louis, Missouri
November 18, 2021
UiPath Community Day Kraków: Devs4Devs ConferenceUiPathCommunity
We are honored to launch and host this event for our UiPath Polish Community, with the help of our partners - Proservartner!
We certainly hope we have managed to spike your interest in the subjects to be presented and the incredible networking opportunities at hand, too!
Check out our proposed agenda below 👇👇
08:30 ☕ Welcome coffee (30')
09:00 Opening note/ Intro to UiPath Community (10')
Cristina Vidu, Global Manager, Marketing Community @UiPath
Dawid Kot, Digital Transformation Lead @Proservartner
09:10 Cloud migration - Proservartner & DOVISTA case study (30')
Marcin Drozdowski, Automation CoE Manager @DOVISTA
Pawel Kamiński, RPA developer @DOVISTA
Mikolaj Zielinski, UiPath MVP, Senior Solutions Engineer @Proservartner
09:40 From bottlenecks to breakthroughs: Citizen Development in action (25')
Pawel Poplawski, Director, Improvement and Automation @McCormick & Company
Michał Cieślak, Senior Manager, Automation Programs @McCormick & Company
10:05 Next-level bots: API integration in UiPath Studio (30')
Mikolaj Zielinski, UiPath MVP, Senior Solutions Engineer @Proservartner
10:35 ☕ Coffee Break (15')
10:50 Document Understanding with my RPA Companion (45')
Ewa Gruszka, Enterprise Sales Specialist, AI & ML @UiPath
11:35 Power up your Robots: GenAI and GPT in REFramework (45')
Krzysztof Karaszewski, Global RPA Product Manager
12:20 🍕 Lunch Break (1hr)
13:20 From Concept to Quality: UiPath Test Suite for AI-powered Knowledge Bots (30')
Kamil Miśko, UiPath MVP, Senior RPA Developer @Zurich Insurance
13:50 Communications Mining - focus on AI capabilities (30')
Thomasz Wierzbicki, Business Analyst @Office Samurai
14:20 Polish MVP panel: Insights on MVP award achievements and career profiling
Mitigating the Impact of State Management in Cloud Stream Processing SystemsScyllaDB
Stream processing is a crucial component of modern data infrastructure, but constructing an efficient and scalable stream processing system can be challenging. Decoupling compute and storage architecture has emerged as an effective solution to these challenges, but it can introduce high latency issues, especially when dealing with complex continuous queries that necessitate managing extra-large internal states.
In this talk, we focus on addressing the high latency issues associated with S3 storage in stream processing systems that employ a decoupled compute and storage architecture. We delve into the root causes of latency in this context and explore various techniques to minimize the impact of S3 latency on stream processing performance. Our proposed approach is to implement a tiered storage mechanism that leverages a blend of high-performance and low-cost storage tiers to reduce data movement between the compute and storage layers while maintaining efficient processing.
Throughout the talk, we will present experimental results that demonstrate the effectiveness of our approach in mitigating the impact of S3 latency on stream processing. By the end of the talk, attendees will have gained insights into how to optimize their stream processing systems for reduced latency and improved cost-efficiency.
The DealBook is our annual overview of the Ukrainian tech investment industry. This edition comprehensively covers the full year 2023 and the first deals of 2024.
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Chris Swan
Have you noticed the OpenSSF Scorecard badges on the official Dart and Flutter repos? It's Google's way of showing that they care about security. Practices such as pinning dependencies, branch protection, required reviews, continuous integration tests etc. are measured to provide a score and accompanying badge.
You can do the same for your projects, and this presentation will show you how, with an emphasis on the unique challenges that come up when working with Dart and Flutter.
The session will provide a walkthrough of the steps involved in securing a first repository, and then what it takes to repeat that process across an organization with multiple repos. It will also look at the ongoing maintenance involved once scorecards have been implemented, and how aspects of that maintenance can be better automated to minimize toil.
Understanding Insider Security Threats: Types, Examples, Effects, and Mitigat...Bert Blevins
Today’s digitally connected world presents a wide range of security challenges for enterprises. Insider security threats are particularly noteworthy because they have the potential to cause significant harm. Unlike external threats, insider risks originate from within the company, making them more subtle and challenging to identify. This blog aims to provide a comprehensive understanding of insider security threats, including their types, examples, effects, and mitigation techniques.
Sustainability requires ingenuity and stewardship. Did you know Pigging Solutions pigging systems help you achieve your sustainable manufacturing goals AND provide rapid return on investment.
How? Our systems recover over 99% of product in transfer piping. Recovering trapped product from transfer lines that would otherwise become flush-waste, means you can increase batch yields and eliminate flush waste. From raw materials to finished product, if you can pump it, we can pig it.
Best Practices for Effectively Running dbt in Airflow.pdfTatiana Al-Chueyr
As a popular open-source library for analytics engineering, dbt is often used in combination with Airflow. Orchestrating and executing dbt models as DAGs ensures an additional layer of control over tasks, observability, and provides a reliable, scalable environment to run dbt models.
This webinar will cover a step-by-step guide to Cosmos, an open source package from Astronomer that helps you easily run your dbt Core projects as Airflow DAGs and Task Groups, all with just a few lines of code. We’ll walk through:
- Standard ways of running dbt (and when to utilize other methods)
- How Cosmos can be used to run and visualize your dbt projects in Airflow
- Common challenges and how to address them, including performance, dependency conflicts, and more
- How running dbt projects in Airflow helps with cost optimization
Webinar given on 9 July 2024
論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...Toru Tamaki
Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, Volker Tresp, Philip Torr "A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models" arXiv2023
https://arxiv.org/abs/2307.12980
Choose our Linux Web Hosting for a seamless and successful online presencerajancomputerfbd
Our Linux Web Hosting plans offer unbeatable performance, security, and scalability, ensuring your website runs smoothly and efficiently.
Visit- https://onliveserver.com/linux-web-hosting/
Quantum Communications Q&A with Gemini LLM. These are based on Shannon's Noisy channel Theorem and offers how the classical theory applies to the quantum world.
BT & Neo4j: Knowledge Graphs for Critical Enterprise Systems.pptx.pdfNeo4j
Presented at Gartner Data & Analytics, London Maty 2024. BT Group has used the Neo4j Graph Database to enable impressive digital transformation programs over the last 6 years. By re-imagining their operational support systems to adopt self-serve and data lead principles they have substantially reduced the number of applications and complexity of their operations. The result has been a substantial reduction in risk and costs while improving time to value, innovation, and process automation. Join this session to hear their story, the lessons they learned along the way and how their future innovation plans include the exploration of uses of EKG + Generative AI.
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc
Six months into 2024, and it is clear the privacy ecosystem takes no days off!! Regulators continue to implement and enforce new regulations, businesses strive to meet requirements, and technology advances like AI have privacy professionals scratching their heads about managing risk.
What can we learn about the first six months of data privacy trends and events in 2024? How should this inform your privacy program management for the rest of the year?
Join TrustArc, Goodwin, and Snyk privacy experts as they discuss the changes we’ve seen in the first half of 2024 and gain insight into the concrete, actionable steps you can take to up-level your privacy program in the second half of the year.
This webinar will review:
- Key changes to privacy regulations in 2024
- Key themes in privacy and data governance in 2024
- How to maximize your privacy program in the second half of 2024
Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Em...Erasmo Purificato
Slide of the tutorial entitled "Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Emerging Trends" held at UMAP'24: 32nd ACM Conference on User Modeling, Adaptation and Personalization (July 1, 2024 | Cagliari, Italy)
2. OpenStack
An open source IaaS project
• Apache 2 license
Managed by the OpenStack Foundation
• More than 150 companies joined it including:
• AMD, Intel, Canonical, SUSE Linux, Red Hat, Cisco, Dell,
HP, IBM, NEC, Vmware
Portable
• Mostly used on Linux
Written in Python
3. Releases
Very aggressive schedule
• Twice a year
• Essex
• Apr 2012
• Hyper-V removed
• Folsom
• October 2012
• Cloudbase Solutions involvement - Hyper-V re-included
• Grizzly
• Apr 2013
• Havana
• October 2013
4. Reasons for success
• Most cloud providers want a platform to embed and extend
• E.g. Rackspace, HP
• Most cloud engineers want a platform which is easy to mantain and troubleshoot
• Python, due to it’s dynamic nature, fits the role and is widely known by Linux admins and DevOps
• Great scalability
• Great support for multiple hypervisors
• Nowadays hypervisors are “simple” components in the global architecture, not the focal point anymore
• Difference between products is getting narrower
5. Reasons for success
• Great support for virtual networking
• OpenVSwitch is becoming the “standard de facto” in SDN
• Compatibility with EC2 and S3 API
• Excellent project lifecycle
• Code review
• Unit / System / integration tests
6. Drawbacks
• Setup experience
• Lack of unified deployment tools
• Lack of proper documentation
• Some “youth” issues
• The project is anyway maturing really fast
• The UI is not at the level of the competition
• E.g. Cloudstack
7. Target market
• Public clouds
• Big private clouds
• Not really suitable for mid / small clouds
• Flavors limit flexibility
• Failover clusters are not considered in typical Openstack
deployments, due to large scalability requirements
• Complex installation
• Lack of maturity requires troubleshooting experience
• Not suited for poorly scalable applications
8. Pets vs Cattle
• 2 ways of looking at your VMs
• Pets
• You name them
• Each one has a specific role
• If one “dies”, it’s hard to replace
• Poor scalability
• Cattle
• You don’t name them
• If one “dies”, another will take it’s place
• Great scalability
9. OpenStack Architecture
• Distributed Components
• Can be deployed on a single server or multiple servers
• Public RESTful API
• Bindings available for various languages (including .Net)
• Queues (AMQP) for private APIs
• Great scalability
• Relational DBs for storing the configuration
• MySQL, PostgreSQL, SQLite (and SQL server)
• UI
• CLI tools
• Web dashboard
10. Deployment
• How do you deploy 1000+ physical servers?
• Puppet
• We are working on the CI infrastructure with Microsoft
• Chef
• Crowbar
• Uses mainly Chef
• Dell, Suse, etc
• We are currently adding Hyper-V support for 1.6 and 2.0
11. Crowbar
• Big part of Dell’s cloud strategy
• Bare-metal deployment
• vendor independent
• PXE booting
• Sledgehammer image boots and gets configuration
• Provisioning via Crowbar web site
• Barclams
• Configuration
• Chef recipes
13. Hyper-V 2012 availability
• FREE edition
• Full Hypervisor
• Minimum OS support
• Minimum impact on security updates, etc
• Windows Server 2012
• Just enable the related role
• Windows 8
• For workstation / testing / development usage
• Hyper-V 2012 R2 coming with lots of new features!
14. Components
Compute (Nova)
Object Storage (Swift)
Block Storage (Cinder)
Image Service (Glance)
Networking (Quantum)
Dashboard (Horizon)
Identity (Keystone)
Metering (Ceilometer) - new
Orchestration (Heat) - new
17. Component interaction
• AMQP
• RabbitMQ
• Apache Qpid
• API
• RESTful services
• Database
• Any Python SQLAlchemy provider
• Mostly MySQL
• SQL Server required various bug fixes
18. Keystone
• Single point of integration for:
• Authentication
• Catalog
• Policy
• Token
• Plugin based architecture for backends, e.g.:
• SQL
• LDAP, including Active Directory
• PAM
• Provides Restful APIs
19. Swift
Scalable and fault tolerant object storage
• Achieved via replication
Optional cache (e.g. memcache) used to improve
performance
Objects served via RESTful APIs or raw HTTP requests
Lifecycle independent from the other OpenStack projects
21. Glance images
• Created by providing:
• Container format
• Bare, OVF, AKI, ARI, AMI
• Disk Format
• RAW, VHD, VMDK, VDI, ISO, QCow2, AKI, ARI, AMI
• Metadata properties
• E.g.: hypervisor_type
22. Cinder
• Block storage management
• cinder-api
• cinder-volume
• Mainly iSCSI
• Driver based architecture:
• EMC
• Netapp
• Nexenta
• HP Lefthand
• IBM XIV
• NFS
• LVM
• Windows Storage Server 2012
• And more…
23. Nova
Compute
• nova-compute
Scheduler
• nova-scheduler
API (including metadata)
• nova-api
• EC2 API compatibility
• Metadata API for guests
Console
• nova-novncproxy
• nova-xvpvncproxy
• nova-consoleauth
No-DB-Compute (Grizzly)
• nova-conductor
Network (deprecated, replaced by Quantum)
• nova –network
Volume (deprecated, replaced by Cinder)
• nova-volume
24. nova-compute
Includes drivers for most hypervisors:
• Libvirt (KVM, QEMU)
• Xen
• Hyper-V
• ESX / ESXi / vCenter
• Baremetal
• Mixed deployments are common
• e.g. KVM and Hyper-V
Starting with Grizzly, no database access is needed
• Uses AMQP for scalability
25. nova-scheduler
Scheduling instance deployment is a fundamental feature
• Makes sure that instances are distributed in the best possible
way, based on a set of rules called “filters”
• ComputeFilter checks the image compatibility with the host
(e.g. CPU architecture, hypervisor)
• Custom filters can be easily implemented in Python
26. Compute driver actions
• Driver actions include
• Spawn / destroy VMs
• Retrieving VM info
• Retrieving host info
• Change VM status
• start, stop, reboot, pause, suspend, etc
• Snapshot management
• A snapshot is an image created from an existing VM
• Live migration
• Cold migration / resize
• Volumes attach / detach
• Rescue / unrescue
• Networking (deprecated)
27. OpenStack instances
• Virtual machines (instances) are spawned based on the
following data:
• Glance image
• Hypervisors specific template disk (e.g. VHD, QCow2)
• Flavor
• RAM
• Root disk size
• Ephemeral disk size
• vCPUs
• Specific networks can be specified as well
• Creates one vNic per network
28. Hyper-V driver
WMI
• Currently V1 namespace (root/virtualization)
• V2 namespace used for live migration
• Complete migration to V2 for Havana
Grizzly OS support
• Windows Server / Hyper-V 2008 R2, 2012
• Windows 8 supported for development
Havana OS Support
• New features will target 2012 and above only
29. Hyper-V driver disks
• Copy on write (CoW) implemented with differencing
disks
• Reconnected and merged during snapshots
• Reconnected and merged on resize
• Not necessary for VHDX (Havana)
30. Quantum
• Network connectivity as a service
• Plugin based architecture
• Official plugins:
• Open vSwitch
• Cisco UCS/Nexus
• Linux bridge
• Nicira NVP
• Ryu OpenFlow
• NEC OpenFlow
• Big Switch OpenFlow
• Hyper-V
31. Hyper-V plugin
Quantum plugin
• Runs in quantum-server
• Database based configuration
• Networks, subnets, ports
Quantum agent
• Runs on any compute node
• Supports:
• Local (private vSwitch)
• Flat
• VLAN
• NVGRE (Havana)
• Communicates with plugin via AMQP
L3 support:
• Using the Linux L3 and DHCP plugins
32. Hyper-V plugin
Nova compute driver
• Creates the vNics (one per network)
• Does not connect them
Quantum agent
• Creates the vswitch ports
• Sets the VLAN ID and proper vswitch mode
• For NVGRE creates instances of:
• MSFT_NetVirtualizationLookupRecordSettingData
• etc
33. Quantum OVS interop
• Quantum plugin / agent AMQP RPC protocol is compatible
with OpenVSwitch!!
• You can use the OVS plugin with Hyper-V agents (or vice
versa)
• Limited to compatible L2 protocols: Flat / VLAN
• You can use the L3 and DHCP agents with the Hyper-V
plugin
• Using the OVS L2 agent on the networking nodes
• Great for interoperability (e.g. KVM + Hyper-V)
35. Why OpenVSwitch?
• De facto standard for SDN
• Great interoperability
• Hyper-V / KVM / etc
• OpenFlow
• Tunnelling
• VXLAN
• GRE
• Quantum OVS plugin
• No need for a different one
36. OpenVSwitch porting
• Porting Posix calls to Windows API
• Same CLI as on Linux
• Open Source
• Windows kernel drivers
• NDIS Hyper-V extensions
• VXLAN, GRE
• Best performance
• Support for Hyper-V 2012 and above
38. Horizon
• Main UI
• Implemented in Python with Django
• Typically deployed on Apache with mod_wsgi
• Includes console access
• Currently only VNC
• RDP to be added for Havana
40. Dashboard integration
• Hyper-V uses RDP for accessing the console instead of
VNC
• By default it accepts connections on port 2179
• Not the RDP connection on 3389!
• Access to any guest: Windows, Linux, FreeRDP, etc
• The VM id is provided as part of an RDP protocol
additional buffer called PCB (Pre Connection Buffer)
• Authentication is performed against the host, not the
guest!
41. Dashboard integration
Decoupling from VNC
• Create plugin that embed the canvas in the Dashboard
Authentication layer
• Keystone token validation
Logic for retrieving VM id from OpenStack instance ID
• Ask Nova about the instance details, in particular the Hyper-V host running it
• Get cached hypervisor credentials from keyring or similar
• Query the Hyper-V host about the RDP console port (2179 by default)
• Retrieve the VM id from the host given the instance name
• Perform connection via wsgate
Webcast: http://bit.ly/VgwMvs
42. FreeRDP
• FreeRDP is an amazing RDP open source client.
• Apache 2 license
• Multiplatform
• Linux
• Mac OS X
• Windows
• It provides support for the PCB parameter required by
Hyper-V
• We are using it as a standalone client on the free
Hyper-V server and on Linux, Mac OS X
43. FreeRDP - Web Gateway
Client
• HTML5
• Works on any modern browser
• Web sockets for data transfer
• Canvas for painting
• Can be embedded in any web site
Server proxy
• (wsgate) written in C++
• Currently working on Linux
• can work on Windows
45. Metadata access
• Metadata needs to be provided to the guest:
• Metadata API, accessed via 169.254.169.254
• Openstack or EC2 format
• ConfigDriveV2
46. Cloud-Init
Project provided by Canonical
• Currently Linux specific
• GPL
• Windows Cloud-Init (cloudbase-init)
• Written in Python
• Apache 2
• Very easy to install
• Supports any Hypervisor
47. Windows Cloud-Init
• Cloudbase-init (aka Windows Cloud-Init)
• Similar to the Linux cloud-init
• License: Apache 2
• Porting Cloud-Init to Linux was not possible
• It requires some major refactoring
• Supported data sources:
• OpenStack HTTP metadata
• ConfigDrive V2
• EC2 HTTP metadata (contributed by Mirantis)
50. Password generation
• New Grizzly feature
1. Password is randomly generated
• Can be forced to the value injected by Nova
2. Password in encrypted with the SSH public key
3. Password is POSTed to the metadata service
4. Password can be retrieved by the client and decrypted
with the SSH private key
• nova get-password vm1 .ssh/id_rsa
52. Unattended mode
• Like all our installers, it can be executed in fully
unattended mode:
• msiexec /i CloudbaseInitSetup.msi /qn /l*v log.txt
NETWORKADAPTERNAME="Intel(R) PRO/1000 MT
Network Connection"
55. OpenStack WS2012
• Complete with:
• Drivers / tools:
• VirtIO / XenServer Tools
• Cloudbase-Init
• Sysprepped
• Eval edition can be upgraded with a simple uder_data script:
• DISM /online /Set-Edition:ServerStandard /ProductKey:XXXXX-
XXXXX-XXXXX-XXXXX-XXXXX /AcceptEula
• Make sure to respect the eval license!!
56. Nova Compute Installer
• Independent Python environment to avoid conflicts with
existing applications
• Installs and registers all the required dependencies
• Generates dynamically a nova.conf file based on the
parameters provided by the user
57. Nova Compute Installer
• Creates a new Hyper-V external switch if required
• Registers nova-compute as a service and starts it
• Registers quantum-hyperv-agent as a service and
starts it (optional)
• Enables the Microsoft iSCSI Initiator service (optional)
58. Nova Compute Installer
• Enables and configures Hyper-V Live Migration
(optional )
• FreeRDP for Hyper-V console access
• Installs a command prompt shortcut in the applications
menu for a ready made OpenStack prompt (optional)
• Can be executed fully unattended and automated
59. Unattended mode
• Can be installed in fully unattended mode, great for:
• Chef, Puppet, Group Policies…
• msiexec /i HyperVNovaCompute.msi /qn /l*v log.txt
ADDLOCAL=HyperVNovaCompute,QuantumHyperVAgent,iSCSISWInitiator
,OpenStackCmdPrompt INSTALLDIR=C:OpenStackNova
GLANCEHOST=glancehost GLANCEPORT=9292 RABBITHOST=rabbithost
RABBITPORT=5672 RABBITPASSWORD=12345678
NOVASQLCONNECTION=mysql://user:password@host/nova
INSTANCESPATH=C:HyperV ADDVSWITCH=0 VSWITCHNAME=external1
LIMITCPUFEATURES=”" USECOWIMAGES=1 LOGDIR=C:log
ENABLELOGGING=1 VERBOSELOGGING=1
QUANTUMURL=http://quantumhost:9696
QUANTUMADMINTENANTNAME=service
QUANTUMADMINUSERNAME=quantum
QUANTUMADMINPASSWORD=12345678
QUANTUMADMINAUTHURL=http://keystonehost:35357/v2.0
67. VLAN drivers issues
• BEWARE: Some Windows NIC drivers disable VLAN access
by default!
• HKEY_LOCAL_MACHINESYSTEMCurrentControlSetCo
ntrolClass{4d36e972-e325-11ce-bfc1-08002be10318}
• Look in all the child keys xxxx, e.g. 0001, 0002 etc for a
value "VLanFiltering". Make sure that if present it's set to 0.
• In case of changes, reboot the server or restart the
corresponding adapters.
69. Havana!!
• Planning for Havana now!
• Console integration
• More Quantum
• Ceilometer Hyper-V agent
• Heat Windows Cloud-Init support
• VHDX support
• Ephemeral storage
• Support for more image formats (e.g. AMI/ARI/AKI)
• Nova rescue
• Active Directory Keystone scalability
• VDI support
• Fibre channel
• VM dynamic memory support
• Cinder improvements…