Azure Synapse is the evolution of Azure SQL Data Warehouse, combining big data, data storage and data integration into a single service for end-to-end cloud scale analytics. It provides unlimited analytics with unparalleled speed to gain insights. Azure Synapse brings together enterprise data warehousing and big data analytics to give a unified experience with the advantages of both worlds.
This document provides an introduction to Azure Synapse Analytics, a modern data warehousing solution that combines enterprise data warehousing and big data analytics. It discusses how Azure Synapse Analytics allows for data ingestion, preparation, storage, and serves/visualizes data. It also covers how to integrate data with Azure Data Factory or Azure Synapse Pipelines, use Apache Spark pools for big data engineering, and ingest data using Apache Spark notebooks in Azure Synapse Analytics.
Big Data and Data Warehousing Together with Azure Synapse Analytics (SQLBits ...Michael Rys
SQLBits 2020 presentation on how you can build solutions based on the modern data warehouse pattern with Azure Synapse Spark and SQL including demos of Azure Synapse.
NOVA SQL User Group - Azure Synapse Analytics Overview - May 2020Timothy McAliley
Jim Boriotti presents an overview and demo of Azure Synapse Analytics, an integrated data platform for business intelligence, artificial intelligence, and continuous intelligence. Azure Synapse Analytics includes Synapse SQL for querying with T-SQL, Synapse Spark for notebooks in Python, Scala, and .NET, and Synapse Pipelines for data workflows. The demo shows how Azure Synapse Analytics provides a unified environment for all data tasks through the Synapse Studio interface.
The new Microsoft Azure SQL Data Warehouse (SQL DW) is an elastic data warehouse-as-a-service and is a Massively Parallel Processing (MPP) solution for "big data" with true enterprise class features. The SQL DW service is built for data warehouse workloads from a few hundred gigabytes to petabytes of data with truly unique features like disaggregated compute and storage allowing for customers to be able to utilize the service to match their needs. In this presentation, we take an in-depth look at implementing a SQL DW, elastic scale (grow, shrink, and pause), and hybrid data clouds with Hadoop integration via Polybase allowing for a true SQL experience across structured and unstructured data.
Building Modern Data Platform with Microsoft AzureDmitry Anoshin
This document provides an overview of building a modern cloud analytics solution using Microsoft Azure. It discusses the role of analytics, a history of cloud computing, and a data warehouse modernization project. Key challenges covered include lack of notifications, logging, self-service BI, and integrating streaming data. The document proposes solutions to these challenges using Azure services like Data Factory, Kafka, Databricks, and SQL Data Warehouse. It also discusses alternative implementations using tools like Matillion ETL and Snowflake.
Modernizing to a Cloud Data ArchitectureDatabricks
Organizations with on-premises Hadoop infrastructure are bogged down by system complexity, unscalable infrastructure, and the increasing burden on DevOps to manage legacy architectures. Costs and resource utilization continue to go up while innovation has flatlined. In this session, you will learn why, now more than ever, enterprises are looking for cloud alternatives to Hadoop and are migrating off of the architecture in large numbers. You will also learn how elastic compute models’ benefits help one customer scale their analytics and AI workloads and best practices from their experience on a successful migration of their data and workloads to the cloud.
Big data architectures and the data lakeJames Serra
The document provides an overview of big data architectures and the data lake concept. It discusses why organizations are adopting data lakes to handle increasing data volumes and varieties. The key aspects covered include:
- Defining top-down and bottom-up approaches to data management
- Explaining what a data lake is and how Hadoop can function as the data lake
- Describing how a modern data warehouse combines features of a traditional data warehouse and data lake
- Discussing how federated querying allows data to be accessed across multiple sources
- Highlighting benefits of implementing big data solutions in the cloud
- Comparing shared-nothing, massively parallel processing (MPP) architectures to symmetric multi-processing (
Azure Data Factory is a cloud-based data integration service that orchestrates and automates the movement and transformation of data. Key concepts in Azure Data Factory include pipelines, datasets, linked services, and activities. Pipelines contain activities that define actions on data. Datasets represent data structures. Linked services provide connection information. Activities include data movement and transformation. Azure Data Factory supports importing data from various sources and transforming data using technologies like HDInsight Hadoop clusters.
The document discusses migrating a data warehouse to the Databricks Lakehouse Platform. It outlines why legacy data warehouses are struggling, how the Databricks Platform addresses these issues, and key considerations for modern analytics and data warehousing. The document then provides an overview of the migration methodology, approach, strategies, and key takeaways for moving to a lakehouse on Databricks.
Azure Data Factory is a cloud-based data integration service that orchestrates and automates the movement and transformation of data. In this session we will learn how to create data integration solutions using the Data Factory service and ingest data from various data stores, transform/process the data, and publish the result data to the data stores.
Azure DataBricks for Data Engineering by Eugene PolonichkoDimko Zhluktenko
This document provides an overview of Azure Databricks, a Apache Spark-based analytics platform optimized for Microsoft Azure cloud services. It discusses key components of Azure Databricks including clusters, workspaces, notebooks, visualizations, jobs, alerts, and the Databricks File System. It also outlines how data engineers can leverage Azure Databricks for scenarios like running ETL pipelines, streaming analytics, and connecting business intelligence tools to query data.
The document introduces data engineering and provides an overview of the topic. It discusses (1) what data engineering is, how it has evolved with big data, and the required skills, (2) the roles of data engineers, data scientists, and data analysts in working with big data, and (3) the structure and schedule of an upcoming meetup on data engineering that will use an agile approach over monthly sprints.
Azure Databricks - An Introduction (by Kris Bock)Daniel Toomey
Azure Databricks is a fast, easy to use, and collaborative Apache Spark-based analytics platform optimized for Azure. It allows for interactive collaboration through a unified workspace, enables sharing of insights through integration with Power BI, and provides native integration with other Azure services. It also offers enterprise-grade security through integration with Azure Active Directory and compliance features.
Intuit's Data Mesh - Data Mesh Leaning Community meetup 5.13.2021Tristan Baker
Past, present and future of data mesh at Intuit. This deck describes a vision and strategy for improving data worker productivity through a Data Mesh approach to organizing data and holding data producers accountable. Delivered at the inaugural Data Mesh Leaning meetup on 5/13/2021.
Data Lakehouse, Data Mesh, and Data Fabric (r1)James Serra
So many buzzwords of late: Data Lakehouse, Data Mesh, and Data Fabric. What do all these terms mean and how do they compare to a data warehouse? In this session I’ll cover all of them in detail and compare the pros and cons of each. I’ll include use cases so you can see what approach will work best for your big data needs.
Introduction SQL Analytics on Lakehouse ArchitectureDatabricks
This document provides an introduction and overview of SQL Analytics on Lakehouse Architecture. It discusses the instructor Doug Bateman's background and experience. The course goals are outlined as describing key features of a data Lakehouse, explaining how Delta Lake enables a Lakehouse architecture, and defining features of the Databricks SQL Analytics user interface. The course agenda is then presented, covering topics on Lakehouse Architecture, Delta Lake, and a Databricks SQL Analytics demo. Background is also provided on Lakehouse architecture, how it combines the benefits of data warehouses and data lakes, and its key features.
Data Lakehouse Symposium | Day 1 | Part 2Databricks
The world of data architecture began with applications. Next came data warehouses. Then text was organized into a data warehouse.
Then one day the world discovered a whole new kind of data that was being generated by organizations. The world found that machines generated data that could be transformed into valuable insights. This was the origin of what is today called the data lakehouse. The evolution of data architecture continues today.
Come listen to industry experts describe this transformation of ordinary data into a data architecture that is invaluable to business. Simply put, organizations that take data architecture seriously are going to be at the forefront of business tomorrow.
This is an educational event.
Several of the authors of the book Building the Data Lakehouse will be presenting at this symposium.
This document discusses the future of data and the Azure data ecosystem. It highlights that by 2025 there will be 175 zettabytes of data in the world and the average person will have over 5,000 digital interactions per day. It promotes Azure services like Power BI, Azure Synapse Analytics, Azure Data Factory and Azure Machine Learning for extracting value from data through analytics, visualization and machine learning. The document provides overviews of key Azure data and analytics services and how they fit together in an end-to-end data platform for business intelligence, artificial intelligence and continuous intelligence applications.
This document provides an agenda and summary for a Data Analytics Meetup (DAM) on March 27, 2018. The agenda covers topics such as disruption opportunities in a changing data landscape, transitioning from traditional to modern BI architectures using Azure, Azure SQL Database vs Data Warehouse, data integration with Azure Data Factory and SSIS, Analysis Services, Power BI reporting, and a wrap-up. The document discusses challenges around data growth, digital transformation, and the shrinking time for companies to adapt to disruption. It provides overviews and comparisons of Azure SQL Database, Data Warehouse, and related Azure services to help modernize analytics architectures.
This webinar by Volodymyr Trishyn (Senior Software Engineer, Consultant, GlobalLogic) was delivered at On Air webinar #15 on July 31, 2020.
Webinar agenda:
- SQL Database
- Azure SQL Data Warehouse
- Azure SQL Elastic Database Pool
- Geo-replication
- Distributed Transactions
- Transaction Isolation Level
- Table Partitioning
- Materialized View Pattern
More details and presentation: https://www.globallogic.com/ua/about/events/webinar-azure-sql/
Self-serve analytics journey at Celtra: Snowflake, Spark, and DatabricksGrega Kespret
Celtra provides a platform for streamlined ad creation and campaign management used by customers including Porsche, Taco Bell, and Fox to create, track, and analyze their digital display advertising. Celtra’s platform processes billions of ad events daily to give analysts fast and easy access to reports and ad hoc analytics. Celtra’s Grega Kešpret leads a technical dive into Celtra’s data-pipeline challenges and explains how it solved them by combining Snowflake’s cloud data warehouse with Spark to get the best of both.
Topics include:
- Why Celtra changed its pipeline, materializing session representations to eliminate the need to rerun its pipeline
- How and why it decided to use Snowflake rather than an alternative data warehouse or a home-grown custom solution
- How Snowflake complemented the existing Spark environment with the ability to store and analyze deeply nested data with full consistency
- How Snowflake + Spark enables production and ad hoc analytics on a single repository of data
Ai big dataconference_eugene_polonichko_azure data lake Olga Zinkevych
Topic of presentation: Azure Data Lake: what is it? why is it? where is it?
The main points of the presentation:
What is Azure Data Lake? Why does this technology call Microsoft Big Data? Azure Data Lake includes all the capabilities required to make it easy for developers, data scientists, and analysts to store data of any size, shape, and speed, and do all types of processing and analytics across platforms and languages. It removes the complexities of ingesting and storing all of your data while making it faster to get up and running with batch, streaming, and interactive analytics.
http://dataconf.com.ua/index.php#agenda
#dataconf
#AIBDConference
Eugene Polonichko "Architecture of modern data warehouse"Lviv Startup Club
The document discusses the architecture of a modern data warehouse using Microsoft technologies. It describes traditional data warehousing approaches and outlines ten characteristics of a modern data warehouse. It then details Microsoft's approach using Azure Data Factory to ingest diverse data types into Azure Blob Storage, Azure Databricks for analytics and data transformation, and Azure SQL Data Warehouse for combined structured data. It also discusses technologies for storage, visualization, and links for further information.
Slides: Proven Strategies for Hybrid Cloud Computing with Mainframes — From A...DATAVERSITY
Mainframes continue to perform mission-critical transaction processing and contain massive amounts of core business data. But digital transformation initiatives and cloud computing have created both opportunities and challenges for unlocking and utilizing this data. Qlik and AWS will share some of the proven strategies from successful customer deployments across a range of different mainframe to cloud use cases, including legacy application modernization, data analytics, and data migrations.
In this presentation, you will learn how to:
• Replicate very large volumes of mainframe data in real-time to the cloud
• Automate the creation of analytics-ready data lakes and data warehouses
• Achieve a 30% reduction in cost of compute
This was a very interesting conference, TIC students oriented where I take him to the azure ecosystem for data warehousing architecture and best practices to reach powerful Business Intelligence Solutions according to the new era
Building near real-time HTAP solutions using Synapse Link for Azure Cosmos DBTimothy McAliley
This document discusses using Azure Cosmos DB and Azure Synapse Analytics together to enable near real-time analytics on operational data. It notes that running analytics directly on an operational database can impact performance. Instead, it recommends using Azure Synapse Link for Cosmos DB to automatically sync operational data to Azure Synapse Analytics for analytics and reporting. This allows separating OLTP and OLAP workloads while still providing near real-time insights. It then demonstrates this solution for a retailer doing supply chain management and sales forecasting across thousands of locations.
The document discusses Azure Synapse Analytics and its core services. It provides an overview of Azure Synapse Analytics, its high-level architecture, components and features. It discusses Azure Synapse Studio, Azure Synapse Data Integration, Synapse SQL Pools, Apache Spark for Azure Synapse and Azure Synapse security. It also discusses Azure HDInsight, its features, architecture, metastore best practices, migration practices and security and DevOps.
TahseenFiroz is a senior systems engineer with over 3 years of experience in Microsoft business intelligence technologies including SQL Server, SSIS, SSAS, SSRS, and Power View. He has extensive experience performing ETL processes, data modeling, performance tuning, and report development. His skills include SQL Server, SSIS, SSAS, Power BI, and SSRS. He has worked on several projects for Microsoft involving building data warehouses and analytic solutions for partners and customer support systems.
Oracle Business Intelligence is a product of Oracle Corporation. It is a Data Warehousing BI tool. It is very user friendly. OBIEE is one of the most emerging reporting tools ever since Oracle has taken over Siebel. In the coming days there are going to be many existing and new projects that will be migrated to Obiee from their existing reporting tools. And above all Obiee is very easy to learn and fun to learn, you do not need coding skills, but just a little logic and familiarity with the tool. And we are training on version 11g (11.1.1.6) which is a new release.
Streaming Real-time Data to Azure Data Lake Storage Gen 2Carole Gunst
Check out this presentation to learn the basics of using Attunity Replicate to stream real-time data to Azure Data Lake Storage Gen2 for analytics projects.
Dans cette session nous vous présenterons les différentes manières d'utiliser SQL Server dans une infrastructure Cloud (Microsoft Azure). Seront présentés des scénarios hybrides, de migration, de backup, et d'hébergement de bases de données SQL Server en mode IaaS ou PaaS.
How to Use a Semantic Layer on Big Data to Drive AI & BI ImpactDATAVERSITY
Learn about using a semantic layer to make data accessible and how to accelerate the business impact of AI and BI at your organization.
This session will offer practical advice on how to drive AI & BI business outcomes with an effective data strategy that leverages a semantic layer.
You will learn how to achieve quantifiable results by modernizing your data and analytics stack with a semantic layer that delivers an order of magnitude better query performance, increased data team productivity, lower query compute costs, and improved Speed-to-Insights.
Attend this session to learn about:
- Gaining business alignment and reducing data prep for your AI and BI teams.
- Making a consistent set of business metrics “analytics-ready” and accessible.
- Accelerating end-to-end query performance while optimizing cloud resources.
- Treating “data as a product” and how to drive business value for all consumers.
Logical Data Warehouse: How to Build a Virtualized Data Services LayerDataWorks Summit
The document discusses the emergence of logical data warehouses in response to big data. It describes how a logical data warehouse uses virtualization, distributed processing, and other techniques to provide a unified view of data across different repositories like Hadoop, relational databases and NoSQL stores. It also discusses how organizations can optimize resources by offloading analytical workloads from their enterprise data warehouse to Hadoop clusters to reduce costs while still using existing code and applications.
Similar to Data warehouse con azure synapse analytics (20)
Este documento describe la evolución de los grandes datos y la analítica, incluyendo el aumento de fuentes de datos, la comprensión de su valor, y la disminución de costos de hardware. También resume los componentes clave de Hadoop como HDFS, MapReduce, Hive y otros para el procesamiento y análisis de grandes cantidades de datos.
Creando tu primer ambiente de AI en Azure ML y SQL ServerEduardo Castro
Este documento proporciona una introducción a cómo crear el primer entorno de inteligencia artificial en Azure. Explica brevemente los beneficios de la inteligencia artificial y el aprendizaje automático para los negocios. Luego describe algunos de los servicios principales de Azure que pueden usarse para analizar datos, desarrollar modelos de aprendizaje automático y implementar soluciones de IA, como Azure Machine Learning, Databricks y HDInsight.
El documento describe las diferentes características y capacidades de seguridad disponibles en Azure SQL Database y Azure SQL Data Warehouse. Incluye gráficos que muestran el número de vulnerabilidades abordadas desde 2010 hasta 2018 y describe opciones como cifrado de datos en tránsito y en reposo, autenticación multifactor, firewalls, detección de amenazas, auditoría y más. El objetivo es ayudar a los clientes a proteger y auditar sus datos de manera segura en la nube.
Este documento describe cómo integrar Azure Synapse con MLflow para habilitar el seguimiento de experimentos de aprendizaje automático y el registro y despliegue de modelos en Azure Machine Learning. Explica cómo configurar los cuadernos de Azure Synapse para usar MLflow conectado a un área de trabajo de Azure Machine Learning, registrar modelos entrenados en Synapse en el registro de modelos de Azure ML y desplegarlos para su uso.
SQL Server can be installed on Windows Server 2022. Eduardo Castro provides a demonstration of how to install SQL Server on the latest Windows server operating system. His demonstration is available at a GitHub link that tracks an issue regarding documentation on installing SQL Server with Windows Server 2022.
El documento describe las nuevas características de SQL Server 2022, incluyendo la integración bidireccional con Azure SQL para replicación de datos, Azure Synapse Link para transferencia automática de cambios a Synapse Analytics, integración con Azure Purview para detección y clasificación de datos, mejoras en rendimiento a través de Query Store y optimización de planes, y mejoras en seguridad, disponibilidad y resolución de conflictos de réplicas.
SQL Server 2022 está habilitado para Azure para recuperación ante desastres, análisis y seguridad. Ofrece nuevas innovaciones como inteligencia de consultas integrada para mejorar el rendimiento, compatibilidad con almacenamiento de objetos y funciones extendidas de T-SQL para nuevos escenarios.
Machine Learning con Azure Managed InstanceEduardo Castro
En esta presentación mostramos las opciones para implementar Machine Learning dentro de Azure, así como las formas de configurar y utilizar Python dentro de Azure Managed Instance
El documento describe las nuevas características de SQL Server 2022, incluyendo la integración bidireccional con Azure SQL para replicación de datos, Azure Synapse Link para transferencia automática de cambios a Synapse Analytics, integración con Azure Purview para detección y clasificación de datos, mejoras en rendimiento a través de Query Store y optimización de planes, nuevas funciones de seguridad como ledger inmutable, y automatización de conflictos de réplicas en entornos de múltiples escrituras.
SQL Server can be installed on Windows Server 2022. Eduardo Castro provides a demonstration of how to install SQL Server on the latest Windows server operating system. His demonstration is available at a GitHub link that tracks an issue regarding documentation on installing SQL Server with Windows Server 2022.
Este documento presenta una introducción a Apache Spark y Azure Databricks. Explica que Spark es un motor de procesamiento de datos a gran escala de código abierto que incluye características como Spark SQL, aprendizaje automático, procesamiento de flujos y grafos. Luego describe cómo Azure Databricks es una plataforma unificada para análisis que utiliza Spark y ofrece mejor rendimiento, procesamiento de grandes volúmenes de datos y arquitectura de clústeres. Finalmente, incluye una demostración de las capacidades de
Este documento proporciona una introducción a los pronósticos con SQL Server 2019, discutiendo métodos como promedios móviles, suavizado exponencial, proyección de tendencias y regresión lineal. También describe cómo SQL Server 2019 permite a los científicos de datos y desarrolladores interactuar directamente con los datos y realizar análisis avanzados dentro de la base de datos, lo que puede aplicarse a soluciones como detección de fraude, pronósticos de ventas y mantenimiento predictivo.
Que hay de nuevo en el Azure Data Lake Storage Gen2Eduardo Castro
Este documento proporciona una actualización sobre las novedades de Azure Data Lake Storage. Incluye mejoras en el rendimiento, escalabilidad de costos, seguridad, soporte para almacenamiento de blobs y sistemas de archivos jerárquicos, y una vista previa de las integraciones con Azure Event Grid y Azure Synapse Analytics.
Azure Synapse Analytics es un servicio de análisis que combina big data, almacenamiento de datos e integración de datos en un solo servicio con escalabilidad en la nube. Ofrece análisis de datos end-to-end con tiempos de respuesta en segundos utilizando SQL, Python, R y otros lenguajes. Incluye características como ingesta de datos, almacenamiento de datos, análisis SQL, machine learning integrado y más.
Este documento presenta los Servicios Cognitivos de Microsoft, que proporcionan APIs de visión, habla, lenguaje y análisis de datos para permitir que las aplicaciones tengan capacidades como reconocimiento facial, detección de emociones, extracción de frases clave y comprensión del lenguaje natural. Los servicios cognitivos se pueden integrar fácilmente en aplicaciones y ayudan a los equipos de datos a resolver problemas en áreas como la atención médica, la seguridad y el comercio minorista.
Script de paso a paso de configuración de Secure EnclavesEduardo Castro
El documento proporciona instrucciones para configurar un equipo HGS como host protegido y otro equipo con SQL Server para usar enclaves seguros con Always Encrypted. Se explica cómo instalar el servicio de protección de host en HGS, configurar el dominio HGS, configurar la atestación de claves y obtener la dirección IP de HGS. Luego, se indica cómo configurar el equipo SQL Server como host protegido, generar y registrar su clave de host, e indicarle dónde debe realizar la atestación. Finalmente, se habilitan los en
Introducción a conceptos de SQL Server Secure EnclavesEduardo Castro
Este documento describe varias técnicas de cifrado de datos, incluido el cifrado de datos en reposo, en uso y en tránsito. Se centra en particular en Always Encrypted, una solución que permite cifrar datos sensibles en las columnas de una base de datos de forma que se mantengan las consultas enriquecidas. Explica cómo los datos cifrados se almacenan de forma segura utilizando claves maestras de columna almacenadas externamente, y cómo las aplicaciones pueden recuperar datos desencriptados de forma segura mediante el uso de encl
Este documento presenta Azure SQL Data Warehouse, un servicio de almacenamiento de datos relacionales completamente administrado en la nube. Explica que ofrece escalabilidad elástica, permitiendo aumentar o disminuir los recursos de computación y almacenamiento según sea necesario. También destaca que está integrado con otros servicios de Azure como Power BI, Azure Data Factory y Azure Machine Learning.
Quality Patents: Patents That Stand the Test of TimeAurora Consulting
Is your patent a vanity piece of paper for your office wall? Or is it a reliable, defendable, assertable, property right? The difference is often quality.
Is your patent simply a transactional cost and a large pile of legal bills for your startup? Or is it a leverageable asset worthy of attracting precious investment dollars, worth its cost in multiples of valuation? The difference is often quality.
Is your patent application only good enough to get through the examination process? Or has it been crafted to stand the tests of time and varied audiences if you later need to assert that document against an infringer, find yourself litigating with it in an Article 3 Court at the hands of a judge and jury, God forbid, end up having to defend its validity at the PTAB, or even needing to use it to block pirated imports at the International Trade Commission? The difference is often quality.
Quality will be our focus for a good chunk of the remainder of this season. What goes into a quality patent, and where possible, how do you get it without breaking the bank?
** Episode Overview **
In this first episode of our quality series, Kristen Hansen and the panel discuss:
⦿ What do we mean when we say patent quality?
⦿ Why is patent quality important?
⦿ How to balance quality and budget
⦿ The importance of searching, continuations, and draftsperson domain expertise
⦿ Very practical tips, tricks, examples, and Kristen’s Musts for drafting quality applications
https://www.aurorapatents.com/patently-strategic-podcast.html
Kief Morris rethinks the infrastructure code delivery lifecycle, advocating for a shift towards composable infrastructure systems. We should shift to designing around deployable components rather than code modules, use more useful levels of abstraction, and drive design and deployment from applications rather than bottom-up, monolithic architecture and delivery.
UiPath Community Day Kraków: Devs4Devs ConferenceUiPathCommunity
We are honored to launch and host this event for our UiPath Polish Community, with the help of our partners - Proservartner!
We certainly hope we have managed to spike your interest in the subjects to be presented and the incredible networking opportunities at hand, too!
Check out our proposed agenda below 👇👇
08:30 ☕ Welcome coffee (30')
09:00 Opening note/ Intro to UiPath Community (10')
Cristina Vidu, Global Manager, Marketing Community @UiPath
Dawid Kot, Digital Transformation Lead @Proservartner
09:10 Cloud migration - Proservartner & DOVISTA case study (30')
Marcin Drozdowski, Automation CoE Manager @DOVISTA
Pawel Kamiński, RPA developer @DOVISTA
Mikolaj Zielinski, UiPath MVP, Senior Solutions Engineer @Proservartner
09:40 From bottlenecks to breakthroughs: Citizen Development in action (25')
Pawel Poplawski, Director, Improvement and Automation @McCormick & Company
Michał Cieślak, Senior Manager, Automation Programs @McCormick & Company
10:05 Next-level bots: API integration in UiPath Studio (30')
Mikolaj Zielinski, UiPath MVP, Senior Solutions Engineer @Proservartner
10:35 ☕ Coffee Break (15')
10:50 Document Understanding with my RPA Companion (45')
Ewa Gruszka, Enterprise Sales Specialist, AI & ML @UiPath
11:35 Power up your Robots: GenAI and GPT in REFramework (45')
Krzysztof Karaszewski, Global RPA Product Manager
12:20 🍕 Lunch Break (1hr)
13:20 From Concept to Quality: UiPath Test Suite for AI-powered Knowledge Bots (30')
Kamil Miśko, UiPath MVP, Senior RPA Developer @Zurich Insurance
13:50 Communications Mining - focus on AI capabilities (30')
Thomasz Wierzbicki, Business Analyst @Office Samurai
14:20 Polish MVP panel: Insights on MVP award achievements and career profiling
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
Sustainability requires ingenuity and stewardship. Did you know Pigging Solutions pigging systems help you achieve your sustainable manufacturing goals AND provide rapid return on investment.
How? Our systems recover over 99% of product in transfer piping. Recovering trapped product from transfer lines that would otherwise become flush-waste, means you can increase batch yields and eliminate flush waste. From raw materials to finished product, if you can pump it, we can pig it.
Implementations of Fused Deposition Modeling in real worldEmerging Tech
The presentation showcases the diverse real-world applications of Fused Deposition Modeling (FDM) across multiple industries:
1. **Manufacturing**: FDM is utilized in manufacturing for rapid prototyping, creating custom tools and fixtures, and producing functional end-use parts. Companies leverage its cost-effectiveness and flexibility to streamline production processes.
2. **Medical**: In the medical field, FDM is used to create patient-specific anatomical models, surgical guides, and prosthetics. Its ability to produce precise and biocompatible parts supports advancements in personalized healthcare solutions.
3. **Education**: FDM plays a crucial role in education by enabling students to learn about design and engineering through hands-on 3D printing projects. It promotes innovation and practical skill development in STEM disciplines.
4. **Science**: Researchers use FDM to prototype equipment for scientific experiments, build custom laboratory tools, and create models for visualization and testing purposes. It facilitates rapid iteration and customization in scientific endeavors.
5. **Automotive**: Automotive manufacturers employ FDM for prototyping vehicle components, tooling for assembly lines, and customized parts. It speeds up the design validation process and enhances efficiency in automotive engineering.
6. **Consumer Electronics**: FDM is utilized in consumer electronics for designing and prototyping product enclosures, casings, and internal components. It enables rapid iteration and customization to meet evolving consumer demands.
7. **Robotics**: Robotics engineers leverage FDM to prototype robot parts, create lightweight and durable components, and customize robot designs for specific applications. It supports innovation and optimization in robotic systems.
8. **Aerospace**: In aerospace, FDM is used to manufacture lightweight parts, complex geometries, and prototypes of aircraft components. It contributes to cost reduction, faster production cycles, and weight savings in aerospace engineering.
9. **Architecture**: Architects utilize FDM for creating detailed architectural models, prototypes of building components, and intricate designs. It aids in visualizing concepts, testing structural integrity, and communicating design ideas effectively.
Each industry example demonstrates how FDM enhances innovation, accelerates product development, and addresses specific challenges through advanced manufacturing capabilities.
Fluttercon 2024: Showing that you care about security - OpenSSF Scorecards fo...Chris Swan
Have you noticed the OpenSSF Scorecard badges on the official Dart and Flutter repos? It's Google's way of showing that they care about security. Practices such as pinning dependencies, branch protection, required reviews, continuous integration tests etc. are measured to provide a score and accompanying badge.
You can do the same for your projects, and this presentation will show you how, with an emphasis on the unique challenges that come up when working with Dart and Flutter.
The session will provide a walkthrough of the steps involved in securing a first repository, and then what it takes to repeat that process across an organization with multiple repos. It will also look at the ongoing maintenance involved once scorecards have been implemented, and how aspects of that maintenance can be better automated to minimize toil.
Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Em...Erasmo Purificato
Slide of the tutorial entitled "Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Emerging Trends" held at UMAP'24: 32nd ACM Conference on User Modeling, Adaptation and Personalization (July 1, 2024 | Cagliari, Italy)
INDIAN AIR FORCE FIGHTER PLANES LIST.pdfjackson110191
These fighter aircraft have uses outside of traditional combat situations. They are essential in defending India's territorial integrity, averting dangers, and delivering aid to those in need during natural calamities. Additionally, the IAF improves its interoperability and fortifies international military alliances by working together and conducting joint exercises with other air forces.
Best Programming Language for Civil EngineersAwais Yaseen
The integration of programming into civil engineering is transforming the industry. We can design complex infrastructure projects and analyse large datasets. Imagine revolutionizing the way we build our cities and infrastructure, all by the power of coding. Programming skills are no longer just a bonus—they’re a game changer in this era.
Technology is revolutionizing civil engineering by integrating advanced tools and techniques. Programming allows for the automation of repetitive tasks, enhancing the accuracy of designs, simulations, and analyses. With the advent of artificial intelligence and machine learning, engineers can now predict structural behaviors under various conditions, optimize material usage, and improve project planning.
Mitigating the Impact of State Management in Cloud Stream Processing SystemsScyllaDB
Stream processing is a crucial component of modern data infrastructure, but constructing an efficient and scalable stream processing system can be challenging. Decoupling compute and storage architecture has emerged as an effective solution to these challenges, but it can introduce high latency issues, especially when dealing with complex continuous queries that necessitate managing extra-large internal states.
In this talk, we focus on addressing the high latency issues associated with S3 storage in stream processing systems that employ a decoupled compute and storage architecture. We delve into the root causes of latency in this context and explore various techniques to minimize the impact of S3 latency on stream processing performance. Our proposed approach is to implement a tiered storage mechanism that leverages a blend of high-performance and low-cost storage tiers to reduce data movement between the compute and storage layers while maintaining efficient processing.
Throughout the talk, we will present experimental results that demonstrate the effectiveness of our approach in mitigating the impact of S3 latency on stream processing. By the end of the talk, attendees will have gained insights into how to optimize their stream processing systems for reduced latency and improved cost-efficiency.
Best Practices for Effectively Running dbt in Airflow.pdfTatiana Al-Chueyr
As a popular open-source library for analytics engineering, dbt is often used in combination with Airflow. Orchestrating and executing dbt models as DAGs ensures an additional layer of control over tasks, observability, and provides a reliable, scalable environment to run dbt models.
This webinar will cover a step-by-step guide to Cosmos, an open source package from Astronomer that helps you easily run your dbt Core projects as Airflow DAGs and Task Groups, all with just a few lines of code. We’ll walk through:
- Standard ways of running dbt (and when to utilize other methods)
- How Cosmos can be used to run and visualize your dbt projects in Airflow
- Common challenges and how to address them, including performance, dependency conflicts, and more
- How running dbt projects in Airflow helps with cost optimization
Webinar given on 9 July 2024
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc
Six months into 2024, and it is clear the privacy ecosystem takes no days off!! Regulators continue to implement and enforce new regulations, businesses strive to meet requirements, and technology advances like AI have privacy professionals scratching their heads about managing risk.
What can we learn about the first six months of data privacy trends and events in 2024? How should this inform your privacy program management for the rest of the year?
Join TrustArc, Goodwin, and Snyk privacy experts as they discuss the changes we’ve seen in the first half of 2024 and gain insight into the concrete, actionable steps you can take to up-level your privacy program in the second half of the year.
This webinar will review:
- Key changes to privacy regulations in 2024
- Key themes in privacy and data governance in 2024
- How to maximize your privacy program in the second half of 2024
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-In
Data warehouse con azure synapse analytics
1. 08-May-20 7:12 AM
1
Azure Synapse es la evolución de Azure SQL Data Warehouse,
combinando big data, almacenamiento de datos e integración de datos
en un único servicio para análisis de extremo a extremo a escala de nube.
Azure Synapse Analytics
Servicio de análisis ilimitado con un tiempo inigualable para obtener información
2. 08-May-20 7:12 AM
2
INGEST
Data warehouse moderno
PREPARE TRANSFORM
& ENRICH
SERVE
STORE
VISUALIZE
On-premises data
Cloud data
SaaS data
Integrated data platform for BI, AI and continuous intelligence
Platform
Azure
Data Lake Storage
Common Data Model
Enterprise Security
Optimized for Analytics
METASTORE
SECURITY
MANAGEMENT
MONITORING
DATA INTEGRATION
Analytics Runtimes
PROVISIONED ON-DEMAND
Form Factors
SQL
Languages
Python .NET Java Scala R
Experience Synapse Analytics Studio
Artificial Intelligence / Machine Learning / Internet of Things
Intelligent Apps / Business Intelligence
3. 08-May-20 7:12 AM
3
Plataforma de datos integrada para BI, IA e inteligencia continua
Platform
Azure
Data Lake Storage
Common Data Model
Enterprise Security
Optimized for Analytics
METASTORE
SECURITY
MANAGEMENT
MONITORING
DATA INTEGRATION
Analytics Runtimes
PROVISIONED ON-DEMAND
Form Factors
SQL
Languages
Python .NET Java Scala R
Experience Synapse Analytics Studio
Inteligencia Artificial / Aprendizaje Automático / Internet de las
cosas/ Aplicaciones inteligentes / Inteligencia empresarial
Servicios conectados
Azure Data Catalog
Azure Data Lake Storage
Azure Data Share
Azure Databricks
Azure HDInsight
Azure Machine Learning
Power BI
3rd Party Integration
Arquitecturas elásticas
Híbrido
Analizar todos los datosComputación
optimizada para cargas
de trabajo
Autoservicio gobernadoSin silos de datos
4. 08-May-20 7:12 AM
4
Tiempo Costo Riesgo
Plataforma: Rendimiento
• Azure Synapse aprovecha el ecosistema de Azure y las
mejoras principales del motor de SQL Server para producir
mejoras masivas en el rendimiento.
• Estos beneficios no requieren ninguna configuración del
cliente y se proporcionan de fábrica para cada almacén de
datos
• Gen2 adaptive caching – utilizando unidades de estado
sólido (NVMe) de memoria no volátil para aumentar el
ancho de banda de E/S disponible para las consultas.
• Azure FPGA-accelerated networking enhancements – para
mover datos a velocidades de hasta 1 GB/s por nodo para
mejorar las consultas
• Instant data movement – aprovecha el paralelismo
multinúcleo en los servidores SQL Server subyacentes para
mover datos de forma eficiente entre nodos de proceso.
• Query Optimization –optimización de consultas
distribuidas
7. 08-May-20 7:12 AM
7
Gestión de la
carga de
trabajo
Scale-In Isolation
Coste predecible
Elasticidaden línea
Eficiente paracargasde trabajo impredecibles
Intra Cluster Workload Isolation
(Scale In)
Marketing
CREATE WORKLOAD GROUP Sales
WITH
(
[ MIN_PERCENTAGE_RESOURCE = 60 ]
[ CAP_PERCENTAGE_RESOURCE = 100 ]
[ MAX_CONCURRENCY = 6 ] )
40%
Compute
1000c DWU
60%
Sales
60%
100%
Seguridad integral
Category Feature
Data Protection
Data in Transit
Data Encryption at Rest
Data Discovery and Classification
Access Control
Object Level Security (Tables/Views)
Row Level Security
Column Level Security
Dynamic Data Masking
SQL Login
Authentication Azure Active Directory
Multi-Factor Authentication
Virtual Networks
Network Security Firewall
Azure ExpressRoute
Thread Detection
Threat Protection Auditing
Vulnerability Assessment
8. 08-May-20 7:12 AM
8
Integración de
datos
Data Warehouse Reporting
Integración de datos de Synapse
Más de 90 conectores listos para usar
Sin servidor, sin infraestructura que
administrar
Ingestión sostenida de 4 GB/s
CSV, AVRO, ORC, Parquet, JSON support
9. 08-May-20 7:12 AM
9
Integración de datos de Synapse
Code First
Code Free
GUI based
+ many more
Power BI Azure Machine Learning
Azure Data Share Ecosystem
Azure Synapse Analytics
10. 08-May-20 7:12 AM
10
Data Integration Data Warehouse Reporting
Almacenamiento optimizado para el rendimiento
Elastic Architecture Columnar Storage Columnar Ordering Table Partitioning
Nonclustered Indexes Hash Distribution Materialized Views Resultset Cache
11. 08-May-20 7:12 AM
11
Migración de tablas de base de datos
CREATE TABLE StoreSales (
[sales_city] varchar(60),
[sales_year] int,
[sales_state] char(2),
[item_sk] int,
[sales_zip] char(10),
[sales_date] date,
[customer_sk] int)
WITH(
CLUSTERED COLUMNSTORE INDEX ORDER ([customer_sk]),
DISTRIBUTION = HASH([sales_zip],[item_sk]),
PARTITION ([sales_year] RANGE RIGHT FOR VALUES (1998,1999,2000,2001,2002,2003)))
Vista de base de
datos
Migración Materialized Views
Views
12. 08-May-20 7:12 AM
12
Migración de vista de base de
datos
Vista Vista materializada
Abstrae estructura a los usuarios YES YES
Requiere una referencia explícita YES No
Mejora el rendimiento No YES
Se requiere almacenamiento adicional No YES
Asegurable YES YES
Soporte completo de SQL
YES No
Migración de vista de base de datos
CREATE VIEW vw_TopSalesState
AS
SELECT
SubQ.StateAbbrev,
SubQ.FirstSoldDate,
(SubQ.SalesPrice / sum(SubQ.SalesPrice) OVER (order by (select null)))*100,
(1- (SalesPrice/ListPrice))*100 AS Discount,
RANK() OVER (order by (1- (SalesPrice/ListPrice))) AS StateDiscRank
FROM (
SELECT
s_state AS StateAbbrev,
MIN(d_date) AS FirstSoldDate,
SUM([ss_list_price]) AS ListPrice,
SUM([ss_sales_price]) AS SalesPrice
FROM [tpcds10TB].[store_sales2] ss
INNER JOIN [tpcds10TB].store s on s.[s_store_sk] = ss.[ss_store_sk]
INNER JOIN [tpcds10TB].[date_dim] d on d.[d_date_sk] = ss.ss_sold_date_sk
GROUP BY
s_state) AS SubQ
13. 08-May-20 7:12 AM
13
Migración de la vista materializada de la base de datos
CREATE MATERIALIZED VIEW [dbo].[mvw_StoreSalesSummary]
WITH (DISTRIBUTION = HASH(ss_store_sk))
AS
SELECT
s_state,
c_birth_country,
ss_store_sk AS ss_store_sk,
ss_sold_date_sk AS ss_sold_date_sk,
SUM([ss_list_price]) AS [ss_list_price],
SUM([ss_sales_price]) AS [ss_sales_price],
count_big(*) AS cb
FROM [tpcds10TB].[store_sales2] ss
INNER JOIN [tpcds10TB].customer c ON c.[c_customer_sk] = ss.[ss_customer_sk]
INNER JOIN [tpcds10TB].store s on s.[s_store_sk] = ss.[ss_store_sk]
GROUP BY
s_state,c_birth_country,ss_store_sk, ss_sold_date_sk
Customer
65
Million
Rows
Store
1500
Rows
Store Sales
26
Billion
Rows
Materialized View
287
Million
Rows
Data Integration Data Warehouse Informes
14. 08-May-20 7:12 AM
14
Synapse Connected Service: Power BI
Experiencia integrada de
creación de Power BI
Publicar en Power BI
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
CREATE MATERIALZIED VIEW vw_ProductSales
WITH (DISTRIBUTION = HASH(ProductKey))
AS
SELECT
ProductName
ProductKey,
SUM(Amount) AS TotalSales
FROM
FactSales fs
INNER JOIN DimProduct dp ON fs.prodkey = dp.prodkey
GROUP BY
ProductName,
ProductKey
15. 08-May-20 7:12 AM
15
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
ProductName ProductKey TotalSales
Product A 5453 784,943.00
Product B 763 48,723.00
… … …
FactSales Table
10B Records
DimProduct Table
1,000 Records
FactSales
DimProduct
FactInventory
Table
mvw_ProductSales
1,000 Records
SELECT
ProductName
ProductKey,
SUM(Amount) AS TotalSales
FROM
FactSales fs
INNER JOIN DimProduct dp
GROUP BY
ProductName,
ProductKey
FactInventory
Escalado a
Petabytes
Result set Cache
Automaticquery matching
Implicitcreatingfrom queryactivity
Resilient to cluster elasticity
Execution2
Cache Hit
~.2 seconds
Execution1
Cache Miss
Regular Execution
16. 08-May-20 7:12 AM
16
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
CREATE MATERIALZIED VIEW vw_ProductSales
WITH (DISTRIBUTION = HASH(ProductKey))
AS
SELECT
ProductName
ProductKey,
SUM(Amount) AS TotalSales
FROM
FactSales fs
INNER JOIN DimProduct dp ON fs.prodkey = dp.prodkey
GROUP BY
ProductName,
ProductKey
ProductName ProductKey TotalSales
Product A 5453 784,943.00
Product B 763 48,723.00
… … …
FactSales Table
10B Records
DimProduct Table
1,000 Records
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
FactSales
DimProduct
FactInventory
Table
mvw_ProductSales
1,000 Records
SELECT
ProductName
ProductKey,
SUM(Amount) AS TotalSales
FROM
FactSales fs
INNER JOIN DimProduct dp
GROUP BY
ProductName,
ProductKey
FactInventory
17. 08-May-20 7:12 AM
17
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
SELECT
c_customerkey,
c_nationkey,
SUM(l_quantity),
SUM(l_extendedprice)
FROM [dbo].[lineitem_MonthPartition] l
INNER JOIN [dbo].[orders] o on o.o_orderkey = l.l_orderkey
INNER JOIN [dbo].[customer] c on c.c_customerkey = o.o_customerkey
GROUP BY
c_customerkey,
c_nationkey
[dbo].[lineitem_MonthPartition] HASH(l_orderkey)
[dbo].[orders] HASH(o_orderkey)
[dbo].[customer] HASH(c_customerkey)
Table Distributions
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
LineItem Orders
Collocated Join (DistributionAligned)
Customer
Non-collocatedJoin (Shuffle Required)
FROM [dbo].[lineitem_MonthPartition] l
INNER JOIN [dbo].[orders] o on o.o_orderkey = l.l_orderkey
INNER JOIN [dbo].[customer] c on c.c_customerkey = o.o_customerkey
18. 08-May-20 7:12 AM
18
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
(Shuffle Required)
LineItem Orders
Collocated Join (DistributionAligned)
Stage 1
Customer
Stage 2
#temp (Orders + Lineitem)
Nation
Collocated Join (Replicate Aligned)
Collocated Join (DistributionAligned)
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
CREATE MATERIALIZED VIEW mvw_CustomerSales
WITH (DISTRIBUTION = HASH(o_custkey))
AS
SELECT
o_custkey,
l_shipdate,
SUM(l_quantity) AS l_quantity,
SUM(l_extendedprice) AS l_extendedprice
FROM [dbo].[lineitem_MonthPartition] l
INNER JOIN [dbo].[orders] o on o.o_orderkey = l.l_orderkey
WHERE
l_shipdate >= CONVERT(DATETIME, '1998-11-01', 103)
GROUP BY
o_custkey,
l_shipdate
19. 08-May-20 7:12 AM
19
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
Legend
mvw_CustomerSales
Nation
Customer
<replicated table>
Collocated Join (DistributionAligned)
Collocated Join (Replicate Aligned)
Escalado a
Petabytes
Materialized Views
Transactionalconsistentlyto datamodification
AutomaticQueryOptimizermatching
275
5
0
50
100
150
200
250
300
No MaterializedView WithMaterializedView
Seconds
Query Execution Time
20. 08-May-20 7:12 AM
20
Power BI
Materialized Views
Tables
Escalado a
Petabytes
Power BI
DirectQuery
Composite Models
Aggregation Tables