This document discusses GraphQL and DGraph with GO. It begins by introducing GraphQL and some popular GraphQL implementations in GO like graphql-go. It then discusses DGraph, describing it as a distributed, high performance graph database written in GO. It provides examples of using the DGraph GO client to perform CRUD operations, querying for single and multiple objects, committing transactions, and more.
Deep learning (DL) is still one of the fastest developing areas in machine learning. As models increase their complexity and data sets grow in size, your model training can last hours or even days. In this session we will explore some of the trends in Deep Neural Networks to accelerate training using parallelize/distribute deep learning.
We will also present how to apply some of these strategies using Cloudera Data Science Workbenck and some popular (DL) open source frameworks like Uber Horovod, Tensorflow and Keras.
Speakers
Rafael Arana, Senior Solutions Architect
Cloudera
Zuling Kang, Senior Solutions Architect
Cloudera Inc.
This document introduces Apache Cassandra, a distributed column-oriented NoSQL database. It discusses Cassandra's architecture, data model, query language (CQL), and how to install and run Cassandra. Key points covered include Cassandra's linear scalability, high availability and fault tolerance. The document also demonstrates how to use the nodetool utility and provides guidance on backing up and restoring Cassandra data.
The document discusses big data and Hadoop concepts. It covers Hadoop operations like put, get, scan, filter, delete as well as join and group by. It also discusses the different types of data access patterns like random write, sequential read, sequential write and random read. The document focuses on big data, Hadoop operations, and data access patterns.
Google BigQuery is a cloud data warehouse and spreadsheet database that allows users to import, store, and query data in various formats like CSV, JSON, and Google Sheets. It provides a sandbox account with 10GB of free storage and 1TB of free queries per month. To use it, users create a BigQuery project, import data into datasets and tables, and then query the data using SQL syntax.
Apache kafka performance(throughput) - without data loss and guaranteeing dat...
Apache Kafak의 성능이 특정환경(데이터 유실일 발생하지 않고, 데이터 전송순서를 반드시 보장)에서 어느정도 제공하는지 확인하기 위한 테스트 결과 공유
데이터 전송순서를 보장하기 위해서는 Apache Kafka cluster로 partition을 분산할 수 없게되므로, 성능향상을 위한 장점을 사용하지 못하게 된다.
이번 테스트에서는 Apache Kafka의 단위 성능, 즉 partition 1개에 대한 성능만을 측정하게 된다.
향후, partition을 증가할 경우 본 테스트의 1개 partition 단위 성능을 기준으로 예측이 가능할 것 같다.
This document summarizes Spark SQL and DataFrames in Spark. It notes that Spark SQL is part of the core Spark distribution and allows running SQL and HiveQL queries. DataFrames provide a way to select, filter, aggregate and plot structured data like in R and Pandas. DataFrames allow writing less code through a high-level API and reading less data by using optimized formats and partitioning. The optimizer can optimize queries across functions and push down predicates to read less data. This allows creating and running Spark programs faster.
How Dashtable Helps Dragonfly Maintain Low Latency
Dashtable is a hashtable implementation inside Dragonfly. It supports incremental resizes and fast, cache-friendly operations. In this talk, we will learn how Dashtable helps Dragonfly to keep its tail latency in check. In Dashtable, long-tail latencies have been reduced by a factor of 1000x, but P999 are 7x longer. Find out why we still think this is a good tradeoff.
GraphQL is a query language for APIs that allows clients to request specific data rather than entire resources. It addresses common problems with REST APIs like over-fetching data. Several GraphQL libraries exist for Java including graphql-java, graphql-java-kickstart, graphql-dgs-framework, and spring-graphql. These libraries differ in features like code generation, error handling, testing support, and available clients. Spring GraphQL is considered the most mature option as it integrates directly with Spring Boot and allows testing all code.
The document discusses GraphQL and Relay Modern. It begins with an overview of REST and some of its issues. It then covers what GraphQL is, how it was created at Facebook, and some of its key features like being version-free and having a hierarchical data structure. The document also discusses Relay Modern and what it provides for managing GraphQL queries and mutations. It includes examples of using GraphQL schemas and Relay to fetch and display basic and list data.
This document discusses GraphQL and Relay Modern. It begins with an overview of REST and its issues. It then covers what GraphQL is, how it was created at Facebook, and its motivations. Key aspects of GraphQL discussed include its single endpoint, hierarchical nature, strong typing, response mirroring the query, and introspection capabilities. The document also discusses Relay Modern and what it provides like declarative queries and mutations. It provides examples of simple GraphQL schemas and Relay components. In the end, it discusses subscriptions, routing, debugging, and resources for learning more.
GraphQL and its schema as a universal layer for database access
GraphQL is a query language mostly used to streamline access to REST APIs. It is seeing tremendous growth and adoption, in organizations like Airbnb, Coursera, Docker, GitHub, Twitter, Uber, and Facebook, where it was invented.
As REST APIs are proliferating, the promise of accessing them all through a single query language and hub, which is what GraphQL and GraphQL server implementations bring, is alluring.
A significant recent addition to GraphQL was SDL, its schema definition language. SDL enables developers to define a schema governing interaction with the back-end that GraphQL servers can then implement and enforce.
Prisma is a productized version of the data layer leveraging GraphQL to access any database. Prisma works with MySQL, Postgres, and MongoDB, and is adding to this list.
Prisma sees the GraphQL community really coming together around the idea of schema-first development, and wants to use GraphQL SDL as the foundation for all interfaces between systems.
Neo4j Morpheus: Interweaving Table and Graph Data with SQL and Cypher in Apac...
This document discusses property graphs and how they are represented and queried using Morpheus, a graph query engine for Apache Spark.
Morpheus allows querying property graphs using Cypher and represents property graphs using DataFrames, with node and relationship data stored in tables. It integrates with various data sources and supports federated queries across multiple property graphs. The document provides examples of loading property graph data from sources like JSON, SQL databases and Neo4j, creating graph projections, running analytical queries, and recommending businesses based on graph algorithms.
The document discusses Prisma and GraphQL. It provides an overview of GraphQL concepts like schema, queries, and resolvers. It then covers the typical architecture of a GraphQL server including the schema definition, resolver functions, and server setup. Finally, it introduces Prisma as a database access layer that can be used to build GraphQL servers.
GraphQL is query language for APIs, but what are the advantages and how would one implement such in their microservices/APIs. In this session, I will go through the basics of GraphQL, different aspects of GraphQL and architecture of such APIs. How 4 different ways we can implement GraphQL for a Springboot microservice/API.
This document discusses GraphQL, a query language for APIs and a runtime for fulfilling queries with existing data. It provides examples of basic queries, nested fields, connections, arguments, and fragments. It also covers GraphQL types including scalars, schemas, definitions, predicates, and data resolution. GraphQL allows clients to define the structure of the data required, and specifies querying, modifying, and transmitting data between client and server.
Advanced Data Science with Apache Spark-(Reza Zadeh, Stanford)
The document discusses graph computations and Pregel, an API for graph processing. It introduces Pregel's vertex-centric programming model where computation is organized into supersteps and depends only on neighboring vertices. Examples like PageRank are shown implemented in Pregel. GraphX is also introduced as a library providing Pregel-like abstractions on Spark. The document then discusses distributing matrix computations, covering partitioning schemes for matrices and how to distribute operations like multiplication and singular value decomposition (SVD) across a cluster.
ScalaTo July 2019 - No more struggles with Apache Spark workloads in production
Scala Toronto July 2019 event at 500px.
Pure Functional API Integration
Apache Spark Internals tuning
Performance tuning
Query execution plan optimisation
Cats Effects for switching execution model runtime.
Discovery / experience with Monix, Scala Future.
GraphQL is query language for APIs, but what are the advantages and how would one implement such in their microservices/APIs. In this session, I will go through the basics of GraphQL, different aspects of GraphQL and architecture of such APIs. There will be a demo/live-coding on, how 4 different ways we can implement GraphQL for a Springboot microservice/API. Lots of examples, live coding and helpful comparison on structure, usage and implementations of GraphQL in Springboot & Java world.
Bigdam is a planet-scale data ingestion pipeline designed for large-scale data ingestion. It addresses issues with the traditional pipeline such as imperfectqueue throughput limitations, latency in queries from event collectors, difficulty maintaining event collector code, many small temporary and imported files. The redesigned pipeline includes Bigdam-Gateway for HTTP endpoints, Bigdam-Pool for distributed buffer storage, Bigdam-Scheduler to schedule import tasks, Bigdam-Queue as a high throughput queue, and Bigdam-Import for data conversion and import. Consistency is ensured through at-least-once design and deduplication is performed at the end of the pipeline for simplicity and reliability. Components are designed to scale out horizontally.
Deep learning (DL) is still one of the fastest developing areas in machine learning. As models increase their complexity and data sets grow in size, your model training can last hours or even days. In this session we will explore some of the trends in Deep Neural Networks to accelerate training using parallelize/distribute deep learning.
We will also present how to apply some of these strategies using Cloudera Data Science Workbenck and some popular (DL) open source frameworks like Uber Horovod, Tensorflow and Keras.
Speakers
Rafael Arana, Senior Solutions Architect
Cloudera
Zuling Kang, Senior Solutions Architect
Cloudera Inc.
This document introduces Apache Cassandra, a distributed column-oriented NoSQL database. It discusses Cassandra's architecture, data model, query language (CQL), and how to install and run Cassandra. Key points covered include Cassandra's linear scalability, high availability and fault tolerance. The document also demonstrates how to use the nodetool utility and provides guidance on backing up and restoring Cassandra data.
The document discusses big data and Hadoop concepts. It covers Hadoop operations like put, get, scan, filter, delete as well as join and group by. It also discusses the different types of data access patterns like random write, sequential read, sequential write and random read. The document focuses on big data, Hadoop operations, and data access patterns.
Google BigQuery is a cloud data warehouse and spreadsheet database that allows users to import, store, and query data in various formats like CSV, JSON, and Google Sheets. It provides a sandbox account with 10GB of free storage and 1TB of free queries per month. To use it, users create a BigQuery project, import data into datasets and tables, and then query the data using SQL syntax.
Apache kafka performance(throughput) - without data loss and guaranteeing dat...SANG WON PARK
Apache Kafak의 성능이 특정환경(데이터 유실일 발생하지 않고, 데이터 전송순서를 반드시 보장)에서 어느정도 제공하는지 확인하기 위한 테스트 결과 공유
데이터 전송순서를 보장하기 위해서는 Apache Kafka cluster로 partition을 분산할 수 없게되므로, 성능향상을 위한 장점을 사용하지 못하게 된다.
이번 테스트에서는 Apache Kafka의 단위 성능, 즉 partition 1개에 대한 성능만을 측정하게 된다.
향후, partition을 증가할 경우 본 테스트의 1개 partition 단위 성능을 기준으로 예측이 가능할 것 같다.
Beyond SQL: Speeding up Spark with DataFramesDatabricks
This document summarizes Spark SQL and DataFrames in Spark. It notes that Spark SQL is part of the core Spark distribution and allows running SQL and HiveQL queries. DataFrames provide a way to select, filter, aggregate and plot structured data like in R and Pandas. DataFrames allow writing less code through a high-level API and reading less data by using optimized formats and partitioning. The optimizer can optimize queries across functions and push down predicates to read less data. This allows creating and running Spark programs faster.
How Dashtable Helps Dragonfly Maintain Low LatencyScyllaDB
Dashtable is a hashtable implementation inside Dragonfly. It supports incremental resizes and fast, cache-friendly operations. In this talk, we will learn how Dashtable helps Dragonfly to keep its tail latency in check. In Dashtable, long-tail latencies have been reduced by a factor of 1000x, but P999 are 7x longer. Find out why we still think this is a good tradeoff.
GraphQL is a query language for APIs that allows clients to request specific data rather than entire resources. It addresses common problems with REST APIs like over-fetching data. Several GraphQL libraries exist for Java including graphql-java, graphql-java-kickstart, graphql-dgs-framework, and spring-graphql. These libraries differ in features like code generation, error handling, testing support, and available clients. Spring GraphQL is considered the most mature option as it integrates directly with Spring Boot and allows testing all code.
The document discusses GraphQL and Relay Modern. It begins with an overview of REST and some of its issues. It then covers what GraphQL is, how it was created at Facebook, and some of its key features like being version-free and having a hierarchical data structure. The document also discusses Relay Modern and what it provides for managing GraphQL queries and mutations. It includes examples of using GraphQL schemas and Relay to fetch and display basic and list data.
This document discusses GraphQL and Relay Modern. It begins with an overview of REST and its issues. It then covers what GraphQL is, how it was created at Facebook, and its motivations. Key aspects of GraphQL discussed include its single endpoint, hierarchical nature, strong typing, response mirroring the query, and introspection capabilities. The document also discusses Relay Modern and what it provides like declarative queries and mutations. It provides examples of simple GraphQL schemas and Relay components. In the end, it discusses subscriptions, routing, debugging, and resources for learning more.
GraphQL and its schema as a universal layer for database accessConnected Data World
GraphQL is a query language mostly used to streamline access to REST APIs. It is seeing tremendous growth and adoption, in organizations like Airbnb, Coursera, Docker, GitHub, Twitter, Uber, and Facebook, where it was invented.
As REST APIs are proliferating, the promise of accessing them all through a single query language and hub, which is what GraphQL and GraphQL server implementations bring, is alluring.
A significant recent addition to GraphQL was SDL, its schema definition language. SDL enables developers to define a schema governing interaction with the back-end that GraphQL servers can then implement and enforce.
Prisma is a productized version of the data layer leveraging GraphQL to access any database. Prisma works with MySQL, Postgres, and MongoDB, and is adding to this list.
Prisma sees the GraphQL community really coming together around the idea of schema-first development, and wants to use GraphQL SDL as the foundation for all interfaces between systems.
Neo4j Morpheus: Interweaving Table and Graph Data with SQL and Cypher in Apac...Databricks
This document discusses property graphs and how they are represented and queried using Morpheus, a graph query engine for Apache Spark.
Morpheus allows querying property graphs using Cypher and represents property graphs using DataFrames, with node and relationship data stored in tables. It integrates with various data sources and supports federated queries across multiple property graphs. The document provides examples of loading property graph data from sources like JSON, SQL databases and Neo4j, creating graph projections, running analytical queries, and recommending businesses based on graph algorithms.
The document discusses Prisma and GraphQL. It provides an overview of GraphQL concepts like schema, queries, and resolvers. It then covers the typical architecture of a GraphQL server including the schema definition, resolver functions, and server setup. Finally, it introduces Prisma as a database access layer that can be used to build GraphQL servers.
GraphQL is query language for APIs, but what are the advantages and how would one implement such in their microservices/APIs. In this session, I will go through the basics of GraphQL, different aspects of GraphQL and architecture of such APIs. How 4 different ways we can implement GraphQL for a Springboot microservice/API.
This document discusses GraphQL, a query language for APIs and a runtime for fulfilling queries with existing data. It provides examples of basic queries, nested fields, connections, arguments, and fragments. It also covers GraphQL types including scalars, schemas, definitions, predicates, and data resolution. GraphQL allows clients to define the structure of the data required, and specifies querying, modifying, and transmitting data between client and server.
Advanced Data Science with Apache Spark-(Reza Zadeh, Stanford)Spark Summit
The document discusses graph computations and Pregel, an API for graph processing. It introduces Pregel's vertex-centric programming model where computation is organized into supersteps and depends only on neighboring vertices. Examples like PageRank are shown implemented in Pregel. GraphX is also introduced as a library providing Pregel-like abstractions on Spark. The document then discusses distributing matrix computations, covering partitioning schemes for matrices and how to distribute operations like multiplication and singular value decomposition (SVD) across a cluster.
ScalaTo July 2019 - No more struggles with Apache Spark workloads in productionChetan Khatri
Scala Toronto July 2019 event at 500px.
Pure Functional API Integration
Apache Spark Internals tuning
Performance tuning
Query execution plan optimisation
Cats Effects for switching execution model runtime.
Discovery / experience with Monix, Scala Future.
GraphQL is query language for APIs, but what are the advantages and how would one implement such in their microservices/APIs. In this session, I will go through the basics of GraphQL, different aspects of GraphQL and architecture of such APIs. There will be a demo/live-coding on, how 4 different ways we can implement GraphQL for a Springboot microservice/API. Lots of examples, live coding and helpful comparison on structure, usage and implementations of GraphQL in Springboot & Java world.
Remember the last time you tried to write a MapReduce job (obviously something non trivial than a word count)? It sure did the work, but has lot of pain points from getting an idea to implement it in terms of map reduce. Did you wonder how life will be much simple if you had to code like doing collection operations and hence being transparent* to its distributed nature? Did you want/hope for more performant/low latency jobs? Well, seems like you are in luck.
In this talk, we will be covering a different way to do MapReduce kind of operations without being just limited to map and reduce, yes, we will be talking about Apache Spark. We will compare and contrast Spark programming model with Map Reduce. We will see where it shines, and why to use it, how to use it. We’ll be covering aspects like testability, maintainability, conciseness of the code, and some features like iterative processing, optional in-memory caching and others. We will see how Spark, being just a cluster computing engine, abstracts the underlying distributed storage, and cluster management aspects, giving us a uniform interface to consume/process/query the data. We will explore the basic abstraction of RDD which gives us so many awesome features making Apache Spark a very good choice for your big data applications. We will see this through some non trivial code examples.
Session at the IndicThreads.com Confence held in Pune, India on 27-28 Feb 2015
http://www.indicthreads.com
http://pune15.indicthreads.com
- Deepak Shevani works at Microsoft for the Azure group and recently became interested in GraphQL.
- GraphQL was created by Facebook in 2015 to improve client-server communication when rebuilding their native mobile apps. It defines a query language and provides a server-side runtime to fetch the desired data.
- GraphQL allows clients to get precisely the data they need through queries, reducing over- and under-fetching of data compared to REST APIs.
Cascading is a Java framework that allows users to define data processing workflows on Hadoop clusters more easily. The author discusses connecting Cascading to the Starfish profiler and optimizer to enable automated optimization of Cascading workflows. Key points are:
1) Cascading workflows are translated to DAGs of Hadoop jobs for profiling and optimization.
2) The Cascading API is modified to use the Hadoop New API to interface with Starfish.
3) Experiments show the Starfish optimizer providing speedups of up to 1.3x for several real-world Cascading workflows.
GraphQL is a query language for APIs that was created by Facebook in 2012. It allows clients to define the structure of the data required, and exactly the data they need from the server. This prevents over- and under-fetching of data. GraphQL has grown in popularity with the release of tools like Apollo and GraphQL code generation. GraphQL can be used to build APIs that integrate with existing backend systems and databases, with libraries like Express GraphQL and GraphQL Yoga making it simple to create GraphQL servers.
Code-first GraphQL Server Development with PrismaNikolas Burk
This document discusses code-first and SDL-first approaches to building GraphQL schemas and servers. It defines the terminology and compares the two approaches. Code-first involves programmatically defining types and resolvers, while SDL-first uses a string-based schema definition language. Both have tradeoffs like inconsistencies or lack of tooling for SDL-first, and lack of documentation for code-first. Prisma is introduced as a tool that can generate a GraphQL schema from a database using either approach. The document concludes with a demonstration of building a GraphQL server and schema with Prisma and Nexus using a code-first approach.
GraphQL across the stack: How everything fits togetherSashko Stubailo
My talk from GraphQL Summit 2017!
In this talk, I talk about a future for GraphQL which builds on the idea that GraphQL enables lots of tools to work together seamlessly across the stack. I present this through the lens of 3 examples: Caching, performance tracing, and schema stitching.
Stay tuned for the video recording from GraphQL Summit!
GraphQL is a syntax that describes how to ask for data, and is generally used to load data from a server to a client. GraphQL has three main characteristics:
It lets the client specify exactly what data it needs.
It makes it easier to aggregate data from multiple sources.
It uses a type system to describe data.
Advanced Techniques for Cyber Security Analysis and Anomaly DetectionBert Blevins
Cybersecurity is a major concern in today's connected digital world. Threats to organizations are constantly evolving and have the potential to compromise sensitive information, disrupt operations, and lead to significant financial losses. Traditional cybersecurity techniques often fall short against modern attackers. Therefore, advanced techniques for cyber security analysis and anomaly detection are essential for protecting digital assets. This blog explores these cutting-edge methods, providing a comprehensive overview of their application and importance.
Comparison Table of DiskWarrior Alternatives.pdfAndrey Yasko
To help you choose the best DiskWarrior alternative, we've compiled a comparison table summarizing the features, pros, cons, and pricing of six alternatives.
An invited talk given by Mark Billinghurst on Research Directions for Cross Reality Interfaces. This was given on July 2nd 2024 as part of the 2024 Summer School on Cross Reality in Hagenberg, Austria (July 1st - 7th)
The DealBook is our annual overview of the Ukrainian tech investment industry. This edition comprehensively covers the full year 2023 and the first deals of 2024.
Quality Patents: Patents That Stand the Test of TimeAurora Consulting
Is your patent a vanity piece of paper for your office wall? Or is it a reliable, defendable, assertable, property right? The difference is often quality.
Is your patent simply a transactional cost and a large pile of legal bills for your startup? Or is it a leverageable asset worthy of attracting precious investment dollars, worth its cost in multiples of valuation? The difference is often quality.
Is your patent application only good enough to get through the examination process? Or has it been crafted to stand the tests of time and varied audiences if you later need to assert that document against an infringer, find yourself litigating with it in an Article 3 Court at the hands of a judge and jury, God forbid, end up having to defend its validity at the PTAB, or even needing to use it to block pirated imports at the International Trade Commission? The difference is often quality.
Quality will be our focus for a good chunk of the remainder of this season. What goes into a quality patent, and where possible, how do you get it without breaking the bank?
** Episode Overview **
In this first episode of our quality series, Kristen Hansen and the panel discuss:
⦿ What do we mean when we say patent quality?
⦿ Why is patent quality important?
⦿ How to balance quality and budget
⦿ The importance of searching, continuations, and draftsperson domain expertise
⦿ Very practical tips, tricks, examples, and Kristen’s Musts for drafting quality applications
https://www.aurorapatents.com/patently-strategic-podcast.html
Implementations of Fused Deposition Modeling in real worldEmerging Tech
The presentation showcases the diverse real-world applications of Fused Deposition Modeling (FDM) across multiple industries:
1. **Manufacturing**: FDM is utilized in manufacturing for rapid prototyping, creating custom tools and fixtures, and producing functional end-use parts. Companies leverage its cost-effectiveness and flexibility to streamline production processes.
2. **Medical**: In the medical field, FDM is used to create patient-specific anatomical models, surgical guides, and prosthetics. Its ability to produce precise and biocompatible parts supports advancements in personalized healthcare solutions.
3. **Education**: FDM plays a crucial role in education by enabling students to learn about design and engineering through hands-on 3D printing projects. It promotes innovation and practical skill development in STEM disciplines.
4. **Science**: Researchers use FDM to prototype equipment for scientific experiments, build custom laboratory tools, and create models for visualization and testing purposes. It facilitates rapid iteration and customization in scientific endeavors.
5. **Automotive**: Automotive manufacturers employ FDM for prototyping vehicle components, tooling for assembly lines, and customized parts. It speeds up the design validation process and enhances efficiency in automotive engineering.
6. **Consumer Electronics**: FDM is utilized in consumer electronics for designing and prototyping product enclosures, casings, and internal components. It enables rapid iteration and customization to meet evolving consumer demands.
7. **Robotics**: Robotics engineers leverage FDM to prototype robot parts, create lightweight and durable components, and customize robot designs for specific applications. It supports innovation and optimization in robotic systems.
8. **Aerospace**: In aerospace, FDM is used to manufacture lightweight parts, complex geometries, and prototypes of aircraft components. It contributes to cost reduction, faster production cycles, and weight savings in aerospace engineering.
9. **Architecture**: Architects utilize FDM for creating detailed architectural models, prototypes of building components, and intricate designs. It aids in visualizing concepts, testing structural integrity, and communicating design ideas effectively.
Each industry example demonstrates how FDM enhances innovation, accelerates product development, and addresses specific challenges through advanced manufacturing capabilities.
Are you interested in dipping your toes in the cloud native observability waters, but as an engineer you are not sure where to get started with tracing problems through your microservices and application landscapes on Kubernetes? Then this is the session for you, where we take you on your first steps in an active open-source project that offers a buffet of languages, challenges, and opportunities for getting started with telemetry data.
The project is called openTelemetry, but before diving into the specifics, we’ll start with de-mystifying key concepts and terms such as observability, telemetry, instrumentation, cardinality, percentile to lay a foundation. After understanding the nuts and bolts of observability and distributed traces, we’ll explore the openTelemetry community; its Special Interest Groups (SIGs), repositories, and how to become not only an end-user, but possibly a contributor.We will wrap up with an overview of the components in this project, such as the Collector, the OpenTelemetry protocol (OTLP), its APIs, and its SDKs.
Attendees will leave with an understanding of key observability concepts, become grounded in distributed tracing terminology, be aware of the components of openTelemetry, and know how to take their first steps to an open-source contribution!
Key Takeaways: Open source, vendor neutral instrumentation is an exciting new reality as the industry standardizes on openTelemetry for observability. OpenTelemetry is on a mission to enable effective observability by making high-quality, portable telemetry ubiquitous. The world of observability and monitoring today has a steep learning curve and in order to achieve ubiquity, the project would benefit from growing our contributor community.
Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Em...Erasmo Purificato
Slide of the tutorial entitled "Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Emerging Trends" held at UMAP'24: 32nd ACM Conference on User Modeling, Adaptation and Personalization (July 1, 2024 | Cagliari, Italy)
Mitigating the Impact of State Management in Cloud Stream Processing SystemsScyllaDB
Stream processing is a crucial component of modern data infrastructure, but constructing an efficient and scalable stream processing system can be challenging. Decoupling compute and storage architecture has emerged as an effective solution to these challenges, but it can introduce high latency issues, especially when dealing with complex continuous queries that necessitate managing extra-large internal states.
In this talk, we focus on addressing the high latency issues associated with S3 storage in stream processing systems that employ a decoupled compute and storage architecture. We delve into the root causes of latency in this context and explore various techniques to minimize the impact of S3 latency on stream processing performance. Our proposed approach is to implement a tiered storage mechanism that leverages a blend of high-performance and low-cost storage tiers to reduce data movement between the compute and storage layers while maintaining efficient processing.
Throughout the talk, we will present experimental results that demonstrate the effectiveness of our approach in mitigating the impact of S3 latency on stream processing. By the end of the talk, attendees will have gained insights into how to optimize their stream processing systems for reduced latency and improved cost-efficiency.
RPA In Healthcare Benefits, Use Case, Trend And Challenges 2024.pptxSynapseIndia
Your comprehensive guide to RPA in healthcare for 2024. Explore the benefits, use cases, and emerging trends of robotic process automation. Understand the challenges and prepare for the future of healthcare automation
Best Practices for Effectively Running dbt in Airflow.pdfTatiana Al-Chueyr
As a popular open-source library for analytics engineering, dbt is often used in combination with Airflow. Orchestrating and executing dbt models as DAGs ensures an additional layer of control over tasks, observability, and provides a reliable, scalable environment to run dbt models.
This webinar will cover a step-by-step guide to Cosmos, an open source package from Astronomer that helps you easily run your dbt Core projects as Airflow DAGs and Task Groups, all with just a few lines of code. We’ll walk through:
- Standard ways of running dbt (and when to utilize other methods)
- How Cosmos can be used to run and visualize your dbt projects in Airflow
- Common challenges and how to address them, including performance, dependency conflicts, and more
- How running dbt projects in Airflow helps with cost optimization
Webinar given on 9 July 2024
Sustainability requires ingenuity and stewardship. Did you know Pigging Solutions pigging systems help you achieve your sustainable manufacturing goals AND provide rapid return on investment.
How? Our systems recover over 99% of product in transfer piping. Recovering trapped product from transfer lines that would otherwise become flush-waste, means you can increase batch yields and eliminate flush waste. From raw materials to finished product, if you can pump it, we can pig it.
TrustArc Webinar - 2024 Data Privacy Trends: A Mid-Year Check-InTrustArc
Six months into 2024, and it is clear the privacy ecosystem takes no days off!! Regulators continue to implement and enforce new regulations, businesses strive to meet requirements, and technology advances like AI have privacy professionals scratching their heads about managing risk.
What can we learn about the first six months of data privacy trends and events in 2024? How should this inform your privacy program management for the rest of the year?
Join TrustArc, Goodwin, and Snyk privacy experts as they discuss the changes we’ve seen in the first half of 2024 and gain insight into the concrete, actionable steps you can take to up-level your privacy program in the second half of the year.
This webinar will review:
- Key changes to privacy regulations in 2024
- Key themes in privacy and data governance in 2024
- How to maximize your privacy program in the second half of 2024
How RPA Help in the Transportation and Logistics Industry.pptxSynapseIndia
Revolutionize your transportation processes with our cutting-edge RPA software. Automate repetitive tasks, reduce costs, and enhance efficiency in the logistics sector with our advanced solutions.
Coordinate Systems in FME 101 - Webinar SlidesSafe Software
If you’ve ever had to analyze a map or GPS data, chances are you’ve encountered and even worked with coordinate systems. As historical data continually updates through GPS, understanding coordinate systems is increasingly crucial. However, not everyone knows why they exist or how to effectively use them for data-driven insights.
During this webinar, you’ll learn exactly what coordinate systems are and how you can use FME to maintain and transform your data’s coordinate systems in an easy-to-digest way, accurately representing the geographical space that it exists within. During this webinar, you will have the chance to:
- Enhance Your Understanding: Gain a clear overview of what coordinate systems are and their value
- Learn Practical Applications: Why we need datams and projections, plus units between coordinate systems
- Maximize with FME: Understand how FME handles coordinate systems, including a brief summary of the 3 main reprojectors
- Custom Coordinate Systems: Learn how to work with FME and coordinate systems beyond what is natively supported
- Look Ahead: Gain insights into where FME is headed with coordinate systems in the future
Don’t miss the opportunity to improve the value you receive from your coordinate system data, ultimately allowing you to streamline your data analysis and maximize your time. See you there!
2. James tan
@ Okestra Systems Sdn Bhd | https://okestra.co/ | james@okestra.co | 0123221812
2
Workflow management simplified
Let’s make GO fun!
With AI (soon)
3. What is GraphQL?
GraphQL is a query language for APIs and a runtime for fulfilling those queries
with your existing data.
3
7. Multiple Round
Trip
7
● fetch /pages/1
● fetch /comments?pageID=1
● fetch /users/2
query page($id: Int) {
page(ID: $id) {
title
description
comments {
id
message
created_by {
id
name
}
}
}
}
REST:
GraphQL:
8. Multiple Round
Trip
8
● fetch /pages/1
● fetch /comments?pageID=1
● fetch /users/2
query page($id: Int) {
page(ID: $id) {
title
description
comments {
id
message
created_by {
id
name
}
}
}
announcements() {
edges {
node: {
id
title
message
}
}
}
REST:
GraphQL:
9. Over fetching data
9
● Frontend: fetch /pages
● Admin: fetch /pages
Frontend:
Admin:
title date
description
title date
description
...
ID title Created by Created on Published
1 Title here ... ... Yes
2 comments
5 comments
[
{ id: 1, title: “title”,
description: “...”,
created_at: “...”,
created_by: { name: “...” },
Comments_count: 2
}, …
]
REST Frontend & admin:
10. Over fetching data
10
Frontend:
Admin:
title date
description
title date
description
...
ID title Created by Created on Published
1 Title here ... ... Yes
2 comments
5 comments
query page($id: Int) {
page(ID: $id) {
title
description
Comment_count
…
}}
GraphQL frontend:
query page($id: Int) {
page(ID: $id) {
Title
…
}}
GraphQL admin:
11. Schema Documentation
11
Allow API Discovery and exploration
REST:
schema {
query: Query
mutation: Mutation
}
type Query {
user(email: String!): User
users(first: Int, after: String): UsersConnection!
}
type Mutation {
createUser(email: String!, password: String!): User
}
12. Implementing GraphQL
● graphql-go: An implementation of GraphQL for Go / Golang.
● graph-gophers/graphql-go: An active implementation of GraphQL in Golang (was
https://github.com/graph-gophers/graphql-go).
● GQLGen - Go generate based graphql server library.
● graphql-relay-go: A Go/Golang library to help construct a graphql-go server supporting
react-relay.
● machinebox/graphql: An elegant low-level HTTP client for GraphQL.
● samsarahq/thunder: A GraphQL implementation with easy schema building, live queries,
and batching
12
https://graphql.org/code/#go
13. Graphql-go
● minimal API
● support for context.Context
● support for the OpenTracing standard
● schema type-checking against resolvers
● resolvers are matched to the schema based on method sets (can resolve a GraphQL schema with a Go
interface or Go struct).
● handles panics in resolvers
● parallel execution of resolvers
● Subscriptions
https://github.com/graph-gophers/graphql-go
13
24. What is DGraph?
24
Dgraph is a distributed, low-latency, high throughput graph database, written in Go. It puts a lot of emphasis on good
design, concurrency and minimizing network calls required to execute a query in a distributed environment.
Why build Dgraph?
We think graph databases are currently second class citizens. They are not considered mature enough to be run as the
sole database, and get run alongside other SQL/NoSQL databases. Also, we’re not happy with the design decisions of
existing graph databases, which are either non-native or non-distributed, don’t manage underlying data or suffer from
performance issues.
Is Dgraph production ready?
We recommend Dgraph to be used in production at companies.
https://dgraph.io/
26. What is Graph database?
26
What is Graph?
A graph is composed of two elements: a node and a relationship.
Each node represents an entity (a person, place, thing, category or other piece of data), and each relationship
represents how two nodes are associated. This general-purpose structure allows you to model all kinds of
scenarios – from a system of roads, to a network of devices, to a population’s medical history or anything else
defined by relationships.
source: https://neo4j.com/why-graph-databases/
27. What is Graph database?
27
A graph database is an online database management system with Create, Read, Update and Delete (CRUD)
operations working on a graph data model.
source: https://neo4j.com/why-graph-databases/
29. Performance
Example SQL with “Join”s
SELECT 'paper_review'::text AS type, paper_reviews.id, paper_reviews.user_id, concat_ws(' '::text, users.first_name, users.last_name) AS
user_full_name, users.title AS user_title, users.photo AS user_photo, users.organization AS user_organization, users.username,
paper_reviews.paper_id, NULL::character varying AS language_framework, NULL::text AS performance_description, paper_reviews.score_tech,
paper_reviews.score_original,
( SELECT count(comments.id) AS count FROM public.comments WHERE ((comments.source_id = paper_reviews.id) AND (comments.vote = 1))) AS
likes_count,
( SELECT count(comments.id) AS count FROM public.comments WHERE ((comments.source_id = paper_reviews.id) AND (comments.vote = 0))) AS
comments_count FROM ((public.paper_reviews LEFT JOIN public.papers ON ((paper_reviews.paper_id = papers.id))) LEFT JOIN public.users ON
((paper_reviews.user_id = users.id)))
UNION SELECT 'paper_code'::text AS type, paper_codes.id, paper_codes.user_id, concat_ws(' '::text, users.first_name, users.last_name) AS
user_full_name, users.title AS user_title, ...
Why Graph database?
29
31. Flexibility
SQL based:
More joins!, more indexing, more optimisations, more group by OR more sub select, or refactor
NoSQL based:
More round trip for additional collection, more indexing
GraphDB
Add relation, add to graphql query, more indexing (for non relation field)
Why Graph database?
31
33. DGraph Setup
● Zero node: controls the Dgraph cluster, assigns servers to a group and re-balances data between server
groups
● Alpha node: hosts predicates and indexes
● Ratel: UI to run queries, mutations & altering schema (alike pgadmin)
https://docs.dgraph.io/get-started/
https://tour.dgraph.io/intro/1/
https://gist.github.com/u007/f47f9e42ae4f7a94b29c542955d698bc - docker-compose example
33