As businesses grow, so does the complexity of their software. New features, new models, and new background processes all continue to be added. . .and developers struggle to make sense of it all. Yet the end user demands a swift and functional experience when interacting with your application. It is paramount to be open to alternative patterns that help tame complex, high-demand services. Two such patterns are command-query responsibility segregation (CQRS) and event sourcing (ES).
Command-query responsibility segregation is an architectural pattern for user-facing applications that extends from the now standard Model-View-Controller (MVC) pattern and is an alternative to the CRUD pattern. At its core, CQRS is about changing how we think of and work with our data by introducing two types of models: all user actions become commands, and a read-only query model powers our views. Commands and queries are logistically separated, providing additional decoupling of our application. CQRS also calls for changes in how we store and structure our data.
Enter event sourcing. Instead of persisting the current state of our domain objects or entities, we record historical events about our data. The key advantage is that we can examine our application data at any point in time, rather than just the current state. This pattern changes how we persist and process our data but is surprisingly efficient.
While each of the two patterns can be used exclusively, they complement each other beautifully and facilitate the construction of decoupled, scalable applications or individual services. Stephen Pember explores the fundamentals of each pattern and offers several examples and demonstration code to show how one might actually go about implementing CQRS and ES. Steve discusses task-based UIs and domain-driven design as he outlines some of the advantages—and challenges—that ThirdChannel has seen when developing systems using CQRS and ES over the past year.
The document discusses which database to use for different situations. It begins with explaining why a relational database may not be suitable for all problems and then describes different database categories including key-value stores, column family databases, document databases, graph databases, and Hadoop. It notes the characteristics and uses of each database type. The document concludes that the choice depends on factors like data structure, scalability needs, and workload.
This document discusses how DevOps practices can help organizations accelerate innovation through software delivery. It outlines the core components of DevOps as self-service, automation, and collaboration. It then walks through the software development lifecycle from developing to operating software. Throughout, it emphasizes practices like continuous delivery, automated testing and configuration, zero-downtime deployments, monitoring, and designing systems to withstand failures using techniques like Netflix's "Chaos Monkey" which intentionally causes failures to test resiliency. The overall message is that DevOps can help organizations innovate faster by breaking down silos and automating the process of delivering high quality software.
I gave this presentation at the Sydney Continuous Delivery Meetup Group. The main goal was to talk about Performance Metrics that you should monitor along the pipeline. I examples in 4 different areas where deployments failed and how metrics would have helped preventing these problems
Top .NET, Java & Web Performance Mistakes - Meetup Jan 2015Andreas Grabner
Top .NET, Java & Web Performance Problems. Why these apps failed, how to avoid it and which metrics to look at, e.g: # of Busy vs. Idle Worker Threads, Connection Pool Acquisition Time, # Exceptions Thrown, ...
BTD2015 - Your Place In DevTOps is Finding Solutions - Not Just Bugs!Andreas Grabner
This is about leveling-up and REVOLUTIONIZING Testing as part of your Agile/DevOps Transformation.
You can contribute more than testing functionality. You need to Level-Up your skill set by understanding the apps you are testing. # Images, # JS Files, # SQL Statements, Connection Pool Utilization and Garbage Collection Activity have to be added to your portfolio.
Check these metrics when you do your functional testing and report regressions to your engineers even though the functionality is still good. But you just uncovered an Architectural regression that will lead to a scalabilty and performance problem.
Finding these problems early will eliminate a lot of wasted and unplanned time later on in the lifecycle. that is your contribution to delivering software faster with better quality
Mobile User Experience:Auto Drive through Performance MetricsAndreas Grabner
Believe it or not - 85% of mobile apps are removed after first usage! In this presentation - given at the APM Meetup in Singapore in April 2015 - I talked about the challenges, best practices and especially metrics to avoid this situation.
Key Points of the Presentation
The two key trends "Internet of Things" and "DevOps" play a big role in our life when we talk about User Experience and especially mobile user experience. In this presentation I tell you what metrics to use to make sure you deliver your ideas faster to your mobile end users but also ensuring the right quality and user experience so that your users stay loyal and dont delete the mobile app after first usage.
The basics you need to know to get up and running with Chaos Monkey in your Amazon Web Service's Cloud enviornment.
Links:
CloudFormation Template:
https://github.com/joehack3r/aws/blob/master/cloudformation/templates/chaosMonkey.json
Simian Army Quick Start Guide:
https://github.com/Netflix/SimianArmy/wiki/Quick-Start-Guide
Chaos Monkey Configuration:
https://github.com/Netflix/SimianArmy/wiki/Chaos-Settings
Chaos Monkey Army:
https://github.com/Netflix/SimianArmy/wiki/The-Chaos-Monkey-Army
Finding Anything: Real-time Search with IndexTankYogiWanKenobi
IndexTank provides a real-time search service that allows developers to add powerful search capabilities to their applications without having to implement it themselves. It allows for features like real-time indexing and searching, custom scoring, autocomplete, faceting, geo search, and advanced text search. The speaker discusses how he used IndexTank to build search features for two sample applications in a short period of time with little effort. He also highlights reddit.com as a case study of a large site that uses IndexTank to improve its search capabilities.
Benchmarking: You're Doing It Wrong (StrangeLoop 2014)Aysylu Greenberg
Knowledge of how to set up good benchmarks is invaluable in understanding performance of the system. Writing correct and useful benchmarks is hard, and verification of the results is difficult and prone to errors. When done right, benchmarks guide teams to improve the performance of their systems. When done wrong, hours of effort may result in a worse performing application, upset customers or worse! In this talk, we will discuss what you need to know to write better benchmarks. We will look at examples of bad benchmarks and learn about what biases can invalidate the measurements, in the hope of correctly applying our new-found skills and avoiding such pitfalls in the future.
This presentation was given as part of a Dynatrace Lunch & Learn event. APM (=Application Performance Management) allows us to transform the way we develop, deploy and run software.
Here are some ideas how APM can be (r)evolutionized
Reactive programming and Hystrix fault tolerance by Max MyslyvtsevJavaDayUA
Reactive programming is a new paradigm that provides asynchronous event-based flow control. Java implementation is called rxJava and is being developed by Netfix. They have also released Hystrix — a non-functional layer that provides fault tolerance and latency features which also exposes reactive API.
React has a different design than other frameworks and will influence front end Javascript frameworks for years.
I'll give an overview of what sets it apart and walk through some example code to get you up and running with React.
This is from an invited talk I gave at the Pittsburgh Perl Workshop a few years back. It's not often that I get a chance to talk to developers, so I thought I'd take advantage of it and yell at them a bit ;-)
This document provides tips for improving email communication and reducing unnecessary emails. It suggests strengthening email subjects using an Action, Summary, Background, Close format. Emails should be kept concise by chunking key points, eliminating attachments, and adding next steps. Reply-All and carbon copy should be used appropriately as they can lead to more emails. Sending fewer, more targeted emails that are timely, relevant, complete, compliant and professional will improve communication.
Optimizing Git LFS Migration Through Repository Data-miningAtlassian
This document discusses optimizing migration to Git LFS (Large File Storage) by analyzing repository data. It recommends setting a Git LFS file size cutoff between 100KB and 1000KB based on metrics like clone size increase over time, number of files tracked, and workspace size without large files. The analysis found that a cutoff below 800KB produced the best results for these metrics when migrating a large 75GB repository to Git LFS.
Slides from my talk at QCon New York on how Netflix increases resiliency through failure, covering the Chaos Monkey, Chaos Gorilla, Latency Monkey, and others from the Simian Army.
This document discusses scaling a web application, particularly those built with PHP and MySQL. It begins with introductions and then outlines various strategies for scaling applications and databases. For applications, it recommends profiling code and queries to identify bottlenecks, optimizing frameworks, caching, and monitoring. For databases, it suggests technologies like Memcached, database replication using master-slave, sharding, MySQL Cluster, and storage engines. The overall message is that scaling requires understanding applications and systems, identifying pain points, and having a plan to optimize performance as needs grow.
The speaker discusses their many failures in technology careers over 12 years. Some key lessons are that failure is inevitable, it's okay not to be a "success story", no matter how hard you work failures can still happen, it's important to ask for help when you don't know what you're doing, own your mistakes, and have a plan for when failures occur so you can learn from them and try again. The speaker provides examples of failures with companies going under, building unsuccessful products, data migrations going wrong, but in each case emphasizes learning from mistakes and having contingencies to remedy problems.
Agile: the Good, the Bad and the Ugly - Webinar by Clarke Ching Agile - Septe...MARRIS Consulting
Webinar by Clarke Ching Agile and ToC expert. Agile: the Good, the Bad and the Ugly. If your Agile is broken then this is how to fix it!
Your Agile teams are busy. Busy delivering. Busy improving. Your quality is amazing. Rework is low. The product looks great. Your users love it. You are a high performing team!
But your internal customers say your teams are slow. This session will teach you how to use the Theory of Constraints to figure out how to speed up, by finding the one thing that’s slowing them down.
This webinar will cover how, in an Agile environment:
- to better control scope creep,
- to reinforce your relationship with the I.T. Development team’s client,
- to be able to make commitments and honour them and
- to decide where your bottleneck should be.
About the speaker
Clarke Ching is a computer scientist with an MBA who discovered Goldratt’s Theory of Constraints (ToC) in 2003 and has been using it ever since to accelerate Agile initiatives. He is fascinated by Agile and obsessed with ToC.
He wrote the amazon best-sellers Rolling Rocks Downhill and The Bottleneck Rules. Rolling Rocks Downhill teaches 3 things: the fundamentals of Agile combined with ToC; how to use those fundamentals to deliver big projects faster and on time; and how to deliver quietly huge transformations. It’s been featured in The Guardian newspaper and The Spectator magazine. It was one of Barbara Oakley’s top 10 books of 2019. It was the #2 best-selling Leadership book on amazon.com, just behind Steven Covey’s 7-habits book.
He has been Agile / Lean / ToC expert in: GE Energy, Dell, Royal London (life insurance & pensions), Gazprom and Standard Life Aberdeen among other organizations. He is the past Chairperson of Agile Scotland. He is a lecturer at Victoria University School Of Management in New Zealand where he now lives.
Today he is the founder and Chief Productivity Officer of Odd Socks Consulting
Talk - Essential Big-O for the DevOps engineer: quantifying scalability of software systems"?
Big-O analysis is the workhorse of scalability analysis of software systems, but many people don't appreciate its importance in DevOps. This talk will introduce the essential intuitions and techniques behind Big-O analysis, with the goal of facilitating more precise communications between DevOps and software engineers who speak in Big-O terms, and applying the principles directly to DevOps scalability problems such as networking.
Gr8conf US 2015 - Intro to Event Sourcing with GroovySteve Pember
As Grails developers we generally build systems where our data is directly linked to the current state of our application; one row in the database equates to one entity’s current state. Only ever knowing the current state of the data is adequate for many systems, but imagine the possibilities if one had access to the state of the data at any point in time.
Enter Event Sourcing: instead of persisting the current state of our Domain Objects, we store historical events about our data. This pattern changes how we store and process our data, but is surprisingly lightweight and performant. In this talk I will present the basic concepts of Event Sourcing and the positive effects it can have on analytics and performance. We’ll see how naturally it couples with the Event-oriented world of modern Reactive systems, and how easily it can be implemented in Groovy. We’ll examine some practical use cases and example implementations. Event Sourcing will change how you think about your data.
Nondeterministic Software for the Rest of UsTomer Gabel
A talk given at GeeCON 2018 in Krakow, Poland.
Classically-trained (if you can call it that) software engineers are used to clear problem statements and clear success and acceptance criteria. Need a mobile front-end for your blog? Sure! Support instant messaging for a million concurrent users? No problem! Store and serve 50TB of JSON blobs? Presto!
Unfortunately, it turns out modern software often includes challenges that we have a hard time with: those without clear criteria for correctness, no easy way to measure performance and success is about more than green dashboards. Your blog platform better have a spam filter, your instant messaging service has to have search, and your blobs will inevitably be fed into some data scientist's crazy contraption.
In this talk I'll share my experiences of learning to deal with non-deterministic problems, what made the process easier for me and what I've learned along the way. With any luck, you'll have an easier time of it!
The document discusses various data science applications at Bol.com including measuring user interactions on the website, forecasting product demand, and building recommendation systems. It provides examples and details for each application. For measuring, it notes Bol is able to process user event data with a 1-2 second lag compared to 25-30 seconds for another company. For recommendations, it highlights improvements from moving the service to the cloud including faster response times and being able to generate new predictions in 30 minutes instead of 24 hours. For forecasting demand, it outlines the process and techniques used including starting small, experimenting fast, and scaling up over time using various machine learning models and cloud technologies.
The document discusses various topics related to surviving in a microservices environment. It begins by outlining some benefits of microservices such as reduced coupling, continuous delivery, and efficient scaling. It then covers infrastructure topics like managing logs, metrics, deployments, builds, and environments. Architecture topics discussed include overall design, technologies, testing approaches, communication methods, and data persistence. The document also addresses team communication and processes. It concludes by providing some miscellaneous advice for working with microservices.
Surviving in a Microservices environment -abridgedSteve Pember
Many presentations on Microservices offer a high-level view; rarely does one hear what it’s like to work in such an environment. Individual services are somewhat trivial to develop, but now you suddenly have countless others to track. You’ll become obsessed over how they communicate. You’ll have to start referring to the whole thing as “the Platform”. You will have to take on some considerable DevOps work and start learning about deployment pipelines, metrics, and logging.
Don’t panic. In this presentation we’ll discuss what we learned over the past four years by highlighting our mistakes. We’ll examine what a development lifecycle might look like for adding a new service, developing a feature, or fixing bugs. We’ll see how team communication is more important than one might realize. Most importantly, we’ll show how - while an individual service is simple - the infrastructure demands are now much more complicated: your organization will need to introduce and become increasingly dependent on various technologies, procedures, and tools - ranging from the ELK stack to Grafana to Kubernetes. Lastly, you’ll come away with the understanding that your resident SREs will become the most valued members of your team.
PlayStation and Lucene - Indexing 1M documents per second: Presented by Alexa...Lucidworks
The document discusses search capabilities for the PlayStation Network. It describes how the initial system indexed 200,000 documents per second for the PS Store, but more capabilities were needed. It then details the challenges in moving from a relational database to NoSQL to support indexing 1 million documents per second across multiple services for 65 million monthly active users.
Leveraging Analytics In Gaming - Tiny Mogul GamesInMobi
'Analytics In Gaming' and how you can use it to improve the game's acquisition, retention and engagement' by Rajdeep Gumaste, Product Manager - Tiny Mogul Games.
Troublefree troubleshooting ian campbell sps jhb 2019Ian Campbell
The document provides instructions for attendees of an event hosted by SPS Events in Johannesburg, South Africa. It notes that session schedules may not be printed for all attendees and can be found by session room doors or online. Attendees are asked to provide feedback on sessions and to stay for the prize giving at the end of the day. They are also encouraged to interact with sponsors and speakers, take selfies with speakers to enter a photo competition, and share their learnings on social media using the #SPSJHB hashtag.
[DSC Europe 23] Vladislav Belov - ChatBot Learning Assistant with Large Langu...DataScienceConferenc1
Join this talk as I reveal the fascinating journey of creating a cutting-edge bot assistant powered by Large Language Models, paving the way from POC to MVP. Prepare to dive deeper into: A unique multi-route architecture that forms the backbone of our bot's performance., Ingenious solutions to combat latency challenges for a seamless user experience., Techniques to tackle out-of-distribution questions and enhance the bot's adaptability. Don't miss the opportunity to discover valuable insights that could elevate your own bot development process. See you there!
Presentation at my company to all the Interns about What DevOps is to me and why I'm passionate about it.
NOTE: Liberally gathered stuffs from the internetz. If I did something wrong by doing so or by you let's chat. I want to work with you to make it better :)
Building a strong Tableau community is important for successful deployment. It provides formalized definitions and knowledge, engagement, and a place to get questions answered. To build a community, organizations should use existing internal forums and wikis to provide documentation like data sources and dictionaries. The platform already used in an organization should be utilized to ensure flexibility. Regular "Tableau Doctor" sessions, either in-person or remote, can help many users per week. Communities need roles like the Tableau Doctor to answer questions and the Tableau Workflow Developer Community to help with APIs.
This document provides an overview of a training on using the Taskman tool to manage tasks and projects. It begins with discussing motivation for using Taskman, including avoiding chaos with tasks in emails and spreadsheets, better prioritizing work, and increasing transparency. It then covers agile methodologies like Scrum and Kanban used at EEA. Key policies for EEA's workflow process include limiting work in progress, fast feedback loops, and focusing on finishing work. Using these policies helped reduce lead times on projects significantly. The document concludes with reviewing Taskman roles and permissions.
Within 5 days, the document describes how to:
1. Apply the Design Sprint methodology to quickly test ideas and identify which to pursue, 2. Use storyboarding and paper prototyping to make ideas more concrete, 3. Obtain feedback from real users on a digital prototype to determine if the initial idea is good and worth developing further. The process enables identifying the right solution to a problem with limited resources in a short timeframe.
[AIIM17] It’s Harvest Time in the Information Garden - Dan AntionAIIM International
We’ve been collecting information for many years, driven by the usual suspects: compliance and fear. Now it’s time to take advantage of the information we’ve gathered by shifting our focus from the people who felt they had to keep it to the people who can actually use it. In short, it’s time to reap the benefits of the hard work we have already done. Learn how American Nuclear Insurers is using their information today, the process that got them there, and the technology it took to make it happen.
Learn about the current state of Information Management in AIIM’s latest report: http://info.aiim.org/2017-state-of-information-management
Surviving in a microservices environmentSteve Pember
Many presentations on Microservices offer a high level view; rarely does one hear what it’s like to work in such an environment. Individual services are somewhat trivial to develop, but now you suddenly have countless others to track. You’ll become obsessed over how they communicate. You’ll have to start referring to the whole thing as “the Platform”. You will have to take on some DevOps work and start learning about deployment pipelines, metrics, and logging.
Don’t panic. In this presentation we’ll discuss what we learned over the past three years. We’ll examine what a development lifecycle might look like for adding a new service, developing a feature, or fixing bugs. We’ll dive a bit into DevOps and see how one will become dependent on various metric and centralized logging tools, like Kubernetes and the ELK stack. Finally we’ll talk about team communication and organization... and how they are likely the most important tool for surviving a Microservices development team.
Sometimes, they just don’t get it.
We’re just trying to do the right thing here. Isn’t our success dependent on our users being able to shop, buy, apply or contact us through our web site or app? So if we’re dependent on our users, shouldn’t we at least involve them somehow in the design process?
Not so easy.
For some of “those” people, design is easy. Don’t we already know what the problem is and what design we can use to fix it? Can’t we just leverage best practices? Why do we even need to test the design if we’re experts? No one ever says these things, right?
In the real world, user-centered design and usability is ironically, not that easy to adapt. It’s counterintuitive because it’s such hard work to make things easy. What we have to do is to make what we do easy to understand and easy to choose. This session may not change your reality, but by sharing in some lessons learned, hopefully you’ll have the tools to help change some minds.
Coaching teams in creative problem solvingFlowa Oy
Agile has helped teams to collaborate and organize work better. That’s great. Better teamwork and better understanding of the work definitely helps a team to do right things. Agile has also lead the way toward technical practices such as Continuous Integration and Delivery, Test Driven Development and SOLID-architecture principles. Great, these things definitely help the team to do things right.
Then again, most of the time in software projects goes into problem solving and similar creative acts. Agile has relatively little to give on these areas. Currently, agile is not about creativity nor is it about problem solving.
This coaching circle session will focus on the creative core of software development: solving creatively novel, original and broad problems more effectively all the time. I will introduce some principles and tools I’ve found useful when helping people to solve hard problems and to find creative solutions.
Similar to Reactive Streams and the Wide World of Groovy (20)
Spring I_O 2024 - Flexible Spring with Event Sourcing.pptxSteve Pember
Event Sourcing is a modern but non-trivial data model for building scalable and powerful systems. Instead of mapping a single Entity to a single row in a datastore, in an Event Sourced system we persist all changes for an Entity in an append-only journal. This design provides a wealth of benefits: a built-in Audit Trail, Time-Based reporting, powerful Error Recovery, and more. It creates flexible, scalable systems and can easily evolve to meet changing organizational demands. That is, once you have some experience with it. Event Sourcing is straightforward in concept, but it does bring additional complexity and a learning curve that can be intimidating. People coming from traditional ORM systems often wonder: how does one model relations between Entities? How is Optimistic Locking handled? What about datastore constraints?
Based on over eight years of experience with building ES systems in Spring applications, we will demonstrate the basics of Event Sourcing and some of the common patterns. We will see how simple it can be to model events with available tools like Spring Data JPA, JOOQ, and the integration between Spring and Axon. We’ll walk through sample code in an application that demonstrates many of these techniques. However, it’s also not strictly about the code; we’ll see how a process called Event Modeling can be a powerful design tool to align Subject Matter Experts, Product, and Engineering. Attendees will leave with an understanding of the basic Event Sourcing patterns, and hopefully a desire to start creating their own Journals.
Anatomy of a Spring Boot App with Clean Architecture - Spring I/O 2023Steve Pember
In this presentation we will present the general philosophy of Clean Architecture, Hexagonal Architecture, and Ports & Adapters: discussing why these approaches are useful and general guidelines for introducing them to your code. Chiefly, we will show how to implement these patterns within your Spring (Boot) Applications. Through a publicly available reference app, we will demonstrate what these concepts can look like within Spring and walkthrough a handful of scenarios: isolating core business logic, ease of testing, and adding a new feature or two.
SACon 2019 - Surviving in a Microservices EnvironmentSteve Pember
Many presentations on microservices offer a high-level view of the architecture; rarely do you hear what it’s like to work in such an environment. Stephen Pember shares his experience migrating from a monolith to microservices across several companies, highlighting the mistakes made along the way and offering advice.
Over the past few years, Gradle has become a popular build tool in the JVM space. This is not surprising, considering the power and the features it brings, compared with its competitors. However, one thing Gradle lacks is history and the collective knowledge at the same level of other alternatives: how does one organize a Gradle project in an ‘idiomatic’ fashion?
We feel that we’ve put together a decent build pipeline for each of our microservices over the years, and each one starts with their build.gradle file(s). We’d like to share it, although we’re not sure if it’s the ‘correct’ way.
In this talk, we’ll walk through a sample project structure and build process. We’ll discuss the various checks and tools we use (e.g. Sonar, CodeNarc, Jenkins) at each step of the build. We’ll explain how each of the components in the process work for us, and share samples of our Groovy scripts. Most importantly, though, we’d like to hear what the audience are using in their builds!
Reactive applications & reactive programming result in flexible, concise, performant code and are a superior alternative to the old thread-based programming model. The reactive approach has gained popularity for a simple reason: we need alternative designs and architectures to meet today’s demands. However, it can be difficult to shift one’s mind to think in reactive terms, particularly when one realizes that we must be Reactive up and down the entire programming stack.
In this talk we’ll explore what it means to be ‘Reactive’. We’ll examine some of the more interesting tools available to us, some of which come from the Groovy community. Specifically we’ll cover Ratpack, RxGroovy, React, and RabbitMq - along with examples and a sample implementation. We’ll demonstrate how effectively they can work together at each level of the stack - from the front end, to the back end, to handling http requests and message queue events - and how easy it can be to go Reactive all the way down.
Event storage offers many practical benefits to distributed systems providing complete state changes over time, but there are a number of challenges when building an event store mechanism. Stephen Pember explores some of the problems you may encounter and shares real-world patterns for working with event storage.
Harnessing Spark and Cassandra with GroovySteve Pember
This talk is an introduction to a powerful combination in the big data space: Apache Spark and Cassandra. Spark is a cluster-computing framework that allows users to perform calculations against resilient in-memory datasets using a functional programming interface. Cassandra is a linearly scalable, fault tolerant, decentralized datastore. These two technologies are complicated, but integrate well and provide such a level of utility that whole companies have formed around them.
In this talk we’ll learn how Spark and Cassandra can be leveraged within your Groovy Application: Spark normally asks for a Scala environment. We’ll talk about Spark and Cassandra from a high level and walk through code examples. We’ll discuss the pitfalls of working with these technologies - like modeling your data appropriately to ensure even distribution in Cassandra and general packaging woes with Spark - and ways to avoid them. Finally, we’ll explore how we at ThirdChannel are using these technologies.
Surviving in a Microservices EnvironmentSteve Pember
The document discusses various topics related to surviving in a microservices environment. It addresses questions around infrastructure, architecture, team communication and provides advice. Key points include the importance of centralized logging and monitoring, avoiding tight coupling between services, ensuring an overall architectural vision, and being reluctant to add new process unless something goes wrong. The document emphasizes that most of the challenge with microservices is in infrastructure.
Are you tired of Hibernate? Is GORM is too heavy for your current project? Do you like having more control over your SQL? Do you like flexible DSLs? Try JOOQ!
JOOQ (Java Object Oriented Querying) is light-weight alternative to classic data access solutions or ORMs like Hibernate, JPA, JDBC, and GORM. JOOQ's goal is to give the developer a flexible DSL for building typesafe, database agnostic SQL queries, and attempts to convince the developer of a ‘database-first’ approach to building their application. In this talk we���ll quickly present an introduction to JOOQ from a high level, discuss its features, and see several examples of how we’re using JOOQ to great effect with many Spring Boot and Ratpack apps within our platform.
Richer Data History with Event Sourcing (SpringOne 2GX 2015Steve Pember
A common pattern in application development is to build systems where the data is directly linked to the current state of the application; one row in the database equates to one entity’s current state. Only ever knowing the current state of the data is adequate for many systems, but imagine the possibilities if one had access to the state of the data at any point in time. Enter Event Sourcing: instead of persisting the current state of our Domain Objects or Entities, we record historical events about our data. This pattern changes how we persist and process our data, but is surprisingly lightweight. In this talk I will present the basic concepts of Event Sourcing and the positive effects it can have on analytics and performance. We’ll discuss how storing historical events provides extremely powerful views into our data at any point in time. We’ll see how naturally it couples with the Event-oriented world of modern Reactive systems, and how easily it can be implemented in Groovy. We’ll examine some practical use cases and example implementations in Ratpack. Event Sourcing will change how you think about your data.
The document discusses concerns around designing microservices. It begins with an overview of microservices, noting they are an approach to building distributed applications that are independently developed and deployed, contain a single context or responsibility, and communicate simply using technologies like HTTP or message queues. The document then covers advantages like loose coupling, ability to use the right tool for each job, and facilitating continuous delivery. It also discusses challenges like the complexity of distributed systems and potential for services to fail. Finally, it outlines considerations for service design, including having each service represent a single bounded context and authority, being resilient, fast, and efficient.
With the rise of opinionated, full-featured frameworks and Object-Relational Mappers, we – as software developers – will generally build systems where our data is directly linked to the current state of our application; one row in the database equates to one entity’s state within the system. Only ever knowing the current state of the data is adequate for many systems, but imagine the possibilities if one had access to the states of all entities at any given time, and how that state was reached. It is just as important to understand the current state as it is to understand the steps that it took to reach the current state.
Enter Event Sourcing: instead of persisting the current state of our entities, we store historical events about our data. This pattern changes how we store and process our data, but is surprisingly lightweight and performant. In this talk I will present the basic concepts behind Event Sourcing and the positive implications it has on usability, visualization, and analytics within our applications. We’ll see how naturally it couples with the Event-oriented world of Reactive systems. Finally, we’ll examine some practical use cases and when one should and should not consider implementing the pattern. Event Sourcing will completely change how you think about your data.
Managing a Microservices Development Team (And advanced Microservice concerns)Steve Pember
So you’re decided to make the transition to a Microservice Architecture. You’ve spent time doing the research. You’ve designed out the responsibilities of each service with your team. You’ve read and memorized the entire article Martin Fowler wrote on the subject. Now, you’re running a team that’s tasked with building some Microservices. You’re built or extracted your first services. You’ve been successfully transmitting data between these services. What next? What should you be aware of? What should keep you up at night?
In this talk we’ll begin with a brief introduction to the architecture pattern before covering some of the more advanced topics when developing Microservices, focusing primarily on team management and service design philosophy. We’ll discuss the CAP theorem and why it should be your obsession. We’ll look at how Conway’s Law should be taken seriously and how it can serve as a warning to facilitate better communication between teams. Finally we’ll examine some common pitfalls of Microservices architectures and how they can be mitigated.
Reactive Microservice Architecture with Groovy and GrailsSteve Pember
Steve Pember gave a presentation on reactive oriented architecture with Grails and Groovy. He discussed some key points:
1. Monolithic applications will not scale well as they grow in complexity, which can negatively impact development and maintenance.
2. Service oriented architecture (SOA) and microservices can help break applications into independent components, but SOA implementations have issues like increased complexity from interconnected services.
3. Architecture choices are more important than any specific framework. Microservices aim to distill SOA principles by focusing each component on a single context to reduce complexity.
Why Reactive Architecture Will Take Over The World (and why we should be wary...Steve Pember
The natural tendency for application developers is to construct their code in a procedural, monolithic pattern. Veteran Developers know that this leads to error prone, unscalable, slow software – yet it is alarmingly prevalent. There have been several architectural patterns that have risen over the years which have attempted to mitigate this problem. We’ve heard of Service Oriented Architecture, Integration Patterns, and Event-Driven Systems, but the Reactive pattern has the best chance for success.
In this talk I will discuss the tenants of the Reactive Pattern and the importance of moving away from Monolithic architectures into Reactive. We will examine Spring Integration and the Grails Async features (along with Netty and RabbitMQ) in order to show they can quickly and effectively help your application to become Reactive. Finally, I will argue that the JVM is the best foundation currently for this architecture – but that if we’re not careful, NodeJS may be the most popular.
Richer Data History in Groovy with Event SourcingSteve Pember
As Grails developers we reflexively lean on the default pattern for data storage: a single row represents the current state of a Domain Object. For most applications, only ever knowing the current state is just fine. However, understanding the manipulations that got your data to that state are just as important as the current state itself, especially in a large system with multiple users.
Enter Event Sourcing: instead of persisting the current state of our Domain Objects, we store historical events about our data. In this talk I will discuss the basic concepts of Event Sourcing. We will discuss the advantages – particularly around performance and analytics – and disadvantages of using this pattern. We will see how easily it can be implemented in Groovy and used within a Grails application. Finally, we’ll examine some practical use cases and when one would consider implementation.
Additionally, there's a code demo that is not covered in the slides, but can be found here:
https://github.com/spember/greach2014-es-demo
Distributed Reactive Architecture: Extending SOA with EventsSteve Pember
This document discusses moving from a traditional monolithic architecture to a distributed reactive architecture using Rabbit MQ. Some benefits mentioned include improved scalability, resilience, and responsiveness by embracing event-driven architecture and message brokers. Rabbit MQ is recommended as it is language agnostic, supports message persistence and recovery, and allows for additional decoupling, though it has some performance overhead from its features. Case studies like Twitter are presented where moving to a reactive architecture improved their ability to handle load.
Message Oriented Architecture - Gr8conf US 2013Steve Pember
This deck is a set of slides for a talk I gave at Gr8conf US 2013 in Minneapolis, MN. It discusses the advantages of embracing an SOA or MODA (Message Oriented decoupled Architecture) in order to combat complexity and increase scalability. In particular, it focuses on RabbitMQ as the Message Broker technology.
Mitigating the Impact of State Management in Cloud Stream Processing SystemsScyllaDB
Stream processing is a crucial component of modern data infrastructure, but constructing an efficient and scalable stream processing system can be challenging. Decoupling compute and storage architecture has emerged as an effective solution to these challenges, but it can introduce high latency issues, especially when dealing with complex continuous queries that necessitate managing extra-large internal states.
In this talk, we focus on addressing the high latency issues associated with S3 storage in stream processing systems that employ a decoupled compute and storage architecture. We delve into the root causes of latency in this context and explore various techniques to minimize the impact of S3 latency on stream processing performance. Our proposed approach is to implement a tiered storage mechanism that leverages a blend of high-performance and low-cost storage tiers to reduce data movement between the compute and storage layers while maintaining efficient processing.
Throughout the talk, we will present experimental results that demonstrate the effectiveness of our approach in mitigating the impact of S3 latency on stream processing. By the end of the talk, attendees will have gained insights into how to optimize their stream processing systems for reduced latency and improved cost-efficiency.
Quantum Communications Q&A with Gemini LLM. These are based on Shannon's Noisy channel Theorem and offers how the classical theory applies to the quantum world.
Blockchain technology is transforming industries and reshaping the way we conduct business, manage data, and secure transactions. Whether you're new to blockchain or looking to deepen your knowledge, our guidebook, "Blockchain for Dummies", is your ultimate resource.
Details of description part II: Describing images in practice - Tech Forum 2024BookNet Canada
This presentation explores the practical application of image description techniques. Familiar guidelines will be demonstrated in practice, and descriptions will be developed “live”! If you have learned a lot about the theory of image description techniques but want to feel more confident putting them into practice, this is the presentation for you. There will be useful, actionable information for everyone, whether you are working with authors, colleagues, alone, or leveraging AI as a collaborator.
Link to presentation recording and transcript: https://bnctechforum.ca/sessions/details-of-description-part-ii-describing-images-in-practice/
Presented by BookNet Canada on June 25, 2024, with support from the Department of Canadian Heritage.
Measuring the Impact of Network Latency at TwitterScyllaDB
Widya Salim and Victor Ma will outline the causal impact analysis, framework, and key learnings used to quantify the impact of reducing Twitter's network latency.
Support en anglais diffusé lors de l'événement 100% IA organisé dans les locaux parisiens d'Iguane Solutions, le mardi 2 juillet 2024 :
- Présentation de notre plateforme IA plug and play : ses fonctionnalités avancées, telles que son interface utilisateur intuitive, son copilot puissant et des outils de monitoring performants.
- REX client : Cyril Janssens, CTO d’ easybourse, partage son expérience d’utilisation de notre plateforme IA plug & play.
UiPath Community Day Kraków: Devs4Devs ConferenceUiPathCommunity
We are honored to launch and host this event for our UiPath Polish Community, with the help of our partners - Proservartner!
We certainly hope we have managed to spike your interest in the subjects to be presented and the incredible networking opportunities at hand, too!
Check out our proposed agenda below 👇👇
08:30 ☕ Welcome coffee (30')
09:00 Opening note/ Intro to UiPath Community (10')
Cristina Vidu, Global Manager, Marketing Community @UiPath
Dawid Kot, Digital Transformation Lead @Proservartner
09:10 Cloud migration - Proservartner & DOVISTA case study (30')
Marcin Drozdowski, Automation CoE Manager @DOVISTA
Pawel Kamiński, RPA developer @DOVISTA
Mikolaj Zielinski, UiPath MVP, Senior Solutions Engineer @Proservartner
09:40 From bottlenecks to breakthroughs: Citizen Development in action (25')
Pawel Poplawski, Director, Improvement and Automation @McCormick & Company
Michał Cieślak, Senior Manager, Automation Programs @McCormick & Company
10:05 Next-level bots: API integration in UiPath Studio (30')
Mikolaj Zielinski, UiPath MVP, Senior Solutions Engineer @Proservartner
10:35 ☕ Coffee Break (15')
10:50 Document Understanding with my RPA Companion (45')
Ewa Gruszka, Enterprise Sales Specialist, AI & ML @UiPath
11:35 Power up your Robots: GenAI and GPT in REFramework (45')
Krzysztof Karaszewski, Global RPA Product Manager
12:20 🍕 Lunch Break (1hr)
13:20 From Concept to Quality: UiPath Test Suite for AI-powered Knowledge Bots (30')
Kamil Miśko, UiPath MVP, Senior RPA Developer @Zurich Insurance
13:50 Communications Mining - focus on AI capabilities (30')
Thomasz Wierzbicki, Business Analyst @Office Samurai
14:20 Polish MVP panel: Insights on MVP award achievements and career profiling
The Rise of Supernetwork Data Intensive ComputingLarry Smarr
Invited Remote Lecture to SC21
The International Conference for High Performance Computing, Networking, Storage, and Analysis
St. Louis, Missouri
November 18, 2021
How RPA Help in the Transportation and Logistics Industry.pptxSynapseIndia
Revolutionize your transportation processes with our cutting-edge RPA software. Automate repetitive tasks, reduce costs, and enhance efficiency in the logistics sector with our advanced solutions.
Kief Morris rethinks the infrastructure code delivery lifecycle, advocating for a shift towards composable infrastructure systems. We should shift to designing around deployable components rather than code modules, use more useful levels of abstraction, and drive design and deployment from applications rather than bottom-up, monolithic architecture and delivery.
Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Em...Erasmo Purificato
Slide of the tutorial entitled "Paradigm Shifts in User Modeling: A Journey from Historical Foundations to Emerging Trends" held at UMAP'24: 32nd ACM Conference on User Modeling, Adaptation and Personalization (July 1, 2024 | Cagliari, Italy)
Advanced Techniques for Cyber Security Analysis and Anomaly DetectionBert Blevins
Cybersecurity is a major concern in today's connected digital world. Threats to organizations are constantly evolving and have the potential to compromise sensitive information, disrupt operations, and lead to significant financial losses. Traditional cybersecurity techniques often fall short against modern attackers. Therefore, advanced techniques for cyber security analysis and anomaly detection are essential for protecting digital assets. This blog explores these cutting-edge methods, providing a comprehensive overview of their application and importance.
Choose our Linux Web Hosting for a seamless and successful online presencerajancomputerfbd
Our Linux Web Hosting plans offer unbeatable performance, security, and scalability, ensuring your website runs smoothly and efficiently.
Visit- https://onliveserver.com/linux-web-hosting/
論文紹介:A Systematic Survey of Prompt Engineering on Vision-Language Foundation ...Toru Tamaki
Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, Volker Tresp, Philip Torr "A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models" arXiv2023
https://arxiv.org/abs/2307.12980
34. THIRDCHANNEL @svpember
What is Rx?
• Collections + Time
• A Single Abstraction over data from different sources
• Observer Pattern with Push-based iterators
• Stream Based Functional Programming
• … with Extensions for Reactive Programming
• Async is easy
• Backpressure
81. Don’t Unsubscribe from Observables
Programmatically complete them
when another Observable fires
83. THIRDCHANNEL @svpember
AutoComplete Requirements
• Wait 250 ms between keypresses before querying
• If no keys are pressed, no query
• Successful queries should render movies
• Any new queries should kill in-flight queries