Get the Message Across: Seamlessly Transport Data to Apps, Anywhere
•
2 likes•346 views
This document discusses how Pivotal Cloud Foundry and Solace can be used together to seamlessly transport data between applications across distributed environments. It notes that Pivotal Cloud Foundry provides capabilities like elastic scaling and automation, while Solace provides an event mesh to enable event-driven communication between applications on Cloud Foundry, legacy systems, IoT devices, and across public and private clouds. A case study is presented of how a global bank used Pivotal Cloud Foundry and Solace to build a fault-tolerant global hybrid cloud data fabric and enable new applications to integrate with legacy systems.
Report
Share
Get the Message Across: Seamlessly Transport Data to Apps, Anywhere
- 1. © Copyright 2017 Pivotal Software, Inc. All rights Reserved. Version 1.0 Get the Message Across Seamlessly transport data to apps, anywhere, with Pivotal and Solace Kamala Dasika Product Marketing Pivotal Jonathan Schabowsky Sr. Architect, Office of the CTO Solace Michael Hilmen Sr. Systems Engineer Solace
- 2. Who is Winning the Cloud Wars? Worldwide IaaS Public Cloud Services Market Grew in 2017 1 ○ $18.2 billion -> $23.5 billion (29.5 Percent) ○ Amazon, Microsoft, Alibaba, Google Worldwide on-premise server sales ALSO grew in 2017 2 Most business are moving towards integrating on-premise and off-premise cloud environments3 ○ 58% of the Voice of the Enterprise (VoTE) Cloud Transformation, Organizational Dynamics 2017 survey 1 Gartner PR, Aug 1 2018 2 Gartner PR Mar 8 2018 3 451 Research Aug 3, 2018
- 3. Age of enlightenment Each cloud provider has something unique to offer ○ Regions ○ Services catalog – DBs, Application Integration, ML/AI platforms etc. ○ Security and compliance levels Some clouds are better suited for certain workloads Growing adoption of cloud platforms like Cloud Foundry and Kubernetes to abstract infrastructure Better ecosystem to facilitate Microservices, API-first architectures
- 4. Modernization is a priority Needs High Dev Efficiency Needs Low Dev Efficiency App Rarely changes App Frequently changes • Traditional apps • Leave as-is • Re-architect • Replatform • CI/CD • Evaluate • Evaluate
- 5. Cover w/ Image Diff platform and workloads need to talk to each other Version Control of Code AND Pipeline Testing & Building, Deployment to Environments MONOLITHIC APPLICATIONS DATA SERVICES MICROSERVICES COTS
- 6. © Solace 6 Cloud Private Public On Premises IoT iPaaS / APIs SaaS Legacy Application Infrastructure How do you seamlessly transport data among apps in this distributed environment?
- 7. © Solace 7 • Event-driven microservices • Event broker • Event mesh How do you seamlessly transport data among apps in this distributed environment?
- 9. What are microservices? © Solace 9 • Small, single purpose • Componentization • Continuous deployment of large, complex apps • Decentralized data management • Infrastructure automation • Smart endpoints and dumb pipes
- 10. Gartner says • Optimal software design model for delivery agility • By 2022, 70% of new digital business solutions will require event-sourced, real-time situational awareness • 80% of new business ecosystems will require support for event processing Source: Gartner “Business Events, Business Moments and Event Thinking in Digital Business” 4 August 2017, Yefim Natis. Why event-driven microservices? © Solace 10
- 11. So why are RESTful request/reply interactions still the norm? 11
- 12. Areas where RESTful API’s enhance architecture © Solace 12 • Externally facing APIs: – Provides easy mechanism for others, outside the organization, to access stored data – Security: authentication, authorization is taken care of via API mgmt. platforms. Policies can be defined to the resource/operation level – Versioning: since you cannot change all external users, API mgmt. can mediate for backwards compatibility • Internal Queries/Commands: – Since the query/command and returned data is generally useless to other services it is truly a one-to- one, request/reply paradigm – Data must be stored in order to be queried, so the data itself has already been in motion. – Frequent queries can be cached to increase performance and decrease latency NoSQL Database API Gateway Microservice B Microservice A Relational Database Query/Command API API CRUD B2B/B2C Client InternetEnterprise <Data> <Data> CRUD • Authenticate • Authorize • Versioning
- 13. Microservices require inter-process communication: Choreography • 4 microservices must be invoked for a “Create Account” use-case – Choreography does not require an orchestrator which is good – A->B->C->D… Pretty simple right? • Problems: – Failure Scenarios – Poor User Experience – Poor Scalability/Resource utilization – Inability to add new services – One-to-many pattern not easy • Problems are symptoms of tight service coupling © Solace 13 Microservice C NoSQL Database Relational Database Users Web Apps Microservice B Microservice D Microservice A Create Account Relational Database NoSQL Database API API API API API API API API API API API API API
- 14. Why RESTful APIs are Not Enough Microservices need: • Loose coupling • 1:many distribution • Deferred execution • Reliable delivery © Solace 14 Why? • To add more functionality quickly & with low effort • Rarely will a piece of data be consumed only once • Provide better response times, be more robust, be more efficient • Tolerant of faults, speed mismatches And….. û Using REST APIs increases coupling û Must not build this into your apps ü You need a stateful intermediary ü That intermediary is an event broker Microservices need decoupling and asynchronous interactions…they need events!
- 15. • Event broker • Event mesh Enhancing microservices with events © Solace 15
- 16. • Microservices are not islands onto themselves: – Greenfield apps easily communicate with microservices – Existing legacy (mainframe, monoliths, systems of record), Mobile, IoT also must communicate • The Event Broker enables agility by: – Providing inter-process communication between microservices – connecting the new with the old Microservices + existing enterprise © Solace 16 Event Broker Mainframe Systems of Record Monolithic/ Legacy Apps Microservice Event Driven Apps Internet of Things Mobile / B2B Event Broker API Gateway Internal Audiences Partners Customers
- 17. A system that reliably transmits events to interested apps and microservices across environments. What’s an Event Broker? © Solace 17
- 18. Poor User Experience… Reduce Response Time © Solace 18 • Deferred Execution Pattern: – Execute the minimal set of services required to provide user acknowledgement – Event broker persists produced events for future processing by other services – At a future time, all required services will be executed – example of eventual consistency • Enables: – Quicker response times back to UI – Ensures that state is persisted for future processing – Task execution to occur in background – Supports long running operations Microservice C NoSQL Database Relational Database API Gateway Microservice B Microservice D Microservice A Relational Database NoSQL Database Event Broker Web App Minimal Execution Set Event: Notification Acct. Open REST Query Deferred Execution Total Elapsed Time: 8 Seconds vs 19 Seconds 2 Sec 3 Sec 1 Sec 1 Sec 3 Sec 5 Sec 3 Sec 1 Sec • No impact on user experience • Parallel processing • A service failure does not impact Ux
- 19. Agility: Add new Services © Solace 19 • Example: addition of fraud detection service (Microservice E): • Can execute in Parallel with C and D – Requires no code changes to C – E does not know about D • No error logic about D must be added – User Experience remains the same since services are decoupled and we leveraged deferred execution – Resource utilization and scalability remains the same: great • Extremely Agile Microservice C NoSQL Database Relational Database API Gateway Microservice B Microservice D Microservice A Relational Database NoSQL Database Event Broker Web App REST Command Async Synchronous Event: Notification Acct. Open REST Query Microservice E
- 20. Agility: Add new Services… Turn REST APIs interactions into events © Solace 20 • Example: In order to detect Fraud, Microservice E needs all requests and responses between the Web App and Microservice A • The Event Broker supports REST APIs by proxying them: – All data is passed through the event broker, As-Is (Including all headers) to Microservice A. – Microservice A does not know these requests are being proxied • Just as it did not know about the API GW – Requests and Responses are then made available as regular events Microservice C NoSQL Database Relational Database API Gateway Microservice B Microservice D Microservice A Relational Database NoSQL Database Event Broker Web App REST Command Async Synchronous REST Proxy REST Query Microservice E Event Broker Fraud Detection Event, REQ, RESPEvent: Notification Acct. Open
- 21. One-to-Many Pattern… Can broker and enhance REST © Solace 21 • Event brokers can enable REST based microservices to be event driven: – Send HTTP POST to publish – Receive HTTP POST (webhook) to consume – Do not have to use a messaging API – Machine Learning could consume events via yet a different protocol Microservice DMicroservice A HTTP POST Request (Webhook) HTTP POST Request EventBroker Microservice E Fraud Detection 200OK Response 200OK Response Microservice C Spring Cloud Streams Event ACK AMQP Event ACK
- 22. Eventing + Microservices… Value Summary © Solace 22 Microservices Require: • Loose coupling • 1:many distribution • Deferred execution • Reliable delivery Value: • To add more functionality quickly & with low effort • Rarely will a piece of data be consumed only once • Provide better response times, be more robust, be more efficient • Tolerant of faults, speed mismatches
- 23. A network layer of interconnected event brokers. A configurable and dynamic infrastructure layer for decoupled applications, systems and devices, that makes event communications agile, reliable, and fast across the enterprise. What is an Event Mesh? © Solace 23
- 24. Event Driven Microservices on the Event Mesh Service Mesh PaaS/Cloud/Virtualization Network Event Mesh Request/Reply Microservices Hybrid Microservices Event-Driven Microservices
- 25. Event Broker Distributed = Event Mesh © Solace 25 • Benefits of an Event Mesh – Asynchronous, 1-many event distribution regardless of location – Resilient to WAN outages – Smart routing can ensure data is only transmitted if there is an interested party – Easy to audit data movement, enforce common security policies – Supports compression to reduce bandwidth costs/utilization • Makes service interactions and data movement between IaaS, Clouds and On-Premise easy • Supports Disaster Recovery (DR) use cases Event Broker Mainframe Systems of Record Distributed Apps Legacy Messaging Event Driven Microservices Internet of Things Mobile / Web Event Broker API Gateway Internal Audiences Partners Customers Event Broker Event Broker Event Broker Event Broker
- 27. 27Presentation Title ©Solace — Confidential Patterns Publish/subscribe + Queuing + Request/reply + Streaming Open Protocols & APIs REST/HTTP AMQP MQTT Websockets JMS Interactions Command + Query + Event Solace provides an advanced event broker
- 28. Solace makes it easy to create new applications and microservices. Why Solace for PCF? © Solace 28
- 29. Elastic Runtime PubSub+ Message Brokers Other PCF Tiles Other PCF Tiles Diego Loggregator Service Brokers Blobstore PCF Services & Add Ons Apps Manager Auto-Scaling API Gateway Data Sync IPSec Add-on Redis MySQL Push Notifications GemFire Single Sign-On Metrics Server Ops Metrics Notifications BOSH Pivotal Elastic Runtime Solace in PCF – Solace Tile
- 30. • Takes customers from: • To: A Unifying Eventing/Messaging Platform Legacy AppMainframe Mainframe Legacy App Mainframe On-Premise Data Center
- 31. Enabling Event-Driven Communications in Context of Layers of Enterprise Requirements © Solace 31 Enabling Event-Driven Communications Performance Security / Governance High Availability / Disaster Recovery Elastic Scalability Flexibility / Future-proofing • API, environment and protocol agnostic • Authentication, encryption, auditability of data and events • Robustness, replication and real-time recovery • Automated provisioning, cloud bursting • High throughput, low latency
- 32. Why Solace? Simple… Pattern/Feature Consolidation Streaming (Kafka) Y N N Y N N N N N Y Feature Queuing (Rabbit, Active, TIBCO, IBM MQ) Pub/Sub Y P2P Y Request/Reply Y Persistent QoS Y Non-Persistent QoS Y DMQ Y Wildcard Routing, Selector Filtering Y Prioritized Delivery Y JMS API Y Replay N Y Y Y Y Y Y Y Y Y Y © Solace — Confidential 32
- 33. What’s in this for developers? 33 © Solace
- 34. Developer Experience • Modern, Cloud native event broker • Service broker provisions instances of the PubSub+ VMR • Self-service experience – no more waiting on the messaging team to configure resources • Imminent support for Spring Cloud Streams • Easily connects applications and microservices that reside in different orgs/spaces • Use the API’s / Protocols along with the language that you are most comfortable Stop waiting for integration and resources…that’s agility!
- 35. Case Study How PCF + Solace enable an event mesh for a global consumer bank © Solace 35
- 36. Next-Gen Digital Transformation Investment Requires Next-Gen Messaging • Deep integration into PCF: – Scaling – Automation • Support new dev-models: – CI/CD integration – Microservices, multiprotocol • Without giving up: – Resiliency & Fault-tolerance – Performance – Enterprise features Bank Private Cloud Cloud Native Apps bosh
- 37. Cannot Afford to Leave Legacy Apps & Data Isolated • Enables new applications to access data from legacy • Allows legacy applications to be updated by new apps • Supports gradual migration of legacy apps to the new paradigm • Increases stability, resilience and performanceBank Private Cloud Cloud Native Apps bosh BW Legacy Apps
- 38. Leaner Operations ⇒ Vertical Scaling+Cloud Burst • For event-driven peaks, leverage 3rd-party cloud via hybrid-cloud architecture Bank Private Cloud Cloud Native Apps bosh BW Legacy Apps Reduces year-round footprint through vertical-scaling Public Cloud ... and then decommission it.
- 39. Fault Tolerant Global Hybrid Cloud Data Fabric NAM PROD APAC PROD EMEA PROD • Globally-replicated data fabric • Security, encryption & authentication • Authorization / Entitlements • WAN optimizations & compression • Filtering & routing • Private and public cloud integration builtin for temporary burst to 3rd party cloud
- 40. Embedded OS (Windows & Linux) NSX-T CPI (15 methods) v1 v2 v3 ... CVEs Product Updates Java | .NET | NodeJS Pivotal Application Service (PAS) Application Code & Frameworks Buildpacks | Spring Boot | Spring Cloud | Steeltoe Elastic | Packaged Software | Spark Pivotal Container Service (PKS) >cf push >kubectl run YOU build the containerWE build the container vSphere Azure & Azure StackGoogle CloudAWSOpenstack Pivotal Network “3Rs” Github Concourse Concourse Pivotal Services Marketplace Pivotal and Partner Products Continuous delivery Public Cloud Services Customer Managed Services OpenServiceBrokerAPI Repair — CVEs Repave Rotate — Credhub How everything comes together
- 41. © Copyright 2017 Pivotal Software, Inc. All rights Reserved. Version 1.0 Get the Message Across Seamlessly transport data to apps, anywhere, with Pivotal and Solace Kamala Dasika Product Marketing Pivotal Jonathan Schabowsky Sr. Architect, Office of the CTO Solace Michael Hilmen Sr. Systems Engineer Solace