SlideShare a Scribd company logo

Above the cloud joarder kamal

•Download as PPTX, PDF•
•
Joarder Kamal
Joarder Kamal

Current Trends and Future Directions in cloud computing were discussed. Key points included: - Cloud computing provides on-demand access to computing resources and pay-per-use model. - Major cloud platforms offer Infrastructure as a Service (IaaS), Platform as a Service (PaaS) and Software as a Service (SaaS). - Big data and NoSQL databases are enabling organizations to analyze large and diverse datasets. - Future directions may include newSQL databases, software defined datacenters, and harnessing big data for intelligence.

1 of 44
Current Trends and Future Directions Joarder Kamal @joarderm | joarder.kamal@gmail.com
At a Glance ... • Cloud Computing – A brief background • Computing as Utility • The era of Big Data • Develop applications in the Cloud • Potential research topics
Cloud Computing
• Illusion of infinite computing resource • Elimination of up front investment • Ability to pay-per usage on-demand • A commercial reality What’s In It? IaaS and PaaS SaaS
Leveraging existing h/w  Amazon  Google  Microsoft Leveraging existing platforms  Amazon Web Services  Google App Engine  Microsoft Azure  Facebook Platform  Heroku/Engine Yard/App Scale New opportunities  Salesforce/Taleo Dropbox /Instagram  Netflix/Hulu/YouTube  Google Apps/Office 360  SAP/IBM  Splunk  Many more ... Became a Reality ...
Run, Run, Run ...
Designed for failure SLA and Guarantees Performance Availability Elasticity Scalability
The business novel tells the story of an IT manager who has ninety days to rescue an over-budget and late IT initiative code named The Phoenix Project.
A great Illusion ... A Big White Box Trust Vs. Liability Standardisation
Utility Computing
Cloud as an Utility ... “If computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility ... The computer utility could become the basis of a new and important industry.” — John McCarthy, at MIT centennial celebration in 1961
XaaS – The Future ... Apps, Devices, and Communication Tech. Emergency and Military Services Wearable tech. and all types of Sensor networks Service of Services
Image taken from: http://www.progrexion.com/blog/marketing/the-internet-of-things-has-changed-marketing-and-sales-forever.html
Mobile Cloud Computing Smart Objects Smart Owners Auth/Location Server Emergency Response Smart Sensors Smart Detection Smart Query Smart Response Multiplayer Mobile Games
Big Data
Volume Velocity Variety Veracity
Above the cloud joarder kamal
NoSQL ... Not Only SQL  Key-Value Stores – Dynamo DB/Riak/MemcacheDB  Column Stores – HBase/Cassandra  Document Stores – MongoDB/CouchDB  Graph Databases – Neo4j Enabler technologies …  DFS/Consistent Hashing/Zookeeper/Bloom Filters  SSD/Flash Storage – In-memory databases
"NoSQL" has Nothing to Do With SQL – Michael Stonebraker, 2009 Reason for introducing NoSQL systems:  Performance  Complexity/Flexibility/Scalability Reducing the overheads from SQL systems:  Locking, Logging, Latching, Buffering  Data Structures – B+ Tree Vs. LSM Tree
Data Replication and Partitioning  Replication – store the same data objects into multiple servers  Partitioning/Sharding – split data objects into multiple servers based on the app. design (i.e. functional decomposition) or load distribution + High availability + Localisation + Read scalability - Hard to scale - Update propagation overhead - Data inconsistency Full/Partial/Hybrid Replication Partitioning/Sharding + High scalability + Load distribution + Write scalability - Multi-table operations - Partition size - App. and mgt. complexity A B C A B A B C A B C B C A B A B C A C C
CAP Confusion ... CAP (Consistency, Availability, Partitioning) Principle The system designer must choose between Consistency and Availability in the face of network Partition. This trade-off comes from the fact that to ensure ‘high availability’ in case of failure (i.e., crash-recovery, partition, Byzantine, etc.) data should be replicated across physical machines.
First-Tier Cloud Services Presentation Tier (Stateless) (DNS, Cache, CDN, Web Servers) Second-Tier Cloud Services Application/Logic Tier (Stateful) (Memcached, Session and Key-Value Stores, App Servers) Inner-Tier Cloud Services Database/Persistence Tier (Index Files, Relational/Semi-Relational DDMS) Back-End Cloud Services Analytics Tier (Hadoop, Map-Reduce Batch Processing) User Request/Reply App. Data Request High BASE Scalability ACID Weak form of Convergent Consistency (Non-Durable Data) Soft State Services with Scalable Consistency (Semi-Durable Data) Hard State Services with Sequential Consistency (Durable Data) 3 Model View 3-Tier Web App. with MVC Pattern Controller Client App End-User Tier (Browser, Desktop/Mobile App)Consistency Low 1 2 3 4 5 6 7 ACID – Atomicity, Consistency, Isolation, Durability BASE – Basically Available Soft-state Eventual consistency
Revisiting Design Space PACELC: If there is a partition (P), how does the system trade-off exist between availability and consistency (A and C); else (E) when the system is running as normal in the absence of partitions, how does the system trade- off exist between latency (L) and consistency (C)?
The Future ... • NewSQL Movement – Modern RDBMS for OLTP • Reduce the impact of distributed transactions • Short-lived transactions - touch a small subset of data • Repetitive queries with different data – Workload-aware – Examples: • Google Spanner • SAP HANA/Amazon RDS • VoltDB/H-Store
Developing Software for the Cloud
Developer Client App (Web Browser) Persistence Tier App Logic Tier Presentation Tier Small-to-Medium Scale Deployment Caching Web and Proxy Servers Multiple App Servers Slave DB Master DB High Availability Web Caching Content Delivery Networks Cloud Platforms Map Reduce Execution, Parallel Computations Sharded-Nothing Cluster High Scalability Warehouse Scale Deployment Architecture ...
 No need to install. Can be consumed online  No dependencies on the hardware  Interaction between multiple users within the same or federated Cloud  Stateless service compositions, interactions, and integrations  Not all Web apps are SaaS SaaS ...
 Don’t Repeat Yourself (DRY) principle  Behaviour-driven design (BDD) and user stories  Test-driven development (TDD)  Design patterns for SaaS  Model View Controller pattern  Refactoring, Agile, Scrum ...  API – lots of them !! Developing SaaS ...
Silo to SOA to SaaS In 2002 Amazon CEO Jeff Bezos wrote an email to his engineers – 1. “All teams will henceforth expose their data and functionality through service interfaces. 2. Team must communicate with each other through these interfaces. 3. There will no other form of inter-process communication allowed: no direct linking, no direct reads of another team's data store, no shared memory model, no back-doors whatever. The only communication allowed is via service interface calls over the network. 4. It doesn't matter what [API protocol] technology you use. 5. Service interfaces, without exception must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exception. 6. Anyone who doesn't do this will be fired. 7. Thank you; have a nice day!” (Courtesy: Prof. Armando Fox and Prof. David Patterson, UC Berkeley) (Link: https://class.coursera.org/saas-2012-002/lecture/index)
Engineering SaaS ...
REST ... Html Xml JOSN Resource Data e.g., “Facebook Updates” ‘GET’ [Event] ‘POST’ [Status] ‘DELETE’ [Wall-Post] Human Another Web Service Another Machine 5 key REST principles: 1. Give every “thing” an ID (Noun) 2. Link things together 3. Use standard methods (Verbs) 4. Resources with multiple representations 5. Communicate statelessly Representational State Transfer
Engineering Software is More Than Programming
The Future
Data Centres Hardware Networking Platforms SLA and Policies Computing Storage I’ve always been more interested in the future than in the past. —Grace Murray Hopper
Above the cloud joarder kamal
Research Directions
IaaS Automation ...
PaaS Automation ...
Software Defined Data Centres Available at http://www.omaramin.me/emcworld-2013-emc-blueprints/
Building the Cloud ... FIWARE Lab http://www.fi-ware.org/
In-Memory Databases Applications and Analytics merge together to enable:  Better insight and interaction  Better decision making
Big Data Intelligence ... Statistical Machine Learning Predictive Analytics Recommender Systems
Network Science ... My Facebook Network -- Blue (School Friends), Yellow (Uni (Undergrad) Friends), Cyan (Work Colleagues Own Org.), Red/Left (Work Colleagues Others), Red/Right (Uni (Doctorate) Friends), Violet/Left (Uni (Masters) Friends - Home), Violet/Right (Uni (Masters) Friends - Abroad), Green/Middle (Wife's Friends) My Facebook Network in 2013
Thank you! I tried to choose the freely available images on the Internet and acknowledge the efforts of their original creators.

More Related Content

Above the cloud joarder kamal

  • 1. Current Trends and Future Directions Joarder Kamal @joarderm | joarder.kamal@gmail.com
  • 2. At a Glance ... • Cloud Computing – A brief background • Computing as Utility • The era of Big Data • Develop applications in the Cloud • Potential research topics
  • 4. • Illusion of infinite computing resource • Elimination of up front investment • Ability to pay-per usage on-demand • A commercial reality What’s In It? IaaS and PaaS SaaS
  • 5. Leveraging existing h/w  Amazon  Google  Microsoft Leveraging existing platforms  Amazon Web Services  Google App Engine  Microsoft Azure  Facebook Platform  Heroku/Engine Yard/App Scale New opportunities  Salesforce/Taleo Dropbox /Instagram  Netflix/Hulu/YouTube  Google Apps/Office 360  SAP/IBM  Splunk  Many more ... Became a Reality ...
  • 7. Designed for failure SLA and Guarantees Performance Availability Elasticity Scalability
  • 8. The business novel tells the story of an IT manager who has ninety days to rescue an over-budget and late IT initiative code named The Phoenix Project.
  • 9. A great Illusion ... A Big White Box Trust Vs. Liability Standardisation
  • 11. Cloud as an Utility ... “If computers of the kind I have advocated become the computers of the future, then computing may someday be organized as a public utility just as the telephone system is a public utility ... The computer utility could become the basis of a new and important industry.” — John McCarthy, at MIT centennial celebration in 1961
  • 12. XaaS – The Future ... Apps, Devices, and Communication Tech. Emergency and Military Services Wearable tech. and all types of Sensor networks Service of Services
  • 13. Image taken from: http://www.progrexion.com/blog/marketing/the-internet-of-things-has-changed-marketing-and-sales-forever.html
  • 14. Mobile Cloud Computing Smart Objects Smart Owners Auth/Location Server Emergency Response Smart Sensors Smart Detection Smart Query Smart Response Multiplayer Mobile Games
  • 18. NoSQL ... Not Only SQL  Key-Value Stores – Dynamo DB/Riak/MemcacheDB  Column Stores – HBase/Cassandra  Document Stores – MongoDB/CouchDB  Graph Databases – Neo4j Enabler technologies …  DFS/Consistent Hashing/Zookeeper/Bloom Filters  SSD/Flash Storage – In-memory databases
  • 19. "NoSQL" has Nothing to Do With SQL – Michael Stonebraker, 2009 Reason for introducing NoSQL systems:  Performance  Complexity/Flexibility/Scalability Reducing the overheads from SQL systems:  Locking, Logging, Latching, Buffering  Data Structures – B+ Tree Vs. LSM Tree
  • 20. Data Replication and Partitioning  Replication – store the same data objects into multiple servers  Partitioning/Sharding – split data objects into multiple servers based on the app. design (i.e. functional decomposition) or load distribution + High availability + Localisation + Read scalability - Hard to scale - Update propagation overhead - Data inconsistency Full/Partial/Hybrid Replication Partitioning/Sharding + High scalability + Load distribution + Write scalability - Multi-table operations - Partition size - App. and mgt. complexity A B C A B A B C A B C B C A B A B C A C C
  • 21. CAP Confusion ... CAP (Consistency, Availability, Partitioning) Principle The system designer must choose between Consistency and Availability in the face of network Partition. This trade-off comes from the fact that to ensure ‘high availability’ in case of failure (i.e., crash-recovery, partition, Byzantine, etc.) data should be replicated across physical machines.
  • 22. First-Tier Cloud Services Presentation Tier (Stateless) (DNS, Cache, CDN, Web Servers) Second-Tier Cloud Services Application/Logic Tier (Stateful) (Memcached, Session and Key-Value Stores, App Servers) Inner-Tier Cloud Services Database/Persistence Tier (Index Files, Relational/Semi-Relational DDMS) Back-End Cloud Services Analytics Tier (Hadoop, Map-Reduce Batch Processing) User Request/Reply App. Data Request High BASE Scalability ACID Weak form of Convergent Consistency (Non-Durable Data) Soft State Services with Scalable Consistency (Semi-Durable Data) Hard State Services with Sequential Consistency (Durable Data) 3 Model View 3-Tier Web App. with MVC Pattern Controller Client App End-User Tier (Browser, Desktop/Mobile App)Consistency Low 1 2 3 4 5 6 7 ACID – Atomicity, Consistency, Isolation, Durability BASE – Basically Available Soft-state Eventual consistency
  • 23. Revisiting Design Space PACELC: If there is a partition (P), how does the system trade-off exist between availability and consistency (A and C); else (E) when the system is running as normal in the absence of partitions, how does the system trade- off exist between latency (L) and consistency (C)?
  • 24. The Future ... • NewSQL Movement – Modern RDBMS for OLTP • Reduce the impact of distributed transactions • Short-lived transactions - touch a small subset of data • Repetitive queries with different data – Workload-aware – Examples: • Google Spanner • SAP HANA/Amazon RDS • VoltDB/H-Store
  • 26. Developer Client App (Web Browser) Persistence Tier App Logic Tier Presentation Tier Small-to-Medium Scale Deployment Caching Web and Proxy Servers Multiple App Servers Slave DB Master DB High Availability Web Caching Content Delivery Networks Cloud Platforms Map Reduce Execution, Parallel Computations Sharded-Nothing Cluster High Scalability Warehouse Scale Deployment Architecture ...
  • 27.  No need to install. Can be consumed online  No dependencies on the hardware  Interaction between multiple users within the same or federated Cloud  Stateless service compositions, interactions, and integrations  Not all Web apps are SaaS SaaS ...
  • 28.  Don’t Repeat Yourself (DRY) principle  Behaviour-driven design (BDD) and user stories  Test-driven development (TDD)  Design patterns for SaaS  Model View Controller pattern  Refactoring, Agile, Scrum ...  API – lots of them !! Developing SaaS ...
  • 29. Silo to SOA to SaaS In 2002 Amazon CEO Jeff Bezos wrote an email to his engineers – 1. “All teams will henceforth expose their data and functionality through service interfaces. 2. Team must communicate with each other through these interfaces. 3. There will no other form of inter-process communication allowed: no direct linking, no direct reads of another team's data store, no shared memory model, no back-doors whatever. The only communication allowed is via service interface calls over the network. 4. It doesn't matter what [API protocol] technology you use. 5. Service interfaces, without exception must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exception. 6. Anyone who doesn't do this will be fired. 7. Thank you; have a nice day!” (Courtesy: Prof. Armando Fox and Prof. David Patterson, UC Berkeley) (Link: https://class.coursera.org/saas-2012-002/lecture/index)
  • 31. REST ... Html Xml JOSN Resource Data e.g., “Facebook Updates” ‘GET’ [Event] ‘POST’ [Status] ‘DELETE’ [Wall-Post] Human Another Web Service Another Machine 5 key REST principles: 1. Give every “thing” an ID (Noun) 2. Link things together 3. Use standard methods (Verbs) 4. Resources with multiple representations 5. Communicate statelessly Representational State Transfer
  • 34. Data Centres Hardware Networking Platforms SLA and Policies Computing Storage I’ve always been more interested in the future than in the past. —Grace Murray Hopper
  • 39. Software Defined Data Centres Available at http://www.omaramin.me/emcworld-2013-emc-blueprints/
  • 40. Building the Cloud ... FIWARE Lab http://www.fi-ware.org/
  • 41. In-Memory Databases Applications and Analytics merge together to enable:  Better insight and interaction  Better decision making
  • 42. Big Data Intelligence ... Statistical Machine Learning Predictive Analytics Recommender Systems
  • 43. Network Science ... My Facebook Network -- Blue (School Friends), Yellow (Uni (Undergrad) Friends), Cyan (Work Colleagues Own Org.), Red/Left (Work Colleagues Others), Red/Right (Uni (Doctorate) Friends), Violet/Left (Uni (Masters) Friends - Home), Violet/Right (Uni (Masters) Friends - Abroad), Green/Middle (Wife's Friends) My Facebook Network in 2013
  • 44. Thank you! I tried to choose the freely available images on the Internet and acknowledge the efforts of their original creators.

Editor's Notes

  1. You can remove this slide if you need. But we will strongly appreciate that you help us to spread the voice and let your colleagues and audience to download our free templates. Here are a free resources: Prepare your presentations: SlideHunter.com lets you download free templates for your presentations. Share online: SlideOnline.com lets you upload presentations for free.