SlideShare a Scribd company logo
Blair Layton, Business Development Manager,
AWS, APAC
September 14, 2017
AWS Workshop Series
Databases on AWS
Scaling Applications & Modern Data Architectures
Databases on AWS: Scaling Applications & Modern Data Architectures
Introduction to AWS
AWS Regions 16 Regions – 44 Availability Zones – 77 Edge Locations
* As of 13 September 2017

Recommended for you

[AWS Builders] Effective AWS Glue
[AWS Builders] Effective AWS Glue[AWS Builders] Effective AWS Glue
[AWS Builders] Effective AWS Glue

The document discusses AWS Glue, a fully managed ETL service. It provides an overview of Glue's programming environment and data processing model. It then gives several examples of optimizing Glue job performance, including processing many small files, a few large files, optimizing parallelism with JDBC partitions, Python performance, and using the new Python shell job type.

aws_builders_301
Amazon EMR과 SageMaker를 이��하여 데이터를 준비하고 머신러닝 모델 개발 하기
Amazon EMR과 SageMaker를 이용하여 데이터를 준비하고 머신러닝 모델 개발 하기Amazon EMR과 SageMaker를 이용하여 데이터를 준비하고 머신러닝 모델 개발 하기
Amazon EMR과 SageMaker를 이용하여 데이터를 준비하고 머신러닝 모델 개발 하기

Amazon SageMaker는 머신러닝 프로젝트를 위한 통합 플랫폼입니다. SageMaker의 기능 중 Amazon SageMaker Studio는 머신러닝 통합 개발환경을 제공하여, 데이터를 준비에서부터 모델을 빌드, 교육 및 배포하는 데 필요한 모든 단계를 수행할 수 있습니다. Amazon EMR은 Apache Spark, Apache Hive 및 Presto와 같은 오픈 소스 분석 프레임워크를 사용하여 대규모 분산 데이터 처리 작업, 대화형 SQL 쿼리 및 ML 애플리케이션을 실행하기 위한 빅 데이터 플랫폼입니다. 이 세션에서는 데이터 과학자와 ML 엔지니어가 ML 워크플로우에서 분산 빅 데이터 프레임워크를 쉽게 사용할 수 있도록 상호 서비스 간의 통합에 대하여 데모를 통해 알아봅니다.

aws for data webinar
AWS DMS를 통한 오라클 DB 마이그레이션 방법 - AWS Summit Seoul 2017
AWS DMS를 통한 오라클 DB 마이그레이션 방법 - AWS Summit Seoul 2017AWS DMS를 통한 오라클 DB 마이그레이션 방법 - AWS Summit Seoul 2017
AWS DMS를 통한 오라클 DB 마이그레이션 방법 - AWS Summit Seoul 2017

aws summit seoul 2017day2 track50900
Deploy faster wherever you like 16 Regions – 44 Availability Zones – 77 Edge Locations
* As of 13 September 2017
6
Region
Availability Zone A Availability Zone B
Concepts: Regions, Availability Zones and Networking
Account Support
Support
Managed
Services
Professional
Services
Partner
Ecosystem
Training &
Certification
Solution
Architects
Account
Management
Security &
Pricing Reports
Technical Acct.
Management
Marketplace
Business
Applications
DevOps Tools
Business
Intelligence
Security
Networking
Database &
Storage
SaaS
Subscriptions
Operating
Systems
Mobile
Build, Test,
Monitor Apps
Push
Notifications
Build, Deploy,
Manage APIs
Device Testing
Identity
Enterprise
Applications
Document
Sharing
Email &
Calendaring
Hosted
Desktops
Application
Streaming
Backup
Game
Development
3D Game
Engine
Multi-player
Backends
Mgmt. Tools
Monitoring
Auditing
Service Catalog
Server
Management
Configuration
Tracking
Optimization
Resource
Templates
Automation
Analytics
Query Large
Data Sets
Elasticsearch
Business
Analytics
Hadoop/Spark
Real-time Data
Streaming
Orchestration
Workflows
Managed
Search
Managed ETL
Artificial
Intelligence
Voice & Text
Chatbots
Machine
Learning
Text-to-Speech
Image Analysis
Deep Learning
IoT
Rules Engine
Local Compute
and Sync
Device
Shadows
Device
Gateway
Registry
Hybrid
Devices & Edge
Systems
Data
Integration
Integrated
Networking
Resource
Management
VMware on
AWS
Identity
Federation
Migration
Application
Discovery
Application
Migration
Database
Migration
Server
Migration
Data Migration
Infrastructure Regions
Availability
Zones
Points of
Presence
Compute Containers
Event-driven
Computing
Virtual
Machines
Simple Servers Auto Scaling Batch
Web
Applications
Storage Object Storage Archive Block Storage
Managed File
Storage
Exabyte-scale
Data Transport
Database MariaDB
Data
Warehousing
NoSQLAurora MySQL Oracle SQL ServerPostgreSQL
Application
Services
Transcoding Step Functions Messaging
Security
Certificate
Management
Web App.
Firewall
Identity &
Access
Key Storage &
Management
DDoS
Protection
Application
Analysis
Active
Directory
Dev Tools
Private Git
Repositories
Continuous
Delivery
Build, Test, and
Debug
Deployment
Networking
Isolated
Resources
Dedicated
Connections
Load Balancing Scalable DNSGlobal CDN
The AWS
Platform
* As of 1 September 2017
2010
61
516
1,017
159
2012 2014 2016
AWS has been continually expanding its services to support virtually any cloud workload, and it
now has more than 90 services that range from compute, storage, networking, database,
analytics, application services, deployment, management, developer, mobile, Internet of Things
(IoT), Artificial Intelligence (AI), security, hybrid and enterprise applications. AWS has launched a
total of 795 new features and/or services year to date* - for a total of 3,708 new features and/or
services since inception in 2006.
AWS Pace of Innovation

Recommended for you

AWS Summit Seoul 2023 | Amazon EKS, 중요한 건 꺾이지 않는 안정성
AWS Summit Seoul 2023 | Amazon EKS, 중요한 건 꺾이지 않는 안정성AWS Summit Seoul 2023 | Amazon EKS, 중요한 건 꺾이지 않는 안정성
AWS Summit Seoul 2023 | Amazon EKS, 중요한 건 꺾이지 않는 안정성

Amazon EKS를 안정적으로 운영하는 것은 무엇보다 중요지만 어려운 것 중 하나입니다. 운영 중 발생하는 문제들을 수동으로 대처하던 방식에서 사전에 예방할 수 있는 방식으로 변화해야 하고, 잠재적인 장애 위험을 식별해야 합니다. 이 세션에서는 Amazon EKS를 소개하고, Best Practice를 기반으로 안정성을 높일 수 있는 방법을 알아봅니다.

Building Data Lakes with AWS
Building Data Lakes with AWSBuilding Data Lakes with AWS
Building Data Lakes with AWS

The document discusses building data lakes with AWS. It recommends using Amazon S3 as the storage layer for the data lake due to its scalability, durability and integration with other AWS analytics services. It also recommends using AWS Glue to catalog and ingest data into the data lake through automated crawlers. This allows for easy discovery, querying and analysis of data in the lake.

awsamazon web servicescloud
Enterprise workloads on AWS
Enterprise workloads on AWSEnterprise workloads on AWS
Enterprise workloads on AWS

The document provides best practices for migrating enterprise workloads to AWS. It discusses common business drivers for cloud migration like agility, cost reduction, and digital transformation. Case studies are presented showing organizations achieving benefits like cost savings, improved productivity, and reduced risk by migrating applications and infrastructure to AWS. The migration process involves assessment, planning, and executing the migration. Various migration strategies and tools are outlined to help simplify and accelerate migrating workloads to AWS.

aws-initiate-rome-2019
Strengthen your security posture
Leverage security
enhancements from 1M+
customer experiences
Benefit from AWS
industry leading
security teams 24/7,
365 days a year
Security infrastructure
built to satisfy military, global
banks, and other high-
sensitivity organizations
Over 50 global
compliance
certifications and
accreditations
“We work closely with AWS to
develop a security model, which we
believe enables us to operate more
securely in the public cloud than we
can in our own data centers.”
Rob Alexander - CIO, Capital One
Access a deep set of cloud security tools
Virtual Private Cloud
Isolated cloud resources
Web Application Firewall
Filter Malicious Web Traffic
Shield
DDoS protection
Certificate Manager
Provision, manage, and
deploy SSL/TSL certificates
Networking
Key Management Service
Manage creation and control
of encryption keys
CloudHSM
Hardware-based key storage
Server-Side Encryption
Flexible data encryption
options
Encryption
IAM
Manage user access and
encryption keys
SAML Federation
SAML 2.0 support to allow
on-prem identity integration
Directory Service
Host and manage Microsoft
Active Directory
Organizations
Manage settings for multiple
accounts
Identity & Management
Service Catalog
Create and use standardized
products
Config
Track resource inventory and
changes
CloudTrail
Track user activity and API
usage
CloudWatch
Monitor resources and
applications
Inspector
Analyze application security
Artifact
Self-service for AWS’
compliance reports
Compliance
More assurance programs than anyone
Certifications /
Attestations
C5 [Germany], Cyber Essentials Plus [UK], DoD SRG, FedRAMP, FIPS, IRAP [Australia],
ISO 27001, ISO 27017, ISO 27018, ISO 9001, MLPS Level 3 [China],
MTCS Tier 3 [Singapore], PCI DSS Level 1, SEC Rule 17a-4(f), SOC 1, SOC 2, SOC 3
Laws,
Regulations,
and Privacy
DNB [Netherlands], DPA – 1998 [U.K.], EAR, EU Data Protection Directive,
EU Model Clauses, FERPA, Gramm-Leach-Bliley Act (GLBA), HIPAA, HITECH, IRS 1075,
ITAR, My Number Act [Japan], PDPA – 2010 [Malaysia], PDPA – 2012 [Singapore],
PIPEDA [Canada], Privacy Act [Australia], Privacy Act [New Zealand],
Spanish DPA Authorization, VPAT / Section 508
Alignments and
Frameworks
CIS, CJIS, CLIA, CMS Edge, CMSR, CSA, EU-US Privacy Shield, FISC [Japan], FISMA,
G-Cloud [U.K.], GxP (FDA CFR 21 Part 11), ICREA, IT Grundschutz [Germany], MITA 3.0,
MPAA, NIST, PHR, UK Cloud Security Principles, Uptime Institute Tiers
Global Enterprise Customers
General Electric Capital One BMW
Johnson &
Johnson Merck Nordstrom
“There is no public cloud infrastructure provider
that has more robust enterprise capabilities.”
Marc Benioff, Chairman & CEO, Salesforce

Recommended for you

Cost Optimisation on AWS
Cost Optimisation on AWSCost Optimisation on AWS
Cost Optimisation on AWS

Moving from an on-premises environment into AWS is just the start of the journey towards cost optimisation. In this session we’ll look at a range of ways in which our customers can understand their costs and increase their return-on-investment: building the business case; selecting the right models for the right workloads; benefiting from tiered pricing aggregation; using data to drive the choice of AWS services; implementation of intelligent auto-scaling; and, where appropriate, re-platforming to make use of new architectural patterns such as Serverless.

awspslondonsummitpreday
Deep Dive: Amazon RDS
Deep Dive: Amazon RDSDeep Dive: Amazon RDS
Deep Dive: Amazon RDS

For more training on AWS, visit: https://www.qa.com/amazon AWS Loft | London - Deep Dive: Amazon RDS by Toby Knight, Manager Solutions Architecture, 18 April 2016

awsaws cloudaws-loft-london-2016
Common Workloads on the AWS Cloud
Common Workloads on the AWS CloudCommon Workloads on the AWS Cloud
Common Workloads on the AWS Cloud

Hundreds of thousands of customers have joined the AWS community and use AWS solutions to build their businesses. In this session we will provide an overview of running four common workloads on the AWS Cloud. This session will provide insights into running four common workloads on the AWS Cloud. This includes Websites, Backup and Recovery, Disaster Recovery and Content Delivery. We will discuss the merits of running each workload on AWS and show reference architectures which will provide a quick start to migrate your own workloads to the cloud. The session will also provide an overview of the core AWS services these workloads consume such as cover Compute, Storage, Networking and Database in addition to some others. Reasons to attend: Learn about running some of the most common workloads on the AWS Cloud. Learn how you can implement services from AWS to build efficient, cost-effective and reliable architectures. Hear best practices for architecting your application, and how to scale your infrastructure in the cloud. Discover more about cloud computing and the basics of the AWS Cloud including Amazon EC2, Amazon S3 and Amazon RDS.

awsapww2014awsamazon web services
AWS Positioned as a Leader in the Gartner Magic Quadrant for Cloud
Infrastructure as a Service, Worldwide*
AWS is positioned
highest in execution
and furthest in vision
within the Leaders
Quadrant
*Gartner, Magic Quadrant for Cloud Infrastructure as a Service, Worldwide, Leong, Lydia, Petri, Gregor, Gill, Bob, Dorosh, Mike, August 32016
This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from AWS : http://www.gartner.com/doc/reprints?id=1-2G2O5FC&ct=150519&st=sb
Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as
statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
AWS Positioned as a Leader in the Gartner Magic Quadrant for
Operational Database Management Systems*
*Gartner, Magic Quadrant for Public Cloud Storage Services, Worldwide, Bala, Raj, Chandrasekran, 26 July 2016
This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from AWS : http://www.gartner.com/doc/reprints?id=1-2IH2LGI&ct=150626&st=sb
Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as
statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
DBaaS report
“AWS not only has the
largest adoption of DBaaS, it also offers
the widest range of offerings to support
analytical, operational, and transactional
workloads.”
“AWS’s key strengths lay in its dynamic
scale, automated administration, flexibility
of database offerings, strong security,
and high-availability capabilities, which
make it a preferred choice for customers”
The Forrester Wave™ is copyrighted by Forrester Research, Inc. Forrester and Forrester Wave™ are trademarks of Forrester Research, Inc. The Forrester Wave™ is a graphical representation of
Forrester's call on a market and is plotted using a detailed spreadsheet with exposed scores, weightings, and comments. Forrester does not endorse any vendor, product, or service depicted in the
Forrester Wave. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change.
The AWS Cloud
Eliminate costly technical debt and reallocate resources so
you can deliver high-value, revenue-generating projects faster.
Innovate faster and solidify your competitive advantage by
merging startup agility with enterprise experience and resources.
Reduce risk by focusing resources dedicated to security, compliance
and availability to the most important areas of your business.
"AWS is our trusted partner that is going to run our company for the next 140 years.”
Jim Fowler – CIO, General Electric

Recommended for you

Amazon S3 고급 활용 기법 - AWS Summit Seoul 2017
Amazon S3 고급 활용 기법  - AWS Summit Seoul 2017Amazon S3 고급 활용 기법  - AWS Summit Seoul 2017
Amazon S3 고급 활용 기법 - AWS Summit Seoul 2017

day2 track3aws summit seoul 20171720
KB국민카드 - 클라우드 기반 분석 플랫폼 혁신 여정 - 발표자: 박창용 과장, 데이터전략본부, AI혁신부, KB카드│강병억, Soluti...
KB국민카드 - 클라우드 기반 분석 플랫폼 혁신 여정 - 발표자: 박창용 과장, 데이터전략본부, AI혁신부, KB카드│강병억, Soluti...KB국민카드 - 클라우드 기반 분석 플랫폼 혁신 여정 - 발표자: 박창용 과장, 데이터전략본부, AI혁신부, KB카드│강병억, Soluti...
KB국민카드 - 클라우드 기반 분석 플랫폼 혁신 여정 - 발표자: 박창용 과장, 데이터전략본부, AI혁신부, KB카드│강병억, Soluti...

온프레미스 분석 플랫폼에는 자원 증설 비용, 자원 관리 비용, 신규 자원 도입 및 환경 설정의 리드타임 등 다양한 측면에서의 한계가 존재합니다. 이에 KB국민카드에서는 기존 분석 플랫폼의 한계를 극복함과 동시에 시너지를 낼 수 있는 클라우드 기반 분석 플랫폼을 설계 및 도입하였습니다. 본 사례 소개는 KB국민카드의 데이터 혁신 여정과 노하우를 소개합니다.

aws data roadshow 2023
AWS Cloud trail
AWS Cloud trailAWS Cloud trail
AWS Cloud trail

Training for AWS Solutions Architect at http://zekelabs.com/courses/amazon-web-services-training-bangalore/.This slide describes about cloud trail key concepts, workflow and event history ___________________________________________________ zekeLabs is a Technology training platform. We provide instructor led corporate training and classroom training on Industry relevant Cutting Edge Technologies like Big Data, Machine Learning, Natural Language Processing, Artificial Intelligence, Data Science, Amazon Web Services, DevOps, Cloud Computing and Frameworks like Django,Spring, Ruby on Rails, Angular 2 and many more to Professionals. Reach out to us at www.zekelabs.com or call us at +91 8095465880 or drop a mail at info@zekelabs.com

cloud trailzekelabscloud trail workflow
What is a
Large Scale Event?
What is a Large Scale Event
An event where you need more capacity than normally
allocated for a period of time
Typically from minutes to days, but could be a couple of
weeks
Often associated with a sudden surge of users
Hard to architect and provision for at a reasonable cost
Consumers get angry when it all goes wrong!
Databases on AWS: Scaling Applications & Modern Data Architectures
Databases on AWS: Scaling Applications & Modern Data Architectures

Recommended for you

AWS Summit Seoul 2023 | 다중 계정 및 하이브리드 환경에서 안전한 IAM 체계 만들기
AWS Summit Seoul 2023 | 다중 계정 및 하이브리드 환경에서 안전한 IAM 체계 만들기AWS Summit Seoul 2023 | 다중 계정 및 하이브리드 환경에서 안전한 IAM 체계 만들기
AWS Summit Seoul 2023 | 다중 계정 및 하이브리드 환경에서 안전한 IAM 체계 만들기

많은 기업들의 클라우드 환경이 확대되고 사용자 작업 환경 및 워크로드가 다양한 액세스 유형과 위치를 가짐에 따라 통합적인 IAM 관리가 요구되고 있습니다. 이 세션에서는 AWS 계정, 보안 주체, 권한 정책 등 IAM 구성 요소에 대한 이해와 이를 바탕으로 다중 AWS 계정 및 하이브리드 환경에서 IAM, IAM Identity Center, IAM Roles Anywhere를 활용하여 AWS 리소스의 안전한 사용을 돕는 베스트 프랙티스를 소개합니다.

Cost Optimisation on AWS
Cost Optimisation on AWSCost Optimisation on AWS
Cost Optimisation on AWS

This document discusses cost optimization strategies on AWS. It provides examples of cost savings achieved by companies that migrated applications to AWS including a 14 million dollar annual savings for GE. It outlines approaches for architecting efficiently for cost, optimizing usage costs over time, and taking advantage of AWS pricing benefits like reserved instances, spot instances, and different storage options. The document emphasizes optimizing through proactive monitoring and billing tools, leveraging the various EC2 pricing plans, and combining options for further savings.

awsome-day-amsterdam-march-2017
금융권 최신 AWS 도입 사례 총정리 – 신한 제주 은행, KB손해보험 사례를 중심으로 - 지성국 사업 개발 담당 이사, AWS / 정을용...
금융권 최신 AWS 도입 사례 총정리 – 신한 제주 은행, KB손해보험 사례를 중심으로 - 지성국 사업 개발 담당 이사, AWS / 정을용...금융권 최신 AWS 도입 사례 총정리 – 신한 제주 은행, KB손해보험 사례를 중심으로 - 지성국 사업 개발 담당 이사, AWS / 정을용...
금융권 최신 AWS 도입 사례 총정리 – 신한 제주 은행, KB손해보험 사례를 중심으로 - 지성국 사업 개발 담당 이사, AWS / 정을용...

금융권 최신 AWS 도입 사례 총정리 – 신한 제주 은행, KB손해보험 사례를 중심으로 지성국 사업 개발 담당 이사, AWS 정을용 수석, 신한 DS 노용헌 이사, 메가존 금융권 클라우드 규제 환경에서도 많은 고객들이 AWS를 사용하여 업무를 혁신하여 왔습니다. 크게 보면 새로운 사업에 AWS를 활용하여 혁신할 수 있는지와 기존 업무를 AWS로 신속하게 안전하게 이전하는 것으로 누누어 볼 수 있습니다. 첫 번째 사례로 신한 제주 은행 지니앱 개발 사례를 통하여 신한 DS가 비금융 앱인 "제주 지니" 프로젝트에 AWS기반 데브옵스 체계를 금융권 최초로 구축한 사례를 ���개합니다. 단순히, Infra form factor만 클라우드를 활용하던 관행에서 벗어나 개발, 테스트, 스테이징, 배포 등 CI/CD 전 과정을 AWS상에서 자동으로 구현하는 과정을 통하여 기존 On premise 대비 AWS 클라우드의 장점 및 이를 통해 배운 점 등을 공유합니다. 다음으로는 해외 북미 법인 DC 전체를 AWS 로 All-in 마이그레이션을 통하여 운영중인 K 손해보험 사례를 공유합니다. K 손해보험의 미주법인은 새롭게 시행된 미국의 강력한 사이버 보안 정책이 23 NYCRR500을 준수하면서 On premise 환경보다 우월한 보안 요건을 준수하기 위하여 AWS 클라우드 검토하게 되었으며 AWS의 금융전문파트너인 메가존과 함께 뉴저지와 캘리포니아에 위치한 2개의 데이터센터를 모두 AWS 클라우드로 이관하는데 성공하였습니다. 약 6개월간 진행된 해당 프로젝트에 대한 사례소개 및 이를 통한 비용 절감의 효과를 공유해 드리며 현재도 계속 지원하고 있는 메가존의 AWS 클라우드 매니지드 서비스의 효과도 함께 전달 드립니다.

aws_summit_seoul_2019day1day1_track3
Databases on AWS: Scaling Applications & Modern Data Architectures
Databases on AWS: Scaling Applications & Modern Data Architectures
Databases on AWS: Scaling Applications & Modern Data Architectures
What is a Large Scale Event?
For you, it could be as simple as needing twice as much
capacity for a short promotion
Everyone’s Large Scale Event is different, but the
underlying concepts are the same

Recommended for you

AWS Cloud 환경으로​ DB Migration 전략 수립하기
AWS Cloud 환경으로​ DB Migration 전략 수립하기AWS Cloud 환경으로​ DB Migration 전략 수립하기
AWS Cloud 환경으로​ DB Migration 전략 수립하기

BespinGlobal 컨설팅 본부 ​ 최정식 위원(js.choi@bespinglobal.com)​ 데이터 마이그레이션 세미나 - 데이터로 날자 Helping You Adopt Cloud | 가트너 선정 아시아 No.1 클라우드 MSP, 성공적인 클라우드 도입을 위한 전략, 구축, 운영 및 관리 서비스 제공

awsclouddb migration
AWS Certified Solutions Architect Professional Course S15-S18
AWS Certified Solutions Architect Professional Course S15-S18AWS Certified Solutions Architect Professional Course S15-S18
AWS Certified Solutions Architect Professional Course S15-S18

This deck contains the slides from our AWS Certified Solutions Architect Professional video course. It covers: Section 15 Analytics Services Section 16 Monitoring, Logging and Auditing Section 17 Security: Defense in Depth Section 18 Cost Management Full course can be found here: https://digitalcloud.training/courses/aws-certified-solutions-architect-professional-video-course/

aws certificationaws trainingaws certified solutions architect professional
AWS-Vizalytics-March-2018 2.pdf
AWS-Vizalytics-March-2018 2.pdfAWS-Vizalytics-March-2018 2.pdf
AWS-Vizalytics-March-2018 2.pdf

The document discusses AWS and Vizalytics technology for smart cities solutions. It provides an overview of AWS cloud capabilities for infrastructure, analytics, IoT, machine learning and other services. It also presents customer stories of how cities like Chicago and Transport for London have used AWS to power applications and platforms for open data, transportation, and other smart city initiatives. Finally, it outlines a logical architecture for a smart city platform leveraging AWS cloud services for data ingestion, processing, storage in a data lake, and analysis.

specialistdayssmartcitiesiot
What Problems do you Face?
Unknown infrastructure requirements
• Cost?
Short duration of the event
• Massive investment in infrastructure that is otherwise idle or
underutilized
• Often tight deadlines to get the system live
Legacy system integration
Understanding system bahaviour, required metrics
Getting the right architecture
Finding the right talent
Databases on AWS: Scaling Applications & Modern Data Architectures
You Don’t Want This!
One question is
constant!

Recommended for you

Canberra Symposium Keynote
Canberra Symposium KeynoteCanberra Symposium Keynote
Canberra Symposium Keynote

This document summarizes a presentation given by Teresa Carlson at the AWS Government, Education and Nonprofits Symposium in Canberra, Australia. Carlson discussed how cloud computing has become the new normal for many organizations. She provided examples of successful government adoption models and how AWS addresses security, compliance, procurement and culture issues. Carlson also presented statistics on AWS's growing customer base and the rapid pace of innovation, with over 500 new features and services launched in 2014.

awsgovsymposium
Canberra Symposium Keynote
Canberra Symposium KeynoteCanberra Symposium Keynote
Canberra Symposium Keynote

This document summarizes a presentation given by Teresa Carlson from Amazon Web Services at the AWS Government, Education and Nonprofits Symposium in Canberra, Australia. Carlson discussed how cloud computing has become the new normal for many organizations. She provided examples of successful government adoption models and addressed common issues around security, compliance, procurement and culture. Carlson also presented statistics on AWS's growing customer base and the rapid pace of innovation, with over 500 new features and services launched in 2014.

awsgovsymposium
AWS Webcast - AWS haystax afb oct 17(1)
AWS Webcast - AWS haystax afb oct 17(1)AWS Webcast - AWS haystax afb oct 17(1)
AWS Webcast - AWS haystax afb oct 17(1)

This document discusses how AWS provides scalable infrastructure and services to power public safety applications in the cloud. It highlights Haystax's use of AWS to build a public safety cloud that provides situational awareness tools to various government agencies. The cloud enables collaborative data sharing across jurisdictions and devices to help agencies focus on their critical public safety missions. AWS offers scalable, secure, and cost-effective infrastructure that can rapidly scale up or down based on an agency's needs.

How do we scale,
especially the
database?
So let’s start from day
one, user one ( you )
Day One, User One
A single EC2 Instance
• With full stack on this host
• Web app
• Database
• Management
• Etc.
A single Elastic IP
Route53 for DNS
EC2
Instance
Elastic IP
Amazon
Route 53
User
“We’re gonna need a bigger box”
Simplest approach
Can now leverage PIOPs
High I/O instances
High memory instances
High CPU instances
High storage instances
Easy to change instance sizes
Will hit an endpoint eventually
x1.32xlarge
m4.large
t2.micro

Recommended for you

AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar
AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar
AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar

Learn how the AWS cloud can cost effectively provide the scalable computing resources, storage services, and analytic tools that enable running different kinds of Emergency Preparedness and Collaborative workloads.

cloudanalyticcomputing
AWSome Day Galway Intro
AWSome Day Galway IntroAWSome Day Galway Intro
AWSome Day Galway Intro

The document provides an overview of Amazon Web Services' (AWS) global infrastructure footprint and capabilities. It notes that AWS has 11 regions across the world with 4 more upcoming, along with 53 edge locations and over 29 availability zones. It highlights that AWS has over 1 million active customers, including 900+ government agencies, 3,400+ educational institutions, and 1,000+ financial services organizations operating in 190 countries. The document also emphasizes AWS' continual expansion of infrastructure and server capacity.

awsome day
Demystifying Cloud Security: Lessons Learned for the Public Sector
Demystifying Cloud Security: Lessons Learned for the Public SectorDemystifying Cloud Security: Lessons Learned for the Public Sector
Demystifying Cloud Security: Lessons Learned for the Public Sector

As government agencies expand the use of cloud services, security continues to be a top priority for program managers, policymakers, and cloud service providers (CSPs). Governments and agencies worldwide are moving workloads with varying levels of sensitivity to the cloud. This session will feature agency-level security risk management practices and address common myths about security in the cloud. Participants will gain insight into how governments are leveraging cloud computing to improve their security posture and more quickly benefit from economies of scale. Mark Ryland, Chief Solutions Architect, Amazon Web Services, WWPS

Day One, User One:
We could potentially get to a
few hundred to a few
thousand depending on
application complexity and
traffic
No failover
No redundancy
Too many eggs in one
basket
EC2
Instance
Elastic IP
Amazon
Route 53
User
Day Two, User >1
First let’s separate out our
single host into more than one.
Web
Database
• Make use of a database
service?
Web
Instance
Database
Instance
Elastic IP
Amazon
Route 53
User
Start with the right
databases for the job
So decide wisely.
Look for the key
points of scale.

Recommended for you

How PagerDuty Achieved End-to-End Visibility with Splunk and AWS Webinar Deck
How PagerDuty Achieved End-to-End Visibility with Splunk and AWS Webinar DeckHow PagerDuty Achieved End-to-End Visibility with Splunk and AWS Webinar Deck
How PagerDuty Achieved End-to-End Visibility with Splunk and AWS Webinar Deck

Splunk® offers a leading platform for Operational Intelligence, enabling AWS users to look closely at machine data and gain actionable insights that can help make your organization more productive, profitable, competitive, and secure. Join us to learn how Splunk and AWS together can provide the end-to-end visibility needed to respond proactively and as quickly as possible to rapidly evolving security environments. Learn how Splunk and AWS together can provide the end-to-end visibility needed to respond proactively and as quickly as possible to rapidly evolving security environments. Speakers: David Wall, Country Manager ANZ & Head of Asia Pacific & Japan & Arup Chakrabarti, Director of Engineering - Pager Duty Myles Hosford, Security Solution Architect - AWS Richard Smith, Strategic Alliances - Splunk

awsawscloudapacipc
AWSome Day Intro Oslo 20160218
AWSome Day Intro Oslo 20160218AWSome Day Intro Oslo 20160218
AWSome Day Intro Oslo 20160218

The document provides an overview of AWSome Day and why customers use AWS. It discusses how AWS enables agility for customers (#1 reason for moving to AWS), provides a broad and deep platform (#2 reason), allows customers to innovate at scale (#3 reason), and offers cost savings and flexibility (#4 reason). It then gives examples of how different Nordic companies are using AWS for development and testing, new workloads, supplementing existing workloads, migrating applications, data center migration, and moving IT entirely to the cloud.

nordics
Keynote & Introduction
Keynote & IntroductionKeynote & Introduction
Keynote & Introduction

The document discusses Amazon Web Services (AWS) and why customers use AWS. It notes that the top reasons are: 1) Agility and scalability allowing customers to quickly scale resources, 2) The broad platform of services offered by AWS, and 3) AWS's ability to support innovation through frequent new feature and service launches. It provides examples of how customers from startups to large enterprises are using AWS across different industry domains.

awsome-day-amsterdam-march-2017
User >100
First let’s separate out our
single host into more than one.
Web
Database
• Use RDS to make your life
easier
Web
Instance
Elastic IP
RDS DB
Instance
Amazon
Route 53
User
User > 1000
Next let’s address our lack of
failover and redundancy issues
Elastic Load Balancing
Another web instance
• In another Availability Zone
Enable Amazon RDS multi-AZ
Web
Instance
RDS DB Instance
Active (Multi-AZ)
Availability Zone Availability Zone
Web
Instance
RDS DB Instance
Standby (Multi-AZ)
Elastic Load
Balancing
Amazon
Route 53
User
User >10 ks–100 ks
RDS DB Instance
Active (Multi-AZ)
Availability Zone Availability Zone
RDS DB Instance
Standby (Multi-AZ)
Elastic Load
Balancing
RDS DB Instance
Read Replica
RDS DB Instance
Read Replica
RDS DB Instance
Read Replica
RDS DB Instance
Read Replica
Web
Instance
Web
Instance
Web
Instance
Web
Instance
Web
Instance
Web
Instance
Web
Instance
Web
Instance
Amazon
Route 53
User
This will take us pretty far
honestly, but we care about
performance and efficiency,
so let’s clean this up a bit

Recommended for you

AWSome Day Intro Stockholm 201509
AWSome Day Intro Stockholm 201509AWSome Day Intro Stockholm 201509
AWSome Day Intro Stockholm 201509

This document provides an overview of Amazon Web Services (AWS) and why customers use AWS. It discusses how AWS enables agility for customers, allows them to avoid undifferentiated heavy lifting of managing infrastructure, provides a broad platform for innovation at scale, and offers cost savings and flexibility through various pricing models. The document then highlights how a variety of Nordic companies are using AWS across different use cases like development and testing, new workloads, supplementing existing workloads and infrastructure, migrating applications, data center migration, and moving entire IT operations to the cloud.

nordics
AWSome Day Thailand Keynote 2015
AWSome Day Thailand Keynote 2015AWSome Day Thailand Keynote 2015
AWSome Day Thailand Keynote 2015

This keynote presentation from an Amazon Web Services solutions architect discusses how cloud computing is changing businesses across many industries and workloads. The architect highlights examples of startup and enterprise customers benefiting from AWS, including faster development times, reduced costs compared to data centers, and innovation opportunities. The presentation also outlines the security, management tools, and career opportunities available through AWS, emphasizing how cloud skills are in high demand.

ec2lambdaamazon web services
How News Corp Secured Their Digital Transformation through Identity and Acces...
How News Corp Secured Their Digital Transformation through Identity and Acces...How News Corp Secured Their Digital Transformation through Identity and Acces...
How News Corp Secured Their Digital Transformation through Identity and Acces...

News Corp secured their digital transformation through identity and access management by migrating from their legacy on-premises solution to Okta on AWS. The migration provided a better user experience, simplified access to cloud applications, and aligned with global identity standards. Over 9,000 employees were enrolled in Okta within two weeks without business disruption. News Corp benefited from faster enrollment, increased mobile authentication, and automated user lifecycle management.

awsawscloudapacipc
Shift Some Load Around
Let’s lighten the load on our
web and database instances
Move static content from the web
instance to Amazon S3 and
CloudFront
Move dynamic content from the
Elastic Load Balancing to
CloudFront
Move session/state and DB
caching to ElastiCache or
DynamoDB
Web
Instance
RDS DB Instance
Active (Multi-AZ)
Availability Zone
Elastic Load
Balancing
Amazon S3
Amazon
CloudFront
Amazon
Route 53
User
ElastiCache
Amazon
DynamoDB
User >500k+
Availability Zone
Amazon
Route 53
User
Amazon S3
Amazon
Cloudfront
Availability Zone
Elastic Load
Balancing
DynamoDB
RDS DB Instance
Read Replica
Web
Instance
Web
Instance
Web
Instance
ElastiCache RDS DB Instance
Read Replica
Web
Instance
Web
Instance
Web
Instance
ElastiCacheRDS DB Instance
Standby (Multi-AZ)
RDS DB Instance
Active (Multi-AZ)
Time to make some
radical improvements at
the web & app layers
SOAing
Move services into their own tiers
or modules. Treat each of these
as 100% separate pieces of your
infrastructure and scale them
independently.
Amazon.com and AWS do this
extensively! It offers flexibility and
greater understanding of each
component.

Recommended for you

AWS in Financial Services
AWS in Financial ServicesAWS in Financial Services
AWS in Financial Services

AWS provides a broad range of cloud computing services for financial services customers. It has experience building and managing infrastructure since 2006 and offers over 40 services that support any workload. AWS has a global footprint with regions and availability zones around the world and a history of rapid innovation and price reductions. Major financial institutions like Capital One and Simple Bank use AWS services to reduce costs, increase agility, and focus on their customers rather than managing infrastructure.

fintech success awsawsaws cloud
Establishing a Scalable, Resilient Web Architecture | AWS Public Sector Summi...
Establishing a Scalable, Resilient Web Architecture | AWS Public Sector Summi...Establishing a Scalable, Resilient Web Architecture | AWS Public Sector Summi...
Establishing a Scalable, Resilient Web Architecture | AWS Public Sector Summi...

Amazon Web Services (AWS) provides an ideal platform for running web architectures. This session describes the foundational services required for deploying an example web architecture. It covers Amazon EC2, Amazon EBS, Elastic Load Balancing, Auto Scaling, Amazon S3, Amazon RDS, and Amazon Machine Images (AMIs) and relates overviews of the services back to the example web architecture. After the initial architecture discussion, we will describe the usage of Amazon S3 for scalable content, Elastic Load Balancing, and Auto Scaling to provide high availability.

aws public sector summitcloud computingaws cloud
Optimize your azure architecture
Optimize your azure architectureOptimize your azure architecture
Optimize your azure architecture

What we've learn from 150+ azure deployments? we will cover the following: * Whats new in azure * Know your portal * Basic billing and subscription management * From Servers to Services - IaaS to PaaS * What we've learn from 150 azure deployments * High Availability from the cost perspective - the SQL example

cloud computingasaf nakashazure
Loose Coupling Sets You Free!
The looser they're coupled, the bigger they scale
• Use independent components
• Design everything as a black box
• Decouple interactions
• Favor services with built in redundancy and scalability than
building your own
Controller A Controller B
Controller A Controller B
Q Q
Tight Coupling
Use Amazon SQS as Buffers
Loose Coupling
Users > 1 Million
RDS DB Instance
Active (Multi-AZ)
Availability Zone
Elastic Load
Balancer
RDS DB Instance
Read Replica
RDS DB Instance
Read Replica
Web
Instance
Web
Instance
Web
Instance
Web
Instance
Amazon
Route 53
User
Amazon S3
Amazon
Cloudfront
Amazon
DynamoDB
Amazon SQS
ElastiCache
Worker
Instance
Worker
Instance
Amazon
CloudWatch
Internal App
Instance
Internal App
Instance
Amazon SES
The next big steps
From 5 to 10 Million Users
You may start to run into issues with your database around
contention on the write master.
How can you solve it?
Federation (splitting into multiple DBs based on function)
Sharding (splitting one data set up across multiple hosts)
Moving some functionality to other types of DBs (NoSQL)

Recommended for you

Module 1: AWS Introduction and History - AWSome Day Online Conference - APAC
Module 1: AWS Introduction and History - AWSome Day Online Conference - APACModule 1: AWS Introduction and History - AWSome Day Online Conference - APAC
Module 1: AWS Introduction and History - AWSome Day Online Conference - APAC

This module will cover cloud computing concepts and AWS global Infrastructure. • Recognize terminology and concepts as they relate to the AWS platform and navigate the AWS Management Console. • Understand the foundational services, including Amazon Elastic Compute Cloud (EC2), Amazon Virtual Private Cloud (VPC), Amazon Simple Storage Service (S3), and Amazon Elastic Block Store (EBS). • Understand the security measures AWS provides and key concepts of AWS Identity and Access Management (IAM). • Understand AWS database services, including Amazon DynamoDB and Amazon Relational Database Service (RDS). • Understand AWS management tools, including Auto Scaling, Amazon CloudWatch, Elastic Load Balancing (ELB), and AWS Trusted Advisor. Presenters today: • John Balsillie Senior Technical Trainer, AWS APAC • Karthik Chandy Senior Technical Trainer, AWS APAC

awsawscloudawsome day online conference
AWS 클라우드가 이끄는 공공기관 혁신 :: Brad Coughlan :: AWS Summit Seoul 2016
AWS 클라우드가 이끄는 공공기관 혁신 :: Brad Coughlan :: AWS Summit Seoul 2016AWS 클라우드가 이끄는 공공기관 혁신 :: Brad Coughlan :: AWS Summit Seoul 2016
AWS 클라우드가 이끄는 공공기관 혁신 :: Brad Coughlan :: AWS Summit Seoul 2016

Brad Coughlan discussed how more public sector customers are adopting cloud services due to the freedoms cloud provides, such as freedom to build, freedom to extract value from data, freedom to migrate systems, and freedom to innovate quickly while maintaining security and compliance. AWS provides services that help customers gain these freedoms, such as databases, analytics tools, migration assistance, and security and identity management services. Coughlan argued cloud adoption allows public sectors to better serve citizens.

혁신public sectoramazon web services
The 2014 AWS Enterprise Summit Keynote
The 2014 AWS Enterprise Summit Keynote The 2014 AWS Enterprise Summit Keynote
The 2014 AWS Enterprise Summit Keynote

Enabling the New IT Org Today’s technology leaders recognize that the cloud is disrupting the way in which they collaborate and deliver technology solutions throughout their enterprise. In this session, experienced and emerging leadership teams will learn how companies are leveraging the cloud to reimagine the traditional people, process, and technology alignment model. Attendees will be presented with key considerations to better enable the alignment of executives, developers, system administrators, and end users with the objective of increasing business value, agility, and innovation.

awsamazon web servicescloud computing
Database Federation
• Split up databases by function or
purpose
• Harder to do cross-function
queries
• Essentially delays the need for
something like sharding or
NoSQL until much further down
the line
• Won’t help with single huge
functions or tables
ForumsDB
UsersDB
ProductsDB
Sharded Horizontal Scaling
• More complex at the
application layer
• ORM support can help
• No practical limit on
scalability
• Operational complexity and
sophistication
• Shard by function or key
space
• RDBMS or NoSQL
User ShardID
002345 A
002346 B
002347 C
002348 B
002349 A
A
B
C
Shifting Functionality to NoSQL
Similar in a sense to federation
Again, think about the earlier points for when you need NoSQL
vs SQL
Leverage hosted services like Amazon DynamoDB
Consider these use cases:
• Leaderboards and scoring
• Rapid ingest of clickstream or log data
• Temporary data needs (cart data)
• “Hot” tables
• Metadata or lookup tables
Amazon
DynamoDB
From 5 to 10 Million Users
You may start to run into issues with speed and performance of
your applications
Make sure you have monitoring, metrics, & logging in place
• If you can’t build it internally, outsource it! (third-party SaaS)
Pay attention to what customers are saying works well vs.
what doesn’t, and use this as direction
Try to work on squeezing as much performance out of each
service or component

Recommended for you

CloudPassage Best Practices for Automatic Security Scaling
CloudPassage Best Practices for Automatic Security ScalingCloudPassage Best Practices for Automatic Security Scaling
CloudPassage Best Practices for Automatic Security Scaling

Organizations that are transitioning from a traditional data center to an on-demand IT environment, such as AWS, are quickly finding that automating and scaling legacy security services for comprehensive workload security can be challenging. In light of these challenges, it is necessary to deploy a security solution that employs the same versatility and elasticity as the cloud workloads it is meant to protect. CloudPassage® Halo® provides virtually instant visibility and continuous protection for servers in any combination of data centers, private clouds and public clouds like AWS. Join Xero and CloudPassage to learn about best practices for migrating your security workloads to the cloud. Join us to learn: - Best practices for maintaining workload security - How you can align cloud security deployment methods with on-premises deployment methods - Key considerations for architecting your infrastructure to scale quickly and securely Who should attend: CTOs, CIOs, CISOs, Directors and Managers of Security, IT Administers, IT Architects and IT Security Engineers

awsamazon web servicescloud computing
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...

Il Forecasting è un processo importante per tantissime aziende e viene utilizzato in vari ambiti per cercare di prevedere in modo accurato la crescita e distribuzione di un prodotto, l’utilizzo delle risorse necessarie nelle linee produttive, presentazioni finanziarie e tanto altro. Amazon utilizza delle tecniche avanzate di forecasting, in parte questi servizi sono stati messi a disposizione di tutti i clienti AWS. In questa sessione illustreremo come pre-processare i dati che contengono una componente temporale e successivamente utilizzare un algoritmo che a partire dal tipo di dato analizzato produce un forecasting accurato.

Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...

La varietà e la quantità di dati che si crea ogni giorno accelera sempre più velocemente e rappresenta una opportunità irripetibile per innovare e creare nuove startup. Tuttavia gestire grandi quantità di dati può apparire complesso: creare cluster Big Data su larga scala sembra essere un investimento accessibile solo ad aziende consolidate. Ma l’elasticità del Cloud e, in particolare, i servizi Serverless ci permettono di rompere questi limiti. Vediamo quindi come è possibile sviluppare applicazioni Big Data rapidamente, senza preoccuparci dell’infrastruttura, ma dedicando tutte le risorse allo sviluppo delle nostre le nostre idee per creare prodotti innovativi.

Customer Examples
Gumi Asia: Singaporean
Gaming Company
Sizing for Peak Loads
Promotions cause huge spikes in user activity
Auto-scaling works for the web and middle tier
RDS instances have to be sized for peak loads
Adopted our recommendations in a staged approach
Amazon
Route 53
CloudFront
Availability Zone #1
Amazon S3
Availability Zone #2
Amazon EC2Amazon EC2
Auto Scaling
Geo Routing
US East
Amazon
CloudWatch
RDS DB Instance
Active (Multi-AZ)
RDS DB Instance
Standby (Multi-AZ)
User

Recommended for you

Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate

Ora puoi utilizzare Amazon Elastic Kubernetes Service (EKS) per eseguire pod Kubernetes su AWS Fargate, il motore di elaborazione serverless creato per container su AWS. Questo rende più semplice che mai costruire ed eseguire le tue applicazioni Kubernetes nel cloud AWS.In questa sessione presenteremo le caratteristiche principali del servizio e come distribuire la tua applicazione in pochi passaggi

Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS

Vent'anni fa Amazon ha attraversato una trasformazione radicale con l'obiettivo di aumentare il ritmo dell'innovazione. In questo periodo abbiamo imparato come cambiare il nostro approccio allo sviluppo delle applicazioni ci ha permesso di aumentare notevolmente l'agilità, la velocità di rilascio e, in definitiva, ci ha consentito di creare applicazioni più affidabili e scalabili. In questa sessione illustreremo come definiamo le applicazioni moderne e come la creazione di app moderne influisce non solo sull'architettura dell'applicazione, ma sulla struttura organizzativa, sulle pipeline di rilascio dello sviluppo e persino sul modello operativo. Descriveremo anche approcci comuni alla modernizzazione, compreso l'approccio utilizzato dalla stessa Amazon.com.

Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot

L’utilizzo dei container è in continua crescita. Se correttamente disegnate, le applicazioni basate su Container sono molto spesso stateless e flessibili. I servizi AWS ECS, EKS e Kubernetes su EC2 possono sfruttare le istanze Spot, portando ad un risparmio medio del 70% rispetto alle istanze On Demand. In questa sessione scopriremo insieme quali sono le caratteristiche delle istanze Spot e come possono essere utilizzate facilmente su AWS. Impareremo inoltre come Spreaker sfrutta le istanze spot per eseguire applicazioni di diverso tipo, in produzione, ad una frazione del costo on-demand!

Amazon
Route 53
CloudFront
Availability Zone #1
Amazon S3
Availability Zone #2
Amazon EC2Amazon EC2
Auto Scaling
Geo Routing
US East
User
Amazon
CloudWatch
RDS DB Instance
Active (Multi-AZ)
RDS DB Instance
Standby (Multi-AZ)
RDS DB
instance read
replica
Amazon
Route 53
CloudFront
Availability Zone #1
Amazon S3
DynamoDB
Availability Zone #2
Amazon EC2Amazon EC2
Auto Scaling
Geo Routing
US East
User
Amazon
CloudWatch
RDS DB Instance
Active (Multi-AZ)
RDS DB Instance
Standby (Multi-AZ)
RDS DB
instance read
replica
Amazon
Route 53
CloudFront
Availability Zone #1
Amazon S3
DynamoDB
Availability Zone #2
Amazon EC2
ElastiCache
Memcached
Amazon EC2
Auto Scaling
Geo Routing
US East
User
Amazon
CloudWatch
RDS DB Instance
Active (Multi-AZ)
RDS DB Instance
Standby (Multi-AZ)
RDS DB
instance read
replica
Amazon
Route 53
CloudFront
Availability Zone #1
Amazon S3
DynamoDB
Availability Zone #2
Amazon EC2
ElastiCache
(Redis Master)
ElastiCache
Memcached
Amazon EC2
Redis Slave
Auto Scaling
Geo Routing
US East
User
Amazon
CloudWatch
RDS DB Instance
Active (Multi-AZ)
RDS DB Instance
Standby (Multi-AZ)
RDS DB
instance read
replica
Amazon Redshift

Recommended for you

Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service

In recent months, many customers have been asking us the question – how to monetise Open APIs, simplify Fintech integrations and accelerate adoption of various Open Banking business models. Therefore, AWS and FinConecta would like to invite you to Open Finance marketplace presentation on October 20th. Event Agenda : Open banking so far (short recap) • PSD2, OB UK, OB Australia, OB LATAM, OB Israel Intro to Open Finance marketplace • Scope • Features • Tech overview and Demo The role of the Cloud The Future of APIs • Complying with regulation • Monetizing data / APIs • Business models • Time to market One platform for all: a Strategic approach Q&A

bankingawsfinconecta
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...

Per creare valore e costruire una propria offerta differenziante e riconoscibile, le startup di successo sanno come combinare tecnologie consolidate con componenti innovativi creati ad hoc. AWS fornisce servizi pronti all'utilizzo e, allo stesso tempo, permette di personalizzare e creare gli elementi differenzianti della propria offerta. Concentrandoci sulle tecnologie di Machine Learning, vedremo come selezionare i servizi di intelligenza artificiale offerti da AWS e, anche attraverso una demo, come costruire modelli di Machine Learning personalizzati utilizzando SageMaker Studio.

OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...

Con l'approccio tradizionale al mondo IT per molti anni è stato difficile implementare tecniche di DevOps, che finora spesso hanno previsto attività manuali portando di tanto in tanto a dei downtime degli applicativi interrompendo l'operatività dell'utente. Con l'avvento del cloud, le tecniche di DevOps sono ormai a portata di tutti a basso costo per qualsiasi genere di workload, garantendo maggiore affidabilità del sistema e risultando in dei significativi miglioramenti della business continuity. AWS mette a disposizione AWS OpsWork come strumento di Configuration Management che mira ad automatizzare e semplificare la gestione e i deployment delle istanze EC2 per mezzo di workload Chef e Puppet. Scopri come sfruttare AWS OpsWork a garanzia e affidabilità del tuo applicativo installato su Instanze EC2.

Lessons Learned
Listen to AWS Business Development and Solution
Architects ;)
Gaming promotions much easier to handle
Unpredicted loads also easier to handle
Senior operations person moving to a new game
Customers get a much better gaming experience!
Singaporean Telco
Customer Success Stories
Telecommunications Company
iPhone 5s/5c, 6/6+, 7 and Samsung Note 3-8 and S4-8
Needed a system to handle a huge number of concurrent
requests
Failed previously at the iPhone5 launch
Management directive to succeed at all costs!
Telco
Availability Zone
Elastic Load
Balancer
Web
Instance
Web
Instance
Web
Instance
Web
Instance
Amazon
Route 53
User
Amazon S3
Amazon
Cloudfront
Amazon
DynamoDB
ElastiCache
Amazon
CloudWatch
ElastiCache

Recommended for you

Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads

Vuoi conoscere le opzioni per eseguire Microsoft Active Directory su AWS? Quando si spostano carichi di lavoro Microsoft in AWS, è importante considerare come distribuire Microsoft Active Directory per supportare la gestione, l'autenticazione e l'autorizzazione dei criteri di gruppo. In questa sessione, discuteremo le opzioni per la distribuzione di Microsoft Active Directory su AWS, incluso AWS Directory Service per Microsoft Active Directory e la distribuzione di Active Directory su Windows su Amazon Elastic Compute Cloud (Amazon EC2). Trattiamo argomenti quali l'integrazione del tuo ambiente Microsoft Active Directory locale nel cloud e l'utilizzo di applicazioni SaaS, come Office 365, con AWS Single Sign-On.

Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS

Dal riconoscimento facciale al riconoscimento di frodi o difetti di fabbricazione, l'analisi di immagini e video che sfruttano tecniche di intelligenza artificiale, si stanno evolvendo e raffinando a ritmi elevati. In questo webinar esploreremo le possibilità messe a disposizione dai servizi AWS per applicare lo stato dell'arte delle tecniche di computer vision a scenari reali.

Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare

Amazon Web Services e VMware organizzano un evento virtuale gratuito il prossimo mercoledì 14 Ottobre dalle 12:00 alle 13:00 dedicato a VMware Cloud ™ on AWS, il servizio on demand che consente di eseguire applicazioni in ambienti cloud basati su VMware vSphere® e di accedere ad una vasta gamma di servizi AWS, sfruttando a pieno le potenzialità del cloud AWS e tutelando gli investimenti VMware esistenti. Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi. La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.

Great Success!
Tested with 150,000 concurrent users
All phones gone within 2 minutes
No phones misallocated or unallocated
Management said the system was too fast!
Actual launch went smoothly
Lessons
AWS can provide infrastructure for applications to scale to
very high concurrent users
Managed services allow for quick deployment and changes
to infrastructure
Impossible for the customer to execute internally
Massive cost savings, even with huge over provisioning
New, improved system now developed for iPhone 8
“With our systems on AWS, we
can scale our resources more
than 130-fold in 30 minutes,
enabling us to support more
than 2,500 orders per second”
KT Chiu
Founder and Chief Executive Officer
TixCraft
A Modern Data Architecture for
Microservices

Recommended for you

Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS

Molte aziende oggi, costruiscono applicazioni con funzionalità di tipo ledger ad esempio per verificare lo storico di accrediti o addebiti nelle transazioni bancarie o ancora per tenere traccia del flusso supply chain dei propri prodotti. Alla base di queste soluzioni ci sono i database ledger che permettono di avere un log delle transazioni trasparente, immutabile e crittograficamente verificabile, ma sono strumenti complessi e onerosi da gestire. Amazon QLDB elimina la necessità di costruire sistemi personalizzati e complessi fornendo un database ledger serverless completamente gestito. In questa sessione scopriremo come realizzare un'applicazione serverless completa che utilizzi le funzionalità di QLDB.

API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web

Con l’ascesa delle architetture di microservizi e delle ricche applicazioni mobili e Web, le API sono più importanti che mai per offrire agli utenti finali una user experience eccezionale. In questa sessione impareremo come affrontare le moderne sfide di progettazione delle API con GraphQL, un linguaggio di query API open source utilizzato da Facebook, Amazon e altro e come utilizzare AWS AppSync, un servizio GraphQL serverless gestito su AWS. Approfondiremo diversi scenari, comprendendo come AppSync può aiutare a risolvere questi casi d’uso creando API moderne con funzionalità di aggiornamento dati in tempo reale e offline. Inoltre, impareremo come Sky Italia utilizza AWS AppSync per fornire aggiornamenti sportivi in tempo reale agli utenti del proprio portale web.

Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare

Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi. La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali. In queste slide, gli esperti AWS e VMware presentano semplici e pratici accorgimenti per facilitare e semplificare la migrazione dei carichi di lavoro Oracle accelerando la trasformazione verso il cloud, approfondiranno l’architettura e dimostreranno come sfruttare a pieno le potenzialità di VMware Cloud ™ on AWS.

What to Expect from the Session
• Microservices at Amazon
• Overview and Challenges
• Key Elements and Benefits
• Two Pizza Teams
• Data Architecture Challenges
• Transactions and Rollbacks
• Streams
• Master Data Management
• Choosing a Data Store
• Aggregation
Microservices at Amazon
Microservices at Amazon
Service-Oriented Architecture
(SOA)
Single-purpose
Connect only through APIs
Connect over HTTPS
“Microservices”
Monolithic vs. SOA vs. Microservices
Microservices:
Many very small components
Business logic lives inside of
single service domain
Simple wire protocols(HTTP
with XML/JSON)
API driven with SDKs/Clients
SOA:
Fewer more sophisticated
components
Business logic can live across
domains
Enterprise Service Bus like
layers between services
Middleware

Recommended for you

Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS

1) The document discusses building a minimum viable product (MVP) using Amazon Web Services (AWS). 2) It provides an example of an MVP for an omni-channel messenger platform that was built from 2017 to connect ecommerce stores to customers via web chat, Facebook Messenger, WhatsApp, and other channels. 3) The founder discusses how they started with an MVP in 2017 with 200 ecommerce stores in Hong Kong and Taiwan, and have since expanded to over 5000 clients across Southeast Asia using AWS for scaling.

start-up-day-hk-2020
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck

This document discusses pitch decks and fundraising materials. It explains that venture capitalists will typically spend only 3 minutes and 44 seconds reviewing a pitch deck. Therefore, the deck needs to tell a compelling story to grab their attention. It also provides tips on tailoring different types of decks for different purposes, such as creating a concise 1-2 page teaser, a presentation deck for pitching in-person, and a more detailed read-only or fundraising deck. The document stresses the importance of including key information like the problem, solution, product, traction, market size, plans, team, and ask.

start-up-day-hk-2020
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers

This document discusses building serverless web applications using AWS services like API Gateway, Lambda, DynamoDB, S3 and Amplify. It provides an overview of each service and how they can work together to create a scalable, secure and cost-effective serverless application stack without having to manage servers or infrastructure. Key services covered include API Gateway for hosting APIs, Lambda for backend logic, DynamoDB for database needs, S3 for static content, and Amplify for frontend hosting and continuous deployment.

start-up-day-hk-2020
Monolithic vs. SOA vs. Microservices
SOA
Coarse-grained
Microservices
Fine-grained
Monolithic
Single Unit
Microservice Challenges
Distributed computing is hard
Transactions
• Multiple Databases across multiple services
Eventual Consistency
Lots of moving parts
Service discovery
Increase coordination
Increase message routing
Key Elements of Microservices…
Some core concepts are common to all services
• Service registration, discovery, wiring, administration
• State management
• Service metadata
• Service versioning
• Caching
Low Friction Deployment
Automated Management and Monitoring
Key Elements of Microservices…
Eliminates any long-term commitment to a technology stack
Polyglot ecosystem
Polyglot persistence
• Decompose Databases
• Database per microservice pattern
Allows easy use of Canary and Blue-Green deployments

Recommended for you

Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials

This document provides tips for fundraising from startup founders Roland Yau and Sze Lok Chan. It discusses generating competition to create urgency for investors, fundraising in parallel rather than sequentially, having a clear fundraising narrative focused on what you do and why it's compelling, and prioritizing relationships with people over firms. It also notes how the pandemic has changed fundraising, with examples of deals done virtually during this time. The tips emphasize being fully prepared before fundraising and cultivating connections with investors in advance.

start-up-day-hk-2020
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...

This document discusses Amazon's machine learning services for building conversational interfaces and extracting insights from unstructured text and audio. It describes Amazon Lex for creating chatbots, Amazon Comprehend for natural language processing tasks like entity extraction and sentiment analysis, and how they can be used together for applications like intelligent call centers and content analysis. Pre-trained APIs simplify adding machine learning to apps without requiring ML expertise.

start-up-day-hk-2020
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service

Amazon Elastic Container Service (Amazon ECS) è un servizio di gestione dei container altamente scalabile, che semplifica la gestione dei contenitori Docker attraverso un layer di orchestrazione per il controllo del deployment e del relativo lifecycle. In questa sessione presenteremo le principali caratteristiche del servizio, le architetture di riferimento per i differenti carichi di lavoro e i semplici passi necessari per poter velocemente migrare uno o più dei tuo container.

Key Elements of Microservices…
Each microservice is:
• Elastic: scales up or down independently of other services
• Resilient: services provide fault isolation boundaries
• Composable: uniform APIs for each service
• Minimal: highly cohesive set of entities
• Complete: loosely coupled with other services
Controller A Controller B
Controller A Controller B
Q Q
Tight Coupling
Loose Coupling
Microservices Benefits
Fast to develop
Rapid deployment
Parallel development & deployment
Closely integrated with DevOps
• Now ”DevSecOps”
Improved scalability, availability & fault tolerance
More closely aligned to business domain
Two-pizza teams
Full ownership
Full accountability
Aligned incentives
“DevOps”
Principles of the Two Pizza Team
How do Two Pizza Teams work?
We call them “Service teams”
Own the “primitives” they build:
• Product planning (roadmap)
• Development work
• Operational/Client support work
“You build it, you run it”
Part of a larger concentrated org (Amazon.com, AWS,
Prime, etc)

Recommended for you

Data Architecture Challenges
Challenge: Centralized Database
user-svc account-svccart-svc
DB
Applications often have a
monolithic data store
• Difficult to make schema changes
• Technology lock-in
• Vertical scaling
• Single point of failure
Centralized Database – Anti-pattern
Applications often have a
monolithic data store
• Difficult to make schema changes
• Technology lock-in
• Vertical scaling
• Single point of failure
user-svc account-svccart-svc
DB
Decentralized Data Stores
account-svccart-svc
DynamoDB RDS
user-svc
ElastiCache RDS
Polyglot Persistence
Each service chooses it’s data
store technology
Low impact schema changes
Independent scalability
Data is gated through the
service API

Recommended for you

Challenge: Transactional Integrity
Polyglot persistence generally translates into
eventual consistency
Asynchronous calls allow non-blocking, but
returns need to be handled properly
How about transactional integrity?
• Event-sourcing – Capture changes as
sequence of events
• Staged commit
• Rollback on failure
ERROR
STATE?
ROLLBACK?
Best Practice: Use Correlation IDs
09-02-2015 15:03:24 ui-svc INFO [uuid-123] ……
09-02-2015 15:03:25 catalog-svc INFO [uuid-123] ……
09-02-2015 15:03:26 checkout-svc ERROR [uuid-123] ……
09-02-2015 15:03:27 payment-svc INFO [uuid-123] ……
09-02-2015 15:03:27 shipping-svc INFO [uuid-123] ……
ui-svc
catalog-
svc
checkout-
svc
shipping-
svc
payment-
svc
request correlation id:
“uuid-123”
correlation id:
“uuid-123”
Best Practice: Microservice owns Rollback
Every microservice should expose
it’s own “rollback” method
This method could just rollback
changes, or trigger subsequent
actions
• Could send a notification
If you implement staged commit,
also expose a commit function
Microservice
Function 1
Rollback
Commit
(optional)
Event-Driven: DynamoDB Streams
If async, consider event-driven
approach with DynamoDB Streams
Don’t need to manage function
execution failure, DDB Streams
automatically retries until successful
“Attach” yourself to the data of interest
Microservice

Recommended for you

Challenge: Report Errors / Rollback
What if functions fail? (business logic failure,
not code failure)
Create a “Transaction Manager”
microservice that notifies all relevant
microservices to rollback or take action
DynamoDB is the trigger for the clean-up
function (could be SQS, Kinesis etc.)
Use Correlation ID to identify relations
mm-svc
Transaction
Manager
Function
DDB Streams
API Call
Error Table
Challenge: Report Errors / Rollback
ERROR
DynamoDB
Error Table
Transaction
Manager
Function
Kinesis
Error Stream
SQS
Error Queue
Rollback
(correlation-id)
Rollback
(correlation-id)
Rollback
(correlation-id)
Rollback
(correlation-id)
Challenge: Code Error
Lambda Execution Error because of
faulty code
Leverage Cloudwatch Logs to
process error message and call
Transaction Manager
Set Cloudwatch Logs Metric Filter to
look for Error/Exception and call
Lambda Handler upon Alarm state
ui-svc
Cloudwatch
Logs
Cloudwatch
Alarm
Transaction
Manager
Function
Beware: Stream Model with AWS Lambda
DynamoDB Streams and Kinesis streams directly work
with AWS Lambda, however AWS Lambda needs to
acknowledge processing the message correctly
If Lambda fails to process the message, the stream
horizon will not be moved forward, creating a “jam”
Solution: Monitor AWS Lambda Error Cloudwatch
Metric and react when error rate of same “Correlation ID”
keeps increasing

Recommended for you

MDM – Keep Data Consistent
Databases
AWS Lambda
“Cleanup”
Function
Cloudwatch
Scheduled Event
Perform Master Data Management
(MDM) to keep data consistent
Create AWS Lambda function to
check consistencies across
microservices and “cleanup”
Create Cloudwatch Event
to schedule the function
(e.g. hourly basis)
Choosing a Datastore
Storage & DB options in AWS
Amazon
RDS
Amazon
DynamoDB
Amazon
Elasticsearch
Service
Amazon
S3
Amazon
Kinesis
Amazon
ElastiCache
In-Memory NoSQL SQL SearchObject Streaming
Amazon
Redshift
Amazon
Glacier
Service

Recommended for you

Challenge: What Service to Use?
Many problems can be solved with NoSQL, RDBMS or
even in-memory cache technologies
Non-functional requirements can help identify appropriate
services
Solution: Classify your organizations non-functional
requirements and map them to service capabilities
Determine Your Non-Functional Requirements
Requirement
Latency > 1s 200 ms -1s 20 ms – 200 ms < 20 ms
Durability 99.99 99.999 99.9999 > 99.9999
Storage Scale < 256 GB 256 GB – 1 TB 1 TB – 16 TB > 16 TB
Availability 99 99.9 99.95 > 99.95
Data Class Public Important Secret Top Secret
Recoverability 12 – 24 hours 1 – 12 hours 5 mins – 1 hour < 5 mins
Skills None Average Good Expert
This is only an example. Your company’s classifications will be different
There will be other requirements such as regulatory compliance too.
Map Non-Functional Requirements to Services
Service Latency Durability Storage Availability Recoverability from AZ Failure
(RPO, RTO)
RDS
< 100 ms > 99.8 (EBS) 6 TB (SQL
Server 16 TB)
99.95 0s and 90s (MAZ)
Aurora < 100 ms > 99.9 64 TB > 99.95 0s and < 30s (MAZ)
Aurora + ElastiCache < 1 ms > 99.9 64 TB > 99.95 0s and < 30s (MAZ)
DynamoDB < 10 ms > 99.9 No Limit > 99.99 0s and 0s
DynamoDB / DAX < 1 ms > 99.9 No Limit > 99.99 0s and 0s
ElastiCache Redis < 1 ms N/A 3.5 TiB 99.95 0s and < 30s (MAZ)
Elasticsearch < 200 ms > 99.9 150 TB 99.95 0s and < 30s (Zone Aware)
S3 < 500 ms 99.999999999 No Limit 99.99 0s and 0s
The information below is not exact and does not represent SLAs
Finalizing Your Data Store Choices
After mapping your non-functional requirements to services you
should have a short list to choose from
Functional requirements such as geospatial data and query support
will refine the list further
You may institute standards to make data store selection simpler and
also make it easier for people to move between teams, e.g Redis over
Memcached and PostgreSQL over MySQL. These can still be
overridden, but require justification to senior management

Recommended for you

Challenge: Reporting and Analytics
Data is now spread across a number of isolated polyglot
data stores
Consolidation and aggregation required
Solution: Pull data from required microservices, push
data to data aggregation service, use pub/sub, or use a
composite service (anti-pattern).
Aggregation
usr svc
Pull model
Data Aggregation
Application
account svc cart svc
Pull
Aggregation
usr svc
Pull model Push model
Data Aggregation
Application
account svc cart svc
usr svc
account svc
cart svc
Data
Aggregation
Application
Push
Pull
Aggregation
usr svc
Pull model Push model
Data Aggregation
Application
usr svc
Data
Aggregation
Application
Pub/Sub
account svc cart svc
account svc
cart svc
Pub Sub
usr svc
account svc
cart svc
Data
Aggregation
Application
Push
Pull

Recommended for you

Aggregation
usr svc
Pull model Push model
Data Aggregation
Application
usr svc
Data
Aggregation
Application
Pub/Sub Composite
Composite Data Service
usr account cart
account svc cart svc
account svc
cart svc
Pub Sub
usr svc
account svc
cart svc
Data
Aggregation
Application
Push
Pull
A Few Thoughts
Use Non-Functional Requirements to help identify the
right data store(s) for each microservice
Use polyglot persistence to avoid bottlenecks, schema
issues and allow independent scalability (and cache)
Embrace eventual consistency and design fault-tolerant
business processes which can recover
Think ahead and plan your analytics requirements as
part of the overall architecture
Learn from our Customers
Beware of Costs
Many microservices with redundant, isolated data stores
can blow out costs
One customer in India with 300 microservices is now
looking at costs reduction
Primary, standby, read replicas and cache per microservice
with databases using PIOPs storage
Great performance, scale and resilience, but expensive

Recommended for you

Invest in Governance and Architecture
Giving each team independence is empowering
However, architects still need to understand the core
components of the distributed system and enforce
standards
An Indonesian customer is changing to microservices now,
but doesn’t have governance, architecture or standards in
place
Debugging distributed system is already proving complex
Standard logging, error handing and oversight will help

More Related Content

What's hot

Orchestrating the Cloud
Orchestrating the CloudOrchestrating the Cloud
Orchestrating the Cloud
Amazon Web Services
 
AWS RDS
AWS RDSAWS RDS
AWS RDS
Mahesh Raj
 
Become an IAM Policy Ninja
Become an IAM Policy NinjaBecome an IAM Policy Ninja
Become an IAM Policy Ninja
Amazon Web Services
 
[AWS Builders] Effective AWS Glue
[AWS Builders] Effective AWS Glue[AWS Builders] Effective AWS Glue
[AWS Builders] Effective AWS Glue
Amazon Web Services Korea
 
Amazon EMR과 SageMaker를 이용하여 데이터를 준비하고 머신러닝 모델 개발 하기
Amazon EMR과 SageMaker를 이용하여 데이터를 준비하고 머신러닝 모델 개발 하기Amazon EMR과 SageMaker를 이용하여 데이터를 준비하고 머신러닝 모델 개발 하기
Amazon EMR과 SageMaker를 이용하여 데이터를 준비하고 머신러닝 모델 개발 하기
Amazon Web Services Korea
 
AWS DMS를 통한 오라클 DB 마이그레이션 방법 - AWS Summit Seoul 2017
AWS DMS를 통한 오라클 DB 마이그레이션 방법 - AWS Summit Seoul 2017AWS DMS를 통한 오라클 DB 마이그레이션 방법 - AWS Summit Seoul 2017
AWS DMS를 통한 오라클 DB 마이그레이션 방법 - AWS Summit Seoul 2017
Amazon Web Services Korea
 
AWS Summit Seoul 2023 | Amazon EKS, 중요한 건 꺾이지 않는 안정성
AWS Summit Seoul 2023 | Amazon EKS, 중요한 건 꺾이지 않는 안정성AWS Summit Seoul 2023 | Amazon EKS, 중요한 건 꺾이지 않는 안정성
AWS Summit Seoul 2023 | Amazon EKS, 중요한 건 꺾이지 않는 안정성
Amazon Web Services Korea
 
Building Data Lakes with AWS
Building Data Lakes with AWSBuilding Data Lakes with AWS
Building Data Lakes with AWS
Amazon Web Services
 
Enterprise workloads on AWS
Enterprise workloads on AWSEnterprise workloads on AWS
Enterprise workloads on AWS
Amazon Web Services
 
Cost Optimisation on AWS
Cost Optimisation on AWSCost Optimisation on AWS
Cost Optimisation on AWS
Amazon Web Services
 
Deep Dive: Amazon RDS
Deep Dive: Amazon RDSDeep Dive: Amazon RDS
Deep Dive: Amazon RDS
Amazon Web Services
 
Common Workloads on the AWS Cloud
Common Workloads on the AWS CloudCommon Workloads on the AWS Cloud
Common Workloads on the AWS Cloud
Amazon Web Services
 
Amazon S3 고급 활용 기법 - AWS Summit Seoul 2017
Amazon S3 고급 활용 기법  - AWS Summit Seoul 2017Amazon S3 고급 활용 기법  - AWS Summit Seoul 2017
Amazon S3 고급 활용 기법 - AWS Summit Seoul 2017
Amazon Web Services Korea
 
KB국민카드 - 클라우드 기반 분석 플랫폼 혁신 여정 - 발표자: 박창용 과장, 데이터전략본부, AI혁신부, KB카드│강병억, Soluti...
KB국민카드 - 클라우드 기반 분석 플랫폼 혁신 여정 - 발표자: 박창용 과장, 데이터전략본부, AI혁신부, KB카드│강병억, Soluti...KB국민카드 - 클라우드 기반 분석 플랫폼 혁신 여정 - 발표자: 박창용 과장, 데이터전략본부, AI혁신부, KB카드│강병억, Soluti...
KB국민카드 - 클라우드 기반 분석 플랫폼 혁신 여정 - 발표자: 박창용 과장, 데이터전략본부, AI혁신부, KB카드│강병억, Soluti...
Amazon Web Services Korea
 
AWS Cloud trail
AWS Cloud trailAWS Cloud trail
AWS Cloud trail
zekeLabs Technologies
 
AWS Summit Seoul 2023 | 다중 계정 및 하이브리드 환경에서 안전한 IAM 체계 만들기
AWS Summit Seoul 2023 | 다중 계정 및 하이브리드 환경에서 안전한 IAM 체계 만들기AWS Summit Seoul 2023 | 다중 계정 및 하이브리드 환경에서 안전한 IAM 체계 만들기
AWS Summit Seoul 2023 | 다중 계정 및 하이브리드 환경에서 안전한 IAM 체계 만들기
Amazon Web Services Korea
 
Cost Optimisation on AWS
Cost Optimisation on AWSCost Optimisation on AWS
Cost Optimisation on AWS
Amazon Web Services
 
금융권 최신 AWS 도입 사례 총정리 – 신한 제주 은행, KB손해보험 사례를 중심으로 - 지성국 사업 개발 담당 이사, AWS / 정을용...
금융권 최신 AWS 도입 사례 총정리 – 신한 제주 은행, KB손해보험 사례를 중심으로 - 지성국 사업 개발 담당 이사, AWS / 정을용...금융권 최신 AWS 도입 사례 총정리 – 신한 제주 은행, KB손해보험 사례를 중심으로 - 지성국 사업 개발 담당 이사, AWS / 정을용...
금융권 최신 AWS 도입 사례 총정리 – 신한 제주 은행, KB손해보험 사례를 중심으로 - 지성국 사업 개발 담당 이사, AWS / 정을용...
Amazon Web Services Korea
 
AWS Cloud 환경으로​ DB Migration 전략 수립하기
AWS Cloud 환경으로​ DB Migration 전략 수립하기AWS Cloud 환경으로​ DB Migration 전략 수립하기
AWS Cloud 환경으로​ DB Migration 전략 수립하기
BESPIN GLOBAL
 
AWS Certified Solutions Architect Professional Course S15-S18
AWS Certified Solutions Architect Professional Course S15-S18AWS Certified Solutions Architect Professional Course S15-S18
AWS Certified Solutions Architect Professional Course S15-S18
Neal Davis
 

What's hot (20)

Orchestrating the Cloud
Orchestrating the CloudOrchestrating the Cloud
Orchestrating the Cloud
 
AWS RDS
AWS RDSAWS RDS
AWS RDS
 
Become an IAM Policy Ninja
Become an IAM Policy NinjaBecome an IAM Policy Ninja
Become an IAM Policy Ninja
 
[AWS Builders] Effective AWS Glue
[AWS Builders] Effective AWS Glue[AWS Builders] Effective AWS Glue
[AWS Builders] Effective AWS Glue
 
Amazon EMR과 SageMaker를 이용하여 데이터를 준비하고 머신러닝 모델 개발 하기
Amazon EMR과 SageMaker를 이용하여 데이터를 준비하고 머신러닝 모델 개발 하기Amazon EMR과 SageMaker를 이용하여 데이터를 준비하고 머신러닝 모델 개발 하기
Amazon EMR과 SageMaker를 이용하여 데이터를 준비하고 머신러닝 모델 개발 하기
 
AWS DMS를 통한 오라클 DB 마이그레이션 방법 - AWS Summit Seoul 2017
AWS DMS를 통한 오라클 DB 마이그레이션 방법 - AWS Summit Seoul 2017AWS DMS를 통한 오라클 DB 마이그레이션 방법 - AWS Summit Seoul 2017
AWS DMS를 통한 오라클 DB 마이그레이션 방법 - AWS Summit Seoul 2017
 
AWS Summit Seoul 2023 | Amazon EKS, 중요한 건 꺾이지 않는 안정성
AWS Summit Seoul 2023 | Amazon EKS, 중요한 건 꺾이지 않는 안정성AWS Summit Seoul 2023 | Amazon EKS, 중요한 건 꺾이지 않는 안정성
AWS Summit Seoul 2023 | Amazon EKS, 중요한 건 꺾이지 않는 안정성
 
Building Data Lakes with AWS
Building Data Lakes with AWSBuilding Data Lakes with AWS
Building Data Lakes with AWS
 
Enterprise workloads on AWS
Enterprise workloads on AWSEnterprise workloads on AWS
Enterprise workloads on AWS
 
Cost Optimisation on AWS
Cost Optimisation on AWSCost Optimisation on AWS
Cost Optimisation on AWS
 
Deep Dive: Amazon RDS
Deep Dive: Amazon RDSDeep Dive: Amazon RDS
Deep Dive: Amazon RDS
 
Common Workloads on the AWS Cloud
Common Workloads on the AWS CloudCommon Workloads on the AWS Cloud
Common Workloads on the AWS Cloud
 
Amazon S3 고급 활용 기법 - AWS Summit Seoul 2017
Amazon S3 고급 활용 기법  - AWS Summit Seoul 2017Amazon S3 고급 활용 기법  - AWS Summit Seoul 2017
Amazon S3 고급 활용 기법 - AWS Summit Seoul 2017
 
KB국민카드 - 클라우드 기반 분석 플랫폼 혁신 여정 - 발표자: 박창용 과장, 데이터전략본부, AI혁신부, KB카드│강병억, Soluti...
KB국민카드 - 클라우드 기반 분석 플랫폼 혁신 여정 - 발표자: 박창용 과장, 데이터전략본부, AI혁신부, KB카드│강병억, Soluti...KB국민카드 - 클라우드 기반 분석 플랫폼 혁신 여정 - 발표자: 박창용 과장, 데이터전략본부, AI혁신부, KB카드│강병억, Soluti...
KB국민카드 - 클라우드 기반 분석 플랫폼 혁신 여정 - 발표자: 박창용 과장, 데이터전략본부, AI혁신부, KB카드│강병억, Soluti...
 
AWS Cloud trail
AWS Cloud trailAWS Cloud trail
AWS Cloud trail
 
AWS Summit Seoul 2023 | 다중 계정 및 하이브리드 환경에서 안전한 IAM 체계 만들기
AWS Summit Seoul 2023 | 다중 계정 및 하이브리드 환경에서 안전한 IAM 체계 만들기AWS Summit Seoul 2023 | 다중 계정 및 하이브리드 환경에서 안전한 IAM 체계 만들기
AWS Summit Seoul 2023 | 다중 계정 및 하이브리드 환경에서 안전한 IAM 체계 만들기
 
Cost Optimisation on AWS
Cost Optimisation on AWSCost Optimisation on AWS
Cost Optimisation on AWS
 
금융권 최신 AWS 도입 사례 총정리 – 신한 제주 은행, KB손해보험 사례를 중심으로 - 지성국 사업 개발 담당 이사, AWS / 정을용...
금융권 최신 AWS 도입 사례 총정리 – 신한 제주 은행, KB손해보험 사례를 중심으로 - 지성국 사업 개발 담당 이사, AWS / 정을용...금융권 최신 AWS 도입 사례 총정리 – 신한 제주 은행, KB손해보험 사례를 중심으로 - 지성국 사업 개발 담당 이사, AWS / 정을용...
금융권 최신 AWS 도입 사례 총정리 – 신한 제주 은행, KB손해보험 사례를 중심으로 - 지성국 사업 개발 담당 이사, AWS / 정을용...
 
AWS Cloud 환경으로​ DB Migration 전략 수립하기
AWS Cloud 환경으로​ DB Migration 전략 수립하기AWS Cloud 환경으로​ DB Migration 전략 수립하기
AWS Cloud 환경으로​ DB Migration 전략 수립하기
 
AWS Certified Solutions Architect Professional Course S15-S18
AWS Certified Solutions Architect Professional Course S15-S18AWS Certified Solutions Architect Professional Course S15-S18
AWS Certified Solutions Architect Professional Course S15-S18
 

Similar to Databases on AWS: Scaling Applications & Modern Data Architectures

AWS-Vizalytics-March-2018 2.pdf
AWS-Vizalytics-March-2018 2.pdfAWS-Vizalytics-March-2018 2.pdf
AWS-Vizalytics-March-2018 2.pdf
Amazon Web Services
 
Canberra Symposium Keynote
Canberra Symposium KeynoteCanberra Symposium Keynote
Canberra Symposium Keynote
Amazon Web Services
 
Canberra Symposium Keynote
Canberra Symposium KeynoteCanberra Symposium Keynote
Canberra Symposium Keynote
Amazon Web Services
 
AWS Webcast - AWS haystax afb oct 17(1)
AWS Webcast - AWS haystax afb oct 17(1)AWS Webcast - AWS haystax afb oct 17(1)
AWS Webcast - AWS haystax afb oct 17(1)
Amazon Web Services
 
AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar
AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar
AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar
Amazon Web Services
 
AWSome Day Galway Intro
AWSome Day Galway IntroAWSome Day Galway Intro
AWSome Day Galway Intro
Amazon Web Services
 
Demystifying Cloud Security: Lessons Learned for the Public Sector
Demystifying Cloud Security: Lessons Learned for the Public SectorDemystifying Cloud Security: Lessons Learned for the Public Sector
Demystifying Cloud Security: Lessons Learned for the Public Sector
Amazon Web Services
 
How PagerDuty Achieved End-to-End Visibility with Splunk and AWS Webinar Deck
How PagerDuty Achieved End-to-End Visibility with Splunk and AWS Webinar DeckHow PagerDuty Achieved End-to-End Visibility with Splunk and AWS Webinar Deck
How PagerDuty Achieved End-to-End Visibility with Splunk and AWS Webinar Deck
Amazon Web Services
 
AWSome Day Intro Oslo 20160218
AWSome Day Intro Oslo 20160218AWSome Day Intro Oslo 20160218
AWSome Day Intro Oslo 20160218
Amazon Web Services
 
Keynote & Introduction
Keynote & IntroductionKeynote & Introduction
Keynote & Introduction
Amazon Web Services
 
AWSome Day Intro Stockholm 201509
AWSome Day Intro Stockholm 201509AWSome Day Intro Stockholm 201509
AWSome Day Intro Stockholm 201509
Amazon Web Services
 
AWSome Day Thailand Keynote 2015
AWSome Day Thailand Keynote 2015AWSome Day Thailand Keynote 2015
AWSome Day Thailand Keynote 2015
Hwee Bee Tan
 
How News Corp Secured Their Digital Transformation through Identity and Acces...
How News Corp Secured Their Digital Transformation through Identity and Acces...How News Corp Secured Their Digital Transformation through Identity and Acces...
How News Corp Secured Their Digital Transformation through Identity and Acces...
Amazon Web Services
 
AWS in Financial Services
AWS in Financial ServicesAWS in Financial Services
AWS in Financial Services
Amazon Web Services
 
Establishing a Scalable, Resilient Web Architecture | AWS Public Sector Summi...
Establishing a Scalable, Resilient Web Architecture | AWS Public Sector Summi...Establishing a Scalable, Resilient Web Architecture | AWS Public Sector Summi...
Establishing a Scalable, Resilient Web Architecture | AWS Public Sector Summi...
Amazon Web Services
 
Optimize your azure architecture
Optimize your azure architectureOptimize your azure architecture
Optimize your azure architecture
Asaf Nakash
 
Module 1: AWS Introduction and History - AWSome Day Online Conference - APAC
Module 1: AWS Introduction and History - AWSome Day Online Conference - APACModule 1: AWS Introduction and History - AWSome Day Online Conference - APAC
Module 1: AWS Introduction and History - AWSome Day Online Conference - APAC
Amazon Web Services
 
AWS 클라우드가 이끄는 공공기관 혁신 :: Brad Coughlan :: AWS Summit Seoul 2016
AWS 클라우드가 이끄는 공공기관 혁신 :: Brad Coughlan :: AWS Summit Seoul 2016AWS 클라우드가 이끄는 공공기관 혁신 :: Brad Coughlan :: AWS Summit Seoul 2016
AWS 클라우드가 이끄는 공공기관 혁신 :: Brad Coughlan :: AWS Summit Seoul 2016
Amazon Web Services Korea
 
The 2014 AWS Enterprise Summit Keynote
The 2014 AWS Enterprise Summit Keynote The 2014 AWS Enterprise Summit Keynote
The 2014 AWS Enterprise Summit Keynote
Amazon Web Services
 
CloudPassage Best Practices for Automatic Security Scaling
CloudPassage Best Practices for Automatic Security ScalingCloudPassage Best Practices for Automatic Security Scaling
CloudPassage Best Practices for Automatic Security Scaling
Amazon Web Services
 

Similar to Databases on AWS: Scaling Applications & Modern Data Architectures (20)

AWS-Vizalytics-March-2018 2.pdf
AWS-Vizalytics-March-2018 2.pdfAWS-Vizalytics-March-2018 2.pdf
AWS-Vizalytics-March-2018 2.pdf
 
Canberra Symposium Keynote
Canberra Symposium KeynoteCanberra Symposium Keynote
Canberra Symposium Keynote
 
Canberra Symposium Keynote
Canberra Symposium KeynoteCanberra Symposium Keynote
Canberra Symposium Keynote
 
AWS Webcast - AWS haystax afb oct 17(1)
AWS Webcast - AWS haystax afb oct 17(1)AWS Webcast - AWS haystax afb oct 17(1)
AWS Webcast - AWS haystax afb oct 17(1)
 
AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar
AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar
AWS Webcast - Emergency Preparedness and Interagency Collaboration Webinar
 
AWSome Day Galway Intro
AWSome Day Galway IntroAWSome Day Galway Intro
AWSome Day Galway Intro
 
Demystifying Cloud Security: Lessons Learned for the Public Sector
Demystifying Cloud Security: Lessons Learned for the Public SectorDemystifying Cloud Security: Lessons Learned for the Public Sector
Demystifying Cloud Security: Lessons Learned for the Public Sector
 
How PagerDuty Achieved End-to-End Visibility with Splunk and AWS Webinar Deck
How PagerDuty Achieved End-to-End Visibility with Splunk and AWS Webinar DeckHow PagerDuty Achieved End-to-End Visibility with Splunk and AWS Webinar Deck
How PagerDuty Achieved End-to-End Visibility with Splunk and AWS Webinar Deck
 
AWSome Day Intro Oslo 20160218
AWSome Day Intro Oslo 20160218AWSome Day Intro Oslo 20160218
AWSome Day Intro Oslo 20160218
 
Keynote & Introduction
Keynote & IntroductionKeynote & Introduction
Keynote & Introduction
 
AWSome Day Intro Stockholm 201509
AWSome Day Intro Stockholm 201509AWSome Day Intro Stockholm 201509
AWSome Day Intro Stockholm 201509
 
AWSome Day Thailand Keynote 2015
AWSome Day Thailand Keynote 2015AWSome Day Thailand Keynote 2015
AWSome Day Thailand Keynote 2015
 
How News Corp Secured Their Digital Transformation through Identity and Acces...
How News Corp Secured Their Digital Transformation through Identity and Acces...How News Corp Secured Their Digital Transformation through Identity and Acces...
How News Corp Secured Their Digital Transformation through Identity and Acces...
 
AWS in Financial Services
AWS in Financial ServicesAWS in Financial Services
AWS in Financial Services
 
Establishing a Scalable, Resilient Web Architecture | AWS Public Sector Summi...
Establishing a Scalable, Resilient Web Architecture | AWS Public Sector Summi...Establishing a Scalable, Resilient Web Architecture | AWS Public Sector Summi...
Establishing a Scalable, Resilient Web Architecture | AWS Public Sector Summi...
 
Optimize your azure architecture
Optimize your azure architectureOptimize your azure architecture
Optimize your azure architecture
 
Module 1: AWS Introduction and History - AWSome Day Online Conference - APAC
Module 1: AWS Introduction and History - AWSome Day Online Conference - APACModule 1: AWS Introduction and History - AWSome Day Online Conference - APAC
Module 1: AWS Introduction and History - AWSome Day Online Conference - APAC
 
AWS 클라우드가 이끄는 공공기관 혁신 :: Brad Coughlan :: AWS Summit Seoul 2016
AWS 클라우드가 이끄는 공공기관 혁신 :: Brad Coughlan :: AWS Summit Seoul 2016AWS 클라우드가 이끄는 공공기관 혁신 :: Brad Coughlan :: AWS Summit Seoul 2016
AWS 클라우드가 이끄는 공공기관 혁신 :: Brad Coughlan :: AWS Summit Seoul 2016
 
The 2014 AWS Enterprise Summit Keynote
The 2014 AWS Enterprise Summit Keynote The 2014 AWS Enterprise Summit Keynote
The 2014 AWS Enterprise Summit Keynote
 
CloudPassage Best Practices for Automatic Security Scaling
CloudPassage Best Practices for Automatic Security ScalingCloudPassage Best Practices for Automatic Security Scaling
CloudPassage Best Practices for Automatic Security Scaling
 

More from Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
Amazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
Amazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
Amazon Web Services
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Amazon Web Services
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
Amazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
Amazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Amazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
Amazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Amazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
Amazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
Amazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
Amazon Web Services
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
Amazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
Amazon Web Services
 

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

Databases on AWS: Scaling Applications & Modern Data Architectures

  • 1. Blair Layton, Business Development Manager, AWS, APAC September 14, 2017 AWS Workshop Series Databases on AWS Scaling Applications & Modern Data Architectures
  • 4. AWS Regions 16 Regions – 44 Availability Zones – 77 Edge Locations * As of 13 September 2017
  • 5. Deploy faster wherever you like 16 Regions – 44 Availability Zones – 77 Edge Locations * As of 13 September 2017
  • 6. 6 Region Availability Zone A Availability Zone B Concepts: Regions, Availability Zones and Networking
  • 7. Account Support Support Managed Services Professional Services Partner Ecosystem Training & Certification Solution Architects Account Management Security & Pricing Reports Technical Acct. Management Marketplace Business Applications DevOps Tools Business Intelligence Security Networking Database & Storage SaaS Subscriptions Operating Systems Mobile Build, Test, Monitor Apps Push Notifications Build, Deploy, Manage APIs Device Testing Identity Enterprise Applications Document Sharing Email & Calendaring Hosted Desktops Application Streaming Backup Game Development 3D Game Engine Multi-player Backends Mgmt. Tools Monitoring Auditing Service Catalog Server Management Configuration Tracking Optimization Resource Templates Automation Analytics Query Large Data Sets Elasticsearch Business Analytics Hadoop/Spark Real-time Data Streaming Orchestration Workflows Managed Search Managed ETL Artificial Intelligence Voice & Text Chatbots Machine Learning Text-to-Speech Image Analysis Deep Learning IoT Rules Engine Local Compute and Sync Device Shadows Device Gateway Registry Hybrid Devices & Edge Systems Data Integration Integrated Networking Resource Management VMware on AWS Identity Federation Migration Application Discovery Application Migration Database Migration Server Migration Data Migration Infrastructure Regions Availability Zones Points of Presence Compute Containers Event-driven Computing Virtual Machines Simple Servers Auto Scaling Batch Web Applications Storage Object Storage Archive Block Storage Managed File Storage Exabyte-scale Data Transport Database MariaDB Data Warehousing NoSQLAurora MySQL Oracle SQL ServerPostgreSQL Application Services Transcoding Step Functions Messaging Security Certificate Management Web App. Firewall Identity & Access Key Storage & Management DDoS Protection Application Analysis Active Directory Dev Tools Private Git Repositories Continuous Delivery Build, Test, and Debug Deployment Networking Isolated Resources Dedicated Connections Load Balancing Scalable DNSGlobal CDN The AWS Platform
  • 8. * As of 1 September 2017 2010 61 516 1,017 159 2012 2014 2016 AWS has been continually expanding its services to support virtually any cloud workload, and it now has more than 90 services that range from compute, storage, networking, database, analytics, application services, deployment, management, developer, mobile, Internet of Things (IoT), Artificial Intelligence (AI), security, hybrid and enterprise applications. AWS has launched a total of 795 new features and/or services year to date* - for a total of 3,708 new features and/or services since inception in 2006. AWS Pace of Innovation
  • 9. Strengthen your security posture Leverage security enhancements from 1M+ customer experiences Benefit from AWS industry leading security teams 24/7, 365 days a year Security infrastructure built to satisfy military, global banks, and other high- sensitivity organizations Over 50 global compliance certifications and accreditations “We work closely with AWS to develop a security model, which we believe enables us to operate more securely in the public cloud than we can in our own data centers.” Rob Alexander - CIO, Capital One
  • 10. Access a deep set of cloud security tools Virtual Private Cloud Isolated cloud resources Web Application Firewall Filter Malicious Web Traffic Shield DDoS protection Certificate Manager Provision, manage, and deploy SSL/TSL certificates Networking Key Management Service Manage creation and control of encryption keys CloudHSM Hardware-based key storage Server-Side Encryption Flexible data encryption options Encryption IAM Manage user access and encryption keys SAML Federation SAML 2.0 support to allow on-prem identity integration Directory Service Host and manage Microsoft Active Directory Organizations Manage settings for multiple accounts Identity & Management Service Catalog Create and use standardized products Config Track resource inventory and changes CloudTrail Track user activity and API usage CloudWatch Monitor resources and applications Inspector Analyze application security Artifact Self-service for AWS’ compliance reports Compliance
  • 11. More assurance programs than anyone Certifications / Attestations C5 [Germany], Cyber Essentials Plus [UK], DoD SRG, FedRAMP, FIPS, IRAP [Australia], ISO 27001, ISO 27017, ISO 27018, ISO 9001, MLPS Level 3 [China], MTCS Tier 3 [Singapore], PCI DSS Level 1, SEC Rule 17a-4(f), SOC 1, SOC 2, SOC 3 Laws, Regulations, and Privacy DNB [Netherlands], DPA – 1998 [U.K.], EAR, EU Data Protection Directive, EU Model Clauses, FERPA, Gramm-Leach-Bliley Act (GLBA), HIPAA, HITECH, IRS 1075, ITAR, My Number Act [Japan], PDPA – 2010 [Malaysia], PDPA – 2012 [Singapore], PIPEDA [Canada], Privacy Act [Australia], Privacy Act [New Zealand], Spanish DPA Authorization, VPAT / Section 508 Alignments and Frameworks CIS, CJIS, CLIA, CMS Edge, CMSR, CSA, EU-US Privacy Shield, FISC [Japan], FISMA, G-Cloud [U.K.], GxP (FDA CFR 21 Part 11), ICREA, IT Grundschutz [Germany], MITA 3.0, MPAA, NIST, PHR, UK Cloud Security Principles, Uptime Institute Tiers
  • 12. Global Enterprise Customers General Electric Capital One BMW Johnson & Johnson Merck Nordstrom “There is no public cloud infrastructure provider that has more robust enterprise capabilities.” Marc Benioff, Chairman & CEO, Salesforce
  • 13. AWS Positioned as a Leader in the Gartner Magic Quadrant for Cloud Infrastructure as a Service, Worldwide* AWS is positioned highest in execution and furthest in vision within the Leaders Quadrant *Gartner, Magic Quadrant for Cloud Infrastructure as a Service, Worldwide, Leong, Lydia, Petri, Gregor, Gill, Bob, Dorosh, Mike, August 32016 This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from AWS : http://www.gartner.com/doc/reprints?id=1-2G2O5FC&ct=150519&st=sb Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
  • 14. AWS Positioned as a Leader in the Gartner Magic Quadrant for Operational Database Management Systems* *Gartner, Magic Quadrant for Public Cloud Storage Services, Worldwide, Bala, Raj, Chandrasekran, 26 July 2016 This graphic was published by Gartner, Inc. as part of a larger research document and should be evaluated in the context of the entire document. The Gartner document is available upon request from AWS : http://www.gartner.com/doc/reprints?id=1-2IH2LGI&ct=150626&st=sb Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner's research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
  • 15. DBaaS report “AWS not only has the largest adoption of DBaaS, it also offers the widest range of offerings to support analytical, operational, and transactional workloads.” “AWS’s key strengths lay in its dynamic scale, automated administration, flexibility of database offerings, strong security, and high-availability capabilities, which make it a preferred choice for customers” The Forrester Wave™ is copyrighted by Forrester Research, Inc. Forrester and Forrester Wave™ are trademarks of Forrester Research, Inc. The Forrester Wave™ is a graphical representation of Forrester's call on a market and is plotted using a detailed spreadsheet with exposed scores, weightings, and comments. Forrester does not endorse any vendor, product, or service depicted in the Forrester Wave. Information is based on best available resources. Opinions reflect judgment at the time and are subject to change.
  • 16. The AWS Cloud Eliminate costly technical debt and reallocate resources so you can deliver high-value, revenue-generating projects faster. Innovate faster and solidify your competitive advantage by merging startup agility with enterprise experience and resources. Reduce risk by focusing resources dedicated to security, compliance and availability to the most important areas of your business. "AWS is our trusted partner that is going to run our company for the next 140 years.” Jim Fowler – CIO, General Electric
  • 17. What is a Large Scale Event?
  • 18. What is a Large Scale Event An event where you need more capacity than normally allocated for a period of time Typically from minutes to days, but could be a couple of weeks Often associated with a sudden surge of users Hard to architect and provision for at a reasonable cost Consumers get angry when it all goes wrong!
  • 24. What is a Large Scale Event? For you, it could be as simple as needing twice as much capacity for a short promotion Everyone’s Large Scale Event is different, but the underlying concepts are the same
  • 25. What Problems do you Face? Unknown infrastructure requirements • Cost? Short duration of the event • Massive investment in infrastructure that is otherwise idle or underutilized • Often tight deadlines to get the system live Legacy system integration Understanding system bahaviour, required metrics Getting the right architecture Finding the right talent
  • 29. How do we scale, especially the database?
  • 30. So let’s start from day one, user one ( you )
  • 31. Day One, User One A single EC2 Instance • With full stack on this host • Web app • Database • Management • Etc. A single Elastic IP Route53 for DNS EC2 Instance Elastic IP Amazon Route 53 User
  • 32. “We’re gonna need a bigger box” Simplest approach Can now leverage PIOPs High I/O instances High memory instances High CPU instances High storage instances Easy to change instance sizes Will hit an endpoint eventually x1.32xlarge m4.large t2.micro
  • 33. Day One, User One: We could potentially get to a few hundred to a few thousand depending on application complexity and traffic No failover No redundancy Too many eggs in one basket EC2 Instance Elastic IP Amazon Route 53 User
  • 34. Day Two, User >1 First let’s separate out our single host into more than one. Web Database • Make use of a database service? Web Instance Database Instance Elastic IP Amazon Route 53 User
  • 35. Start with the right databases for the job
  • 36. So decide wisely. Look for the key points of scale.
  • 37. User >100 First let’s separate out our single host into more than one. Web Database • Use RDS to make your life easier Web Instance Elastic IP RDS DB Instance Amazon Route 53 User
  • 38. User > 1000 Next let’s address our lack of failover and redundancy issues Elastic Load Balancing Another web instance • In another Availability Zone Enable Amazon RDS multi-AZ Web Instance RDS DB Instance Active (Multi-AZ) Availability Zone Availability Zone Web Instance RDS DB Instance Standby (Multi-AZ) Elastic Load Balancing Amazon Route 53 User
  • 39. User >10 ks–100 ks RDS DB Instance Active (Multi-AZ) Availability Zone Availability Zone RDS DB Instance Standby (Multi-AZ) Elastic Load Balancing RDS DB Instance Read Replica RDS DB Instance Read Replica RDS DB Instance Read Replica RDS DB Instance Read Replica Web Instance Web Instance Web Instance Web Instance Web Instance Web Instance Web Instance Web Instance Amazon Route 53 User
  • 40. This will take us pretty far honestly, but we care about performance and efficiency, so let’s clean this up a bit
  • 41. Shift Some Load Around Let’s lighten the load on our web and database instances Move static content from the web instance to Amazon S3 and CloudFront Move dynamic content from the Elastic Load Balancing to CloudFront Move session/state and DB caching to ElastiCache or DynamoDB Web Instance RDS DB Instance Active (Multi-AZ) Availability Zone Elastic Load Balancing Amazon S3 Amazon CloudFront Amazon Route 53 User ElastiCache Amazon DynamoDB
  • 42. User >500k+ Availability Zone Amazon Route 53 User Amazon S3 Amazon Cloudfront Availability Zone Elastic Load Balancing DynamoDB RDS DB Instance Read Replica Web Instance Web Instance Web Instance ElastiCache RDS DB Instance Read Replica Web Instance Web Instance Web Instance ElastiCacheRDS DB Instance Standby (Multi-AZ) RDS DB Instance Active (Multi-AZ)
  • 43. Time to make some radical improvements at the web & app layers
  • 44. SOAing Move services into their own tiers or modules. Treat each of these as 100% separate pieces of your infrastructure and scale them independently. Amazon.com and AWS do this extensively! It offers flexibility and greater understanding of each component.
  • 45. Loose Coupling Sets You Free! The looser they're coupled, the bigger they scale • Use independent components • Design everything as a black box • Decouple interactions • Favor services with built in redundancy and scalability than building your own Controller A Controller B Controller A Controller B Q Q Tight Coupling Use Amazon SQS as Buffers Loose Coupling
  • 46. Users > 1 Million RDS DB Instance Active (Multi-AZ) Availability Zone Elastic Load Balancer RDS DB Instance Read Replica RDS DB Instance Read Replica Web Instance Web Instance Web Instance Web Instance Amazon Route 53 User Amazon S3 Amazon Cloudfront Amazon DynamoDB Amazon SQS ElastiCache Worker Instance Worker Instance Amazon CloudWatch Internal App Instance Internal App Instance Amazon SES
  • 47. The next big steps
  • 48. From 5 to 10 Million Users You may start to run into issues with your database around contention on the write master. How can you solve it? Federation (splitting into multiple DBs based on function) Sharding (splitting one data set up across multiple hosts) Moving some functionality to other types of DBs (NoSQL)
  • 49. Database Federation • Split up databases by function or purpose • Harder to do cross-function queries • Essentially delays the need for something like sharding or NoSQL until much further down the line • Won’t help with single huge functions or tables ForumsDB UsersDB ProductsDB
  • 50. Sharded Horizontal Scaling • More complex at the application layer • ORM support can help • No practical limit on scalability • Operational complexity and sophistication • Shard by function or key space • RDBMS or NoSQL User ShardID 002345 A 002346 B 002347 C 002348 B 002349 A A B C
  • 51. Shifting Functionality to NoSQL Similar in a sense to federation Again, think about the earlier points for when you need NoSQL vs SQL Leverage hosted services like Amazon DynamoDB Consider these use cases: • Leaderboards and scoring • Rapid ingest of clickstream or log data • Temporary data needs (cart data) • “Hot” tables • Metadata or lookup tables Amazon DynamoDB
  • 52. From 5 to 10 Million Users You may start to run into issues with speed and performance of your applications Make sure you have monitoring, metrics, & logging in place • If you can’t build it internally, outsource it! (third-party SaaS) Pay attention to what customers are saying works well vs. what doesn’t, and use this as direction Try to work on squeezing as much performance out of each service or component
  • 55. Sizing for Peak Loads Promotions cause huge spikes in user activity Auto-scaling works for the web and middle tier RDS instances have to be sized for peak loads Adopted our recommendations in a staged approach
  • 56. Amazon Route 53 CloudFront Availability Zone #1 Amazon S3 Availability Zone #2 Amazon EC2Amazon EC2 Auto Scaling Geo Routing US East Amazon CloudWatch RDS DB Instance Active (Multi-AZ) RDS DB Instance Standby (Multi-AZ) User
  • 57. Amazon Route 53 CloudFront Availability Zone #1 Amazon S3 Availability Zone #2 Amazon EC2Amazon EC2 Auto Scaling Geo Routing US East User Amazon CloudWatch RDS DB Instance Active (Multi-AZ) RDS DB Instance Standby (Multi-AZ) RDS DB instance read replica
  • 58. Amazon Route 53 CloudFront Availability Zone #1 Amazon S3 DynamoDB Availability Zone #2 Amazon EC2Amazon EC2 Auto Scaling Geo Routing US East User Amazon CloudWatch RDS DB Instance Active (Multi-AZ) RDS DB Instance Standby (Multi-AZ) RDS DB instance read replica
  • 59. Amazon Route 53 CloudFront Availability Zone #1 Amazon S3 DynamoDB Availability Zone #2 Amazon EC2 ElastiCache Memcached Amazon EC2 Auto Scaling Geo Routing US East User Amazon CloudWatch RDS DB Instance Active (Multi-AZ) RDS DB Instance Standby (Multi-AZ) RDS DB instance read replica
  • 60. Amazon Route 53 CloudFront Availability Zone #1 Amazon S3 DynamoDB Availability Zone #2 Amazon EC2 ElastiCache (Redis Master) ElastiCache Memcached Amazon EC2 Redis Slave Auto Scaling Geo Routing US East User Amazon CloudWatch RDS DB Instance Active (Multi-AZ) RDS DB Instance Standby (Multi-AZ) RDS DB instance read replica Amazon Redshift
  • 61. Lessons Learned Listen to AWS Business Development and Solution Architects ;) Gaming promotions much easier to handle Unpredicted loads also easier to handle Senior operations person moving to a new game Customers get a much better gaming experience!
  • 63. Customer Success Stories Telecommunications Company iPhone 5s/5c, 6/6+, 7 and Samsung Note 3-8 and S4-8 Needed a system to handle a huge number of concurrent requests Failed previously at the iPhone5 launch Management directive to succeed at all costs!
  • 64. Telco Availability Zone Elastic Load Balancer Web Instance Web Instance Web Instance Web Instance Amazon Route 53 User Amazon S3 Amazon Cloudfront Amazon DynamoDB ElastiCache Amazon CloudWatch ElastiCache
  • 65. Great Success! Tested with 150,000 concurrent users All phones gone within 2 minutes No phones misallocated or unallocated Management said the system was too fast! Actual launch went smoothly
  • 66. Lessons AWS can provide infrastructure for applications to scale to very high concurrent users Managed services allow for quick deployment and changes to infrastructure Impossible for the customer to execute internally Massive cost savings, even with huge over provisioning New, improved system now developed for iPhone 8
  • 67. “With our systems on AWS, we can scale our resources more than 130-fold in 30 minutes, enabling us to support more than 2,500 orders per second” KT Chiu Founder and Chief Executive Officer TixCraft
  • 68. A Modern Data Architecture for Microservices
  • 69. What to Expect from the Session • Microservices at Amazon • Overview and Challenges • Key Elements and Benefits • Two Pizza Teams • Data Architecture Challenges • Transactions and Rollbacks • Streams • Master Data Management • Choosing a Data Store • Aggregation
  • 71. Microservices at Amazon Service-Oriented Architecture (SOA) Single-purpose Connect only through APIs Connect over HTTPS “Microservices”
  • 72. Monolithic vs. SOA vs. Microservices Microservices: Many very small components Business logic lives inside of single service domain Simple wire protocols(HTTP with XML/JSON) API driven with SDKs/Clients SOA: Fewer more sophisticated components Business logic can live across domains Enterprise Service Bus like layers between services Middleware
  • 73. Monolithic vs. SOA vs. Microservices SOA Coarse-grained Microservices Fine-grained Monolithic Single Unit
  • 74. Microservice Challenges Distributed computing is hard Transactions • Multiple Databases across multiple services Eventual Consistency Lots of moving parts Service discovery Increase coordination Increase message routing
  • 75. Key Elements of Microservices… Some core concepts are common to all services • Service registration, discovery, wiring, administration • State management • Service metadata • Service versioning • Caching Low Friction Deployment Automated Management and Monitoring
  • 76. Key Elements of Microservices… Eliminates any long-term commitment to a technology stack Polyglot ecosystem Polyglot persistence • Decompose Databases • Database per microservice pattern Allows easy use of Canary and Blue-Green deployments
  • 77. Key Elements of Microservices… Each microservice is: • Elastic: scales up or down independently of other services • Resilient: services provide fault isolation boundaries • Composable: uniform APIs for each service • Minimal: highly cohesive set of entities • Complete: loosely coupled with other services Controller A Controller B Controller A Controller B Q Q Tight Coupling Loose Coupling
  • 78. Microservices Benefits Fast to develop Rapid deployment Parallel development & deployment Closely integrated with DevOps • Now ”DevSecOps” Improved scalability, availability & fault tolerance More closely aligned to business domain
  • 79. Two-pizza teams Full ownership Full accountability Aligned incentives “DevOps” Principles of the Two Pizza Team
  • 80. How do Two Pizza Teams work? We call them “Service teams” Own the “primitives” they build: • Product planning (roadmap) • Development work • Operational/Client support work “You build it, you run it” Part of a larger concentrated org (Amazon.com, AWS, Prime, etc)
  • 82. Challenge: Centralized Database user-svc account-svccart-svc DB Applications often have a monolithic data store • Difficult to make schema changes • Technology lock-in • Vertical scaling • Single point of failure
  • 83. Centralized Database – Anti-pattern Applications often have a monolithic data store • Difficult to make schema changes • Technology lock-in • Vertical scaling • Single point of failure user-svc account-svccart-svc DB
  • 84. Decentralized Data Stores account-svccart-svc DynamoDB RDS user-svc ElastiCache RDS Polyglot Persistence Each service chooses it’s data store technology Low impact schema changes Independent scalability Data is gated through the service API
  • 85. Challenge: Transactional Integrity Polyglot persistence generally translates into eventual consistency Asynchronous calls allow non-blocking, but returns need to be handled properly How about transactional integrity? • Event-sourcing – Capture changes as sequence of events • Staged commit • Rollback on failure ERROR STATE? ROLLBACK?
  • 86. Best Practice: Use Correlation IDs 09-02-2015 15:03:24 ui-svc INFO [uuid-123] …… 09-02-2015 15:03:25 catalog-svc INFO [uuid-123] …… 09-02-2015 15:03:26 checkout-svc ERROR [uuid-123] …… 09-02-2015 15:03:27 payment-svc INFO [uuid-123] …… 09-02-2015 15:03:27 shipping-svc INFO [uuid-123] …… ui-svc catalog- svc checkout- svc shipping- svc payment- svc request correlation id: “uuid-123” correlation id: “uuid-123”
  • 87. Best Practice: Microservice owns Rollback Every microservice should expose it’s own “rollback” method This method could just rollback changes, or trigger subsequent actions • Could send a notification If you implement staged commit, also expose a commit function Microservice Function 1 Rollback Commit (optional)
  • 88. Event-Driven: DynamoDB Streams If async, consider event-driven approach with DynamoDB Streams Don’t need to manage function execution failure, DDB Streams automatically retries until successful “Attach” yourself to the data of interest Microservice
  • 89. Challenge: Report Errors / Rollback What if functions fail? (business logic failure, not code failure) Create a “Transaction Manager” microservice that notifies all relevant microservices to rollback or take action DynamoDB is the trigger for the clean-up function (could be SQS, Kinesis etc.) Use Correlation ID to identify relations mm-svc Transaction Manager Function DDB Streams API Call Error Table
  • 90. Challenge: Report Errors / Rollback ERROR DynamoDB Error Table Transaction Manager Function Kinesis Error Stream SQS Error Queue Rollback (correlation-id) Rollback (correlation-id) Rollback (correlation-id) Rollback (correlation-id)
  • 91. Challenge: Code Error Lambda Execution Error because of faulty code Leverage Cloudwatch Logs to process error message and call Transaction Manager Set Cloudwatch Logs Metric Filter to look for Error/Exception and call Lambda Handler upon Alarm state ui-svc Cloudwatch Logs Cloudwatch Alarm Transaction Manager Function
  • 92. Beware: Stream Model with AWS Lambda DynamoDB Streams and Kinesis streams directly work with AWS Lambda, however AWS Lambda needs to acknowledge processing the message correctly If Lambda fails to process the message, the stream horizon will not be moved forward, creating a “jam” Solution: Monitor AWS Lambda Error Cloudwatch Metric and react when error rate of same “Correlation ID” keeps increasing
  • 93. MDM – Keep Data Consistent Databases AWS Lambda “Cleanup” Function Cloudwatch Scheduled Event Perform Master Data Management (MDM) to keep data consistent Create AWS Lambda function to check consistencies across microservices and “cleanup” Create Cloudwatch Event to schedule the function (e.g. hourly basis)
  • 95. Storage & DB options in AWS Amazon RDS Amazon DynamoDB Amazon Elasticsearch Service Amazon S3 Amazon Kinesis Amazon ElastiCache In-Memory NoSQL SQL SearchObject Streaming Amazon Redshift Amazon Glacier
  • 97. Challenge: What Service to Use? Many problems can be solved with NoSQL, RDBMS or even in-memory cache technologies Non-functional requirements can help identify appropriate services Solution: Classify your organizations non-functional requirements and map them to service capabilities
  • 98. Determine Your Non-Functional Requirements Requirement Latency > 1s 200 ms -1s 20 ms – 200 ms < 20 ms Durability 99.99 99.999 99.9999 > 99.9999 Storage Scale < 256 GB 256 GB – 1 TB 1 TB – 16 TB > 16 TB Availability 99 99.9 99.95 > 99.95 Data Class Public Important Secret Top Secret Recoverability 12 – 24 hours 1 – 12 hours 5 mins – 1 hour < 5 mins Skills None Average Good Expert This is only an example. Your company’s classifications will be different There will be other requirements such as regulatory compliance too.
  • 99. Map Non-Functional Requirements to Services Service Latency Durability Storage Availability Recoverability from AZ Failure (RPO, RTO) RDS < 100 ms > 99.8 (EBS) 6 TB (SQL Server 16 TB) 99.95 0s and 90s (MAZ) Aurora < 100 ms > 99.9 64 TB > 99.95 0s and < 30s (MAZ) Aurora + ElastiCache < 1 ms > 99.9 64 TB > 99.95 0s and < 30s (MAZ) DynamoDB < 10 ms > 99.9 No Limit > 99.99 0s and 0s DynamoDB / DAX < 1 ms > 99.9 No Limit > 99.99 0s and 0s ElastiCache Redis < 1 ms N/A 3.5 TiB 99.95 0s and < 30s (MAZ) Elasticsearch < 200 ms > 99.9 150 TB 99.95 0s and < 30s (Zone Aware) S3 < 500 ms 99.999999999 No Limit 99.99 0s and 0s The information below is not exact and does not represent SLAs
  • 100. Finalizing Your Data Store Choices After mapping your non-functional requirements to services you should have a short list to choose from Functional requirements such as geospatial data and query support will refine the list further You may institute standards to make data store selection simpler and also make it easier for people to move between teams, e.g Redis over Memcached and PostgreSQL over MySQL. These can still be overridden, but require justification to senior management
  • 101. Challenge: Reporting and Analytics Data is now spread across a number of isolated polyglot data stores Consolidation and aggregation required Solution: Pull data from required microservices, push data to data aggregation service, use pub/sub, or use a composite service (anti-pattern).
  • 102. Aggregation usr svc Pull model Data Aggregation Application account svc cart svc Pull
  • 103. Aggregation usr svc Pull model Push model Data Aggregation Application account svc cart svc usr svc account svc cart svc Data Aggregation Application Push Pull
  • 104. Aggregation usr svc Pull model Push model Data Aggregation Application usr svc Data Aggregation Application Pub/Sub account svc cart svc account svc cart svc Pub Sub usr svc account svc cart svc Data Aggregation Application Push Pull
  • 105. Aggregation usr svc Pull model Push model Data Aggregation Application usr svc Data Aggregation Application Pub/Sub Composite Composite Data Service usr account cart account svc cart svc account svc cart svc Pub Sub usr svc account svc cart svc Data Aggregation Application Push Pull
  • 106. A Few Thoughts Use Non-Functional Requirements to help identify the right data store(s) for each microservice Use polyglot persistence to avoid bottlenecks, schema issues and allow independent scalability (and cache) Embrace eventual consistency and design fault-tolerant business processes which can recover Think ahead and plan your analytics requirements as part of the overall architecture
  • 107. Learn from our Customers
  • 108. Beware of Costs Many microservices with redundant, isolated data stores can blow out costs One customer in India with 300 microservices is now looking at costs reduction Primary, standby, read replicas and cache per microservice with databases using PIOPs storage Great performance, scale and resilience, but expensive
  • 109. Invest in Governance and Architecture Giving each team independence is empowering However, architects still need to understand the core components of the distributed system and enforce standards An Indonesian customer is changing to microservices now, but doesn’t have governance, architecture or standards in place Debugging distributed system is already proving complex Standard logging, error handing and oversight will help