From Data To Insights

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
From Data To Insights
Orit Alul
Solutions Architect, Amazon Web Services

© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
What to expect from this session?
• The data architecture challenges
• Architectural principles
• Applying the architectural principles in practice
• Combining data and artificial intelligence
• Lake Formation Demo
• Summary

Business Monitoring
Business Insights
New Business Opportunity
Business Optimization
Business Transformation
Evolving Tools and Methods
AI/MLSQL Query

The Data Architecture Challenges
• Discovering the data
• Maintaining a short time-to-insight
• Analyzing the data by different personas
• Being cost efficient

Architectural Principles
• Build decoupled systems
• Data → Store → Process → Store → Analyze → Insights
• Use the right tool for the job
• Data structure, latency, throughput, access patterns
• Leverage managed and serverless services
• Scalable/elastic, available, reliable, secure, no/low admin
• Use log-centric design patterns
• Immutable logs (data lake), materialized views
• Be cost-conscious
• Big data ≠ big cost
• AI/ML enable your applications

Sounds good!
But, How do I practically apply those principles…?

Let’s build together!
Use case: smart analyzer for tweets

Use case: smart analyzer for tweets
• Our goal is to get smart insights on a stream of tweets related to a specific topic
• Get the general sentiment around a topic
• Get the highlights of a topic
• Enable data scientists to run queries
• Present the highlights in a simple graphical way
• Short time-to-insight
• Cost efficiency

Smart analyzer for tweets: accessories
• Tweepy - An easy-to-use Python library for accessing the Twitter API.
• How to scale sentiment analysis using Amazon Comprehend, AWS Glue and Amazon
Athena – blog post by Roy Hasson

The proposed architecture
Data → Store → Process → Store → Analyze → Answers

Amazon Kinesis Data Firehose
• Easily load streaming data into AWS
• Seamless elasticity
• Direct-to-data store integration
AMAZON S3
AMAZON REDSHIFT

Demo

AWS Glue - ETL Service
Data Catalog ETL Job authoring
Discover data and extract
schema
Auto-generates
customizable ETL code in
Python and Spark

AWS Glue Data Catalog
Central Metadata Catalog for the data lake
• Unified metadata repository
across relational databases,
Amazon RDS, Amazon Redshift,
and Amazon S3.
AWS Glue Data Catalog
Central Metadata Catalog for the data lake

Entities
Key Phrases
Language
Sentiment
Amazon
Comprehend
AMAZON COMPREHEND
Discover valuable insights from text

AI Services
ML Services
ML Frameworks + Infrastructure
P O L L Y T R A N S C R I B E T R A N S L A T E C O M P R E H E N D
& C O M P R E H E N D
M E D I C A L
L E X F O R E C A S TR E K O G N I T I O N
I M A G E
R E K O G N I T I O N
V I D E O
T E X T R A C T P E R S O N A L I Z E
Amazon SageMaker
F P G A SE C 2 P 3
& P 3 D N
E C 2 G 4
E C 2 C 5
I N F E R E N T I AG R E E N G R A S S E L A S T I C
I N F E R E N C E
D L
C O N T A I N E R S
& A M I s
E L A S T I C
K U B E R N E T E S
S E R V I C E
E L A S T I C
C O N T A I N E R
S E R V I C E
T H E A W S M L S T A C K
Broadest and deepest set of capabilities

Amazon Athena - Interactive Analysis
Interactive query service to analyze data in Amazon S3 using standard SQL
SQL
Query Instantly Pay per query Open Easy
$

Visualize with
Amazon QuickSight

Amazon QuickSight
Fast, easy to use, serverless analytics at 1/10th the cost of traditional BI
Empower
everyone
Seamless
connectivity
Fast analysis Serverless

Summary
üBuild decoupled systems
üUse the right tool for the job
üLeverage managed and serverless services
üUse log-centric design patterns
üBe cost-conscious
üAI/ML enable your applications

From Data To Insights

More Related Content

From Data To Insights