The internet of things (beyond data management and analytics) jonathan cooper ellis

1© Cloudera, Inc. All rights reserved.
Jonathan Cooper-Ellis (JCE) // Solutions Architect
The Internet of Things:
Beyond Data Management & Analytics

Top Level Concepts in IoT
Core Platform / Cloud /
Information Technology (IT)
Edge
WAN
Operational
Technology (OT)

End-to-End IoT
Core Platform / Cloud / Information Technology (IT)
IoT Data Platform
•  Time Series Analysis
•  Machine Learning
•  Contextual Enrichment
•  Advanced Analytics
Core Platform
•  Cloud & On-Premise
•  Scalable Data Storage,
Processing, & Analytics
Cloud
Data Hub
Enterprise Data
Sources
IoT Hub
IoT Hub
•  Device Registry &
Management
•  Data & Control
Message Routing
Edge
Data Center
IoT Gateway
IoT Gateway
•  Edge Analytics,
Processing, &
Storage
•  Core Platform
Connectivity
Devices/Things
•  Sensors & Actuators
•  Low-powered compute
WAN

Complete IoT Stack
Multi-service Gateway
IoT Integration Platform
Data Management | Advanced Analytics
IoT / M2M Communication
3. Data acquisition & control
8. Information processing & integration
4. Data pre-processing
7. Device Management
9. Data Management
1. Data endpoints
5. Edge intelligence & analytics
2. Field communication
6. Back-end communication
11. Enterprise Applications & Services
10. BI & Advanced Analytics
Foundation/Security/Management
Enterprise IT & Business Applications

IoT Device
A computer integrated with the physical world
through one or more connected sensors or actuators.

Sensors
Sensors detect changes in the physical world.

Sensor Examples

Actuators
Actuators affect changes in the physical world.

Actuator Examples

Controller
Sensors/actuators are physically integrated with a computer, called a controller.
Ø  Sensors require a computer to collect the information they generate.
Ø  Actuators require a computer to control what they do.
Often (though not necessarily) the computers that sensors and actuators are
integrated with are very low powered, known as microcontrollers.

Controller/Microcontroller Examples

Controller + Sensor and/or Actuator = IoT Device

Field Communication
Connectivity technologies and communication protocols used by IoT devices to
exchange information within a remote local- or personal-area network
(LAN/PAN).

Edge Network
Multiple IoT devices communicating with each other over a local-area network
(LAN) or personal-area network (PAN) form an edge network.
This type of communication is commonly known as machine-to-machine (M2M),
and the enabling technology was the predecessor for modern IoT. Often this falls
under the category of Operational Technology (OT).
In the OT world, Supervisory Control and Data Acquisition (SCADA) systems are
extremely common.

Edge Network Example: SCADA System
HMI
SCADA
PLC
PLC

17�� Cloudera, Inc. All rights reserved.
Edge Network Example: Smart Home

Edge Network Example: Connected Car

LAN/PAN Connectivity
Short-range (~10cm–150m max. distance) connectivity technologies:
•  Ethernet
•  WiFi
•  RFID
•  Bluetooth / Bluetooth Low Energy (BLE)
•  ZigBee
•  Z-Wave
•  6LowPAN
•  Thread
•  NFC

Field Communication Protocols
OT:
•  OPC / OPC-DA / OPC-UA
•  Modbus
•  S7
•  BACnet
•  DDS
Automotive:
•  Controller Area Network (CAN)
Modern IoT:
•  MQTT
•  CoAP
And so many more…

IoT Gateway
A computer connected to one or more IoT devices, or being itself an IoT device,
capable of communicating with a core platform over a wide-area network (WAN).

IoT Gateway: Common Functions
In addition to communicating over a WAN, IoT gateways are often responsible for:
ü  Collecting data from local IoT devices
ü  Sending control messages to local IoT devices
ü  Routing messages to/from a core platform
ü  Local data storage
ü  Local data processing
ü  Filtering messages
ü  Edge analytics / intelligence

2 PB of data/car/ year 1 – 2 TB of data / day 1 – 5 TB of data / day
Why the obsession with edge analytics?
The WAN is always the bottleneck.

IoT Gateway: Edge Analytics
Traditionally, edge analytics has generally meant applying rules.
More recently, edge analytics has evolved to include ML model serving.
But wait, where do the models come from?!

Core Platform Training & Edge Serving
q  Insufficient resources at edge for
training (lacking storage & compute)
q  Inadequate network for streaming data
to core platform for serving
•  Bandwidth too low
•  Cost too high
•  Latency too high

IoT Gateway: Hardware Specifications
IoT gateway hardware requirements varies significantly depending on use-case.
Low Powered High Powered

Back-End Communication
Connectivity technologies and communication protocols used by IoT gateways to
exchange information with a core platform over a wide-area network (WAN).

WAN Connectivity
Long-range (~30km–10000km+ max. distance) connectivity technologies:
•  Cable / Fiber
•  Cellular (GSM / 3G / 4G / 5G / LTE / LTE Cat-M)
•  Satellite
•  Sigfox
•  LoRa
•  NB-IOT
•  LPWAN

Back-End Communication Protocols
Back-end communication protocols are relatively well standardized.
Messaging protocols:
•  AMQP
•  MQTT
•  Kafka
REST protocols:
•  HTTP
•  CoAP

IoT Hub
A core platform service (or collection or services) responsible for
device management and enabling secure communication between
IoT gateways and other core platform services.

IoT Hub: Common Functions
IoT hubs are generally responsible for:
ü  Maintaining a device registry
ü  Device management capabilities
ü  Ensuring secure bidirectional communication over an untrusted WAN
ü  Authentication
ü  Authorization
ü  Routing messages to/from other core platform services

Data Hub
The component within the core platform responsible for scalable storage,
processing, and analysis of IoT data (i.e. telemetry, audio, image, video) and
contextual data.

IoT Analytics
The goal of IoT analytics is generally to leverage a combination of IoT data and
contextual data to create actionable insights for both humans and machines.
Ø  Human intelligence
ü  Operations monitoring
ü  BI/reporting
Ø  Artificial intelligence
ü  Machine learning
ü  Automated decision-making

IoT Data Characteristics
IoT data is:
Ø  Mostly time-series, increasingly image/video/audio
Ø  High volume
Ø  Generated from a variety of data sources
Ø  Diverse data structures and schemas
Ø  Either in streams (real-time) or batches
Ø  Often perishable
Combining sensor data with contextual data is the key
to value creation from IoT.

Digital Twin
IoT technology enables the concept of a digital twin (aka device twin),
which is a virtual model of a physical asset.
The state of an asset (sensor readings) and its capabilities (actuators) are
exposed via APIs on top of an object-oriented programming model.
The virtual model can be used for analytics and simulations, and changes made to
it can be propagated to the physical world.

Digital Twin Example: Drones

IoT
Gateways
Data Mgt. and Analytics
IoT
Hub
End-to-End IoT Architecture: Functional
Application Data
Applications
Telemetry
Data
Management
OT Middleware
OT Middleware
Smart
Services
Machine
Learning
Business
Logic
Device
Management
Device
connectivity
Administration
Platform-as-a-Service
Ingest
Machine Learning
Store
Analyze
Process
Connected
“Things”
Telemetry
Data
Management Telemetry
Data
Application
Integration

IoT
Gateways
Data Mgt. and Analytics
IoT
Hub
Integrating IoT Operating Technology, Data Management, Analytics, and Applications
End-to-End IoT Architecture: Overview
Application Data
ApplicationsConnected
“Things”
Telemetry
Data
Application
Integration
Real-Time
Analytics
Data Ingest
Real-Time
Processing
Data Storage
Data Security
Machine
Learning
Telemetry
Data
Management
Telemetry
Data
Management
Deep data
analysis &
insights
Analytics Output
Analytics Output
Prediction / Alert
Actions
Protocol
Translation
Intelligent
Filtering Aggregation Routing

Thank you
Jonathan Cooper-Ellis (JCE)
jce@cloudera.com

The internet of things (beyond data management and analytics) jonathan cooper ellis

More Related Content

The internet of things (beyond data management and analytics) jonathan cooper ellis