NoSQL meets Microservices - Michael Hackstein

www.arangodb.com
NoSQL meets Microservices
Michael Hackstein
@mchacki

Michael Hackstein
‣ ArangoDB Core Team
‣ Web Frontend
‣ Graph visualisation
‣ Graph features
‣ Host of cologne.js
‣ Master’s Degree 
(spec. Databases and 
Information Systems)
2

Responsibilities
5
LoadBalancer
Application Code
Consistency
Joins

app
Monolith breakdown
6
server.route({
method: "GET", path: "/app",
handler: function (req, reply) {
var sum = cpuWaste();
reply(dataLookup["key" + sum]);
}
});
var cpuWaste = function() {
var sum = Math.floor(Math.random() * 10000);
for (var j = 0; j < 1000000; ++j) {
sum += j; sum %= 10000;
}
return sum;
};
var dataLookup = {};
for (var j = 0; j < 500000; ++j) {
dataLookup["key" + j] =
"This is test text number " + j + ".";
}

memService
cpuService
app
Monolith breakdown
6
server.route({
}
});
for (var j = 0; j < 1000000; ++j) {
sum += j; sum %= 10000;
}
return sum;
};
for (var j = 0; j < 500000; ++j) {
}

memService
cpuService
app
Monolith breakdown
6
server.route({
}
});
for (var j = 0; j < 1000000; ++j) {
sum += j; sum %= 10000;
}
return sum;
};
for (var j = 0; j < 500000; ++j) {
}
server.route({
method: "GET", path: "/mem/{id}",
handler: function (request, reply) {
reply(dataLookup["key" + request.params.id]);
}
});

memService
cpuService
app
Monolith breakdown
6
server.route({
}
});
for (var j = 0; j < 1000000; ++j) {
sum += j; sum %= 10000;
}
return sum;
};
for (var j = 0; j < 500000; ++j) {
}
server.route({
}
});
server.route({
method: "GET", path: "/cpu",
reply(cpuWaste());
}
});

memService
cpuService
app
Monolith breakdown
6
for (var j = 0; j < 1000000; ++j) {
sum += j; sum %= 10000;
}
return sum;
};
for (var j = 0; j < 500000; ++j) {
}
server.route({
}
});
server.route({
method: "GET", path: "/cpu",
reply(cpuWaste());
}
});
server.route({
method: "GET",
path: "/app",
request('http://127.0.0.1:9000/cpu',
function (error, response, body) {
if (!error
&& response.statusCode == 200) {
request('http://127.0.0.1:9000/mem/'
+ body,
function (error, response, body) {
if (!error
&& response.statusCode == 200) {
reply(body);
} else {
reply({error: error});
}
});
} else {
reply({error: error});
}
});
}
});

Introduction to NoSQL
7
NoSQL World
Documents - JSON
Graphs
Key Value
{
“type“: "pants",
“waist": 32,
“length”: 34,
“color": "blue",
“material”: “cotton"
}
{
“type“: "television",
“diagonal screen size": 46,
“hdmi inputs": 3,
“wall mountable": true,
“built-in digital tuner": true,
“dynamic contrast ratio”: “50,000:1”,
Resolution”: “1920x1080”
}
{
“type": "sweater",
“color": "blue",
“size": “M”,
“material”: “wool”,
“form”: “turtleneck"
}
{
“type": "sweater",
“color": "blue",
“size": “M”,
“material”: “wool”,
“form”: “turtleneck"
}
‣ Map value data to unique string keys (identiﬁers)
‣ Treat data as opaque (data has no schema)
‣ Can implement scaling and partitioning easily
‣ Focussed on m-to-n relations between entities
‣ Stores property graphs: entities and edges can have
attributes
‣ Easily query paths of variable length
‣ Normally based on key-value stores (each document still
has a unique key)
‣ Allow to save documents with logical similarity in
“collections”
‣ Treat data records as attribute-structured documents
(data is no more opaque)
‣ Often allow querying and indexing document attributes

Polyglot Modeling
‣ If you have structured data
➡ Use a document store
‣ If you have relations between entities and want to eﬃciently query
them by arbitrary steps in between
➡ Use a graph database
‣ If you manage the data structure yourself and do not need
complex queries
➡ Use a key-value store
‣ If you have structured data in graph format, or data model might
change
➡ Use a multi-model database
8
Use the right tool for the job

Using different databases
9
LoadBalancer

Responsibilities
10
LoadBalancer
Application Code
Consistency
Joins

Solution 1: Writes only to one master
11

neo4jService
mongoDBService
productService
Polyglot Setup
13
MATCH (me:Users {id: "' + id + '"})
-[:Friend]-> (:Users)
-[:Bought]-> (p)
return p.id as id
products.find({$or: req.payload})

neo4jService
mongoDBService
productService
Polyglot Setup
13
MATCH (me:Users {id: "' + id + '"})
-[:Friend]-> (:Users)
-[:Bought]-> (p)
return p.id as id
products.find({$or: req.payload})
6 network hops
+ 2 queries

Live Demo
Neo4J + MongoDB
14
https://github.com/mchacki/ms-meets-nosql-examples

Multi Model Database
‣ Can natively store several kinds of data models:
‣ Key-value pairs
‣ Documents
‣ Graphs
‣ Delivers query mechanisms for all data models
‣ ACID transactions
‣ Cross collection joins
15

Using a Multi-Model Database
16
LoadBalancer
Application Code
Consistency
Joins

Using a Multi-Model Database
16
LoadBalancer
Application Code Consistency
Joins

Foxx
‣ Customized REST API on top of ArangoDB
‣ Microservice framework
‣ Integrate with other microservices
‣ Reuse your Node.js code and NPM modules
‣ Built-in authentication using OAuth2.0 or HTTP-Basic Auth
‣ Operations are encapsulated in the database
‣ low network traﬃc, direct data access
‣ increases data privacy
/
(~(
) ) /_/
( _-----_(@ @)
( /
/|/--| V
" " " "
17

foxx
productService
Foxx Setup
18
RETURN GRAPH_NEIGHBORS('ecom',
@id, {direction: 'outbound',
minDepth: 2, maxDepth: 2,
neighborCollectionRestriction: 'Products',
includeData: true
})
4 network hops
+ 1 query

Live Demo
Foxx
19
https://github.com/mchacki/ms-meets-nosql-examples

‣ open source and free (Apache 2 license)
‣ sharding & replication
‣ JavaScript throughout (V8 built into server)
‣ drivers for a wide range of languages
‣ web frontend
‣ good & complete documentation
‣ professional as well as community support
20
An overview of other features

Thank you
‣ Further questions?
‣ Follow me on twitter/github: @mchacki
‣ Write me a mail: mchacki@arangodb.com
‣ Join or google group: https://groups.google.com/forum/#!forum/arangodb
‣ Visit us at our booth
‣ Free T-shirts
21

NoSQL meets Microservices - Michael Hackstein

Related slideshows

More Related Content

NoSQL meets Microservices - Michael Hackstein