Demystifying NoSQL - All Things Open - October 2020
- 1. DEMYSTIFYING NOSQL The SQL Developer's Guide Matthew Groves | Developer Advocate
- 2. 2 • Go to • pigeonhole.at/ATO2020 • Join Q&A • "Demystifying NoSQL" • Ask questions (anonymously if you want) Any questions?
- 4. 4Questions: pigeonhole.at/ATO2020 NoSQL is just a buzzword • Only moderately useful in discussion • Defines what something isn't • Not even accurate anymore • It does NOT mean anti-SQL
- 5. 01/ 02/ 03/ Why? NoSQL History Choose Your Own Adventure Summary/Next Steps AGENDA
- 7. 7Questions: pigeonhole.at/ATO2020 • Poor ad-hoc query capability • Low-level languages/tooling • Lack of interoperability Before relational
- 8. 8Questions: pigeonhole.at/ATO2020 • E.F. Codd invented the relational model • Designed to optimize for limited disk space • "Although it is logically unnecessary to store both a relation and some permutation of it, performance considerations could make it advisable." • Alpha Before SQL: Relational Databases
- 9. 9Questions: pigeonhole.at/ATO2020 • Created by Don Chamberlin & Raymond Boyce • Designed to be English-friendly • BCNF (Boyce-Codd Normal Form) • "SQL" and "relational" are now synonyms SQL
- 10. 10Questions: pigeonhole.at/ATO2020 • Scaling • Performance • Inflexibility / Impedance mismatch Criticisms/tradeoffs of SQL/relational
- 12. 12Questions: pigeonhole.at/ATO2020 Scaling The Free Lunch is Over (by Herb Sutter) http://www.gotw.ca/publications/concurrency-ddj.htm
- 13. 13Questions: pigeonhole.at/ATO2020 Impedance mismatch ID Username DateCreated 1 mgroves 2019-06-13 2 agroves 2019-06-14 . . . . . . CartID Item Price Qty 1 hat 12.99 1 1 socks 11.99 1 2 t-shirt 15.99 1 . . . . . . . . public class ShoppingCart { public int Id; public string Username; public List<Items> Items; } ShoppingCart ShoppingCartItems
- 15. 15Questions: pigeonhole.at/ATO2020 A relational database may be… Disclaimer!
- 17. 17Questions: pigeonhole.at/ATO2020 You're in a maze of twisty little passages, all alike • Types: Key/value, Document, Graph, Wide column • Data migration / synchronization • Scaling / high availability • ACID / transactions • Mobile • Kubernetes • Use Cases • JSON in Postgres/MySQL/etc • Something else I didn't think of • Why don't you demo something already? • Modeling: Document, Graph, Key/value, Wide table • Schema (or lack thereof) • Access Patterns: Key/value, Query, Search • Performance • Cloud / DBaaS / on-prem • Getting started • Options / vendors • Security • Summary
- 18. 18Questions: pigeonhole.at/ATO2020 Key/Value KEY1 { type: “json” } KEY2 <data type=“xml”></data> KEY3 Lorem ipsum … 00100101000100101 KEYn < Back
- 19. 19Questions: pigeonhole.at/ATO2020 Document Database key: person_5209 { "name": "Matt", "email": "me@mgroves.com", "favoriteFoods": [ "pizza", "cheesecake", "donuts" ] } key: person_5210 { "name": "Alice", "email": "alice@example.org", "age": 29, "favoriteFoods": [ "tofu", "quinoa", "Funyuns" ] } < Back
- 21. 21Questions: pigeonhole.at/ATO2020 Wide Table user_id name favoriteFoods age 5209 Matt ["pizza","cheesecake","donuts"] 5210 Alice ["tofu","quinoa","Funyuns"] 29 < Back
- 26. 26Questions: pigeonhole.at/ATO2020 Scaling: primary/secondary (write) primary secondaries DATA client DATA DATA
- 27. 27Questions: pigeonhole.at/ATO2020 Scaling: primary/secondary (read) primary secondaries DATA client
- 28. 28Questions: pigeonhole.at/ATO2020 Scaling: primary/secondary (failover) primary secondaries
- 29. 29Questions: pigeonhole.at/ATO2020 Scaling: shared nothing (write) DATA1 client DATA1 DATA1 MAP MAP (cache) DATA2 DATA2 DATA2
- 30. 30Questions: pigeonhole.at/ATO2020 Scaling: shared nothing (read) DATA client MAP MAP (cache)
- 31. 31Questions: pigeonhole.at/ATO2020 Scaling: shared nothing (failover) DATA DATA DATA DATA < Back
- 33. 33Questions: pigeonhole.at/ATO2020 Naïve NoSQL version // document 1 { "user": "Matthew Groves", "dateCreated": "2018-03-22T13:57:31.2311892-04:00", } // document 2 { "name": "widget", "price": 19.99, "quantity": 2, "cart": 1} // document 3 { "name": "sprocket", "price": 17.89, "quantity": 1, "cart": 1} // document 4 { "name": "doodad", "price": 20.99, "quantity": 5, "cart": 1}
- 34. 34Questions: pigeonhole.at/ATO2020 ACID/transactions { "user": "Matthew Groves", "dateCreated": "2018-03-22T13:57:31.2311892-04:00", "items": [ { "name": "widget", "price": 19.99, "quantity": 2}, { "name": "sprocket", "price": 17.89, "quantity": 1}, { "name": "doodad", "price": 20.99, "quantity": 5} ] }
- 36. 36Questions: pigeonhole.at/ATO2020 Mobile: Local { "user": "Matthew Groves", "dateCreated": "2018-03-22T13:57:31.2311892-04:00", "items": [ { "name": "widget", "price": 19.99, "quantity": 2}, { "name": "sprocket", "price": 17.89, "quantity": 1}, { "name": "doodad", "price": 20.99, "quantity": 5} ] }
- 37. 37Questions: pigeonhole.at/ATO2020 Mobile: Sync { "user": "Matthew Groves", "dateCreated": "2018-03-22T13:57:31.2311892-04:00", "items": [ { "name": "widget", "price": 19.99, "quantity": 2}, { "name": "sprocket", "price": 17.89, "quantity": 1}, { "name": "doodad", "price": 20.99, "quantity": 5} ] } { "user": "Matthew Groves", "dateCreated": "2018-03-22T13:57:31.2311892-04:00", "items": [ { "name": "widget", "price": 19.99, "quantity": 2}, { "name": "sprocket", "price": 17.89, "quantity": 1}, { "name": "doodad", "price": 20.99, "quantity": 5} ] } < Back
- 39. 39Questions: pigeonhole.at/ATO2020 Kubernetes in the Cloud GKE < Back
- 40. 40Questions: pigeonhole.at/ATO2020 Use Cases • Caching • Session • User profile • Catalog • Content management • Personalization • Customer 360 • IoT • Householding • Mainframe offloading • Communication • Gaming • Advertising • Travel booking/reservations • Loyalty programs • Fraud monitoring • Social media • Finance • E-Commerce • Inventory < Back
- 41. 41Questions: pigeonhole.at/ATO2020 Is Postgres the new NoSQL? ID Name Email Preferences 801 Matt me@mgroves.com { "language" : "en", "timeZone" : "EDT", "theme": "Dark" } 802 Alice alice@example.org { "language" : "sp", "timeZone" : "PDT", "theme": "Light" } 803 Bob bob@example.org { "language" : "en" } … … … … JSON_OBJECT() JSON_OBJECTAGG() JSON_ARRAY() JSON_ARRAYAGG() … etc …
- 42. 42Questions: pigeonhole.at/ATO2020 Is Postgres the new NoSQL?
- 43. 43Questions: pigeonhole.at/ATO2020 Is Postgres the new NoSQL?
- 44. 44Questions: pigeonhole.at/ATO2020 Is Postgres the new NoSQL? • Complete analysis and opinion: https://blog.couchbase.com/postgres- jsonb-and-nosql/ • Querying limits • Indexing • Tooling • Documentation • Don Chamberlain (co-creator of SQL) weighs in on SQL 2016 vs SQL++: http://bit.ly/comparingSql
- 45. 45Questions: pigeonhole.at/ATO2020 Is Postgres the new NoSQL? < Back
- 46. 46Questions: pigeonhole.at/ATO2020 Data Modeling: Relational
- 47. 47Questions: pigeonhole.at/ATO2020 Data Modeling: Document { "user": "Matthew Groves", "items": [ { "name": "widget", "price": 19.99, "quantity": 2}, { "name": "sprocket", "price": 17.89, "quantity": 1}, { "name": "doodad", "price": 20.99, "quantity": 5} ] }
- 48. 48Questions: pigeonhole.at/ATO2020 Data Modeling: Document { "user": "Matthew Groves", "blueCheckmark": false } { "userId" : "user::802", "text" : "Just signed up for twitter, it's pretty cool!", "dtTweeted" : "2019-08-10" } { "userId" : "user::802", "text" : "Hmm, I'm having my doubts about twitter", "dtTweeted" : "2019-08-11" } { "userId" : "user::802", "text" : "Twitter has ruined my life!", "dtTweeted" : "2019-08-12" } < Back
- 49. 49Questions: pigeonhole.at/ATO2020 Data Modeling: Graph https://neo4j.com/blog/data-modeling-basics/
- 50. 50Questions: pigeonhole.at/ATO2020 Data Modeling: Graph https://neo4j.com/blog/data-modeling-basics/
- 51. 51Questions: pigeonhole.at/ATO2020 Data Modeling: Relational https://neo4j.com/blog/data-modeling-basics/
- 52. 52Questions: pigeonhole.at/ATO2020 Data Modeling: Relation(al) vs Relation(ship) < Back
- 53. 53Questions: pigeonhole.at/ATO2020 Data Modeling: Wide table Relational Wide table http://bit.ly/cassandraModeling < Back
- 54. 54Questions: pigeonhole.at/ATO2020 Data Modeling: Key/Value Ideas for creating keys: • Natural • Human readable • Deterministic (compound key) • Semantic • Unique/random Example design for a blog: • author::matt • author::matt::blog • blog::csharp_9_features • blog::csharp_9_features::comments < Back
- 55. 55Questions: pigeonhole.at/ATO2020 Schema, or lack thereof { "name" : "Matt", "isVerified": false, "salary" : 50000 } { "name" : "Alice", "isVerified": true, "salary" : 100000, "favoriteFood" : "Funyunns" } document 1 document 2 { "name" : "Bob", "isVerified": false, "salaryUSD": 100000 } document 3
- 56. 56Questions: pigeonhole.at/ATO2020 Schema, or lack thereof
- 57. 57Questions: pigeonhole.at/ATO2020 Can I live without schema enforcement?
- 58. 58Questions: pigeonhole.at/ATO2020 Schema, or lack thereof
- 59. 59Questions: pigeonhole.at/ATO2020 Schema Validation in NoSQL < Back
- 60. 60Questions: pigeonhole.at/ATO2020 Access pattern: key/value Get Set (insert) Replace (update) Upsert Delete < Back
- 61. 61Questions: pigeonhole.at/ATO2020 Access pattern: Query (map/reduce)
- 62. 62Questions: pigeonhole.at/ATO2020 Access pattern: Declarative Query MongoDB query DynamoDB query CouchDB query ("Mango")
- 63. 63Questions: pigeonhole.at/ATO2020 Access Pattern: SQL++ (SQL for JSON) Couchbase (N1QL) CosmosDB SQL PartiQL
- 67. 67Questions: pigeonhole.at/ATO2020 Access pattern: Cypher/Gremlin Gremlin (CosmosDB) Cypher (Neo4j) < Back
- 68. 68Questions: pigeonhole.at/ATO2020 Access Patterns: Search (Full Text Search) ElasticSearch Couchbase (FTS) < Back
- 70. 70Questions: pigeonhole.at/ATO2020 Performance: "Benchmarks" • LOSERS • CHUMPS • US, OF COURSE
- 73. 73Questions: pigeonhole.at/ATO2020 • Cloud • DBaaS – Database as a Service • K8S service • VM • On-premises • K8S cluster • VM • Bare metal Cloud / DBaaS / on-premises
- 74. 74Questions: pigeonhole.at/ATO2020 Cloud / DBaaS / on-premises Database DBaaS K8S operator VM/bare Elasticsearch ✔️ ✔️ ✔️ MongoDB ✔️ ✔️ ✔️ Redis ✔️ alpha ✔️ Cassandra ✔️ ❌* ✔️ Microsoft Access 😕 😕 😕 DynamoDB ✔️ ❌ ❌ CosmosDB ✔️ ❌ ❌ Couchbase ✔️ ✔️ ✔️ Neo4j ✔️ ❌ ✔️ < Back
- 77. 77Questions: pigeonhole.at/ATO2020 http://bit.ly/proofconcept Proof of Concept < Back
- 83. 83Questions: pigeonhole.at/ATO2020 Security: I said don't panic! http://www.theregister.co.uk/2017/01/09/mongodb/ https://www.itworld.com/article/3115247/fairware-ransomware-infects-servers-through-exposed-redis- instances.html
- 84. 84Questions: pigeonhole.at/ATO2020 Security: I said don't panic! http://www.zdnet.com/article/elasticsearch-ransomware-attacks-now-number-in-the- thousands/ https://www.zdnet.com/google-amp/article/unsecured-mongodb-databases-expose-kremlins-backdoor-into-russian-businesses/
- 85. 85Questions: pigeonhole.at/ATO2020 Security: Unsafe by default? < Back
- 86. 86Questions: pigeonhole.at/ATO2020 Something else I didn't think of < Back
- 88. 88Questions: pigeonhole.at/ATO2020 It all started with scaling… Vertical Horizontal
- 89. 89Questions: pigeonhole.at/ATO2020 NoSQL is a lousy buzzword
- 90. 90Questions: pigeonhole.at/ATO2020 Think about the "why"
- 92. THANK YOU