Grokking TechTalk #33: Architecture of AI-First Systems - Engineering for Big Data & AI
- 1. Architectures of AI systems Engineering for Big Data & AI HCMC, Sep 6th 2019 herve@quod.aiHerve Roussel
- 3. Is this data engineering? UploadData.java upload_data.py
- 4. cat console.log | grep “ERROR” > errors.log Is this data engineering?
- 5. Data engineering? Transformed dataEvent data Program
- 7. cat console.log | grep “ERROR” > errors.log Is this data engineering? Event data Transform Transformed data
- 8. What is Big Data Engineering ?
- 9. Where is Big Data?
- 10. How to query news feed? SELECT * FROM posts INNER JOIN friends WHERE ... ORDER BY posts.timestamp DESC
- 11. Notify? Web, mobile? Who can see this? Racist? Vulgar? Is this a face? Who’s this? Friend? Celebrity? Courtney likes. Is that good or bad? Paddy commented. Is that good or bad? Chris posted. Is that good or bad? Anybody tagged? What rank in feed?
- 13. Is Big Data just for big companies? 300K QPS [R] 6K QPS [W] As of JULY 8, 2013 1B+ QPM [P] 250M+ QPM [R] 400M LOC [P] 1.8 TB per year [P]
- 14. Data Engineering Augmented dataEvent data Program
- 15. Event data Transform Augmented data Big Data Engineering + AI
- 16. Pipeline (Transform) Source (Event data) Sink (Augmented data)
- 17. What is a source ?
- 18. Synchronous_ ( 10-100 ms )_ Where is data coming from? Main data Event source Why split? Asynchronous_ ( 3-5 s )_
- 19. What’s in an event data? Post { id: 12345, content: “hello world”, created_at: … updated_at: … author_id: 67890, … } PostCreatedEvent { story_id: 12345, type: “story_posted” … }
- 20. Job 1 Job 2 Scheduler What’s batch processing?
- 21. Which DB for event source?
- 22. ● Volume? ● Velocity? QPS reads? QPS writes? ● Latency? ● Cost? Storage & R/W ● How to write? ○ Integrity? ○ Consistency? ○ Durability? ○ Version? ● How to read? ○ Random access or sequential? ○ Full text search? ○ Geo distance? How to store events?
- 23. MySQL MongoDB JSON on S3 (or GCS) 30 GB OK Good Very good 10K WPS OK Good Very good 1K RPS OK Good Very good Range read OK Good Very good Cost $$ $$$ $ MySQL MongoDB 30 GB OK Good 10K WPS OK Good 1K RPS OK Good Sequential read OK Good Cost $$ $$$ How to store events?
- 24. Who wants to become architect?
- 25. Job 1 Job 2 Scheduler What’s the problem with batch? LATENCY
- 26. How to process real-time? Stream processing
- 27. How can 2 processes talk?
- 28. QUEUE Why not use database?
- 29. Importance MySQL Kafka Redis 10K WPS 1.0 5 10 10 1K RPS 1.0 5 10 10 Sequential read 1.0 10 (with B-TREE) 10 10 (using Lists) Order guarantee 0.2 10 0 10 Durability 0.1 10 5 (but perf. hit) 0 Deployability 0.5 10 5 7.5 Score 5.6 / 10 6.6 / 10 7.15 / 10 Why not database?
- 30. What is a transform ?
- 32. Functional vs OOP Librarian .startShift() Catalog.open() Library.close() Books.create() Operations on things Add more things find(book) assign(book) Things with operations Add more operations remove(book) load_cover(book)
- 33. Functional vs OOP find_similar(vid_uploaded) transcribe_captions(vid_uploaded ) Things with operations Add more operations alert_subscribers(vid_uploaded) generate_thumbnails(vid_uploaded )
- 34. What’s supporting data? Transform Supporting data event { id: 12345, type: “story_posted” user_id: 67890 coordinates: [ 10.76, 106.66 ] } Friends or city DB
- 35. Who uses ext. supporting data?
- 36. API vs Pipeline: availability? Requests in thread Long running
- 37. API vs Pipeline: performance? 100ms ⇓ 10ms 100ms * 300,000/60/60 = 9H ⇓ 10ms * 300,000/60/60 = 55 min
- 38. Where is the data coming from? Is this a face? Who’s this? Friend? Celebrity?
- 39. Data pipelines & AI TransformAI model
- 40. How can 2 processes talk? Transform AI model
- 41. What is a sink ?
- 42. Which DB to sink to?
- 43. What to do with the sink? Write Read Data scientist Sales
- 44. What are the read use cases? Give me summary report of last month’s activity Give me posts that contain the words Donald Trump, Trump or President Give me all posts by female, age 18-35 Aggregation Full text search Bulk data, filtered
- 45. ACID
- 46. Denormalization: good or bad?
- 47. What is BCNF?
- 48. What’s distributed data systems?
- 49. Why re-run the pipeline? TransformAI model Transform v2
- 50. Idempotency & backfill f(f(x)) = f(x) POST “/BankAccount/AddFunds” { value: 1000, token: TX123 }
- 51. Another reason for backfill?
- 52. What if the AI model improves? TransformAI model v2
- 53. AI systems ≠ traditional systems? 93.2% ProbabilisticDeterministic
- 54. Store output of model v1 or v2? AI Model v1 ( accuracy: 83.1% ) AI Model v2 ( accuracy: ?? )
- 55. What have we learned ?
- 56. Source: Uber Engineering [DE] Collect data [DE] Process data [DS] Build DL model [BE/FE] Use DL model in app [DA] Validate DL model
- 57. Which NFR for Big Data? • Scalability • Availability • Interoperability • Portability • Modifiability • Maintainability • Testability • Usability • Buildability • Deployability • Ease of Development • Performance • Security • Localization • Legal • Reusability • Supportability • Monitorability
- 58. • Deployability • Ease of Development • Performance • Security • Localization • Legal • Reusability • Supportability • Monitorability Which NFR for Big Data? • Scalability • Availability • Interoperability • Portability • Modifiability • Maintainability • Testability • Usability • Buildability
- 59. Main data + Materialized view Event data ⇓ Pipeline ⇓ Augmented data What have we learned?
- 60. Want to learn more about AI & Big Data? We’re hiring: ● Big Data Engineer, in training (Java) ● Big Data Engineer (Java) ● Data Scientist (Python) http://bit.ly/quod-ai-join herve@quod.aiHerve Roussel