NodeJS ecosystem

3 HY NODEJS?
Little’s Law for Scalability and Fault Tolerance
L = λW
λ - Average request arrival rate W -Average time each request is processed by the system
L - Number of concurrent requests handled by the system
In our system, 1000 requests, on average, arrive each second
Each takes 0.5 seconds to process
How many requests does our system need to handle concurrently?
1000*0.5 = 500
1. If we know and we can figure out the rate of requests we can support
2.To handle more requests, we need to increase , our capacity, or decrease , our processing time, or latency

4 HY NODEJS?
Time
WEB CLIENT
SERVER
FOO
BAR
MICROSERVICES
HTTP SERVICE
•FOO and BAR, each take 500ms on average to return a response,
•Processing latency, is 1 second. That’s our
•We’ve allowed the web server to spawn up to 2000 threads (that’s now our )
How many requests can our system handle per second ?
λ = L/W = 2000/1 = 2000

5 HY NODEJS?
L is a feature of the environment (hardware, OS, etc.) and its limiting factors.
It is the minimum of all limits constraining the number of concurrent requests.
What Dominates the Capacity (L) ? A server can support several tens-of-thousands requests A server can support 100K to over a million concurrent requests Assuming a request size of 1 MB this can be over several hundreds-of- thousands requests Again, this can be over several hundreds-of-thousands requests So, L is somewhere between 100K and 1 million requests? Oh no no… Wait a minute… It usually has somewhere between 2K and 15K threads. With thread- per-connection model a server can serve < 20K requests
L = MIN(1,2,3,4,5 ) L is completely dominated by the number of threads the OS can support without adding latency

6 HY NODEJS?
Time
WEB CLIENT
SERVER
FOO
BAR
MICROSERVICES
HTTP SERVICE
•FOO and BAR, each take 500ms on average to return a response,
•Processing latency, is 1 second. That’s our
•We’ve allowed the web server to spawn up to 2000 threads (that’s now our )
If we call FOO and BAR parallely or asynchronously
How many requests can our system handle per second ?
λ = 2000/500 = 4000 per second

7 HY NODEJS?
��Even with multiple threads Context switching is not free
•Execution stacks(threads) take up memory
•Cannot use an OS thread for each connection
Problem 1
Problem 2
We do I/O which is or synchronous Typical blocking things:-
•Calls out to web services
•Reads/Writes on the Database

8 HAT IS NODEJS?
•Server side Javascript runtime
•It has Google Chrome’s V8 under the hood
•Nodejs.org: “Node.js is a platform built on Chrome’s JavaScript runtime for easily building fastnetwork applications. Node.js uses an , model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.”
Little’s Law...huh

9
NodeJS under the hood…
Callbacks + Polling Epoll, POSIX AIO,Kqueue …
Socket,http etc…
Open source JavaScript engine, platform Optimizer, JIT, inline caching, GC
Cross-platform asychronous I/O. Event Loop,Worker Thread Pool

ultithreaded Server | Architecture Summary
Synchronous, blocking I/O = simpler way of performing I/O. More threads because of the direct association to connections. This causes more memory and higher CPU usage due to more context switching among threads

inglethreaded Server | Architecture Summary
Event loop (main thread) at the front and asynchronous I/O at the kernel level. By not directly associating connections and threads, needs only a main event loop thread and (kernel) threads to perform I/O. Fewer threads , consequently yield , it uses less memory and also less CPU.

17
THE PAYPAL STORY
•On November 22, 2013 PayPal's engineering team posted on the PayPal engineering blog a post about PayPal's move from Java back end to Node.js back end for its web applications.
•The first web application to get the node treatment was the
•Two teams were building the same application with the exact same functionality one in Java and other in Javascript.
•The application contained three routes and each route made a between 2-5 API requests, orchestrated the data and rendered the page.

18
The PAYPAL STORY
•Double the requests per second vs. the Java application.
•35% decrease in the average response time for the same page. This resulted in the pages being served 200ms faster— something users will definitely notice.

19
THE BENEFITS Using JavaScript on both the front- end and the back-end removed an artificial boundary between the browser and server, allowing engineers to code both.
2.Built almost
3.Written in
4.Constructed with vs. the Java application. time for the same page

Wohoooooo………
Should I use NodeJS
for everything now?
No…

22
NODE.JS SCALES ON A PER PROCESSOR BASIS AS WELL AS ACROSS SERVERS.
Independently scale the subsystems A JavaEE 3-tier app is actually not written with Horizontal clustering in mind

23
Case Study - The Playfield
Reference: http://www.techferry.com/eBooks/NodeJS-vs-J2EE- Stack.html Two small applications simulating following use cases were written. One using Spring/Hibernate running on Tomcat with MySQL database at backend and another application using NodeJS with MySQL database at backend. Jmeter has been used to fire similar test load with varying load patterns to both the applications. 1.Use Case - Write a record : A module to insert a record into one single DB table and the backend returns a success/decline json response. 2.Use Case - Read a record : A module to read the same record by passing a query parameter and the backend returns the record as json string.

24
BENCHMARKING – CPU & MEMORY
Average CPU and memory usage were very low for NodeJS for serving same load pattern
WRITE A RECORD - User/Loops (100/1000)
READ A RECORD - User/Loops (100/1000)

25
BENCHMARKING – HITS PER SEC(Scales may differ)
NodeJs was able to server better hits per second smoothly over time
WRITE A RECORD - User/Loops (100/1000)

26
BENCHMARKING – Response Times vs Threads (Scales may differ)
NodeJs was able to server better hits per second smoothly over time.(Scales below are different)
READ A RECORD - User/Loops (100/1000)

27
Scenario
NODE JS
JAVA/J2EE
System Resource usage
NodeJS consistently uses comparatively very low CPU and Memory because of fewer threads used for processing
J2EE model scales with threads and hence it is CPU and memory hungry. CPU and Memory usage was comparatively very high
Simple URL based read/writes and increasing User Load
As the load and concurrency grow , NodeJS's scalability potential for lightweight operations improves
Since each request executes as thread, thread pool size can not be set infinite so with a given thread pool size requests will have to wait for their turn which work well on moderate load but performance degrades on extremely high Twitter kind of loads.
NodeJS vs JEE - Comparison Summary

28
Scenario
NODE JS
JAVA/J2EE
CPU Intensive operations
Should be very carefully chosen in this scenario.
Should be preferred due to multi threading capabilities that can execute such operations in parallel without blocking other requests.
Independent and Blocking Asynchronous I/O operations. Like mix of DB calls, Web Service calls, File Read/Write etc.
Such operations can be executed in parallel while NodeJS instead of waiting on blocking I/O operations can process more requests
Performance degrades on higher loads. All sub operations in each request will be performed sequentially even though they are independent. So initially it may scale up to some parallel threads or requests after which waiting time will keep most of the threads waiting (blocking resources)
NodeJS vs JEE - Comparison Summary

30
ECOSYSTEM 1-1
NODEJS (Upcoming)
JAVA/J2EE(Mature)
Language
Javascript
Java
Runtime
Node JS
JVM
Server/Middleware
Express.JS(Async)
J2EE compliant Web Server(Non Async)
Module Repository
Node Packaged Modules
Mostly Maven
Paas Support
Multiple…
•Cloudbees
•GoGrid
•Cloudcat
Full Stack
Javascript
all across
•No Java only stack
Libraries
MODULES Package.json
JARS
MANIFEST.MF

31
NODEJS (Upcoming)
JAVA/J2EE(Mature)
Unit Testing Frameworks
NodeUnit
Mocha(TDD) + Chai(assertion library)
Nock(Mock Http Services)
Junit
EasyMock
Build Tools
•Grunt
•node-ant (Apache Ant adapter )
Ant, Maven, Gradle
Configuration
config-js
Mostly XML based
Standardization of Code Structure
KrakenJS
Maven/ pattern based
Auto Restart
•forever
•Supervisor
•Needs to be explicitly scripted
ECOSYSTEM 1-1

32
NODEJS (Upcoming)
JAVA/J2EE(Mature)
Web MVC
Angular.js
Ember.js
Backbone.js
• Struts
•Spring MVC
•JSF
IOC Framework
Event loop and handlers yield an IOC
Spring
ORM frameworks
Support exists all across, but still nascent
•JPA
•Hibernate
•EJB Entity Beans
IDE Support
•Sublime Text
•Eclipse
•Eclipse
•IntelliJ
Debugging
•Cluster2 – Live Production Debugging
•node-inspector
•Eclipse v8 Plugin
•JDT
Profiling
•V8-profiler
•node -profiler
•JProfiler
•JMeter
ECOSYSTEM 1-1

33
ECOSYSTEM 1-1
NODEJS (Upcoming)
JAVA/J2EE(Mature)
Logging frameworks
Winston
Bunyan
Log4Js
log4j
logback slf4j
Monitoring
node-monitor
APPDynamics
Dynatrace, JMX console
API Support(Persistance, Cache, MQ/File System)
HTTP
File System
Socket.io
Redis
MongoDB
MySQL
PostgreSQL
Memcached
Cassandra
RabbitMQ
Riak
Oracle etc.. and counting…
Most backend systems

36
NODE IS GOOD FOR…
CHAT
JSON API on top of an object
based DB (such as MongoDB).
QUEUED
INPUTS
DATA STREAMING
PROXY
APPLICATION MONITORING DASHBOARD
BROKERAGE - STOCK TRADER’S DASHBOARD
SYSTEM MONITORING DASHBOARD

?
What about the Callback Challenges?

40
Can Node be used for Compute Intensive tasks?

41
COMPUTE INTENSIVE WITH NODE
*Courtesy: http://neilk.net/blog/2013/04/30/why-you-should-use-nodejs-for-CPU-bound-tasks/
Yohooo!!…Worker pools, Cluster, Web Workers to the rescue..

43
NODE.JS TRANSACTION/CONNECTION POOLING SUPPORT
Knex.js is a "batteries included" SQL query builder for Postgres, MySQL, MariaDB and SQLite3, designed to be flexible, portable, and fun to use. It features both traditional node style callbacks as well as a promise interface for cleaner async flow control, a stream interface, full featured query and schema builders, transaction support, connection pooling and standardized responses between different query clients and dialects.
Still unconquered
to the rescue…

45
WHY JAVA NIO FAILS
Java still has downsides compared to Node.js in that many core Java libraries are not non-blocking friendly (such as JDBC).
Async in Node.js is completely different. Everything is non-blocking, from file reads to database access

46
What about Design Patterns in Javascript?

51
BUSINESS BENEFITS
Motivation
•Fast Prototyping
•Continuous Delivery Productivity
•An Enterprise Scale Web Server In 5 Lines
•>80K Modules On NPM (Growing Ecosystem) Developer Joy More Happy, Works Better, More Developers Join Cost Savings
•Fewer Developers
•Smaller Iterations
•Less Hardware footprint

54
NODE IS GROWING FASTER THAN EVER

56
References
•http://blog.paralleluniverse.co/2014/02/04/littles- law/
•http://www.google.com/googlebooks/chrome/small_16.html
•http://blog.cloudfoundry.org/2012/06/27/future- proofing-your-apps-cloud-foundry-and-node-js/
•http://strongloop.com/node-js/infographic/

NodeJS ecosystem

Related slideshows

More Related Content

NodeJS ecosystem