The document provides an introduction to Node.js basics including how its asynchronous and event-driven architecture saves I/O costs compared to traditional multi-threaded servers, an overview of its usage in production by companies like LinkedIn and Yahoo, and how to organize code in Node.js using classes, modules, npm, and the EventEmitter utility class.
Report
Share
Report
Share
1 of 65
More Related Content
Intro2 nodejs 2pm
1. Welcome!
Register to Cloud Foundry:
Go to http://www.cloudfoundry.com
• Click Register - link
• Use Promo-code: cloudtoday
Download Node.js
• Go to http://nodejs.org/
Getting Started:
Go to http://www.cloudfoundry.com
• Click “Get Started” menu
1
3. Agenda
1. About Node.js
• Internal working of Node.js
• Buzz around Node.js
• Who is using it
• What kind of apps are being built
2. Coding in Node.js
• Sync v/s Async coding (Callbacks)
• Classes & Modules (CommonJS)
• npm & package.json
• Node.js EventEmitters
3. Node.js & Cloud Foundry (w/ demo)
• Hello World app in Cloud Foundry
• Using Sticky Sessions
• CloudFoundry Module & connecting to Redis, MongoDB etc.
• Express.js (RESTful) app
• Socket.io + Express.js (Real-time) app
3
4. What is Node.js
Node.js is a platform to build fast and scalable network applications. It is
built on Google Chrome‟s v8 engine & implements event-driven, non-
blocking I/O model.
- It is ~80% C/C++ & ~20% JS (APIs)
- Uses CommonJS module system.
- Executes JavaScript on the server
- Built by Ryan Dahl
- Sponsored by Joyent
Ryan Dahl
(Node.js creator)
4
6. What is the biggest advantage of Node.js?
Biggest thing Node.js brings to the table
(other than JS, of course) is savings in I/O cost
6
7. The cost of I/O
http://blog.mixu.net/2011/02/01/understanding-the-node-js-event-loop/
7
8. So how does Node.js save I/O cost?
Node.js saves I/O cost by implementing
event driven, Non-blocking I/O model
8
9. Event-driven, non-blocking I/O platform/server
What exactly is a event-driven, non-blocking server? How is it
different from a multi-threaded server?
Multi-threaded blocking server
v/s
Event-driven, non-blocking server
9
10. Multi-threaded server - Threads are spawned for every connection
User1 i/o request
T1
DB
User2 i/o request T2
Blocking
I/O
User3 T3 T4 T5
Refreshes 2 times
User4 T7 T8 FS
T6 T9
refreshes 3 times
Multi threaded
server
T Thread
Because every I/o is blocking, server
spawns a thread in a thread-pool to support
multiple requests
10
11. Non-blocking & Evented I/O (Node.js server)
T1
User1 i/o request V8 DB
T1
T1
V8 POSIX
Event loop V8
Async
User2 i/o request
Threads
T1 delegate i/o to Non-blocking
t2 I/O
V8 libeio t1
User3
Refreshes 2 times Single t4
thread t3
serves T1
all users V8
User4
refreshes 3 times T1 FS
V8
i/o result returned
T1 T1
2 EL after x time
V8 T1 V8
T1 V8 Thread running V8
V8 JS code (Single threaded)
Node.js
t1 POSIX threads doing
t2
async I/O (multi-threaded)
t3 Everything except your (JS) code is runs in parallel (by libuv)
11
12. Event-driven, non-blocking I/O server
Real-world example of the two designs?
Multi-threaded blocking server (Apache)
v/s
Event-driven, non-blocking server (Nginx)
12
13. Apache V/s Nginx: performance
Reqs/sec v/s concurrent connections
At ~4000 concurrent connections,
- Nginx can serve ~9000 reqs/sec
- Apache can serve ~3000 reqs/sec
Ref: http://blog.webfaction.com/a-little-holiday-present
13
17. Other things going on for Node.js
8,000+ libraries/modules/servers
High-level library categories
Web frameworks Oracle Multiple protocols Command Line Option Parsers
Routers NoSQL and Key/Value HTTP Parser Generators
Mongo FTP Other Parsers
Static file servers Debugging / Console Utilities
Microframeworks Hive E-mail
Compression
Redis XMPP
Frameworks Graphics
CouchDB Other networking Sound
Middleware
Other NoSQL implementations RPC Payment Gateways
JSGI Miscellaneous and multiple DB Web Sockets & Ajax API clients
Connect Templating Message Queues Control flow / Async goodies
Other middleware CSS Engines Class systems I18n and L10n modules
Other Content Management Systems Testing / Spec Frameworks Boilerplates
Database Build and Deployment Wrappers Continuous Integration Tools
MS SQL Server Package Management Systems Parsers DDD, CQRS, EventSourcing
Module Loader JSON Desktop application related
PostgreSQL JavaScript threads
MySQL OpenSSL / Crypto / Hashing XML
Other
SQLite SMTP
TCP / IP
https://github.com/joyent/node/wiki/modules
17
18. Other things going on for Node.js
Node in Production!
• LinkedIn, Yahoo!, Yammer, eBay, Twitter etc.
• >1000 other companies/startups are using it in production
All kinds of interesting apps:
End-user apps:
• Real-time apps
• Mobile apps
• Web sites
• Hardware programming (robots!)
Platform apps (Servers / Services):
• Node-http-proxy - Node.js implementation of reverse proxy like nginx
• LdapJS.org – - Node.js implementation of LDAP server itself
• SMTP – Node.js implementation of SMTP server itself
• XMPP, SSH, RPC, many more.
18
19. Agenda – part 2
1. About Node.js
• Internal working of Node.js
• Buzz around Node.js
• Who is using it
• What kind of apps are being built
2. Coding in Node.js
• Sync v/s Async coding (Callbacks)
• Classes & Modules (CommonJS)
• npm & package.json
• Node.js EventEmitters
3. Node.js & Cloud Foundry (w/ demo)
• Hello World app in Cloud Foundry
• Using Sticky Sessions
• CloudFoundry Module & connecting to Redis, MongoDB etc.
• Express.js (RESTful) app
• Socket.io + Express.js (Real-time) app
19
20. Let‟s look at the code..
How does async code differ from sync(regular) code?
Synchronous code
v/s
Asynchronous Code
20
21. Callbacks – Control flow
Use case: Let‟s say we have an item‟s id and want to get its name from DB and print it
//Synchronous & blocking code //Async & non-blocking code
function getItemNameById(id) { function getItemNameById(id, callback) {
//blocks or waits for DB db.get(id, callback); //step 2
return db.get(id); //step 2 //nothing is returned here
} }
var name = getItemNameById(100); //step 1 //step 3
Some internal function calls the callback w/ result
//print name in step 3
console.log(name); //step 3 //You create a callback helper function
function displayHelperCallback(name) {
console.log(name); //step 4
}
Things to note: //pass callback function to consume the result
1. Async code doesn‟t directly „return‟ getItemNameById(100, displayHelperCallback); //step 1
anything
2. Instead, it takes a
function(callback) & calls that
function when result becomes
available
21
22. Callbacks – Control flow (detailed version in Node.js)
//YOUR APP
var db = require(„db‟);
function getItemNameById(id, callback) {
db.get(id, callback); //step 2
}
//You create a callback helper function
function displayHelperCallback(name) {
console.log(name); //step 103
}
//pass callback function to consume the result //step 5
getItemNameById(100, displayHelperCallback); //step 1 V8 is free to run other functions in the event-
loop.
//INTERNALS OF A DB LIBRARY (HIGH LEVEL)
//step 5, step 6 ..step 100
function db() { Say v8 notices 95 other things to do (in the
this.dbConnection = net.connection(); // connects to DB
event loop), it starts executing them one by
}
db.protorype.get = function(id, callback) { one.
var self = this; //step 3 & //step4 is dbConnection.read (below)
this.dbConnection.read(id, function(result, callback) { Step 5 At some point b/w step 3 and step 100,
self. receiveFromDB(result, callback);//step 101 returns result & asks to run
}); dbConnection.write‟s callback.
}
db.protorype.receiveFromDB = function(result, callback) { This event goes to the back of the queue as
callback(result); //Execute callback step step 102 step 101
}
22
24. JavaScript Classes (util.inherits)
Node.js provides handy util.inherits function to inherit a class.
- This also provides „subclass.super_‟ to access super class‟ functions
var require(„util‟); //import util module
//Super Class
function Automobile(license, model) {
this.license = license;
this.model = model;
}
Automobile.prototype.getModel = function() {
return model;
}
//Sub class
function Car(license, model) {
Automobile.call(this, license, model);
}
util.inherits(Car, Automobile);
24
25. CommonJS modules
//Automobile.js file Things to note:
function Automobile(license, model) { 1. Allows keeping JS code in
this.license = license;
separate files
this.model = model;
}
2. Use “exports.<name>” to export
Automobile.prototype.getModel = function() { something
return model;
} 1. Use require(„path/to/module‟) to
exports.Automobile = Automobile; import it
2. use require(„module‟).<name> to
//Car.js file
access things inside module
var util = require('util');
var module = require('./Automobile');
var Automobile = module.Automobile;
function Car(license, model) {
Automobile.call(this, license, model);
}
util.inherits(Car, Automobile);
console.log(new Car("1232", "BMW").model); //prints BMW
25
26. CommonJS modules: Exporting multiple things
//myModule.js file Things to note:
exports.myFunction = function () { 1. You can directly export
return „hi there‟;
function, arrays, variables
}
exports.myArray = [„foo‟, „bar‟];
2. You can export multiple things
exports.myVariable = „I‟m a variable‟; from one file using „exports‟
//app.js file
var myModule = require('./myModule');
console.log(myModule.myFunction()); //prints „‟hi there‟
console.log(myModule.myArray[1]); //prints „bar‟
console.log(myModule.myVariable); //prints I‟m a variable‟
26
27. CommonJS modules: „exports‟ v/s „module.exports‟
//myModule.js file Things to note:
module.exports = function () { If you want to export only one
return „hi there‟;
class/function.. so that it can be
}
used directly by the
recipient, you can use:
module.exports = <something>;
Warning:
If you use both module.exports
//app.js file
var myFunction = require('./myModule');
and exports.bla, exports.bla will
NOT be exported(ignored)
console.log(myModule.myFunction()); //prints „‟hi there‟
27
28. Installing external modules – npm (Node Package Manager)
Use npm (Node Package Manager) to install modules
npm install <moduleName>
e.x. npm install express
Modules are copied into ./node_modules folder
/myapp
/myapp/node_modules/express
Things to note:
1. npm = Node Package Manager
2. It is a CLI to install modules from http://search.npmjs.org
3. LOCAL: npm install express
1. It installs in myapp/node_modules/express
4. GLOBAL: npm install express -g
1. It installs in /usr/local/lib/node_modules/ (default)
2. Installs executable files in /usr/local/.bin (default)
5. Use GLOBAL when the library has some shell script & want to reuse it
for different apps
28
29. package.json – Describe app & dependencies in a file
//package.json Things to note:
1. If you use package.json, you can
{ simply do:
npm install (w/o any module
"name": ”MyApp",
names)
"description": ”My awesome twitter app",
"version": "2.5.8", 1. Keep package.json in root
"author": ”Raja <rajar@vmware.com>", directory
"dependencies": {
"express": “2.3.4”, 2. Using package.json is preferred
"mime": "", over individual npm install
"connect-redis": ">= 0.0.1" <module>
}
1. You need to have all the modules
} pre-installed (i.e. npm install)
before uploading your app to
Cloud Foundry
29
30. Node.js EventEmitter (A utility class that allows emitting events)
EventEmitter class implements Observer pattern and provides on and emit
APIs
- It is used when creating (not using) an async library.
Node.js EventEmitter
30
31. Events – Node.js EventEmitter (A node.js utility class that allows emitting events)
//Simplified EventEmitter (Observer pattern)
function EventEmitter() {
//store events and callbacks like {event1: [callback1, callback2] , event2 : [cb3,
cb4]…}
this.eventNameAndCallbackList = {};
}
//Allow others to add a callback(function) for a event name(string)
EventEmitter.prototype.on = function(eventName, callback) {
//add eventName and callback to eventNameAndCallbackList
};
//When an event is emitted, call each callbacks in a loop
EventEmitter.prototype.emit = function(eventName) {
for(var i =0; i < currentCallbacks.length ; i++) {
currentCallbacks[i](); //call each callback
}
};
exports.EventEmitter = EventEmitter;
31
32. Events – Node.js EventEmitter (continued)
Say you are writing an I/O library & writing readFromDB function but don‟t know how to handle
async DB result.
You can solve it by..
1. Inheriting your class from EventEmitter
2. Then you can use its „emit‟ function to an event when data arrives (asynchronously)
3. You ask people who‟ll be using your library to implement „on‟ function
//myIOModule.js
var util = require('util');
var events = require('events');
//myIOClass is a subclass of events.EventEmitter class
var MyIOClass = function () {
events.EventEmitter.call(this);
};
util.inherits(MyIOClass, events.EventEmitter);
MyIOClass.prototype.readFromDB = function(query){
// <--reads data code here -->
this.emit('data', "some data");
}
exports.MyIOClass = MyIOClass; //export the class
32
33. Events – Node.js EventEmitter (continued)
Say you are an end-user trying to use DB library to read result from DB..
1. You‟ll have to implement „on‟ function for the given event name („data‟) and set a callback
2. DB libraries internal function will call your callback when the result comes back
//app.js
var myIOModule = require('./myIOModule');
var myIOClass = new myIOModule.MyIOClass();
myIOClass.on('data', function (data) {
console.log(data);
});
myIOClass.readFromDB('select * from users');
33
34. Events – A library can emit multiple events
I/O libraries usually emit multiple events..
connected, disconnected, error, ready, data, result etc.
//So you can listen to all of them..
function myFunction() {
db.on(„error‟, function(e) {
console.error(e);
});
db.on(„connect‟, function() { //db is connected
db.query(user);
});
db.on(„disconnect‟, function(){
console.log(„db disconnected‟);
});
db.connect(„127.0.0.1‟, „100‟);
}
34
35. Events – Error/Exception handling
//Say there was an exception trying to connect to db.
Function () {
try {
db.connect(„127.0.0.1‟, „4000‟); // failed to connect; connectionException
} catch (e) {
console.error(e);
}
}
Above try/catch won‟t handle it because the act of connection itself is an i/o
//Say there was an exception trying to connect to db.
Function () {
//Typically I/O libraries triggers „error‟ event (or callback). We‟ll need to listen to that
event
db.on(„error‟, function(e) {
console.error(e);
});
db.connect(„127.0.0.1‟, „100‟); // failed to connect; connectionException
}
35
36. Agenda – part 3
1. About Node.js
• Internal working of Node.js
• Buzz around Node.js
• Who is using it
• What kind of apps are being built
2. Coding in Node.js
• Sync v/s Async coding (Callbacks)
• Classes & Modules (CommonJS)
• npm & package.json
• Node.js EventEmitters
3. Node.js & Cloud Foundry (w/ demo)
• Hello World app in Cloud Foundry
• Using Sticky Sessions
• CloudFoundry Module & connecting to Redis, MongoDB etc.
• Express.js (RESTful) app
• Socket.io + Express.js (Real-time) app
36
37. Cloud Foundry
Cloud Foundry – Open Platform as a Service
37
38. Cloud Foundry open Platform as a Service
The PaaS of choice for the Cloud era
Simple
• Let‟s developers focus on their code and not wiring middleware
Open
• Avoid lock-in to specific cloud, frameworks or service
• Completely open source from day one
Flexible and Scalable
• Self service, deploy and scale your applications in seconds
• Extensible architecture to “digest” future cloud innovation
38
43. Hello World App on Cloud Foundry
//Simple http server on localhost //Simple http server on Cloud Foundry
var http = require('http'); var http = require('http');
http.createServer(function (req, res) { var host = process.env.VCAP_APP_HOST ||
res.writeHead(200, {'Content-Type': 'text/plain'}); „localhost‟;
res.end('Hello Worldn'); var port = process.env.VCAP_APP_PORT || „3000‟;
}).listen(3000, '127.0.0.1'); http.createServer(function (req, res) {
console.log('Server running at 127.0.0.1:3000'); res.writeHead(200, {'Content-Type': 'text/plain'});
res.end('Hello Worldn');
}).listen(port, host);
console.log('Server running at ‟ + host + “:” + port);
Things to note:
1. Cloud Foundry will provide host and port info in process.env variable
2. You can use https to access your app (up to nginx; http w/in CF)
3. Save your app as app.js or server.js to automatically use that as main node.js
43
45. “cloudfoundry” NodeJS helper module
npm install cloudfoundry
var cloudfoundry = require(„cloudfoundry‟);
cloudfoundry.cloud //is running on Cloud Foundry?
cloudfoundry.host // host
cloudfoundry.port // port
//Example: Say you are using „test-mongodb‟ MongoDB service, you can get its info:
cloudfoundry.mongodb['test-mongodb'].credentials.hostname
cloudfoundry.mongodb['test-mongodb'].credentials.port
cloudfoundry.mongodb['test-mongodb'].credentials.db
cloudfoundry.mongodb['test-mongodb'].credentials.username
cloudfoundry.mongodb['test-mongodb'].credentials.password
Things to note:
1. Cloudfoundry module (built by „igo‟) provides easy access to environment variables
2. For more, please go through https://github.com/cloudfoundry-samples/cloudfoundry
45
46. “cloudfoundry” NodeJS helper module
npm install cloudfoundry
var cloudfoundry = require(„cloudfoundry‟);
cloudfoundry.cloud //is running on Cloud Foundry?
cloudfoundry.host // host
cloudfoundry.port // port
//Example: Say you are using „test-mongodb‟ MongoDB service, you can get its info:
cloudfoundry.mongodb['test-mongodb'].credentials.hostname
cloudfoundry.mongodb['test-mongodb'].credentials.port
cloudfoundry.mongodb['test-mongodb'].credentials.db
cloudfoundry.mongodb['test-mongodb'].credentials.username
cloudfoundry.mongodb['test-mongodb'].credentials.password
Things to note:
1. Cloudfoundry module (built by „igo‟) provides easy access to environment variables
2. For more, please go through https://github.com/cloudfoundry-samples/cloudfoundry
46
47. MongoDB – Example of inserting a user (using mongodb-native module)
var mongodb = require('mongodb').Db;
var conn; // holds connection
//connect to db and get connection obj
//connectionUrl looks like mongodb://username:pwd@host:port/dbName
mongodb.connect(connectionUrl, function(err, connection) {
conn = connection;
});
//add a user
function addUser (userObj, callback) {
//Get the collection that holds users
conn.collection('users', function (err, userCollection) {
//insert user to this collection
userCollection.insert(userObj, {safe:true}, function(err) {
callback(userObj);
});
});
}
//PS: Error handling is not shown
47
48. Demo app
Things to note:
1. Simple MongoDB demo app, adds random users and pulls existing users
2. https://github.com/rajaraodv/mongoapp1
48
50. Hello World App using ExpressJS
var host = process.env.VCAP_APP_HOST ||
'localhost';
Things to note:
var port = process.env.VCAP_APP_PORT || '3000'; 1. ExpressJS is Ruby Sinatra
inspired web framework
var express = require('express');
var app = express.createServer();
2. It is built on top of „Connect‟ –
app.listen(port, host); which itself is a wrapper for
http-module
3. It provides support for:
1. Dev/Prod Configurations
2. Routes
3. Templating
4. Sessions
5. Many, many other
features
50
51. Hello World App using ExpressJS (Middlewares)
var express = require('express');
var app = express.createServer();
//Middlewares
app.use(express.logger()); //logs requests
app.use(express.static(__dirname + „/public‟)); //sets location of public files
app.use(express.bodyParser()); //parses HTTP POST body
Things to Note:
1. Middlewares are functions that helps in common tasks involved
building in web applications
2. They are actually connect-module functions but exposed by
ExpressJS for simplicity
51
52. Hello World App using ExpressJS (Environments)
var express = require('express');
var app = express.createServer();
app.configure('development', function() {
//On error, print exceptions to console & to the web-page itself
app.use(express.errorHandler({ dumpExceptions: true, showStack: true }));
});
app.configure('production', function() {
//On error, this simply says „Internal error Occurred‟
app.use(express.errorHandler({ dumpExceptions: true, showStack: false }));
app.use(express.logger()); //logs requests
});
Things to Note:
1. Express uses NODE_ENV environment variable to set working
environment
2. On CF, to toggle b/w „development‟ and „production‟, you can use..
vmc env-add <appName> NODE_ENV „production‟
52
53. Hello World App using ExpressJS (Routing)
var express = require('express'); Things to note:
var app = express.createServer(); 1. You can use Routing to
app.use(express.bodyParser()); listen to requests to call
different functions
//Receive HTTP GET
app.get('/user', function(req, res) { 2. You can listen to HTTP
//Don‟t let people call to /user POST, PUT etc.
throw new Error(“I‟m private. Call me w/ a user id");
});
app.get('/user/:id', function(req, res){
res.send('user ' + req.params.id);
});
//Receive HTTP POST
app.post(„‟/register”, function(req, res) {
//(Use bodyParser middleware for this)
var body = req.body;
db.save(body.user, body.password);
}
53
54. Hello World App using ExpressJS (Sessions)
var express = require('express');
var app = express.createServer();
//Middlewares
app.use(express.static(__dirname + „/public‟)); //sets location of public files
app.use(express.bodyParser()); //parses HTTP POST body
app.use(express.cookieParser()); //Parses cookies headers
app.use(express.session({secret: 'your secret here}));
Things to note:
1. To create sessions, use cookieParser & session middlewares
2. By default Express uses MemoryStore to store sessions
3. You can use Redis to store sessions
54
55. ExpressJS (Sticky sessions for multi instances)
var express = require('express');
var app = express.createServer();
//Middlewares
app.use(express.static(__dirname + „/public‟)); //sets location of public files
app.use(express.bodyParser()); //parses HTTP POST body
app.use(express.cookieParser()); //Parses cookies headers
app.use(express.session({secret: 'your secret here‟, key: „jsessionid‟ }));
Things to note:
1. Sticky Session is a reverse proxy / load balancer feature to help persistent
connection
2. When Sticky Session is on, request goes from Nginx to the same instance
no matter how many instances of your app is running .
3. Cloud Foundry‟s Nginx provides Sticky Sessions on „jsessionid‟ cookie
4. W/o setting this requests are randomly sent to different instances & you‟ll
have to use external store like Redis to fetch session data
(recommended).
55
59. Socket.io on Cloud Foundry (server side)
var sio = require('socket.io');
var express = require('express');
var app = express.createServer();
var io = sio.listen(app);//listen to express
io.configure(function() {
io.set('log level', 1);
io.set("transports", ["xhr-polling"]); //Currently CF doesn‟t support websockets
});
Things to Note:
1. Socket.io is mainly used to build real-time apps
2. Socket.io provides a single interface to switch b/w various transport techniques like xhr-
polling, websocket, JSONP etc
3. In addition, it provides heartbeats, reconnection, timeouts etc. that are vital for real-time
apps.
4. It works seamlessly with ExpressJS
59
60. Socket.io on Cloud Foundry (server side continued)
//After listening to express..wait for connection from browser
io.sockets.on('connection',
function(socket) {
// When the client/browser emits 'sendchat', this listens and executes
socket.on('sendchat',
function(data) {
// We will echo it back to ALL sockets
io.sockets.emit('updatechat‟, data);
});
});
60
61. Socket.io on Cloud Foundry (client side)
<script src="/socket.io/socket.io.js"></script>//socket.io serves this file from server
var socket = io.connect(document.location.href); //connect to the server
// on connection
socket.on('connect', function() {
console.log("client connected");
});
// Whenever the server emits 'updatechat', this updates the chat body
socket.on('updatechat', function (data) {
$('#conversation').append(data); // append it to my list
});
//When the user enter some data, send it to server
function sendchat() {
var message = $('#chatField').val();
// Emit or tell server to execute 'sendchat‟
socket.emit('sendchat', message);
}
61
63. Summary
1. About Node.js
• Internal working of Node.js
• Buzz around Node.js
• Who is using it
• What kind of apps are being built
2. Coding in Node.js
• Sync v/s Async coding (Callbacks)
• Classes & Modules (CommonJS)
• npm & package.json
• Node.js EventEmitters
3. Node.js & Cloud Foundry (w/ demo)
• Hello World app in Cloud Foundry
• Using Sticky Sessions
• CloudFoundry Module & connecting to Redis, MongoDB etc.
• Express.js (RESTful) app
• Socket.io + Express.js (Real-time) app
63
64. Next Steps
Register to Cloud Foundry:
Go to http://www.cloudfoundry.com
• Click Register - link
• Use Promo-code: cloudtoday
Download Node.js
• Go to http://nodejs.org/
Getting Started:
Go to http://www.cloudfoundry.com
• Click “Get Started” menu
64