Spring one 2012 Groovy as a weapon of maas PaaSification

©2012AmadeusITGroupSA©2012AmadeusITGroupSA
Groovy as a weapon of
massive PaaSification
Nenad Bogojevic
October 2012

©2012AmadeusITGroupSA
Context ?aaS
Why
Groovy?
Groovy,
Java, JSR
Security API
DSL IDE Conclusion

Me
 Nenad Bogojević
 Software Architect at Amadeus
 e-Commerce platforms
 nbogojevic@amadeus.com
 @NenadBo

Amadeus
The leading
provider of IT
solutions to tourism
and travel industry
Provides transaction
processing power
and technology for
travel providers and
travel agencies
Global distribution
system

Travel ecosystem
Airlines
Hotels
Cruises/ Ferries
Tour Operators
Car Rentals
Rail
Airlines own distribution
Airports, ticket offices,
web sites, call centers
Providers
Travel Agencies
TMCs
Traditional agencies
Online agencies
• Travel
Shopping
• Reservation
• Ticketing
• Post Sales
• Customer
Profiles
• Destination
Content
• …
Global
Distribution
System
Leisure
Business
End
consumers

Toronto
Boston
Miami
London
Antwerp Frankfurt
SydneySophia Antipolis Bangalore
Development Footprint
Warsaw
Bogota
Bangkok
Beijing
Tucson

Amadeus Locations

Groovy, Java talents are welcome
http://www.amadeus.com/careers

Amadeus System Constraints
• 600+ millions bookings, 1 billion billable
transactions per year.
Huge Volumes
• 24x7 99.99%
• 4:20 minute coffee pause per month
High Availability
• 20 000 transaction per second - average < 200ms
• 65 000 external messages per second
Fast Performance

Amadeus SaaS Services
 Booking
 Inventory
 Departure control
 Travel IT
 e-Commerce
 Point-of-sale

e-Commerce and point-of-sale platform
 One platform
Multiple products
 Consumers
 Professionals
 Integrators Common
platform
E-Commerce
Corporations
Travel
Agents

Common platform
Web-based
Common
hardware
Common code
Java, JEE,
JavaScript

Airline web site
 Any browser
 Any device
 Specific UI
 Specific logic
 1 billion hits/month
 500 million visitors
 $40 billion in turnover

Corporate travel site
 RIA
 Guided flow
 Integrates with
 3rd party services
 Corporate IS

Travel-agency POS
 Mission critical
 Dual UI
 Terminal, RIA
 Free-flow
 Custimizable UI
 Needs automation
 450,000 terminals
 2,000 TPS

Existing customization technology
 Windows, COM based
 VB, .NET, C#, rudimentary scripting language
 “Shell script” with terminal

Types of extensions
UI change
Guided work-flow
Automate tasks
Add business logic
Batch processing

Three types of developers
 Corporate (airlines, travel agency chains)
 Observe “rules and regulations”
 IT Services (small-medium shops)
 Agile, know their tools, hack
 Travel Agents
 Non-technical, read a book or two
Karin Dalziel, Michael Lokner , Creative Commons

Alphabet soup
IaaS
• Virtualized
hardware
• EC2, Azure
PaaS
• Virtualized
runtime
• AppEngine,
CloudFoundry
• elastic
beanstalk,
Azure
SaaS
• Virtualized
application
• Salesforce,
Google Apps
• Office 365

SaaS platform
Application code
Middleware
Presentation
Transaction Processing Facility

SaaS Scenario
Search flights
Results

PaaS Scenario
Search flights
Results
Customer’s code

PaaS platform
Application code
Middleware
Script execution
environment
A
P
I
Presentation
Customization code
& resources
UI Customization
Transaction processing facility

Language choice?
Executes in
JVM
Dynamic,
scripting
language
A language
with future and
community
Extensions are
often small
pieces of code
Developers
have varying
level of skills

Scripting/dynamic language
Easy Productive
Dynamic
loading

Language choice
4 candidates
JavaScript (Rhino)
Ruby (JRuby)
Python (jython)
Groovy
3 measures
Popularity
Language features
Performance

Some code examples
def keyboard = new Scanner(System.in)
def number = new Random().nextInt(10) + 1
def guess = 0
while (number != guess) {
println "Guess the number which is between 1 and 10: "
guess = keyboard.nextInt()
}
println"Hurray! You guessed correctly!"
function guessNumber() {
// Get a random integer from 1 to 10 inclusive
var num = Math.ceil(Math.random() * 10);
var guess;
while (guess != num) {
guess = prompt('Guess the number between 1 and
10 inclusive');
}
alert('Congratulations!nThe number was ' + num);
}
guessNumber();
import random
target, guess = random.randint(1, 10), 0
while target != guess:
guess = int(input('Guess my number between 1 and 10 until you get it right: '))
print('Thats right!')
n = rand( 10 ) + 1
puts 'Guess the number: '
puts 'Wrong! Guess again: ' until gets.to_i == n
puts 'Well guessed!'
source: http://rosettacode.com

Why Groovy?
Fastest
~10% in our
benchmark
Least
memory
20%-75%
better then
average
On top of
Java
Double eco-
system
Language
itself
Java code
is valid
code
Groovy
proper for
.NET crowd

So, next…
Platform for 10
million users
“powered by
Groovy”
500.000 users
writing Groovy
code

Embedding scripting engine
Groovy offers everything you need to
add it as scripting language
We have our platform and we can plug
it in
But what if we need to support another
scripting or templating language
• Freemarker, javascript

JSR-223
Framework for embedding scripts into Java applications
Independent from language
Scripts can be run inside or outside JVM
Part of JDK 6+. JDK comes with Rhino/Nashorn
javax.script

JSR 223 Services
Script Execution: executing stream of characters in script language
• ScriptEngine
Binding: exposing Java objects to script programs as named
variables
• Bindings, ScriptContext
Compilation: generating intermediate code from script source
• Compilable, CompiledScript
Invocation: reusing intermediate code
• Invocable

Groovy vs. JSR
GroovyShell shell = new GroovyShell();
shell.evaluate(
"println 'Hello World!'; x = 123; return foo * 10");
ScriptEngineManager mgr = new ScriptEngineManager();
ScriptEngine groovyEngine = mgr.getEngineByName(“groovy");
try {
groovyEngine.eval(
"println 'Hello World!'; x = 123; return foo * 10");
} catch (ScriptException ex) {
ex.printStackTrace();
}

Groovy vs. JSR
Binding binding = new Binding();
binding.setVariable("foo", new Integer(2));
GroovyShell shell = new GroovyShell(binding);
shell.evaluate("println 'Hello World!'; x = 123; return foo * 10");
ScriptEngineManager mgr = new ScriptEngineManager();
ScriptEngine groovyEngine = mgr.getEngineByName(“groovy");
Bindigs bindings = groovyEngine.createBindings();
bindings.put("foo", new Integer(2));
try {
groovyEngine.eval(
"println 'Hello World!'; x = 123; return foo * 10 “, bindings);
} catch (ScriptException ex) {
ex.printStackTrace();
}

JSR 223 Interfacing with Java
 Invocable
groovyEngine.eval(
"def hello(s) { println 'Hello ${s}!';}");
Invocable obj = (Invocable) groovyEngine;
inv.invokeMethod(obj, "hello", "Groovy!!" );
 Invocable with interface
groovyEngine.eval(
"def hello(s) { println 'Hello ${s}!';}");
Invocable inv = (Invocable) groovyEngine;
Runnable r = inv.getInterface(Runnable.class);
r.run();

JSR 223 Interfacing with Java
 Invocable
jsEngine.eval(
"function hello(s) { log('Hello ' + s); }");
Invocable obj = (Invocable) groovyEngine;
inv.invokeMethod(obj, "hello", "JavaScript!!" );
 Invocable with interface
jsEngine.eval(
"function hello(s) { log('Hello ' + s); }");
Invocable inv = (Invocable) groovyEngine;
Runnable r = inv.getInterface(Runnable.class);
r.run();

When JSR 223
Multiple
script
technologies
Middleware
services
Common
API
Access to
scripting
resources

Limitations
javax.script is for interfacing with Java code
Exposure of script services is limited
Concrete security is script engine dependent
Works for dynamic typed languages only

PaaSification

Pacification
 We are executing in JVM, sharing the platform
and resources
 What can you do in JVM?
 Consume resources (CPU, disk, threads)
 Spy around:
 Amadeus code,
 Other customers’ code
 Even worse, other customers’ data

Approaches
• One JVM per user,
150K concurrent users
Sandbox
JVM
• Use AOP, instrument
everything
Sandbox
Code
• Replace all classes
with sanitized one
Sandbox
API

AST visiting
 Check every node in AST at compile time
org.codehaus.groovy.control.customizers.CompilationCustomizer
org.codehaus.groovy.ast.GroovyCodeVisitor
 Sandbox API
 Whitelist
 API that you can call
 Blacklist
 java.lang.Object, java.lang.reflect
 Methods in whitelisted classes

AST modification (for stability)
 Timeout scripts
groovy.transform.TimedInterrupt
@TimedInterrupt(value = 10L, unit = TimeUnit.SECONDS)
 Allow interrupting script thread
groovy.transform.ThreadInterrupt
 Allow custom interrupting
groovy.transform.ConditionalInterrupt
@ConditionalInterrupt({++counter<10})

Extension points, where?
 Web service endpoints
 Well documented (xml schemas), public,
 Internal application hooks
 Well…

API on Top of Existing Code
 Web service
 Documented (xml schemas), public
 XML easily integrates with groovy

DOM-like object
 SaaS services communicate using
DOM-like object with Map interface
 groovy.util.Node
 Adding support for groovy
constructs (e.g. closures)
 groovy.lang.DelegatingMetaClass
Marshall
object on
java side
Unmarshall
(parse) on
groovy

UI API
 Depends on UI techology
 XUL, AriaTemplates

New Services
 Build for Scripting

DSL challenges
 Groovy is great for DSL
 But, can we build a great DSL?
 The one that is really used?
 Know the users
 Learn from history

DSL for macros
 Simple
 Send terminal command
 Prompt user
 Variable
 Loops?
 Conditions?
ask “Customer phone?" assign to phone
send “AP ${phone}"
ask “Customer account?" assign to account
send "RM ACC-${account}"

DSL for macros
Run macro
User entry
Prompt user
Continue script
Run script
User entry
Prompt user

What kind of IDE
DSL
• For end users
• Integrated in application itself
UI templating
• Text editor or Web based
• Ace, Eclipse Orion
• Graphical
Groovy
• For professionals only
• Eclipse today

Development environment
 Customers are writing code running …
 … hosted in the Amadeus Data Center
Customization Server
Aria Templates
Groovy

Extend eclipse plugin

How to test
 Develop locally, execute on remotely hosted
platform
 Install platform locally
 Works for Google App Engine,
Azure, Cloud Foundry
 Mock platform locally

Where we are now
In production
Tens of thousands of
LOCs of Groovy
Rolling out to 20,000
users

Challenges
Expand API
Polyglot
IDE & Testing
Resource management

Open Source
 We know the value is in community
 We have contributed to groovy,
 We will keep contributing
 http://www.amadeus.com/blog/17/09/open-for-
business/
 Open sourcing UI http://ariatemplates.com/

THE END
Special thanks
SSE Team
Vincent Bersin, Tiago Fernandez, Mathieu Bruyen,
Sebastien Blanc, Guillaume Cernier (@cernier,
http://linkedin.com/in/cernierg), Vincent Malley, Marco Funaro, Pascal
Cohen, Vincent Gerard, Nicola Rondelle, Bruno Bongiovanni,
Francesco Uliana, Emanuel Dore, Geoffrey Van Staen
Architecture group
Marc Campora, Christophe Allexandre, Fabrice Matrat
Amadeus e-Travel Management and Selling Platform Connect
teams for creating products that are shown here
Wikipedia for images of French Riviera
rosetta.org for code examples
Smartflow DSL slides inspired by
http://dslprez.cloudfoundry.com/DSLPresentation.html by
Corinne Krych, Sebastien Blanc

Spring one 2012 Groovy as a weapon of maas PaaSification

Related slideshows

More Related Content

Spring one 2012 Groovy as a weapon of maas PaaSification