High Volume Web API Management with the WSO2 ESB

1. lean enterprise middlewareHigh Volume Web API Managementwith WSO2 ESBPaul Fremantle, CTO and Co-FounderHiranyaJayathilaka, Associate Technical Lead© WSO2 2011. Not for redistribution.

2. WSO2 OfferingsWSO2 CarbonOn premise product platformConsistent products sharing the same OSGi coreWSO2 StratosA “Cloud-Enabled Application Platform” (CEAP)Multi-tenant, Elastic, Metered/Billed deploymentAn extension of the Carbon platform Same SVN code treeWSO2 StratosLivehttp://cloud.wso2.comA running Platform-as-a-Service (PaaS)An instance of Stratos© WSO2 2011

3. ForresterWaveTM ESB Q2 2011 - Leader

4. © WSO2 2011WSO2 ESB 4.0

5. WSO2 ESB FeaturesThe WSO2 Enterprise Service Bus offers:Easy configuration through an intuitive graphical interface Lean, high-performance design conserves resources through maximum efficiency.Built on the WSO2 Carbon OSGi platform: adding new capabilities as you need them is a snap.Integrates with your existing systems, with support for EDI, CSV, Files, FIX, HL7, SAP, JMS, etcAsynchronous architecture supports very high loads with fixed resources Routing, mediation and transformation are modular, extensible but provide most use-cases simply by sequencing built-in mediatorsFull support for SOAP/WS-* and REST© WSO2 2011

6. Key Features for High Volume Web API Load Balancing and Failover built-inPluggable models can include affinity based LBFull REST Support (XML and JSON)Especially used in Mobile environmentsCan bridge into SOAP or other enterprise systemsMessage Relay Mode and Relay TransportExcellent performance for 100% streamingPriority Execution, Cache and ThrottlingManage workloads within the ESB as well as throttle backendCache content locally if appropriateService ChainingAsynchronous lightweight orchestration© WSO2 2011

7. eBay Case Study – High VolumeRequirements:“The solutions we were using no longer met our needs, so we considered whether to build a new internal system or to adopt third-party technology”, Abhinav Kumar, eBayNeeded enhanced service mediation and orchestration capabilities Scalability and performance to sustain increasing loadsEvaluation:eBay evaluated several industry leading hardware and software ESB solutions, looking at open source, commercially licensed ESBs and hardwareTest production during 2009 holiday season proved that our system was rock solid1m messages/day with zero downtime, zero memory growthResultsShared-nothing architecture on RedHat Linux on x86 hardwareCurrently running at more than 1 billion messages / day Low resource utilization and flat memory usageThe benefits“Using the WSO2 ESBs, we've been able to provide customers and partners with the quality experience they expect on eBay, even as our global customer base has grown,” Abhinav Kumar© WSO2 2011

8. Memory Usage at eBay© WSO2 2011

9. Deployment© WSO2 2011

10. Configuring for High VolumeOperating System JVMESBManagement© WSO2 2011

11. Tuning the OSOptimize core network settings for high throughput network activity

12. /etc/sysctl.conf

13. Increase the number of open file descriptors allowed by the OS

14. /etc/security/limits.confTuning the JVMAllocate sufficient memory for the heap

15. -Xms256m -Xmx2048m -XX:MaxPermSize=256m

16. Consider reducing the new ratio

17. -XX:NewRatio=n

18. Consider using the concurrent mark and sweep collector

19. -XX:+UseConcMarkSweepGC

20. Read more on JVM and GC tuning

21. http://wso2.org/library/articles/2010/11/taming-java-garbage-collectorTuning the ESBConfigure transport thread pools

22. Configured through nhttp.properties file

23. IO dispatcher threads carry out network IO at the wire level

24. Recommended to have one IO dispatcher per CPU core

25. Server workers and client workers mediate the messages

26. More the merrier (But keep memory usage and load in mind)Message Relay ModeWSO2 ESB uses Apache AXIOM and the StAX API for processing XML

27. XML payloads are streamed through the ESB

28. Pull parsing model

29. But for pure routing, load-balancing and header-only mediation we can avoid even this step

30. Message relay mode works around this glitch and enables 100% pure streaming of messages

31. Works regardless of size and format of messagesEnabling Message RelayEnable the binary relay builder (axis2.xml)

32. Instructs the ESB to stream the incoming messages through without touching their payloads

33. Enable the expanding message formatter (axis2.xml)

34. Allows the ESB to send messages that were received through the binary relay builder<messageBuildercontentType="application/xml" class="org.wso2.carbon.relay.BinaryRelayBuilder"/><messageFormattercontentType="application/xml" class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>Above configuration tells the ESB to process all application/xml messages in message relay mode.HTTP Relay TransportBrand new in WSO2 ESB 4.0 release

35. Non blocking HTTP transport implementation specially designed for streaming messages

36. Doesn’t require the binary relay builder and expanding formatter

37. To enable, simply uncomment the relevant entries in the axis2.xml fileError HandlingIn a large scale, high throughput deployment, errors are not unusual – Expect the unexpected

38. Configure endpoints to gracefully handle the common errors

39. Connection timeouts

40. Connection close/reset

41. Connection refused

42. HTTP protocol violations!!!!More on Error HandlingConfigure sequences to clearly log errors and if needed notify system administrators

43. In case the back end server fails to respond, send detailed fault responses to clients – Makes your application more appealing to the customers

44. Pay attention to HTTP error codesMonitoringIn general keep an eye on:

45. CPU usage

46. Memory usage

47. Thread counts

48. Fault counts

49. Latency/Response time

50. Active connectionsMonitoring ToolsUtilities Provided by OS

51. top

52. netstat

53. Mediation statistics

54. JMX Clients

62. Questions?28http://www.flickr.com/photos/oberazzi/

63. Resourceshttp://wso2.com/landing/ebay-the-esb-casestudy/WSO2 ESB 4.0 Download and Libraryhttp://wso2.org/library/esbMessage Relay Modehttp://wso2.org/project/esb/java/4.0.0/docs/message_relay.htmlTwitter: @pzfreoEmail: paul@wso2.comhiranya@wso2.com© WSO2 2011

High Volume Web API Management with the WSO2 ESB

Related slideshows

More Related Content

High Volume Web API Management with the WSO2 ESB