LMAX Disruptor as real-life example
•
57 likes•21,659 views
The document discusses the Disruptor, an ultrafast communication framework. It describes the problems with traditional concurrent queues, such as lock contention. The Disruptor uses a novel ring buffer approach with barriers to allow producers to add events and consumers to process them in parallel without synchronization overhead. This architecture enables it to achieve throughput of over 6 million transactions per second. The Disruptor exploits hardware characteristics to achieve extremely low latency communication between threads.
Report
Share
LMAX Disruptor as real-life example
- 1. Disruptor – Ultrafast communication March 2012 #theedge2012 Guy Raz Nir
- 2. Disruptor » Introduction » The problem … » The (not so good) alternatives » Architecture » Summary Agenda
- 5. Disruptor » QUEUE ! » Communication facility between threads. Disruptor
- 6. Disruptor » London Multi Asset Exchange platform » Can handle up to 6,000,000 TPS ▪ Dual socket, 3GHz quad-core Nehalem processors. » 98% transactions under 38ms. » Average transaction length: 9.22ms LMAX
- 7. Disruptor » To learn about LMAX Disruptor abilities and usability. » To practice “different thinking” in order to solve complex problems. Why are we here ?
- 8. Disruptor "Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius, and a lot of courage to move in the opposite direction.“ Albert Einstein
- 9. Disruptor
- 11. Disruptor “Mechanical Sympathy” Hardware and software working together in harmony * * Martin Thompson’s blog
- 13. Disruptor » Test case: execution 10,000,000 primitive increments. » Single thread execution: ~ 7ms ▪ No concurrency Multi-threading test long value = 0; while (value < 10000000L) { value++;
- 14. Disruptor » Synchronized approach: » Mutual exclusion approach: » AtomicLong: java.util.concurrent.locks.ReentrantLock lock = … lock.lock(); value++; lock.unlock(); synchronized (syncObj) { value++; AtomicLong value = new AtomicLong(0); long result = value.incrementAndGet();
- 15. Disruptor » Single thread, bare execution (value++) ▪ About ~7 milliseconds » Single thread, AtomicLong ▪ 68 milliseconds (x8.5) » Single thread with lock ▪ 125 milliseconds (x15.5) » Single thread with synchronized approach ▪ 450 milliseconds (x56)
- 16. Disruptor » Single thread, bare execution (value++) ▪ About ~7 milliseconds » Two threads, AtomicLong ▪ 270 milliseconds (x33.7) » Two thread with lock ▪ 298 milliseconds (x37.5)
- 17. Disruptor Concurrent execution latency (increment) Time(ms) Number of threads Synchronized CAS Lock
- 18. Disruptor Concurrent execution latency (PI calculation) Time(ms) Number of threads Synchronized Lock
- 19. “A good preliminary design overcomes any lastly patch” Guy (Raz) Nir, The Edge 2012
- 20. Disruptor Core #1 L1 cache Core #2 L1 cache Core #3 L1 cache Core #4 L1 cache L2 cache L2 cache L2 cache L2 cache L3 cache 32KB inst. 32KB data 256KB 2M – 16M 64-bit registers Model CPU architecture
- 21. Disruptor Non-volatile vs volatile Non-volatile Volatile 7 Time(ms)
- 22. Disruptor The (not so good) alternatives
- 23. Disruptor » Linked-list based queue ▪ Requires re-allocation of units ▪ Memory fragmentation ▪ Garbage collection ▪ Bad contention #0 #1 #2 #3 #4
- 24. Disruptor » Cyclic array-based queue ▪ Bad contention #0 #1 #2 #3 #4 Head Tail
- 25. Disruptor java.util.concurrent.ArrayBlockingQueue // Put new element in the queue. public boolean offer(E e, long timeout, TimeUnit unit) { // Consume ‘lock’ for writing. final ReentrantLock lock = this.lock; lock.lock(); } // Take one element from the queue. public E poll() { // Consume ‘lock’ for reading. final ReentrantLock lock = this.lock; lock.lock(); } Sun (Oracle) JDK 1.7.0_u2
- 26. Disruptor » General purpose assumptions: ▪ Multiple readers, multiple writers ▪ Queues can run as big as memory ▪ Other operations that degrade design » No regards to hardware Other problems
- 28. Disruptor » Barriers » Ring buffer » Sequences Main components
- 30. Disruptor The ring buffer 1 2 3 4 MyDataType[] buffer = ...; int offset = sequence % buffer.length; Next read sequence Available sequence
- 31. Disruptor » Array-based cyclic buffer. ▪ Fast index-based accessed. » Allow us to allocate all entries in advance ▪ Save GC time ▪ Continuous block allocation ▪ Save new costs at runtime. The ring buffer 1 2 3 4 5
- 32. Ring Buffer Disruptor Barriers 1 2 3 4 5 Producer Consumer sequence nextSequence
- 33. Disruptor public class StandardProducer { public void offer(Object o) { // ... } } public class DisruptorProducer { private RingBuffer buffer; public void addMessage(String message, long timestamp) { int seq = buffer.writeSequenceNumber++; buffer.data[seq].msg = message; buffer.data[seq].timestamp = timestamp; buffer.availableSequenceNumber = seq; } } X
- 34. Disruptor public class DisruptorConsumer { private RingBuffer buffer; int nextSequenceNumber; public Object take() { while (nextSequenceNumber < buffer.sequenceNumber) { .. } return buffer.get(nextSequenceNumber++); } } Buffer.sequenceNumber My sequence number
- 35. Ring Buffer Disruptor Multi consumers 1 2 3 4 5 Consumer sequence nextSequence = 2 Sequence barrier Consumer nextSequence = 3 Consumer nextSequence = 4
- 36. Disruptor » Allow us to fetch multiple elements. » Using event processors ▪ Callbacks Batches & Events
- 37. Disruptor Code sample – Create ring buffer // // Create a new ring buffer. // RingBuffer<MyEvent> ringBuffer = new RingBuffer<MyEvent>( new MyOwnFactory(), new SingleThreadedClaimStrategy(sizeOfRing), new SleepingWaitStrategy());
- 38. Disruptor Code sample - Producer // Request the next available sequence number. long sequence = buffer.next(); // Fetch the object at the that location. MyEvent event = buffer.get(sequence); // // ... do something with the event. // // Notify the rest of the world this event is ready to be consumed. buffer.publish(sequence);
- 39. Disruptor Code sample - Consumer // Extract a consumer's barrier. SequenceBarrier barrier = ringBuffer.newBarrier(); // Wait for an event to come. barrier.waitFor(nextSequence); // Take the event (data). MyEvent event = ringBuffer.get(nextSequence);
- 40. Disruptor » Disruptor is a smart Queue. » Latest release is 2.8 » Exploit hardware acceleration points. » Won the Duke’s 2011 award for innovation ! Summary
- 41. Disruptor » Google code: ▪ http://code.google.com/p/disruptor/ » Technical paper: ▪ http://disruptor.googlecode.com/files/Disruptor-1.0.pdf » Martin Thompson’s blog: ▪ http://mechanical-sympathy.blogspot.com » Trisha Gee’s blog: ▪ http://mechanitis.blogspot.com/ » InfoQ on Disruptor (session video): ▪ http://www.infoq.com/presentations/LMAX References
Editor's Notes
- 16 threadsAbout 375K TPS per threadAbout 518 billion T/day.1 cent per transaction = 5 billion Euros per day.Won the Duke’s 2011 award for innovation !
- Sun (Oracle) JDK 1.7Intel i7 2600K (SandyBridge) + Overclocking
- Intel i7 2600K SandyBridgeL1 cache speed: 450GB/secCPU – Memory speed: about 18GB (x25 slower than L1)
- Single-threaded example
- Single-threaded example
- Producers works in the same way.Disruptor provide various barriers for various models.