SlideShare a Scribd company logo

TRex Traffic Generator - Hanoch Haim

•
•
H
harryvanhaaren

TRex is an open source, low cost, stateful traffic generator fuelled by DPDK. It generates L4-7 traffic based on pre-processing and a smart replay of real traffic templates. TRex amplifies both client and server side traffic and can scale to 200Gb/sec with one UCS.

1 of 24
Download to read offline
Userspace 2015 | Dublin
Current Situation • Networks include complex L4-7 features, such as • Load Balancer, DPI/AVC, Firewall, NAT • Requires testing with stateful and realistic traffic mix Firewall, NATDPI/AVCLB
What problem is being solved? • Traffic generators for realistic traffic are • Expensive ~$100-500K • Not scalable for high rates • Not flexible • Implication • Limited and late testing • Different benchmarks and test methodologies • Real life bottlenecks and design issues
What is TRex? • Stateful traffic generator • Generates, manipulates and amplifies based on templates of real, captured flows – no TCP/IP stack • High performance: up to 200 Gb/sec • Low cost: C220M UCS-1RU, Cisco internal – Standard server hardware • Flexible and Open Software - DPDK, ZMQ, Python libs • Virtualization (VMXNET3/E1000) • Easy installation and deployment
Stateful Traffic Generation Model
High level software architecture CPU/Sharememory Share memo ry Intel NIC DP -thread 1 CP – thread 0 DP -thread 2 Rx – thread 5 IF0 Client IF1 Server IF2 Client IF3 Server DP -thread 3 DP -thread 4 • DPDK/Multi-Threaded • Scales linearly • ~20 Gb/sec per core • Supports 1/10/40 Gb Intel NICs • Flow-based • Fast event scheduler • Generates flow templates • Can support 1K templates • Scales up to 100K clients,1M servers • Flexible • Client/server generation models • Measures jitter/latency/flow order • NAT translation/IPv6/Tunnels • User Interface • Python API • Benchmark automation • GUI
Flow Generation Total PPS = 𝑘=0 𝑛 𝐶𝑃𝑆 𝑘 × 𝑓𝑙𝑜𝑤_𝑝𝑘𝑡𝑠 𝑘 Total CPS = 𝑘=0 𝑛 𝐶𝑃𝑆 𝑘 Concurrent flows = 𝑘=0 𝑛 𝐶𝑃𝑆 𝑘 × 𝑓𝑙𝑜𝑤_𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛 𝑘 • Example of one flow with four packets FIF FIF FIF
DNS simple profile example
DNS output
HTTP & DNS
HTTP & DNS
Enterprise traffic profile • Includes protocols with Control/Data dependency • SIP • RTSP
EMIX - YAML
EMIX
Demo routing Cisco ASR 1013 ESP100 100Gb/sec 13RU - 4KW UCS-220M2 32GB 2x8 cores 2Ghz 2x82559 NIC (4x10Gb/sec) 0.4KW 1RU , 2K$
-Per port stats table ports | 0 | 1 | 2 | 3 ----------------------------------------------------------------------------------------- opackets | 71602327 | 82914082 | 71601627 | 82913259 obytes | 23990692045 | 67783397687 | 23990683213 | 67783032819 ipackets | 82914006 | 71602322 | 82913203 | 71601615 ibytes | 67783068258 | 23986183633 | 67782693539 | 23986134949 Tx Bw | 3.68 Gbps | 8.84 Gbps | 3.68 Gbps | 8.84 Gbps Cpu Utilization : 41.6 % 15.0 Gb/core Platform_factor : 1.0 Total-Tx : 25.04 Gbps Nat_time_out : 0 Total-Rx : 25.03 Gbps Nat_no_fid : 0 Total-PPS : 5.21 Mpps Total_nat_active: 33 Total-CPS : 103.16 Kcps Total_nat_open : 6697955 Active-flows : 90502 Clients : 248 Socket-util : 0.5795 % Open-flows : 6713475 Servers : 5368 Socket : 90502 Socket/Clients : 364.9 -Latency stats enabled Cpu Utilization : 7.2 % if| tx_ok , rx_ok , rx ,error, average , max , Jitter , max window | , , check, , latency(usec),latency (usec) ,(usec) , -------------------------------------------------------------------------------------------------------------- -- 0 | 65649, 65648, 647320, 0, 61 , 144, 13 | 111 130 138 107 127 110 1 | 65648, 65649, 552181, 0, 37 , 62, 3 | 42 42 43 39 38 39 38 2 | 65649, 65648, 660627, 0, 57 , 144, 11 | 103 139 130 97 126 3 | 65648, 65649, 562860, 0, 37 , 55, 3 | 39 41 39 39 46 39 39 -Rx Check stats enabled ------------------------------------------------------------------------------------------------------------ rx check: avg/max/jitter latency, 63 , 176, 18 | 130 158 159 138 155 175 136 active flows: 647, fif: 52193, drop: 0, errors: 0 ------------------------------------------------------------------------------------------------------------
TRex Traffic Generator - Hanoch Haim
Configure NAT & Firewall
-Global stats enabled Cpu Utilization : 34.9 % 15.9 Gb/core Platform_factor : 1.0 Total-Tx : 22.24 Gbps Nat_time_out : 3910462 Total-Rx : 22.24 Gbps Nat_no_fid : 735 Total-PPS : 4.87 Mpps Total_nat_active: 46 Total-CPS : 102.79 Kcps Total_nat_open : 11287417 Active-flows : 79813 Clients : 248 Socket-util : 0.5110 % Open-flows : 11313545 Servers : 5368 Socket : 79813 Socket/Clients : 321.8 -Latency stats enabled Cpu Utilization : 6.7 % if| tx_ok , rx_ok , rx ,error, average , max , Jitter , max window | , , check, , latency(usec),latency (usec) ,(usec) , -------------------------------------------------------------------------------------------------------------- -- 0 | 110281, 110279, 714052, 0, 204 , 14313, 16 | 807 1276 3439 935 850 5175 1 | 110280, 110280, 644514, 0, 91 , 10907, 5 | 802 653 1344 717 701 3436 2 | 110281, 110280, 720630, 0, 201 , 14355, 14 | 808 1194 3357 972 822 5177 3 | 110280, 110281, 645768, 0, 90 , 10491, 3 | 800 642 1349 706 699 3279 -Rx Check stats enabled ------------------------------------------------------------------------------------------------------------ rx check: avg/max/jitter latency, 180 , 14785, 407 | 2321 2546 4229 1627 1912 5275 active flows: 690, fif: 57593, drop: 14179, errors: 14552 ------------------------------------------------------------------------------------------------------------
TRex Traffic Generator - Hanoch Haim
Roadmap • RPC support • Stateless support (Ostinato like) – Streams – Stats per stream – Any type of packet – GUI to build packet – Emulation • Stateful – TCP stack – SDK for light application
TRex on your laptop (virtualbox)
Resources • Web • GitHub • Manual • Presentation
TRex Traffic Generator - Hanoch Haim

More Related Content

TRex Traffic Generator - Hanoch Haim

  • 2. Current Situation • Networks include complex L4-7 features, such as • Load Balancer, DPI/AVC, Firewall, NAT • Requires testing with stateful and realistic traffic mix Firewall, NATDPI/AVCLB
  • 3. What problem is being solved? • Traffic generators for realistic traffic are • Expensive ~$100-500K • Not scalable for high rates • Not flexible • Implication • Limited and late testing • Different benchmarks and test methodologies • Real life bottlenecks and design issues
  • 4. What is TRex? • Stateful traffic generator • Generates, manipulates and amplifies based on templates of real, captured flows – no TCP/IP stack • High performance: up to 200 Gb/sec • Low cost: C220M UCS-1RU, Cisco internal – Standard server hardware • Flexible and Open Software - DPDK, ZMQ, Python libs • Virtualization (VMXNET3/E1000) • Easy installation and deployment
  • 6. High level software architecture CPU/Sharememory Share memo ry Intel NIC DP -thread 1 CP – thread 0 DP -thread 2 Rx – thread 5 IF0 Client IF1 Server IF2 Client IF3 Server DP -thread 3 DP -thread 4 • DPDK/Multi-Threaded • Scales linearly • ~20 Gb/sec per core • Supports 1/10/40 Gb Intel NICs • Flow-based • Fast event scheduler • Generates flow templates • Can support 1K templates • Scales up to 100K clients,1M servers • Flexible • Client/server generation models • Measures jitter/latency/flow order • NAT translation/IPv6/Tunnels • User Interface • Python API • Benchmark automation • GUI
  • 7. Flow Generation Total PPS = 𝑘=0 𝑛 𝐶𝑃𝑆 𝑘 × 𝑓𝑙𝑜𝑤_𝑝𝑘𝑡𝑠 𝑘 Total CPS = 𝑘=0 𝑛 𝐶𝑃𝑆 𝑘 Concurrent flows = 𝑘=0 𝑛 𝐶𝑃𝑆 𝑘 × 𝑓𝑙𝑜𝑤_𝑑𝑢𝑟𝑎𝑡𝑖𝑜𝑛 𝑘 • Example of one flow with four packets FIF FIF FIF
  • 12. Enterprise traffic profile • Includes protocols with Control/Data dependency • SIP • RTSP
  • 14. EMIX
  • 15. Demo routing Cisco ASR 1013 ESP100 100Gb/sec 13RU - 4KW UCS-220M2 32GB 2x8 cores 2Ghz 2x82559 NIC (4x10Gb/sec) 0.4KW 1RU , 2K$
  • 16. -Per port stats table ports | 0 | 1 | 2 | 3 ----------------------------------------------------------------------------------------- opackets | 71602327 | 82914082 | 71601627 | 82913259 obytes | 23990692045 | 67783397687 | 23990683213 | 67783032819 ipackets | 82914006 | 71602322 | 82913203 | 71601615 ibytes | 67783068258 | 23986183633 | 67782693539 | 23986134949 Tx Bw | 3.68 Gbps | 8.84 Gbps | 3.68 Gbps | 8.84 Gbps Cpu Utilization : 41.6 % 15.0 Gb/core Platform_factor : 1.0 Total-Tx : 25.04 Gbps Nat_time_out : 0 Total-Rx : 25.03 Gbps Nat_no_fid : 0 Total-PPS : 5.21 Mpps Total_nat_active: 33 Total-CPS : 103.16 Kcps Total_nat_open : 6697955 Active-flows : 90502 Clients : 248 Socket-util : 0.5795 % Open-flows : 6713475 Servers : 5368 Socket : 90502 Socket/Clients : 364.9 -Latency stats enabled Cpu Utilization : 7.2 % if| tx_ok , rx_ok , rx ,error, average , max , Jitter , max window | , , check, , latency(usec),latency (usec) ,(usec) , -------------------------------------------------------------------------------------------------------------- -- 0 | 65649, 65648, 647320, 0, 61 , 144, 13 | 111 130 138 107 127 110 1 | 65648, 65649, 552181, 0, 37 , 62, 3 | 42 42 43 39 38 39 38 2 | 65649, 65648, 660627, 0, 57 , 144, 11 | 103 139 130 97 126 3 | 65648, 65649, 562860, 0, 37 , 55, 3 | 39 41 39 39 46 39 39 -Rx Check stats enabled ------------------------------------------------------------------------------------------------------------ rx check: avg/max/jitter latency, 63 , 176, 18 | 130 158 159 138 155 175 136 active flows: 647, fif: 52193, drop: 0, errors: 0 ------------------------------------------------------------------------------------------------------------
  • 18. Configure NAT & Firewall
  • 19. -Global stats enabled Cpu Utilization : 34.9 % 15.9 Gb/core Platform_factor : 1.0 Total-Tx : 22.24 Gbps Nat_time_out : 3910462 Total-Rx : 22.24 Gbps Nat_no_fid : 735 Total-PPS : 4.87 Mpps Total_nat_active: 46 Total-CPS : 102.79 Kcps Total_nat_open : 11287417 Active-flows : 79813 Clients : 248 Socket-util : 0.5110 % Open-flows : 11313545 Servers : 5368 Socket : 79813 Socket/Clients : 321.8 -Latency stats enabled Cpu Utilization : 6.7 % if| tx_ok , rx_ok , rx ,error, average , max , Jitter , max window | , , check, , latency(usec),latency (usec) ,(usec) , -------------------------------------------------------------------------------------------------------------- -- 0 | 110281, 110279, 714052, 0, 204 , 14313, 16 | 807 1276 3439 935 850 5175 1 | 110280, 110280, 644514, 0, 91 , 10907, 5 | 802 653 1344 717 701 3436 2 | 110281, 110280, 720630, 0, 201 , 14355, 14 | 808 1194 3357 972 822 5177 3 | 110280, 110281, 645768, 0, 90 , 10491, 3 | 800 642 1349 706 699 3279 -Rx Check stats enabled ------------------------------------------------------------------------------------------------------------ rx check: avg/max/jitter latency, 180 , 14785, 407 | 2321 2546 4229 1627 1912 5275 active flows: 690, fif: 57593, drop: 14179, errors: 14552 ------------------------------------------------------------------------------------------------------------
  • 21. Roadmap • RPC support • Stateless support (Ostinato like) – Streams – Stats per stream – Any type of packet – GUI to build packet – Emulation • Stateful – TCP stack – SDK for light application
  • 22. TRex on your laptop (virtualbox)
  • 23. Resources • Web • GitHub • Manual • Presentation