Routing, firewalls, security gateways, virtualization and VMware are terms that are often found together in the networking appliance market’s transition to software.
There are many software networking appliances available in VMware’s Marketplace. Yet these software appliances face the same challenges, namely performance and scalability.
A solution to these challenges is to use physical appliances in front of the VMware stack, which has been a common practice for years. However, this solution is more expensive and less flexible than software appliances.
6WIND combines its vRouter software appliances with expertise on VMware ESXi to help enterprises and service providers around the world boost their network performance in VMware deployment use cases. We focus on several examples:
- Security gateways with high performance IPsec, large numbers of IKE sessions, and additional networking features such as bonding, VLAN, etc.
- Firewall appliances to protect the internal network from outside threats using firewall rules while reducing performance bottlenecks.
- Routing appliances that support dynamic routing protocols such as BGP, OSPF, RIP, and also MPLS, for small office and data center use.
6WIND vRouter: Performance for Enterprise and Service Provider Use Cases in VMware Environments
6WIND developed a full-featured networking stack designed as an acceleration engine for the Linux networking stack. It offloads network processing from the Linux stack into what we refer to as our fast path. 6WIND’s fast path runs on a dedicated set of cores (only one core/two threads in this blog example). It has very limited impact on the system’s management, thus, 6WIND’s customers can continue to use their current management tools if they are already familiar with Linux routers.
The standard Linux commands (iproute2, iptables, ethtool, tcpdump, etc.) can be used to configure networking. Alternatively, for non-Linux users, a user-friendly Cisco-like CLI is also available. Both configuration models will be shown below.
At the end of this blog, we will prove that 6WIND’s vRouter outperforms Linux routers by up to 10X in Security Gateway and Routing use cases.
Test Case Introduction
This blog will detail two tests that compare the efficiency of the Linux networking stack versus 6WIND’s vRouter stack for VPN and Routing use cases in a VMware environment.
The following use cases are simple and used only to demonstrate the packet processing performance (with or without IPsec).
However, 6WIND’s vRouter has all the features to handle real and complex use cases:
- Firewalling: iptables and ipset support (ACLs)
- Overlays: VXLAN, GRE
- L2: VLAN, LAG
- High availability including IKE/IPsec HA
- Routing: BGP, OSPF, RIP
- IKE control plane: up to 100K tunnels with a setup rate of 1K tunnels per second
- Traffic engineering: QoS, packet capture
On the hypervisor side, VMware ESXi 6.5 is used running on top of a DELL PowerEdge R530 server with 1x Intel Xeon CPU E5-2680 v3 and 64 GB of RAM.
Both sites are linked with 2x 10GbE port Intel NICs. LAG is used in VMware’s ESXi Distributed vSwitch to leverage these two ports.
Each 6WIND vRouter/Linux Router will use 2 cores / 4 threads (one for the data plane and one for the control plane), and 8GB of RAM.
The Iperf3 tool will be used to generate TCP traffic from tester 1 to tester 2.
ESXi Configuration Tuning to Enhance 6WIND’s vRouter Appliance
Optimizations must be done in the hypervisor to achieve the best performance.
In the ‘Virtual Hardware’ tab of the VM settings, set:
- VM CPU reservation field to its maximal value
- VM CPU limit field to Unlimited
In the ‘VM Options’ tab, ‘Advanced’ part of the VM settings, set:
- cpu.latencySensitivity to High: used to ensure pinning and exclusive affinity of all CPUs of a VNF
Since ESXi 6.5, new tuning options are available to improve the hypervisor’s performance. Before going further, all the settings previously described must be applied.
In the ‘VM Options’ tab, ‘Advanced’ part of the VM settings, press the ‘Configuration Parameters’ button to set:
- ctxPerDev to 1 (where ethernetX is the NIC which will be handled by the 6WIND vRouter): each NIC configured with ctxPerDev will receive a TX thread in the hypervisor. It can be checked in the esxtop output. The ctxPerDev recommendation must be enabled for NICs that are expected to process a high packet load.
- cpu.latencySensitivity.sysContexts to numerical value: system threads (TX and RX) are assigned exclusive physical CPU cores. The numerical value assigned to sched.cpu.latencySensitivity.sysContexts must equal the number of active threads for the VNF. For example, if one receive thread exists and three TX threads have been set using the ctxPerDev command, the value set must be 4. In this example, 4 physical CPU cores must be available and unreserved.
More details are available in this VMware document regarding high performance setups.
To ensure that the updates have been taken into account, a simple esxtop has to be run in the hypervisor shell.
Virtual Border Router Use Case: 6WIND vRouter vs Linux
Let’s consider the following use case: Two hosts located on different sites have to communicate together using TCP. A router is setup at the edge of each site to route the connection. There is no need to secure the connection in this case.
The following diagram illustrates the technical setup used for the benchmark.
The router will simply run Linux in the first case, then 6WIND vRouter in the second case.
Network Configuration
Linux based configuration (for both Linux and 6WIND vRouter tests)
Left Traffic VM
Left Linux router or 6WIND vRouter
Right Linux router or 6WIND vRouter
Right Traffic VM
CLI-based configuration (only for 6WIND vRouter test)
Left 6WIND vRouter
Right 6WIND vRouter test
Results
With multiple TCP flows (multiple source ports), 6WIND’s vRouter achieves more than 5 times the performance of a standard Linux Router.
On top of that, 6WIND’s vRouter core is only at ~30% usage. This means that it is possible to add network processing (i.e. encapsulation, filtering, bonding, etc.) and keep the same throughput, or increase the throughput by adding more fiber between the two sites.
VPN Gateway Use Case: 6WIND vRouter vs Linux
In this use case, sensitive data is exchanged between the two sites (public network, personal data, etc.). Thus, IPsec is needed to secure the traffic between the two sites.
The following diagram illustrates the technical setup used for the benchmark.
The following algorithms will be used: AES-CBC-128/HMAC-SHA1/DH group 14.
The router will simply run Linux in the first case, then 6WIND vRouter in the second case.
Network Configuration
Linux based configuration (for both Linux and 6WIND vRouter tests)
The Traffic VMs’ configurations are the same as in the previous use case.
Left Linux router or 6WIND vRouter
Right Linux router or 6WIND vRouter
CLI based configuration (only for 6WIND vRouter test)
Left 6WIND vRouter
Right 6WIND vRouter
Results
6WIND vRouter for IPsec achieves around 10 times the performance of a standard Linux Router.
Conclusion: 6WIND’s vRouter Rocks ESXi Performance!
6WIND’s vRouter, as demonstrated in this benchmark, allows customers to benefit from the flexibility of a software router without any performance drop for multiple use cases on top of VMware’s ESXi hypervisor.
- 10X performance versus Linux with IPsec for Security or VPN Gateway use cases
- 5X performance with only 30% CPU utilization versus Linux for Routing use cases
When adding multiple business applications, the efficiency of the edge routing stack will be even more important, as it will need to manage a lot of flows, potentially a lot of different protocols and encapsulations, and of course, it will need to scale.
If you are looking for a software router for your VMware architecture, contact us today to request an evaluation of 6WIND vRouter.
Gabriel Frey is Pre-sales Manager at 6WIND.
