Validating 6WIND’s High Performance TCP Networking Stack

As a high performance networking software vendor, 6WIND must use the right tools to measure and improve the performance of its product. This is especially important when measuring our TCP stack with a connection establishment rate of over 1 million Connections per Second (CPS).

In some cases, we can use simple tools because the traffic is simple and can be generated by different software and hardware solutions. We saw this in my previous blog post, which discussed the importance of using tools to allow a fair comparison between different implementations, summarized by VSperf tests to measure virtual switching performance.

But measuring TCP is much more complex as TCP relies on a state machine and implies the use of timers. There are tools such as iPerf that implement state machines, but iPerf is not a high performance test tool and is not the correct tool for measuring millions of concurrent TCP connections. For example, iPerf won’t scale when millions of concurrent TCP connections must be simulated, which is the requirement to measure 6WIND’s TCP stack.

One way to measure 6WIND’s TCP performance is to use our own TCP client to test our TCP server. It would probably work very well of course, and save money on test equipment, but it may not be very satisfying. Or at least, it may not be satisfying for me and my team.

So today, 6WIND is using an IXIA XT80 in order to validate our 6WINDGate TCP stack in a scalable way.

We first identified two scenarios for the simulation tests:

  • Proxy Scenario: 6WINDGate’s TCP stack terminates and initiates TCP traffic
  • Server Scenario: 6WINDGate’s TCP stack terminates traffic at the application level

In both scenarios we could validate the following key performance indicators:

  • Number of concurrent TCP sockets opened
  • Ability to continuously open and close new TCP sockets at high rates
  • TCP throughput

All tests were run on an Intel(R) Xeon(R) Platinum 8170 CPU @ 2.10GHz with 48GB DDR3 RAM and Intel X520 Dual port 10G adapters.

Proxy Scenario

6WINDGate is configured to act as a transparent TCP proxy. It terminates incoming TCP connections and establishes new ones while keeping a set of connections open.

IXIA will simulate the clients and server. Device under Test (DUT) 1-4 are implemented by 6WINDGate running its fast path TCP Stack with the proxy application.

The IXIA is configured to open and maintain a set of concurrent connections; in this example it is 6 million concurrent connections.

With this test we can measure the establishment rate and, according to the requested page size, the bandwidth.

For Connections per Second (CPS) we use 1 Byte pages, therefore the link speed will not limit the socket establishment rate.

For bandwidth we use 128KB pages.

Performance Tests

Now let’s review the Proxy Scenario performance tests.

1.) Socket Rate

Here we demonstrate 6WIND’s industry leading 1 million Connections per Second (CPS).

For our socket rate test, we measure CPS against the number of concurrent connections and number of cores allocated to the 6WINDGate fast path.

We can confirm that the socket establishment rate scales according to the number of cores allocated to the 6WINDGate fast path. Checking the final number of concurrent connections at the end of the test, we can see that the CPS remain steady: there is only a small impact on the establishment rate from the total number of connections.

The socket rate scales with the numbers of cores and independently of the targeted number of concurrent connections.

2.) Throughput

Here we demonstrate 12 Gbps with 2 fast path cores.

Here, using 128KB pages to achieve the maximum throughput, we confirm that performance scales according to the number of cores allocated to the 6WINDGate fast path. As we check the throughput results, remember our test platform is limited to 40G and you can see we reach wirespeed.

We also verify that the throughput is fairly steady regardless of the number of concurrent connections.

Server Scenario

Now we will check the server scenario. 6WINDGate is configured to act as a HTTP 1.1 server. It serves incoming HTTP requests while keeping a set of connections open.

IXIA will simulate clients only. Device under Test (1-4) are implemented by 6WINDGate running its fast path TCP stack with HTTP 1.1 server application.

Performance Tests

Now let’s review the Server Scenario performance tests.

1.) Socket Rate

In this test, we demonstrate 1.47 million Connections per Second (CPS).

Connection rate scales as more CPU resources are allocated to 6WINDGate’s fast path. The maximum is 1.47M sockets opened every second with 2M concurrent sockets opened simultaneously.

We also verify that CPS is fairly steady whatever the total number of concurrent connections.

2.) Throughput

Here we demonstrate 12 Gbps per core.

Here, same as with the proxy test, using 128KB pages to achieve the maximum throughput, we confirm that performance scales according to the number of cores allocated to the 6WINDGate fast path. As we check the throughput results, remember our test platform is limited to 40G and you can see we reach wirespeed.

We also verify that the throughput is fairly steady regardless of the number of concurrent connections.

Conclusion

6WIND was able to validate its fast path TCP Stack with intensive scenarios using the IXIA traffic generator.

Simpler tools such as ab (Apache Bench) or iPerf are sufficient for functional testing, however you’ll need a dedicated traffic generator to validate complex scenarios.

Using IXIA, 6WIND could verify 6WINDGate’s ability to maintain a high number of concurrent TCP sessions opened while continuously opening and closing sockets and generating traffic.

In summary, we have shown the following performance with 6WINDGate’s high performance TCP stack:

  • Socket rate: 1.4 million CPS
  • Throughput: 12 Gbps per core

We see that:

1.) Performance scales with the number of cores whether we measure socket rate or bandwidth.

2.) Socket rate is fairly steady, no matter the number of concurrent connections in the system.

Please visit our TCP Solutions web page where you will find more detailed information of our TCP use cases, including Solution Briefs on TCP solutions for SSL Inspection and Test and Measurement.

We look forward to hearing from you about your TCP performance requirements.


Damien Routier is the Support Team Manager at 6WIND.

Akram Arab is a Support Engineer at 6WIND.