Advanced QoS in vRouter Fast Path
6WIND vRouter provides QoS functionalities, including rate limitation (aka Basic QoS) and class-based QoS (aka Advanced QoS). In a previous blog, we also described how the vRouter uses internal QoS mechanisms to prioritize control plane protocols over data plane traffic and avoid breakage under load.
In this blog, we focus on advanced QoS, whose goal is for example to let a network administrator provide an assured bandwidth for certain types of traffic. With advanced QoS, a user can send packets into traffic classes and then apply different processing to these classes, including policing, shaping and scheduling.
We will explain about the general advanced QoS architecture in the 6WIND vRouter’s high-performance networking stack and explain with a simple use case how to instantiate classes of traffic, program some associated schedulers on output interfaces and configure ACLs to steer ingress traffic into the traffic classes.
Architecture
In the 6WIND architecture, QoS processing happens inside the vRouter’s high-performance networking stack. This is what we call the fast path, which is based on 6WINDGate, and used by Tier 1 customers worldwide for over a decade. More information about the 6WINDGate architecture can be found here.
The QoS architecture in the fast path is as follows:
All worker cores receive traffic according to RSS queues and mark packets according to Netfilter rules. These rules are used to classify packets to different class queues and the scheduler finally emits packets according to each class queue priority. Shaping and policing can also be applied in the process.
Schedulers are running on a specified set of fast path cores which are totally independent from the worker cores to assure ‘reliable scheduling.
Use case
We’ll illustrate the configuration required using a simple use case. VoIP traffic will be scheduled using a specific class while all other traffic will be scheduled with limited bandwidth.
Configure scheduler
We’ll start by configuring a scheduler on each interface that will prioritize traffic to be emitted.
This is done at the fast path level using the fp-cli qos-sched-add command.
Each scheduler attached to an interface defines:
- Policy: strict or deficit weighted round robin (DWRR)
- Class number
- and optionally:
- Maximum emission rate
- Maximum emission burst
- Class queue size in packets
- Maximum class rate
- Maximum class burst
Upon its creation, queues for the different classes are created.
Example:
# fp-cli qos-sched-add enp1s0f0 prio 2 rate 10g qrate 0,2g
On enp1s0f0 a scheduler is created with 2 queues (one for each class). This scheduler is limited to 10 Gbps in emission. It implements a strict policy algorithm: queue 1 is the highest priority and queue 2 the lowest.
The bandwidth of the first queue will not be limited, while the second queue will be limited to 2 Gbps in emission.
The scheduler is now created and ready to process packets submitted to its different queues matching the different classes.
Classify traffic according to marks
Every packet to be sent on a scheduled interface will need to be associated with a class, based on the Netfilter mark set on the packet.
This is done at the fast path level using the fp-cli qos-filter-add command.
This commands defines:
- Scheduler used
- Priority of the filtering rule
- Netfilter mark filter
- Filtering action to associate with a class
Each rule is evaluated according to the order defined to sort marked packets to different classes.
Examples:
Rule 1 sends all packets with Netfilter mark ending with 0x0001 to class 0x1:
# fp-cli qos-filter-add enp1s0f0 1 1/0xffff set-classid 0x1
This class will be used for VoIP traffic.
Rule 2 sends all packets with Netfilter mark ending with 0x0000 to class 0x2:
# fp-cli qos-filter-add enp1s0f0 2 0/0xffff set-classid 0x2
This class will be used for all other traffic.
Identify and mark packets
The final step is to efficiently identify and mark packets belonging to the different classes. The fast path filtering module based on the Netfilter framework allows us to define fine grain rules.
As a reminder, this module supports the following Netfilter matches:
- comment
- connmark
- conntrack (–ctstate only)
- dscp
- frag
- icmp
- icmp6
- limit
- mac
- mark
- multiport
- physdev
- policy
- rpfilter
- sctp
- set
- state
- tcp
- udp
And the following ip set types:
- hash:ip
- hash:net
- hash:mac
- hash:net,net
- hash:net,port
- hash:net,port,net
Configuration of these rules is done using original iptables commands as illustrated by the following example:
All SIP (port 5060) and RTP (port range 10000:20000) traffic used for VoIP have their DSCP set to EF:
# iptables -t mangle -I POSTROUTING -p udp –dport 5060 -j DSCP –set-dscp-class EF
# iptables -t mangle -I POSTROUTING -p udp –dport 10000:20000 -j DSCP –set-dscp-class EF
All packets with the EF DSCP must have their Netfilter mark set to 0x1:
# iptables -t mangle -I POSTROUTING -m dscp –dscp-class EF -j MARK –set-mark 0x1
Multiple rules can be defined according to the traffic that needs to be processed through the QoS module.
Conclusion
QoS configuration is fairly simple as it is about marking the packets, associating a mark with a class and then scheduling the different class queues on the egress interface.
By dedicating cores to the scheduling operation, the impact of QoS on the global system performance is reduced and the benefits of 6WIND’s accelerated data plane are maintained.
This feature is available in 6WINDGate now and will be integrated into 6WIND Turbo Series 2.2 software appliances to be released in Sept 2019.
If you wish to evaluate now, do not hesitate to contact us.
___________________________________________________
Damien Routier is the Support Team Manager at 6WIND. 
