4.3.6. List of vswitchperf testcases

4.3.6.1. Performance testcases

Testcase Name Description
phy2phy_tput LTD.Throughput.RFC2544.PacketLossRatio
phy2phy_forwarding LTD.Forwarding.RFC2889.MaxForwardingRate
phy2phy_learning LTD.AddrLearning.RFC2889.AddrLearningRate
phy2phy_caching LTD.AddrCaching.RFC2889.AddrCachingCapacity
back2back LTD.Throughput.RFC2544.BackToBackFrames
phy2phy_tput_mod_vlan LTD.Throughput.RFC2544.PacketLossRatioFrameModification
phy2phy_cont Phy2Phy Continuous Stream
pvp_cont PVP Continuous Stream
pvvp_cont PVVP Continuous Stream
pvpv_cont Two VMs in parallel with Continuous Stream
phy2phy_scalability LTD.Scalability.Flows.RFC2544.0PacketLoss
pvp_tput LTD.Throughput.RFC2544.PacketLossRatio
pvp_back2back LTD.Throughput.RFC2544.BackToBackFrames
pvvp_tput LTD.Throughput.RFC2544.PacketLossRatio
pvvp_back2back LTD.Throughput.RFC2544.BackToBackFrames
phy2phy_cpu_load LTD.CPU.RFC2544.0PacketLoss
phy2phy_mem_load LTD.Memory.RFC2544.0PacketLoss
phy2phy_tput_vpp VPP: LTD.Throughput.RFC2544.PacketLossRatio
phy2phy_cont_vpp VPP: Phy2Phy Continuous Stream
phy2phy_back2back_vpp VPP: LTD.Throughput.RFC2544.BackToBackFrames
pvp_tput_vpp VPP: LTD.Throughput.RFC2544.PacketLossRatio
pvp_cont_vpp VPP: PVP Continuous Stream
pvp_back2back_vpp VPP: LTD.Throughput.RFC2544.BackToBackFrames
pvvp_tput_vpp VPP: LTD.Throughput.RFC2544.PacketLossRatio
pvvp_cont_vpp VPP: PVP Continuous Stream
pvvp_back2back_vpp VPP: LTD.Throughput.RFC2544.BackToBackFrames

List of performance testcases above can be obtained by execution of:

$ ./vsperf --list

4.3.6.2. Integration testcases

Testcase Name Description
vswitch_vports_add_del_flow vSwitch - configure switch with vports, add and delete flow
vswitch_add_del_flows vSwitch - add and delete flows
vswitch_p2p_tput vSwitch - configure switch and execute RFC2544 throughput test
vswitch_p2p_back2back vSwitch - configure switch and execute RFC2544 back2back test
vswitch_p2p_cont vSwitch - configure switch and execute RFC2544 continuous stream test
vswitch_pvp vSwitch - configure switch and one vnf
vswitch_vports_pvp vSwitch - configure switch with vports and one vnf
vswitch_pvp_tput vSwitch - configure switch, vnf and execute RFC2544 throughput test
vswitch_pvp_back2back vSwitch - configure switch, vnf and execute RFC2544 back2back test
vswitch_pvp_cont vSwitch - configure switch, vnf and execute RFC2544 continuous stream test
vswitch_pvp_all vSwitch - configure switch, vnf and execute all test types
vswitch_pvvp vSwitch - configure switch and two vnfs
vswitch_pvvp_tput vSwitch - configure switch, two chained vnfs and execute RFC2544 throughput test
vswitch_pvvp_back2back vSwitch - configure switch, two chained vnfs and execute RFC2544 back2back test
vswitch_pvvp_cont vSwitch - configure switch, two chained vnfs and execute RFC2544 continuous stream test
vswitch_pvvp_all vSwitch - configure switch, two chained vnfs and execute all test types
vswitch_p4vp_tput 4 chained vnfs, execute RFC2544 throughput test, deployment pvvp4
vswitch_p4vp_back2back 4 chained vnfs, execute RFC2544 back2back test, deployment pvvp4
vswitch_p4vp_cont 4 chained vnfs, execute RFC2544 continuous stream test, deployment pvvp4
vswitch_p4vp_all 4 chained vnfs, execute RFC2544 throughput tests, deployment pvvp4
2pvp_udp_dest_flows RFC2544 Continuous TC with 2 Parallel VMs, flows on UDP Dest Port, deployment pvpv2
4pvp_udp_dest_flows RFC2544 Continuous TC with 4 Parallel VMs, flows on UDP Dest Port, deployment pvpv4
6pvp_udp_dest_flows RFC2544 Continuous TC with 6 Parallel VMs, flows on UDP Dest Port, deployment pvpv6
vhost_numa_awareness vSwitch DPDK - verify that PMD threads are served by the same NUMA slot as QEMU instances
ixnet_pvp_tput_1nic PVP Scenario with 1 port towards IXIA
vswitch_vports_add_del_connection_vpp VPP: vSwitch - configure switch with vports, add and delete connection
p2p_l3_multi_IP_ovs OVS: P2P L3 multistream with unique flow for each IP stream
p2p_l3_multi_IP_mask_ovs OVS: P2P L3 multistream with 1 flow for /8 net mask
pvp_l3_multi_IP_mask_ovs OVS: PVP L3 multistream with 1 flow for /8 net mask
pvvp_l3_multi_IP_mask_ovs OVS: PVVP L3 multistream with 1 flow for /8 net mask
p2p_l4_multi_PORT_ovs OVS: P2P L4 multistream with unique flow for each IP stream
p2p_l4_multi_PORT_mask_ovs OVS: P2P L4 multistream with 1 flow for /8 net and port mask
pvp_l4_multi_PORT_mask_ovs OVS: PVP L4 multistream flows for /8 net and port mask
pvvp_l4_multi_PORT_mask_ovs OVS: PVVP L4 multistream with flows for /8 net and port mask
p2p_l3_multi_IP_arp_vpp VPP: P2P L3 multistream with unique ARP entry for each IP stream
p2p_l3_multi_IP_mask_vpp VPP: P2P L3 multistream with 1 route for /8 net mask
p2p_l3_multi_IP_routes_vpp VPP: P2P L3 multistream with unique route for each IP stream
pvp_l3_multi_IP_mask_vpp VPP: PVP L3 multistream with route for /8 netmask
pvvp_l3_multi_IP_mask_vpp VPP: PVVP L3 multistream with route for /8 netmask
p2p_l4_multi_PORT_arp_vpp VPP: P2P L4 multistream with unique ARP entry for each IP stream and port check
p2p_l4_multi_PORT_mask_vpp VPP: P2P L4 multistream with 1 route for /8 net mask and port check
p2p_l4_multi_PORT_routes_vpp VPP: P2P L4 multistream with unique route for each IP stream and port check
pvp_l4_multi_PORT_mask_vpp VPP: PVP L4 multistream with route for /8 net and port mask
pvvp_l4_multi_PORT_mask_vpp VPP: PVVP L4 multistream with route for /8 net and port mask
vxlan_multi_IP_mask_ovs OVS: VxLAN L3 multistream
vxlan_multi_IP_arp_vpp VPP: VxLAN L3 multistream with unique ARP entry for each IP stream
vxlan_multi_IP_mask_vpp VPP: VxLAN L3 multistream with 1 route for /8 netmask

List of integration testcases above can be obtained by execution of:

$ ./vsperf --integration --list

4.3.6.3. OVS/DPDK Regression TestCases

These regression tests verify several DPDK features used internally by Open vSwitch. Tests can be used for verification of performance and correct functionality of upcoming DPDK and OVS releases and release candidates.

These tests are part of integration testcases and they must be executed with --integration CLI parameter.

Example of execution of all OVS/DPDK regression tests:

$ ./vsperf --integration --tests ovsdpdk_

Testcases are defined in the file conf/integration/01b_dpdk_regression_tests.conf. This file contains a set of configuration options with prefix OVSDPDK_. These parameters can be used for customization of regression tests and they will override some of standard VSPERF configuration options. It is recommended to check OVSDPDK configuration parameters and modify them in accordance with VSPERF configuration.

At least following parameters should be examined. Their values shall ensure, that DPDK and QEMU threads are pinned to cpu cores of the same NUMA slot, where tested NICs are connected.

_OVSDPDK_1st_PMD_CORE
_OVSDPDK_2nd_PMD_CORE
_OVSDPDK_GUEST_5_CORES

4.3.6.3.1. DPDK NIC Support

A set of performance tests to verify support of DPDK accelerated network interface cards. Testcases use standard physical to physical network scenario with several vSwitch and traffic configurations, which includes one and two PMD threads, uni and bidirectional traffic and RFC2544 Continuous or RFC2544 Throughput with 0% packet loss traffic types.

Testcase Name Description
ovsdpdk_nic_p2p_single_pmd_unidir_cont P2P with single PMD in OVS and unidirectional traffic.
ovsdpdk_nic_p2p_single_pmd_bidir_cont P2P with single PMD in OVS and bidirectional traffic.
ovsdpdk_nic_p2p_two_pmd_bidir_cont P2P with two PMDs in OVS and bidirectional traffic.
ovsdpdk_nic_p2p_single_pmd_unidir_tput P2P with single PMD in OVS and unidirectional traffic.
ovsdpdk_nic_p2p_single_pmd_bidir_tput P2P with single PMD in OVS and bidirectional traffic.
ovsdpdk_nic_p2p_two_pmd_bidir_tput P2P with two PMDs in OVS and bidirectional traffic.

4.3.6.3.2. DPDK Hotplug Support

A set of functional tests to verify DPDK hotplug support. Tests verify, that it is possible to use port, which was not bound to DPDK driver during vSwitch startup. There is also a test which verifies a possibility to detach port from DPDK driver. However support for manual detachment of a port from DPDK has been removed from recent OVS versions and thus this testcase is expected to fail.

Testcase Name Description
ovsdpdk_hotplug_attach Ensure successful port-add after binding a device to igb_uio after ovs-vswitchd is launched.
ovsdpdk_hotplug_detach Same as ovsdpdk_hotplug_attach, but delete and detach the device after the hotplug. Note Support of netdev-dpdk/detach has been removed from OVS, so testcase will fail with recent OVS/DPDK versions.

4.3.6.3.3. RX Checksum Support

A set of functional tests for verification of RX checksum calculation for tunneled traffic. Open vSwitch enables RX checksum offloading by default if NIC supports it. It is to note, that it is not possible to disable or enable RX checksum offloading. In order to verify correct RX checksum calculation in software, user has to execute these testcases at NIC without HW offloading capabilities.

Testcases utilize existing overlay physical to physical (op2p) network deployment implemented in vsperf. This deployment expects, that traffic generator sends unidirectional tunneled traffic (e.g. vxlan) and Open vSwitch performs data decapsulation and sends them back to the traffic generator via second port.

Testcase Name Description
ovsdpdk_checksum_l3 Test verifies RX IP header checksum (offloading) validation for tunneling protocols.
ovsdpdk_checksum_l4 Test verifies RX UDP header checksum (offloading) validation for tunneling protocols.

4.3.6.3.4. Flow Control Support

A set of functional testcases for the validation of flow control support in Open vSwitch with DPDK support. If flow control is enabled in both OVS and Traffic Generator, the network endpoint (OVS or TGEN) is not able to process incoming data and thus it detects a RX buffer overflow. It then sends an ethernet pause frame (as defined at 802.3x) to the TX side. This mechanism will ensure, that the TX side will slow down traffic transmission and thus no data is lost at RX side.

Introduced testcases use physical to physical scenario to forward data between traffic generator ports. It is expected that the processing of small frames in OVS is slower than line rate. It means that with flow control disabled, traffic generator will report a frame loss. On the other hand with flow control enabled, there should be 0% frame loss reported by traffic generator.

Testcase Name Description
ovsdpdk_flow_ctrl_rx Test the rx flow control functionality of DPDK PHY ports.
ovsdpdk_flow_ctrl_rx_dynamic Change the rx flow control support at run time and ensure the system honored the changes.

4.3.6.3.5. Multiqueue Support

A set of functional testcases for validation of multiqueue support for both physical and vHost User DPDK ports. Testcases utilize P2P and PVP network deployments and native support of multiqueue configuration available in VSPERF.

Testcase Name Description
ovsdpdk_mq_p2p_rxqs Setup rxqs on NIC port.
ovsdpdk_mq_p2p_rxqs_same_core_affinity Affinitize rxqs to the same core.
ovsdpdk_mq_p2p_rxqs_multi_core_affinity Affinitize rxqs to separate cores.
ovsdpdk_mq_pvp_rxqs Setup rxqs on vhost user port.
ovsdpdk_mq_pvp_rxqs_linux_bridge Confirm traffic received over vhost RXQs with Linux virtio device in guest.
ovsdpdk_mq_pvp_rxqs_testpmd Confirm traffic received over vhost RXQs with DPDK device in guest.

4.3.6.3.6. Vhost User

A set of functional testcases for validation of vHost User Client and vHost User Server modes in OVS.

NOTE: Vhost User Server mode is deprecated and it will be removed from OVS in the future.

Testcase Name Description
ovsdpdk_vhostuser_client Test vhost-user client mode
ovsdpdk_vhostuser_client_reconnect Test vhost-user client mode reconnect feature
ovsdpdk_vhostuser_server Test vhost-user server mode
ovsdpdk_vhostuser_sock_dir Verify functionality of vhost-sock-dir flag

4.3.6.3.7. Virtual Devices Support

A set of functional testcases for verification of correct functionality of virtual device PMD drivers.

Testcase Name Description
ovsdpdk_vdev_add_null_pmd Test addition of port using the null DPDK PMD driver.
ovsdpdk_vdev_del_null_pmd Test deletion of port using the null DPDK PMD driver.
ovsdpdk_vdev_add_af_packet_pmd Test addition of port using the af_packet DPDK PMD driver.
ovsdpdk_vdev_del_af_packet_pmd Test deletion of port using the af_packet DPDK PMD driver.

4.3.6.3.8. NUMA Support

A functional testcase for validation of NUMA awareness feature in OVS.

Testcase Name Description
ovsdpdk_numa Test vhost-user NUMA support. Vhostuser PMD threads should migrate to the same numa slot, where QEMU is executed.

4.3.6.3.9. Jumbo Frame Support

A set of functional testcases for verification of jumbo frame support in OVS. Testcases utilize P2P and PVP network deployments and native support of jumbo frames available in VSPERF.

Testcase Name Description
ovsdpdk_jumbo_increase_mtu_phy_port_ovsdb Ensure that the increased MTU for a DPDK physical port is updated in OVSDB.
ovsdpdk_jumbo_increase_mtu_vport_ovsdb Ensure that the increased MTU for a DPDK vhost-user port is updated in OVSDB.
ovsdpdk_jumbo_reduce_mtu_phy_port_ovsdb Ensure that the reduced MTU for a DPDK physical port is updated in OVSDB.
ovsdpdk_jumbo_reduce_mtu_vport_ovsdb Ensure that the reduced MTU for a DPDK vhost-user port is updated in OVSDB.
ovsdpdk_jumbo_increase_mtu_phy_port_datapath Ensure that the MTU for a DPDK physical port is updated in the datapath itself when increased to a valid value.
ovsdpdk_jumbo_increase_mtu_vport_datapath Ensure that the MTU for a DPDK vhost-user port is updated in the datapath itself when increased to a valid value.
ovsdpdk_jumbo_reduce_mtu_phy_port_datapath Ensure that the MTU for a DPDK physical port is updated in the datapath itself when decreased to a valid value.
ovsdpdk_jumbo_reduce_mtu_vport_datapath Ensure that the MTU for a DPDK vhost-user port is updated in the datapath itself when decreased to a valid value.
ovsdpdk_jumbo_mtu_upper_bound_phy_port Verify that the upper bound limit is enforced for OvS DPDK Phy ports.
ovsdpdk_jumbo_mtu_upper_bound_vport Verify that the upper bound limit is enforced for OvS DPDK vhost-user ports.
ovsdpdk_jumbo_mtu_lower_bound_phy_port Verify that the lower bound limit is enforced for OvS DPDK Phy ports.
ovsdpdk_jumbo_mtu_lower_bound_vport Verify that the lower bound limit is enforced for OvS DPDK vhost-user ports.
ovsdpdk_jumbo_p2p Ensure that jumbo frames are received, processed and forwarded correctly by DPDK physical ports.
ovsdpdk_jumbo_pvp Ensure that jumbo frames are received, processed and forwarded correctly by DPDK vhost-user ports.
ovsdpdk_jumbo_p2p_upper_bound Ensure that jumbo frames above the configured Rx port’s MTU are not accepted

4.3.6.3.10. Rate Limiting

A set of functional testcases for validation of rate limiting support. This feature allows to configure an ingress policing for both physical and vHost User DPDK ports.

NOTE: Desired maximum rate is specified in kilo bits per second and it defines the rate of payload only.

Testcase Name Description
ovsdpdk_rate_create_phy_port Ensure a rate limiting interface can be created on a physical DPDK port.
ovsdpdk_rate_delete_phy_port Ensure a rate limiting interface can be destroyed on a physical DPDK port.
ovsdpdk_rate_create_vport Ensure a rate limiting interface can be created on a vhost-user port.
ovsdpdk_rate_delete_vport Ensure a rate limiting interface can be destroyed on a vhost-user port.
ovsdpdk_rate_no_policing Ensure when a user attempts to create a rate limiting interface but is missing policing rate argument, no rate limitiner is created.
ovsdpdk_rate_no_burst Ensure when a user attempts to create a rate limiting interface but is missing policing burst argument, rate limitiner is created.
ovsdpdk_rate_p2p Ensure when a user creates a rate limiting physical interface that the traffic is limited to the specified policer rate in a p2p setup.
ovsdpdk_rate_pvp Ensure when a user creates a rate limiting vHost User interface that the traffic is limited to the specified policer rate in a pvp setup.
ovsdpdk_rate_p2p_multi_pkt_sizes Ensure that rate limiting works for various frame sizes.

4.3.6.3.11. Quality of Service

A set of functional testcases for validation of QoS support. This feature allows to configure an egress policing for both physical and vHost User DPDK ports.

NOTE: Desired maximum rate is specified in bytes per second and it defines the rate of payload only.

Testcase Name Description
ovsdpdk_qos_create_phy_port Ensure a QoS policy can be created on a physical DPDK port
ovsdpdk_qos_delete_phy_port Ensure an existing QoS policy can be destroyed on a physical DPDK port.
ovsdpdk_qos_create_vport Ensure a QoS policy can be created on a virtual vhost user port.
ovsdpdk_qos_delete_vport Ensure an existing QoS policy can be destroyed on a vhost user port.
ovsdpdk_qos_create_no_cir Ensure that a QoS policy cannot be created if the egress policer cir argument is missing.
ovsdpdk_qos_create_no_cbs Ensure that a QoS policy cannot be created if the egress policer cbs argument is missing.
ovsdpdk_qos_p2p In a p2p setup, ensure when a QoS egress policer is created that the traffic is limited to the specified rate.
ovsdpdk_qos_pvp In a pvp setup, ensure when a QoS egress policer is created that the traffic is limited to the specified rate.

4.3.6.3.12. Custom Statistics

A set of functional testcases for validation of Custom Statistics support by OVS. This feature allows Custom Statistics to be accessed by VSPERF.

These testcases require DPDK v17.11, the latest Open vSwitch(v2.9.90) and the IxNet traffic-generator.

ovsdpdk_custstat_check Test if custom statistics are supported.
ovsdpdk_custstat_rx_error Test bad ethernet CRC counter ‘rx_crc_errors’ exposed by custom statistics.

4.3.6.4. T-Rex in VM TestCases

A set of functional testcases, which use T-Rex running in VM as a traffic generator. These testcases require a VM image with T-Rex server installed. An example of such image is a vloop-vnf image with T-Rex available for download at:

http://artifacts.opnfv.org/vswitchperf/vnf/vloop-vnf-ubuntu-16.04_trex_20180209.qcow2

This image can be used for both T-Rex VM and loopback VM in vm2vm testcases.

NOTE: The performance of T-Rex running inside the VM is lower if compared to T-Rex execution on bare-metal. The user should perform a calibration of the VM maximum FPS capability, to ensure this limitation is understood.

trex_vm_cont T-Rex VM - execute RFC2544 Continuous Stream from T-Rex VM and loop it back through Open vSwitch.
trex_vm_tput T-Rex VM - execute RFC2544 Throughput from T-Rex VM and loop it back through Open vSwitch.
trex_vm2vm_cont T-Rex VM2VM - execute RFC2544 Continuous Stream from T-Rex VM and loop it back through 2nd VM.
trex_vm2vm_tput T-Rex VM2VM - execute RFC2544 Throughput from T-Rex VM and loop it back through 2nd VM.