Up to his point rxqs are sorted by processing cycles they
consumed and assigned to pmds in a round robin manner.
Ian pointed out that on wrap around the most loaded pmd will be
the next one to be assigned an additional rxq and that it would be
better to reverse the pmd order when wraparound occurs.
In other words, change from assigning by rr to assigning in a forward
and reverse cycle through pmds.
Also, now that the algorithm has finalized, document an example.
Suggested-by: Ian Stokes <ian.stokes@intel.com>
Signed-off-by: Kevin Traynor <ktraynor@redhat.com>
Signed-off-by: Darrell Ball <dlu998@gmail.com>
Previously rxqs were assigned to pmds by round robin in
port/queue order.
Now that we have the processing cycles used for existing rxqs,
use that information to try and produced a better balanced
distribution of rxqs across pmds. i.e. given multiple pmds, the
rxqs which have consumed the largest amount of processing cycles
will be placed on different pmds.
The rxqs are sorted by their processing cycles and assigned (in
sorted order) round robin across pmds.
Signed-off-by: Kevin Traynor <ktraynor@redhat.com>
Signed-off-by: Darrell Ball <dlu998@gmail.com>
Commit 5dcde09c80a8 was introduced to make detaching more
automatic without using an additional command beyond
ovs-vsctl del-port <br> <port>.
Sometimes, since commit 5dcde09c80a8, dpdk devices are
not detached when del-port is issued; command example:
sudo ovs-vsctl del-port br0 dpdk1
This can happen when vswitchd is (re)started with an existing
database and devices are already bound to dpdk.
A minimal recipe to reproduce the issue is:
1/ Starting with
darrell@prmh-nsx-perf-server125:~$ sudo ovs-vsctl show
1c50d8ee-b17f-4fac-a595-03b0da8c8275
Bridge "br0"
Port "br0"
Interface "br0"
type: internal
Port "dpdk1"
Interface "dpdk1"
type: dpdk
options: {dpdk-devargs="0000:04:00.1"}
Port "dpdk0"
Interface "dpdk0"
type: dpdk
options: {dpdk-devargs="0000:04:00.0"}
darrell@prmh-nsx-perf-server125:~$ /usr/src/dpdk-16.11/tools/dpdk-devbind.py --status
Network devices using DPDK-compatible driver
============================================
0000:04:00.0 'Ethernet Controller 10-Gigabit X540-AT2' drv=uio_pci_generic unused=ixgbe,vfio-pci
0000:04:00.1 'Ethernet Controller 10-Gigabit X540-AT2' drv=uio_pci_generic unused=ixgbe,vfio-pci
2/ restart vswitchd
3/ run
sudo ovs-vsctl del-port br0 dpdk1
and find the interface is NOT detached; there is
no info log ‘Device '0000:04:00.1' detached’.
A more verbose discussion is here:
https://mail.openvswitch.org/pipermail/ovs-dev/2017-June/333462.html
along with another possible solution.
Since we are nearing the end of a release, a safe approach is needed,
at this time.
One approach is to revert 5dcde09c80a8. This patch does not do that
but reinstates the command ovs-appctl netdev-dpdk/detach to handle
cases when del-port will not work.
To detach the device, run the reinstated command
ovs-appctl netdev-dpdk/detach 0000:04:00.1
Observe console output
‘Device '0000:04:00.1' has been detached’
Fixes: 5dcde09c80a8 ("netdev-dpdk: Fix device leak on port deletion.")
CC: Ilya Maximets <i.maximets@samsung.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Acked-by: Fischetti, Antonio <antonio.fischetti@intel.com>
Signed-off-by: Darrell Ball <dlu998@gmail.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Upgrading to DPDK 17.05.1 stable release adds new
significant features relevant to OVS, including,
but not limited to:
- tun/tap PMD,
- VFIO hotplug support,
- Generic flow API.
Following changes are applied:
- netdev-dpdk: Changes required by DPDK API modifications.
- doc: Because of DPDK API changes, backward compatibility
with previous DPDK releases will be broken, thus all
relevant documentation entries are updated.
- .travis: DPDK version change from 16.11.1 to 17.05.1.
- rhel/openvswitch-fedora.spec.in: DPDK version change
from 16.11 to 17.05.1
Signed-off-by: Michal Weglicki <michalx.weglicki@intel.com>
Acked-by: Kevin Traynor <ktraynor@redhat.com>
Acked-by: Mark Kavanagh <mark.b.kavanagh@intel.com>
Tested-by: Ian Stokes <ian.stokes@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Darrell Ball <dlu998@gmail.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Rx checksum offload is enabled by default on DPDK physical NICs
where available, with reconfiguration through
options:rx-checksum-offload. However, changing rx-checksum-offload
did not result in a reconfiguration of the NIC and wrong status is
reported for it.
As there seems to be diminishing reasons why a user would want
to disable Rx checksum offload, just remove the broken reconfiguration
option.
Fixes: 1a2bb11817a4 ("netdev-dpdk: Enable Rx checksum offloading feature on DPDK physical ports.")
Reported-by: Kevin Traynor <ktraynor@redhat.com>
Suggested-by: Sugesh Chandran <sugesh.chandran@intel.com>
Acked-by: Darrell Ball <dlu998@gmail.com>
Tested-by: Sugesh Chandran <sugesh.chandran@intel.com>
Signed-off-by: Kevin Traynor <ktraynor@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
It's becoming more common that OSes include "ip" but not "ifconfig", so
it's best to avoid using the latter. This commit removes most references
to "ifconfig" and replaces them by "ip". It also adds a build-time check
to make it harder to introduce new uses of "ifconfig".
There are important differences between "ifconfig" and "ip":
- An "ifconfig" command that sets an IP address also brings the interface
up, but a similar "ip addr add" command does not, so it is often necessary
(or at least precautionary) to add an "ip link set <dev> up" command.
- "ifconfig" can infer a netmask from an IP adddress, but "ip" always
assumes /32 if none is given.
- "ifconfig" with address 0.0.0.0 removes any configured IP address, but
"ip addr add" does not, so "ifconfig <dev> 0.0.0.0" must be replaced by
"ip addr del" or "ip addr flush".
Signed-off-by: Ben Pfaff <blp@ovn.org>
Currently, once created device in dpdk will exist forever
even after del-port operation untill we manually call
'ovs-appctl netdev-dpdk/detach <name>', where <name> is not
the port's name but the name of dpdk eth device or pci address.
Few issues with current implementation:
1. Different API for usual (system) and DPDK devices.
(We have to call 'ovs-appctl netdev-dpdk/detach' each
time after 'del-port' to actually free the device)
This is a big issue mostly for virtual DPDK devices.
2. Follows from 1:
For DPDK devices 'del-port' leads just to
'rte_eth_dev_stop' and subsequent 'add-port' will
just start the already existing device. Such behaviour
will not reset the device to initial state as it could
be expected. For example: virtual pcap pmd will continue
reading input file instead of reading it from the beginning.
3. Follows from 2:
After execution of the following commands 'port1' will be
configured with the 'old-options' while 'ovs-vsctl show'
will show us 'new-options' in dpdk-devargs field:
ovs-vsctl add-port port1 -- set interface port1 type=dpdk \
options:dpdk-devargs=<eth_pmd_name1>,<old-options>
ovs-vsctl del-port port1
ovs-vsctl add-port port1 -- set interface port1 type=dpdk \
options:dpdk-devargs=<eth_pmd_name1>,<new-options>
4. Follows from 1:
Not detached device consumes 'port_id'. Since we have very
limited number of 'port_id's (32 in common case) this may
lead to quick exhausting of id pool and inability to add any
other port.
To avoid above issues we need to detach all the attached devices on
port destruction.
appctl 'netdev-dpdk/detach' removed because not needed anymore.
We need to use internal 'attached' variable to track ports on
which rte_eth_dev_attach() was called and returned successfully
to avoid closing and detaching devices that do not support hotplug or
by any other reason attached using the 'dpdk-extra' cmdline options.
CC: Ciara Loftus <ciara.loftus@intel.com>
Fixes: 55e075e65ef9 ("netdev-dpdk: Arbitrary 'dpdk' port naming")
Fixes: 69876ed78611 ("netdev-dpdk: Add support for virtual DPDK PMDs (vdevs)")
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Billy O'Mahony <billy.o.mahony@intel.com>
Update the documentation with the information on the megaflow hits
observed with the default 'emc-insert-inv-prob' value. Also add the
recommended setting for achieving higher forwarding performance.
Signed-off-by: Bhanuprakash Bodireddy <bhanuprakash.bodireddy@intel.com>
CC: Ciara Loftus <ciara.loftus@intel.com>
CC: Georg Schmuecking <georg.schmuecking@ericsson.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Kevin Traynor <ktraynor@redhat.com>
OVS no longer expects DPDK devices to start with dpdk and end with a
portid. Update docs to reflect this.
Signed-off-by: Cian Ferriter <cian.ferriter@intel.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Add instructions on how to retrieve extended statistics for DPDK physical
and vhost ports.
Signed-off-by: Bhanuprakash Bodireddy <bhanuprakash.bodireddy@intel.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Unconditional insertion of EMC entries results in EMC thrashing at high
numbers of parallel flows. When this occurs, the performance of the EMC
often falls below that of the dpcls classifier, rendering the EMC
practically useless.
Instead of unconditionally inserting entries into the EMC when a miss
occurs, use a 1% probability of insertion. This ensures that the most
frequent flows have the highest chance of creating an entry in the EMC,
and the probability of thrashing the EMC is also greatly reduced.
The probability of insertion is configurable, via the
other_config:emc-insert-inv-prob option. This value sets the average
probability of insertion to 1/emc-insert-inv-prob.
For example the following command changes the insertion probability to
(on average) 1 in every 20 packets ie. 1/20 ie. 5%.
ovs-vsctl set Open_vSwitch . other_config:emc-insert-inv-prob=20
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Signed-off-by: Georg Schmuecking <georg.schmuecking@ericsson.com>
Co-authored-by: Georg Schmuecking <georg.schmuecking@ericsson.com>
Acked-by: Kevin Traynor <ktraynor@redhat.com>
Signed-off-by: Daniele Di Proietto <diproiettod@vmware.com>
Update the "deep dive" and "howto" pages with headings that more clearly
indicate the separate lists of OVS or OVN content. Also add a link to
ovn-architecture from the "deep dive" page as it seems quite relevant
there.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
options:dpdk-devargs is always required now. This commit also changes
some of the names from 'dpdk0' to various others.
netdev-dpdk/detach accepts a PCI id instead of a port name.
CC: Ciara Loftus <ciara.loftus@intel.com>
Fixes: 55e075e65ef9("netdev-dpdk: Arbitrary 'dpdk' port naming")
Signed-off-by: Daniele Di Proietto <diproiettod@vmware.com>
Acked-by: Ciara Loftus <ciara.loftus@intel.com>
Strip a couple of closing HTML tags that were left over from when this doc
was converted from the web site to RST.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Prior to this commit, the 'dpdk' port type could only be used for
physical DPDK devices. Now, virtual devices (or 'vdevs') are supported.
'vdev' devices are those which use virtual DPDK Poll Mode Drivers eg.
null, pcap. To add a DPDK vdev, a valid 'dpdk-devargs' must be set for
the given dpdk port. The format expected is 'eth_<driver_name><x>' where
'x' is a number between 0 and RTE_MAX_ETHPORTS -1.
For example to add a port that uses the 'null' DPDK PMD driver:
ovs-vsctl set Interface null0 options:dpdk-devargs=eth_null0
Not all DPDK vdevs have been verified to work at this point in time.
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Acked-by: Stephen Finucane <stephen@that.guru> # docs only
Signed-off-by: Daniele Di Proietto <diproiettod@vmware.com>
'dpdk' ports no longer have naming restrictions. Now, instead of
specifying the dpdk port ID as part of the name, the PCI address of the
device must be specified via the 'dpdk-devargs' option. eg.
ovs-vsctl add-port br0 my-port
ovs-vsctl set Interface my-port type=dpdk
options:dpdk-devargs=0000:06:00.3
The user must no longer hotplug attach DPDK ports by issuing the
specific ovs-appctl netdev-dpdk/attach command. The hotplug is now
automatically invoked when a valid PCI address is set in the
dpdk-devargs. The format for ovs-appctl netdev-dpdk/detach command
has changed in that the user now must specify the relevant PCI address
as input instead of the port name.
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Signed-off-by: Kevin Traynor <ktraynor@redhat.com>
Co-authored-by: Kevin Traynor <ktraynor@redhat.com>
Acked-by: Stephen Finucane <stephen@that.guru> # docs only
Signed-off-by: Daniele Di Proietto <diproiettod@vmware.com>
In order to use dpdk ports in ovs they have to be bound to a DPDK
compatible driver before ovs is started.
This patch adds the possibility to hotplug (or hot-unplug) a device
after ovs has been started. The implementation adds two appctl commands:
netdev-dpdk/attach and netdev-dpdk/detach
After the user attaches a new device, it has to be added to a bridge
using the add-port command, similarly, before detaching a device,
it has to be removed using the del-port command.
Signed-off-by: Mauricio Vasquez B <mauricio.vasquezbernal@studenti.polito.it>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Co-authored-by: Ciara Loftus <ciara.loftus@intel.com>
Acked-by: Stephen Finucane <stephen@that.guru> # docs only
Signed-off-by: Daniele Di Proietto <diproiettod@vmware.com>
ivshmem is a path to the guest using DPDK rings that was
introduced before userspace vhost was available in the OVS-DPDK
datapath. ivshmem is external to OVS but the scheme of using it
with DPDK rings is documented.
Remove ivshmem instruction documentation because:
- The ivshmem library has been removed in DPDK since DPDK 16.11.
- The instructions/scheme provided will not work with current
supported and future DPDK versions.
- The linked patch needed to enable support in QEMU has never
been upstreamed and does not apply to the last 4 QEMU releases.
- Userspace vhost has become the defacto OVS-DPDK path to the guest.
Fixes: 04de404e1bfa ("netdev-dpdk: Add support for DPDK 16.11")
Cc: Ciara Loftus <ciara.loftus@intel.com>
Cc: Stephen Finucane <stephen@that.guru>
Signed-off-by: Kevin Traynor <ktraynor@redhat.com>
Acked-by: Stephen Finucane <stephen@that.guru>
Acked-by: Mauricio Vasquez B <mauricio.vasquez@polito.it>
Signed-off-by: Daniele Di Proietto <diproiettod@vmware.com>
Add Rx checksum offloading feature support on DPDK physical ports. By default,
the Rx checksum offloading is enabled if NIC supports. However,
the checksum offloading can be turned OFF either while adding a new DPDK
physical port to OVS or at runtime.
The rx checksum offloading can be turned off by setting the parameter to
'false'. For eg: To disable the rx checksum offloading when adding a port,
'ovs-vsctl add-port br0 dpdk0 -- \
set Interface dpdk0 type=dpdk options:rx-checksum-offload=false'
OR (to disable at run time after port is being added to OVS)
'ovs-vsctl set Interface dpdk0 options:rx-checksum-offload=false'
Similarly to turn ON rx checksum offloading at run time,
'ovs-vsctl set Interface dpdk0 options:rx-checksum-offload=true'
The Tx checksum offloading support is not implemented due to the following
reasons.
1) Checksum offloading and vectorization are mutually exclusive in DPDK poll
mode driver. Vector packet processing is turned OFF when checksum offloading
is enabled which causes significant performance drop at Tx side.
2) Normally, OVS generates checksum for tunnel packets in software at the
'tunnel push' operation, where the tunnel headers are created. However
enabling Tx checksum offloading involves,
*) Mark every packets for tx checksum offloading at 'tunnel_push' and
recirculate.
*) At the time of xmit, validate the same flag and instruct the NIC to do the
checksum calculation. In case NIC doesnt support Tx checksum offloading,
the checksum calculation has to be done in software before sending out the
packets.
No significant performance improvement noticed with Tx checksum offloading
due to the e overhead of additional validations + non vector packet processing.
In some test scenarios, it introduces performance drop too.
Rx checksum offloading still offers 8-9% of improvement on VxLAN tunneling
decapsulation even though the SSE vector Rx function is disabled in DPDK poll
mode driver.
Signed-off-by: Sugesh Chandran <sugesh.chandran@intel.com>
Acked-by: Jesse Gross <jesse@kernel.org>
Acked-by: Pravin B Shelar <pshelar@ovn.org>
These were found using the 'linkcheck' builder.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Cc: Ben Pfaff <blp@ovn.org>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Combined, the dpdk and dpdk-advanced installation documents provide a
lot of useful information, but most of this information is unrelated to
installation. Rework these documents, completely breaking up the
dpdk-advanced document into multiple smaller documents in other sections
and moving non-install aspects of the dpdk document into these sections.
This aims to tie the DPDK docs into the documentation structure.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Instructions were provided in the userspace tunneling cookbook.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Signed-off-by: Ben Pfaff <blp@ovn.org>
There are many docs that don't need to kept at the top level, along
with many more hidden in random folders. Move them all.
This also allows us to add the '-W' flag to Sphinx, ensuring unindexed
docs result in build failures.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Signed-off-by: Ben Pfaff <blp@ovn.org>
There are a couple of references to these for various build systems.
The website is going to be our "one true resource" for all docs, so
simply remove these references.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Signed-off-by: Ben Pfaff <blp@ovn.org>
This is a dumb move of all 'INSTALL*' docs, with very little
refactoring (mostly updating links and making the titles a little more
consistent. Additional refactoring will be done in subsequent changes.
Signed-off-by: Stephen Finucane <stephen@that.guru>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Create a series of sections, all of which are currently empty, using
the general design established by Jacob Kaplan-Moss and the Django
project [1]. Five sections are provided:
- intro
- tutorials
- topics
- howto
- ref
- faq
- internals
The purpose of each section is described in the documents themselves.
[1] https://jacobian.org/writing/great-documentation/
Signed-off-by: Stephen Finucane <stephen@that.guru>
Signed-off-by: Ben Pfaff <blp@ovn.org>
