Checking for each of the required AVX512 ISA separately will allow the
compiler to generate some AVX512 code where there is some support in the
compiler rather than only generating all AVX512 code when all of it is
supported or no AVX512 code at all.
For example, in GCC 4.9 where there is just support for AVX512F, this
patch will allow building the AVX512 DPIF.
Another example, in GCC 5 and 6, most AVX512 code can be generated, just
without AVX512VPOPCNTDQ support.
Signed-off-by: Cian Ferriter <cian.ferriter@intel.com>
Acked-by: Sunil Pai G <sunil.pai.g@intel.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
No instructions from the AVX512VL ISA are used.
Compilation for AVX512F and AVX512 BW ISA are already enabled in
lib/automake.mk for the dpif-netdev-lookup-avx512-gather.c file because
it's part of the libopenvswitchavx512.la library. They don't need to be
enabled at a function level.
Remove these unnecessary function-level compiler target attributes.
Signed-off-by: Cian Ferriter <cian.ferriter@intel.com>
Acked-by: Sunil Pai G <sunil.pai.g@intel.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
For packets which don't already have a hash calculated,
miniflow_hash_5tuple() calculates the hash of a packet
using the previously built miniflow.
This commit adds IPv4 profile specific hashing which
uses fixed offsets into the packet to improve hashing
performance.
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Co-authored-by: Harry van Haaren <harry.van.haaren@intel.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Co-authored-by: Ilya Maximets <i.maximets@ovn.org>
Signed-off-by: Kumar Amber <kumar.amber@intel.com>
Acked-by: Cian Ferriter <cian.ferriter@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
The code changes here are to handle (1 << i) shifts where 'i' is the
packet index in the batch, and 1 << 31 is an overflow of the signed '1'.
Fixed by adding UINT32_C() around the 1 character, ensuring compiler knows
the 1 is unsigned (and 32-bits). Undefined Behaviour sanitizer is now happy
with the bit-shifts at runtime.
Suggested-by: Ilya Maximets <i.maximets@ovn.org>
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
This commit fixes the minimum packet size for the vlan/ipv4/tcp
traffic profile, which was previously incorrectly set.
This commit also disallows any fragmented IPv4 packets from being
matched in the optimized miniflow-extract, avoiding complexity of
handling fragmented packets and using scalar fallback instead.
The DF (don't fragment) bit is now ignored, and stripped from the
resulting miniflow.
Fixes: aa85a25095 ("dpif-netdev/mfex: Add more AVX512 traffic profiles.")
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Tested-by: Kumar Amber <kumar.amber@intel.com>
Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
This commit improves handling of packets where the allocated memory
is less than 64 bytes. For packets recevied from DPDK ports this
never matters, as an mbuf always pre-allocates enough space, however
this can occur in cases where packet received from a kernel interface
or injected by an OpenFlow controller. The fix is required to
ensure OVS doesn't overread the allocated memory, e.g.:
==49944==ERROR: AddressSanitizer: heap-buffer-overflow on address
0x6060000d8181 at pc 0x000001cb9d24 bp 0x7ffce3b385d0 sp 0x7ffce3b385c8
READ of size 64 at 0x6060000d8181 thread T0
#0 0x1cb9d23 in mfex_avx512_process lib/dpif-netdev-extract-avx512.c:491:26
#1 0x1cb9d23 in mfex_avx512_ip_udp lib/dpif-netdev-extract-avx512.c:625:1
#2 0x18786a1 in dpif_miniflow_extract_autovalidator lib/dpif-netdev-private-extract.c:277:29
#3 0x1cbca5c in dp_netdev_input_outer_avx512 lib/dpif-netdev-avx512.c:159:19
#4 0x1853048 in dp_netdev_process_rxq_port lib/dpif-netdev.c:4900:19
#5 0x1837c76 in dpif_netdev_run lib/dpif-netdev.c:6197:25
#6 0x1727a02 in type_run ofproto/ofproto-dpif.c:370:9
#7 0x16f6e07 in ofproto_type_run ofproto/ofproto.c:1778:31
#8 0x16c1a8b in bridge_run__ vswitchd/bridge.c:3245:9
#9 0x16bd2fd in bridge_run vswitchd/bridge.c:3310:5
#10 0x16db8fe in main vswitchd/ovs-vswitchd.c:127:9
#11 0x7fbc0c5b61a2 in __libc_start_main (/lib64/libc.so.6+0x271a2)
#12 0xedabbd in _start (vswitchd/ovs-vswitchd+0xedabbd)
0x6060000d8181 is located 9 bytes to the right of 56-byte
region [0x6060000d8140,0x6060000d8178)
allocated by thread T0 here:
#0 0xf7b09f in malloc (vswitchd/ovs-vswitchd+0xf7b09f)
#1 0x1aff3b9 in xmalloc__ lib/util.c:137:15
#2 0x1aff3b9 in xmalloc lib/util.c:172:12
#3 0x1afe211 in process_command lib/unixctl.c:310:13
#4 0x1afe211 in run_connection lib/unixctl.c:344:17
#5 0x1afe211 in unixctl_server_run lib/unixctl.c:395:21
#6 0x16db918 in main vswitchd/ovs-vswitchd.c:128:9
#7 0x7fbc0c5b61a2 in __libc_start_main (/lib64/libc.so.6+0x271a2)
The solution implemented uses a mask-to-zero if the available buffer
size is less than 64 bytes, and a branch for which type of load is used.
Fixes: 250ceddcc2d0 ("dpif-netdev/mfex: Add AVX512 based optimized miniflow extract")
Reported-by: Ilya Maximets <i.maximets@ovn.org>
Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
DPIF AVX512 optimizations currently rely on DPDK availability while
they can be used without DPDK.
Besides, checking for availability of some isa only has to be done once
and won't change while a OVS process runs.
Resolve isa availability in constructors by using a simplified query
based on cpuid API that comes from the compiler.
Note: this also fixes the check on BMI2 availability: DPDK had a bug
for this isa, see https://git.dpdk.org/dpdk/commit/?id=aae3037ab1e0.
Suggested-by: Ilya Maximets <i.maximets@ovn.org>
Signed-off-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
This commit tightens the requirements for processing TCP packets
in AVX512, ensuring that there are no TCP options by validating that
the "data offset" field of the TCP header is exactly equal to 5.
This ensures that the TCP header is not too short, and that it does
not contain extra options.
On the IP handling side, improve checks around total packet length.
Now the next protocol is included in the length checks, ensuring that
the IP header reported length is of appropriate size to contain the
next protocol (e.g. UDP requires 8 bytes, TCP requires 20). Note that
the inner protocol is always of a fixed size per profile, so it can be
set using the UDP_ and TCP_ HEADER_LEN defines.
Fixes: 250ceddcc2d0 ("dpif-netdev/mfex: Add AVX512 based optimized miniflow extract")
Reported-by: Ilya Maximets <i.maximets@ovn.org>
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
This commit adds 3 new traffic profile implementations to the
existing avx512 miniflow extract infrastructure. The profiles added are:
- Ether()/IP()/TCP()
- Ether()/Dot1Q()/IP()/UDP()
- Ether()/Dot1Q()/IP()/TCP()
The design of the avx512 code here is for scalability to add more
traffic profiles, as well as enabling CPU ISA. Note that an implementation
is primarily adding static const data, which the compiler then specializes
away when the profile specific function is declared below.
As a result, the code is relatively maintainable, and scalable for new
traffic profiles as well as new ISA, and does not lower performance
compared with manually written code for each profile/ISA.
Note that confidence in the correctness of each implementation is
achieved through autovalidation, unit tests with known packets, and
fuzz tested packets.
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Eelco Chaudron <echaudro@redhat.com>
Acked-by: Flavio Leitner <fbl@sysclose.org>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
This commit adds AVX512 implementations of miniflow extract.
By using the 64 bytes available in an AVX512 register, it is
possible to convert a packet to a miniflow data-structure in
a small quantity instructions.
The implementation here probes for Ether()/IP()/UDP() traffic,
and builds the appropriate miniflow data-structure for packets
that match the probe.
The implementation here is auto-validated by the miniflow
extract autovalidator, hence its correctness can be easily
tested and verified.
Note that this commit is designed to easily allow addition of new
traffic profiles in a scalable way, without code duplication for
each traffic profile.
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Eelco Chaudron <echaudro@redhat.com>
Acked-by: Flavio Leitner <fbl@sysclose.org>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
