This commit adds annotations for thread safety check. And the
check can be conducted by using -Wthread-safety flag in clang.
Co-authored-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
This disentangles "struct nl_dump" from "struct nl_sock", clearing the way
to make the use of either one thread-safe in an obviously correct manner.
Signed-off-by: Ben Pfaff <blp@nicira.com>
The "stress" library was introduced years ago. We intended at the time to
start using it to provoke errors in testing, to make sure that Open vSwitch
was resilient against those errors. The intention was good, but there were
few actual implementations of stress options, and the testing never
materialized.
Rather than adapt the stress library for thread safety, this seems like a
good opportunity to remove it, so this commit does so.
Signed-off-by: Ben Pfaff <blp@nicira.com>
This was the only obvious use of bare malloc() in the tree, other
than in the implementation of wrapper functions.
Signed-off-by: Ben Pfaff <blp@nicira.com>
This patch fixes tests when they are run with "fakeroot debian/rules binary"
command.
The problem was that under fakeroot setsockopt() call could still return
EPERM and lead to a warning message being logged.
Signed-off-by: Ansis Atteka <aatteka@nicira.com>
Some Open vSwitch utilities can do useful work when they are not run as
root. Without this commit, these utilities will log a warning on failure
to use the SO_RCVBUFFORCE socket option if they open any Netlink sockets.
This will always happen, it does not report anything unexpected or
fixable as non-root, and sometimes it makes users wonder if something is
wrong, so there is no benefit to logging it. This commit drops it in that
case.
Signed-off-by: Ben Pfaff <blp@nicira.com>
This is a straight search-and-replace, except that I also removed #include
<assert.h> from each file where there were no assert calls left.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
Casts are sometimes necessary. One common reason that they are necessary
is for discarding a "const" qualifier. However, this can impede
maintenance: if the type of the expression being cast changes, then the
presence of the cast can hide a necessary change in the code that does the
cast. Using CONST_CAST, instead of a bare cast, makes these changes
visible.
Inspired by my own work elsewhere:
http://git.savannah.gnu.org/cgit/pspp.git/tree/src/libpspp/cast.h#n80
Signed-off-by: Ben Pfaff <blp@nicira.com>
Commit 7d7447 (netlink: Postpone choosing sequence numbers until send
time.) broke ovs-brcompatd because it prevented userspace replies to
kernel requests from using the correct sequence numbers. This commit fixes
it.
Atzm Watanabe found the root cause and provided an alternative patch to
avoid the problem.
Reported-by: André Ruß <andre.russ@hybris.com>
Reported-by: Atzm Watanabe <atzm@stratosphere.co.jp>
Tested-by: Atzm Watanabe <atzm@stratosphere.co.jp>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Replaced all instances of Nicira Networks(, Inc) to Nicira, Inc.
Feature #10593
Signed-off-by: Raju Subramanian <rsubramanian@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Typically an nl_sock client can stack-allocate the buffer for receiving
a Netlink message, which provides a performance boost.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Choosing sequence numbers at time of creating a packet means that
nl_sock_transact_multiple() has to search for the sequence number
of a reply, because the sequence numbers of the requests aren't
necessarily sequential. This commit makes it possible to avoid
the search, by deferring choice of sequence numbers until the
time that we send the packets. It doesn't actually modify
nl_sock_transact_multiple(), which will happen in a later commit.
Previously, I was concerned about a theoretical race condition
described in a comment in the old versino of this code:
This implementation uses sequence numbers that are unique
process-wide, to avoid a hypothetical race: send request, close
socket, open new socket that reuses the old socket's PID value,
send request on new socket, receive reply from kernel to old
socket but with same PID and sequence number. (This race could be
avoided other ways, e.g. by preventing PIDs from being quickly
reused).
However, I no longer believe that this can be a real problem,
because Netlink operates synchronously. The reply to a request
will always arrive before the socket can be closed and a new
socket opened with the old socket's PID.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Open vSwitch userspace can set up flows at a high rate, but it is somewhat
"bursty" in opportunities to set up flows, by which I mean that OVS sets up
a batch of flows, then goes off and does some other work for a while, then
sets up another batch of flows, and so on. The result is that, if a large
number of packets that need flow setups come in all at once, then some of
them can overflow the relatively small kernel-to-user buffers.
This commit increases the kernel-to-user buffers from the default of
approximately 120 kB each to 1 MB each. In one somewhat synthetic test
case that I ran based on an "hping3" that generated a load of about 20,000
new flows per second (including both requests and replies), this reduced
the packets dropped at the kernel-to-user interface from about 30% to none.
I expect that it will similarly improve packet loss in workloads where
flow arrival is not easily predictable.
(This has little effect on workloads generated by "ovs-benchmark rate"
because that benchmark is effectively "self-clocking", that is, a new flow
is triggered only by a reply to a request made earlier, which means that
the number of buffered packets at any given has a known, constant upper
limit.)
Bug #10210.
Signed-off-by: Ben Pfaff <blp@nicira.com>
The Netlink code in the Linux kernel has been willing to choose unique
Netlink pids for userspace sockets since at least 2.4.36 and probably
earlier. There's no value in choosing them ourselves.
This simplifies the code and eliminates the possibility of exhausting our
supply of Netlink PIDs.
Using poll() internally in dpif_linux_recv(), instead of relying
on the results of the main loop poll() call, brings netperf CRR
performance back within 1% of par versus the code base before the
poll_fd_woke() optimizations were introduced. It also increases
the ovs-benchmark results by about 5% versus that baseline, too.
My theory is that this is because the main loop takes long enough
that a significant number of packets can arrive during the main
loop itself, so this reduces the time before OVS gets to those
packets.
This reverts commit 1e276d1a10539a8cd97d2ad63c073a9a43f0f1ef.
The poll_fd_woke() and nl_sock_woke() function added in that commit are
no longer used, so there is no reason to keep them in the tree.
Each time we run through the poll loop, we check all file descriptors
that we were waiting on to see if there is data available. However,
this requires a system call and poll already provides information on
which FDs caused the wakeup so it is inefficient as the number of
active FDs grows. This provides a way to check whether a given FD
has data.
In the future, the kernel will use unicast messages instead of
multicast to send upcalls. As a result, we need to be able to
tell it where to direct the traffic. This adds a function to expose
the Netlink pid of a socket so it can be included in messages to the
kernel.
A Netlink socket that receives asynchronous notifications (e.g. from a
multicast group) cannot be used for transactions or dumps, because those
operations would discard asynchronous messages that arrive while waiting
for replies.
This commit documents this issue in a comment on nl_sock_join_mcgroup().
It also removes an internal attempt to avoid mixing multicast reception
with other operations. The attempt was incomplete, because it only
handled dumps even though ordinary transactions are also problematic. It
seems better to remove it than to fix it because, first, all of the
existing users in OVS already separate multicast reception from other
operations and, second, an upcoming commit will start using unicast
Netlink for asynchronous notifications, which has the same issues but
doesn't use nl_sock_join_mcgroup().
The nl_lookup_genl_mcgroup() function can fail on older kernels
which do not support the required netlink interface. Before this
patch, dpif-linux would refuse to create a datapath when this
happened. With this patch, it attempts to use a workaround. If
the workaround fails it simply disables the affected features
without completely disabling the dpif.
Commit e408762f "netlink-socket: New function nl_lookup_genl_mcgroup()"
modified do_lookup_genl_family() to return the Netlink attributes to the
caller, but it still freed the Netlink message itself, which meant that
the attributes pointed into freed memory. This commit fixes the problem.
This commit is not a minimal fix. It refactors do_lookup_genl_family(),
changing the return value from "negative errno value or positive genl
family id" to the more common "zero or positive errno value".
Found by valgrind.
Until now, each attempt to receive a message from a Netlink socket has
taken at least two system calls, one to check the size of the message to
be received and a second one to delete the message from the socket buffer.
This commit switches to a new strategy that requires only one system call
per message received.
In my testing this increases the maximum flow setups per second by a little
over 10%.
The ids for Generic Netlink family names aren't very helpful because they
can vary from machine to machine and even from one boot to the next. So
this change logs their names too.
This only affects logging at DBG level.
Reviewed by Ethan Jackson <ethan@nicira.com>.
nlmsghdr_to_string() wrote sequence numbers in hex, but nl_sock_transact()
wrote them in decimal. This consistently switches to hexadecimal.
Reviewed by Ethan Jackson <ethan@nicira.com>.
It's not safe to use a single Netlink fd to do multiple operations in an
synchronous way. Some of the limitations are fundamental; for example, the
kernel only supports a single "dump" operation at a time. Others are
limitations imposed by the OVS coding style; for example, our Netlink
library is not callback based, so nothing can be done about incoming
messages that can't be handled immediately. Regardless, in OVS multicast
groups, transactions, and dumps cannot coexist on a single nl_sock.
This is only mildly irritating at the moment, but it will become much worse
later on, when dpif-linux shifts to using Netlink dumps for listing various
kinds of datapath entities. When that happens, a dump will be in progress
in situations where the dpif-linux client might want to do other
operations. For example, it is reasonable for the client to list flows
and, in the middle, look up information on vports mentioned in those flows.
It might be possible to simply ban and avoid such nested operations--I have
not even audited the source tree to find out whether we do anything like
that already--but that seems like an unnecessary cramp on our coding style.
Furthermore, it's difficult to explain and justify without understanding
the implementation.
This patch takes another approach, by improving the Netlink socket library
to avoid artificial constraints. When an operation, or a dump, or joining
a multicast group would cause a problem, this patch makes the library
transparently create a separate Netlink socket. This solves the problem
without putting any onerous restrictions on use.
This commit also slightly simplifies netdev_vport_reset_names(). It had
been written to destroy the dump object before the Netlink socket that it
used, but this is no longer necessary and doing it in the opposite order
saved a few lines of code.
Reviewed by Ethan Jackson <ethan@nicira.com>.
When this library was originally implemented, support for Linux 2.4 was
important. The Netlink implementation in Linux only added support for
joining and leaving multicast groups after a socket is bound as of Linux
2.6.14, so the library did not support it either. But the current version
of Open vSwitch targets Linux 2.6.18 and over, so it's fine to add this
support now, and this commit does so.
This will be used more extensively in upcoming commits.
Reviewed by Justin Pettit.
The parts of the netlink module that are related to sockets are
Linux-specific, since only Linux has AF_NETLINK sockets. The rest can be
built anywhere. This commit breaks them into two modules, and builds the
generic one on all platforms.
Acked-by: Jesse Gross <jesse@nicira.com>
