Non-leader revalidator thread uses pthread_barrier_* functions in their
main loop to synchronize with leader thread. However, since those threads
only call poll_block() intermittently, the poll interval check in
poll_block() can wrongly take the time since last call as poll interval
and issue the following warnings:
"Unreasonably long XXXXms poll interval".
To prevent it, this commit implements the barrier struct and operations
for OVS which allow thread to block on barrier via poll_block().
Signed-off-by: Alex Wang <alexw@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Between fork() and execvp() calls in the process_start()
function both child and parent processes share the same
file descriptors. This means that, if a child process
received a signal during this time interval, then it could
potentially write data to a shared file descriptor.
One such example is fatal signal handler, where, if
child process received SIGTERM signal, then it would
write data into pipe. Then a read event would occur
on the other end of the pipe where parent process is
listening and this would make parent process to incorrectly
believe that it was the one who received SIGTERM.
Also, since parent process never reads data from this
pipe, then this bug would make parent process to consume
100% CPU by immediately waking up from the event loop.
This patch will help to avoid this problem by blocking
signals until child closes all its file descriptors.
Signed-off-by: Ansis Atteka <aatteka@nicira.com>
Reported-by: Suganya Ramachandran <suganyar@vmware.com>
Issue: 1255110
This makes the message issued refer to the file and line that called
ovs_mutex_lock(), instead of to the file and line *inside*
ovs_mutex_lock().
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Andy Zhou <azhou@nicira.com>
With glibc, a mutex or rwlock filled with all-zero-bytes is properly
initialized for use, but this is not true for any other libc that OVS
supports. However, OVS gets a lot more testing with glibc than any other
libc. This means that developers keep introducing bugs that do not
manifest on the main development platform.
This commit should help avoid the problem, by reusing the existing 'where'
members to indicate whether a mutex or rwlock has been initialized.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Thread names are occasionally very useful for debugging, but from time to
time we've forgotten to set one. This commit adds the new thread's name
as a parameter to the function to start a thread, to make that mistake
impossible. This also simplifies code, since two function calls become
only one.
This makes a few other changes to the thread creation function:
* Since it is no longer a direct wrapper around a pthread function,
rename it to avoid giving that impression.
* Remove 'pthread_attr_t *' param that every caller supplied as NULL.
* Change 'pthread *' parameter into a return value, for convenience.
The system-stats code hadn't set a thread name, so this fixes that issue.
This patch is a prerequisite for making RCU report the name of a thread
that is blocking RCU synchronization, because the easiest way to do that is
for ovsrcu_quiesce_end() to record the current thread's name.
ovsrcu_quiesce_end() is called before the thread function is called, so it
won't get a name set within the thread function itself. Setting the thread
name earlier, as in this patch, avoids the problem.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Alex Wang <alexw@nicira.com>
Otherwise the udpif revalidator threads can postpone RCU callbacks
essentially forever, especially if there are many revalidator threads and
little network traffic.
Reported-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Alex Wang <alexw@nicira.com>
This allows clients to do more than just increment a counter. The
following commit will make the first use of that feature.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Andy Zhou <azhou@nicira.com>
RCU allows multiple threads to read objects in parallel without any
performance penalty. The following commit will introduce the first use.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Andy Zhou <azhou@nicira.com>
We use the number of cpu cores to determine the number
of threads that we spawn. We are not yet sure what is
the ideal number of OVS userspace threads that can run
on Hyper-V. Till we figure that out, use the same logic
of counting CPU cores in Windows too.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
glibc supports two kinds of rwlocks:
- The default kind of rwlock always allows recursive read-locks to
succeed, but threads blocked on acquiring the write-lock are treated
unfairly, causing them to be delayed indefinitely as long as new
readers continue to come along.
- An alternative "writer nonrecursive" rwlock allows recursive
read-locks to succeed only if there are no threads waiting for the
write-lock. Otherwise, recursive read-lock attempts deadlock in
the presence of blocking write-lock attempts. However, this kind
of rwlock is fair to writer.
POSIX allows the latter behavior, which essentially means that any portable
pthread program cannot try to take read-locks recursively. Since that's
true, we might as well use the latter kind of rwlock with glibc and get the
benefit of fairness of writers.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Joe Stringer <joestringer@nicira.com>
A couple of times I've wanted to create a dynamic data structure that has
thread-specific data, but I've not been able to do that because
PTHREAD_KEYS_MAX is so low (POSIX says at least 128, glibc is only a little
bigger at 1024). This commit introduces a new form of thread-specific data
that supports a large number of items.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
ovsthread_counter is an abstract interface that could be implemented
different ways. The initial implementation is simple but less than
optimally efficient.
Signed-off-by: Ben Pfaff <blp@nicira.com>
ofproto_set_threads() uses the calculation MAX(count_cpu_cores() - 2, 1)
to decide on the default thread count. However, count_cpu_cores() returns
0 if it can't count the number of cores, or 1 if there's only one core,
and that causes the calculation to come out as UINT_MAX-2 or UINT_MAX-1,
respectively, which causes a memory allocation failure later.
There are other ways to fix this problem, too, of course.
Signed-off-by: Ben Pfaff <blp@nicira.com>
This was causing test failures on non-Linux platforms.
Reported-by: Ed Maste <emaste@freebsd.org>
Signed-off-by: Joe Stringer <joestringer@nicira.com>
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
On systems that provide /proc/cpuinfo similar to Linux on x86, this
should allow us to choose a better default value for the number of
upcall handler threads -- in particular, it avoids counting
hyper-thread cores. If /proc/cpuinfo cannot be parsed for any reason,
fall back to using sysconf().
Signed-off-by: Joe Stringer <joestringer@nicira.com>
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
I don't see any other way to make Clang realize that these are the real
mutex implementation functions.
I first noticed these warnings with Clang 1:3.4~svn188890-1~exp1.
I previously used version 1:3.4~svn187484-1~exp1.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
We've seen a number of deadlocks in the tree since thread safety was
introduced. So far, all of these are self-deadlocks, that is, a single
thread acquiring a lock and then attempting to re-acquire the same lock
recursively. When this has happened, the process simply hung, and it was
somewhat difficult to find the cause.
POSIX "error-checking" mutexes check for this specific problem (and
others). This commit switches from other types of mutexes to
error-checking mutexes everywhere that we can, that is, everywhere that
we're not using recursive mutexes. This ought to help find problems more
quickly in the future.
There might be performance advantages to other kinds of mutexes in some
cases. However, the existing mutex type choices were just guesses, so I'd
rather go for easy detection of errors until we know that other mutex
types actually perform better in specific cases. Also, I did a quick
microbenchmark of glibc mutex types on my host and found that the
error checking mutexes weren't any slower than the other types, at least
when the mutex is uncontended.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
I foresee a need for possibly large numbers of instances of "struct
seq" (which is introduced in an upcoming patch). Each struct seq
needs some per-thread data. POSIX has pthread_key_t for this, but
the number of keys can be fairly limited, to as few as 128. It is
reasonable to work around this by using a hash table indexed on the
current thread. That only works if one can get a thread identifier
that is hashable (pthread_t is not). This patch introduces a
hashable thread identifier.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Commit 97be153858b4cd175cbe7862b8e1624bf22ab98a (clang: Add
annotations for thread safety check.) defined 'struct ovs_mutex'
variable in 'atomic_flag' in 'ovs-atomic-pthreads.h'. This
casued "mutex: has incomplete type" error in compilation when
'ovs-atomic-pthreads.h' is included.
This commit goes back to use 'pthread_mutex_t' for that variable
and adds test for the 'atomic_flag' related functions.
Reported-by: Gurucharan Shetty <gshetty@nicira.com>
Signed-off-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
This commit adds annotations for thread safety check. And the
check can be conducted by using -Wthread-safety flag in clang.
Co-authored-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
In C, one can do preprocessor tricks by making a macro expansion include
the macro's own name. We actually used this in the tree to automatically
provide function arguments, e.g.:
int f(int x, const char *file, int line);
#define f(x) f(x, __FILE__, __LINE__)
...
f(1); /* Expands to a call like f(1, __FILE__, __LINE__); */
However it's somewhat confusing, so this commit stops using that trick.
Reported-by: Ed Maste <emaste@freebsd.org>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ed Maste <emaste@freebsd.org>
pthread_once() is portable but it does not allow passing any parameters to
the initialization function, which is often inconvenient, because it means
that the function can only access data declared at file scope. This commit
introduces an alternative with a more convenient interface.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
The only tricky part here is that I'm throwing in annotations to allow
"sparse" to report unbalanced locking.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
