init_clock begins with a memset of 0 of the full clock struct. This
memset at the end of a single struct member just makes extra sure that
it's set to 0, which is unnecessary.
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Most of the information that timeval was reporting for long poll intervals
was comparing per-thread with per-process statistics, which yielded
nonsense a lot of the time.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Alex Wang <alexw@nicira.com>
A new function vlog_insert_module() is introduced to avoid using
list_insert() from the vlog.h header.
Signed-off-by: Thomas Graf <tgraf@noironetworks.com>
Acked-by: Ben Pfaff <blp@nicira.com>
'timewarp_seq' should be initialized only once, while init_clock() is
called multiple times (once for each clock instance).
Found by valgrind
Signed-off-by: Daniele Di Proietto <ddiproietto@vmware.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
Till now, we were initializing 'unix_epoch' through time_init().
But if there was a call directly to xgettimeofday(), we would
miss the initialization causing overflows. This commit fixes it
by pre-calculating the value and assigning it globally.
Also add-in a missing return statement.
Reported-by: Alessandro Pilotti <apilotti@cloudbasesolutions.com>
Reported-by: Linda Sun <lsun@vmware.com>
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
The new command is of the form 'time/warp LARGE_MSECS MSECS'.
It advances the current monotonic time by LARGE_MSECS. This is done MSECS
at a time in each run of the main thread. This gives other threads
time to run after the clock has been advanced by MSECS.
The old command would continue to work.
Rationale: On Windows, process creation is slower. When we have tests
that run 'ovs-appctl time/warp MSECS' hundreds of times in a for loop,
the time it takes to complete the test increases. This is specially
true for bfd tests. For e.g, the 11 bfd tests would take 3.5 minutes
to complete before this change and now takes a little less than 2 minutes.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Otherwise ovsrcu_synchronize() busy-waits in its loop because its
poll_block() un-quiesces, causing the global_seqno to increase, which is
what it waits for.
Reported-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Alex Wang <alexw@nicira.com>
RCU allows multiple threads to read objects in parallel without any
performance penalty. The following commit will introduce the first use.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Andy Zhou <azhou@nicira.com>
The WSAPoll() function, which is similar to poll() doesnot
simply sleep when the fd array is NULL. So use Sleep() instead.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Use GetSystemTimePreciseAsFileTime() for gettimeofday().
GetSystemTimePreciseAsFileTime() provides the result that is more
high resolution than just the microsecond that gittimeofday() in
Linux provides. So we need to remove some additional precision.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
QueryPerformanceCounter() retrieves the current value of the performance
counter, which is a high resolution (<1us) time stamp that can be used for
time-interval measurements. So, use it for MONOTONIC clock.
The GetSystemTimePreciseAsFileTime() function retrieves the current system date
and time with the highest possible level of precision (<1us). Use it for
real time clock. This function returns a counter representing the number of
100-nanosecond intervals since January 1, 1601. To make it compatible with
Linux CLOCK_REALTIME, we need to calculate the 100-nanoseconds counter value
till 01/01/1970.
An upcoming commit implements gettimeofday() using the same clock, so,
carve out a function.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Windows does not have a SIGHUP or SIGALRM. It does have
a SIGINT and SIGTERM. The documentation at msdn says that
SIGINT is not supported for win32 applications because
WIN32 operating systems generate a new thread to specifically
handle Ctrl+C.
This commit handles SIGTERM for Windows. The documentation also
states that nothing generates SIGTERM in Windows, but one can
use raise(SIGTERM) to manage it. The idea for handling SIGTERM
for Windows is to just have a place holder if there is need to
raise() a signal for some other purpose.
We use SIGALRM in timeval.c if we wake up from a sleep after
'deadline'. For Windows, print an error message and then
use SIGTERM.
There is an atexit() function for Windows, so we can call cleanup
functions during exit.
An upcoming commit separately handles Ctrl+C so that we can call
clean up functions for that use case.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Use WaitForMultipleObjects for polling on windows. This works on all kinds
of objects, e.g. sockets, files, especially ioctl calls to the kernel.
poll_fd_wait_event() is used if events need to be passed to pollfds. latch
is signaled with event, to be waited/polled by WaitForMultipleObjects() as
well. Changed array of fds to hmap to check for duplicate fds.
Signed-off-by: Linda Sun <lsun@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
BFD tests have the code like the following.
# wait for a while to stablize everything.
for i in `seq 0 9`; do ovs-appctl time/warp 500; done
They no longer work as intended because BFD code is run in a
separate monitor thread these days. The loop merely "warp"
the time by 5000. The monitor thread should have been woken
at least once, but it's far from "wait for a while to stablize
everything."
This commit mitigates the problem by sleeping a little in the
appctl handler. This is not ideal but makes BFD tests success
on my environment.
Signed-off-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
Signed-off-by: Ben Pfaff <blp@nicira.com>
time_poll() calls log_poll_interval(), which in some circumstances calls
coverage_log(). Before this commit, time_poll() also called
coverage_clear() after log_poll_interval(). This made sense before commit
857165b5fd26 (coverage: Make thread-safe.), because coverage_log() would
log the most recent main loop's coverage counters separately and calling
coverage_clear() beforehand would zero out those counters. However, it
doesn't make sense any longer because the most recent loop's counters are
no longer separately logged and in fact this practice now means that the
most recent loop's counters are omitted from the logged counters.
Therefore, this commit moves the call to coverage_clear() earlier, so that
the most recent loop's counters are included.
Signed-off-by: Ben Pfaff <blp@nicira.com>
This commit makes the main thread wake up all other threads when time is
warped.
Signed-off-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
This commit changes the "ovs-appctl coverage/show" command to show the
the averaged per-second rates for the last few seconds, the last minute
and the last hour, and the total counts of all of the coverage counters.
Signed-off-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
It's only held briefly now and in general a mutex tends to be preferred for
that case.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
Commit 31ef9f5178dee18 (timeval: Remove CACHE_TIME scheme.) inadvertently
removed the ability to warp time forward, for use in tests, except when
time is stopped. This fixes the problem.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
Commit 31ef9f5178 (timeval: Remove CACHE_TIME scheme.) removed
initialization of a rwlock which is still used for some operations.
This restores it.
Signed-off-by: Ben Pfaff <blp@nicira.com>
This commit removes the CACHE_TIME scheme from timeval module. This
is for eliminating the lock contention over the read/write lock of
the cached time. To get the time, the thread now will directly do
the system call 'clock_gettime()'.
As a side effect, timer can only be warpped after timer is stopped
by 'appctl time/stop' command.
Signed-off-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
DEFINE_PER_THREAD_DATA always declared its data item as "static", meaning
that it was only directly visible within a single translation unit.
This commit adds additional forms of per-thread data that allow the data
to be accessible from multiple translation units.
Signed-off-by: Ben Pfaff <blp@nicira.com>
This commit adds annotations for thread safety check. And the
check can be conducted by using -Wthread-safety flag in clang.
Co-authored-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
This should make the "timeval" module thread-safe, except for its
calls into vlog (because vlog is not yet thread-safe).
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ed Maste <emaste@freebsd.org>
The backtrace feature of timeval is useful because it provides a "poor
man's profile" view of Open vSwitch. But it is not likely to be useful in
a multithreaded process, because signal delivery doesn't necessarily follow
the profile when there is more than one thread. (A signal in a
multithreaded process are delivered to an arbitrary thread.)
Another problem with the backtrace feature is that it is difficult for
format_backtraces() to synchronize properly with the signal handler in a
multithreaded process. In a single-threaded process, it can just block
the signal handler, but in a multithreaded process this does not prevent
signal delivery to threads other than the one running format_backtrace().
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ed Maste <emaste@freebsd.org>
These functions will not have the same useful effect when Open vSwitch
becomes multithreaded, because time_disable_restart() will disable time
advancing for every thread, not just for the thread that calls it.
These functions are no longer used, so this commit removes them.
Signed-off-by: Ben Pfaff <blp@nicira.com>
There is no need to do so because the signal handler does not read or
write 'deadline'.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
POSIX says that multithreaded programs must not use sigprocmask() but must
use pthread_sigmask() instead. This commit makes that replacement.
The actual use of signals in Open vSwitch is still not thread safe
following this commit, but this change is a necessary prerequisite for
fixing the other problems.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Other code in the tree uses HAVE_BACKTRACE and then blindly includes
<execinfo.h> if it is present, so this doesn't make anything worse.
Once we do that, HAVE_EXECINFO_H has no further users, so this commit also
removes the check for <execinfo.h>
Reported-by: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Signed-off-by: Ben Pfaff <blp@nicira.com>
backtrace() is really useful, but it is not signal safe everywhere. We
need to reassess whether it is reasonable to use it anywhere, but
immediately we need to disable it on x86-64 (with glibc) because it is
causing segfaults in testing.
Bug #15497.
Reported-by: Ram Jothikumar <rjothikumar@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
Durations longer than 4294967 seconds would unnecessarily overflow in the
multiplication here.
Found by Coverity.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
This is a straight search-and-replace, except that I also removed #include
<assert.h> from each file where there were no assert calls left.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
Occasionally, backtrace() will deadlock in the signal handler
because it does some non signal safe initialization. Specifically,
it opens a shared object. As a work around, this patch forces
backtrace() to run outside of a signal handler, so that future
calls will perform as expected.
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Often when debugging Open vSwitch, one will see in the logs that
CPU usage has been high for some period of time, but it's totally
unclear why. In an attempt to remedy the situation, this patch
logs backtraces taken at regular intervals as a poor man's
profiling alternative.
Signed-off-by: Ethan Jackson <ethan@nicira.com>
With this patch, `ovs-appctl backtrace` will return a unique list
of backtraces and a count of how many times it has been recorded.
This work had previously been done by ovs-parse-backtrace. However,
in future patches poll-loop will begin logging backtraces as a
matter of course. At this point, coalescing the backtraces will
help keep these log messages brief.
Signed-off-by: Ethan Jackson <ethan@nicira.com>
With this patch, timeval will take a backtrace with each SIGALRM
allowing it to retrieve a profiling snapshot instantly. This will
be useful in future patches when backtraces are logged.
Signed-off-by: Ethan Jackson <ethan@nicira.com>
log_poll_interval() is a little bit too aggressive, and is
therefore less useful than it could be. This patch removes the
mean interval calculation, and simply logs if the poll loop took
longer than 1 second instead.
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Commit 00a16895 (timeval: Don't require signals for time_alarm().)
Incorrectly disabled signals when when CACHE_TIME was disabled. In
fact, the reverse was correct. As a result of this bug, OVS would
wake once every 100ms unnecessarily. It shouldn't have affected
correctness otherwise.
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Often, it can be quite difficult to debug performance issues in
Open vSwitch. Typically one needs to run something like gprof, but
that requires rebuilding and installing on the affected system
which is often problematic. This patch adds a light weight
profiling solution which can be used in these situations. The
ovs-appctl backtrace command prints out backtraces taken at 100
millisecond intervals over a 5 second period of time. It is
currently only supported on systems which have the execinfo library
and enable time caching.
Signed-off-by: Ethan Jackson <ethan@nicira.com>
