It may be desirable to make use of time warp functionality in unit
tests.
Separate logic from time/stop unixctl into timeval_stop() and add
a new timeval_warp() interface for directing monotonic clock into
slow path and advancing the current monotonic directly.
This will be used in a patch that implements unit tests for the
cooperative multitasking module.
Signed-off-by: Frode Nordahl <frode.nordahl@canonical.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Martin Kennelly observes that even though this data is available to
humans via the journal/log files, these aren't exactly easy for a
developer to make any kind of behavioral inferences. This kind of
log and counter would be useful when checking on system health to
let us know that an Open vSwitch component is noticing some kind of
system level hiccup.
Add a new coverage counter to track information on these events, and
let a developer or system engineer know how long these events have
occurred with some historical context.
Reported-at: https://lists.linuxfoundation.org/pipermail/ovs-discuss/2023-June/052523.html
Suggested-by: Martin Kennelly <mkennell@redhat.com>
Co-Authored-By: Aaron Conole <aconole@redhat.com>
Signed-off-by: Eelco Chaudron <echaudro@redhat.com>
Acked-by: Simon Horman <horms@ovn.org>
Acked-by: Ilya Maximets <i.maximets@ovn.org>
Signed-off-by: Aaron Conole <aconole@redhat.com>
This commit introduces xnanosleep() for the threads needing high
resolution sleep timeouts.
usleep() that provides microsecond granularity is deprecated and threads
wanting sub-second(ms,us,ns) granularity can use this implementation.
Signed-off-by: Bhanuprakash Bodireddy <bhanuprakash.bodireddy@intel.com>
Acked-by: Alin Gabriel Serdean <aserdean@ovn.org>
Signed-off-by: Ben Pfaff <blp@ovn.org>
[Problem]
Compilation error on newer versions of macOS (Sierra onwards) due to
multiple declarations of clock_gettime.
[Solution]
Have configure check for clock_gettime and check this result in
timeval to avoid incorrectly declaring/defining clock_gettime again.
[Testing]
Source code now successfully builds on macOS.
Signed-off-by: Richard Oliver <richard@richard-oliver.co.uk>
Signed-off-by: Ben Pfaff <blp@ovn.org>
TIME_UPDATE_INTERVAL was removed long time ago.
Now each call leads to time update via syscall and it's
granularity is system dependent.
Fixes: 31ef9f5178de ("timeval: Remove CACHE_TIME scheme.")
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
By reordering the elements in clock structure, pad bytes
can be reduced and also a cache line is saved.
Before: structure size:136, holes:3, sum padbytes:18, cachelines:3
After: structure size:120, holes:1, sum padbytes:2, cachelines:2
Signed-off-by: Bhanuprakash Bodireddy <bhanuprakash.bodireddy@intel.com>
Co-authored-by: Antonio Fischetti <antonio.fischetti@intel.com>
Signed-off-by: Antonio Fischetti <antonio.fischetti@intel.com>
Acked-by: Jarno Rajahalme <jarno@ovn.org>
Acked-by: Daniele Di Proietto <diproiettod@vmware.com>
To easily allow both in- and out-of-tree building of the Python
wrapper for the OVS JSON parser (e.g. w/ pip), move json.h to
include/openvswitch. This also requires moving lib/{hmap,shash}.h.
Both hmap.h and shash.h were #include-ing "util.h" even though the
headers themselves did not use anything from there, but rather from
include/openvswitch/util.h. Fixing that required including util.h
in several C files mostly due to OVS_NOT_REACHED and things like
xmalloc.
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
OS X does not implement clock_gettime(), implement replacement.
Signed-off-by: Lance Richardson <lrichard@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
'Unreasonably long poll interval's are reasonable for PMD threads.
Also reporting of high CPU usage is not necessary.
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
init_clock begins with a memset of 0 of the full clock struct. This
memset at the end of a single struct member just makes extra sure that
it's set to 0, which is unnecessary.
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Most of the information that timeval was reporting for long poll intervals
was comparing per-thread with per-process statistics, which yielded
nonsense a lot of the time.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Alex Wang <alexw@nicira.com>
A new function vlog_insert_module() is introduced to avoid using
list_insert() from the vlog.h header.
Signed-off-by: Thomas Graf <tgraf@noironetworks.com>
Acked-by: Ben Pfaff <blp@nicira.com>
'timewarp_seq' should be initialized only once, while init_clock() is
called multiple times (once for each clock instance).
Found by valgrind
Signed-off-by: Daniele Di Proietto <ddiproietto@vmware.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
Till now, we were initializing 'unix_epoch' through time_init().
But if there was a call directly to xgettimeofday(), we would
miss the initialization causing overflows. This commit fixes it
by pre-calculating the value and assigning it globally.
Also add-in a missing return statement.
Reported-by: Alessandro Pilotti <apilotti@cloudbasesolutions.com>
Reported-by: Linda Sun <lsun@vmware.com>
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
The new command is of the form 'time/warp LARGE_MSECS MSECS'.
It advances the current monotonic time by LARGE_MSECS. This is done MSECS
at a time in each run of the main thread. This gives other threads
time to run after the clock has been advanced by MSECS.
The old command would continue to work.
Rationale: On Windows, process creation is slower. When we have tests
that run 'ovs-appctl time/warp MSECS' hundreds of times in a for loop,
the time it takes to complete the test increases. This is specially
true for bfd tests. For e.g, the 11 bfd tests would take 3.5 minutes
to complete before this change and now takes a little less than 2 minutes.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Otherwise ovsrcu_synchronize() busy-waits in its loop because its
poll_block() un-quiesces, causing the global_seqno to increase, which is
what it waits for.
Reported-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Alex Wang <alexw@nicira.com>
RCU allows multiple threads to read objects in parallel without any
performance penalty. The following commit will introduce the first use.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Andy Zhou <azhou@nicira.com>
The WSAPoll() function, which is similar to poll() doesnot
simply sleep when the fd array is NULL. So use Sleep() instead.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Use GetSystemTimePreciseAsFileTime() for gettimeofday().
GetSystemTimePreciseAsFileTime() provides the result that is more
high resolution than just the microsecond that gittimeofday() in
Linux provides. So we need to remove some additional precision.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
QueryPerformanceCounter() retrieves the current value of the performance
counter, which is a high resolution (<1us) time stamp that can be used for
time-interval measurements. So, use it for MONOTONIC clock.
The GetSystemTimePreciseAsFileTime() function retrieves the current system date
and time with the highest possible level of precision (<1us). Use it for
real time clock. This function returns a counter representing the number of
100-nanosecond intervals since January 1, 1601. To make it compatible with
Linux CLOCK_REALTIME, we need to calculate the 100-nanoseconds counter value
till 01/01/1970.
An upcoming commit implements gettimeofday() using the same clock, so,
carve out a function.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Windows does not have a SIGHUP or SIGALRM. It does have
a SIGINT and SIGTERM. The documentation at msdn says that
SIGINT is not supported for win32 applications because
WIN32 operating systems generate a new thread to specifically
handle Ctrl+C.
This commit handles SIGTERM for Windows. The documentation also
states that nothing generates SIGTERM in Windows, but one can
use raise(SIGTERM) to manage it. The idea for handling SIGTERM
for Windows is to just have a place holder if there is need to
raise() a signal for some other purpose.
We use SIGALRM in timeval.c if we wake up from a sleep after
'deadline'. For Windows, print an error message and then
use SIGTERM.
There is an atexit() function for Windows, so we can call cleanup
functions during exit.
An upcoming commit separately handles Ctrl+C so that we can call
clean up functions for that use case.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Use WaitForMultipleObjects for polling on windows. This works on all kinds
of objects, e.g. sockets, files, especially ioctl calls to the kernel.
poll_fd_wait_event() is used if events need to be passed to pollfds. latch
is signaled with event, to be waited/polled by WaitForMultipleObjects() as
well. Changed array of fds to hmap to check for duplicate fds.
Signed-off-by: Linda Sun <lsun@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
BFD tests have the code like the following.
# wait for a while to stablize everything.
for i in `seq 0 9`; do ovs-appctl time/warp 500; done
They no longer work as intended because BFD code is run in a
separate monitor thread these days. The loop merely "warp"
the time by 5000. The monitor thread should have been woken
at least once, but it's far from "wait for a while to stablize
everything."
This commit mitigates the problem by sleeping a little in the
appctl handler. This is not ideal but makes BFD tests success
on my environment.
Signed-off-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
Signed-off-by: Ben Pfaff <blp@nicira.com>
time_poll() calls log_poll_interval(), which in some circumstances calls
coverage_log(). Before this commit, time_poll() also called
coverage_clear() after log_poll_interval(). This made sense before commit
857165b5fd26 (coverage: Make thread-safe.), because coverage_log() would
log the most recent main loop's coverage counters separately and calling
coverage_clear() beforehand would zero out those counters. However, it
doesn't make sense any longer because the most recent loop's counters are
no longer separately logged and in fact this practice now means that the
most recent loop's counters are omitted from the logged counters.
Therefore, this commit moves the call to coverage_clear() earlier, so that
the most recent loop's counters are included.
Signed-off-by: Ben Pfaff <blp@nicira.com>
This commit makes the main thread wake up all other threads when time is
warped.
Signed-off-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
This commit changes the "ovs-appctl coverage/show" command to show the
the averaged per-second rates for the last few seconds, the last minute
and the last hour, and the total counts of all of the coverage counters.
Signed-off-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
It's only held briefly now and in general a mutex tends to be preferred for
that case.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
Commit 31ef9f5178dee18 (timeval: Remove CACHE_TIME scheme.) inadvertently
removed the ability to warp time forward, for use in tests, except when
time is stopped. This fixes the problem.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
Commit 31ef9f5178 (timeval: Remove CACHE_TIME scheme.) removed
initialization of a rwlock which is still used for some operations.
This restores it.
Signed-off-by: Ben Pfaff <blp@nicira.com>
This commit removes the CACHE_TIME scheme from timeval module. This
is for eliminating the lock contention over the read/write lock of
the cached time. To get the time, the thread now will directly do
the system call 'clock_gettime()'.
As a side effect, timer can only be warpped after timer is stopped
by 'appctl time/stop' command.
Signed-off-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
DEFINE_PER_THREAD_DATA always declared its data item as "static", meaning
that it was only directly visible within a single translation unit.
This commit adds additional forms of per-thread data that allow the data
to be accessible from multiple translation units.
Signed-off-by: Ben Pfaff <blp@nicira.com>
This commit adds annotations for thread safety check. And the
check can be conducted by using -Wthread-safety flag in clang.
Co-authored-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
This should make the "timeval" module thread-safe, except for its
calls into vlog (because vlog is not yet thread-safe).
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ed Maste <emaste@freebsd.org>
The backtrace feature of timeval is useful because it provides a "poor
man's profile" view of Open vSwitch. But it is not likely to be useful in
a multithreaded process, because signal delivery doesn't necessarily follow
the profile when there is more than one thread. (A signal in a
multithreaded process are delivered to an arbitrary thread.)
Another problem with the backtrace feature is that it is difficult for
format_backtraces() to synchronize properly with the signal handler in a
multithreaded process. In a single-threaded process, it can just block
the signal handler, but in a multithreaded process this does not prevent
signal delivery to threads other than the one running format_backtrace().
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ed Maste <emaste@freebsd.org>
These functions will not have the same useful effect when Open vSwitch
becomes multithreaded, because time_disable_restart() will disable time
advancing for every thread, not just for the thread that calls it.
These functions are no longer used, so this commit removes them.
Signed-off-by: Ben Pfaff <blp@nicira.com>
There is no need to do so because the signal handler does not read or
write 'deadline'.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
POSIX says that multithreaded programs must not use sigprocmask() but must
use pthread_sigmask() instead. This commit makes that replacement.
The actual use of signals in Open vSwitch is still not thread safe
following this commit, but this change is a necessary prerequisite for
fixing the other problems.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Other code in the tree uses HAVE_BACKTRACE and then blindly includes
<execinfo.h> if it is present, so this doesn't make anything worse.
Once we do that, HAVE_EXECINFO_H has no further users, so this commit also
removes the check for <execinfo.h>
Reported-by: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Signed-off-by: Ben Pfaff <blp@nicira.com>
