From: Jakub Sitnicki <jkbs@redhat.com>
Facilitate checking coverage counters from scripts and tests with a new
coverage/read-counter command that gets the total count for a counter.
Same could be achieved by scraping the output of coverage/show command
but the difficulties there are that output is in human readable format
and zero-value counters are not listed.
Signed-off-by: Jakub Sitnicki <jkbs@redhat.com>
Acked-by: Han Zhou <hzhou8@ebay.com>
Signed-off-by: Han Zhou <hzhou8@ebay.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
For performance-critical threads like pmd threads, we currently make them
never call coverage_clear() to avoid contention over the global mutex
'coverage_mutex'. So, even though pmd thread still keeps updating their
thread-local coverage count, the count is never attributed to the global
total. But it is useful to have them available.
This commit makes this happen by implementing a non-contending version
of the clear function, coverage_try_clear(). The function will use
the ovs_mutex_trylock() and return immediately if the mutex cannot
be acquired. Since threads like pmd thread are always busy-looping,
the lock will eventually be acquired.
Requested-by: Ilya Maximets <i.maximets@samsung.com>
Signed-off-by: Alex Wang <alexw@nicira.com>
Acked-by: Ilya Maximets <i.maximets@samsung.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Acked-by: Daniele Di Proietto <diproiettod@vmware.com
A new function vlog_insert_module() is introduced to avoid using
list_insert() from the vlog.h header.
Signed-off-by: Thomas Graf <tgraf@noironetworks.com>
Acked-by: Ben Pfaff <blp@nicira.com>
This removes a GCC 4.9 warning (unused-but-set-variable)
Signed-off-by: Daniele Di Proietto <ddiproietto@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
These variables were initialized in an outer scope and then immediately
changed in an inner one, so they might as well be farther in.
Found by clang-analyzer.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
Fixes the following warning on my system.
../lib/coverage.c:54:40: error: incompatible pointer types passing
'unsigned int *' to parameter of type 'size_t *' (aka 'unsigned long
*')
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Use a static array in coverage.c that gets initialized by constructor
functions per coverage definition.
Signed-off-by: Helmut Schaa <helmut.schaa@googlemail.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
With this change, OVS builds and runs fine without linker section support.
CC: Linda Sun <lsun@vmware.com>
CC: Saurabh Shah <ssaurabh@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
The MSVC C library printf() implementation does not support the 'z', 't',
'j', or 'hh' format specifiers. This commit changes the Open vSwitch code
to avoid those format specifiers, switching to standard macros from
<inttypes.h> where available and inventing new macros resembling them
where necessary. It also updates CodingStyle to specify the macros' use
and adds a Makefile rule to report violations.
Signed-off-by: Alin Serdean <aserdean@cloudbasesolutions.com>
Co-authored-by: Ben Pfaff <blp@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
When profiling CPU usage in situations involving high numbers of ports,
coverage_clear() was highlighted as a commonly called function. It
appears that it can be quite expensive to access all of the per-thread
coverage counters when threads are constantly waking up.
This patch makes each thread only do coverage_clear() logic roughly once
per second by introducing per-thread timers. Upcall handler counters may
become less accurate, as these threads may sleep without synchronising
and not wake up for some time. When the main thread is under load at
~90% CPU, this drops to ~85%. Upcall handler threads sitting at ~2.5%
drop to ~1.5%.
Signed-off-by: Joe Stringer <joestringer@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
This commit changes the "ovs-appctl coverage/show" command to show the
the averaged per-second rates for the last few seconds, the last minute
and the last hour, and the total counts of all of the coverage counters.
Signed-off-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
sparse warns if a non-static variable with external linkage has an
initializer at first declaration. This commit suppresses the
warnings issued when adding custom section is not supported by
compiler.
Signed-off-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
This makes each of the coverage counters per-thread. It abandons the
idea of trying to keep track of the number of hits in the "current" poll
loop, since there are many poll loops running, each in its own thread, as
well as the idea of numbering epochs for the same reason. Instead, we
just keep track of overall totals for the process for each coverage
counter, accumulating per-thread counts into the global total each time
a thread's main loop passes through poll_block().
Signed-off-by: Ben Pfaff <blp@nicira.com>
Replaced all instances of Nicira Networks(, Inc) to Nicira, Inc.
Feature #10593
Signed-off-by: Raju Subramanian <rsubramanian@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
I've had a few complaints that ovs-vswitchd logs its coverage counters
at WARN level, but this is mainly wrong: ovs-vswitchd only logs coverage
counters at WARN level when the "coverage/log" command is used through
ovs-appctl. This was even documented.
The reason to log at such a high level was to make it fairly certain that
these messages specifically requested by the admin would not be filtered
out before making it to the log. But it's even better if the admin just
gets the coverage counters as a reply to the ovs-appctl command. So that
is what this commit does.
This commit also improves the documentation of the ovs-appctl command.
Signed-off-by: Ben Pfaff <blp@nicira.com>
I'd always assumed that the exponentially weighted moving average code in
timeval was enough rate-limiting, but I actually encountered a pathological
case some time ago that forced this coverage information to print once a
second or so, which seems too often.
Signed-off-by: Ben Pfaff <blp@nicira.com>
The unixctl library had used the vde2 management protocol since the
early days of Open vSwitch. As Open vSwitch has matured, several
Python daemons have been added to the code base which would benefit
from a unixctl implementations. Instead of implementing the old
unixctl protocol in Python, this patch changes unixctl to use JSON
RPC for which we already have an implementation in both Python and
C. Future patches will need to implement a unixctl library in
Python on top of JSON RPC.
Signed-off-by: Ethan Jackson <ethan@nicira.com>
Otherwise coverage log messages will eventually get rotated out of the
system logs and never reappear (before restarting a given daemon).
Bug #1452.
Signed-off-by: Ben Pfaff <blp@nicira.com>
The protocol used by ovs-appctl has a long-standing bug that there
is no way to distinguish "ovs-appctl a b c" from "ovs-appctl 'a b c'".
This isn't a big deal because none of the current commands really
want to accept arguments that include spaces, but it's kind of a silly
limitation.
At the same time, the internal API is awkward because every user is
stuck doing its own argument parsing, which is no fun.
This commit fixes both problems, by adding shell-like quoting to the
protocol and modifying the internal API from one that passes a string
to one that passes in an array of pre-parsed strings. Command
implementations may now specify how many arguments they expect. This
simplifies some command implementations significantly.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Until now, the collection of coverage counters supported by a given OVS
program was not specific to that program. That means that, for example,
even though ovs-dpctl does not have anything to do with mac_learning, it
still has a coverage counter for it. This is confusing, at best.
This commit fixes the problem on some systems, in particular on ones that
use GCC and the GNU linker. It uses the feature of the GNU linker
described in its manual as:
If an orphaned section's name is representable as a C identifier then
the linker will automatically see PROVIDE two symbols: __start_SECNAME
and __end_SECNAME, where SECNAME is the name of the section. These
indicate the start address and end address of the orphaned section
respectively.
Systems that don't support these features retain the earlier behavior.
This commit also fixes the annoyance that files that include coverage
counters must be listed on COVERAGE_FILES in lib/automake.mk.
This commit also fixes the annoyance that modifying any source file that
includes a coverage counter caused all programs that link against
libopenvswitch.a to relink, even programs that the source file was not
linked into. For example, modifying ofproto/ofproto.c (which includes
coverage counters) caused tests/test-aes128 to relink, even though
test-aes128 does not link again ofproto.o.
Adding a macro to define the vlog module in use adds a level of
indirection, which makes it easier to change how the vlog module must be
defined. A followup commit needs to do that, so getting these widespread
changes out of the way first should make that commit easier to review.
By default, many OVS processes keep track of their time through a poll
loop. If it takes an unusually long time (measured as some distance
from the mean), the processes will log stats it has been keeping about
coverage. It was doing this at level WARN.
On Xen systems, syslog messages written at level INFO and higher are
written to /var/log/messages synchronously. This would mean that there
would be dire messages that it took a few dozen milliseconds to go
through the loop, meanwhile, it would take up to 6(!) seconds writing
those. Meanwhile, the process would do no other processing, which could
be quite serious in the case of a process such as ovs-vswitchd.
This problem was somewhat masked because the time used by this logging
was not used in the calculations for determining how long it was taking
to get through the loop.
This commit lowers the default log level for those coverage messages to
INFO. On Xen systems, it raises the default level at which messages are
written to syslog to WARN.
Diagnosed and fixed with the help of Ian Campbell.
When providing the ability to force coverage printouts to occur, some
code was moved around that allowed the "hash" variable to be used
unitialized. This fixes that.
Thanks to Ben for pointing out the problem.
Bug #1577
Previously, there was no way to induce coverage information to be
displayed; it would only print when the system noticed unusual delays
between polling intervals. Now, production of coverage logs can be
forced with "coverage/log" command in ovs-appctl. Coverage counters may
be reset with "coverage/clear".
