Although multiple jsonrpc monitors can share the same ovsdb monitor,
each change still needs to translated into json object from scratch.
This can be wasteful if multiple jsonrpc monitors are interested in the
same changes.
Json cache improves this by keeping an copy of json object generated
for transaction X to current transaction. When jsonrpc is interested
in a change, the cache is searched first, if an json object is found,
a copy of it is handed back, skipping the regeneration process.
Any commit to the monitor will empty the cache. This can be further
optimized to not throw away the cache if the updated tables and columns
are not being monitored.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
monitor
Store ovsdb monitor in a global hmap. If a newly created ovsdb monitor
object monitors the same tables and columns as an existing one, the
existing monitor will be reused.
With this patch, jsonrpc monitor and ovsdb monitor now have N:1 mapping.
The goals are to:
1) Reduce the cost of maintaining duplicated monitors.
2) Allow for create Json cache for the same updates. Json cache will be
introduced in the following patch.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Currently, each monitor table contains a single hmap 'changes' to
track updates. This patch introduces a new data structure
'ovsdb_monitor_changes' that stores the updates 'rows' tagged by
its first commit transaction id. Each 'ovsdb_monitor_changes' is
refenece counted allowing multiple jsonrpc_monitors to share them.
The next patch will allow each ovsdb monitor table to store a list
of 'ovsdb_monitor_changes'. This patch stores only one, same as
before.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
jsonrpc_monitor_compose_update() seems to fit better than
jsonrpc_monitor_compose_table_update(), since it composes changes
from all tables. Albeit the original one is named after the
<table-updates> object described in RFC 7047.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
With N:1 mappings, multiple jsonrpc server may be servicing the rpc
connection at a different pace. ovsdb-monitor thus needs to maintain
different change sets, depends on connection speed of each rpc
connections. Connections servicing at the same speed can share the
same change set.
Transaction ID is an concept added to describe the change set. One
possible view of the database state is a sequence of changes, more
precisely, commits be applied to it in order, starting from an
initial state, with commit 0. The logic can also be applied to the
jsonrpc monitor; each change it pushes corresponds to commits between
two transaction IDs.
This patch introduces transaction IDs. For ovsdb-monitor, it maintains
n_transactions, starting from 0. Each commit add 1 to the number.
Jsonrpc maintains and 'unflushed' transaction number, corresponding to
the next commit the remote has not seen. jsonrpc's job is simply to
notice there are changes in the ovsdb-monitor that it is interested in,
i.e. 'n_transactions' >= 'unflushed', get the changes in json format,
and push them to the remote site.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Currently, each ovsdb-monitor points to a single jsonrpc_monitor object.
This means there is 1:1 relationship between them.
In case multiple jsonrpc-monitors need to monitor the same tables and
the columns within them, then can share a single ovsdb-monitor, so the
updates only needs to be maintained once.
This patch, with a few following patches, will allow for N:1 mapping
between jsonrpc-monitor and ovsdb-monitor.
Maintaining jsonrpc-monitor pointers in a linked-list is essential
in allowing N:1 mapping. The ovsdb-monitor life cycle
is now reference counted. An empty list means zero references.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Refactoring ovsdb_monitor_get_initial() to not generate JSON object.
It only collect changes within the ovsdb_monitor().
ovsdb_jsonrpc_monitor_compose_table_update() is then used to generate
JSON object.
This change will also make future patch easier.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Added new files monitor.[ch] for monitor backend functions.
There is no functional changes.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Add ovsdb_monitor_destroy() function to properly cleanup ovsdb_monitor.
It is also responsible for unhook from the replica chain.
The replica destroy callback is now called
ovsdb_monitor_destroy_callback()
Minor variable renaming in ovsdb_monitor_create() to make it
more consistent.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
rename ovsdb_jsonrpc_monitor_get_initial() to
ovsdb_monitor_get_initial()
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
split out per monitoring needs_flush() into
ovsdb_monitor_needs_flush().
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Now it simply calls ovsdb_monitor_compose_table_update(), which
actually creates the json object.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Change ovsdb_jsonrpc_parse_monitor_request() to make
ovsdb_monitor_table an opaque object.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
end
jsonrpc-server.c has two main functions. One deals with handling the
jsonrpc connections, the other deals with monitoring the database.
Currently, each jsonrpc connections has its own set of DB monitors.
This can be wasteful if a number of connections shares the same
monitors.
This patch, and a few following refactoring patches attempts to
split the jsonrpc handling front end off the main monitoring
functions within jsonrpc.c.
This patch changes the monitoring functions and data structures from
'ovsdb_jsonrpc_monitor_xxx' into 'ovsdb_monitor_xxx'
This and the following patches move the ovsdb_monitor backend functions
into their own file.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
I realized that this bug existed only a few weeks ago, but it has been
there since the earliest ovsdb-server code.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Alex Wang <alexw@nicira.com>
Raise the connection limit to allow larger number of concurrent
ovsdb-server connections. Note, ovsdb-server may not perform well
at the new limit. It is rather a prelude to further scaling tests and
optimizations.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Without the log message, it was not obvious why ovsdb-server no longer
accepts new connections when the session limit was reached. This patch
adds a log message to make it obvious.
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Russell Bryant <rbryant@redhat.com>
Acked-by: Ben Pfaff <blp@nicira.com>
This function is really of marginal utility. This commit drops it and
makes the existing callers instead open a new pstream with the desired
dscp.
The ulterior motive here is that the set_dscp() function that actually sets
the DSCP on a socket really wants to know the address family (AF_INET vs.
AF_INET6). We could plumb that down through the stream code, and that's
one reasonable option, but I thought that simply eliminating some calls
to set_dscp() where we don't already have the address family handy was
another reasonable way to go.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Alex Wang <alexw@nicira.com>
A new function vlog_insert_module() is introduced to avoid using
list_insert() from the vlog.h header.
Signed-off-by: Thomas Graf <tgraf@noironetworks.com>
Acked-by: Ben Pfaff <blp@nicira.com>
struct list is a common name and can't be used in public headers.
Signed-off-by: Thomas Graf <tgraf@noironetworks.com>
Acked-by: Ben Pfaff <blp@nicira.com>
The following macros are renamed to avoid conflicts with other headers:
* WARN_UNUSED_RESULT to OVS_WARN_UNUSED_RESULT
* PRINTF_FORMAT to OVS_PRINTF_FORMAT
* NO_RETURN to OVS_NO_RETURN
Signed-off-by: Thomas Graf <tgraf@noironetworks.com>
Acked-by: Ben Pfaff <blp@nicira.com>
Also, add some clarifications relative to RFC 7047 to ovsdb-server(1).
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Justin Pettit <jpettit@nicira.com>
Connections that queue up too much data, because they are monitoring a
table that is changing quickly and failing to keep up with the updates,
cause problems with buffer management. Since commit 60533a405b
(jsonrpc-server: Disconnect connections that queue too much data.),
ovsdb-server has dealt with them by disconnecting the connection and
letting them start up again with a fresh copy of the database. However,
this is not ideal because of situations where disconnection happens
repeatedly. For example:
- A manager toggles a column back and forth between two or more values
quickly (in which case the data transmitted over the monitoring
connections always increases quickly, without bound).
- A manager repeatedly extends the contents of some column in some row
(in which case the data transmitted over the monitoring connection
grows with O(n**2) in the length of the string).
A better way to deal with this problem is to combine updates when they are
sent to the monitoring connection, if that connection is not keeping up.
In both the above cases, this reduces the data that must be sent to a
manageable amount. This commit implements this new way.
Bug #1211786.
Bug #1221378.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Andy Zhou <azhou@nicira.com>
This reverts commit 60533a405b.
Connections that queue up too much data, because they are monitoring a
table that is changing quickly and failing to keep up with the updates,
cause problems with buffer management. Since commit 60533a405b
(jsonrpc-server: Disconnect connections that queue too much data.),
ovsdb-server has dealt with them by disconnecting the connection and
letting them start up again with a fresh copy of the database. However,
this is not ideal because of situations where disconnection happens
repeatedly. For example:
- A manager toggles a column back and forth between two or more values
quickly (in which case the data transmitted over the monitoring
connections always increases quickly, without bound).
- A manager repeatedly extends the contents of some column in some row
(in which case the data transmitted over the monitoring connection
grows with O(n**2) in the length of the string).
A better way to deal with this problem is to combine updates when they are
sent to the monitoring connection, if that connection is not keeping up.
In both the above cases, this reduces the data that must be sent to a
manageable amount. An upcoming patch implements this new way. This commit
reverts part of the previous solution that disconnects backlogged
connections, since it is no longer useful.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Andy Zhou <azhou@nicira.com>
The MSVC C library printf() implementation does not support the 'z', 't',
'j', or 'hh' format specifiers. This commit changes the Open vSwitch code
to avoid those format specifiers, switching to standard macros from
<inttypes.h> where available and inventing new macros resembling them
where necessary. It also updates CodingStyle to specify the macros' use
and adds a Makefile rule to report violations.
Signed-off-by: Alin Serdean <aserdean@cloudbasesolutions.com>
Co-authored-by: Ben Pfaff <blp@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
The commit allows a user to add a database file to a
ovsdb-server during run time. One can also remove a
database file from ovsdb-server's control.
Feature #14595.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Co-authored-by: Ben Pfaff <blp@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
The administrator can request that OVSDB bind any available TCP port, but
in that case there is no easy way to find out what port it has bound. This
commit adds that information as the "bound_port" key in the "status"
column.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Consider this situation:
* OVSDB client A executes transactions very quickly for a long time.
* OVSDB client B monitors the tables that A modifies, but (either
because B is connected over a slow network, or because B is slow to
process updates) cannot keep up.
In this situation, the data that ovsdb-server has queued to send B grows
without bound and eventually ovsdb-server runs out of memory. This commit
avoids the problem by noticing that more data is queued to B than necessary
to express the whole contents of the database and dropping the connection
to B. When B reconnects later, it can then fetch the contents of the
database using less data than was previously queued to it.
(This is not entirely hypothetical. We have seen this behavior in
intentional stress tests.)
Bug #15637.
Reported-by: Jeff Merrick <jmerrick@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Until now, ovs-vsctl has kept trying to the database server until it
succeeded or the timeout expired (if one was specified with --timeout).
This meant that if ovsdb-server wasn't running, then ovs-vsctl would hang.
The result was that almost every ovs-vsctl invocation in scripts specified
a timeout on the off-chance that the database server might not be running.
But it's difficult to choose a good timeout. A timeout that is too short
can cause spurious failures. A timeout that is too long causes long delays
if the server really isn't running.
This commit should alleviate this problem. It changes ovs-vsctl's behavior
so that, if it fails to connect to the server, it exits unsuccessfully.
This makes --timeout obsolete for the purpose of avoiding a hang if the
database server isn't running. (--timeout is still useful to avoid a hang
if ovsdb-server is running but ovs-vswitchd is not, for ovs-vsctl commands
that modify the database. --no-wait also avoids that issue.)
Bug #2393.
Bug #15594.
Reported-by: Jeff Merrick <jmerrick@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
ovsdb_session_destroy() was called twice but it should only be called once.
This double-free is unlikely to cause problems in practice because it only
triggers if there were ever more than two outstanding requests in the
session at a time (because the only data being freed is an hmap, which
does not allocate any heap memory unless the hmap has more than two
elements).
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
This is a straight search-and-replace, except that I also removed #include
<assert.h> from each file where there were no assert calls left.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
2012-09-14T05:38:26Z|00001|jsonrpc|WARN|tcp:127.0.0.1:6634: receive error: Con
ovsdb-client: transaction failed (Connection reset by peer)
NOTE: This occurs intermittently depending on how ovsdb-server runs.
Running ovsdb-client on a remote machine increases the possibility.
This is because ovsdb-server closes newly accepted tcp connection.
The following changesets caused it. struct jsonrpc_session::dscp isn't set
based on listening socket's dscp value.
- ovsdb-server creates passive connection and listens on it.
- ovsdb-server accepts connection by ovsdb_jsonrpc_server_run().
The accepted socket inherits from the listening sockets.
ovsdb_jsonrpc_server_run() creates json session, but leaves dscp of
struct jsonrpc_session zero.
- On calling reconfigure_from_db(), it resets dscp value to
all jsonrpc sessions. Eventually jsonrpc_session_set_dscp() is called.
Then jsonrpc_session_force_reconnect() closes the connection.
With this patch,
- struct jsonrpc_session::dscp is correctly set based on
listening sockets dscp value.
- dscp of listening socket is changed dynamically by setsockopt.
This leaves a window where accepted socket may have old dscp.
But it is ignored for now because it would complicates codes
too much.
The related change sets:
- 0442efd9b1
Reapplying the dscp changes: No need to restart DB/OVS on changing
dscp value.
- 59efa47adf
Revert DSCP update changes.
- b2e18db292
No need to restart DB / OVS on changing dscp value.
- f125905cdd
Allow configuring DSCP on controller and manager connections.
Signed-off-by: Isaku Yamahata <yamahata@valinux.co.jp>
Signed-off-by: Mehak Mahajan <mmahajan@nicira.com>
The OVSDB protocol has supported multiple databases for a long time, but
the ovsdb-server implementation only supported one database at a time.
This commit adds support for multiple databases.
Feature #12353.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
Replaced all instances of Nicira Networks(, Inc) to Nicira, Inc.
Feature #10593
Signed-off-by: Raju Subramanian <rsubramanian@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
There isn't a lot of value in sending inactivity probes on unix
sockets. This patch changes the default to disable them.
Signed-off-by: Ethan Jackson <ethan@nicira.com>
valgrind's memory leak detector considers a pointer to the head of a memory
block to be "definitely" a pointer to that memory block but a pointer to
the interior of a memory block only "possibly" a pointer to that memory
block. Open vSwitch hmap_node and list data structures can go anywhere
inside a structure; if they are in the middle of a structure then valgrind
considers pointers to them to be possible leaks. Therefore, this commit
moves some of these from the middle of data structures to the head, to
reduce valgrind's uncertainty.
Signed-off-by: Ben Pfaff <blp@nicira.com>
The changes allow the user to specify a separate dscp value for the
controller connection and the manager connection. The value will take
effect on resetting the connections. If no value is specified a default
value of 192 is chosen for each of the connections.
Feature #10074
Requested-by: Rajiv Ramanathan <rramanathan@nicira.com>
Signed-off-by: Mehak Mahajan <mmahajan@nicira.com>
This provides clients a way to coordinate their access to the database.
This is a voluntary, not mandatory, locking protocols, that is, clients
are not prevented from modifying the database unless they cooperate with
the locking protocol. It is also not related to any of the ACID properties
of database transactions. It is strictly a way for clients to coordinate
among themselves.
The following commit will introduce one user.
An upcoming commit will need to expose the concept of a database session
to the execution engine, to allow the execution engine to query the locks
held by the session. This commit prepares for that by making sessions a
publicly visible data structure.
Inbound managers (e.g. "ptcp:") can have multiple active connections, but
the database schema doesn't allow us to report the status of more than one
at a time. This commit adds a status key-value pair that, when there is
more than one active connection, reports the number that are active. This
at least helps to clarify the issue.
ovsdb_jsonrpc_server keeps track of its remotes in a shash indexed on the
remote name specified in the database Manager record, but
ovsdb_jsonrpc_server_get_remote_status() added the name returned by
jsonrpc_session_get_name() to the shash returned to the ovsdb-server code.
If that name happened to be different (which is entirely possible because
the latter returns a "canonicalized" name in some cases) then the
ovsdb-server code couldn't find it. Furthermore, if an inbound (e.g.
"ptcp:") Manager got a connection and then lost it, the status info in
that Manager never got updated to reflect that, because the code considered
that that "couldn't happen" and didn't bother to do any updates.
This commit simplifies the logic. Now ovsdb-server just asks for a single
status record at a time, using the name that is indexed in the
ovsdb_jsonrpc_server shash, avoiding that whole issue.
Recently I helped debug a scenario where ovsdb-server connected to a remote
manager, then ovs-vsctl deleted the remote manager and soon after re-added
it. The log was difficult to interpret because it showed two successful
connection attempts to the same remote without showing a reason why the
connection was dropped in the first place. Adding this log message would
make it clear that the configuration changed to remove that remote
connection in the meantime.
