Consider this situation:
* OVSDB client A executes transactions very quickly for a long time.
* OVSDB client B monitors the tables that A modifies, but (either
because B is connected over a slow network, or because B is slow to
process updates) cannot keep up.
In this situation, the data that ovsdb-server has queued to send B grows
without bound and eventually ovsdb-server runs out of memory. This commit
avoids the problem by noticing that more data is queued to B than necessary
to express the whole contents of the database and dropping the connection
to B. When B reconnects later, it can then fetch the contents of the
database using less data than was previously queued to it.
(This is not entirely hypothetical. We have seen this behavior in
intentional stress tests.)
Bug #15637.
Reported-by: Jeff Merrick <jmerrick@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Until now, ovs-vsctl has kept trying to the database server until it
succeeded or the timeout expired (if one was specified with --timeout).
This meant that if ovsdb-server wasn't running, then ovs-vsctl would hang.
The result was that almost every ovs-vsctl invocation in scripts specified
a timeout on the off-chance that the database server might not be running.
But it's difficult to choose a good timeout. A timeout that is too short
can cause spurious failures. A timeout that is too long causes long delays
if the server really isn't running.
This commit should alleviate this problem. It changes ovs-vsctl's behavior
so that, if it fails to connect to the server, it exits unsuccessfully.
This makes --timeout obsolete for the purpose of avoiding a hang if the
database server isn't running. (--timeout is still useful to avoid a hang
if ovsdb-server is running but ovs-vswitchd is not, for ovs-vsctl commands
that modify the database. --no-wait also avoids that issue.)
Bug #2393.
Bug #15594.
Reported-by: Jeff Merrick <jmerrick@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
ovsdb_session_destroy() was called twice but it should only be called once.
This double-free is unlikely to cause problems in practice because it only
triggers if there were ever more than two outstanding requests in the
session at a time (because the only data being freed is an hmap, which
does not allocate any heap memory unless the hmap has more than two
elements).
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
This is a straight search-and-replace, except that I also removed #include
<assert.h> from each file where there were no assert calls left.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
2012-09-14T05:38:26Z|00001|jsonrpc|WARN|tcp:127.0.0.1:6634: receive error: Con
ovsdb-client: transaction failed (Connection reset by peer)
NOTE: This occurs intermittently depending on how ovsdb-server runs.
Running ovsdb-client on a remote machine increases the possibility.
This is because ovsdb-server closes newly accepted tcp connection.
The following changesets caused it. struct jsonrpc_session::dscp isn't set
based on listening socket's dscp value.
- ovsdb-server creates passive connection and listens on it.
- ovsdb-server accepts connection by ovsdb_jsonrpc_server_run().
The accepted socket inherits from the listening sockets.
ovsdb_jsonrpc_server_run() creates json session, but leaves dscp of
struct jsonrpc_session zero.
- On calling reconfigure_from_db(), it resets dscp value to
all jsonrpc sessions. Eventually jsonrpc_session_set_dscp() is called.
Then jsonrpc_session_force_reconnect() closes the connection.
With this patch,
- struct jsonrpc_session::dscp is correctly set based on
listening sockets dscp value.
- dscp of listening socket is changed dynamically by setsockopt.
This leaves a window where accepted socket may have old dscp.
But it is ignored for now because it would complicates codes
too much.
The related change sets:
- 0442efd9b1a88d923b56eab6b72b6be8231a49f7
Reapplying the dscp changes: No need to restart DB/OVS on changing
dscp value.
- 59efa47adf3234ec51541405726d033173851285
Revert DSCP update changes.
- b2e18db292cd4962af3248f11e9f17e6eaf9c033
No need to restart DB / OVS on changing dscp value.
- f125905cdd3dc0339ad968c0a70128807884b400
Allow configuring DSCP on controller and manager connections.
Signed-off-by: Isaku Yamahata <yamahata@valinux.co.jp>
Signed-off-by: Mehak Mahajan <mmahajan@nicira.com>
The OVSDB protocol has supported multiple databases for a long time, but
the ovsdb-server implementation only supported one database at a time.
This commit adds support for multiple databases.
Feature #12353.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
Replaced all instances of Nicira Networks(, Inc) to Nicira, Inc.
Feature #10593
Signed-off-by: Raju Subramanian <rsubramanian@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
There isn't a lot of value in sending inactivity probes on unix
sockets. This patch changes the default to disable them.
Signed-off-by: Ethan Jackson <ethan@nicira.com>
valgrind's memory leak detector considers a pointer to the head of a memory
block to be "definitely" a pointer to that memory block but a pointer to
the interior of a memory block only "possibly" a pointer to that memory
block. Open vSwitch hmap_node and list data structures can go anywhere
inside a structure; if they are in the middle of a structure then valgrind
considers pointers to them to be possible leaks. Therefore, this commit
moves some of these from the middle of data structures to the head, to
reduce valgrind's uncertainty.
Signed-off-by: Ben Pfaff <blp@nicira.com>
The changes allow the user to specify a separate dscp value for the
controller connection and the manager connection. The value will take
effect on resetting the connections. If no value is specified a default
value of 192 is chosen for each of the connections.
Feature #10074
Requested-by: Rajiv Ramanathan <rramanathan@nicira.com>
Signed-off-by: Mehak Mahajan <mmahajan@nicira.com>
This provides clients a way to coordinate their access to the database.
This is a voluntary, not mandatory, locking protocols, that is, clients
are not prevented from modifying the database unless they cooperate with
the locking protocol. It is also not related to any of the ACID properties
of database transactions. It is strictly a way for clients to coordinate
among themselves.
The following commit will introduce one user.
An upcoming commit will need to expose the concept of a database session
to the execution engine, to allow the execution engine to query the locks
held by the session. This commit prepares for that by making sessions a
publicly visible data structure.
Inbound managers (e.g. "ptcp:") can have multiple active connections, but
the database schema doesn't allow us to report the status of more than one
at a time. This commit adds a status key-value pair that, when there is
more than one active connection, reports the number that are active. This
at least helps to clarify the issue.
ovsdb_jsonrpc_server keeps track of its remotes in a shash indexed on the
remote name specified in the database Manager record, but
ovsdb_jsonrpc_server_get_remote_status() added the name returned by
jsonrpc_session_get_name() to the shash returned to the ovsdb-server code.
If that name happened to be different (which is entirely possible because
the latter returns a "canonicalized" name in some cases) then the
ovsdb-server code couldn't find it. Furthermore, if an inbound (e.g.
"ptcp:") Manager got a connection and then lost it, the status info in
that Manager never got updated to reflect that, because the code considered
that that "couldn't happen" and didn't bother to do any updates.
This commit simplifies the logic. Now ovsdb-server just asks for a single
status record at a time, using the name that is indexed in the
ovsdb_jsonrpc_server shash, avoiding that whole issue.
Recently I helped debug a scenario where ovsdb-server connected to a remote
manager, then ovs-vsctl deleted the remote manager and soon after re-added
it. The log was difficult to interpret because it showed two successful
connection attempts to the same remote without showing a reason why the
connection was dropped in the first place. Adding this log message would
make it clear that the configuration changed to remove that remote
connection in the meantime.
Only the time connected (if connected) or disconnected (if disconnected) is
currently reported for each manager. Change to reporting both in seconds since
the last connect and disconnect events respectively. An empty value indicates
no previous connection or disconnection.
This can help diagnose certain connectivity problems, e.g. flapping.
Requested-by: Peter Balland <peter@nicira.com>
Bug #4833.
Commit 0b3e7a8b71 (ovsdb-server: Write manager status information to Manager
table.) attempted to provide managers with the ability to debug manager-related
connection problems, but it turns out that reporting "time_in_state" is not
very useful, because the state is constantly changing. What people really want
is the time each manager has been connected or disconnected, depending on the
current connection state.
Replace "time_in_state" key with "time_connected" and "time_disconnected"
keys. Only one exists at a time, and time is in seconds.
Bug #4833.
This commit makes the status of manager connections visible via the Manager
table in the database. Two new columns have been created for this purpose:
'is_connected' and 'status'. The former is a boolean flag, and the latter is a
string-string map which may contain the keys "last_error", "state", and
"time_in_state".
Requested-by: Keith Amidon <keith@nicira.com>
Reviewed by: Ben Pfaff.
Feature #3692.
The OVSDB wire protocol has a "monitor" feature that supports table
replication on a column-by-column basis. This wire protocol in theory
supports replicating a table without replicating any columns in it. In
such a case, the client only tracks the UUIDs of the rows in the table.
However, the ovsdb-server implementation did not support this possibility
properly. This commit fixes the bug.
I'm retaining the "managers" column in the Open_vSwitch table for now, but
I hope that applications transition to using "manager_options" eventually
so that we could drop it.
CC: Andrew Lambeth <wal@nicira.com>
CC: Jeremy Stribling <strib@nicira.com>
Adding a macro to define the vlog module in use adds a level of
indirection, which makes it easier to change how the vlog module must be
defined. A followup commit needs to do that, so getting these widespread
changes out of the way first should make that commit easier to review.
Until now, "monitor" has only allowed the client to choose the kinds of
changes that will be monitored on a per-table basis. However, it makes
sense to be able to choose operations on a per-column basis. The
immediate need for this is to make sure that the final statistics of
deleted Interface records are known at time of deletion, even though the
intermediate values of the statistics are not important.
CC: Jeremy Stribling <strib@nicira.com>
Until now, each part of a transaction commit that is interested in whether
a column's value has changed has had to do a comparison of the old and new
values itself. There can be several interested parties per commit
(generally one for file storage and one for each remove OVSDB connection),
so this seems like too much redundancy. This commit adds a bitmap
to struct ovsdb_txn_row that tracks whether a column's value has actually
changed, to reduce this overhead.
As a convenient side effect of doing these checks up front, it then
becomes easily possible to drop txn_rows (and txn_tables and entire txns)
that become no-ops. (This probably fixes bug #2400, which reported that
some no-ops actually report updates over monitors.)
This also adds protocol compatibility to the database itself and to
ovsdb-client. It doesn't actually add multiple database support to
ovsdb-server, since we don't really need that yet.
Some of these are serious leaks, in that they could leak some amount of
memory for every transaction processed by the database server.
Found with valgrind.
Until this commit, ovsdb-server would send off echo requests when the
connection became idle, but then it would terminate the connection when
the reply arrived, because it didn't recognize that it was a reply to its
own request (!).
JSON-RPC requires that "params" be an array, but we weren't observing this
properly in the ovsdb specifications or code.
Thanks to Jeremy Stribling for pointing out the problem.
