2
0
mirror of https://github.com/openvswitch/ovs synced 2025-08-29 13:27:59 +00:00

73 Commits

Author SHA1 Message Date
Han Zhou
8833e7c8ed ovsdb-idl: Provide API to disable set_db_change_aware request.
For ovsdb clients that are short-lived, e.g. when using
ovn-nbctl/ovn-sbctl to read some metrics from the OVN NB/SB server, they
don't really need to be aware of db changes, because they exit
immediately after getting the initial response for the requested data.
In such use cases, however, the clients still send 'set_db_change_aware'
request, which results in server side error logs when the server tries
to send out the response for the 'set_db_change_aware' request, because
at the moment the client that is supposed to receive the request has
already closed the connection and exited. E.g.:

2023-01-10T18:23:29.431Z|00007|jsonrpc|WARN|unix#3: receive error: Connection reset by peer
2023-01-10T18:23:29.431Z|00008|reconnect|WARN|unix#3: connection dropped (Connection reset by peer)

To avoid such problems, this patch provides an API to allow a client to
choose to not send the 'set_db_change_aware' request.

There was an earlier attempt to fix this [0], but it was not accepted
back then as discussed in the email [1]. It was also discussed in the
emails that an alternative approach is to use notification instead of
request, but that would require protocol changes and taking backward
compatibility into consideration. So this patch takes a different
approach and tries to keep the change small.

[0] http://patchwork.ozlabs.org/project/openvswitch/patch/1594380801-32134-1-git-send-email-dceara@redhat.com/

[1] https://mail.openvswitch.org/pipermail/ovs-discuss/2021-February/050919.html

Reported-by: Girish Moodalbail <gmoodalbail@nvidia.com>
Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2020-July/050343.html
Reported-by: Tobias Hofmann <tohofman@cisco.com>
Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2021-February/050914.html
Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Han Zhou <hzhou@ovn.org>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2023-01-16 20:14:10 +01:00
Numan Siddique
55b9507e68 ovsdb-idl: Add the support to specify the uuid for row insert.
ovsdb-server allows the OVSDB clients to specify the uuid for
the row inserts [1].  Both the C IDL client library and Python
IDL are missing this feature.  This patch adds this support.

In C IDL, for each schema table, a new function is generated -
<schema_table>insert_persistent_uuid(txn, uuid) which can
be used the clients to persist the uuid.

ovs-vsctl and other derivatives of ctl now supports the same
in the generic 'create' command with the option "--id=<UUID>".

In Python IDL, the uuid to persist can be specified in
the Transaction.insert() function.

[1] - a529e3cd1f("ovsdb-server: Allow OVSDB clients to specify the UUID for inserted rows.:)

Acked-by: Adrian Moreno <amorenoz@redhat.com>
Acked-by: Han Zhou <hzhou@ovn.org>
Acked-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Numan Siddique <numans@ovn.org>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-11-30 15:15:57 +01:00
Dumitru Ceara
d94cd0d3ee ovsdb-idl: Support write-only-changed IDL monitor mode.
At a first glance, change tracking should never be allowed for
write-only columns.  However, some clients (e.g., ovn-northd) that are
mostly exclusive writers of a database, use change tracking to avoid
duplicating the IDL row records into a local cache when implementing
incremental processing.

The default behavior of the IDL is to automatically turn a write-only
column into a read-write column whenever the client enables change
tracking for that column.

For the afore mentioned clients, this becomes a performance issue.
Commit 1cc618c32524 ("ovsdb-idl: Fix atomicity of writes that don't
change a column's value.") explains why writes that don't change a
column's value cannot be optimized out early if the column is
read/write.

Furthermore, if there is at least one record in any table that
changed during a transaction, then *all* records that have been
written are added to the transaction, even if their values didn't
change.  If there are many such rows (e.g., like in ovn-northd's
case) this incurs a significant overhead because:
a. the client has to build this large transaction
b. the transaction has to be sent over the network
c. the server needs to parse this (mostly) no-op update

We now introduce new IDL APIs allowing users to set a new monitoring
mode flag, OVSDB_IDL_WRITE_CHANGED_ONLY, to indicate to the IDL that the
atomicity constraints may be relaxed and written columns that don't
change value can be skipped from the current transaction.

We benchmarked ovn-northd performance when using this new mode
against NB and SB databases taken from ovn-kubernetes scale tests.
We noticed that when a minor change is performed to the Northbound
database (e.g., NB_Global.nb_cfg is incremented) the time it takes to
build the Southbound transaction becomes negligible (vs ~1.5 seconds
before this change).

End-to-end ovn-kubernetes scale tests on 120-node clusters also show
significant reduction of latency to bring up pods; both average and P99
latency decreased by ~30%.

Acked-by: Han Zhou <hzhou@ovn.org>
Signed-off-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-04-28 16:57:43 +02:00
Ilya Maximets
066741d9c5 ovsdb-idl: Add memory report function.
Added new function to return memory usage statistics for database
objects inside IDL.  Statistics similar to what ovsdb-server reports.
Not counting _Server database as it should be small, hence doesn't
worth adding extra code to the ovsdb-cs module.  Can be added later
if needed.

ovs-vswitchd is a user in OVS, but this API will be mostly useful for
OVN daemons.

Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Acked-by: Han Zhou <hzhou@ovn.org>
Acked-by: Dumitru Ceara <dceara@redhat.com>
2021-11-04 23:13:13 +01:00
Numan Siddique
7502849e95 ovsdb-idl: Add APIs to query if a table and a column is present.
This patch adds 2 new APIs in the ovsdb-idl client library
 - ovsdb_idl_server_has_table() and ovsdb_idl_server_has_column() to
query if a table and a column is present in the IDL or not.  This
patch also adds IDL helper functions which are auto generated from
the schema which makes it easier for the clients.

These APIs are required for scenarios where the server schema is old and
missing a table or column and the client (built with a new schema
version) does a transaction with the missing table or column.  This
results in a continuous loop of transaction failures.

Related-Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1992705
Signed-off-by: Numan Siddique <nusiddiq@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-08-28 02:59:04 +02:00
Dumitru Ceara
4c0d093b17 ovsdb-idl.at: Make test outputs more predictable.
IDL tests need predictable output from test-ovsdb.

This used to be done by first sorting the output of test-ovsdb and then
applying uuidfilt to predictably translate UUIDs.  This was not
reliable enough in case test-ovsdb processes two or more insert/delete
operations in the same iteration because the order of lines in the
output depends on the automatically generated UUID values.

To fix this we change the way test-ovsdb and test-ovsdb.py generate
outputs and prepend the table name and tracking information before
printing the contents of a row.

All existing ovsdb-idl.at and ovsdb-cluster.at tests are updated to
expect the new output format.

Signed-off-by: Dumitru Ceara <dceara@redhat.com>
Acked-by: Han Zhou <hzhou@ovn.org>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-04-01 13:53:20 +02:00
Ben Pfaff
47da7fa5a0 ovsdb-idl: Improve prototypes.
Adding parameter names makes these prototypes clearer.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Ilya Maximets <i.maximets@ovn.org>
2020-12-18 18:32:14 -08:00
Ben Pfaff
ba67afcf2b ovsdb-idl: Remove prototype for function that is not defined or used.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Ilya Maximets <i.maximets@ovn.org>
2020-12-18 18:32:10 -08:00
Ben Pfaff
8205fbc8f5 Eliminate "whitelist" and "blacklist" terms.
There is one remaining use under datapath.  That change should happen
upstream in Linux first according to our usual policy.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Alin Gabriel Serdean <aserdean@ovn.org>
2020-10-16 19:22:24 -07:00
Mark Michelson
89b522aee3 ovsdb-idl: Add function to reset min_index.
If an administrator removes all of the databases in a cluster from
disk, then ovsdb IDL clients will have a problem. The databases will all
reset their stored indexes to 0, so The IDL client's min_index will be
higher than the indexes of all databases in the cluster. This results in
the client constantly connecting to databases, detecting the data as
"stale", and then attempting to connect to another.

This function provides a way to reset the IDL to an initial state with
min_index of 0. This way, the client will not wrongly detect the
database data as stale and will recover properly.

Notice that this function is not actually used anywhere in this patch.
This will be used by OVN, though, since OVN is the primary user of
clustered OVSDB.

Signed-off-by: Mark Michelson <mmichels@redhat.com>
Acked-by: Han Zhou <hzhou@ovn.org>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2020-05-28 18:36:48 +02:00
Ben Pfaff
695e540334 ovsdb-idl: Improve comments.
Acked-by: Numan Siddique <nusiddiq@redhat.com>
Suggested-by: Numan Siddique <nusiddiq@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2019-07-03 09:47:27 -07:00
Han Zhou
d59050555c ovsdb-idl: Support optionally not shuffling multiple remotes.
This patch allows remotes not being shuffled if desired (mostly for
testing purpose, when we need the order of remotes during retrying
be predictable). By default it still shuffles as how it behaves today.

Signed-off-by: Han Zhou <hzhou8@ebay.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2019-04-15 12:54:41 -07:00
Ben Pfaff
5e07b8f93f ovsdb-idl: New function ovsdb_idl_create_unconnected().
This new function makes it possible to create an instance of the IDL
without connecting it to a remote OVSDB server.  The caller can then
connect and disconnect using ovsdb_idl_set_remote(); the ability to
disconnect is a new feature.

With this patch, the ovsdb_idl 'session' member can be null whereas
previously it was always nonnull.  The scattered changes throughout
ovsdb-idl are to cope with this new possibility.

An upcoming patch will introduce the first user of this new feature.

Signed-off-by: Ben Pfaff <blp@ovn.org>
2018-10-04 12:55:21 -07:00
Lorenzo Bianconi
82b261b9ef Introduce ovs-appctl command to monitor HVs sb connection status
Add 'connection-status' command to ovs-appctl utility in order to check
if a given chassis is currently connected to SB db

Acked-by: Mark Michelson <mmichels@redhat.com>
Co-authored-by: aginwala <aginwala@ebay.com>
Signed-off-by: aginwala <aginwala@ebay.com>
Signed-off-by: Lorenzo Bianconi <lorenzo.bianconi@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2018-07-31 12:49:19 -07:00
Ben Pfaff
d14e007c1f ovsdb-idl: Redesign use of indexes.
The design of the compound index feature in the C OVSDB IDL was unusual.
Indexes were generally referenced only by name rather than by pointer, and
could be obtained only from the top-level ovsdb_idl object.  To iterate or
otherwise search an index required explicitly creating a special
ovsdb_idl_cursor object, which at least seemed somewhat heavy-weight given
that it required a string lookup in a table of indexes.

This commit redesigns the compound index interface.  It discards the use of
names for indexes, instead having clients pass in a pointer to the index
object itself.  It simplifies how indexes are created, gets rid of the need
for explicit cursor objects, and updates all of the users to the new
interface.

The underlying reason for this commit is to make it easier in
ovn-controller to keep track of the dependencies for a given function, by
making the indexes explicit arguments to any function that needs to use
them.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Han Zhou <hzhou8@ebay.com>
2018-06-12 08:24:08 -07:00
Ben Pfaff
1b1d2e6daa ovsdb: Introduce experimental support for clustered databases.
This commit adds support for OVSDB clustering via Raft.  Please read
ovsdb(7) for information on how to set up a clustered database.  It is
simple and boils down to running "ovsdb-tool create-cluster" on one server
and "ovsdb-tool join-cluster" on each of the others and then starting
ovsdb-server in the usual way on all of them.

One you have a clustered database, you configure ovn-controller and
ovn-northd to use it by pointing them to all of the servers, e.g. where
previously you might have said "tcp:1.2.3.4" was the database server,
now you say that it is "tcp:1.2.3.4,tcp:5.6.7.8,tcp:9.10.11.12".

This also adds support for database clustering to ovs-sandbox.

Acked-by: Justin Pettit <jpettit@ovn.org>
Tested-by: aginwala <aginwala@asu.edu>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2018-03-24 12:04:53 -07:00
Ben Pfaff
1456335dc5 ovsdb-idl: Break out database-specific stuff into new data structure.
Until now, a given ovsdb-idl instances has only monitored a single
database.  In an upcoming commit, it will grow to also monitor a second
database that represents the state of the database server itself.  Much of
the work is the same for both databases, so this commit breaks the common
code and data out into new data structures and functions.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Justin Pettit <jpettit@ovn.org>
2018-03-24 12:04:51 -07:00
Yi-Hung Wei
9131abe223 lib, ovsdb: Adapt headers for C++ usage
This patch adds 'extern "C"' in a couple of header files so that
they can be compiled with C++ compilers.

Signed-off-by: Yi-Hung Wei <yihung.wei@gmail.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2017-12-01 13:22:18 -08:00
Lance Richardson
93fe026466 ovsdb-idl: idl compound indexes implementation
This patch adds support for the creation of multicolumn indexes
in the C IDL to enable for efficient search and retrieval of database
rows by key.

Signed-off-by: Esteban Rodriguez Betancourt <estebarb@hpe.com>
Co-authored-by: Lance Richardson <lrichard@redhat.com>
Signed-off-by: Lance Richardson <lrichard@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2017-08-03 14:48:34 -07:00
Ben Pfaff
a0b02897cf db-ctl-base: Drop redundant 'table' field from struct ctl_row_id.
The 'table' field is redundant because the required 'column' field
implies the table that the column is a part of.

This simplifies the users and makes it harder to get these things wrong.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Andy Zhou <azhou@ovn.org>
2017-05-03 08:31:13 -07:00
Andy Zhou
46437c5232 ovsdb-idl: Enhance conditional monitoring API
To allow client to know when the conditional monitoring changes
has been accepted by the OVSDB server and the 'idl' contents has
been updated to match the new conditions.

Signed-off-by: Andy Zhou <azhou@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
2017-01-09 12:48:08 -08:00
Ben Pfaff
0164e367f5 ovsdb-idl: Change interface to conditional monitoring.
Most users of OVSDB react to whatever is currently in their view of the
database, as opposed to keeping track of changes and reacting to those
changes individually.  The interface to conditional monitoring was
different, in that it expected the client to say what to add or remove from
monitoring instead of what to monitor.  This seemed reasonable at the time,
but in practice it turns out that the usual approach actually works better,
because the condition is generally a function of the data visible in the
database.  This commit changes the approach.

This commit also changes the meaning of an empty condition for a table.
Previously, an empty condition meant to replicate every row.  Now, an empty
condition means to replicate no rows.  This is more convenient for code
that gradually constructs conditions, because it does not need special
cases for replicating nothing.

This commit also changes the internal implementation of conditions from
linked lists to arrays.  I just couldn't see an advantage to using linked
lists.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Liran Schour <lirans@il.ibm.com>
2016-12-19 21:02:11 -08:00
Ben Pfaff
11990a5274 ovsdb-idl: Check internal graph in OVSDB tests.
Some upcoming tests will add extra trickiness to the IDL internal graph.
This worries me, because the IDL doesn't have any checks for its graph
consistency.  This commit adds some.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Andy Zhou <azhou@ovn.org>
2016-10-19 11:39:26 -07:00
Ben Pfaff
8ba0e38ac8 ovsdb-idl: Return result of transaction in ovsdb_idl_loop_commit_and_wait().
This will have a caller in the future.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Signed-off-by: Justin Pettit <jpettit@ovn.org>
2016-09-23 12:06:59 -07:00
Ben Pfaff
239fa5bbe6 ovsdb-idl: Style and comment improvements for conditional replication.
The conditional replication code had hardly any comments.  This adds some.

This commit also fixes a number of style problems, factors out some code
into a helper function, and moves some struct declarations from a public
header, that were not used by client code, into more private locations.

Signed-off-by: Ben Pfaff <blp@ovn.org>
2016-08-15 17:12:17 -07:00
Ryan Moats
f1ab6e0606 ovsdb: Add/use partial set updates.
This patchset mimics the changes introduced in

  f199df26 (ovsdb-idl: Add partial map updates functionality.)
  010fe7ae (ovsdb-idlc.in: Autogenerate partial map updates functions.)
  7251075c (tests: Add test for partial map updates.)
  b1048e6a (ovsdb-idl: Fix issues detected in Partial Map Update feature)

but for columns that store sets of values rather than key-value
pairs.  These columns will now be able to use the OVSDB mutate
operation to transmit deltas on the wire rather than use
verify/update and transmit wait/update operations on the wire.

Side effect of modifying the comments in the partial map update
tests.

Signed-off-by: Ryan Moats <rmoats@us.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2016-08-14 16:31:15 -07:00
Ben Pfaff
de32cec780 ovn-nbctl: Add "sync" command to wait for previous changes to take effect.
It's slow to add --wait to every ovn-nbctl command; only the last command
needs it.  But it's sometimes inconvenient to add it to the last command
if it's in a loop, etc.  This makes it possible to separately wait for
the OVN southbound or hypervisors to catch up to the northbound.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Ryan Moats <rmoats@us.ibm.com>
2016-08-08 09:07:18 -07:00
Ben Pfaff
fa183acc65 ovn: Make it possible for CMS to detect when the OVN system is up-to-date.
Until now, there has been no reliable for the CMS (or ovn-nbctl, or
anything else) to detect when changes made to the northbound configuration
have been passed through to the southbound database or to the hypervisors.
This commit adds this feature to the system, by adding sequence numbers
to the northbound and southbound databases and adding code in ovn-nbctl,
ovn-northd, and ovn-controller to keep those sequence numbers up-to-date.

The biggest user-visible change from this commit is new a new option
--wait to ovn-nbctl.  With --wait=sb, ovn-nbctl now waits for ovn-northd
to update the southbound database; with --wait=hv, it waits for the
changes to make their way to Open vSwitch on every hypervisor.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Russell Bryant <russell@ovn.org>
2016-07-26 23:59:53 -07:00
Liran Schour
16ebb90e05 lib: add monitor_cond_change API to C IDL lib
Add to IDL API that allows the user to add and remove clauses on a table's condition
iteratively. IDL maintain tables condition and send monitor_cond_change to the server
upon condition change.
Add tests for conditional monitoring to IDL.

Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2016-07-18 22:58:45 -07:00
Liran Schour
7152b6fa95 lib: add to ovsdb-idl monitor_id
IDL uses now a uuid to specify a monitoring session that is being
sent to the server on "monitor_cond" request.
This uuid will be used to issue ongoing "monitor_cond_change" requests
for this monitoring session.

Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2016-07-18 22:58:44 -07:00
Edward Aymerich
f199df26e8 ovsdb-idl: Add partial map updates functionality.
In the current implementation, every time an element of either a map or set
column has to be modified, the entire content of the column is sent to the
server to be updated. This is not a major problem if the information contained
in the column for the corresponding row is small, but there are cases where
these columns can have a significant amount of elements per row, or these
values are updated frequently, therefore the cost of the modifications becomes
high in terms of time and bandwidth.

In this solution, the ovsdb-idl code is modified to use the RFC 7047 'mutate'
operation, to allow sending partial modifications on map columns to the server.
The functionality is exposed to clients in the vswitch idl. This was
implemented through map operations.

A map operation is defined as an insertion, update or deletion of a key-value
pair inside a map. The idea is to minimize the amount of map operations
that are send to the OVSDB server when a transaction is committed.

In order to keep track of the requested map operations, structs map_op and
map_op_list were defined with accompanying functions to manipulate them. These
functions make sure that only one operation is send to the server for each
key-value that wants to be modified, so multiple operation on a key value are
collapsed into a single operation.

As an example, if a client using the IDL updates several times the value for
the same key, the functions will ensure that only the last value is send to
the server, instead of multiple updates. Or, if the client inserts a key-value,
and later on deletes the key before committing the transaction, then both
actions cancel out and no map operation is send for that key.

To keep track of the desired map operations on each transaction, a list of map
operations (struct map_op_list) is created for every column on the row on which
a map operation is performed. When a new map operation is requested on the same
column, the corresponding map_op_list is checked to verify if a previous
operations was performed on the same key, on the same transaction. If there is
no previous operation, then the new operation is just added into the list. But
if there was a previous operation on the same key, then the previous operation
is collapsed with the new operation into a single operation that preserves the
final result if both operations were to be performed sequentially. This design
keep a small memory footprint during transactions.

When a transaction is committed, the map operations lists are checked and
all map operations that belong to the same map are grouped together into a
single JSON RPC "mutate" operation, in which each map_op is transformed into
the necessary "insert" or "delete" mutators. Then the "mutate" operation is
added to the operations that will be send to the server.

Once the transaction is finished, all map operation lists are cleared and
deleted, so the next transaction starts with a clean board for map operations.

Using different structures and logic to handle map operations, instead of
trying to force the current structures (like 'old' and 'new' datums in the row)
to handle then, ensures that map operations won't mess up with the current
logic to generate JSON messages for other operations, avoids duplicating the
whole map for just a few changes, and is faster for insert and delete
operations, because there is no need to maintain the invariants in the 'new'
datum.

Signed-off-by: Edward Aymerich <edward.aymerich@hpe.com>
Signed-off-by: Arnoldo Lutz <arnoldo.lutz.guevara@hpe.com>
Co-authored-by: Arnoldo Lutz <arnoldo.lutz.guevara@hpe.com>
[blp@ovn.org made style changes and factored out error checking]
Signed-off-by: Ben Pfaff <blp@ovn.org>
2016-05-18 10:14:37 -07:00
RYAN D. MOATS
104aec4e0c Add change tracking documentation
Change tracking is a bit different from what someone with
"classic" database experience might expect, so let's add
the knowledged gained from the experience of making change
tracking work for incremental processing.

Signed-off-by: RYAN D. MOATS <rmoats@us.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2016-05-06 10:05:33 -07:00
RYAN D. MOATS
1b62572da1 ovn-controller: Dynamically reconnect if ovn-remote value changes.
Allows for auto detection and reconnect if the ovn-remote needs
to change.  Ovn-controller test case updated to include testing
this code.

Signed-off-by: RYAN D. MOATS <rmoats@us.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2016-04-14 14:58:18 -07:00
Huang Lei
a4927e36ff ovn-controller: Add external-id 'ovn-remote-probe-interval'
Add a external-id 'ovn-remote-probe-interval' for setting the activity probe
interval of the json session from ovn-controller to the OVN southbound database.

Signed-off-by: Huang Lei <lhuang8@ebay.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2016-03-30 17:06:31 -07:00
Shad Ansari
32d37ce851 ovsdb-idl: Add support for column tracking in IDL.
Recent IDL change tracking patches allow quick traversal of changed
rows. This patch adds additional support to track changed columns.
It allows an IDL client to efficiently check if a specific column
of a row was updated by IDL.

Signed-off-by: Shad Ansari <shad.ansar@hpe.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2016-01-12 11:25:26 -08:00
Shad Ansari
932104f483 ovsdb-idl: Add support for change tracking.
Ovsdb-idl notifies a client that something changed; it does not track
which table, row changed in what way (insert, modify or delete).
As a result, a client has to scan or reconfigure the entire idl after
ovsdb_idl_run(). This is presumably fine for typical ovs schemas where
tables are relatively small. In use-cases where ovsdb is used with
schemas that can have very large tables, the current ovsdb-idl
notification mechanism does not appear to scale - clients need to do a
lot of processing to determine the exact change delta.

This change adds support for:
 - Table and row based change sequence numbers to record the
   most recent IDL change sequence numbers associated with insert,
   modify or delete update on that table or row.
 - Change tracking of specific columns. This ensures that changed
   rows (inserted, modified, deleted) that have tracked columns, are
   tracked by IDL. The client can directly access the changed rows
   with get_first, get_next operations without the need to scan the
   entire table.
   The tracking functionality is not enabled by default and needs to
   be turned on per-column by the client after ovsdb_idl_create()
   and before ovsdb_idl_run().

     /* Example Usage */

     idl = ovsdb_idl_create(...);

     /* Track specific columns */
     ovsdb_idl_track_add_column(idl, column);
     /* Or, track all columns */
     ovsdb_idl_track_add_all(idl);

     for (;;) {
         ovsdb_idl_run(idl);
         seqno = ovsdb_idl_get_seqno(idl);

         /* Process only the changed rows in Table FOO */
         FOO_FOR_EACH_TRACKED(row, idl) {
             /* Determine the type of change from the row seqnos */
             if (foo_row_get_seqno(row, OVSDB_IDL_CHANGE_DELETE)
                    >= seqno)) {
                 printf("row deleted\n");
             } else if (foo_row_get_seqno(row, OVSDB_IDL_CHANGE_MODIFY)
                           >= seqno))
                 printf("row modified\n");
             } else if (foo_row_get_seqno(row, OVSDB_IDL_CHANGE_INSERT)
                           >= seqno))
                 printf("row inserted\n");
             }
         }

         /* All changes processed - clear the change track */
         ovsdb_idl_track_clear(idl);
    }

Signed-off-by: Shad Ansari <shad.ansari@hp.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2015-11-23 08:47:10 -08:00
Alex Wang
a660eac8f1 ovsdb-idl: Move get_initial_snapshot() to ovsdb-idl.
The same function is defined in both ovn-controller.c and
ovn-controller-vtep.c, so worth librarizing.

Signed-off-by: Alex Wang <alexw@nicira.com>
Acked-by: Russell Bryant <rbryant@redhat.com>
2015-08-10 19:53:49 -07:00
Alex Wang
a548a764b8 idl-loop: Move idl-loop into ovsdb-idl library.
idl-loop is needed in implementing other controller (i.e., vtep controller).
So, this commit moves the logic into ovsdb-idl library module.

Signed-off-by: Alex Wang <alexw@nicira.com>
Acked-by: Russell Bryant <rbryant@redhat.com>
2015-08-08 09:49:39 -07:00
Ben Pfaff
d322ffc81f ovsdb-idl: Document that the IDL always presents a consistent view.
We've had this question a couple of times so we might as well document it.

Requested-by: Saurabh Shrivastava (सौरभ श्रीवास्तव) <saurabh@gmail.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
2015-06-11 10:47:47 -07:00
Thomas Graf
cab5044987 lib: Move compiler.h to <openvswitch/compiler.h>
The following macros are renamed to avoid conflicts with other headers:
 * WARN_UNUSED_RESULT to OVS_WARN_UNUSED_RESULT
 * PRINTF_FORMAT to OVS_PRINTF_FORMAT
 * NO_RETURN to OVS_NO_RETURN

Signed-off-by: Thomas Graf <tgraf@noironetworks.com>
Acked-by: Ben Pfaff <blp@nicira.com>
2014-12-15 14:14:47 +01:00
Ben Pfaff
ff495b6378 ovs-vsctl: Allow modifying "immutable" columns if we just created the row.
OVSDB has the concept of "immutable" columns, which are columns whose
values are fixed once a row is inserted.  Until now, ovs-vsctl has not
allowed these columns to be modified at all.  However, this is a little too
strict, because these columns can be set to any value at the time that the
row is inserted.  This commit relaxes the ovs-vsctl requirement, then, to
allow an immutable column's value to be modified if its row has been
inserted within this transaction.

Requested-by: Mukesh Hira <mhira@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
2014-10-08 13:12:45 -07:00
Ansis Atteka
705d7a3930 ovs-vsctl: reconnect to the database if connection was dropped
If ovs-vsctl has to wait for ovs-vswitchd to reconfigure itself
according to the new database, then sometimes ovs-vsctl could
end up stuck in the event loop if OVSDB connection was dropped
while ovs-vsctl was still running.

This patch fixes this problem by letting ovs-vsctl to reconnect
to the OVSDB, if it has to wait cur_cfg field to be updated.

Issue: 1191997
Reported-by: Spiro Kourtessis <spiro@nicira.com>
Signed-Off-By: Ansis Atteka <aatteka@nicira.com>
2014-02-19 14:01:05 -08:00
Ben Pfaff
fba6bd1d3f ovs-vsctl: Try connecting only once for active connections by default.
Until now, ovs-vsctl has kept trying to the database server until it
succeeded or the timeout expired (if one was specified with --timeout).
This meant that if ovsdb-server wasn't running, then ovs-vsctl would hang.
The result was that almost every ovs-vsctl invocation in scripts specified
a timeout on the off-chance that the database server might not be running.
But it's difficult to choose a good timeout.  A timeout that is too short
can cause spurious failures.  A timeout that is too long causes long delays
if the server really isn't running.

This commit should alleviate this problem.  It changes ovs-vsctl's behavior
so that, if it fails to connect to the server, it exits unsuccessfully.
This makes --timeout obsolete for the purpose of avoiding a hang if the
database server isn't running.  (--timeout is still useful to avoid a hang
if ovsdb-server is running but ovs-vswitchd is not, for ovs-vsctl commands
that modify the database.  --no-wait also avoids that issue.)

Bug #2393.
Bug #15594.
Reported-by: Jeff Merrick <jmerrick@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
2013-03-15 16:26:47 -07:00
Ben Pfaff
fe19569ac5 ovsdb-idlc: Make no-op writes to write-only columns cheaper.
For 1000 tunnels with CFM enabled, this reduces CPU use from
about 36% to about 30%.

Bug #15171.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
2013-03-05 21:35:41 -08:00
Ethan Jackson
8cdec72569 idl: Optionally warn when writing to read-write columns.
ovs-vswitchd should only write to write-only columns.  Furthermore,
writing to a column which is not write-only can cause serious
performance degradations.  This patch causes ovs-vswitchd to log
and reject writes to read-write columns.

Signed-off-by: Ethan Jackson <ethan@nicira.com>
2012-09-27 17:23:17 -07:00
Raju Subramanian
e0edde6fee Global replace of Nicira Networks.
Replaced all instances of Nicira Networks(, Inc) to Nicira, Inc.

Feature #10593
Signed-off-by: Raju Subramanian <rsubramanian@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
2012-05-02 17:08:02 -07:00
Ben Pfaff
2f92678735 ovsdb-idl: Improve documentation.
Signed-off-by: Ben Pfaff <blp@nicira.com>
2012-04-12 08:28:14 -07:00
Ben Pfaff
94fbe1aae2 ovsdb-idl: Improve ovsdb_idl_txn_increment() interface.
The previous interface was just bizarre.

Signed-off-by: Ben Pfaff <blp@nicira.com>
2012-04-12 08:28:13 -07:00
Ben Pfaff
854a94d9d2 ovsdb-idl: Simplify transaction retry.
Originally the IDL transaction state machine had a return value
TXN_TRY_AGAIN to signal the client to wait for a change in the database and
then retry its transaction.  However, this logic was incomplete, because
it was possible for the database to change before the reply to the
transaction RPC was received, in which case the client would wait for a
further change.  Commit 4fdfe5ccf84c (ovsdb-idl: Prevent occasional hang
when multiple database clients race.) fixed the problem by breaking
TXN_TRY_AGAIN into two status codes, TXN_AGAIN_WAIT that meant to wait for
a further change and TXN_AGAIN_NOW that meant that a change had already
occurred so try again immediately.

This is correct enough, but it is more complicated than necessary.  It is
simpler and just as correct to use a single "try again" status that
requires the client to wait for a change relative to the database contents
*before* the transaction was committed.  This commit makes that change.
It also changes ovsdb_idl_run()'s return type from bool to void because
its return type is hardly useful anymore.

Signed-off-by: Ben Pfaff <blp@nicira.com>
2012-04-12 08:19:01 -07:00
Ben Pfaff
4fdfe5ccf8 ovsdb-idl: Prevent occasional hang when multiple database clients race.
When a client of the IDL tries to commit a read-modify-write transaction
but the database has changed in the meantime, the IDL tells its client to
wait for the IDL to change and then try the transaction again by returning
TXN_TRY_AGAIN.  The "wait for the IDL to change" part is important because
there's no point in retrying the transaction before the IDL has received
the database updates (the transaction would fail in the same way all over
again).

However, the logic was incomplete: the database update can be received
*before* the reply to the transaction RPC (I think that in the current
ovsdb-server implementation this will always happen, in fact).  When this
happens, the right thing to do is to retry the transaction immediately;
if we wait, then we're waiting for an additional change to the database
that may never come, causing an indefinite hang.

This commit therefore breaks the "try again" IDL commit status code
into two, one that means "try again immediately" and another that means
"wait for a change then try again".  When an update is processed after a
transaction is committed but before the reply is received, the "try again
now" tells the IDL client not to wait for another database change before
retrying its transaction.

Bug #5980.
Reported-by: Ram Jothikumar <rjothikumar@nicira.com>
Reproduced-by: Alex Yip <alex@nicira.com>
2011-10-31 09:15:14 -07:00