2
0
mirror of https://github.com/openvswitch/ovs synced 2025-08-25 11:27:46 +00:00

72 Commits

Author SHA1 Message Date
Ilya Maximets
485ac63d10 ovsdb: Add lazy-copy support for ovsdb_datum objects.
Currently ovsdb-server is using shallow copies of some JSON objects
by keeping a reference counter.  JSON string objects are also used
directly as ovsdb atoms in database rows to avoid extra copies.

Taking this approach one step further ovsdb_datum objects can also
be mostly deduplicated by postponing the copy until it actually
needed.  datum object itself contains a type and 2 pointers to
data arrays.  Adding a one more pointer to a reference counter
we may create a shallow copy of the datum by simply copying type
and pointers and increasing the reference counter.

Before modifying the datum, special function needs to be called
to perform an actual copy of the object, a.k.a. unshare it.
Most of the datum modifications are performed inside the special
functions in ovsdb-data.c, so that is not very hard to track.
A few places like ovsdb-server.c and column mutations are accessing
and changing the data directly, so a few extra unshare() calls
has to be added there.

This change doesn't affect the maximum memory consumption too much,
because most of the copies are short-living.  However, not actually
performing these copies saves up to 40% of CPU time on operations
with large sets.

Reported-at: https://bugzilla.redhat.com/2069089
Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-13 20:33:07 +02:00
Dumitru Ceara
471babb811 treewide: Avoid offsetting NULL pointers.
This is undefined behavior and was reported by UB Sanitizer:
  lib/meta-flow.c:3445:16: runtime error:
    member access within null pointer of type 'struct vl_mf_field'
      0 0x6aad0f in mf_get_vl_mff lib/meta-flow.c:3445
      1 0x6d96d7 in mf_from_oxm_header lib/nx-match.c:260
      2 0x6d9e2e in nx_pull_header__ lib/nx-match.c:341
      3 0x6daafa in nx_pull_header lib/nx-match.c:488
      4 0x6abcb6 in mf_vl_mff_nx_pull_header lib/meta-flow.c:3605
      5 0x73b9be in decode_NXAST_RAW_REG_MOVE lib/ofp-actions.c:2652
      6 0x764ccd in ofpact_decode lib/ofp-actions.inc2:4681
      [...]
  lib/sset.c:315:12: runtime error: applying zero offset to null pointer
      0 0xcc2e6a in sset_at_position lib/sset.c:315:12
      1 0x5734b3 in port_dump_next ofproto/ofproto-dpif.c:4083:20
      [...]
  lib/ovsdb-data.c:2194:56:
  runtime error: applying zero offset to null pointer
      0 0x5e9530 in ovsdb_datum_added_removed lib/ovsdb-data.c:2194:56
      1 0x4d6258 in update_row_ref_count ovsdb/transaction.c:335:17
      2 0x4c360b in for_each_txn_row ovsdb/transaction.c:1572:33
      [...]
  lib/ofpbuf.c:440:30:
  runtime error: applying zero offset to null pointer
      0 0x75066d in ofpbuf_push_uninit lib/ofpbuf.c:440
      1 0x46ac8a in ovnacts_parse lib/actions.c:4190
      2 0x46ad91 in ovnacts_parse_string lib/actions.c:4208
      3 0x4106d1 in test_parse_actions tests/test-ovn.c:1324
      [...]
  lib/ofp-actions.c:3205:22:
  runtime error: applying non-zero offset 2 to null pointer
      0 0x6e1641 in set_field_split_str lib/ofp-actions.c:3205:22
      [...]
  lib/tnl-ports.c:74:12:
  runtime error: applying zero offset to null pointer
      0 0xceffe7 in tnl_port_cast lib/tnl-ports.c:74:12
      1 0xcf14c3 in map_insert lib/tnl-ports.c:116:13
      [...]
  ofproto/ofproto.c:8905:16:
  runtime error: applying zero offset to null pointer
      0 0x556795 in eviction_group_hash_rule ofproto/ofproto.c:8905:16
      1 0x503f8d in eviction_group_add_rule ofproto/ofproto.c:9022:42
      [...]

Also, it's valid to have an empty ofpact list and we should be able to
try to iterate through it.

UB Sanitizer report:
  include/openvswitch/ofp-actions.h:222:12:
  runtime error: applying zero offset to null pointer
      0 0x665d69 in ofpact_end ./include/openvswitch/ofp-actions.h:222:12
      1 0x66b2cf in ofpacts_put_openflow_actions lib/ofp-actions.c:8861:5
      2 0x6ffdd1 in ofputil_encode_flow_mod lib/ofp-flow.c:447:9
      [...]

Signed-off-by: Dumitru Ceara <dceara@redhat.com>
Acked-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-05-17 23:08:13 +02:00
Dumitru Ceara
8ed26a8be3 treewide: Don't pass NULL to library functions that expect non-NULL.
It's actually undefined behavior to pass NULL to standard library
functions that manipulate arrays (e.g., qsort, memcpy, memcmp), even if
the passed number of items is 0.

UB Sanitizer reports:
  ovsdb/monitor.c:408:9: runtime error: null pointer passed as argument 1,
                                        which is declared to never be null
      #0 0x406ae1 in ovsdb_monitor_columns_sort ovsdb/monitor.c:408
      #1 0x406ae1 in ovsdb_monitor_add ovsdb/monitor.c:1683
  [...]
  lib/ovsdb-data.c:1970:5: runtime error: null pointer passed as argument 2,
                                          which is declared to never be null
      #0 0x4071c8 in ovsdb_datum_push_unsafe lib/ovsdb-data.c:1970
      #1 0x471cd0 in ovsdb_datum_apply_diff_in_place lib/ovsdb-data.c:2345
  [...]
  ofproto/ofproto-dpif-rid.c:159:17:
        runtime error: null pointer passed as argument 1,
                       which is declared to never be null
      #0 0x4df5d8 in frozen_state_equal ofproto/ofproto-dpif-rid.c:159
      #1 0x4dfd27 in recirc_find_equal ofproto/ofproto-dpif-rid.c:179
      [...]

Acked-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Dumitru Ceara <dceara@redhat.com>
Acked-by: Paolo Valerio <pvalerio@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-02-14 22:35:42 +01:00
Ilya Maximets
dec4291684 ovsdb-data: Consolidate ovsdb atom and json strings.
ovsdb_atom_string and json_string are basically the same data structure
and ovsdb-server frequently needs to convert one to another.  We can
avoid that by using json_string from the beginning for all ovsdb
strings.  So, the conversion turns into simple json_clone(), i.e.
increment of a reference counter.  This change gives a moderate
performance boost in some scenarios, improves the code clarity and
may be useful for future development.

Acked-by: Mike Pattrick <mkp@redhat.com>
Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-11-30 13:34:03 +01:00
Ilya Maximets
429b114c5a ovsdb-data: Deduplicate string atoms.
ovsdb-server spends a lot of time cloning atoms for various reasons,
e.g. to create a diff of two rows or to clone a row to the transaction.
All atoms, except for strings, contains a simple value that could be
copied in efficient way, but duplicating strings every time has a
significant performance impact.

Introducing a new reference-counted structure 'ovsdb_atom_string'
that allows to not copy strings every time, but just increase a
reference counter.

This change allows to increase transaction throughput in benchmarks
up to 2x for standalone databases and 3x for clustered databases, i.e.
number of transactions that ovsdb-server can handle per second.
It also noticeably reduces memory consumption of ovsdb-server.

Next step will be to consolidate this structure with json strings,
so we will not need to duplicate strings while converting database
objects to json and back.

Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Acked-by: Dumitru Ceara <dceara@redhat.com>
Acked-by: Mark D. Gray <mark.d.gray@redhat.com>
2021-09-24 15:53:46 +02:00
Ilya Maximets
32b51326ef ovsdb-data: Add function to apply diff in-place.
ovsdb_datum_apply_diff() is heavily used in ovsdb transactions, but
it's linear in terms of number of comparisons.  And it also clones
all the atoms along the way.  In most cases size of a diff is much
smaller than the size of the original datum, this allows to perform
the same operation in-place with only O(diff->n * log2(old->n))
comparisons and O(old->n + diff->n) memory copies with memcpy.
Using this function while applying diffs read from the storage gives
a significant performance boost and allows to execute much more
transactions per second.

Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Acked-by: Mark D. Gray <mark.d.gray@redhat.com>
2021-09-24 15:01:38 +02:00
Ilya Maximets
bb12b63176 ovsdb-data: Optimize subtraction of sets.
Current algorithm for ovsdb_datum_subtract looks like this:

  for-each atom in a:
      if atom in b:
          swap(atom, <last atom in 'a'>)
          destroy(atom)
  quicksort(a)

Complexity:

  Na * log2(Nb)  +  (Na - Nb) * log2(Na - Nb)
    Search          Comparisons for quicksort

It's not optimal, especially because Nb << Na in a vast majority of
cases.

Reversing the search phase to look up atoms from 'b' in 'a', and
closing gaps from deleted elements in 'a' by plain memory copy to
avoid quicksort.

Resulted complexity:

  Nb * log2(Na)  +    (Na - Nb)
    Search          Memory copies

Subtraction is heavily used while executing database transactions.
For example, to remove one port from a logical switch in OVN.
Complexity of such operation if original logical switch had 100 ports
goes down from

 100 * log2(1)   = 100 comparisons for search and
  99 * log2(99)  = 656 comparisons for quicksort
                   ------------------------------
                   756 comparisons in total
to only

   1 * log2(100) = 7 comparisons for search
   + memory copy of 99 * sizeof (union ovsdb_atom) bytes.

We could use memmove to close the gaps after removing atoms, but
it will lead to 2 memory copies inside the call, while we can perform
only one to the temporary 'result' and swap pointers.

Performance in cases, where sizes of 'a' and 'b' are comparable,
should not change.  Cases with Nb >> Na should not happen in practice.

All in all, this change allows ovsdb-server to perform several times
more transactions, that removes elements from sets, per second.

Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Acked-by: Han Zhou <hzhou@ovn.org>
Acked-by: Mark D. Gray <mark.d.gray@redhat.com>
2021-09-24 14:55:54 +02:00
Ilya Maximets
51946d2227 ovsdb-data: Optimize union of sets.
Current algorithm of ovsdb_datum_union looks like this:

  for-each atom in b:
      if not bin_search(a, atom):
          push(a, clone(atom))
  quicksort(a)

So, the complexity looks like this:

   Nb * log2(Na)   +    Nb     +   (Na + Nb) * log2(Na + Nb)
   Comparisons        clones       Comparisons for quicksort
   for search

ovsdb_datum_union() is heavily used in database transactions while
new element is added to a set.  For example, if new logical switch
port is added to a logical switch in OVN.  This is a very common
use case where CMS adds one new port to an existing switch that
already has, let's say, 100 ports.  For this case ovsdb-server will
have to perform:

   1 * log2(100)  + 1 clone + 101 * log2(101)
   Comparisons                Comparisons for
   for search                   quicksort.
       ~7           1            ~707
   Roughly 714 comparisons of atoms and 1 clone.

Since binary search can give us position, where new atom should go
(it's the 'low' index after the search completion) for free, the
logic can be re-worked like this:

  copied = 0
  for-each atom in b:
      desired_position = bin_search(a, atom)
      push(result, a[ copied : desired_position - 1 ])
      copied = desired_position
      push(result, clone(atom))
  push(result, a[ copied : Na ])
  swap(a, result)

Complexity of this schema:

   Nb * log2(Na)   +    Nb     +         Na
   Comparisons        clones       memory copy on push
   for search

'swap' is just a swap of a few pointers.  'push' is not a 'clone',
but a simple memory copy of 'union ovsdb_atom'.

In general, this schema substitutes complexity of a quicksort
with complexity of a memory copy of Na atom structures, where we're
not even copying strings that these atoms are pointing to.

Complexity in the example above goes down from 714 comparisons
to 7 comparisons and memcpy of 100 * sizeof (union ovsdb_atom) bytes.

General complexity of a memory copy should always be lower than
complexity of a quicksort, especially because these copies usually
performed in bulk, so this new schema should work faster for any input.

All in all, this change allows to execute several times more
transactions per second for transactions that adds new entries to sets.

Alternatively, union can be implemented as a linear merge of two
sorted arrays, but this will result in O(Na) comparisons, which
is more than Nb * log2(Na) in common case, since Na is usually
far bigger than Nb.  Linear merge will also mean per-atom memory
copies instead of copying in bulk.

'replace' functionality of ovsdb_datum_union() had no users, so it
just removed.  But it can easily be added back if needed in the future.

Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Acked-by: Han Zhou <hzhou@ovn.org>
Acked-by: Mark D. Gray <mark.d.gray@redhat.com>
2021-09-24 14:55:54 +02:00
Ilya Maximets
b2712d026e ovsdb: transaction: Use diffs for strong reference counting.
Currently, even if one reference added to the set of strong references
or removed from it, ovsdb-server will walk through the whole set and
re-count references to other rows.  These referenced rows will also be
added to the transaction in order to re-count their references.

For example, every time Logical Switch Port added to a Logical Switch,
OVN Northbound database server will walk through all ports of this
Logical Switch, clone their rows, and re-count references.  This is
not very efficient.  Instead, it can only increase reference counters
for added references and reduce for removed ones.  In many cases this
will be only one row affected in the Logical_Switch_Port table.

Introducing new function that generates a diff of two datum objects,
but stores added and removed atoms separately, so they can be used
to increase or decrease row reference counters accordingly.

This change allows to perform several times more transactions that
adds or removes strong references to/from sets per second, because
ovsdb-server no longer clones and re-counts rows that are irrelevant
to current transaction.

Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-09-23 00:21:49 +02:00
William Tu
e398275024 ovsdb-idl: Fix NULL deref reported by Coverity.
When 'datum.values' or 'datum.keys' is NULL, some code path calling
into ovsdb_idl_txn_write__ triggers NULL deref.  An example is below:

ovsrec_open_vswitch_set_cur_cfg(const struct ovsrec_open_vswitch
{
    struct ovsdb_datum datum;
    union ovsdb_atom key;

    datum.n = 1;
    datum.keys = &key;

    key.integer = cur_cfg;
//  1. assign_zero: Assigning: datum.values = NULL.
    datum.values = NULL;
//  CID 1421356 (#1 of 1): Explicit null dereferenced (FORWARD_NULL)
//  2. var_deref_model: Passing &datum to ovsdb_idl_txn_write_clone,\
//     which dereferences null datum.values.
    ovsdb_idl_txn_write_clone(&row->header_, &ovsrec_open_vswitch_col
}

And with the following calls:
ovsdb_idl_txn_write_clone
  ovsdb_idl_txn_write__
    6. deref_parm_in_call: Function ovsdb_datum_destroy dereferences
       datum->values
	ovsdb_datum_destroy

And another possible NULL deref is at ovsdb_datum_equals(). Fix the
two by adding additional checks.

Reviewed-by: Yifeng Sun <pkusunyifeng@gmail.com>
Signed-off-by: William Tu <u9012063@gmail.com>
2020-05-12 08:35:31 -07:00
Ilya Maximets
29004db273 ovsdb-data: Don't put strings with digits in quotes.
No need to use quotes for strings like "br0".

Keeping UUIDs always in quotes to avoid different treatment of those
that starts with digits and those that starts with letters.

Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
Acked-by: Ben Pfaff <blp@ovn.org>
2019-07-25 19:43:46 +03:00
Ben Pfaff
bc9bbc4ea2 ovsdb-data: Improve grammar in error message.
"must have exactly one member" is much better than "must have 1 to 1
members".

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Justin Pettit <jpettit@ovn.org>
2018-10-03 18:13:48 -07:00
Ben Pfaff
2f114c7e11 ovsdb-data: Drop redundant initialization from ovsdb_datum_apply_diff().
The call to ovsdb_datum_diff() initializes 'new', so it's not necessary to
also do it in ovsdb_datum_apply_diff().

Found by inspection.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Reviewed-by: Yifeng Sun <pkusunyifeng@gmail.com>
2018-09-26 13:26:05 -07:00
Ben Pfaff
fa37affad3 Embrace anonymous unions.
Several OVS structs contain embedded named unions, like this:

struct {
    ...
    union {
        ...
    } u;
};

C11 standardized a feature that many compilers already implemented
anyway, where an embedded union may be unnamed, like this:

struct {
    ...
    union {
        ...
    };
};

This is more convenient because it allows the programmer to omit "u."
in many places.  OVS already used this feature in several places.  This
commit embraces it in several others.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Justin Pettit <jpettit@ovn.org>
Tested-by: Alin Gabriel Serdean <aserdean@ovn.org>
Acked-by: Alin Gabriel Serdean <aserdean@ovn.org>
2018-05-25 13:36:05 -07:00
Ben Pfaff
53178986d7 ovsdb: Add support for online schema conversion.
With this change, "ovsdb-client convert" can be used to convert a database
from one schema to another without taking the database offline.

This can be useful to minimize downtime for a database during a software
upgrade.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Justin Pettit <jpettit@ovn.org>
2018-03-24 12:04:52 -07:00
Ben Pfaff
500db308e2 util: Document and rely on ovs_assert() always evaluating its argument.
The ovs_assert() macro always evaluates its argument, even when NDEBUG is
defined so that failure is ignored.  This behavior wasn't documented, and
thus a lot of code didn't rely on it.  This commit documents the behavior
and simplifies bits of code that heretofore didn't rely on it.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Reviewed-by: Yifeng Sun <pkusunyifeng@gmail.com>
2018-02-01 11:21:34 -08:00
Ben Pfaff
9041097aee ovsdb-client: Show even constraint-breaking data in "dump" output.
The ovsdb-client "dump" command is a fairly low-level tool that can be
used, among other purposes, to debug the OVSDB protocol.  It's better if
it just prints what the server sends without being too judgmental about it.
Thus, we might as well ignore constraints for the purpose of dumping
tables.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Justin Pettit <jpettit@ovn.org>
2017-12-22 11:51:47 -08:00
Ben Pfaff
fe0fb88551 ovsdb-client: Add new "restore" command.
Signed-off-by: Ben Pfaff <blp@ovn.org>
2017-12-21 16:41:30 -08:00
Ben Pfaff
3865965dd9 ovsdb-error: New function ovsdb_error_to_string_free().
This allows slight code simplifications across the tree.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Tested-by: Yifeng Sun <pkusunyifeng@gmail.com>
Reviewed-by: Yifeng Sun <pkusunyifeng@gmail.com>
2017-12-13 11:32:29 -08:00
Ben Pfaff
65d9759c4f ovsdb-data: Add OVS_WARN_UNUSED_RESULT annotations to function definitions.
The function prototypes in ovsdb-data.h already have these, but it seems
more complete to have the annotation on the definitions too.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Reviewed-by: Greg Rose <gvrose8192@gmail.com>
2017-12-08 13:39:29 -08:00
Lukasz Rzasik
1ab39058cc ovsdb-data: Add support for integer ranges in database commands
Adding / removing a range of integers to a column accepting a set of
integers requires enumarating all of the integers. This patch simplifies
it by introducing 'range' concept to the database commands. Two integers
separated by a hyphen represent an inclusive range.

The patch adds positive and negative tests for the new syntax.
The patch was tested by 'make check'. Covarage was tested by
'make check-lcov'.

Signed-off-by: Lukasz Rzasik <lukasz.rzasik@gmail.com>
Suggested-by: <my_ovs_discuss@yahoo.com>
Suggested-by: Ben Pfaff <blp@ovn.org>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2017-01-05 08:48:10 -08:00
Ben Pfaff
9b03e59d20 ovsdb-idlc: Use ovsdb_datum_from_smap() instead of open-coding it.
There's no reason to have three copies of this code for every smap-type
column.

The code wasn't a perfect match for ovsdb_datum_from_smap(), so this commit
also changes ovsdb_datum_from_smap() to better suit it.  It only had one
caller and the new design is adequate for that caller.

Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Andy Zhou <azhou@ovn.org>
2016-10-19 11:38:31 -07:00
Terry Wilson
ee89ea7b47 json: Move from lib to include/openvswitch.
To easily allow both in- and out-of-tree building of the Python
wrapper for the OVS JSON parser (e.g. w/ pip), move json.h to
include/openvswitch. This also requires moving lib/{hmap,shash}.h.

Both hmap.h and shash.h were #include-ing "util.h" even though the
headers themselves did not use anything from there, but rather from
include/openvswitch/util.h. Fixing that required including util.h
in several C files mostly due to OVS_NOT_REACHED and things like
xmalloc.

Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2016-07-22 17:09:17 -07:00
Ben Warren
3e8a2ad145 Move lib/dynamic-string.h to include/openvswitch directory
Signed-off-by: Ben Warren <ben@skyportsystems.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
2016-03-19 10:02:12 -07:00
Andy Zhou
1a6f1cefc0 lib: add diff and apply diff APIs for ovsdb_datum
When an OVSDB column change its value, it is more efficient to only
send what has changed, rather than sending the entire new copy.
This is analogous to software programmer send patches rather than
the entire source file.

For columns store a single element, the "diff" datum is the same
as the "new" datum.

For columns that store set or map, it is only necessary to send the
information about the elements changed (including addition or removal).
The "diff" for those types are all elements that are changed.

Those APIs are mainly used for implementing a new OVSDB server
"update2" JSON-RPC notification, which encodes modifications
of a column with the contents of those "diff"s. Later patch implements
the "update2" notification.

Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@ovn.org>
2015-12-11 14:22:33 -08:00
Andy Zhou
aefd97b779 lib: avoid set size check when generating diff datum from json
Added ovsdb_transient_datum_from_json() to avoid size check for
the diff datum that is transient in nature.
Suppose a datum contains set, and the max number of elements is 2.
If we are changing from set that contains [A, B], to a set contains
[C, D], the diff datum will contains 4 elements [A, B, C, D].

Thus diff datum should not be constrained by the size limit. However
the datum after diff is applied should not violate the size limit.

Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ben Pfaff <blp@ovn.org>
2015-12-11 14:22:32 -08:00
Thomas Graf
cab5044987 lib: Move compiler.h to <openvswitch/compiler.h>
The following macros are renamed to avoid conflicts with other headers:
 * WARN_UNUSED_RESULT to OVS_WARN_UNUSED_RESULT
 * PRINTF_FORMAT to OVS_PRINTF_FORMAT
 * NO_RETURN to OVS_NO_RETURN

Signed-off-by: Thomas Graf <tgraf@noironetworks.com>
Acked-by: Ben Pfaff <blp@nicira.com>
2014-12-15 14:14:47 +01:00
Ben Pfaff
c70a77673c ovsdb: Remove SPECS in favor of referring to RFC 7047.
Also, add some clarifications relative to RFC 7047 to ovsdb-server(1).

Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Justin Pettit <jpettit@nicira.com>
2014-04-04 13:51:32 -07:00
Ben Pfaff
7bd02255d9 Revert "ovsdb-data: New functions for predicting serialized length of data."
This reverts commit 0ea7bec76d804a2c4efccd3dbdaa3e30cf536a5c.

Connections that queue up too much data, because they are monitoring a
table that is changing quickly and failing to keep up with the updates,
cause problems with buffer management.  Since commit 60533a405b2e
(jsonrpc-server: Disconnect connections that queue too much data.),
ovsdb-server has dealt with them by disconnecting the connection and
letting them start up again with a fresh copy of the database.  However,
this is not ideal because of situations where disconnection happens
repeatedly.  For example:

     - A manager toggles a column back and forth between two or more values
       quickly (in which case the data transmitted over the monitoring
       connections always increases quickly, without bound).

     - A manager repeatedly extends the contents of some column in some row
       (in which case the data transmitted over the monitoring connection
       grows with O(n**2) in the length of the string).

A better way to deal with this problem is to combine updates when they are
sent to the monitoring connection, if that connection is not keeping up.
In both the above cases, this reduces the data that must be sent to a
manageable amount.  An upcoming patch implements this new way.  This commit
reverts part of the previous solution that disconnects backlogged
connections, since it is no longer useful.

Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Andy Zhou <azhou@nicira.com>
2014-04-03 07:53:48 -07:00
Harold Lim
428b2eddc9 Rename NOT_REACHED to OVS_NOT_REACHED
This allows other libraries to use util.h that has already
defined NOT_REACHED.

Signed-off-by: Harold Lim <haroldl@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
2013-12-17 13:16:39 -08:00
Alin Serdean
34582733d9 Avoid printf type modifiers not supported by MSVC C runtime library.
The MSVC C library printf() implementation does not support the 'z', 't',
'j', or 'hh' format specifiers.  This commit changes the Open vSwitch code
to avoid those format specifiers, switching to standard macros from
<inttypes.h> where available and inventing new macros resembling them
where necessary.  It also updates CodingStyle to specify the macros' use
and adds a Makefile rule to report violations.

Signed-off-by: Alin Serdean <aserdean@cloudbasesolutions.com>
Co-authored-by: Ben Pfaff <blp@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
2013-11-25 23:38:59 -08:00
Ben Pfaff
21a0e105f1 ovsdb-data: Make ovsdb_atom_default() thread-safe.
Signed-off-by: Ben Pfaff <blp@nicira.com>
2013-07-22 10:53:11 -07:00
Ben Pfaff
0ea7bec76d ovsdb-data: New functions for predicting serialized length of data.
These will be used for the first time in an upcoming commit.

Signed-off-by: Ben Pfaff <blp@nicira.com>
2013-04-01 13:20:04 -07:00
Ben Pfaff
cb22974d77 Replace most uses of assert by ovs_assert.
This is a straight search-and-replace, except that I also removed #include
<assert.h> from each file where there were no assert calls left.

Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
2013-01-16 16:03:37 -08:00
Ben Pfaff
ebc56baa41 util: New macro CONST_CAST.
Casts are sometimes necessary.  One common reason that they are necessary
is for discarding a "const" qualifier.  However, this can impede
maintenance: if the type of the expression being cast changes, then the
presence of the cast can hide a necessary change in the code that does the
cast.  Using CONST_CAST, instead of a bare cast, makes these changes
visible.

Inspired by my own work elsewhere:
http://git.savannah.gnu.org/cgit/pspp.git/tree/src/libpspp/cast.h#n80

Signed-off-by: Ben Pfaff <blp@nicira.com>
2012-08-03 13:33:13 -07:00
Ben Pfaff
57c8677b51 system-stats: Use "smap" instead of "shash".
"smap" is now the appropriate data structure for a string-to-string map.

Also changes ovsdb_datum_from_shash() into ovsdb_datum_from_smap() since
system-stats related code was the only client.

Signed-off-by: Ben Pfaff <blp@nicira.com>
2012-07-18 10:51:02 -07:00
Raju Subramanian
e0edde6fee Global replace of Nicira Networks.
Replaced all instances of Nicira Networks(, Inc) to Nicira, Inc.

Feature #10593
Signed-off-by: Raju Subramanian <rsubramanian@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
2012-05-02 17:08:02 -07:00
Ben Pfaff
e49190c493 Fix minor memory leaks found by valgrind.
All of these leaks are in normally short-lived programs, so none of them
is very important.

Signed-off-by: Ben Pfaff <blp@nicira.com>
2012-03-28 14:54:48 -07:00
Ben Pfaff
ca261b6535 ovsdb-data: Short-circuit ovsdb_datum_includes_all() in trivial case.
Signed-off-by: Ben Pfaff <blp@nicira.com>
2012-01-11 09:56:11 -08:00
Ben Pfaff
6a6f8d1673 ovsdb-data: Simplify converting an OVSDB datum to JSON by reordering logic.
Putting the "map" case first avoids duplicate tests.

Suggested-by: Reid Price <reid@nicira.com>
2011-08-24 11:57:42 -07:00
Ben Pfaff
be44585c21 tests: Fix the two Python XFAIL tests.
OVS has two Python tests that have always failed, for reasons not
understood, since they were added to the tree.  This commit fixes them.

One problem was that Python was assuming that stdout was encoded in ASCII.
Apparently the only way to "fix" this at runtime is to set PYTHONIOENCODING
to utf_8 in the environment, so this change does that.

Second, it appears that Python really doesn't like to print invalid UTF-8,
so this avoids doing that in python/ovs/json.py, instead just printing
the hexadecimal values of the invalid bytes.  For consistency, it makes
the same change to the C version.

Third, the C version of test-ovsdb doesn't check UTF-8 for consistency, it
just sends it blindly to the OVSDB server, but Python does check it and so
it bails out earlier.  This commit changes the Python version of the
"no invalid UTF-8 sequences in strings" to allow for the slight difference
in output that occurs for that reason.

Finally, test-ovsdb.py needs to convert error messages to Unicode
explicitly before printing them in the "parse-atoms" function.  I don't
really understand why, but now it works.
2011-05-24 11:32:22 -07:00
Ben Pfaff
c5f341ab19 ovsdb: Implement garbage collection. 2011-03-10 11:24:00 -08:00
Ben Pfaff
0dc66db95b ovsdb-data: Expose guts of ovsdb_symbol_table() to clients.
ovs-vsctl will, in upcoming commits, want to more closely examine its
ovsdb_symbol_table structures.  This could be done by providing a more
complete API, but it doesn't seem worth it to me.  This commit instead goes
the other way, exposing the internals to clients.  This commit also
eliminates the ovsdb_symbol_table_find_uncreated() function, which
ovs-vsctl can now implement itself.
2011-03-10 11:24:00 -08:00
Ben Pfaff
e9387de4a2 ovsdb-data: Rename 'used' to 'created' in struct ovsdb_symbol.
The name 'created' better reflects the actual meaning of this member: in
both ovsdb and ovs-vsctl, it is true if a row has been created with the
symbol's UUID and false otherwise.
2011-03-10 11:24:00 -08:00
Ben Pfaff
d198c4beee ovsdb-data: Verify that named-uuid string is an <id>.
The "uuid-name" that creates symbols must be an <id> but we weren't
verifying the same constraint on the "named-uuid"s that refer to symbols,
which was a bit confusing in writing transactions by hand.  This commit
fixes the inconsistency and updates the SPECS file to clarify that a
named-uuid string has to be an <id>.
2011-03-10 11:23:59 -08:00
Justin Pettit
9466d7d4b9 ovsdb-data: Free string leaked in ovsdb_datum_from_string().
Coverity #10725
2011-02-22 09:36:57 -08:00
Ben Pfaff
c6a4125250 table: Add new "bare" output formatting options.
--format=list corresponds to the output format that "ovs-vsctl list" has
always used.

--bare is easier for scripts to parse.
2011-02-08 16:10:05 -08:00
Ethan Jackson
829d41bdc8 ovsdb-data: Fix warnings.
This commit fixes "may be used uninitialized" warnings in
ovsdb-data.c
2011-01-13 10:48:22 -08:00
Ben Pfaff
e4af561537 ovsdb-data: New function ovsdb_datum_from_shash(). 2010-09-23 11:45:35 -07:00
Ben Pfaff
d931cde4d7 ovsdb: Remove unused ovsdb_datum_from_json_unique().
This function was not used outside of the test-ovsdb program.  It seems
like we might as well remove it.
2010-08-25 14:55:47 -07:00