Creating and destroying JSON objects may be time consuming.
Add json_serialized_object_create_with_yield() and
json_destroy_with_yield() functions that make use of the
cooperative multitasking module to yield during processing,
allowing time sensitive tasks in other parts of the program
to be completed during processing.
We keep these new functions private to OVS by adding a new
lib/json.h header file.
The include guard in the public include/openvswitch/json.h is
updated to contain the OPENVSWITCH prefix to be in line with the
other public header files, allowing us to use the non-prefixed
version in our private lib/json.h.
Signed-off-by: Frode Nordahl <frode.nordahl@canonical.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Refactoring of the replication code, so each database is handled
separately from each other. Supposed to work the same way as before
with the only difference that each backup database will have its own
connection to the source and will have its own state machine.
From the user's perspective, the only visible difference is that
ovsdb-server/sync-status appctl now shows the status of each
database separately.
If one of the connections is permanently broken, all the databases
will be switched to active. This is done in order to preserve the
old behavior where we had only one connection.
Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
With the next commit reference counting of json objects will take
significant part of the CPU time for ovsdb-server. Inlining them
to reduce the cost of a function call.
Acked-by: Mike Pattrick <mkp@redhat.com>
Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Introducing a new json type JSON_SERIALIZED_OBJECT. It's not an
actual type that can be seen in a json message on a wire, but
internal type that is intended to hold a serialized version of
some other json object. For this reason it's defined after the
JSON_N_TYPES to not confuse parsers and other parts of the code
that relies on compliance with RFC 4627.
With this JSON type internal users may construct large JSON objects,
parts of which are already serialized. This way, while serializing
the larger object, data from JSON_SERIALIZED_OBJECT can be added
directly to the result, without additional processing.
This will be used by next commits to add pre-serialized JSON data
to the raft_header structure, that can be converted to a JSON
before writing the file transaction on disk or sending to other
servers. Same technique can also be used to pre-serialize json_cache
for ovsdb monitors, this should allow to not perform serialization
for every client and will save some more memory.
Since serialized JSON is just a string, reusing the 'json->string'
pointer for it.
Acked-by: Dumitru Ceara <dceara@redhat.com>
Acked-by: Han Zhou <hzhou@ovn.org>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Several OVS structs contain embedded named unions, like this:
struct {
...
union {
...
} u;
};
C11 standardized a feature that many compilers already implemented
anyway, where an embedded union may be unnamed, like this:
struct {
...
union {
...
};
};
This is more convenient because it allows the programmer to omit "u."
in many places. OVS already used this feature in several places. This
commit embraces it in several others.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Justin Pettit <jpettit@ovn.org>
Tested-by: Alin Gabriel Serdean <aserdean@ovn.org>
Acked-by: Alin Gabriel Serdean <aserdean@ovn.org>
Until now, every time the JSON parser added an object member, it made an
extra copy of the member name and then freed the original copy. This is
wasteful, so this commit eliminates the extra copy.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Reviewed-by: Yifeng Sun <pkusunyifeng@gmail.com>
After profiling OVSDB insert performance it was found
that some significant portion of its time OVSDB is
calling the function json_clone.
Also, the current usages of json_clone never modify the json,
just keeps it to prevent it to be freed.
With that in mind the struct json, json_create, json_clone
and json_destroy were modified to keep a count of how many
references of the json struct are left. Only when that count
reaches zero the json struct is freed.
The old "json_clone" function was renamed as "json_deep_clone".
Some examples of the performance difference:
In these tests a test table with 4 columns (string, string,
bool, integer) was used. All the tests used "commit block".
*** 50 process each inserting 1000 rows ***
Master OVS
Test Duration 131 seconds
Average Inserts Per second 746.2687 inserts/s
Average Insert Duration 134.1382 ms
Minimal Insert Duration 0.166202 ms
Maximum Insert Duration 489.8593 ms
JSON GC Patch
Test Duration 86 seconds
Average Inserts Per second 1176 inserts/s
Average Insert Duration 82.26761 ms
Minimal Insert Duration 0.165448 ms
Maximum Insert Duration 751.2111 ms
*** 5 process each inserting 10000 rows ***
Master OVS
Test Duration 8 seconds
Average Inserts Per second 7142.857 inserts/s
Average Insert Duration 0.656431 ms
Minimal Insert Duration 0.125197 ms
Maximum Insert Duration 11.93203 ms
JSON GC Patch
Test Duration 7 seconds
Average Inserts Per second 8333.333 inserts/s
Average Insert Duration 0.55688 ms
Minimal Insert Duration 0.143233 ms
Maximum Insert Duration 26.26319 ms
Signed-off-by: Esteban Rodriguez Betancourt <estebarb@hpe.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
To easily allow both in- and out-of-tree building of the Python
wrapper for the OVS JSON parser (e.g. w/ pip), move json.h to
include/openvswitch. This also requires moving lib/{hmap,shash}.h.
Both hmap.h and shash.h were #include-ing "util.h" even though the
headers themselves did not use anything from there, but rather from
include/openvswitch/util.h. Fixing that required including util.h
in several C files mostly due to OVS_NOT_REACHED and things like
xmalloc.
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
