2
0
mirror of https://github.com/openvswitch/ovs synced 2025-08-24 02:47:14 +00:00

409 Commits

Author SHA1 Message Date
Adrian Moreno
863d2e1a8c python: Don't exit OFPFlow constructor.
Returning None in a constructor does not make sense and is just error
prone.  Removing what was a leftover from an attempt to handle a common
error case of trying to parse what is commonly outputted by ovs-ofctl.
This should be done by the caller anyway.

Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Acked-by: Mike Pattrick <mkp@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-12-21 19:48:07 +01:00
Adrian Moreno
c395e9810e python: Interpret free keys as output in clone.
clone-like actions can also output to ports by specifying the port name.

Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-12-21 18:36:02 +01:00
Adrian Moreno
542fdad701 python: Fix output=CONTROLLER action.
When CONTROLLER is used as free key, it means output=CONTROLLER which is
handled by decode_controller. However, it must output the KV in the
right format: "output": {"format": "CONTROLLER"}.

Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-12-21 18:36:02 +01:00
Adrian Moreno
1850e5e689 python: Support case-insensitive OpenFlow actions.
OpenFlow actions names can be capitalized so in order to support this,
support case-insensitive KVDecoders and use it in Openflow actions.

Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-12-21 18:36:02 +01:00
Adrian Moreno
75a6e8db9c python: Return list of actions for odp action clone.
Sometimes we don't want to return the result of a nested key-value
decoding as a dictionary but as a list of dictionaries. This happens
when we parse actions where keys can be repeated.

Refactor code that already takes that into account from ofp_act.py to
kv.py and use it for datapath action "clone".

Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Acked-by: Mike Pattrick <mkp@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-12-21 18:36:02 +01:00
Adrian Moreno
d33e548fc7 python: Make key-value matching strict by default.
Currently, if a key is not found in the decoder information, we use the
default decoder which typically returns a string.

This not only means we can go out of sync with the C code without
noticing but it's also error prone as malformed flows could be parsed
without warning.

Make KeyValue parsing strict, raising an error if a decoder is not found
for a key.
This behaviour can be turned off globally by running 'KVDecoders.strict
= False' but it's generally not recommended. Also, if a KVDecoder does
need this default behavior, it can be explicitly configured specifying
it's default decoder.

Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Acked-by: Mike Pattrick <mkp@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-12-21 18:36:02 +01:00
Adrian Moreno
fe204743cb python: Add explicit decoders for all ofp actions.
We were silently relying on some ofp actions to be decoded by the
default decoder which would yield decent string values.

In order to be more safe and robust, add an explicit decoder for all
missing actions.

This patch also reworks the learn action decoding to make it more
explicit and verify all the fields specified in the learn action are
actually valid fields.

Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-12-21 18:36:02 +01:00
Adrian Moreno
c627cfd9cb python: Fix datapath flow decoders.
Fix the following erros in odp decoding:
- Missing push_mpls action
- Typos in collector_set_id, tp_src/tp_dst and csum
- Missing two fields in vxlan match

Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Acked-by: Mike Pattrick <mkp@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-12-21 18:36:02 +01:00
Dumitru Ceara
a787fbbf9d ovsdb-cs: Consider default conditions implicitly acked.
When initializing a monitor table the default monitor condition is
[True] which matches the behavior of the server (to send all rows of
that table).  There's no need to include this default condition in the
initial monitor request so we can consider it implicitly acked by the
server.

This fixes the incorrect (one too large) expected condition sequence
number reported by ovsdb_idl_set_condition() when application is
trying to set a [True] condition for a new table.

Reported-by: Numan Siddique <numans@ovn.org>
Suggested-by: Ilya Maximets <i.maximets@ovn.org>
Signed-off-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-12-13 18:52:10 +01:00
Numan Siddique
55b9507e68 ovsdb-idl: Add the support to specify the uuid for row insert.
ovsdb-server allows the OVSDB clients to specify the uuid for
the row inserts [1].  Both the C IDL client library and Python
IDL are missing this feature.  This patch adds this support.

In C IDL, for each schema table, a new function is generated -
<schema_table>insert_persistent_uuid(txn, uuid) which can
be used the clients to persist the uuid.

ovs-vsctl and other derivatives of ctl now supports the same
in the generic 'create' command with the option "--id=<UUID>".

In Python IDL, the uuid to persist can be specified in
the Transaction.insert() function.

[1] - a529e3cd1f("ovsdb-server: Allow OVSDB clients to specify the UUID for inserted rows.:)

Acked-by: Adrian Moreno <amorenoz@redhat.com>
Acked-by: Han Zhou <hzhou@ovn.org>
Acked-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Numan Siddique <numans@ovn.org>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-11-30 15:15:57 +01:00
Christopher Aubut
a39ee99edc python: idl: Fix idl.Row.__str__ method.
Fixes idl.Row's __str__ method to only print if the column exists on
the object.  The Row object passed to the 'updates' argument of
Idl.notify only contains a subset of columns.  Printing that argument
causes an AttributeError.

Fixes: 6a1c98461b46 ("Add a __str__ method to idl.Row")
Submitted-at: https://github.com/openvswitch/ovs/pull/392
Acked-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Christopher Aubut <christopher@aubut.me>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-09-19 18:33:48 +02:00
Ilya Maximets
78dad3a0c8 python-c-ext: Use designated initializers for type and module.
Python documentation suggests to do so "to avoid listing all the
PyTypeObject fields that you don't care about and also to avoid
caring about the fields' declaration order".  And that does make
sense.

Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-08-30 21:41:32 +02:00
Miro Tomaska
1731ed43c6 python: Do not send non-zero flag for a SSL socket.
pyOpenSSL was recently switched for the Python standard library ssl
module in the cited commit.  Python SSLsocket.send() does not allow
non-zero optional flag and it will explicitly raise an exception for
that.  pyOpenSSL did nothing with this flag but kept it to be
compatible with socket API:
  https://github.com/pyca/pyopenssl/blob/main/src/OpenSSL/SSL.py#L1844

Fixes: 68543dd523bd ("python: Replace pyOpenSSL with ssl.")
Reported-at: https://bugzilla.redhat.com/2115035
Acked-By: Timothy Redaelli <tredaelli@redhat.com>
Signed-off-by: Miro Tomaska <mtomaska@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-08-12 01:23:39 +02:00
Ilya Maximets
434025a154 python: Fix E275 missing whitespace after keyword.
With just released flake8 5.0 we're getting a bunch of E275 errors:

 utilities/bugtool/ovs-bugtool.in:959:23: E275 missing whitespace after keyword
 tests/test-ovsdb.py:623:11: E275 missing whitespace after keyword
 python/setup.py:105:8: E275 missing whitespace after keyword
 python/setup.py:106:8: E275 missing whitespace after keyword
 python/ovs/db/idl.py:145:15: E275 missing whitespace after keyword
 python/ovs/db/idl.py:167:15: E275 missing whitespace after keyword
 make[2]: *** [flake8-check] Error 1

This breaks CI on branches below 2.16.  We don't see a problem right
now on newer branches because we're installing extra dependencies
that backtrack flake8 down to 4.1 or even 3.9.

Acked-by: Mike Pattrick <mkp@redhat.com>
Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-08-04 18:13:20 +02:00
Timothy Redaelli
6a9ec13aa3 python: Use setuptools instead of distutils.
On Python 3.12, distutils will be removed and it's currently (3.10+)
deprecated (see PEP 632).

Since the suggested and simplest replacement is setuptools, this commit
replaces distutils to use setuptools instead.

setuptools < 59.0 doesn't have setuptools.errors and so, in this case,
distutils.errors is still used.

Signed-off-by: Timothy Redaelli <tredaelli@redhat.com>
Acked-by: Mike Pattrick <mkp@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-08-04 14:01:23 +02:00
Ilya Maximets
791c472739 python: Add ability to pass extra libs and cflags for C extension.
In order to correctly link with static libopenvswitch.a library,
users should also provide required cflags and all the libraries
libopenvswitch.a was actually built with and depends on.  Otherwise,
it's not possible to link correctly.

Fixes: 671f93fe42d3 ("python: Allow building json C extension with static OVS library.")
Acked-by: Frode Nordahl <frode.nordahl@canonical.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-29 17:08:28 +02:00
Ilya Maximets
099d1c7454 python-c-ext: Handle initialization failures.
PyModule_AddObject() may fail and it doesn't steal references
in this case.  The error condition should be handled to avoid
possible memory leaks.

And while it's not strictly specified if PyModule_Create may
fail, most of the examples in python documentation include
handling of a NULL case.

Acked-by: Mike Pattrick <mkp@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-29 17:00:12 +02:00
Timothy Redaelli
d3c14abf47 python-c-ext: Fix a couple of build warnings.
ovs/_json.c:67:20: warning: assignment discards ‘const’ qualifier from pointer
target type [-Wdiscarded-qualifiers]

ovs/_json.c:132:27: warning: comparison of integer expressions of different
signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Wsign-compare]

Signed-off-by: Timothy Redaelli <tredaelli@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-22 17:08:03 +02:00
Timothy Redaelli
54ebc235ae python-c-ext: Remove Python 2 support.
Since Python 2 is not supported anymore, remove Python 2 support from C
extension too

Fixes: 1ca0323e7c29 ("Require Python 3 and remove support for Python 2.")
Signed-off-by: Timothy Redaelli <tredaelli@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-22 15:22:01 +02:00
Adrian Moreno
73ba04fd77 python: Add unit tests for filtering engine.
Add unit test for OFFilter class.

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 20:14:24 +02:00
Adrian Moreno
686bb5e729 python: Add unit tests to datapath parsing.
Add unit tests to datapath flow parsing.

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 20:14:24 +02:00
Adrian Moreno
7448cbb4ee python: Add unit tests for openflow parsing.
Add unit tests for OFPFlow class and ip-port range decoder

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 20:14:24 +02:00
Adrian Moreno
3425d01a9c python: Add unit tests for ListParser.
Add unit tests for ListParser class.

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 20:14:24 +02:00
Adrian Moreno
445dceb884 python: Introduce unit tests.
Use pytest to run unit tests as part of the standard testsuite.

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 20:14:24 +02:00
Adrian Moreno
6a71bc09bb python: Add a json encoder to flow fields.
The json encoder can be used to convert Flows to json.

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 20:14:24 +02:00
Adrian Moreno
7e588e82f0 python: Add flow filtering syntax.
Based on pyparsing, create a very simple filtering syntax.

It supports basic logic statements (and, &, or, ||, not, !), numerical
operations (<, >), equality (=, !=), and masking (~=). The latter is only
supported in certain fields (IntMask, EthMask, IPMask).

Masking operation is semantically equivalent to "includes",
therefore:

    ip_src ~= 192.168.1.1

means that ip_src field is either a host IP address equal to 192.168.1.1
or an IPMask that includes it (e.g: 192.168.1.1/24).

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 20:14:24 +02:00
Adrian Moreno
076663b31e python: Add ovs datapath flow parsing.
A ODPFlow is a Flow with the following sections:
ufid
info (e.g: bytes, packets, dp, etc)
match
actions

Only three datapath actions require special handling:
gre: because it has double parenthesis
geneve: because it supports many concatenated lists of options
nat: we reuse the decoder used for openflow actions

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 20:14:24 +02:00
Adrian Moreno
3923b9331d python: Introduce OpenFlow Flow parsing.
Introduce OFPFlow class and all its decoders.

Most of the decoders are generic (from decoders.py). Some have special
syntax and need a specific implementation.

Decoders for nat are moved to the common decoders.py because it's syntax
is shared with other types of flows (e.g: dpif flows).

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 20:14:21 +02:00
Adrian Moreno
1215cf1334 python: Add flow base class.
It simplifies the implementation of different types of flows by creating
the concept of Section (e.g: match, action) and automatic accessors for
all the provided Sections

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 20:13:48 +02:00
Adrian Moreno
61e040fc23 build-aux: Generate ofp field decoders.
Based on meta-field information extracted by extract_ofp_fields,
autogenerate the right decoder to be used.

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 20:13:38 +02:00
Adrian Moreno
d542f0ea85 build-aux: Split extract-ofp-fields.
In order to be able to reuse the core extraction logic, split the command
in two parts. The core extraction logic is moved to python/build while
the command that writes the different files out of the extracted field
info is kept in build-aux.

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 17:40:55 +02:00
Adrian Moreno
7803743a0e python: Add list parser.
Some openflow or dpif flows encode their arguments in lists, eg:
"some_action(arg1,arg2,arg3)". In order to decode this in a way that can
be then stored and queried, add ListParser and ListDecoders classes
that parse lists into KeyValue instances.

The ListParser / ListDecoders mechanism is quite similar to KVParser and
KVDecoders. Since the "key" of the different KeyValue objects is now
ommited, it has to be provided by ListDecoders.

For example, take the openflow action "resubmit" that can be written as:

    resubmit([port],[table][,ct])

Can be decoded by creating a ListDecoders instance such as:

    ListDecoders([
                  ("port", decode_default),
                  ("table", decode_int),
                  ("ct", decode_flag),
                ])

Naturally, the order of the decoders must be kept.

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 17:40:50 +02:00
Adrian Moreno
dcd17a896c python: Add mask, ip and eth decoders.
Add more decoders that can be used by KVParser.

For IPv4 and IPv6 addresses, create a new class that wraps
netaddr.IPAddress.
For Ethernet addresses, create a new class that wraps netaddr.EUI.
For Integers, create a new class that performs basic bitwise mask
comparisons

netaddr is added as a new shoft dependency:
- extras_require in setup.py
- Suggests in deb and rpm packages

Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 17:40:46 +02:00
Adrian Moreno
12bc968e26 python: Add generic Key-Value parser.
Most of ofproto and dpif flows are based on key-value pairs. These
key-value pairs can be represented in several ways, eg: key:value,
key=value, key(value).

Add the following classes that allow parsing of key-value strings:
* KeyValue: holds a key-value pair
* KeyMetadata: holds some metadata associated with a KeyValue such as
  the original key and value strings and their position in the global
  string
* KVParser: is able to parse a string and extract it's key-value pairs
  as KeyValue instances. Before creating the KeyValue instance it tries
  to decode the value via the KVDecoders
* KVDecoders holds a number of decoders that KVParser can use to decode
  key-value pairs. It accepts a dictionary of keys and callables to
  allow users to specify what decoder (i.e: callable) to use for each
  key

Also, flake8 seems to be incorrectly reporting an error (E203) in:
"slice[index + offset : index + offset]" which is PEP8 compliant. So,
ignore this error.

Acked-by: Terry Wilson <twilson@redhat.com>
Acked-by: Eelco Chaudron <echaudro@redhat.com>
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 17:40:36 +02:00
Frode Nordahl
671f93fe42 python: Allow building json C extension with static OVS library.
Allow caller of setup.py to pass in libopenvswitch.a as an object
for linking through the use of LDFLAGS environment variable when
not building a shared openvswitch library.

To accomplish this set the `enable_shared` environment variable to
'no'.

Example:
    LDFLAGS=lib/libopenvswitch.a enable_shared=no setup.py install

Signed-off-by: Frode Nordahl <frode.nordahl@canonical.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-07-15 13:45:55 +02:00
Dumitru Ceara
6835d4b01e python: Add Python bindings TODO file.
For now include the IDL related TODO items as discussed at:
https://mail.openvswitch.org/pipermail/ovs-dev/2022-April/393516.html

Signed-off-by: Dumitru Ceara <dceara@redhat.com>
Acked-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-06-28 13:47:34 +02:00
Terry Wilson
7d35554425 python: idl: Raise AttributeError from uuid_to_row.
Prior to 4e3966e64, when calling _uuid_to_row, it would raise an
AttributeError when trying to access base.ref_table.rows if the
referenced table was not registered. When called from
Row.__getattr__(), this would appropriately raise an AttributeError.

After 4e3966e64, a KeyError would be raised, which is not expected
from a getattr() or hasattr() call, which could break existing
code.

Fixes: 4e3966e64bed ("python: Politely handle misuse of table.condition.")
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-05-02 19:17:53 +02:00
Terry Wilson
4e3966e64b python: Politely handle misuse of table.condition.
Before 46d44cf3b, it was technically possible to assign a monitor
condition directly to Idl.tables[table_name].condition. If done
before the connection was established, it would successfully apply
the condition (where cond_change() actually would fail).

Although this wasn't meant to be supported, several OpenStack
projects made use of this. After 46d44cf3b, .condition is no
longer a list, but a ConditionState. Assigning a list to it breaks
the Idl.

The Neutron and ovsdbapp projects have patches in-flight to
use Idl.cond_change() if ConditionState exists, as it now works
before connection as well, but here could be other users that also
start failing when upgrading to OVS 2.17.

Instead of directly adding attributes to TableSchema, this adds
the IdlTable/IdlColumn objects which hold Idl-specific data and
adds a 'condition' property to TableSchema that maintains the old
interface.

Fixes: 46d44cf3be0d ("python: idl: Add monitor_cond_since support.")
Signed-off-by: Terry Wilson <twilson@redhat.com>
Acked-by: Dumitru Ceara <dceara@redhat.com>
Acked-By: Timothy Redaelli <tredaelli@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-04-27 00:36:28 +02:00
Wentao Jia
d29491eeb4 python: idl: Set cond_changed to true if condition change requested.
cond_changed set to true if _req_cond (requested condition change)
is not none.  This can avoid falling into an endless poll loop,
because cond_changed is true will trigger immediate_wake().

Fixes: 46d44cf3be0d ("python: idl: Add monitor_cond_since support.")
Signed-off-by: Wentao Jia <wentao.jia@easystack.cn>
Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-04-04 22:52:12 +02:00
Wentao Jia
e3de0bd82d python: idl: Set cond_changed to false if last id is zero.
After reconnection, cond_changed will be set to true, poll will be
called and never block causing cpu high load forever.

Fixes: 46d44cf3be0d ("python: idl: Add monitor_cond_since support.")
Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Wentao Jia <wentao.jia@easystack.cn>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-04-04 21:39:54 +02:00
Ilya Maximets
6de8868d19 reconnect: Fix broken inactivity probe if there is no other reason to wake up.
The purpose of reconnect_deadline__() function is twofold:

1. Its result is used to tell if the state has to be changed right now
   in reconnect_run().
2. Its result also used to determine when the process need to wake up
   and call reconnect_run() for a next time, i.e. when the state may
   need to be changed next time.

Since introduction of the 'receive-attempted' feature, the function
returns LLONG_MAX if the deadline is in the future.  That works for
the first case, but doesn't for the second one, because we don't
really know when we need to call reconnect_run().

This is the problem for applications where jsonrpc connection is the
only source of wake ups, e.g. ovn-northd.  When the network goes down
silently, e.g. server looses IP address due to DHCP failure, ovn-northd
will sleep in the poll loop indefinitely after being told that it
doesn't need to call reconnect_run() (deadline == LLONG_MAX).

Fixing that by actually returning the expected time if it is in the
future, so we will know when to wake up.  In order to keep the
'receive-attempted' feature, returning 'now + 1' in case where the
time has already passed, but receive wasn't attempted.  That will
trigger a fast wake up, so the application will be able to attempt the
receive even if there was no real events.  In a correctly written
application we should not fall into this case more than once in a row.
'+ 1' ensures that we will not transition into a different state
prematurely, i.e. before the receive is actually attempted.

Fixes: 4241d652e465 ("jsonrpc: Avoid disconnecting prematurely due to long poll intervals.")
Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-02-24 17:04:32 +01:00
Dumitru Ceara
5202710a78 python: idl: Clear last_id on reconnect if condition changes in-flight.
When reconnecting, if there are condition changes already sent to the
server but not yet acked, reset the db's 'last-id', esentially clearing
the local cache after reconnect.

This is needed because the client cannot easily differentiate between
the following cases:
a. either the server already processed the requested monitor
   condition change but the FSM was restarted before the
   client was notified.  In this case the client should
   clear its local cache because it's out of sync with the
   monitor view on the server side.
b. OR the server hasn't processed the requested monitor
   condition change yet.

Fixes: 46d44cf3be0d ("python: idl: Add monitor_cond_since support.")
Signed-off-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-01-31 21:23:47 +01:00
Dumitru Ceara
718dc8fca7 python: idl: Resend requested but not acked conditions when reconnecting.
When reconnecting forget about in-flight monitor condition changes
if the user requested a newer condition already.

This matches the C implementation, in ovsdb_cs_db_sync_condition().

Fixes: 46d44cf3be0d ("python: idl: Add monitor_cond_since support.")
Signed-off-by: Dumitru Ceara <dceara@redhat.com>
Acked-By: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-01-31 21:23:47 +01:00
Terry Wilson
46d44cf3be python: idl: Add monitor_cond_since support.
Add support for monitor_cond_since / update3 to python-ovs to
allow more efficient reconnections when connecting to clustered
OVSDB servers.

Signed-off-by: Terry Wilson <twilson@redhat.com>
Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2022-01-06 16:45:56 +01:00
Ilya Maximets
dec4291684 ovsdb-data: Consolidate ovsdb atom and json strings.
ovsdb_atom_string and json_string are basically the same data structure
and ovsdb-server frequently needs to convert one to another.  We can
avoid that by using json_string from the beginning for all ovsdb
strings.  So, the conversion turns into simple json_clone(), i.e.
increment of a reference counter.  This change gives a moderate
performance boost in some scenarios, improves the code clarity and
may be useful for future development.

Acked-by: Mike Pattrick <mkp@redhat.com>
Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-11-30 13:34:03 +01:00
Terry Wilson
c041042c12 python: idl: Avoid pre-allocating column defaults.
Many python implementations pre-allocate space for multiple
objects in empty dicts and lists. Using a custom dict-like object
that only generates these objects when they are accessed can save
memory.

On a fairly pathological case where the DB has 1000 networks each
with 100 ports, with only 'name' fields set, this saves around
300MB of memory.

One could argue that if values are not going to change from their
defaults, then users should not be monitoring those columns, but
it's also probably good to not waste memory even if user code is
sub-optimal.

Signed-off-by: Terry Wilson <twilson@redhat.com>
Acked-by: Dumitru Ceara <dceara@redhat.com>
Acked-by: Flavio Fernandes <flavio@flaviof.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-11-30 01:24:00 +01:00
Terry Wilson
04b017e3a2 python: db: Avoid allocation of an attr dict/row+column.
Python objects normally have a dictionary named __dict__ allocated
for handling dynamically assigned attributes. Depending on
architecture and Python version, that empty dict may be between
64 and 280 bytes.

Seeing as Atom and Datum objects do not need dynamic attribute
support and there can be millions of rows in a database, avoiding
this allocation with __slots__ can save 100s of MBs of memory per
Idl process.

Signed-off-by: Terry Wilson <twilson@redhat.com>
Acked-by: Timothy Redaelli <tredaelli@redhat.com>
Tested-by: Timothy Redaelli <tredaelli@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-11-09 00:27:10 +01:00
Timothy Redaelli
68543dd523 python: Replace pyOpenSSL with ssl.
Currently, pyOpenSSL is half-deprecated upstream and so it's removed on
some distributions (for example on CentOS Stream 9,
https://issues.redhat.com/browse/CS-336), but since OVS only
supports Python 3 it's possible to replace pyOpenSSL with "import ssl"
included in base Python 3.

Stream recv and send had to be splitted as _recv and _send, since SSLError
is a subclass of socket.error and so it was not possible to except for
SSLWantReadError and SSLWantWriteError in recv and send of SSLStream.

TCPstream._open cannot be used in SSLStream, since Python ssl module
requires the SSL socket to be created before connecting it, so
SSLStream._open needs to create the socket, create SSL socket and then
connect the SSL socket.

Reported-by: Timothy Redaelli <tredaelli@redhat.com>
Reported-at: https://bugzilla.redhat.com/1988429
Signed-off-by: Timothy Redaelli <tredaelli@redhat.com>
Acked-by: Terry Wilson <twilson@redhat.com>
Tested-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-11-03 16:00:04 +01:00
Timothy Redaelli
3f550fa538 python: socket-util: Split inet_open_active function and use connect_ex.
In an upcoming patch, PyOpenSSL will be replaced with Python ssl module,
but in order to do an async connection with Python ssl module the ssl
socket must be created when the socket is created, but before the
socket is connected.

So, inet_open_active function is splitted in 3 parts:
- inet_create_socket_active: creates the socket and returns the family and
  the socket, or (error, None) if some error needs to be returned.
- inet_connect_active: connect the socket and returns the errno (it
  returns 0 if errno is EINPROGRESS or EWOULDBLOCK).

connect is replaced by connect_ex, since Python suggest to use it for
asynchronous connects and it's also cleaner since inet_connect_active
returns errno that connect_ex already returns, moreover due to a Python
limitation connect cannot not be used with ssl module.

inet_open_active function is changed in order to use the new functions
inet_create_socket_active and inet_connect_active.

Signed-off-by: Timothy Redaelli <tredaelli@redhat.com>
Acked-by: Terry Wilson <twilson@redhat.com>
Tested-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-11-03 15:24:44 +01:00
Terry Wilson
34fbdc4108 python: idl: Avoid sending transactions when the DB is not synced up.
This ports the C IDL change f50714b to the Python IDL:

Until now the code here would happily try to send transactions to the
database server even if the database connection was not in the correct
state.  In some cases this could lead to strange behavior, such as sending
a database transaction for a database that the IDL had just learned did not
exist on the server.

Signed-off-by: Terry Wilson <twilson@redhat.com>
Acked-by: Dumitru Ceara <dceara@redhat.com>
Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
2021-10-12 17:49:11 +02:00