Several OVS structs contain embedded named unions, like this:
struct {
...
union {
...
} u;
};
C11 standardized a feature that many compilers already implemented
anyway, where an embedded union may be unnamed, like this:
struct {
...
union {
...
};
};
This is more convenient because it allows the programmer to omit "u."
in many places. OVS already used this feature in several places. This
commit embraces it in several others.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Justin Pettit <jpettit@ovn.org>
Tested-by: Alin Gabriel Serdean <aserdean@ovn.org>
Acked-by: Alin Gabriel Serdean <aserdean@ovn.org>
Check if we are dealing with a Unicode string that needs
encoding for both Python 2 & 3.
Also, do the encoding the same way for Python 2 & 3 and avoid using
negation to make the code simpler.
Signed-off-by: Jakub Sitnicki <jkbs@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Encoding from 'unicode' to 'str' that has been added to the Stream class
in commit 2254074e3067 ("python: fix python3 encode/decode on Windows")
conflicts with SSLStream which already contains a quirk for pyopenssl
that does the same thing.
This results in a double encoding attempt when SSL is used and we crash
and burn due to:
Traceback (most recent call last):
File "../.././test-ovsdb.py", line 874, in <module>
main(sys.argv)
File "../.././test-ovsdb.py", line 869, in main
func(*args)
File "../.././test-ovsdb.py", line 655, in do_idl
idl_set(idl, command, step)
File "../.././test-ovsdb.py", line 526, in idl_set
status = txn.commit_block()
File "/home/jkbs/src/ovs/python/ovs/db/idl.py", line 1405, in commit_block
status = self.commit()
File "/home/jkbs/src/ovs/python/ovs/db/idl.py", line 1388, in commit
if not self.idl._session.send(msg):
File "/home/jkbs/src/ovs/python/ovs/jsonrpc.py", line 540, in send
return self.rpc.send(msg)
File "/home/jkbs/src/ovs/python/ovs/jsonrpc.py", line 244, in send
self.run()
File "/home/jkbs/src/ovs/python/ovs/jsonrpc.py", line 203, in run
retval = self.stream.send(self.output)
File "/home/jkbs/src/ovs/python/ovs/stream.py", line 808, in send
return super(SSLStream, self).send(buf)
File "/home/jkbs/src/ovs/python/ovs/stream.py", line 391, in send
buf = buf.encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 83: ordinal not in range(128)
Remove the quirk from SSLStream as the base class now does encoding.
Reported-by: Marcin Mirecki <mmirecki@redhat.com>
Signed-off-by: Jakub Sitnicki <jkbs@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
This adds multi-column index support for the Python IDL that is
similar to the feature in the C IDL. Since it adds sortedcontainers
as a dependency and some distros don't yet package it, the library
is copied in-tree and used if sortedcontainers is not installed.
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
This is aimed at an upcoming database clustering implementation, where it's
desirable to try all of the cluster members quickly before backing off to
retry them again in sequence.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Russell Bryant <russell@ovn.org>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Justin Pettit <jpettit@ovn.org>
Long ago, a <monitor-requests> object in the OVSDB protocol mapped a table
name to a single <monitor-request>. Since then, it has mapped a table name
to an *array of* <monitor-request> objects, but the OVSDB IDL has never
been updated to use the modern form. This commit makes that change.
Reported-by: Anil Jangam <anilj.mailing@gmail.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Justin Pettit <jpettit@ovn.org>
On Python 3 hasattr only intercepts AttributeError exception.
On Python2, instead, hasattr intercepts all the exceptions.
This means __getattr__ shouldn't return KeyError when the attribute
doesn't exists, but it should raise AttributeError instead.
Fixes: 2d54d8011e14 ("Python-IDL: getattr after mutate fix")
Signed-off-by: Timothy Redaelli <tredaelli@redhat.com>
Signed-off-by: Russell Bryant <russell@ovn.org>
It can only receive 4096 bytes of data each time in jsonrpc,
when there are similar and Chinese characters occupy multiple bytes,
it may receive half a character, this time the decoding will be abnormal.
We need to receive the completed character to decode.
Signed-off-by: Guoshuai Li <ligs@dtdream.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
This patch removes a useless conversion to/from JSON in the
processing of any 'modify' operations inside the process_update2
method in Python IDL implementation.
Previous code will make resources creation take longer as the number
of elements in the row grows because of that JSON conversion. This
patch eliminates it and now the time remains consant regardless
of the database contents improving performance and scaling.
Reported-by: Daniel Alvarez <dalvarez@redhat.com>
Reported-at: https://mail.openvswitch.org/pipermail/ovs-discuss/2018-February/046263.html
Signed-off-by: Daniel Alvarez <dalvarez@redhat.com>
Acked-by: Terry Wilson <twilson@redhat.com>
Tested-By: Terry Wilson <twilson@redhat.com>
Acked-by: Han Zhou <hzhou8@ebay.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
An IDL schema is an OVSDB schema with some extra stuff in it. So far, all
of the extras have been at the top level. This commit makes it possible
for IDL schemas to have extra information at the table and column levels as
long as it is in an "extensions" member.
No extensions are actually supported yet.
Signed-off-by: Ben Pfaff <blp@ovn.org>
An IDL schema is an OVSDB schema with some extra stuff in it: an idlPrefix
and an idlHeader at the top level to indicate what ovsdb-idlc needs to
generate the interface definitions. This commit adds support for two more
optional IDL schema extensions that allow extra code to be written to the
.c and .h file that ovsdb-idlc generates.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Perl is unfashionable and Python is more widely available and understood,
so this commit converts one of the OVS uses of Perl into Python.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Aaron Conole <aconole@redhat.com>
Found while looking over the code.
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Suggested-by: Alin Gabriel Serdean <aserdean@ovn.org>
Acked-by: Alin Gabriel Serdean <aserdean@ovn.org>
Acked-by: Anand Kumar <kumaranand@vmware.com>
Signed-off-by: Alin Gabriel Serdean <aserdean@ovn.org>
At the moment we have WSAEventSelect in each if branch.
Since the call to the function is similar, we can move
it outside the if branch and create some local variables
which will be passed to WSAEventSelect.
This patch also remove the keyword argument passed when
the event for the connection overlapped structure is created.
The argument is not needed since it does not change the value
from the default one.
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Signed-off-by: Alin Gabriel Serdean <aserdean@ovn.org>
Acked-by: Alin Gabriel Serdean <aserdean@ovn.org>
At the moment the sockets on Windows use the same events
that are being created for the pipes.
This is not correct because they should be different events.
This patch introduces a new event which should be used for sockets.
The new event needs to be set on automatic reset with its initial
state not signaled.
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Co-authored-by: Alin Gabriel Serdean <aserdean@ovn.org>
Signed-off-by: Alin Gabriel Serdean <aserdean@ovn.org>
Tested-by: Alin Gabriel Serdean <aserdean@ovn.org>
Acked-by: Russell Bryant <russell@ovn.org>
Bump the security around named pipes to be more restrictive: disable network
access and allow only administrators and above to access the named pipes.
Acked-by: Anand Kumar <kumaranand@vmware.com>
Acked-by: Alin Gabriel Serdean <aserdean@ovn.org>
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Signed-off-by: Alin Gabriel Serdean <aserdean@ovn.org>
Move the return value at the end of the function
regardless of the pending/non-pending operation.
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Acked-by: Russell Bryant <russell@ovn.org>
Acked-by: Alin Gabriel Serdean <aserdean@ovn.org>
Signed-off-by: Alin Gabriel Serdean <aserdean@ovn.org>
The function poll from poller should return a list of tuples
containing the events and their types.
On Windows the event type is not returned at the moment.
Instead of returning zero all the time, we check to see
the type of event and we set it accordingly before returning
the list.
This is used only for debugging purposes inside the function
"__log_wakeup" later on.
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Acked-by: Russell Bryant <russell@ovn.org>
Acked-by: Alin Gabriel Serdean <aserdean@ovn.org>
Signed-off-by: Alin Gabriel Serdean <aserdean@ovn.org>
Fix double encoding/decoding on data, caused by
'get_decoded_buffer' and 'get_encoded_buffer'.
The functions 'get_decoded_buffer' and 'get_encoded_buffer'
from winutils have been removed. They are no longer
necessary since the buffers received/returned are already
in the right form.
The necessary encoding has been moved before any sending
function (this also includes named pipes send on Windows).
Co-authored-by: Alin Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Signed-off-by: Alin Serdean <aserdean@cloudbasesolutions.com>
Acked-by: Lance Richardson <lrichard@redhat.com>
Signed-off-by: Russell Bryant <russell@ovn.org>
Ensure that JSON is utf-8 encoded and that bytes sent/received on
the stream sockets are in utf-8 form. Add a test case to verify
that unicode data can be sent/received successfully using Python
IDL module.
Co-authored-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Lance Richardson <lrichard@redhat.com>
Signed-off-by: Russell Bryant <russell@ovn.org>
In python3 maxint is not defined, but maxsize is defined in both
python2 and python3.
The put_text() will not automatically use a value which came in as
float due to a pior math function and python3 will throw an exception.
The simple answer is to convert it with int() and move on.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
It's becoming more common that OSes include "ip" but not "ifconfig", so
it's best to avoid using the latter. This commit removes most references
to "ifconfig" and replaces them by "ip". It also adds a build-time check
to make it harder to introduce new uses of "ifconfig".
There are important differences between "ifconfig" and "ip":
- An "ifconfig" command that sets an IP address also brings the interface
up, but a similar "ip addr add" command does not, so it is often necessary
(or at least precautionary) to add an "ip link set <dev> up" command.
- "ifconfig" can infer a netmask from an IP adddress, but "ip" always
assumes /32 if none is given.
- "ifconfig" with address 0.0.0.0 removes any configured IP address, but
"ip addr add" does not, so "ifconfig <dev> 0.0.0.0" must be replaced by
"ip addr del" or "ip addr flush".
Signed-off-by: Ben Pfaff <blp@ovn.org>
Centos provides pyOpenSSL version pyOpenSSL-0.13.1-3.el7.x86_64.
There are 2 issues using this version, which this patch fixes
- The test case "simple idl verify notify - SSL" is skipped.
This is because "python -m OpenSSL.SSL" is used to detect the
presence of pyOpenSSL package. pyOpenSSL v0.13 has C python
modules because of which the above command returns 1.
So this patch fixes this by using 'python -c "import OpenSSL.SSL"'.
- The SSL.Context class does not have the function "set_session_cache_mode"
defined. Our usage here was only relevant for server-side connections,
(pssl), which is not yet supported by python-ovs, so just remove the
usage of this function. The default cache mode (server) will just
be ignored.
I have not tested with older versions (< 0.13) of pyOpenSSL.
Signed-off-by: Numan Siddique <nusiddiq@redhat.com>
Acked-by: Lance Richardson <lrichard@redhat.com>
Tested-by: Marcin Mirecki <mmirecki@redhat.com>
Signed-off-by: Russell Bryant <rbryant@redhat.com>
This patch is adding a new parameter called "probe_interval" to the
constructor of the Idl class. This new parameter will be used to tune
the database connection probing for that IDL session, some users might
want to tune it to be less agressive than the current 5s default in OVS
or even disable it.
Reported-at: https://bugs.launchpad.net/networking-ovn/+bug/1680146
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
Acked-by: Daniel Alvarez <dalvarez@redhat.com>
Signed-off-by: Russell Bryant <russell@ovn.org>
https://review.openstack.org/#/c/432906/
flake8-import-order adds 3 new flake8 warnings:
I100: Your import statements are in the wrong order.
I101: The names in your from import are in the wrong order.
I201: Missing newline between sections or imports.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Since __txn_process_reply always returns None, the existing code
will always hit the final else for replies and log a debug message
about receiving an unexpected reply. In the C version,
ovsdb_idl_txn_process_reply returns true any time the txn is found,
so that behavior is duplicated here.
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Russell Bryant <russell@ovn.org>
Wait for clients to read from the pipe before disconnecting the server.
Signed-off-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
There is still plenty of opportunity for improvement, but this new
ovs-fields(7) manpage is much more comprehensive than ovs-ofctl(8)
could be.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Justin Pettit <jpettit@ovn.org>
This makes diagrams in ASCII output look about as good as one might
reasonably expect, so that in ovn-architecture(7), for example, this:
. 9 bits: reserved (0)
. 15 bits: ingress port
. 16 bits: egress port
. 24 bits: datapath
now gets formatted as:
9 15 16 24
+--------+------------+-----------+--------+
|reserved|ingress port|egress port|datapath|
+--------+------------+-----------+--------+
0
which isn't perfect but certainly more evocative than a bulleted list.
This will be more useful in upcoming commits that start using diagrams more
frequently.
Signed-off-by: Ben Pfaff <blp@ovn.org>
When OVSDB server is aborted,
the SSL send function will throw SSL.SysCallError exception,
which we need to catch and return it's -errno.
While SSL.WantWriteError exception needs to return -EAGAIN
based on its parent class, not EAGAIN
Signed-off-by: Guoshuai Li <ligs@dtdream.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
When XML is used for writing manpages, in the case that there is a
header tag followed by <dl>, the nroff python utility indents the <dl>
tag (and children) an extra level which is unnecessary and makes the
formatting inconsistent between manpages written directly in nroff vs
manpages written in XML and converted to nroff. Fix the indentation by
removing the extraneous .RS / .RE tags added to generated nroff in this
case.
This fixes the formatting of ovn/utilities/ovn-nbctl.8 man page.
Signed-off-by: Joe Stringer <joe@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
stream_or_pstream_needs_probes always return 0. This causes TCP/SSL
connection not be probed, and no reconnect when the connection
is aborted
Signed-off-by: Guoshuai Li <ligs@dtdream.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Instead of using os.fork (not supported on Windows),
subprocess.Popen is used and os.pipe was replaced
with Windows pipes.
To be able to identify the child process, an extra
parameter was added to daemon process '--pipe-handle'.
This parameter contains the parent Windows pipe handle
which is used by the child to notify the parent about
the startup.
The PID file is created directly on Windows, without
using a temporary file because the symbolic link does
not inherit the file lok set on the temporary file.
Signed-off-by: Paul-Daniel Boca <pboca@cloudbasesolutions.com>
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Acked-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions>
Tested-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
Unix sockets (AF_UNIX) are not supported on Windows.
The replacement of Unix sockets on Windows is implemented
using named pipes, we are trying to mimic the behaviour
of unix sockets.
Instead of using Unix sockets to communicate
between components Named Pipes are used. This
makes the python sockets compatible with the
Named Pipe used in Windows applications.
Signed-off-by: Paul-Daniel Boca <pboca@cloudbasesolutions.com>
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Acked-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions>
Tested-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
This patch adds a new python module which contains
helper functions. These will be neccessary for the
Windows implementation.
They cover the following aspects: sockets and namedpipes.
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Acked-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
Most users of OVSDB react to whatever is currently in their view of the
database, as opposed to keeping track of changes and reacting to those
changes individually. The interface to conditional monitoring was
different, in that it expected the client to say what to add or remove from
monitoring instead of what to monitor. This seemed reasonable at the time,
but in practice it turns out that the usual approach actually works better,
because the condition is generally a function of the data visible in the
database. This commit changes the approach.
This commit also changes the meaning of an empty condition for a table.
Previously, an empty condition meant to replicate every row. Now, an empty
condition means to replicate no rows. This is more convenient for code
that gradually constructs conditions, because it does not need special
cases for replicating nothing.
This commit also changes the internal implementation of conditions from
linked lists to arrays. I just couldn't see an advantage to using linked
lists.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Liran Schour <lirans@il.ibm.com>
The do_handshake() function throws the exception OpenSSL.SSL.SysCallError
when the peer's SSL connection is closed, And the recv() function also
throws the exception OpenSSL.SSL.SysCallError when the peer's SSL
connection is abnormally closed, This commit catches the exception and
return error's errno.
Similarly, the recv() function also throws the exception
OpenSSL.SSL.ZeroReturnError when the peer's SSL connection is closed. This
exception refers to TCP connection normal closed, return (0, "")
Signed-off-by: Guoshuai Li <ligs@dtdream.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Numan Siddique <nusiddiq@redhat.com>
pycodestyle >= 2.1.0 reports E305 otherwise, and the flake8-check step
fails.
Signed-off-by: Daniele Di Proietto <diproiettod@vmware.com>
Acked-by: Ben Pfaff <blp@ovn.org>
This commit returns the updated column value when getattr is done
after a mutate operation is performed (but before the commit).
Signed-off-by: Amitabha Biswas <azbiswas@gmail.com>
Reported-by: Richard Theis <rtheis@us.ibm.com>
Reported-at: http://openvswitch.org/pipermail/dev/2016-September/080120.html
Fixes: a59912a0ee8e ("python: Add support for partial map and set updates")
Signed-off-by: Russell Bryant <russell@ovn.org>
SSL support is added to the ovs/stream.py. pyOpenSSL library is used
to support SSL. If this library is not present, then the SSL stream
is not registered with the Stream class.
Signed-off-by: Numan Siddique <nusiddiq@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
In a call like "ovsrec_bridge_update_ports_delvalue(bridge, port)", there's
no reason for the port argument to be nonconst, because the call doesn't
do anything to the port at all--it only searches the list of ports in the
bridge for that particular port and, if it finds it, removes it.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Russell Bryant <russell@ovn.org>
This patch fixes the scenario, where the mutate operation on a row
is sent in the same transaction as row insert operation. It was
obvserved that this mutate operation was not getting committed
to the OVSDB.
To get around the above problem the "where" condition in an
mutate operation is modified to use the named-uuid to identify
a row created in the current transaction.
Signed-off-by: Amitabha Biswas <abiswas@us.ibm.com>
Suggested-by: Richard Theis <rtheis@us.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Use struct uuid * on [add|remove]_clause on columns which are references to
tables. That prevents use-after-free errors.
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
This patch fixes a couple of bugs in commit a59912a0
(python: add support for partial map and partial set updates)
and reverses a simplication added in commit 884d9bad
(Simplify partial map Py3 IDL test) to make the Python3 test
cases passes.
The following changes have been made:
1. Allow multiple map updates on the same column in a transaction.
2. Partial map Py3 IDL test can now support multiple elements.
3. SetAttr overrides pre-existing insert and remove updates.
4. addvalue/delvalue contains unique elements
Signed-off-by: Amitabha Biswas <abiswas@us.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
