This patch is adding a new parameter called "probe_interval" to the
constructor of the Idl class. This new parameter will be used to tune
the database connection probing for that IDL session, some users might
want to tune it to be less agressive than the current 5s default in OVS
or even disable it.
Reported-at: https://bugs.launchpad.net/networking-ovn/+bug/1680146
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
Acked-by: Daniel Alvarez <dalvarez@redhat.com>
Signed-off-by: Russell Bryant <russell@ovn.org>
https://review.openstack.org/#/c/432906/
flake8-import-order adds 3 new flake8 warnings:
I100: Your import statements are in the wrong order.
I101: The names in your from import are in the wrong order.
I201: Missing newline between sections or imports.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Since __txn_process_reply always returns None, the existing code
will always hit the final else for replies and log a debug message
about receiving an unexpected reply. In the C version,
ovsdb_idl_txn_process_reply returns true any time the txn is found,
so that behavior is duplicated here.
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Russell Bryant <russell@ovn.org>
Wait for clients to read from the pipe before disconnecting the server.
Signed-off-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
There is still plenty of opportunity for improvement, but this new
ovs-fields(7) manpage is much more comprehensive than ovs-ofctl(8)
could be.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Justin Pettit <jpettit@ovn.org>
This makes diagrams in ASCII output look about as good as one might
reasonably expect, so that in ovn-architecture(7), for example, this:
. 9 bits: reserved (0)
. 15 bits: ingress port
. 16 bits: egress port
. 24 bits: datapath
now gets formatted as:
9 15 16 24
+--------+------------+-----------+--------+
|reserved|ingress port|egress port|datapath|
+--------+------------+-----------+--------+
0
which isn't perfect but certainly more evocative than a bulleted list.
This will be more useful in upcoming commits that start using diagrams more
frequently.
Signed-off-by: Ben Pfaff <blp@ovn.org>
When OVSDB server is aborted,
the SSL send function will throw SSL.SysCallError exception,
which we need to catch and return it's -errno.
While SSL.WantWriteError exception needs to return -EAGAIN
based on its parent class, not EAGAIN
Signed-off-by: Guoshuai Li <ligs@dtdream.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
When XML is used for writing manpages, in the case that there is a
header tag followed by <dl>, the nroff python utility indents the <dl>
tag (and children) an extra level which is unnecessary and makes the
formatting inconsistent between manpages written directly in nroff vs
manpages written in XML and converted to nroff. Fix the indentation by
removing the extraneous .RS / .RE tags added to generated nroff in this
case.
This fixes the formatting of ovn/utilities/ovn-nbctl.8 man page.
Signed-off-by: Joe Stringer <joe@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
stream_or_pstream_needs_probes always return 0. This causes TCP/SSL
connection not be probed, and no reconnect when the connection
is aborted
Signed-off-by: Guoshuai Li <ligs@dtdream.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Instead of using os.fork (not supported on Windows),
subprocess.Popen is used and os.pipe was replaced
with Windows pipes.
To be able to identify the child process, an extra
parameter was added to daemon process '--pipe-handle'.
This parameter contains the parent Windows pipe handle
which is used by the child to notify the parent about
the startup.
The PID file is created directly on Windows, without
using a temporary file because the symbolic link does
not inherit the file lok set on the temporary file.
Signed-off-by: Paul-Daniel Boca <pboca@cloudbasesolutions.com>
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Acked-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions>
Tested-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
Unix sockets (AF_UNIX) are not supported on Windows.
The replacement of Unix sockets on Windows is implemented
using named pipes, we are trying to mimic the behaviour
of unix sockets.
Instead of using Unix sockets to communicate
between components Named Pipes are used. This
makes the python sockets compatible with the
Named Pipe used in Windows applications.
Signed-off-by: Paul-Daniel Boca <pboca@cloudbasesolutions.com>
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Acked-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions>
Tested-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
This patch adds a new python module which contains
helper functions. These will be neccessary for the
Windows implementation.
They cover the following aspects: sockets and namedpipes.
Signed-off-by: Alin Balutoiu <abalutoiu@cloudbasesolutions.com>
Acked-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
Most users of OVSDB react to whatever is currently in their view of the
database, as opposed to keeping track of changes and reacting to those
changes individually. The interface to conditional monitoring was
different, in that it expected the client to say what to add or remove from
monitoring instead of what to monitor. This seemed reasonable at the time,
but in practice it turns out that the usual approach actually works better,
because the condition is generally a function of the data visible in the
database. This commit changes the approach.
This commit also changes the meaning of an empty condition for a table.
Previously, an empty condition meant to replicate every row. Now, an empty
condition means to replicate no rows. This is more convenient for code
that gradually constructs conditions, because it does not need special
cases for replicating nothing.
This commit also changes the internal implementation of conditions from
linked lists to arrays. I just couldn't see an advantage to using linked
lists.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Liran Schour <lirans@il.ibm.com>
The do_handshake() function throws the exception OpenSSL.SSL.SysCallError
when the peer's SSL connection is closed, And the recv() function also
throws the exception OpenSSL.SSL.SysCallError when the peer's SSL
connection is abnormally closed, This commit catches the exception and
return error's errno.
Similarly, the recv() function also throws the exception
OpenSSL.SSL.ZeroReturnError when the peer's SSL connection is closed. This
exception refers to TCP connection normal closed, return (0, "")
Signed-off-by: Guoshuai Li <ligs@dtdream.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Numan Siddique <nusiddiq@redhat.com>
pycodestyle >= 2.1.0 reports E305 otherwise, and the flake8-check step
fails.
Signed-off-by: Daniele Di Proietto <diproiettod@vmware.com>
Acked-by: Ben Pfaff <blp@ovn.org>
This commit returns the updated column value when getattr is done
after a mutate operation is performed (but before the commit).
Signed-off-by: Amitabha Biswas <azbiswas@gmail.com>
Reported-by: Richard Theis <rtheis@us.ibm.com>
Reported-at: http://openvswitch.org/pipermail/dev/2016-September/080120.html
Fixes: a59912a0ee8e ("python: Add support for partial map and set updates")
Signed-off-by: Russell Bryant <russell@ovn.org>
SSL support is added to the ovs/stream.py. pyOpenSSL library is used
to support SSL. If this library is not present, then the SSL stream
is not registered with the Stream class.
Signed-off-by: Numan Siddique <nusiddiq@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
In a call like "ovsrec_bridge_update_ports_delvalue(bridge, port)", there's
no reason for the port argument to be nonconst, because the call doesn't
do anything to the port at all--it only searches the list of ports in the
bridge for that particular port and, if it finds it, removes it.
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Russell Bryant <russell@ovn.org>
This patch fixes the scenario, where the mutate operation on a row
is sent in the same transaction as row insert operation. It was
obvserved that this mutate operation was not getting committed
to the OVSDB.
To get around the above problem the "where" condition in an
mutate operation is modified to use the named-uuid to identify
a row created in the current transaction.
Signed-off-by: Amitabha Biswas <abiswas@us.ibm.com>
Suggested-by: Richard Theis <rtheis@us.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Use struct uuid * on [add|remove]_clause on columns which are references to
tables. That prevents use-after-free errors.
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
This patch fixes a couple of bugs in commit a59912a0
(python: add support for partial map and partial set updates)
and reverses a simplication added in commit 884d9bad
(Simplify partial map Py3 IDL test) to make the Python3 test
cases passes.
The following changes have been made:
1. Allow multiple map updates on the same column in a transaction.
2. Partial map Py3 IDL test can now support multiple elements.
3. SetAttr overrides pre-existing insert and remove updates.
4. addvalue/delvalue contains unique elements
Signed-off-by: Amitabha Biswas <abiswas@us.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Allow the python IDL to use mutate operations more freely
by mimicing the partial map and partial set operations now
available in the C IDL.
Unit tests for both of these types of operations are included.
They are not carbon copies of the C tests, because testing
idempotency is a bit difficult for the current python IDL
test harness.
Signed-off-by: Ryan Moats <rmoats@us.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
On windows a path containint ':' is considered an absolute path.
Signed-off-by: Paul-Daniel Boca <pboca@cloudbasesolutions.com>
Acked-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
On Windows if this exception is triggered then it will raise an exception while in the
exception handler.
Signed-off-by: Paul-Daniel Boca <pboca@cloudbasesolutions.com>
Acked-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
This is needed for lockf function used to lock the PID file on Windows.
ioctl and fcntl functions are not implemented at this time because they are
not used by any script.
Signed-off-by: Paul-Daniel Boca <pboca@cloudbasesolutions.com>
Acked-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
On Windows the CTL filename doesn't contain the pid of the process.
Signed-off-by: Paul-Daniel Boca <pboca@cloudbasesolutions.com>
Acked-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
SIGHUP and SIGALRM are not available on Windows.
Signed-off-by: Paul-Daniel Boca <pboca@cloudbasesolutions.com>
Acked-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
signal.alarm is not available in Windows and would trigger an exception
when called. Implemented this to mentain compatibility between
Windows and Linux for python tests.
Signed-off-by: Paul-Daniel Boca <pboca@cloudbasesolutions.com>
Acked-by: Alin Gabriel Serdean <aserdean@cloudbasesolutions.com>
Signed-off-by: Gurucharan Shetty <guru@ovn.org>
When python IDL calls the "notify" function after processing the "update2"
message from ovsdb-server, it is suppose to send the old values of the
updated columns as the last parameter. But the recent commit "897c8064"
sends the updated values. This breaks the behaviour.
This patch fixes this issue. It also updates the description of
the 'updates' param of the notify function to make it more clear.
Fixes: 897c8064 ("python: move Python idl to work with monitor_cond")
Signed-off-by: Numan Siddique <nusiddiq@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
There is no particularly good reason to use our own Python JSON
serialization implementation when serialization can be done faster
with Python's built-in JSON library.
A few tests were changed due to Python's default JSON library
returning slightly more precise floating point numbers.
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
ovs.db.idl.Datum.from_python fails to handle set type value, while set
type is also a common iterable sequence, just like list and tuple.
No reason IDL caller must to turn set type parameters to list or tuple
type. Otherwise, they will fail to insert data, but get no exception.
Reported-at: https://bugs.launchpad.net/networking-ovn/+bug/1605573
Signed-off-by: Zong Kai LI <zealokii@gmail.com>
Acked-by: Richard Theis <rtheis@us.ibm.com>
Tested-by: Richard Theis <rtheis@us.ibm.com>
Signed-off-by: Russell Bryant <russell@ovn.org>
To easily allow both in- and out-of-tree building of the Python
wrapper for the OVS JSON parser (e.g. w/ pip), move json.h to
include/openvswitch. This also requires moving lib/{hmap,shash}.h.
Both hmap.h and shash.h were #include-ing "util.h" even though the
headers themselves did not use anything from there, but rather from
include/openvswitch/util.h. Fixing that required including util.h
in several C files mostly due to OVS_NOT_REACHED and things like
xmalloc.
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Add to IDL API that allows the user to add and remove clauses on a table's condition
iteratively. IDL maintain tables condition and send monitor_cond_change to the server
upon condition change.
Add tests for conditional monitoring to IDL.
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Python idl works now with "monitor_cond" method. Add test
for backward compatibility with old "monitor" method.
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Disable the syslog logger in case on Windows, '/dev/log' doesn't exist.
Seems like on Python34 a default handler is added to the logger and it prints
even if no handler is set by us.
Signed-off-by: Paul-Daniel Boca <pboca@cloudbasesolutions.com>
Acked-by: Daniele Di Proietto <diproiettod@vmware.com>
The pure Python in-tree JSON parser is *much* slower than the
in-tree C JSON parser. A local test parsing a 100Mb JSON file
showed the Python version taking 270 seconds. With the C wrapper,
it took under 4 seconds.
The C extension will be used automatically if it can be built. If
the extension fails to build, a warning is displayed and the build
is restarted without the extension.
The Serializer class is replaced with Python's built-in
JSON library since the ability to process chunked data is not
needed in that case.
The extension should work with both Python 2.7 and Python 3.3+.
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
The / operation in Python 2 is "floor division" for int/long types
while in Python 3 is "true division". This means that the
significand can become a float with the existing code in Python 3.
This, in turn, can result in a parse of something like [1.10e1]
returning 11 in Python 2 and 11.0 in Python 3. Switching to the
// operator resolves this difference.
The JSON tests do not catch this difference because the built-in
serializer prints floats with the %.15g format which will convert
floats with no fractional part to an integer representation.
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Instead of checking the raw version, use the six.PY2 and six.PY3 helpers
to determine if Python 2 or Python 3 are in use.
In one case, the check was to determine if the Python version was >=
2.6. We now only support >= 2.7, so this check would always be true.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Acked-by: Miguel Angel Ajo <majopela@redhat.com>
An immutable weak reference is a hole in the constraint system: if
referenced rows are deleted, then the weak reference needs to change.
Therefore, force columsn that contain weak references to be mutable.
Reported-by: "Elluru, Krishna Mohan" <elluru.kri.mohan@hpe.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Ryan Moats <rmoats@us.ibm.com>
Also update the Python ovs package info to note that both Python 2 and 3
are supported.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Python 3 has separate types for strings and bytes. Python 2 used the
same type for both. We need to convert strings to bytes before writing
them out to a socket. We also need to convert data read from the socket
to a string.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Python 3 no longer supports __cmp__. Instead, we have to implement the
"rich comparison" operators. We implement __eq__ and __lt__ and use
functools.total_ordering to implement the rest.
In one case, no __cmp__ method was provided and instead relied on the
default behavior provided in Python 2. We have to implement the
comparisons explicitly for Python 3.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Comparing None to an integer worked in Python 2, but fails in Python 3.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
