Add to IDL API that allows the user to add and remove clauses on a table's condition
iteratively. IDL maintain tables condition and send monitor_cond_change to the server
upon condition change.
Add tests for conditional monitoring to IDL.
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Python idl works now with "monitor_cond" method. Add test
for backward compatibility with old "monitor" method.
Signed-off-by: Liran Schour <lirans@il.ibm.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Disable the syslog logger in case on Windows, '/dev/log' doesn't exist.
Seems like on Python34 a default handler is added to the logger and it prints
even if no handler is set by us.
Signed-off-by: Paul-Daniel Boca <pboca@cloudbasesolutions.com>
Acked-by: Daniele Di Proietto <diproiettod@vmware.com>
The pure Python in-tree JSON parser is *much* slower than the
in-tree C JSON parser. A local test parsing a 100Mb JSON file
showed the Python version taking 270 seconds. With the C wrapper,
it took under 4 seconds.
The C extension will be used automatically if it can be built. If
the extension fails to build, a warning is displayed and the build
is restarted without the extension.
The Serializer class is replaced with Python's built-in
JSON library since the ability to process chunked data is not
needed in that case.
The extension should work with both Python 2.7 and Python 3.3+.
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
The / operation in Python 2 is "floor division" for int/long types
while in Python 3 is "true division". This means that the
significand can become a float with the existing code in Python 3.
This, in turn, can result in a parse of something like [1.10e1]
returning 11 in Python 2 and 11.0 in Python 3. Switching to the
// operator resolves this difference.
The JSON tests do not catch this difference because the built-in
serializer prints floats with the %.15g format which will convert
floats with no fractional part to an integer representation.
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Instead of checking the raw version, use the six.PY2 and six.PY3 helpers
to determine if Python 2 or Python 3 are in use.
In one case, the check was to determine if the Python version was >=
2.6. We now only support >= 2.7, so this check would always be true.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Acked-by: Miguel Angel Ajo <majopela@redhat.com>
An immutable weak reference is a hole in the constraint system: if
referenced rows are deleted, then the weak reference needs to change.
Therefore, force columsn that contain weak references to be mutable.
Reported-by: "Elluru, Krishna Mohan" <elluru.kri.mohan@hpe.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
Acked-by: Ryan Moats <rmoats@us.ibm.com>
Also update the Python ovs package info to note that both Python 2 and 3
are supported.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Python 3 has separate types for strings and bytes. Python 2 used the
same type for both. We need to convert strings to bytes before writing
them out to a socket. We also need to convert data read from the socket
to a string.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Python 3 no longer supports __cmp__. Instead, we have to implement the
"rich comparison" operators. We implement __eq__ and __lt__ and use
functools.total_ordering to implement the rest.
In one case, no __cmp__ method was provided and instead relied on the
default behavior provided in Python 2. We have to implement the
comparisons explicitly for Python 3.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Comparing None to an integer worked in Python 2, but fails in Python 3.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
I've hit several bugs in this Python 3 work where the fix was some code
needed to be converted to use isinstance(). This has been primarily
around deadling with the changes to unicode handling. Go ahead and
convert the rest of the direct type comparisons to use isinstance(), as
it could avoid a bug I haven't hit yet and it's more Pythonic, anyway.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
six.unichr() is equivalent to unichr() in Python 2
and chr() in Python 3.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
sys.maxint does not exist in Python 3, as an int does not have a max
value anymore (except as limited by implementation details and system
resources).
sys.maxsize works as a reasonable substitute as it's the same as
sys.maxint. The Python 3.0 release notes have this to say:
The sys.maxint constant was removed, since there is no longer a limit
to the value of integers. However, sys.maxsize can be used as an
integer larger than any practical list or string index. It conforms to
the implementation’s “natural” integer size and is typically the same
as sys.maxint in previous releases on the same platform (assuming the
same build options).
sys.maxsize is documented as:
An integer giving the maximum value a variable of type Py_ssize_t can
take. It’s usually 2**31 - 1 on a 32-bit platform and 2**63 - 1 on a
64-bit platform.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
StringIO.StringIO in Python 2 became io.StringIO in Python 3. Use
six.StringIO which is an alias for the two cases.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
This code asserted that the callback argument was of type
types.FunctionType. It's more pythonic to just check that the argument
is callable, and not specifically that it's a function. There are other
ways to implement a callback than types.FunctionType.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
types.StringTypes does not exist in Python 3. We can use
six.string_types, instead.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Python 2 had str and unicode. Python 3 only has str, which is always a
unicode string. Drop use of unicode with the help of six.text_type
(unicode in py2 and str in py3) and six.string_types ([str, unicode] in
py2 and [str] in py3).
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Python 2 has both long and int types. Python 3 only has int, which
behaves like long.
In the case of needing a set of integer types, we can use
six.integer_types which includes int and long for Python 2 and just int
for Python 3.
We can convert all cases of long(value) to int(value), because as of
Python 2.4, when the result of an operation would be too big for an int,
the type is automatically converted to a long.
There were several places in this patch doing type comparisons. The
preferred way to do this is using the isinstance() or issubclass()
built-in functions, so I converted the similar checks nearby while I was
at it.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
In Python 2, dict.items(), dict.keys(), and dict.values() returned a
list. dict.iteritems(), dict.iterkeys(), and dict.itervalues() returned
an iterator.
As of Python 3, dict.iteritems(), dict.itervalues(), and dict.iterkeys()
are gone. items(), keys(), and values() now return an iterator.
In the case where we want an iterator, we now use the six.iter*()
helpers. If we want a list, we explicitly create a list from the
iterator.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Python 2 had range() and xrange(). xrange() is more efficient, but
behaves differently so range() was retained for compatibility. Python 3
only has range() and it behaves like Python 2's xrange().
Remove explicit use of xrange() and use six.moves.range() to
make sure we're using xrange() from Python 2 or range() from Python 3.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Fix imports of xmlrpclib to be compatible with Python 3. Python 2 had
xmlrpclib (client) and SimpleXMLRPCServer (server). In Python 3, these
have been renamed to xmlrpc.client and xmlrpc.server.
The solution implemented here is to use the six library. It may seem
excessive for this particular issue, but the six library provides
helpers for Python 2 and 3 compatibility for many different issues.
This is just the first of many uses of the six library.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
The print statement from Python 2 is a function in Python 3. Enable
print function support for Python 2 and convert print statements to
function calls.
Enable the H233 flake8 warning. If the hacking plugin is installed,
this will generate warnings for print statement usage not compatible
with Python 3.
H233 Python 3.x incompatible use of print operator
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Python 3 removed support for tuple parameter unpacking.
https://www.python.org/dev/peps/pep-3113/
Instead of:
def foo((a, b)):
print(a)
print(b)
you should do:
def foo(a_b):
a, b = a_b
print(a)
print(b)
but in both uses here, the values were never used so the fix is even
simpler.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
This patch fixes just the Python 3 problems found by running:
python3 setup.py install
There are still many other issues to be fixed, but this is a start.
Signed-off-by: Terry Wilson <twilson@redhat.com>
[russell@ovn.org resolved conflicts with current master]
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
This patch resolves the following warnings from flake8:
E111 indentation is not a multiple of four
E112 expected an indented block
E113 unexpected indentation
It's critical to have correct indentation in Python code, so it seemed
worth enabling these warnings.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
class Vlog now inherits from "object". This is a "new style" Python
class, which isn't new at all at this point. This was introduced back
in Python 2.2, and some Python 2 code won't work as expected without it.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Fix the following pep8 errors:
E201 whitespace after '('
E203 whitespace before ','
E222 multiple spaces after operator
E225 missing whitespace around operator
E226 missing whitespace around arithmetic operator
E231 missing whitespace after ':'
E241 multiple spaces after ':'
E251 unexpected spaces around keyword / parameter equals
E261 at least two spaces before inline comment
E262 inline comment should start with '# '
E265 block comment should start with '# '
E271 multiple spaces after keyword
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Resolve pep8 errors:
E711 comparison to None should be 'if cond is None:'
The reason comparing against None with "is None" is preferred over
"== None" is because a class can define its own equality operator and
produce bizarre unexpected behavior. Using "is None" has a very
explicit meaning that can not be overridden.
E721 do not compare types, use 'isinstance()'
This one is actually a mistake by the tool in most cases.
'from ovs.db import types' looks just like types from the Python stdlib.
In those cases, use the full ovs.db.types name. Fix one case where it
actually was types from the stdlib.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Resolve pep8 errors E302 and E303:
E302 expected 2 blank lines, found 1
E303 too many blank lines (3)
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
This code referred to "rows" where it meant to refer to "fetched_rows".
The patch resolves flake8 error:
F821 undefined name 'rows'
python/build/nroff.py used a function fatal() that was not defined,
which raised the same type of error.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
This resolves the following flake8 error types:
F841 local variable 'e' is assigned to but never used
F401 'exceptions' imported but unused
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
If flake8 is installed, run it at build time. Similar to most Makefile
targets, run it once and then only run again if the files change.
flake8 is set to ignore all error and warning types that currently occur.
Future patches will remove items from the ignore list as they are
resolved.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
There is currently no mechanism in IDL to fetch specific column values
on-demand without having to register them for monitoring. In the case
where the column represent a frequently changing entity (e.g. counter),
and the reads are relatively infrequent (e.g. CLI client), there is a
significant overhead in replication.
This patch adds support in the Python IDL to register a subset of the
columns of a table as "readonly". Readonly columns are not replicated.
Users may "fetch" the readonly columns of a row on-demand. Once fetched,
the columns are not updated until the next fetch by the user. Writes by
the user to readonly columns does not change the value (both locally or
on the server).
The two main user visible changes in this patch are:
- The SchemaHelper.register_columns() method now takes an optionaly
argument to specify the subset of readonly column(s)
- A new Row.fetch(columns) method to fetch values of readonly columns(s)
Usage:
------
# Schema file includes all columns, including readonly
schema_helper = ovs.db.idl.SchemaHelper(schema_file)
# Register interest in columns with 'r' and 's' as readonly
schema_helper.register_columns("simple", [i, r, s], [r, s])
# Create Idl and jsonrpc, and wait for update, as usual
...
# Fetch value of column 'r' for a specific row
row.fetch('r')
txn.commit_block()
print row.r
print getattr(row, 'r')
# Writing to readonly column has no effect (locally or on server)
row.r = 3
print row.r # prints fetched value not 3
Signed-off-by: Shad Ansari <shad.ansari@hp.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
This is useful so that one can write, e.g.
<p>The following shows how to add 1 to variable <var>x</var>:</p>
<pre>
<var>x</var> = <var>x</var> + 1;
</pre>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Justin Pettit <jpettit@nicira.com>
This will be used in documentation for an upcoming change, to document
how Geneve OVN options are encoded.
The code in this change is from a series (not yet submitted) that makes
much more extensive use of it for documenting protocol headers.
Signed-off-by: Ben Pfaff <blp@nicira.com>
The recommended Google Python style is multi_word_names, not
multiWordNames.
There are lots of other places where the style could be improved.
I started here because I was working in this code anyway and because
this code is only used at build time and not installed, so that it
can't break any third-party code.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Justin Pettit <jpettit@nicira.com>
When opening a JSONRPC connection, the health probes
are incorrectly getting turned off for connections
that need probes.
In other words, when stream_or_pstream_needs_probes()
return non-zero, the probes are gettting disabled as
the probe interval is getting set to zero. This leads
to incorrect behavior such that probes are:
- not getting turned off for unix: connections
- getting turned off for tcp:/ssl: connections
The changes in this commit fix this issue.
Signed-off-by: Sumit Garg <sumit@extremenetworks.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
When 'alert' was turned off on a column, the code was erroring out when
value for that column was being set in a newly inserted row. This is
because the row._data was None at this time.
It seems that new rows are not initialized to defaults and that's why the
NULL error happens. IMO a newly inserted row should automatically get
intialized to default values. This new behavior can be implemented as a
separate improvement sometime in the future.
For now, I don't see an issue with adding the additional check. This new
check can continue as-is even after the new behavior is implemented.
Signed-off-by: Sumit Garg <sumit@extremenetworks.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
A bool (has_lock) was being accessed as a function call leading to a
runtime exception.
Signed-off-by: Sumit Garg <sumit@extremenetworks.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
A long time ago, the Open vSwitch build did not depend on Python (whereas
the runtime did), so the "make dist" based distribution included the
results of Python build tools. Later, the build began using Python,
but the distribution still included some of those results, because no one
had gone to the trouble of changing them. This commit changes the
Makefiles not to distribute Python-generated files but instead to just
generate them at build time.
Signed-off-by: Ben Pfaff <blp@nicira.com>
