I've hit several bugs in this Python 3 work where the fix was some code
needed to be converted to use isinstance(). This has been primarily
around deadling with the changes to unicode handling. Go ahead and
convert the rest of the direct type comparisons to use isinstance(), as
it could avoid a bug I haven't hit yet and it's more Pythonic, anyway.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
six.unichr() is equivalent to unichr() in Python 2
and chr() in Python 3.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
sys.maxint does not exist in Python 3, as an int does not have a max
value anymore (except as limited by implementation details and system
resources).
sys.maxsize works as a reasonable substitute as it's the same as
sys.maxint. The Python 3.0 release notes have this to say:
The sys.maxint constant was removed, since there is no longer a limit
to the value of integers. However, sys.maxsize can be used as an
integer larger than any practical list or string index. It conforms to
the implementation’s “natural” integer size and is typically the same
as sys.maxint in previous releases on the same platform (assuming the
same build options).
sys.maxsize is documented as:
An integer giving the maximum value a variable of type Py_ssize_t can
take. It’s usually 2**31 - 1 on a 32-bit platform and 2**63 - 1 on a
64-bit platform.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
StringIO.StringIO in Python 2 became io.StringIO in Python 3. Use
six.StringIO which is an alias for the two cases.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
This code asserted that the callback argument was of type
types.FunctionType. It's more pythonic to just check that the argument
is callable, and not specifically that it's a function. There are other
ways to implement a callback than types.FunctionType.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
types.StringTypes does not exist in Python 3. We can use
six.string_types, instead.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Python 2 had str and unicode. Python 3 only has str, which is always a
unicode string. Drop use of unicode with the help of six.text_type
(unicode in py2 and str in py3) and six.string_types ([str, unicode] in
py2 and [str] in py3).
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Python 2 has both long and int types. Python 3 only has int, which
behaves like long.
In the case of needing a set of integer types, we can use
six.integer_types which includes int and long for Python 2 and just int
for Python 3.
We can convert all cases of long(value) to int(value), because as of
Python 2.4, when the result of an operation would be too big for an int,
the type is automatically converted to a long.
There were several places in this patch doing type comparisons. The
preferred way to do this is using the isinstance() or issubclass()
built-in functions, so I converted the similar checks nearby while I was
at it.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
In Python 2, dict.items(), dict.keys(), and dict.values() returned a
list. dict.iteritems(), dict.iterkeys(), and dict.itervalues() returned
an iterator.
As of Python 3, dict.iteritems(), dict.itervalues(), and dict.iterkeys()
are gone. items(), keys(), and values() now return an iterator.
In the case where we want an iterator, we now use the six.iter*()
helpers. If we want a list, we explicitly create a list from the
iterator.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Python 2 had range() and xrange(). xrange() is more efficient, but
behaves differently so range() was retained for compatibility. Python 3
only has range() and it behaves like Python 2's xrange().
Remove explicit use of xrange() and use six.moves.range() to
make sure we're using xrange() from Python 2 or range() from Python 3.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Fix imports of xmlrpclib to be compatible with Python 3. Python 2 had
xmlrpclib (client) and SimpleXMLRPCServer (server). In Python 3, these
have been renamed to xmlrpc.client and xmlrpc.server.
The solution implemented here is to use the six library. It may seem
excessive for this particular issue, but the six library provides
helpers for Python 2 and 3 compatibility for many different issues.
This is just the first of many uses of the six library.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
The print statement from Python 2 is a function in Python 3. Enable
print function support for Python 2 and convert print statements to
function calls.
Enable the H233 flake8 warning. If the hacking plugin is installed,
this will generate warnings for print statement usage not compatible
with Python 3.
H233 Python 3.x incompatible use of print operator
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Python 3 removed support for tuple parameter unpacking.
https://www.python.org/dev/peps/pep-3113/
Instead of:
def foo((a, b)):
print(a)
print(b)
you should do:
def foo(a_b):
a, b = a_b
print(a)
print(b)
but in both uses here, the values were never used so the fix is even
simpler.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
This patch fixes just the Python 3 problems found by running:
python3 setup.py install
There are still many other issues to be fixed, but this is a start.
Signed-off-by: Terry Wilson <twilson@redhat.com>
[russell@ovn.org resolved conflicts with current master]
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
This patch resolves the following warnings from flake8:
E111 indentation is not a multiple of four
E112 expected an indented block
E113 unexpected indentation
It's critical to have correct indentation in Python code, so it seemed
worth enabling these warnings.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
class Vlog now inherits from "object". This is a "new style" Python
class, which isn't new at all at this point. This was introduced back
in Python 2.2, and some Python 2 code won't work as expected without it.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Fix the following pep8 errors:
E201 whitespace after '('
E203 whitespace before ','
E222 multiple spaces after operator
E225 missing whitespace around operator
E226 missing whitespace around arithmetic operator
E231 missing whitespace after ':'
E241 multiple spaces after ':'
E251 unexpected spaces around keyword / parameter equals
E261 at least two spaces before inline comment
E262 inline comment should start with '# '
E265 block comment should start with '# '
E271 multiple spaces after keyword
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Resolve pep8 errors:
E711 comparison to None should be 'if cond is None:'
The reason comparing against None with "is None" is preferred over
"== None" is because a class can define its own equality operator and
produce bizarre unexpected behavior. Using "is None" has a very
explicit meaning that can not be overridden.
E721 do not compare types, use 'isinstance()'
This one is actually a mistake by the tool in most cases.
'from ovs.db import types' looks just like types from the Python stdlib.
In those cases, use the full ovs.db.types name. Fix one case where it
actually was types from the stdlib.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
Resolve pep8 errors E302 and E303:
E302 expected 2 blank lines, found 1
E303 too many blank lines (3)
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
This code referred to "rows" where it meant to refer to "fetched_rows".
The patch resolves flake8 error:
F821 undefined name 'rows'
python/build/nroff.py used a function fatal() that was not defined,
which raised the same type of error.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
This resolves the following flake8 error types:
F841 local variable 'e' is assigned to but never used
F401 'exceptions' imported but unused
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
If flake8 is installed, run it at build time. Similar to most Makefile
targets, run it once and then only run again if the files change.
flake8 is set to ignore all error and warning types that currently occur.
Future patches will remove items from the ignore list as they are
resolved.
Signed-off-by: Russell Bryant <russell@ovn.org>
Acked-by: Ben Pfaff <blp@ovn.org>
There is currently no mechanism in IDL to fetch specific column values
on-demand without having to register them for monitoring. In the case
where the column represent a frequently changing entity (e.g. counter),
and the reads are relatively infrequent (e.g. CLI client), there is a
significant overhead in replication.
This patch adds support in the Python IDL to register a subset of the
columns of a table as "readonly". Readonly columns are not replicated.
Users may "fetch" the readonly columns of a row on-demand. Once fetched,
the columns are not updated until the next fetch by the user. Writes by
the user to readonly columns does not change the value (both locally or
on the server).
The two main user visible changes in this patch are:
- The SchemaHelper.register_columns() method now takes an optionaly
argument to specify the subset of readonly column(s)
- A new Row.fetch(columns) method to fetch values of readonly columns(s)
Usage:
------
# Schema file includes all columns, including readonly
schema_helper = ovs.db.idl.SchemaHelper(schema_file)
# Register interest in columns with 'r' and 's' as readonly
schema_helper.register_columns("simple", [i, r, s], [r, s])
# Create Idl and jsonrpc, and wait for update, as usual
...
# Fetch value of column 'r' for a specific row
row.fetch('r')
txn.commit_block()
print row.r
print getattr(row, 'r')
# Writing to readonly column has no effect (locally or on server)
row.r = 3
print row.r # prints fetched value not 3
Signed-off-by: Shad Ansari <shad.ansari@hp.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>
This is useful so that one can write, e.g.
<p>The following shows how to add 1 to variable <var>x</var>:</p>
<pre>
<var>x</var> = <var>x</var> + 1;
</pre>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Justin Pettit <jpettit@nicira.com>
This will be used in documentation for an upcoming change, to document
how Geneve OVN options are encoded.
The code in this change is from a series (not yet submitted) that makes
much more extensive use of it for documenting protocol headers.
Signed-off-by: Ben Pfaff <blp@nicira.com>
The recommended Google Python style is multi_word_names, not
multiWordNames.
There are lots of other places where the style could be improved.
I started here because I was working in this code anyway and because
this code is only used at build time and not installed, so that it
can't break any third-party code.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Justin Pettit <jpettit@nicira.com>
When opening a JSONRPC connection, the health probes
are incorrectly getting turned off for connections
that need probes.
In other words, when stream_or_pstream_needs_probes()
return non-zero, the probes are gettting disabled as
the probe interval is getting set to zero. This leads
to incorrect behavior such that probes are:
- not getting turned off for unix: connections
- getting turned off for tcp:/ssl: connections
The changes in this commit fix this issue.
Signed-off-by: Sumit Garg <sumit@extremenetworks.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
When 'alert' was turned off on a column, the code was erroring out when
value for that column was being set in a newly inserted row. This is
because the row._data was None at this time.
It seems that new rows are not initialized to defaults and that's why the
NULL error happens. IMO a newly inserted row should automatically get
intialized to default values. This new behavior can be implemented as a
separate improvement sometime in the future.
For now, I don't see an issue with adding the additional check. This new
check can continue as-is even after the new behavior is implemented.
Signed-off-by: Sumit Garg <sumit@extremenetworks.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
A bool (has_lock) was being accessed as a function call leading to a
runtime exception.
Signed-off-by: Sumit Garg <sumit@extremenetworks.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
A long time ago, the Open vSwitch build did not depend on Python (whereas
the runtime did), so the "make dist" based distribution included the
results of Python build tools. Later, the build began using Python,
but the distribution still included some of those results, because no one
had gone to the trouble of changing them. This commit changes the
Makefiles not to distribute Python-generated files but instead to just
generate them at build time.
Signed-off-by: Ben Pfaff <blp@nicira.com>
This makes life easier for testing at the point you start to separate your
environment into multiple machines.
Also work on the manpage a little.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Alex Wang <alexw@nicira.com>
It is useful to make the notification events that Idl processes
accessible to users of the library. This will make it possible to
keep external systems in sync, but does not impose any particular
notification pattern.
The Row.from_json() call is added to be able to convert the 'old'
JSON response on an update to a Row object to make it easy for
users of notify() to see what changed, though this usage of Row
is quite different than Idl's typical use.
Signed-off-by: Terry Wilson <twilson@redhat.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
This adds very basic support for setuptools so that the OVS Python
lib can be added to PyPI.
This currently uses the Open vSwitch version number and the
generated dirs.py, though there is no real reason to tie the
Python libraries releases or version numbers to the main project's.
Signed-off-by: Terry Wilson <twilson@redhat.com>
Acked-by: Russell Bryant <rbryant@redhat.com>
Acked-by: Kyle Mestery <mestery@mestery.com>
[blp@nicira.com adjusted automake.mk]
Signed-off-by: Ben Pfaff <blp@nicira.com>
The set_dscp() function, until now, tried to set the DSCP as IPv4 and as
IPv6. This worked OK on Linux, where an ENOPROTOOPT error made it really
clear which one was wrong, but FreeBSD uses EINVAL instead, which has
multiple meanings and which it therefore seems somewhat risky to ignore.
Instead, this commit just tries to set the correct address family's DSCP
option.
Tested by Alex Wang on FreeBSD 9.3.
Reported-by: Atanu Ghosh <atanu@acm.org>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Co-authored-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Alex Wang <alexw@nicira.com>
Tested-by: Alex Wang <alexw@nicira.com>
commit 7905aae3fc1633c2c44c8fdb9e9d3a3d6e63439b
("vlog: Don't fail syslog initialization in chroot.")
uses os.path.isfile("/dev/log"), which tests if the given path
is a regular file, to see if syslog can be usable.
However, /dev/log is not a regular file for platforms I looked at.
* On Ubuntu 14.04 and CentOS 6.5, /dev/log is a socket
* On NetBSD-6, /dev/log is a symlink to a socket
Replace the test with os.path.exists() so that it can work
as intended for these platforms.
Signed-off-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
Acked-by: Gurucharan Shetty <gshetty@nicira.com>
When OVS unit tests are run inside chroot environment,
there is no syslog infrastructure available. In a
situation like that, don't fail or log additional messages
to syslog by increasing the severity level of syslog very high
(log messages would continue to be logged to console and file).
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Thomas Graf <tgraf@noironetworks.com>
When Open vSwitch is run in hundreds of hypervisors, it is
useful to collect log messages through log collectors. To
collect log messages like this, it is useful to log them
in a particular RFC5424 facility in the local system. The
log collectors can then be used to collect logs anytime
desired.
This commit provides a sysadmin the ability to specify the
facility through which the log messages are logged.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
In OVS, we currently use the term 'facility' to mean the place
where we log (syslog, console or file). In Linux's syslog() and
rfc5424, the term 'facility' is used to specify what type of program
is logging the message (e.g: LOG_DAEMON). This causes confusion
while reading vlog's code. This commit changes the term 'facility'
to 'destination'.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
The Open vSwitch "make" output was still pretty verbose even when
configured with --enable-silent-rules. This cleans it up.
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Joe Stringer <joestringer@nicira.com>
There is no 'errno' field in socket.error. Instead use the
get_exception_errno() function to get the error number.
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
This patch also checks the system platform as clock_gettime
could exist on different platforms but with different values of
CLOCK_MONOTONIC and different definitions of 'struct timespec'.
In this case, the system call would be expected to catch the
error, which is dangerous.
This patch ensures Linux, NetBSD and FreeBSD platforms use
clock_gettime with their corresponding correct values and
definitions. All other platforms use time.time().
Signed-off-by: Ryan Wilson <wryan@nicira.com>
Acked-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
Signed-off-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
