Merge branch 'trac1314'

tests/lettuce/README.tutorial

@@ -50,7 +50,7 @@ will need to expand these, but we will look at them shortly.
 This file defines a feature, just under the feature name we can
 provide a description of the feature.
 
-The one scenario we have no has no steps, so if we run it we should
+The one scenario we have has no steps, so if we run it we should
 see something like:
 
 -- output
@@ -84,7 +84,7 @@ So let's add a step that starts bind10.
         When I start bind10 with configuration example.org.config
 --
 
-This is not good enough; it will fire of the process, but setting up
+This is not good enough; it will start the process, but setting up
 b10-auth may take a few moments, so we need to add a step to wait for
 it to be started before we continue.
 
@@ -121,7 +121,7 @@ regular expression itself), so if the step is defined with "do foo bar", the
 scenario can add words for readability "When I do foo bar".
 
 Each captured group will be passed as an argument to the function we define.
-For bind10, i defined a configuration file, a cmdctl port, and a process
+For bind10, I defined a configuration file, a cmdctl port, and a process
 name. The first two should be self-evident, and the process name is an
 optional name we give it, should we want to address it in the rest of the
 tests. This is most useful if we want to start multiple instances. In the

tests/lettuce/configurations/ixfr-out/testset1-config.db

@@ -0,0 +1 @@
+{"Xfrin": {"zones": [{"use_ixfr": true, "class": "IN", "name": "example.com.", "master_addr": "178.18.82.80"}]}, "version": 2, "Logging": {"loggers": [{"debuglevel": 99, "severity": "DEBUG", "output_options": [{"output": "stderr", "flush": true}], "name": "*"}]}, "Auth": {"database_file": "data/ixfr-out/zones.slite3", "listen_on": [{"port": 47806, "address": "::"}, {"port": 47806, "address": "0.0.0.0"}]}}

tests/lettuce/features/ixfr_out_bind10.feature

@@ -0,0 +1,195 @@
+Feature: IXFR out
+    Tests for IXFR-out, specific for BIND 10 behaviour.
+    These are (part of) the tests as described on
+    http://bind10.isc.org/wiki/IxfrSystemTests
+
+    # A lot of these tests test specific UDP behaviour.
+    #
+    # Where possible, we use the TCP equivalent. Some of the behaviour
+    # tested is UDP-specific though. In either case, a comment above
+    # the test shows how and why it differs from the test specification,
+    # or why it is commented out for now.
+    # When we do implement UDP IXFR, we should probably keep the TCP
+    # tests, and add them to the test specification, so we still have a
+    # 1-to-1 mapping between these tests and the specification document.
+    #
+    # These tests use a zone with just a few records, the first serial
+    # is 2, and it is incremented in steps of 2, up to serial 22.
+    # Each updates either deletes or adds the www.example.com A record.
+    # Version 2 has the record, then the update to version 4 deletes it,
+    # the update to 6 adds it again, and so on, until version 22 (where
+    # the last update has added it again)
+    #
+    # Some of the tests (scenario 1 tests 3 and 4, and scenario 2 tests 1 and
+    # 2 may still not work if we replicate BIND 9's behaviour; it always
+    # responds to UDP IXFR requests with just the SOA, and it does not do
+    # AXFR-style IXFR if the number of changes exceeds the size of the zone)
+    #
+    # So in effect, there is only one test that is currently active (scenario
+    # 1 test 7)
+
+
+    Scenario: Test Set 1
+        Given I have bind10 running with configuration ixfr-out/testset1-config.db
+        Then wait for bind10 xfrout to start
+        The SOA serial for example.com should be 22
+
+        #
+        # Test 1
+        #
+        # We don't support UDP yet, and for TCP we currently return full zone,
+        # so this test is currently skipped
+        #
+        #When I do an IXFR transfer of example.com 123 over udp
+        #The transfer result should have 1 RRs
+        #The full result of the last transfer should be
+        #"""
+        #example.com. 3600 IN SOA ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        #"""
+
+        #
+        # Test 2
+        #
+        # Original test specification was for UDP, using TCP for now
+        #
+        #When I do an IXFR transfer of example.com 22 over udp
+        When I do an IXFR transfer of example.com 22 over tcp
+        The transfer result should have 1 RRs
+        The full result of the last transfer should be
+        """
+        example.com. 3600 IN SOA ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        """
+
+        #
+        # Test 3
+        #
+        # Original test specification was for UDP, using TCP for now
+        #
+        #When I do an IXFR transfer of example.com 20 over udp
+        When I do an IXFR transfer of example.com 20 over tcp
+        The transfer result should have 5 RRs
+        The full result of the last transfer should be
+        """
+        example.com. 3600 IN SOA ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        example.com. 3600 IN SOA ns.example.com. admin.example.com. 20 28800 7200 604800 18000
+        example.com. 3600 IN SOA ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        www.example.com. 3600 IN A 192.0.2.1
+        example.com. 3600 IN SOA ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        """
+
+        #
+        # Test 4
+        #
+        # Original test specification was for UDP, using TCP for now
+        #
+        #When I do an IXFR transfer of example.com 18 over udp
+        When I do an IXFR transfer of example.com 18 over tcp
+        The transfer result should have 8 RRs
+        The full result of the last transfer should be
+        """
+        example.com. 3600 IN SOA ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        example.com. 3600 IN SOA ns.example.com. admin.example.com. 18 28800 7200 604800 18000
+        www.example.com. 3600 IN A 192.0.2.1
+        example.com. 3600 IN SOA ns.example.com. admin.example.com. 20 28800 7200 604800 18000
+        example.com. 3600 IN SOA ns.example.com. admin.example.com. 20 28800 7200 604800 18000
+        example.com. 3600 IN SOA ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        www.example.com. 3600 IN A 192.0.2.1
+        example.com. 3600 IN SOA ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        """
+
+        #
+        # Test 5
+        #
+        # This test does not have a TCP equivalent, so it is skipped.
+        #
+        #When I do an IXFR transfer of example.com 2 over udp
+        #The transfer result should have 1 RRs
+        #The full result of the last transfer should be
+        #"""
+        #example.com. 3600 IN SOA ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        #"""
+
+        #
+        # Test 6
+        #
+        # This test does not have a TCP equivalent, so it is skipped.
+        #
+        #When I do an IXFR transfer of example.com 5 over udp
+        #The transfer result should have 1 RRs
+        #The full result of the last transfer should be
+        #"""
+        #example.com. 3600 IN SOA ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        #"""
+
+        #
+        # Test 7
+        #
+        When I do an IXFR transfer of example.com 14 over tcp
+        The transfer result should have 14 RRs
+        The full result of the last transfer should be
+        """
+        example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 14 28800 7200 604800 18000
+        www.example.com.        3600    IN      A       192.0.2.1
+        example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 16 28800 7200 604800 18000
+        example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 16 28800 7200 604800 18000
+        example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 18 28800 7200 604800 18000
+        www.example.com.        3600    IN      A       192.0.2.1
+        example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 18 28800 7200 604800 18000
+        www.example.com.        3600    IN      A       192.0.2.1
+        example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 20 28800 7200 604800 18000
+        example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 20 28800 7200 604800 18000
+        example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        www.example.com.        3600    IN      A       192.0.2.1
+        example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        """
+
+    Scenario: Test Set 2
+        Given I have bind10 running with configuration ixfr-out/testset1-config.db
+        Then wait for bind10 xfrout to start
+        The SOA serial for example.com should be 22
+
+        #
+        # Test 1
+        #
+        # Original test specification was for UDP, using TCP for now
+        #
+        #When I do an IXFR transfer of example.com 19 over udp
+        When I do an IXFR transfer of example.com 19 over tcp
+        The transfer result should have 5 RRs
+        The full result of the last transfer should be
+        """
+        example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        example.com.            3600    IN      NS      ns.example.com.
+        ns.example.com.         3600    IN      A       192.0.2.1
+        www.example.com.        3600    IN      A       192.0.2.1
+        example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        """
+
+        #
+        # Test 2
+        #
+        # This test has no TCP equivalent
+        #
+        #When I do an IXFR transfer of example.com 6 over udp
+        #The transfer result should have 5 RRs
+        #The full result of the last transfer should be
+        #"""
+        #example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        #example.com.            3600    IN      NS      ns.example.com.
+        #ns.example.com.         3600    IN      A       192.0.2.1
+        #www.example.com.        3600    IN      A       192.0.2.1
+        #example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        #"""
+
+        #
+        # Test 3
+        #
+        # This test has no TCP equivalent
+        #
+        #When I do an IXFR transfer of example.com 2 over udp
+        #The transfer result should have 1 RRs
+        #The full result of the last transfer should be
+        #"""
+        #example.com.            3600    IN      SOA     ns.example.com. admin.example.com. 22 28800 7200 604800 18000
+        #"""

tests/lettuce/features/terrain/bind10_control.py

@@ -79,6 +79,20 @@ def wait_for_auth(step, process_name):
     world.processes.wait_for_stderr_str(process_name, ['AUTH_SERVER_STARTED'],
                                         False)
 
+@step('wait for bind10 xfrout (?:of (\w+) )?to start')
+def wait_for_xfrout(step, process_name):
+    """Wait for b10-xfrout to run. This is done by blocking until the message
+       XFROUT_NEW_CONFIG_DONE is logged.
+       Parameters:
+       process_name ('of <name', optional): The name of the BIND 10 instance
+                    to wait for. Defaults to 'bind10'.
+    """
+    if process_name is None:
+        process_name = "bind10"
+    world.processes.wait_for_stderr_str(process_name,
+                                        ['XFROUT_NEW_CONFIG_DONE'],
+                                        False)
+
 @step('have bind10 running(?: with configuration ([\S]+))?' +\
       '(?: with cmdctl port (\d+))?' +\
       '(?: as ([\S]+))?')

tests/lettuce/features/terrain/querying.py

@@ -179,7 +179,7 @@ class QueryResult(object):
         """
         pass
 
-@step('A query for ([\w.]+) (?:type ([A-Z]+) )?(?:class ([A-Z]+) )?' +
+@step('A query for ([\w.]+) (?:type ([A-Z0-9]+) )?(?:class ([A-Z]+) )?' +
       '(?:to ([^:]+)(?::([0-9]+))? )?should have rcode ([\w.]+)')
 def query(step, query_name, qtype, qclass, addr, port, rcode):
     """

tests/lettuce/features/terrain/transfer.py

@@ -0,0 +1,138 @@
+# Copyright (C) 2011  Internet Systems Consortium.
+#
+# Permission to use, copy, modify, and distribute this software for any
+# purpose with or without fee is hereby granted, provided that the above
+# copyright notice and this permission notice appear in all copies.
+#
+# THE SOFTWARE IS PROVIDED "AS IS" AND INTERNET SYSTEMS CONSORTIUM
+# DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# INTERNET SYSTEMS CONSORTIUM BE LIABLE FOR ANY SPECIAL, DIRECT,
+# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING
+# FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
+# NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION
+# WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+# This script provides transfer (ixfr/axfr) test functionality
+# It provides steps to perform the client side of a transfer,
+# and inspect the results.
+#
+# Like querying.py, it uses dig to do the transfers, and
+# places its output in a result structure
+#
+# This is done in a different file with different steps than
+# querying, because the format of dig's output is
+# very different than that of normal queries
+
+from lettuce import *
+import subprocess
+import re
+
+class TransferResult(object):
+    """This object stores transfer results, which is essentially simply
+       a list of RR strings. These are stored, as read from dig's output,
+       in the list 'records'. So for an IXFR transfer it contains
+       the exact result as returned by the server.
+       If this list is empty, the transfer failed for some reason (dig
+       does not really show error results well, unfortunately).
+       We may add some smarter inspection functionality to this class
+       later.
+    """
+    def __init__(self, args):
+        """Perform the transfer by calling dig, and store the results.
+           args is the array of arguments to pass to Popen(), this
+           is passed as is since for IXFR and AXFR there can be very
+           different options"""
+        self.records = []
+
+        # Technically, using a pipe here can fail; since we don't expect
+        # large output right now, this works, but should we get a test
+        # where we do have a lot of output, this could block, and we will
+        # need to read the output in a different way.
+        dig_process = subprocess.Popen(args, 1, None, None, subprocess.PIPE,
+                                       None)
+        result = dig_process.wait()
+        assert result == 0
+        for l in dig_process.stdout:
+            line = l.strip()
+            if len(line) > 0 and line[0] != ';':
+                self.records.append(line)
+
+@step('An AXFR transfer of ([\w.]+)(?: from ([^:]+)(?::([0-9]+))?)?')
+def perform_axfr(step, zone_name, address, port):
+    """
+    Perform an AXFR transfer, and store the result as an instance of
+    TransferResult in world.transfer_result.
+
+    Step definition:
+    An AXFR transfer of <zone_name> [from <address>:<port>]
+
+    Address defaults to 127.0.0.1
+    Port defaults to 47806
+    """
+    if address is None:
+        address = "127.0.0.1"
+    if port is None:
+        port = 47806
+    args = [ 'dig', 'AXFR', '@' + str(address), '-p', str(port), zone_name ]
+    world.transfer_result = TransferResult(args)
+
+@step('An IXFR transfer of ([\w.]+) (\d+)(?: from ([^:]+)(?::([0-9]+))?)?(?: over (tcp|udp))?')
+def perform_ixfr(step, zone_name, serial, address, port, protocol):
+    """
+    Perform an IXFR transfer, and store the result as an instance of
+    TransferResult in world.transfer_result.
+
+    Step definition:
+    An IXFR transfer of <zone_name> <serial> [from <address>:port] [over <tcp|udp>]
+
+    Address defaults to 127.0.0.1
+    Port defaults to 47806
+    If either tcp or udp is specified, only this protocol will be used.
+    """
+    if address is None:
+        address = "127.0.0.1"
+    if port is None:
+        port = 47806
+    args = [ 'dig', 'IXFR=' + str(serial), '@' + str(address), '-p', str(port), zone_name ]
+    if protocol is not None:
+        assert protocol == 'tcp' or protocol == 'udp', "Unknown protocol: " + protocol
+        if protocol == 'tcp':
+            args.append('+tcp')
+        elif protocol == 'udp':
+            args.append('+notcp')
+    world.transfer_result = TransferResult(args)
+
+@step('transfer result should have (\d+) rrs?')
+def check_transfer_result_count(step, number_of_rrs):
+    """
+    Check the number of rrs in the transfer result object created by
+    the AXFR transfer or IXFR transfer step.
+
+    Step definition:
+    transfer result should have <number> rr[s]
+
+    Fails if the number of RRs is not equal to number
+    """
+    assert int(number_of_rrs) == len(world.transfer_result.records),\
+        "Got " + str(len(world.transfer_result.records)) +\
+        " records, expected " + str(number_of_rrs)
+
+@step('full result of the last transfer should be')
+def check_full_transfer_result(step):
+    """
+    Check the complete output from the last transfer call.
+
+    Step definition:
+    full result of the last transfer should be <multiline value>
+
+    Whitespace is normalized in both the multiline value and the
+    output, but the order of the output is not.
+    Fails if there is any difference between the two. Prints
+    full output and expected value upon failure.
+    """
+    records_string = "\n".join(world.transfer_result.records)
+    records_string = re.sub("[ \t]+", " ", records_string)
+    expect = re.sub("[ \t]+", " ", step.multiline)
+    assert records_string.strip() == expect.strip(),\
+        "Got:\n'" + records_string + "'\nExpected:\n'" + expect + "'"