From f4566355e7bcea43a1d3992d7d2b2aac18548a53 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Fri, 22 Jun 2012 14:01:44 +0200 Subject: [PATCH 01/69] [1708] msgq support in DHCPv6 added. --- doc/devel/02-dhcp.dox | 8 +- doc/guide/bind10-guide.html | 138 ++--- doc/guide/bind10-guide.txt | 582 +++++++++--------- doc/guide/bind10-guide.xml | 58 +- doc/guide/bind10-messages.html | 2 +- src/bin/dhcp6/Makefile.am | 3 + src/bin/dhcp6/ctrl_dhcp6_srv.cc | 159 +++++ src/bin/dhcp6/ctrl_dhcp6_srv.h | 123 ++++ src/bin/dhcp6/dhcp6_srv.cc | 13 +- src/bin/dhcp6/dhcp6_srv.h | 4 +- src/bin/dhcp6/main.cc | 74 +-- src/bin/dhcp6/tests/Makefile.am | 5 +- .../dhcp6/tests/ctrl_dhcp6_srv_unittest.cc | 85 +++ src/bin/dhcp6/tests/dhcp6_test.py | 4 +- 14 files changed, 793 insertions(+), 465 deletions(-) create mode 100644 src/bin/dhcp6/ctrl_dhcp6_srv.cc create mode 100644 src/bin/dhcp6/ctrl_dhcp6_srv.h create mode 100644 src/bin/dhcp6/tests/ctrl_dhcp6_srv_unittest.cc diff --git a/doc/devel/02-dhcp.dox b/doc/devel/02-dhcp.dox index b6c3093e92..5217f73971 100644 --- a/doc/devel/02-dhcp.dox +++ b/doc/devel/02-dhcp.dox @@ -72,11 +72,13 @@ * DHCPv6 server component does not support relayed traffic yet, as * support for relay decapsulation is not implemented yet. * - * DHCPv6 server component does not listen to BIND10 message queue. - * * DHCPv6 server component does not use BIND10 logging yet. * - * DHCPv6 server component is not integrated with boss yet. + * @section dhcpv6Session BIND10 message queue integration + * + * DHCPv4 server component is now integrated with BIND10 message queue. + * It follows the same principle as DHCPv4. See \ref dhcpv4Session for + * details. * * @page libdhcp libdhcp++ * diff --git a/doc/guide/bind10-guide.html b/doc/guide/bind10-guide.html index 7a1a120a7b..11dcf00957 100644 --- a/doc/guide/bind10-guide.html +++ b/doc/guide/bind10-guide.html @@ -1,4 +1,4 @@ -BIND 10 Guide

BIND 10 Guide

Administrator Reference for BIND 10

This is the reference guide for BIND 10 version +BIND 10 Guide

BIND 10 Guide

Administrator Reference for BIND 10

This is the reference guide for BIND 10 version 20120405.

Copyright © 2010-2012 Internet Systems Consortium, Inc.

Abstract

BIND 10 is a framework that features Domain Name System (DNS) suite and Dynamic Host Configuration Protocol (DHCP) servers managed by Internet Systems Consortium (ISC). It @@ -10,9 +10,9 @@ The most up-to-date version of this document (in PDF, HTML, and plain text formats), along with other documents for BIND 10, can be found at http://bind10.isc.org/docs. -

Table of Contents

Preface
1. Acknowledgements
1. Introduction
1.1. Supported Platforms
1.2. Required Software
1.3. Starting and Stopping the Server
1.4. Managing BIND 10
2. Installation
2.1. Building Requirements
2.2. Quick start
2.3. Installation from source
2.3.1. Download Tar File
2.3.2. Retrieve from Git
2.3.3. Configure before the build
2.3.4. Build
2.3.5. Install
2.3.6. Install Hierarchy
3. Starting BIND10 with bind10
3.1. Starting BIND 10
3.2. Configuration of started processes
4. Command channel
5. Configuration manager
6. Remote control daemon
6.1. Configuration specification for b10-cmdctl
7. Control and configure user interface
8. Authoritative Server
8.1. Server Configurations
8.2. Data Source Backends
8.2.1. In-memory Data Source
8.2.2. In-memory Data Source With SQLite3 Backend
8.2.3. Reloading an In-memory Data Source
8.2.4. Disabling In-memory Data Sources
8.3. Loading Master Zones Files
9. Incoming Zone Transfers
9.1. Configuration for Incoming Zone Transfers
9.2. Enabling IXFR
9.3. Secondary Manager
9.4. Trigger an Incoming Zone Transfer Manually
9.5. Incoming Transfers with In-memory Datasource
10. Outbound Zone Transfers
11. Dynamic DNS Update
11.1. Enabling Dynamic Update
11.2. Access Control
11.3. Miscellaneous Operational Issues
12. Recursive Name Server
12.1. Access Control
12.2. Forwarding
13. DHCPv4 Server
13.1. DHCPv4 Server Usage
13.2. DHCPv4 Server Configuration
13.3. Supported standards
13.4. DHCPv4 Server Limitations
14. DHCPv6 Server
14.1. DHCPv6 Server Usage
14.2. DHCPv6 Server Configuration
14.3. Supported DHCPv6 Standards
14.4. DHCPv6 Server Limitations
15. libdhcp++ library
15.1. Interface detection
15.2. DHCPv4/DHCPv6 packet handling
16. Statistics
17. Logging
17.1. Logging configuration
17.1.1. Loggers
17.1.2. Output Options
17.1.3. Example session
17.2. Logging Message Format

List of Tables

3.1.

Preface

Table of Contents

1. Acknowledgements

1. Acknowledgements

ISC would like to acknowledge generous support for +

Table of Contents

Preface
1. Acknowledgements
1. Introduction
1.1. Supported Platforms
1.2. Required Software
1.3. Starting and Stopping the Server
1.4. Managing BIND 10
2. Installation
2.1. Building Requirements
2.2. Quick start
2.3. Installation from source
2.3.1. Download Tar File
2.3.2. Retrieve from Git
2.3.3. Configure before the build
2.3.4. Build
2.3.5. Install
2.3.6. Install Hierarchy
3. Starting BIND10 with bind10
3.1. Starting BIND 10
3.2. Configuration of started processes
4. Command channel
5. Configuration manager
6. Remote control daemon
6.1. Configuration specification for b10-cmdctl
7. Control and configure user interface
8. Authoritative Server
8.1. Server Configurations
8.2. Data Source Backends
8.2.1. In-memory Data Source
8.2.2. In-memory Data Source With SQLite3 Backend
8.2.3. Reloading an In-memory Data Source
8.2.4. Disabling In-memory Data Sources
8.3. Loading Master Zones Files
9. Incoming Zone Transfers
9.1. Configuration for Incoming Zone Transfers
9.2. Enabling IXFR
9.3. Secondary Manager
9.4. Trigger an Incoming Zone Transfer Manually
9.5. Incoming Transfers with In-memory Datasource
10. Outbound Zone Transfers
11. Dynamic DNS Update
11.1. Enabling Dynamic Update
11.2. Access Control
11.3. Miscellaneous Operational Issues
12. Recursive Name Server
12.1. Access Control
12.2. Forwarding
13. DHCPv4 Server
13.1. DHCPv4 Server Usage
13.2. DHCPv4 Server Configuration
13.3. Supported standards
13.4. DHCPv4 Server Limitations
14. DHCPv6 Server
14.1. DHCPv6 Server Usage
14.2. DHCPv6 Server Configuration
14.3. Supported DHCPv6 Standards
14.4. DHCPv6 Server Limitations
15. libdhcp++ library
15.1. Interface detection
15.2. DHCPv4/DHCPv6 packet handling
16. Statistics
17. Logging
17.1. Logging configuration
17.1.1. Loggers
17.1.2. Output Options
17.1.3. Example session
17.2. Logging Message Format

List of Tables

3.1.

Preface

Table of Contents

1. Acknowledgements

1. Acknowledgements

ISC would like to acknowledge generous support for BIND 10 development of DHCPv4 and DHCPv6 components provided - by Comcast.

Chapter 1. Introduction

Table of Contents

1.1. Supported Platforms
1.2. Required Software
1.3. Starting and Stopping the Server
1.4. Managing BIND 10

+ by Comcast.

Chapter 1. Introduction

Table of Contents

1.1. Supported Platforms
1.2. Required Software
1.3. Starting and Stopping the Server
1.4. Managing BIND 10

BIND is the popular implementation of a DNS server, developer interfaces, and DNS tools. BIND 10 is a rewrite of BIND 9. BIND 10 is written in C++ and Python @@ -23,7 +23,7 @@

This guide covers the experimental prototype of BIND 10 version 20120405. -

1.1. Supported Platforms

+

1.1. Supported Platforms

BIND 10 builds have been tested on (in no particular order) Debian GNU/Linux 5 and unstable, Ubuntu 9.10, NetBSD 5, Solaris 10 and 11, FreeBSD 7 and 8, CentOS Linux 5.3, @@ -171,7 +171,7 @@ and, of course, DNS. These include detailed developer documentation and code examples. -

Chapter 2. Installation

Table of Contents

2.1. Building Requirements
2.2. Quick start
2.3. Installation from source
2.3.1. Download Tar File
2.3.2. Retrieve from Git
2.3.3. Configure before the build
2.3.4. Build
2.3.5. Install
2.3.6. Install Hierarchy

2.1. Building Requirements

+

Chapter 2. Installation

Table of Contents

2.1. Building Requirements
2.2. Quick start
2.3. Installation from source
2.3.1. Download Tar File
2.3.2. Retrieve from Git
2.3.3. Configure before the build
2.3.4. Build
2.3.5. Install
2.3.6. Install Hierarchy

2.1. Building Requirements

In addition to the run-time requirements, building BIND 10 from source code requires various development include headers.

Note

@@ -233,14 +233,14 @@ the Git code revision control system or as a downloadable tar file. It may also be available in pre-compiled ready-to-use packages from operating system vendors. -

2.3.1. Download Tar File

+

2.3.1. Download Tar File

Downloading a release tar file is the recommended method to obtain the source code.

The BIND 10 releases are available as tar file downloads from ftp://ftp.isc.org/isc/bind10/. Periodic development snapshots may also be available. -

2.3.2. Retrieve from Git

+

2.3.2. Retrieve from Git

Downloading this "bleeding edge" code is recommended only for developers or advanced users. Using development code in a production environment is not recommended. @@ -274,7 +274,7 @@ autoheader, automake, and related commands. -

2.3.3. Configure before the build

+

2.3.3. Configure before the build

BIND 10 uses the GNU Build System to discover build environment details. To generate the makefiles using the defaults, simply run: @@ -305,16 +305,16 @@

If the configure fails, it may be due to missing or old dependencies. -

2.3.4. Build

+

2.3.4. Build

After the configure step is complete, to build the executables from the C++ code and prepare the Python scripts, run: 

$ make

-

2.3.5. Install

+

2.3.5. Install

To install the BIND 10 executables, support files, and documentation, run: 

$ make install

-

Note

The install step may require superuser privileges.

2.3.6. Install Hierarchy

+

Note

The install step may require superuser privileges.

2.3.6. Install Hierarchy

The following is the layout of the complete BIND 10 installation:

  • bin/ — @@ -406,7 +406,7 @@ during startup or shutdown. Unless specified, the component is started in usual way. This is the list of components that need to be started in a special way, with the value of special used for them: -

    Table 3.1. 

    ComponentSpecialDescription
    b10-authauthAuthoritative server
    b10-resolverresolverThe resolver
    b10-cmdctlcmdctlThe command control (remote control interface)


    +

    Table 3.1. 

    ComponentSpecialDescription
    b10-authauthAuthoritative server
    b10-resolverresolverThe resolver
    b10-cmdctlcmdctlThe command control (remote control interface)


    The kind specifies how a failure of the component should be handled. If it is set to “dispensable” @@ -634,12 +634,12 @@ shutdown the details and relays (over a b10-msgq command channel) the configuration on to the specified module.

    -

Chapter 8. Authoritative Server

Table of Contents

8.1. Server Configurations
8.2. Data Source Backends
8.2.1. In-memory Data Source
8.2.2. In-memory Data Source With SQLite3 Backend
8.2.3. Reloading an In-memory Data Source
8.2.4. Disabling In-memory Data Sources
8.3. Loading Master Zones Files

+

Chapter 8. Authoritative Server

Table of Contents

8.1. Server Configurations
8.2. Data Source Backends
8.2.1. In-memory Data Source
8.2.2. In-memory Data Source With SQLite3 Backend
8.2.3. Reloading an In-memory Data Source
8.2.4. Disabling In-memory Data Sources
8.3. Loading Master Zones Files

The b10-auth is the authoritative DNS server. It supports EDNS0 and DNSSEC. It supports IPv6. Normally it is started by the bind10 master process. -

8.1. Server Configurations

+

8.1. Server Configurations

b10-auth is configured via the b10-cfgmgr configuration manager. The module name is “Auth”. @@ -736,7 +736,7 @@ This may be a temporary setting until then. if configured.)

-

8.2. Data Source Backends

Note

+

8.2. Data Source Backends

Note

For the development prototype release, b10-auth supports a SQLite3 data source backend and in-memory data source backend. @@ -815,7 +815,7 @@ This may be a temporary setting until then. and/or zones[0] for the relevant zone as needed.) -

8.3. Loading Master Zones Files

+

8.3. Loading Master Zones Files

RFC 1035 style DNS master zone files may imported into a BIND 10 SQLite3 data source by using the b10-loadzone utility. @@ -844,7 +844,7 @@ This may be a temporary setting until then. If you reload a zone already existing in the database, all records from that prior zone disappear and a whole new set appears. -

Chapter 9. Incoming Zone Transfers

Table of Contents

9.1. Configuration for Incoming Zone Transfers
9.2. Enabling IXFR
9.3. Secondary Manager
9.4. Trigger an Incoming Zone Transfer Manually
9.5. Incoming Transfers with In-memory Datasource

+

Chapter 9. Incoming Zone Transfers

Table of Contents

9.1. Configuration for Incoming Zone Transfers
9.2. Enabling IXFR
9.3. Secondary Manager
9.4. Trigger an Incoming Zone Transfer Manually
9.5. Incoming Transfers with In-memory Datasource

Incoming zones are transferred using the b10-xfrin process which is started by bind10. When received, the zone is stored in the corresponding BIND 10 @@ -858,7 +858,7 @@ This may be a temporary setting until then. IXFR. Due to some implementation limitations of the current development release, however, it only tries AXFR by default, and care should be taken to enable IXFR. -

9.1. Configuration for Incoming Zone Transfers

+

9.1. Configuration for Incoming Zone Transfers

In practice, you need to specify a list of secondary zones to enable incoming zone transfers for these zones (you can still trigger a zone transfer manually, without a prior configuration @@ -874,7 +874,7 @@ This may be a temporary setting until then. > config commit

(We assume there has been no zone configuration before). -

9.2. Enabling IXFR

+

9.2. Enabling IXFR

As noted above, b10-xfrin uses AXFR for zone transfers by default. To enable IXFR for zone transfers for a particular zone, set the use_ixfr @@ -926,13 +926,13 @@ This may be a temporary setting until then. (i.e. no SOA record for it), b10-zonemgr will automatically tell b10-xfrin to transfer the zone in. -

9.4. Trigger an Incoming Zone Transfer Manually

+

9.4. Trigger an Incoming Zone Transfer Manually

To manually trigger a zone transfer to retrieve a remote zone, you may use the bindctl utility. For example, at the bindctl prompt run: 

> Xfrin retransfer zone_name="foo.example.org" master=192.0.2.99

-

9.5. Incoming Transfers with In-memory Datasource

+

9.5. Incoming Transfers with In-memory Datasource

In the case of an incoming zone transfer, the received zone is first stored in the corresponding BIND 10 datasource. In case the secondary zone is served by an in-memory datasource @@ -987,7 +987,7 @@ Xfrout/transfer_acl[0] {"action": "ACCEPT"} any (default)

TSIGs in the incoming messages and to sign responses.

Note

The way to specify zone specific configuration (ACLs, etc) is likely to be changed. -

Chapter 11. Dynamic DNS Update

Table of Contents

11.1. Enabling Dynamic Update
11.2. Access Control
11.3. Miscellaneous Operational Issues

+

Chapter 11. Dynamic DNS Update

Table of Contents

11.1. Enabling Dynamic Update
11.2. Access Control
11.3. Miscellaneous Operational Issues

BIND 10 supports the server side of the Dynamic DNS Update (DDNS) protocol as defined in RFC 2136. This service is provided by the b10-ddns @@ -1034,7 +1034,7 @@ Xfrout/transfer_acl[0] {"action": "ACCEPT"} any (default)

this feature.

-

11.1. Enabling Dynamic Update

+

11.1. Enabling Dynamic Update

First off, it must be made sure that a few components on which b10-ddns depends are configured to run, which are b10-auth @@ -1093,7 +1093,7 @@ Xfrout/transfer_acl[0] {"action": "ACCEPT"} any (default)

specifying it. But for it to shutdown gracefully this parameter should also be specified.

-

11.2. Access Control

+

11.2. Access Control

By default b10-ddns rejects any update requests from any clients by returning a response with an RCODE of REFUSED. @@ -1192,7 +1192,7 @@ DDNS/zones[0]/update_acl[1] {"action": "ACCEPT", "from": "::1", "key": "key. arbitrary clients. There have been other troubles that could have been avoided if the ACL could be checked before the prerequisite check. -

11.3. Miscellaneous Operational Issues

+

11.3. Miscellaneous Operational Issues

Unlike BIND 9, BIND 10 currently does not support automatic resigning of DNSSEC-signed zone when it's updated via DDNS. It could be possible to resign the updated zone afterwards @@ -1234,7 +1234,7 @@ DDNS/zones[0]/update_acl[1] {"action": "ACCEPT", "from": "::1", "key": "key. IXFR. This is done automatically; it does not require specific configuration to make this possible. -

Chapter 12. Recursive Name Server

Table of Contents

12.1. Access Control
12.2. Forwarding

+

Chapter 12. Recursive Name Server

Table of Contents

12.1. Access Control
12.2. Forwarding

The b10-resolver process is started by bind10. @@ -1268,7 +1268,7 @@ DDNS/zones[0]/update_acl[1] {"action": "ACCEPT", "from": "::1", "key": "key.

(Replace the “2” as needed; run “config show - Resolver/listen_on” if needed.)

12.1. Access Control

+ Resolver/listen_on” if needed.)

12.1. Access Control

By default, the b10-resolver daemon only accepts DNS queries from the localhost (127.0.0.1 and ::1). The Resolver/query_acl configuration may @@ -1301,7 +1301,7 @@ DDNS/zones[0]/update_acl[1] {"action": "ACCEPT", "from": "::1", "key": "key.

(Replace the “2” as needed; run “config show Resolver/query_acl” if needed.)

Note

This prototype access control configuration - syntax may be changed.

12.2. Forwarding

+ syntax may be changed.

12.2. Forwarding

To enable forwarding, the upstream address and port must be configured to forward queries to, such as: @@ -1362,7 +1362,7 @@ DDNS/zones[0]/update_acl[1] {"action": "ACCEPT", "from": "::1", "key": "key. or 

> config remove Boss/components b10-dhcp4
 > config commit

- At start, the server will detect available network interfaces + During start-up the server will detect available network interfaces and will attempt to open UDP sockets on all interfaces that are up, running, are not loopback, and have IPv4 address assigned. @@ -1372,13 +1372,8 @@ DDNS/zones[0]/update_acl[1] {"action": "ACCEPT", "from": "::1", "key": "key. will respond to them with OFFER and ACK, respectively. Since the DHCPv4 server opens privileged ports, it requires root - access. Make sure you run this daemon as root.

Note

- Integration with bind10 is - planned. Ultimately, b10-dhcp4 will not - be started directly, but rather via - bind10. Please be aware of this planned - change. -

13.2. DHCPv4 Server Configuration

+ access. Make sure you run this daemon as root. +

13.2. DHCPv4 Server Configuration

The DHCPv4 server does not have a lease database implemented yet nor any support for configuration, so every time the same set of configuration options (including the same fixed address) @@ -1458,22 +1453,21 @@ const std::string HARDCODED_SERVER_ID = "192.0.2.1";

significant limitations. See Section 14.4, “DHCPv6 Server Limitations” for details.

- The DHCPv6 server is implemented as b10-dhcp6 - daemon. As it is not configurable yet, it is fully autonomous, - that is it does not interact with b10-cfgmgr. - To start DHCPv6 server, simply input: - - 

-#cd src/bin/dhcp6
-#./b10-dhcp6
-

- - Depending on your installation, b10-dhcp6 - binary may reside in src/bin/dhcp6 in your source code - directory, in /usr/local/bin/b10-dhcp6 or other directory - you specified during compilation. - - At start, server will detect available network interfaces + b10-dhcp6 is a BIND10 component and is being + run under BIND10 framework. To add a DHCPv6 process to the set of running + BIND10 services, you can use following commands in bindctl: + 

> config add Boss/components b10-dhcp6
+> config set Boss/components/b10-dhcp6/kind dispensable
+> config commit

+

+ To shutdown running b10-dhcp6, please use the + following command: + 

> Dhcp6 shutdown

+ or + 

> config remove Boss/components b10-dhcp6
+> config commit

+

+ During start-up the server will detect available network interfaces and will attempt to open UDP sockets on all interfaces that are up, running, are not loopback, are multicast-capable, and have IPv6 address assigned. @@ -1484,13 +1478,7 @@ const std::string HARDCODED_SERVER_ID = "192.0.2.1";

Since the DHCPv6 server opens privileged ports, it requires root access. Make sure you run this daemon as root. -

Note

- Integration with bind10 is - planned. Ultimately, b10-dhcp6 will not - be started directly, but rather via - bind10. Please be aware of this planned - change. -

14.2. DHCPv6 Server Configuration

+

14.2. DHCPv6 Server Configuration

The DHCPv6 server does not have lease database implemented yet or any support for configuration, so every time the same set of configuration options (including the same fixed address) @@ -1498,7 +1486,7 @@ const std::string HARDCODED_SERVER_ID = "192.0.2.1";

At this stage of development, the only way to alter server configuration is to tweak its source code. To do so, please - edit src/bin/dhcp6/dhcp6_srv.cc file and modify following + edit src/bin/dhcp6/dhcp6_srv.cc file, modify the following parameters and recompile: 

 const std::string HARDCODED_LEASE = "2001:db8:1::1234:abcd";
@@ -1592,7 +1580,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

} }

-

Chapter 17. Logging

Table of Contents

17.1. Logging configuration
17.1.1. Loggers
17.1.2. Output Options
17.1.3. Example session
17.2. Logging Message Format

17.1. Logging configuration

+

Chapter 17. Logging

Table of Contents

17.1. Logging configuration
17.1.1. Loggers
17.1.2. Output Options
17.1.3. Example session
17.2. Logging Message Format

17.1. Logging configuration

The logging system in BIND 10 is configured through the Logging module. All BIND 10 modules will look at the @@ -1601,7 +1589,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

-

17.1.1. Loggers

+

17.1.1. Loggers

Within BIND 10, a message is logged through a component called a "logger". Different parts of BIND 10 log messages @@ -1622,7 +1610,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

(what to log), and the output_options (where to log). -

17.1.1.1. name (string)

+

17.1.1.1. name (string)

Each logger in the system has a name, the name being that of the component using it to log messages. For instance, if you want to configure logging for the resolver module, @@ -1695,7 +1683,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

“Auth.cache” logger will appear in the output with a logger name of “b10-auth.cache”). -

17.1.1.2. severity (string)

+

17.1.1.2. severity (string)

This specifies the category of messages logged. Each message is logged with an associated severity which @@ -1711,7 +1699,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

-

17.1.1.3. output_options (list)

+

17.1.1.3. output_options (list)

Each logger can have zero or more output_options. These specify where log @@ -1721,7 +1709,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

The other options for a logger are: -

17.1.1.4. debuglevel (integer)

+

17.1.1.4. debuglevel (integer)

When a logger's severity is set to DEBUG, this value specifies what debug messages should be printed. It ranges @@ -1730,7 +1718,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

If severity for the logger is not DEBUG, this value is ignored. -

17.1.1.5. additive (true or false)

+

17.1.1.5. additive (true or false)

If this is true, the output_options from the parent will be used. For example, if there are two @@ -1744,18 +1732,18 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

-

17.1.2. Output Options

+

17.1.2. Output Options

The main settings for an output option are the destination and a value called output, the meaning of which depends on the destination that is set. -

17.1.2.1. destination (string)

+

17.1.2.1. destination (string)

The destination is the type of output. It can be one of: -

  • console
  • file
  • syslog

17.1.2.2. output (string)

+

  • console
  • file
  • syslog

17.1.2.2. output (string)

Depending on what is set as the output destination, this value is interpreted as follows: @@ -1785,12 +1773,12 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

The other options for output_options are: -

17.1.2.2.1. flush (true of false)

+

17.1.2.2.1. flush (true of false)

Flush buffers after each log message. Doing this will reduce performance but will ensure that if the program terminates abnormally, all messages up to the point of termination are output. -

17.1.2.2.2. maxsize (integer)

+

17.1.2.2.2. maxsize (integer)

Only relevant when destination is file, this is maximum file size of output files in bytes. When the maximum size is reached, the file is renamed and a new file opened. @@ -1799,11 +1787,11 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

etc.)

If this is 0, no maximum file size is used. -

17.1.2.2.3. maxver (integer)

+

17.1.2.2.3. maxver (integer)

Maximum number of old log files to keep around when rolling the output file. Only relevant when destination is “file”. -

17.1.3. Example session

+

17.1.3. Example session

In this example we want to set the global logging to write to the file /var/log/my_bind10.log, @@ -1964,7 +1952,7 @@ Logging/loggers[0]/output_options[0]/maxver 8 integer (modified) And every module will now be using the values from the logger named “*”. -

17.2. Logging Message Format

+

17.2. Logging Message Format

Each message written by BIND 10 to the configured logging destinations comprises a number of components that identify the origin of the message and, if the message indicates diff --git a/doc/guide/bind10-guide.txt b/doc/guide/bind10-guide.txt index e38b43a34b..bc015154e1 100644 --- a/doc/guide/bind10-guide.txt +++ b/doc/guide/bind10-guide.txt @@ -4,7 +4,7 @@ Administrator Reference for BIND 10 This is the reference guide for BIND 10 version 20120405. - Copyright (c) 2010-2012 Internet Systems Consortium, Inc. + Copyright © 2010-2012 Internet Systems Consortium, Inc. Abstract @@ -168,12 +168,12 @@ Preface 1. Acknowledgements -1. Acknowledgements +1. Acknowledgements ISC would like to acknowledge generous support for BIND 10 development of DHCPv4 and DHCPv6 components provided by Comcast. -Chapter 1. Introduction +Chapter 1. Introduction Table of Contents @@ -193,7 +193,7 @@ Chapter 1. Introduction This guide covers the experimental prototype of BIND 10 version 20120405. -1.1. Supported Platforms +1.1. Supported Platforms BIND 10 builds have been tested on (in no particular order) Debian GNU/Linux 5 and unstable, Ubuntu 9.10, NetBSD 5, Solaris 10 and 11, @@ -202,7 +202,7 @@ Chapter 1. Introduction planned for BIND 10 to build, install and run on Windows and standard Unix-type platforms. -1.2. Required Software +1.2. Required Software BIND 10 requires at least Python 3.1 (http://www.python.org/). It has also been tested with Python 3.2. @@ -229,7 +229,7 @@ Chapter 1. Introduction installation nor standard packages collections. You may need to install them separately. -1.3. Starting and Stopping the Server +1.3. Starting and Stopping the Server BIND 10 is modular. Part of this modularity is accomplished using multiple cooperating processes which, together, provide the server functionality. @@ -242,47 +242,46 @@ Chapter 1. Introduction processes as needed. The processes started by the bind10 command have names starting with "b10-", including: - o b10-auth -- Authoritative DNS server. This process serves DNS - requests. - o b10-cfgmgr -- Configuration manager. This process maintains all of the + o b10-auth — Authoritative DNS server. This process serves DNS requests. + o b10-cfgmgr — Configuration manager. This process maintains all of the configuration for BIND 10. - o b10-cmdctl -- Command and control service. This process allows - external control of the BIND 10 system. - o b10-ddns -- Dynamic DNS update service. This process is used to handle + o b10-cmdctl — Command and control service. This process allows external + control of the BIND 10 system. + o b10-ddns — Dynamic DNS update service. This process is used to handle incoming DNS update requests to allow granted clients to update zones for which BIND 10 is serving as a primary server. - o b10-msgq -- Message bus daemon. This process coordinates communication + o b10-msgq — Message bus daemon. This process coordinates communication between all of the other BIND 10 processes. - o b10-resolver -- Recursive name server. This process handles incoming + o b10-resolver — Recursive name server. This process handles incoming queries. - o b10-sockcreator -- Socket creator daemon. This process creates sockets + o b10-sockcreator — Socket creator daemon. This process creates sockets used by network-listening BIND 10 processes. - o b10-stats -- Statistics collection daemon. This process collects and + o b10-stats — Statistics collection daemon. This process collects and reports statistics data. - o b10-stats-httpd -- HTTP server for statistics reporting. This process + o b10-stats-httpd — HTTP server for statistics reporting. This process reports statistics data in XML format over HTTP. - o b10-xfrin -- Incoming zone transfer service. This process is used to + o b10-xfrin — Incoming zone transfer service. This process is used to transfer a new copy of a zone into BIND 10, when acting as a secondary server. - o b10-xfrout -- Outgoing zone transfer service. This process is used to + o b10-xfrout — Outgoing zone transfer service. This process is used to handle transfer requests to send a local zone to a remote secondary server, when acting as a master server. - o b10-zonemgr -- Secondary manager. This process keeps track of timers + o b10-zonemgr — Secondary manager. This process keeps track of timers and other necessary information for BIND 10 to act as a slave server. These are ran automatically by bind10 and do not need to be run manually. -1.4. Managing BIND 10 +1.4. Managing BIND 10 Once BIND 10 is running, a few commands are used to interact directly with the system: - o bindctl -- interactive administration interface. This is a low-level + o bindctl — interactive administration interface. This is a low-level command-line tool which allows a developer or an experienced administrator to control BIND 10. - o b10-loadzone -- zone file loader. This tool will load standard + o b10-loadzone — zone file loader. This tool will load standard masterfile-format zone files into BIND 10. - o b10-cmdctl-usermgr -- user access control. This tool allows an + o b10-cmdctl-usermgr — user access control. This tool allows an administrator to authorize additional users to manage BIND 10. The tools and modules are covered in full detail in this guide. In @@ -292,7 +291,7 @@ Chapter 1. Introduction Python for the message bus, configuration backend, and, of course, DNS. These include detailed developer documentation and code examples. -Chapter 2. Installation +Chapter 2. Installation Table of Contents @@ -314,7 +313,7 @@ Chapter 2. Installation 2.3.6. Install Hierarchy -2.1. Building Requirements +2.1. Building Requirements In addition to the run-time requirements, building BIND 10 from source code requires various development include headers. @@ -340,7 +339,7 @@ Chapter 2. Installation Visit the wiki at http://bind10.isc.org/wiki/SystemSpecificNotes for system-specific installation tips. -2.2. Quick start +2.2. Quick start Note @@ -351,48 +350,48 @@ Chapter 2. Installation To quickly get started with BIND 10, follow these steps. - 1. Install required run-time and build dependencies. - 2. Download the BIND 10 source tar file from +  1. Install required run-time and build dependencies. +  2. Download the BIND 10 source tar file from ftp://ftp.isc.org/isc/bind10/. - 3. Extract the tar file: +  3. Extract the tar file: $ gzcat bind10-VERSION.tar.gz | tar -xvf - - 4. Go into the source and run configure: +  4. Go into the source and run configure: $ cd bind10-VERSION $ ./configure - 5. Build it: +  5. Build it: $ make - 6. Install it (to default /usr/local): +  6. Install it (to default /usr/local): $ make install - 7. Start the server: +  7. Start the server: $ /usr/local/sbin/bind10 - 8. Test it; for example: +  8. Test it; for example: $ dig @127.0.0.1 -c CH -t TXT authors.bind - 9. Load desired zone file(s), for example: +  9. Load desired zone file(s), for example: $ b10-loadzone your.zone.example.org - 10. Test the new zone. + 10. Test the new zone. -2.3. Installation from source +2.3. Installation from source BIND 10 is open source software written in C++ and Python. It is freely available in source code form from ISC via the Git code revision control system or as a downloadable tar file. It may also be available in pre-compiled ready-to-use packages from operating system vendors. - 2.3.1. Download Tar File + 2.3.1. Download Tar File Downloading a release tar file is the recommended method to obtain the source code. @@ -401,7 +400,7 @@ Chapter 2. Installation ftp://ftp.isc.org/isc/bind10/. Periodic development snapshots may also be available. - 2.3.2. Retrieve from Git + 2.3.2. Retrieve from Git Downloading this "bleeding edge" code is recommended only for developers or advanced users. Using development code in a production environment is @@ -416,7 +415,7 @@ Chapter 2. Installation The latest development code, including temporary experiments and un-reviewed code, is available via the BIND 10 code revision control system. This is powered by Git and all the BIND 10 development is public. - The leading development is done in the "master". + The leading development is done in the “master”. The code can be checked out from git://git.bind10.isc.org/bind10; for example: @@ -429,7 +428,7 @@ Chapter 2. Installation the --install switch. This will run autoconf, aclocal, libtoolize, autoheader, automake, and related commands. - 2.3.3. Configure before the build + 2.3.3. Configure before the build BIND 10 uses the GNU Build System to discover build environment details. To generate the makefiles using the defaults, simply run: @@ -464,14 +463,14 @@ Chapter 2. Installation If the configure fails, it may be due to missing or old dependencies. - 2.3.4. Build + 2.3.4. Build After the configure step is complete, to build the executables from the C++ code and prepare the Python scripts, run: $ make - 2.3.5. Install + 2.3.5. Install To install the BIND 10 executables, support files, and documentation, run: @@ -481,22 +480,22 @@ Chapter 2. Installation The install step may require superuser privileges. - 2.3.6. Install Hierarchy + 2.3.6. Install Hierarchy The following is the layout of the complete BIND 10 installation: - o bin/ -- general tools and diagnostic clients. - o etc/bind10-devel/ -- configuration files. - o lib/ -- libraries and python modules. - o libexec/bind10-devel/ -- executables that a user wouldn't normally run + o bin/ — general tools and diagnostic clients. + o etc/bind10-devel/ — configuration files. + o lib/ — libraries and python modules. + o libexec/bind10-devel/ — executables that a user wouldn't normally run directly and are not run independently. These are the BIND 10 modules which are daemons started by the bind10 tool. - o sbin/ -- commands used by the system administrator. - o share/bind10-devel/ -- configuration specifications. - o share/man/ -- manual pages (online documentation). - o var/bind10-devel/ -- data source and configuration databases. + o sbin/ — commands used by the system administrator. + o share/bind10-devel/ — configuration specifications. + o share/man/ — manual pages (online documentation). + o var/bind10-devel/ — data source and configuration databases. -Chapter 3. Starting BIND10 with bind10 +Chapter 3. Starting BIND10 with bind10 Table of Contents @@ -523,7 +522,7 @@ Chapter 3. Starting BIND10 with bind10 b10-cmdctl for administration tools to communicate with the system, and b10-stats for statistics collection. -3.1. Starting BIND 10 +3.1. Starting BIND 10 To start the BIND 10 service, simply run bind10. Run it with the --verbose switch to get additional debugging or diagnostic output. @@ -532,9 +531,9 @@ Chapter 3. Starting BIND10 with bind10 If the setproctitle Python module is detected at start up, the process names for the Python-based daemons will be renamed to better identify them - instead of just "python". This is not needed on some operating systems. + instead of just “python”. This is not needed on some operating systems. -3.2. Configuration of started processes +3.2. Configuration of started processes The processes to be started can be configured, with the exception of the b10-sockcreator, b10-msgq and b10-cfgmgr. @@ -561,7 +560,7 @@ Chapter 3. Starting BIND10 with bind10 usual way. This is the list of components that need to be started in a special way, with the value of special used for them: - Table 3.1. + Table 3.1.  +------------------------------------------------------------------------+ | Component | Special | Description | @@ -575,11 +574,11 @@ Chapter 3. Starting BIND10 with bind10 +------------------------------------------------------------------------+ The kind specifies how a failure of the component should be handled. If it - is set to "dispensable" (the default unless you set something else), it - will get started again if it fails. If it is set to "needed" and it fails + is set to “dispensable” (the default unless you set something else), it + will get started again if it fails. If it is set to “needed” and it fails at startup, the whole bind10 shuts down and exits with error exit code. But if it fails some time later, it is just started again. If you set it - to "core", you indicate that the system is not usable without the + to “core”, you indicate that the system is not usable without the component and if such component fails, the system shuts down no matter when the failure happened. This is the behaviour of the core components (the ones you can't turn off), but you can declare any other components as @@ -592,10 +591,10 @@ Chapter 3. Starting BIND10 with bind10 the default is enough. There are other parameters we didn't use in our example. One of them is - "address". It is the address used by the component on the b10-msgq message + “address”. It is the address used by the component on the b10-msgq message bus. The special components already know their address, but the usual ones don't. The address is by convention the thing after b10-, with the first - letter capitalized (eg. b10-stats would have "Stats" as its address). + letter capitalized (eg. b10-stats would have “Stats” as its address). The last one is process. It is the name of the process to be started. It defaults to the name of the component if not set, but you can use this to @@ -636,11 +635,11 @@ Chapter 3. Starting BIND10 with bind10 locking the sqlite database, if used. The configuration might be changed to something more convenient in future. -Chapter 4. Command channel +Chapter 4. Command channel The BIND 10 components use the b10-msgq message routing daemon to communicate with other BIND 10 components. The b10-msgq implements what is - called the "Command Channel". Processes intercommunicate by sending + called the “Command Channel”. Processes intercommunicate by sending messages on the command channel. Example messages include shutdown, get configurations, and set configurations. This Command Channel is not used for DNS message passing. It is used only to control and monitor the BIND @@ -650,7 +649,7 @@ Chapter 4. Command channel default, BIND 10 uses port 9912 for the b10-msgq service. It listens on 127.0.0.1. -Chapter 5. Configuration manager +Chapter 5. Configuration manager The configuration manager, b10-cfgmgr, handles all BIND 10 system configuration. It provides persistent storage for configuration, and @@ -662,7 +661,7 @@ Chapter 5. Configuration manager The administrator doesn't connect to it directly, but uses a user interface to communicate with the configuration manager via b10-cmdctl's - REST-ful interface. b10-cmdctl is covered in Chapter 6, Remote control + REST-ful interface. b10-cmdctl is covered in Chapter 6, Remote control daemon. Note @@ -686,10 +685,10 @@ Chapter 5. Configuration manager The configuration manager does not have any command line arguments. Normally it is not started manually, but is automatically started using - the bind10 master process (as covered in Chapter 3, Starting BIND10 with + the bind10 master process (as covered in Chapter 3, Starting BIND10 with bind10). -Chapter 6. Remote control daemon +Chapter 6. Remote control daemon Table of Contents @@ -702,7 +701,7 @@ Chapter 6. Remote control daemon When b10-cmdctl starts, it firsts asks b10-cfgmgr about what modules are running and what their configuration is (over the b10-msgq channel). Then - it will start listening on HTTPS for clients -- the user interface -- such + it will start listening on HTTPS for clients — the user interface — such as bindctl. b10-cmdctl directly sends commands (received from the user interface) to @@ -729,7 +728,7 @@ Chapter 6. Remote control daemon /usr/local/etc/bind10-devel/cmdctl-accounts.csv. This comma-delimited file lists the accounts with a user name, hashed password, and salt. (A sample file is at /usr/local/share/bind10-devel/cmdctl-accounts.csv. It contains - the user named "root" with the password "bind10".) + the user named “root” with the password “bind10”.) The administrator may create a user account with the b10-cmdctl-usermgr tool. @@ -740,14 +739,14 @@ Chapter 6. Remote control daemon connection is stateless and times out in 1200 seconds by default. This can be redefined by using the --idle-timeout command line argument. -6.1. Configuration specification for b10-cmdctl +6.1. Configuration specification for b10-cmdctl The configuration items for b10-cmdctl are: key_file cert_file accounts_file The control commands are: print_settings shutdown -Chapter 7. Control and configure user interface +Chapter 7. Control and configure user interface Note @@ -767,7 +766,7 @@ Chapter 7. Control and configure user interface b10-cfgmgr which then stores the details and relays (over a b10-msgq command channel) the configuration on to the specified module. -Chapter 8. Authoritative Server +Chapter 8. Authoritative Server Table of Contents @@ -789,10 +788,10 @@ Chapter 8. Authoritative Server DNSSEC. It supports IPv6. Normally it is started by the bind10 master process. -8.1. Server Configurations +8.1. Server Configurations b10-auth is configured via the b10-cfgmgr configuration manager. The - module name is "Auth". The configuration data items are: + module name is “Auth”. The configuration data items are: database_file This is an optional string to define the path to find the SQLite3 @@ -801,8 +800,8 @@ Chapter 8. Authoritative Server datasources datasources configures data sources. The list items include: type - to define the required data source type (such as "memory"); class - to optionally select the class (it defaults to "IN"); and zones to + to define the required data source type (such as “memory”); class + to optionally select the class (it defaults to “IN”); and zones to define the file path name, the filetype (e.g., sqlite3), and the origin (default domain). By default, this is empty. @@ -833,7 +832,7 @@ Chapter 8. Authoritative Server There are plans to solve the problem such that the server handles it by itself. But until it is actually implemented, it is - recommended to alter the configuration -- remove the wildcard + recommended to alter the configuration — remove the wildcard addresses and list all addresses explicitly. Then the server will answer on the same interface the request came on, preserving the correct address. @@ -848,9 +847,9 @@ Chapter 8. Authoritative Server loadzone loadzone tells b10-auth to load or reload a zone file. The arguments include: class which optionally defines the class (it - defaults to "IN"); origin is the domain name of the zone; and + defaults to “IN”); origin is the domain name of the zone; and datasrc optionally defines the type of datasource (it defaults to - "memory"). + “memory”). Note @@ -866,7 +865,7 @@ Chapter 8. Authoritative Server argument to select the process ID to stop. (Note that the BIND 10 boss process may restart this service if configured.) -8.2. Data Source Backends +8.2. Data Source Backends Note @@ -879,13 +878,13 @@ Chapter 8. Authoritative Server /usr/local/var/bind10-devel/zone.sqlite3. (The full path is what was defined at build configure time for --localstatedir. The default is /usr/local/var/.) This data file location may be changed by defining the - "database_file" configuration. + “database_file” configuration. - 8.2.1. In-memory Data Source + 8.2.1. In-memory Data Source The following commands to bindctl provide an example of configuring an - in-memory data source containing the "example.com" zone with the zone file - named "example.com.zone": + in-memory data source containing the “example.com” zone with the zone file + named “example.com.zone”: > config add Auth/datasources > config set Auth/datasources[0]/type "memory" @@ -897,11 +896,11 @@ Chapter 8. Authoritative Server The authoritative server will begin serving it immediately after it is loaded. - 8.2.2. In-memory Data Source With SQLite3 Backend + 8.2.2. In-memory Data Source With SQLite3 Backend The following commands to bindctl provide an example of configuring an - in-memory data source containing the "example.org" zone with a SQLite3 - backend file named "example.org.sqlite3": + in-memory data source containing the “example.org” zone with a SQLite3 + backend file named “example.org.sqlite3”: > config add Auth/datasources > config set Auth/datasources[1]/type "memory" @@ -914,14 +913,14 @@ Chapter 8. Authoritative Server The authoritative server will begin serving it immediately after it is loaded. - 8.2.3. Reloading an In-memory Data Source + 8.2.3. Reloading an In-memory Data Source Use the Auth loadzone command in bindctl to reload a changed master file into memory; for example: > Auth loadzone origin="example.com" - 8.2.4. Disabling In-memory Data Sources + 8.2.4. Disabling In-memory Data Sources By default, the memory data source is disabled; it must be configured explicitly. To disable all the in-memory zones, specify a null list for @@ -938,7 +937,7 @@ Chapter 8. Authoritative Server (Replace the list number(s) in datasources[0] and/or zones[0] for the relevant zone as needed.) -8.3. Loading Master Zones Files +8.3. Loading Master Zones Files RFC 1035 style DNS master zone files may imported into a BIND 10 SQLite3 data source by using the b10-loadzone utility. @@ -969,7 +968,7 @@ Chapter 8. Authoritative Server If you reload a zone already existing in the database, all records from that prior zone disappear and a whole new set appears. -Chapter 9. Incoming Zone Transfers +Chapter 9. Incoming Zone Transfers Table of Contents @@ -987,13 +986,13 @@ Chapter 9. Incoming Zone Transfers started by bind10. When received, the zone is stored in the corresponding BIND 10 data source, and its records can be served by b10-auth. In combination with b10-zonemgr (for automated SOA checks), this allows the - BIND 10 server to provide "secondary" service. + BIND 10 server to provide “secondary” service. The b10-xfrin process supports both AXFR and IXFR. Due to some implementation limitations of the current development release, however, it only tries AXFR by default, and care should be taken to enable IXFR. -9.1. Configuration for Incoming Zone Transfers +9.1. Configuration for Incoming Zone Transfers In practice, you need to specify a list of secondary zones to enable incoming zone transfers for these zones (you can still trigger a zone @@ -1010,7 +1009,7 @@ Chapter 9. Incoming Zone Transfers (We assume there has been no zone configuration before). -9.2. Enabling IXFR +9.2. Enabling IXFR As noted above, b10-xfrin uses AXFR for zone transfers by default. To enable IXFR for zone transfers for a particular zone, set the use_ixfr @@ -1033,7 +1032,7 @@ Chapter 9. Incoming Zone Transfers be implemented in a near future version, at which point we will enable IXFR by default. -9.3. Secondary Manager +9.3. Secondary Manager The b10-zonemgr process is started by bind10. It keeps track of SOA refresh, retry, and expire timers and other details for BIND 10 to perform @@ -1059,14 +1058,14 @@ Chapter 9. Incoming Zone Transfers for it), b10-zonemgr will automatically tell b10-xfrin to transfer the zone in. -9.4. Trigger an Incoming Zone Transfer Manually +9.4. Trigger an Incoming Zone Transfer Manually To manually trigger a zone transfer to retrieve a remote zone, you may use the bindctl utility. For example, at the bindctl prompt run: > Xfrin retransfer zone_name="foo.example.org" master=192.0.2.99 -9.5. Incoming Transfers with In-memory Datasource +9.5. Incoming Transfers with In-memory Datasource In the case of an incoming zone transfer, the received zone is first stored in the corresponding BIND 10 datasource. In case the secondary zone @@ -1076,9 +1075,9 @@ Chapter 9. Incoming Zone Transfers The administrator doesn't have to do anything for b10-auth to serve the new version of the zone, except for the configuration such as the one - described in Section 8.2.2, "In-memory Data Source With SQLite3 Backend". + described in Section 8.2.2, “In-memory Data Source With SQLite3 Backend”. -Chapter 10. Outbound Zone Transfers +Chapter 10. Outbound Zone Transfers The b10-xfrout process is started by bind10. When the b10-auth authoritative DNS server receives an AXFR or IXFR request, b10-auth @@ -1127,7 +1126,7 @@ Chapter 10. Outbound Zone Transfers The way to specify zone specific configuration (ACLs, etc) is likely to be changed. -Chapter 11. Dynamic DNS Update +Chapter 11. Dynamic DNS Update Table of Contents @@ -1149,8 +1148,8 @@ Chapter 11. Dynamic DNS Update etc). If the zone has been changed as a result, it will internally notify b10-xfrout so that other secondary servers will be notified via the DNS notify protocol. In addition, if b10-auth serves the updated zone from its - in-memory cache (as described in Section 8.2.2, "In-memory Data Source - With SQLite3 Backend"), b10-ddns will also notify b10-auth so that + in-memory cache (as described in Section 8.2.2, “In-memory Data Source + With SQLite3 Backend”), b10-ddns will also notify b10-auth so that b10-auth will re-cache the updated zone content. The b10-ddns component supports requests over both UDP and TCP, and both @@ -1172,7 +1171,7 @@ Chapter 11. Dynamic DNS Update But right now it's considered a lower priority task and there is no specific plan of implementing this feature. -11.1. Enabling Dynamic Update +11.1. Enabling Dynamic Update First off, it must be made sure that a few components on which b10-ddns depends are configured to run, which are b10-auth and b10-zonemgr. In @@ -1187,7 +1186,7 @@ Chapter 11. Dynamic DNS Update data source storing the zone data be writable. In the current implementation this means the zone must be stored in an SQLite3-based data source. Also, right now, the b10-ddns component configures itself with the - data source referring to the "database_file" configuration parameter of + data source referring to the “database_file” configuration parameter of b10-auth. So this information must be configured correctly before starting b10-ddns. @@ -1221,13 +1220,13 @@ Chapter 11. Dynamic DNS Update because b10-ddns would start and work without specifying it. But for it to shutdown gracefully this parameter should also be specified. -11.2. Access Control +11.2. Access Control By default b10-ddns rejects any update requests from any clients by returning a response with an RCODE of REFUSED. To allow updates to take effect, an access control rule (called update ACL) with a policy allowing updates must explicitly be configured. Update ACL must be configured per - zone basis in the "zones" configuration parameter of b10-ddns. This is a + zone basis in the “zones” configuration parameter of b10-ddns. This is a list of per-zone configurations regarding DDNS. Each list element consists of the following parameters: @@ -1235,7 +1234,7 @@ Chapter 11. Dynamic DNS Update The zone's origin name class - The RR class of the zone (normally "IN", and in that case can be + The RR class of the zone (normally “IN”, and in that case can be omitted in configuration) update_acl @@ -1246,7 +1245,7 @@ Chapter 11. Dynamic DNS Update In general, an update ACL rule that allows an update request should be configured with a TSIG key. This is an example update ACL that allows - updates to the zone named "example.org" of RR class "IN" from clients that + updates to the zone named “example.org” of RR class “IN” from clients that send requests signed with a TSIG whose key name is "key.example.org" (and refuses all others): @@ -1257,7 +1256,7 @@ Chapter 11. Dynamic DNS Update > config add DDNS/zones[0]/update_acl {"action": "ACCEPT", "key": "key.example.org"} > config commit - The TSIG key must be configured system wide (see Chapter 10, Outbound Zone + The TSIG key must be configured system wide (see Chapter 10, Outbound Zone Transfers.) Multiple rules can be specified in the ACL, and an ACL rule can consist of @@ -1293,7 +1292,7 @@ Chapter 11. Dynamic DNS Update which is rejecting any requests in the case of b10-ddns. Other actions than "ACCEPT", namely "REJECT" and "DROP", can be used, too. - See Chapter 12, Recursive Name Server about their effects. + See Chapter 12, Recursive Name Server about their effects. Currently update ACL can only control updates per zone basis; it's not possible to specify access control with higher granularity such as for @@ -1313,7 +1312,7 @@ Chapter 11. Dynamic DNS Update clients. There have been other troubles that could have been avoided if the ACL could be checked before the prerequisite check. -11.3. Miscellaneous Operational Issues +11.3. Miscellaneous Operational Issues Unlike BIND 9, BIND 10 currently does not support automatic resigning of DNSSEC-signed zone when it's updated via DDNS. It could be possible to @@ -1322,20 +1321,20 @@ Chapter 11. Dynamic DNS Update operation. In general, it's not advisable to allow DDNS for a signed zone at this moment. - Also unlike BIND 9, it's currently not possible to "freeze" a zone + Also unlike BIND 9, it's currently not possible to “freeze” a zone temporarily in order to suspend DDNS while you manually update the zone. If you need to make manual updates to a dynamic zone, you'll need to temporarily reject any updates to the zone via the update ACLs. Dynamic updates are only applicable to primary zones. In order to avoid updating secondary zones via DDNS requests, b10-ddns refers to the - "secondary_zones" configuration of b10-zonemgr. Zones listed in - "secondary_zones" will never be updated via DDNS regardless of the update + “secondary_zones” configuration of b10-zonemgr. Zones listed in + “secondary_zones” will never be updated via DDNS regardless of the update ACL configuration; b10-ddns will return a response with an RCODE of NOTAUTH as specified in RFC 2136. If you have a "conceptual" secondary zone whose content is a copy of some external source but is not updated via the standard zone transfers and therefore not listed in - "secondary_zones", be careful not to allow DDNS for the zone; it would be + “secondary_zones”, be careful not to allow DDNS for the zone; it would be quite likely to lead to inconsistent state between different servers. Normally this should not be a problem because the default update ACL rejects any update requests, but you may want to take an extra care about @@ -1346,7 +1345,7 @@ Chapter 11. Dynamic DNS Update can be retrieved in the form of outbound IXFR. This is done automatically; it does not require specific configuration to make this possible. -Chapter 12. Recursive Name Server +Chapter 12. Recursive Name Server Table of Contents @@ -1377,26 +1376,26 @@ Chapter 12. Recursive Name Server > config set Resolver/listen_on[2]/port 53 > config commit - (Replace the "2" as needed; run "config show Resolver/listen_on" if + (Replace the “2” as needed; run “config show Resolver/listen_on” if needed.) -12.1. Access Control +12.1. Access Control By default, the b10-resolver daemon only accepts DNS queries from the localhost (127.0.0.1 and ::1). The Resolver/query_acl configuration may be used to reject, drop, or allow specific IPs or networks. This configuration list is first match. - The configuration's action item may be set to "ACCEPT" to allow the - incoming query, "REJECT" to respond with a DNS REFUSED return code, or - "DROP" to ignore the query without any response (such as a blackhole). For + The configuration's action item may be set to “ACCEPT” to allow the + incoming query, “REJECT” to respond with a DNS REFUSED return code, or + “DROP” to ignore the query without any response (such as a blackhole). For more information, see the respective debugging messages: RESOLVER_QUERY_ACCEPTED, RESOLVER_QUERY_REJECTED, and RESOLVER_QUERY_DROPPED. The required configuration's from item is set to an IPv4 or IPv6 address, addresses with an network mask, or to the special lowercase keywords - "any6" (for any IPv6 address) or "any4" (for any IPv4 address). + “any6” (for any IPv6 address) or “any4” (for any IPv4 address). For example to allow the 192.168.1.0/24 network to use your recursive name server, at the bindctl prompt run: @@ -1406,14 +1405,14 @@ Chapter 12. Recursive Name Server > config set Resolver/query_acl[2]/from "192.168.1.0/24" > config commit - (Replace the "2" as needed; run "config show Resolver/query_acl" if + (Replace the “2” as needed; run “config show Resolver/query_acl” if needed.) Note This prototype access control configuration syntax may be changed. -12.2. Forwarding +12.2. Forwarding To enable forwarding, the upstream address and port must be configured to forward queries to, such as: @@ -1429,7 +1428,7 @@ Chapter 12. Recursive Name Server > config set Resolver/forward_addresses [] > config commit -Chapter 13. DHCPv4 Server +Chapter 13. DHCPv4 Server Table of Contents @@ -1449,7 +1448,7 @@ Chapter 13. DHCPv4 Server clients. Even though principles of both DHCPv4 and DHCPv6 are somewhat similar, these are two radically different protocols. BIND10 offers server implementations for both DHCPv4 and DHCPv6. This chapter is about DHCP for - IPv4. For a description of the DHCPv6 server, see Chapter 14, DHCPv6 + IPv4. For a description of the DHCPv6 server, see Chapter 14, DHCPv6 Server. The DHCPv4 server component is currently under intense development. You @@ -1457,7 +1456,7 @@ Chapter 13. DHCPv4 Server developers mailing list. The DHCPv4 and DHCPv6 components in BIND10 architecture are internally - code named "Kea". + code named “Kea”. Note @@ -1465,43 +1464,44 @@ Chapter 13. DHCPv4 Server servers. That means that while they are capable of performing DHCP configuration, they are not fully functional yet. In particular, neither has functional lease databases. This means that they will assign the same, - fixed, hardcoded addresses to any client that will ask. See Section 13.4, - "DHCPv4 Server Limitations" and Section 14.4, "DHCPv6 Server Limitations" + fixed, hardcoded addresses to any client that will ask. See Section 13.4, + “DHCPv4 Server Limitations” and Section 14.4, “DHCPv6 Server Limitations” for detailed description. -13.1. DHCPv4 Server Usage +13.1. DHCPv4 Server Usage BIND10 provides the DHCPv4 server component since December 2011. It is a skeleton server and can be described as an early prototype that is not fully functional yet. It is mature enough to conduct first tests in lab - environment, but it has significant limitations. See Section 13.4, "DHCPv4 - Server Limitations" for details. + environment, but it has significant limitations. See Section 13.4, “DHCPv4 + Server Limitations” for details. - The DHCPv4 server is implemented as b10-dhcp4 daemon. As it is not - configurable yet, it is fully autonomous, that is it does not interact - with b10-cfgmgr. To start DHCPv4 server, simply input: + b10-dhcp4 is a BIND10 component and is being run under BIND10 framework. + To add a DHCPv4 process to the set of running BIND10 services, you can use + following commands in bindctl: - #cd src/bin/dhcp4 - #./b10-dhcp4 + > config add Boss/components b10-dhcp4 + > config set Boss/components/b10-dhcp4/kind dispensable + > config commit - Depending on your installation, b10-dhcp4 binary may reside in - src/bin/dhcp4 in your source code directory, in /usr/local/bin/b10-dhcp4 - or other directory you specified during compilation. At start, the server - will detect available network interfaces and will attempt to open UDP - sockets on all interfaces that are up, running, are not loopback, and have - IPv4 address assigned. The server will then listen to incoming traffic. - Currently supported client messages are DISCOVER and REQUEST. The server - will respond to them with OFFER and ACK, respectively. Since the DHCPv4 - server opens privileged ports, it requires root access. Make sure you run - this daemon as root. + To shutdown running b10-dhcp4, please use the following command: - Note + > Dhcp4 shutdown - Integration with bind10 is planned. Ultimately, b10-dhcp4 will not be - started directly, but rather via bind10. Please be aware of this planned - change. + or -13.2. DHCPv4 Server Configuration + > config remove Boss/components b10-dhcp4 + > config commit + + During start-up the server will detect available network interfaces and + will attempt to open UDP sockets on all interfaces that are up, running, + are not loopback, and have IPv4 address assigned. The server will then + listen to incoming traffic. Currently supported client messages are + DISCOVER and REQUEST. The server will respond to them with OFFER and ACK, + respectively. Since the DHCPv4 server opens privileged ports, it requires + root access. Make sure you run this daemon as root. + +13.2. DHCPv4 Server Configuration The DHCPv4 server does not have a lease database implemented yet nor any support for configuration, so every time the same set of configuration @@ -1522,60 +1522,55 @@ Chapter 13. DHCPv4 Server Lease database and configuration support is planned for 2012. -13.3. Supported standards +13.3. Supported standards The following standards and draft standards are currently supported: - o RFC2131: Supported messages are DISCOVER, OFFER, REQUEST, and ACK. - o RFC2132: Supported options are: PAD (0), END(255), Message Type(53), + o RFC2131: Supported messages are DISCOVER, OFFER, REQUEST, and ACK. + o RFC2132: Supported options are: PAD (0), END(255), Message Type(53), DHCP Server Identifier (54), Domain Name (15), DNS Servers (6), IP Address Lease Time (51), Subnet mask (1), and Routers (3). -13.4. DHCPv4 Server Limitations +13.4. DHCPv4 Server Limitations These are the current limitations of the DHCPv4 server software. Most of them are reflections of the early stage of development and should be - treated as "not implemented yet", rather than actual limitations. + treated as “not implemented yet”, rather than actual limitations. - o During initial IPv4 node configuration, the server is expected to send + o During initial IPv4 node configuration, the server is expected to send packets to a node that does not have IPv4 address assigned yet. The server requires certain tricks (or hacks) to transmit such packets. This is not implemented yet, therefore DHCPv4 server supports relayed traffic only (that is, normal point to point communication). - o b10-dhcp4 provides a single, fixed, hardcoded lease to any client that + o b10-dhcp4 provides a single, fixed, hardcoded lease to any client that asks. There is no lease manager implemented. If two clients request addresses, they will both get the same fixed address. - o b10-dhcp4 does not support any configuration mechanisms yet. The whole + o b10-dhcp4 does not support any configuration mechanisms yet. The whole configuration is currently hardcoded. The only way to tweak - configuration is to directly modify source code. See see Section 13.2, - "DHCPv4 Server Configuration" for details. - o Upon start, the server will open sockets on all interfaces that are - not loopback, are up and running and have IPv4 address. Support for - multiple interfaces is not coded in reception routines yet, so if you - are running this code on a machine that has many interfaces and - b10-dhcp4 happens to listen on wrong interface, the easiest way to - work around this problem is to turn down other interfaces. This - limitation will be fixed shortly. - o PRL (Parameter Request List, a list of options requested by a client) + configuration is to directly modify source code. See see Section 13.2, + “DHCPv4 Server Configuration” for details. + o Upon start, the server will open sockets on all interfaces that are + not loopback, are up and running and have IPv4 address. + o PRL (Parameter Request List, a list of options requested by a client) is currently ignored and server assigns DNS SERVER and DOMAIN NAME options. - o b10-dhcp4 does not support BOOTP. That is a design choice. This + o b10-dhcp4 does not support BOOTP. That is a design choice. This limitation is permanent. If you have legacy nodes that can't use DHCP and require BOOTP support, please use latest version of ISC DHCP http://www.isc.org/software/dhcp. - o Interface detection is currently working on Linux only. See - Section 15.1, "Interface detection" for details. - o b10-dhcp4 does not verify that assigned address is unused. According + o Interface detection is currently working on Linux only. See + Section 15.1, “Interface detection” for details. + o b10-dhcp4 does not verify that assigned address is unused. According to RFC2131, the allocating server should verify that address is no used by sending ICMP echo request. - o Address renewal (RENEW), rebinding (REBIND), confirmation (CONFIRM), + o Address renewal (RENEW), rebinding (REBIND), confirmation (CONFIRM), duplication report (DECLINE) and release (RELEASE) are not supported yet. - o DNS Update is not supported yet. - o -v (verbose) command line option is currently the default, and cannot + o DNS Update is not supported yet. + o -v (verbose) command line option is currently the default, and cannot be disabled. -Chapter 14. DHCPv6 Server +Chapter 14. DHCPv6 Server Table of Contents @@ -1590,14 +1585,14 @@ Chapter 14. DHCPv6 Server Dynamic Host Configuration Protocol for IPv6 (DHCPv6) is specified in RFC3315. BIND10 provides DHCPv6 server implementation that is described in this chapter. For a description of the DHCPv4 server implementation, see - Chapter 13, DHCPv4 Server. + Chapter 13, DHCPv4 Server. The DHCPv6 server component is currently under intense development. You may want to check out BIND10 DHCP (Kea) wiki and recent posts on BIND10 developers mailing list. The DHCPv4 and DHCPv6 components in BIND10 architecture are internally - code named "Kea". + code named “Kea”. Note @@ -1605,43 +1600,45 @@ Chapter 14. DHCPv6 Server servers. That means that while they are capable of performing DHCP configuration, they are not fully functional yet. In particular, neither has functional lease databases. This means that they will assign the same, - fixed, hardcoded addresses to any client that will ask. See Section 13.4, - "DHCPv4 Server Limitations" and Section 14.4, "DHCPv6 Server Limitations" + fixed, hardcoded addresses to any client that will ask. See Section 13.4, + “DHCPv4 Server Limitations” and Section 14.4, “DHCPv6 Server Limitations” for detailed description. -14.1. DHCPv6 Server Usage +14.1. DHCPv6 Server Usage BIND10 provides the DHCPv6 server component since September 2011. It is a skeleton server and can be described as an early prototype that is not fully functional yet. It is mature enough to conduct first tests in lab - environment, but it has significant limitations. See Section 14.4, "DHCPv6 - Server Limitations" for details. + environment, but it has significant limitations. See Section 14.4, “DHCPv6 + Server Limitations” for details. - The DHCPv6 server is implemented as b10-dhcp6 daemon. As it is not - configurable yet, it is fully autonomous, that is it does not interact - with b10-cfgmgr. To start DHCPv6 server, simply input: + b10-dhcp6 is a BIND10 component and is being run under BIND10 framework. + To add a DHCPv6 process to the set of running BIND10 services, you can use + following commands in bindctl: - #cd src/bin/dhcp6 - #./b10-dhcp6 + > config add Boss/components b10-dhcp6 + > config set Boss/components/b10-dhcp6/kind dispensable + > config commit - Depending on your installation, b10-dhcp6 binary may reside in - src/bin/dhcp6 in your source code directory, in /usr/local/bin/b10-dhcp6 - or other directory you specified during compilation. At start, server will - detect available network interfaces and will attempt to open UDP sockets - on all interfaces that are up, running, are not loopback, are - multicast-capable, and have IPv6 address assigned. The server will then - listen to incoming traffic. Currently supported client messages are - SOLICIT and REQUEST. The server will respond to them with ADVERTISE and - REPLY, respectively. Since the DHCPv6 server opens privileged ports, it - requires root access. Make sure you run this daemon as root. + To shutdown running b10-dhcp6, please use the following command: - Note + > Dhcp6 shutdown - Integration with bind10 is planned. Ultimately, b10-dhcp6 will not be - started directly, but rather via bind10. Please be aware of this planned - change. + or -14.2. DHCPv6 Server Configuration + > config remove Boss/components b10-dhcp6 + > config commit + + During start-up the server will detect available network interfaces and + will attempt to open UDP sockets on all interfaces that are up, running, + are not loopback, are multicast-capable, and have IPv6 address assigned. + The server will then listen to incoming traffic. Currently supported + client messages are SOLICIT and REQUEST. The server will respond to them + with ADVERTISE and REPLY, respectively. Since the DHCPv6 server opens + privileged ports, it requires root access. Make sure you run this daemon + as root. + +14.2. DHCPv6 Server Configuration The DHCPv6 server does not have lease database implemented yet or any support for configuration, so every time the same set of configuration @@ -1649,7 +1646,7 @@ Chapter 14. DHCPv6 Server At this stage of development, the only way to alter server configuration is to tweak its source code. To do so, please edit - src/bin/dhcp6/dhcp6_srv.cc file and modify following parameters and + src/bin/dhcp6/dhcp6_srv.cc file, modify the following parameters and recompile: const std::string HARDCODED_LEASE = "2001:db8:1::1234:abcd"; @@ -1661,50 +1658,50 @@ Chapter 14. DHCPv6 Server Lease database and configuration support is planned for 2012. -14.3. Supported DHCPv6 Standards +14.3. Supported DHCPv6 Standards The following standards and draft standards are currently supported: - o RFC3315: Supported messages are SOLICIT, ADVERTISE, REQUEST, and + o RFC3315: Supported messages are SOLICIT, ADVERTISE, REQUEST, and REPLY. Supported options are SERVER_ID, CLIENT_ID, IA_NA, and IAADDRESS. - o RFC3646: Supported option is DNS_SERVERS. + o RFC3646: Supported option is DNS_SERVERS. -14.4. DHCPv6 Server Limitations +14.4. DHCPv6 Server Limitations These are the current limitations of the DHCPv6 server software. Most of them are reflections of the early stage of development and should be - treated as "not implemented yet", rather than actual limitations. + treated as “not implemented yet”, rather than actual limitations. - o Relayed traffic is not supported. - o b10-dhcp6 provides a single, fixed, hardcoded lease to any client that + o Relayed traffic is not supported. + o b10-dhcp6 provides a single, fixed, hardcoded lease to any client that asks. There is no lease manager implemented. If two clients request addresses, they will both get the same fixed address. - o b10-dhcp6 does not support any configuration mechanisms yet. The whole + o b10-dhcp6 does not support any configuration mechanisms yet. The whole configuration is currently hardcoded. The only way to tweak - configuration is to directly modify source code. See see Section 14.2, - "DHCPv6 Server Configuration" for details. - o Upon start, the server will open sockets on all interfaces that are + configuration is to directly modify source code. See see Section 14.2, + “DHCPv6 Server Configuration” for details. + o Upon start, the server will open sockets on all interfaces that are not loopback, are up, running and are multicast capable and have IPv6 address. Support for multiple interfaces is not coded in reception routines yet, so if you are running this code on a machine that has many interfaces and b10-dhcp6 happens to listen on wrong interface, the easiest way to work around this problem is to turn down other interfaces. This limitation will be fixed shortly. - o ORO (Option Request Option, a list of options requested by a client) + o ORO (Option Request Option, a list of options requested by a client) is currently ignored and server assigns DNS SERVER option. - o Temporary addresses are not supported yet. - o Prefix delegation is not supported yet. - o Address renewal (RENEW), rebinding (REBIND), confirmation (CONFIRM), + o Temporary addresses are not supported yet. + o Prefix delegation is not supported yet. + o Address renewal (RENEW), rebinding (REBIND), confirmation (CONFIRM), duplication report (DECLINE) and release (RELEASE) are not supported yet. - o DNS Update is not supported yet. - o Interface detection is currently working on Linux only. See - Section 15.1, "Interface detection" for details. - o -v (verbose) command line option is currently the default, and cannot + o DNS Update is not supported yet. + o Interface detection is currently working on Linux only. See + Section 15.1, “Interface detection” for details. + o -v (verbose) command line option is currently the default, and cannot be disabled. -Chapter 15. libdhcp++ library +Chapter 15. libdhcp++ library Table of Contents @@ -1722,33 +1719,22 @@ Chapter 15. libdhcp++ library is designed to be portable, universal library useful for any kind of DHCP-related software. -15.1. Interface detection +15.1. Interface detection Both DHCPv4 and DHCPv6 components share network interface detection routines. Interface detection is currently only supported on Linux systems. - For non-Linux systems, there is currently stub implementation provided. As - DHCP servers need to know available addresses, there is a simple mechanism - implemented to provide that information. User is expected to create - interfaces.txt file. Format of this file is simple. It contains list of - interfaces along with available address on each interface. This mechanism - is temporary and is going to be removed as soon as interface detection - becomes available on non-Linux systems. Here is an example of the - interfaces.txt file: + For non-Linux systems, there is currently stub implementation provided. + Interface manager detects loopback interfaces only as their name (lo or + lo0) can be easily predicted. Please contact BIND10 development team if + you are interested in running DHCP components on systems other than Linux. - # For DHCPv6, please specify link-local address (starts with fe80::) - # If in doubt, check output of 'ifconfig -a' command. - eth0 fe80::21e:8cff:fe9b:7349 - - # For DHCPv4, please use following format: - #eth0 192.0.2.5 - -15.2. DHCPv4/DHCPv6 packet handling +15.2. DHCPv4/DHCPv6 packet handling TODO: Describe packet handling here, with pointers to wiki -Chapter 16. Statistics +Chapter 16. Statistics The b10-stats process is started by bind10. It periodically collects statistics data from various modules and aggregates it. @@ -1780,7 +1766,7 @@ Chapter 16. Statistics } -Chapter 17. Logging +Chapter 17. Logging Table of Contents @@ -1794,13 +1780,13 @@ Chapter 17. Logging 17.2. Logging Message Format -17.1. Logging configuration +17.1. Logging configuration The logging system in BIND 10 is configured through the Logging module. All BIND 10 modules will look at the configuration in Logging to see what should be logged and to where. - 17.1.1. Loggers + 17.1.1. Loggers Within BIND 10, a message is logged through a component called a "logger". Different parts of BIND 10 log messages through different loggers, and @@ -1813,78 +1799,78 @@ Chapter 17. Logging (the component that is generating the messages), the severity (what to log), and the output_options (where to log). - 17.1.1.1. name (string) + 17.1.1.1. name (string) Each logger in the system has a name, the name being that of the component using it to log messages. For instance, if you want to configure logging - for the resolver module, you add an entry for a logger named "Resolver". + for the resolver module, you add an entry for a logger named “Resolver”. This configuration will then be used by the loggers in the Resolver module, and all the libraries used by it. If you want to specify logging for one specific library within the module, you set the name to module.library. For example, the logger used by the - nameserver address store component has the full name of "Resolver.nsas". + nameserver address store component has the full name of “Resolver.nsas”. If there is no entry in Logging for a particular library, it will use the configuration given for the module. To illustrate this, suppose you want the cache library to log messages of severity DEBUG, and the rest of the resolver code to log messages of severity INFO. To achieve this you specify two loggers, one with the name - "Resolver" and severity INFO, and one with the name "Resolver.cache" with + “Resolver” and severity INFO, and one with the name “Resolver.cache” with severity DEBUG. As there are no entries for other libraries (e.g. the - nsas), they will use the configuration for the module ("Resolver"), so + nsas), they will use the configuration for the module (“Resolver”), so giving the desired behavior. - One special case is that of a module name of "*" (asterisks), which is + One special case is that of a module name of “*” (asterisks), which is interpreted as any module. You can set global logging options by using this, including setting the logging configuration for a library that is - used by multiple modules (e.g. "*.config" specifies the configuration + used by multiple modules (e.g. “*.config” specifies the configuration library code in whatever module is using it). If there are multiple logger specifications in the configuration that might match a particular logger, the specification with the more specific logger name takes precedence. For example, if there are entries for for - both "*" and "Resolver", the resolver module -- and all libraries it uses - -- will log messages according to the configuration in the second entry - ("Resolver"). All other modules will use the configuration of the first - entry ("*"). If there was also a configuration entry for "Resolver.cache", + both “*” and “Resolver”, the resolver module — and all libraries it uses — + will log messages according to the configuration in the second entry + (“Resolver”). All other modules will use the configuration of the first + entry (“*”). If there was also a configuration entry for “Resolver.cache”, the cache library within the resolver would use that in preference to the - entry for "Resolver". + entry for “Resolver”. One final note about the naming. When specifying the module name within a logger, use the name of the module as specified in bindctl, e.g. - "Resolver" for the resolver module, "Xfrout" for the xfrout module, etc. + “Resolver” for the resolver module, “Xfrout” for the xfrout module, etc. When the message is logged, the message will include the name of the logger generating the message, but with the module name replaced by the name of the process implementing the module (so for example, a message - generated by the "Auth.cache" logger will appear in the output with a - logger name of "b10-auth.cache"). + generated by the “Auth.cache” logger will appear in the output with a + logger name of “b10-auth.cache”). - 17.1.1.2. severity (string) + 17.1.1.2. severity (string) This specifies the category of messages logged. Each message is logged with an associated severity which may be one of the following (in descending order of severity): - o FATAL - o ERROR - o WARN - o INFO - o DEBUG + o FATAL + o ERROR + o WARN + o INFO + o DEBUG When the severity of a logger is set to one of these values, it will only log messages of that severity, and the severities above it. The severity may also be set to NONE, in which case all messages from that logger are inhibited. - 17.1.1.3. output_options (list) + 17.1.1.3. output_options (list) Each logger can have zero or more output_options. These specify where log messages are sent to. These are explained in detail below. The other options for a logger are: - 17.1.1.4. debuglevel (integer) + 17.1.1.4. debuglevel (integer) When a logger's severity is set to DEBUG, this value specifies what debug messages should be printed. It ranges from 0 (least verbose) to 99 (most @@ -1892,80 +1878,80 @@ Chapter 17. Logging If severity for the logger is not DEBUG, this value is ignored. - 17.1.1.5. additive (true or false) + 17.1.1.5. additive (true or false) If this is true, the output_options from the parent will be used. For - example, if there are two loggers configured; "Resolver" and - "Resolver.cache", and additive is true in the second, it will write the - log messages not only to the destinations specified for "Resolver.cache", + example, if there are two loggers configured; “Resolver” and + “Resolver.cache”, and additive is true in the second, it will write the + log messages not only to the destinations specified for “Resolver.cache”, but also to the destinations as specified in the output_options in the - logger named "Resolver". + logger named “Resolver”. - 17.1.2. Output Options + 17.1.2. Output Options The main settings for an output option are the destination and a value called output, the meaning of which depends on the destination that is set. - 17.1.2.1. destination (string) + 17.1.2.1. destination (string) The destination is the type of output. It can be one of: - o console - o file - o syslog + o console + o file + o syslog - 17.1.2.2. output (string) + 17.1.2.2. output (string) Depending on what is set as the output destination, this value is interpreted as follows: - destination is "console" + destination is “console” - The value of output must be one of "stdout" (messages printed to - standard output) or "stderr" (messages printed to standard error). + The value of output must be one of “stdout” (messages printed to + standard output) or “stderr” (messages printed to standard error). - Note: if output is set to "stderr" and a lot of messages are + Note: if output is set to “stderr” and a lot of messages are produced in a short time (e.g. if the logging level is set to DEBUG), you may occasionally see some messages jumbled up together. This is due to a combination of the way that messages are written to the screen and the unbuffered nature of the standard error stream. If this occurs, it is recommended that - output be set to "stdout". + output be set to “stdout”. - destination is "file" + destination is “file” The value of output is interpreted as a file name; log messages will be appended to this file. - destination is "syslog" + destination is “syslog” The value of output is interpreted as the syslog facility (e.g. local0) that should be used for log messages. The other options for output_options are: - 17.1.2.2.1. flush (true of false) + 17.1.2.2.1. flush (true of false) Flush buffers after each log message. Doing this will reduce performance but will ensure that if the program terminates abnormally, all messages up to the point of termination are output. - 17.1.2.2.2. maxsize (integer) + 17.1.2.2.2. maxsize (integer) Only relevant when destination is file, this is maximum file size of output files in bytes. When the maximum size is reached, the file is renamed and a new file opened. (For example, a ".1" is appended to the - name -- if a ".1" file exists, it is renamed ".2", etc.) + name — if a ".1" file exists, it is renamed ".2", etc.) If this is 0, no maximum file size is used. - 17.1.2.2.3. maxver (integer) + 17.1.2.2.3. maxver (integer) Maximum number of old log files to keep around when rolling the output - file. Only relevant when destination is "file". + file. Only relevant when destination is “file”. - 17.1.3. Example session + 17.1.3. Example session In this example we want to set the global logging to write to the file /var/log/my_bind10.log, at severity WARN. We want the authoritative server @@ -2064,9 +2050,9 @@ Chapter 17. Logging > config remove Logging/loggers[1] > config commit - And every module will now be using the values from the logger named "*". + And every module will now be using the values from the logger named “*”. -17.2. Logging Message Format +17.2. Logging Message Format Each message written by BIND 10 to the configured logging destinations comprises a number of components that identify the origin of the message diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml index 1bdc0053f5..48862c4858 100644 --- a/doc/guide/bind10-guide.xml +++ b/doc/guide/bind10-guide.xml @@ -2455,7 +2455,7 @@ then change those defaults with config set Resolver/forward_addresses[0]/address > config commit - At start, the server will detect available network interfaces + During start-up the server will detect available network interfaces and will attempt to open UDP sockets on all interfaces that are up, running, are not loopback, and have IPv4 address assigned. @@ -2465,17 +2465,8 @@ then change those defaults with config set Resolver/forward_addresses[0]/address will respond to them with OFFER and ACK, respectively. Since the DHCPv4 server opens privileged ports, it requires root - access. Make sure you run this daemon as root. - - - - Integration with bind10 is - planned. Ultimately, b10-dhcp4 will not - be started directly, but rather via - bind10. Please be aware of this planned - change. - - + access. Make sure you run this daemon as root. + @@ -2640,22 +2631,25 @@ const std::string HARDCODED_SERVER_ID = "192.0.2.1"; - The DHCPv6 server is implemented as b10-dhcp6 - daemon. As it is not configurable yet, it is fully autonomous, - that is it does not interact with b10-cfgmgr. - To start DHCPv6 server, simply input: + b10-dhcp6 is a BIND10 component and is being + run under BIND10 framework. To add a DHCPv6 process to the set of running + BIND10 services, you can use following commands in bindctl: + > config add Boss/components b10-dhcp6 +> config set Boss/components/b10-dhcp6/kind dispensable +> config commit + - -#cd src/bin/dhcp6 -#./b10-dhcp6 - + + To shutdown running b10-dhcp6, please use the + following command: + > Dhcp6 shutdown + or + > config remove Boss/components b10-dhcp6 +> config commit + - Depending on your installation, b10-dhcp6 - binary may reside in src/bin/dhcp6 in your source code - directory, in /usr/local/bin/b10-dhcp6 or other directory - you specified during compilation. - - At start, server will detect available network interfaces + + During start-up the server will detect available network interfaces and will attempt to open UDP sockets on all interfaces that are up, running, are not loopback, are multicast-capable, and have IPv6 address assigned. @@ -2668,16 +2662,6 @@ const std::string HARDCODED_SERVER_ID = "192.0.2.1"; access. Make sure you run this daemon as root. - - - Integration with bind10 is - planned. Ultimately, b10-dhcp6 will not - be started directly, but rather via - bind10. Please be aware of this planned - change. - - -

@@ -2691,7 +2675,7 @@ const std::string HARDCODED_SERVER_ID = "192.0.2.1"; At this stage of development, the only way to alter server configuration is to tweak its source code. To do so, please - edit src/bin/dhcp6/dhcp6_srv.cc file and modify following + edit src/bin/dhcp6/dhcp6_srv.cc file, modify the following parameters and recompile: const std::string HARDCODED_LEASE = "2001:db8:1::1234:abcd"; diff --git a/doc/guide/bind10-messages.html b/doc/guide/bind10-messages.html index 456aec4f05..1766c0f3c2 100644 --- a/doc/guide/bind10-messages.html +++ b/doc/guide/bind10-messages.html @@ -1,4 +1,4 @@ -BIND 10 Messages Manual

BIND 10 Messages Manual

This is the messages manual for BIND 10 version +BIND 10 Messages Manual

BIND 10 Messages Manual

This is the messages manual for BIND 10 version 20120405.

Copyright © 2011-2012 Internet Systems Consortium, Inc.

Abstract

BIND 10 is a Domain Name System (DNS) suite managed by Internet Systems Consortium (ISC). It includes DNS libraries and modular components for controlling authoritative and diff --git a/src/bin/dhcp6/Makefile.am b/src/bin/dhcp6/Makefile.am index 16b17eddf8..7947903f05 100644 --- a/src/bin/dhcp6/Makefile.am +++ b/src/bin/dhcp6/Makefile.am @@ -33,6 +33,7 @@ BUILT_SOURCES = spec_config.h pkglibexec_PROGRAMS = b10-dhcp6 b10_dhcp6_SOURCES = main.cc dhcp6_srv.cc dhcp6_srv.h +b10_dhcp6_SOURCES += ctrl_dhcp6_srv.cc ctrl_dhcp6_srv.h if USE_CLANGPP # Disable unused parameter warning caused by some of the @@ -44,6 +45,8 @@ b10_dhcp6_LDADD = $(top_builddir)/src/lib/exceptions/libexceptions.la b10_dhcp6_LDADD += $(top_builddir)/src/lib/asiolink/libasiolink.la b10_dhcp6_LDADD += $(top_builddir)/src/lib/log/liblog.la b10_dhcp6_LDADD += $(top_builddir)/src/lib/dhcp/libdhcp++.la +b10_dhcp6_LDADD += $(top_builddir)/src/lib/config/libcfgclient.la +b10_dhcp6_LDADD += $(top_builddir)/src/lib/cc/libcc.la b10_dhcp6dir = $(pkgdatadir) b10_dhcp6_DATA = dhcp6.spec diff --git a/src/bin/dhcp6/ctrl_dhcp6_srv.cc b/src/bin/dhcp6/ctrl_dhcp6_srv.cc new file mode 100644 index 0000000000..461d5f1d64 --- /dev/null +++ b/src/bin/dhcp6/ctrl_dhcp6_srv.cc @@ -0,0 +1,159 @@ +// Copyright (C) 2012 Internet Systems Consortium, Inc. ("ISC") +// +// Permission to use, copy, modify, and/or distribute this software for any +// purpose with or without fee is hereby granted, provided that the above +// copyright notice and this permission notice appear in all copies. +// +// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH +// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY +// AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, +// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM +// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE +// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR +// PERFORMANCE OF THIS SOFTWARE. + +#include +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +using namespace std; +using namespace isc::util; +using namespace isc::dhcp; +using namespace isc::util; +using namespace isc::data; +using namespace isc::cc; +using namespace isc::config; +using namespace isc::asiolink; + +namespace isc { +namespace dhcp { + +ControlledDhcpv6Srv* ControlledDhcpv6Srv::server_ = NULL; + +ConstElementPtr +ControlledDhcpv6Srv::dhcp6ConfigHandler(ConstElementPtr new_config) { + cout << "b10-dhcp6: Received new config:" << new_config->str() << endl; + ConstElementPtr answer = isc::config::createAnswer(0, + "Thank you for sending config."); + return (answer); +} + +ConstElementPtr +ControlledDhcpv6Srv::dhcp6CommandHandler(const string& command, ConstElementPtr args) { + cout << "b10-dhcp6: Received new command: [" << command << "], args=" + << args->str() << endl; + if (command == "shutdown") { + if (ControlledDhcpv6Srv::server_) { + ControlledDhcpv6Srv::server_->shutdown(); + } else { + cout << "Server not initialized yet or already shut down." << endl; + ConstElementPtr answer = isc::config::createAnswer(1, + "Shutdown failure."); + return (answer); + } + ConstElementPtr answer = isc::config::createAnswer(0, + "Shutting down."); + return (answer); + } + + ConstElementPtr answer = isc::config::createAnswer(1, + "Unrecognized command."); + + return (answer); +} + +void ControlledDhcpv6Srv::sessionReader(void) { + // Process one asio event. If there are more events, iface_mgr will call + // this callback more than once. + if (server_) { + server_->io_service_.run_one(); + } +} + +void ControlledDhcpv6Srv::establishSession() { + + string specfile; + if (getenv("B10_FROM_BUILD")) { + specfile = string(getenv("B10_FROM_BUILD")) + + "/src/bin/dhcp6/dhcp6.spec"; + } else { + specfile = string(DHCP6_SPECFILE_LOCATION); + } + + /// @todo: Check if session is not established already. Throw, if it is. + + cout << "b10-dhcp6: my specfile is " << specfile << endl; + + cc_session_ = new Session(io_service_.get_io_service()); + + config_session_ = new ModuleCCSession(specfile, *cc_session_, + dhcp6ConfigHandler, + dhcp6CommandHandler, false); + config_session_->start(); + + /// Integrate the asynchronous I/O model of BIND 10 configuration + /// control with the "select" model of the DHCP server. This is + /// fully explained in \ref dhcpv6Session. + int ctrl_socket = cc_session_->getSocketDesc(); + cout << "b10-dhcp6: Control session started, socket=" + << ctrl_socket << endl; + IfaceMgr::instance().set_session_socket(ctrl_socket, sessionReader); +} + +void ControlledDhcpv6Srv::disconnectSession() { + if (config_session_) { + delete config_session_; + config_session_ = NULL; + } + if (cc_session_) { + cc_session_->disconnect(); + delete cc_session_; + cc_session_ = NULL; + } + + // deregister session socket + IfaceMgr::instance().set_session_socket(IfaceMgr::INVALID_SOCKET, NULL); +} + +ControlledDhcpv6Srv::ControlledDhcpv6Srv(uint16_t port /*= DHCP6_SERVER_PORT*/) + :Dhcpv6Srv(port), cc_session_(NULL), config_session_(NULL) { + server_ = this; // remember this instance for use in callback +} + +void ControlledDhcpv6Srv::shutdown() { + io_service_.stop(); // Stop ASIO transmissions + Dhcpv6Srv::shutdown(); // Initiate DHCPv6 shutdown procedure. +} + +ControlledDhcpv6Srv::~ControlledDhcpv6Srv() { + disconnectSession(); + + server_ = NULL; // forget this instance. There should be no callback anymore + // at this stage anyway. +} + +isc::data::ConstElementPtr +ControlledDhcpv6Srv::execDhcpv6ServerCommand(const std::string& command_id, + isc::data::ConstElementPtr args) { + try { + return (dhcp6CommandHandler(command_id, args)); + } catch (const Exception& ex) { + ConstElementPtr answer = isc::config::createAnswer(1, ex.what()); + return (answer); + } +} + + +}; +}; diff --git a/src/bin/dhcp6/ctrl_dhcp6_srv.h b/src/bin/dhcp6/ctrl_dhcp6_srv.h new file mode 100644 index 0000000000..91fc80acb2 --- /dev/null +++ b/src/bin/dhcp6/ctrl_dhcp6_srv.h @@ -0,0 +1,123 @@ +// Copyright (C) 2012 Internet Systems Consortium, Inc. ("ISC") +// +// Permission to use, copy, modify, and/or distribute this software for any +// purpose with or without fee is hereby granted, provided that the above +// copyright notice and this permission notice appear in all copies. +// +// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH +// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY +// AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, +// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM +// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE +// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR +// PERFORMANCE OF THIS SOFTWARE. + +#ifndef CTRL_DHCPV6_SRV_H +#define CTRL_DHCPV6_SRV_H + +#include +#include +#include +#include +#include + +namespace isc { +namespace dhcp { + +/// @brief Controlled version of the DHCPv6 server +/// +/// This is a class that is responsible for establishing connection +/// with msqg (receving commands and configuration). This is an extended +/// version of Dhcpv6Srv class that is purely a DHCPv6 server, without +/// external control. ControlledDhcpv6Srv should be used in typical BIND10 +/// (i.e. featuring msgq) environment, while Dhcpv6Srv should be used in +/// embedded environments. +/// +/// For detailed explanation or relations between main(), ControlledDhcpv6Srv, +/// Dhcpv6Srv and other classes, see \ref dhcpv6Session. +class ControlledDhcpv6Srv : public isc::dhcp::Dhcpv6Srv { +public: + + /// @brief Constructor + /// + /// @param port UDP port to be opened for DHCP traffic + ControlledDhcpv6Srv(uint16_t port = DHCP6_SERVER_PORT); + + /// @brief Destructor. + ~ControlledDhcpv6Srv(); + + /// @brief Establishes msgq session. + /// + /// Creates session that will be used to receive commands and updated + /// configuration from boss (or indirectly from user via bindctl). + void establishSession(); + + /// @brief Terminates existing msgq session. + /// + /// This method terminates existing session with msgq. After calling + /// it, no further messages over msgq (commands or configuration updates) + /// may be received. + /// + /// It is ok to call this method when session is disconnected already. + void disconnectSession(); + + /// @brief Initiates shutdown procedure for the whole DHCPv6 server. + void shutdown(); + + /// @brief Session callback, processes received commands. + /// + /// @param command_id text represenation of the command (e.g. "shutdown") + /// @param args optional parameters + /// + /// @return status of the command + static isc::data::ConstElementPtr + execDhcpv6ServerCommand(const std::string& command, + isc::data::ConstElementPtr args); + +protected: + /// @brief Static pointer to the sole instance of the DHCP server. + /// + /// This is required for config and command handlers to gain access to + /// the server + static ControlledDhcpv6Srv* server_; + + /// @brief A callback for handling incoming configuration updates. + /// + /// As pointer to this method is used a callback in ASIO used in + /// ModuleCCSession, it has to be static. + /// + /// @param new_config textual representation of the new configuration + /// + /// @return status of the config update + static isc::data::ConstElementPtr + dhcp6ConfigHandler(isc::data::ConstElementPtr new_config); + + /// @brief A callback for handling incoming commands. + /// + /// @param command textual representation of the command + /// @param args parameters of the command + /// + /// @return status of the processed command + static isc::data::ConstElementPtr + dhcp6CommandHandler(const std::string& command, isc::data::ConstElementPtr args); + + /// @brief Callback that will be called from iface_mgr when command/config arrives. + /// + /// This static callback method is called from IfaceMgr::receive6() method, + /// when there is a new command or configuration sent over msgq. + static void sessionReader(void); + + /// @brief IOService object, used for all ASIO operations. + isc::asiolink::IOService io_service_; + + /// @brief Helper session object that represents raw connection to msgq. + isc::cc::Session* cc_session_; + + /// @brief Session that receives configuation and commands + isc::config::ModuleCCSession* config_session_; +}; + +}; // namespace isc::dhcp +}; // namespace isc + +#endif diff --git a/src/bin/dhcp6/dhcp6_srv.cc b/src/bin/dhcp6/dhcp6_srv.cc index 293e6004ae..55a925c1ed 100644 --- a/src/bin/dhcp6/dhcp6_srv.cc +++ b/src/bin/dhcp6/dhcp6_srv.cc @@ -48,12 +48,12 @@ Dhcpv6Srv::Dhcpv6Srv(uint16_t port) { IfaceMgr::instance(); } catch (const std::exception &e) { cout << "Failed to instantiate InterfaceManager:" << e.what() << ". Aborting." << endl; - shutdown = true; + shutdown_ = true; } if (IfaceMgr::instance().countIfaces() == 0) { cout << "Failed to detect any network interfaces. Aborting." << endl; - shutdown = true; + shutdown_ = true; } // Now try to open IPv6 sockets on detected interfaces. @@ -63,7 +63,7 @@ Dhcpv6Srv::Dhcpv6Srv(uint16_t port) { setServerID(); - shutdown = false; + shutdown_ = false; } Dhcpv6Srv::~Dhcpv6Srv() { @@ -72,8 +72,13 @@ Dhcpv6Srv::~Dhcpv6Srv() { IfaceMgr::instance().closeSockets(); } +void Dhcpv6Srv::shutdown() { + cout << "b10-dhcp6: DHCPv6 server shutdown." << endl; + shutdown_ = true; +} + bool Dhcpv6Srv::run() { - while (!shutdown) { + while (!shutdown_) { // client's message and server's response Pkt6Ptr query = IfaceMgr::instance().receive6(); diff --git a/src/bin/dhcp6/dhcp6_srv.h b/src/bin/dhcp6/dhcp6_srv.h index b6ae6376f2..9d1bf19497 100644 --- a/src/bin/dhcp6/dhcp6_srv.h +++ b/src/bin/dhcp6/dhcp6_srv.h @@ -67,6 +67,8 @@ public: /// critical error. bool run(); + /// @brief Instructs the server to shut down. + void shutdown(); protected: /// @brief Processes incoming SOLICIT and returns response. /// @@ -184,7 +186,7 @@ protected: /// indicates if shutdown is in progress. Setting it to true will /// initiate server shutdown procedure. - volatile bool shutdown; + volatile bool shutdown_; }; }; // namespace isc::dhcp diff --git a/src/bin/dhcp6/main.cc b/src/bin/dhcp6/main.cc index 62c0d20807..1cd52913bf 100644 --- a/src/bin/dhcp6/main.cc +++ b/src/bin/dhcp6/main.cc @@ -13,47 +13,34 @@ // PERFORMANCE OF THIS SOFTWARE. #include - -#include -#include -#include -#include -#include -#include -#include - -#include #include - -#include -#if 0 -// TODO cc is not used yet. It should be eventually -#include -#include -#endif - -#include #include - -#include -#include "dhcp6/dhcp6_srv.h" +#include +#include using namespace std; -using namespace isc::util; - -using namespace isc; using namespace isc::dhcp; +/// This file contains entry point (main() function) for standard DHCPv6 server +/// component for BIND10 framework. It parses command-line arguments and +/// instantiates ControlledDhcpv6Srv class that is responsible for establishing +/// connection with msgq (receiving commands and configuration) and also +/// creating Dhcpv6 server object as well. +/// +/// For detailed explanation or relations between main(), ControlledDhcpv6Srv, +/// Dhcpv6Srv and other classes, see \ref dhcpv6Session. + namespace { -bool verbose_mode = false; +const char* const DHCP6_NAME = "b10-dhcp6"; void usage() { - cerr << "Usage: b10-dhcp6 [-v]" + cerr << "Usage: b10-dhcp6 [-v]" << endl; cerr << "\t-v: verbose output" << endl; - cerr << "\t-p number: specify non-standard port number 1-65535 (useful for testing only)" << endl; + cerr << "\t-p number: specify non-standard port number 1-65535 " + << "(useful for testing only)" << endl; exit(EXIT_FAILURE); } } // end of anonymous namespace @@ -63,6 +50,7 @@ main(int argc, char* argv[]) { int ch; int port_number = DHCP6_SERVER_PORT; // The default. Any other values are // useful for testing only. + bool verbose_mode = false; // Should server be verbose? while ((ch = getopt(argc, argv, "vp:")) != -1) { switch (ch) { @@ -84,7 +72,13 @@ main(int argc, char* argv[]) { } } - cout << "My pid=" << getpid() << endl; + // Initialize logging. If verbose, we'll use maximum verbosity. + isc::log::initLogger(DHCP6_NAME, + (verbose_mode ? isc::log::DEBUG : isc::log::INFO), + isc::log::MAX_DEBUG_LEVEL, NULL); + + cout << "b10-dhcp6: My pid=" << getpid() << ", binding to port " + << port_number << ", verbose " << (verbose_mode?"yes":"no") << endl; if (argc - optind > 0) { usage(); @@ -92,24 +86,18 @@ main(int argc, char* argv[]) { int ret = EXIT_SUCCESS; - // TODO remainder of auth to dhcp6 code copy. We need to enable this in - // dhcp6 eventually -#if 0 - Session* cc_session = NULL; - Session* statistics_session = NULL; - ModuleCCSession* config_session = NULL; -#endif try { - string specfile; - if (getenv("B10_FROM_BUILD")) { - specfile = string(getenv("B10_FROM_BUILD")) + - "/src/bin/auth/dhcp6.spec"; - } else { - specfile = string(DHCP6_SPECFILE_LOCATION); - } + + cout << "b10-dhcp6: Initiating DHCPv6 server operation." << endl; + + ControlledDhcpv6Srv* server = new ControlledDhcpv6Srv(port_number); + server->run(); + delete server; + server = NULL; cout << "[b10-dhcp6] Initiating DHCPv6 operation." << endl; + /// @todo: pass verbose to the actual server once logging is implemented Dhcpv6Srv* srv = new Dhcpv6Srv(port_number); srv->run(); diff --git a/src/bin/dhcp6/tests/Makefile.am b/src/bin/dhcp6/tests/Makefile.am index ac20d56830..298be41a13 100644 --- a/src/bin/dhcp6/tests/Makefile.am +++ b/src/bin/dhcp6/tests/Makefile.am @@ -42,9 +42,10 @@ if HAVE_GTEST TESTS += dhcp6_unittests -dhcp6_unittests_SOURCES = ../dhcp6_srv.h ../dhcp6_srv.cc +dhcp6_unittests_SOURCES = ../dhcp6_srv.h ../dhcp6_srv.cc ../ctrl_dhcp6_srv.cc dhcp6_unittests_SOURCES += dhcp6_unittests.cc dhcp6_unittests_SOURCES += dhcp6_srv_unittest.cc +dhcp6_unittests_SOURCES += ctrl_dhcp6_srv_unittest.cc if USE_CLANGPP # Disable unused parameter warning caused by some of the @@ -59,6 +60,8 @@ dhcp6_unittests_LDADD += $(top_builddir)/src/lib/asiolink/libasiolink.la dhcp6_unittests_LDADD += $(top_builddir)/src/lib/dhcp/libdhcp++.la dhcp6_unittests_LDADD += $(top_builddir)/src/lib/log/liblog.la dhcp6_unittests_LDADD += $(top_builddir)/src/lib/exceptions/libexceptions.la +dhcp6_unittests_LDADD += $(top_builddir)/src/lib/config/libcfgclient.la +dhcp6_unittests_LDADD += $(top_builddir)/src/lib/cc/libcc.la endif diff --git a/src/bin/dhcp6/tests/ctrl_dhcp6_srv_unittest.cc b/src/bin/dhcp6/tests/ctrl_dhcp6_srv_unittest.cc new file mode 100644 index 0000000000..1e88f83bba --- /dev/null +++ b/src/bin/dhcp6/tests/ctrl_dhcp6_srv_unittest.cc @@ -0,0 +1,85 @@ +// Copyright (C) 2012 Internet Systems Consortium, Inc. ("ISC") +// +// Permission to use, copy, modify, and/or distribute this software for any +// purpose with or without fee is hereby granted, provided that the above +// copyright notice and this permission notice appear in all copies. +// +// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH +// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY +// AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, +// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM +// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE +// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR +// PERFORMANCE OF THIS SOFTWARE. + +#include +#include +#include +#include + +#include +#include + +#include +#include +#include + +using namespace std; +using namespace isc; +using namespace isc::dhcp; +using namespace isc::asiolink; +using namespace isc::data; +using namespace isc::config; + +namespace { + +class NakedControlledDhcpv6Srv: public ControlledDhcpv6Srv { + // "naked" DHCPv6 server, exposes internal fields +public: + NakedControlledDhcpv6Srv():ControlledDhcpv6Srv(DHCP6_SERVER_PORT + 10000) { } +}; + +class CtrlDhcpv6SrvTest : public ::testing::Test { +public: + CtrlDhcpv6SrvTest() { + } + + ~CtrlDhcpv6SrvTest() { + }; +}; + +TEST_F(CtrlDhcpv6SrvTest, commands) { + + ControlledDhcpv6Srv* srv = NULL; + ASSERT_NO_THROW({ + srv = new ControlledDhcpv6Srv(DHCP6_SERVER_PORT + 10000); + }); + + // use empty parameters list + ElementPtr params(new isc::data::MapElement()); + int rcode = -1; + + // case 1: send bogus command + ConstElementPtr result = ControlledDhcpv6Srv::execDhcpv6ServerCommand("blah", params); + ConstElementPtr comment = parseAnswer(rcode, result); + EXPECT_EQ(1, rcode); // expect failure (no such command as blah) + + // case 2: send shutdown command without any parameters + result = ControlledDhcpv6Srv::execDhcpv6ServerCommand("shutdown", params); + comment = parseAnswer(rcode, result); + EXPECT_EQ(0, rcode); // expect success + + const pid_t pid(getpid()); + ConstElementPtr x(new isc::data::IntElement(pid)); + params->set("pid", x); + + // case 3: send shutdown command with 1 parameter: pid + result = ControlledDhcpv6Srv::execDhcpv6ServerCommand("shutdown", params); + comment = parseAnswer(rcode, result); + EXPECT_EQ(0, rcode); // expect success + + + delete srv; +} + +} // end of anonymous namespace diff --git a/src/bin/dhcp6/tests/dhcp6_test.py b/src/bin/dhcp6/tests/dhcp6_test.py index cf04f60be7..230588fd73 100644 --- a/src/bin/dhcp6/tests/dhcp6_test.py +++ b/src/bin/dhcp6/tests/dhcp6_test.py @@ -1,4 +1,4 @@ -# Copyright (C) 2011,2012 Internet Systems Consortium. +# copyright (C) 2011,2012 Internet Systems Consortium. # # Permission to use, copy, modify, and distribute this software for any # purpose with or without fee is hereby granted, provided that the above @@ -131,7 +131,7 @@ class TestDhcpv6Daemon(unittest.TestCase): print(" not that is can bind sockets correctly. Please ignore binding errors.") (returncode, output, error) = self.runCommand(["../b10-dhcp6", "-v"]) - self.assertEqual( str(output).count("[b10-dhcp6] Initiating DHCPv6 operation."), 1) + self.assertEqual( str(output).count("b10-dhcp6: Initiating DHCPv6 server operation."), 1) def test_portnumber_0(self): print("Check that specifying port number 0 is not allowed.") From f788c425a50d504d7d23d3945b67f2ad25513495 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Fri, 22 Jun 2012 14:05:39 +0200 Subject: [PATCH 02/69] [1708] Minor fixes in dhcp4 - This is a small follow-up to 1651. --- src/bin/dhcp4/ctrl_dhcp4_srv.cc | 2 +- src/bin/dhcp4/main.cc | 9 +++------ src/bin/dhcp4/tests/ctrl_dhcp4_srv_unittest.cc | 6 +++--- 3 files changed, 7 insertions(+), 10 deletions(-) diff --git a/src/bin/dhcp4/ctrl_dhcp4_srv.cc b/src/bin/dhcp4/ctrl_dhcp4_srv.cc index b0526830f8..bb0e77fe30 100644 --- a/src/bin/dhcp4/ctrl_dhcp4_srv.cc +++ b/src/bin/dhcp4/ctrl_dhcp4_srv.cc @@ -86,7 +86,7 @@ void ControlledDhcpv4Srv::establishSession() { string specfile; if (getenv("B10_FROM_BUILD")) { specfile = string(getenv("B10_FROM_BUILD")) + - "/src/bin/auth/dhcp4.spec"; + "/src/bin/dhcp4/dhcp4.spec"; } else { specfile = string(DHCP4_SPECFILE_LOCATION); } diff --git a/src/bin/dhcp4/main.cc b/src/bin/dhcp4/main.cc index d6c4602b72..ab3ae1bc1d 100644 --- a/src/bin/dhcp4/main.cc +++ b/src/bin/dhcp4/main.cc @@ -14,18 +14,13 @@ #include #include -#include #include #include #include -#include -#include using namespace std; using namespace isc::dhcp; - - /// This file contains entry point (main() function) for standard DHCPv4 server /// component for BIND10 framework. It parses command-line arguments and /// instantiates ControlledDhcpv4Srv class that is responsible for establishing @@ -44,7 +39,8 @@ usage() { cerr << "Usage: b10-dhcp4 [-v]" << endl; cerr << "\t-v: verbose output" << endl; - cerr << "\t-p number: specify non-standard port number 1-65535 (useful for testing only)" << endl; + cerr << "\t-p number: specify non-standard port number 1-65535 " + << "(useful for testing only)" << endl; exit(EXIT_FAILURE); } } // end of anonymous namespace @@ -94,6 +90,7 @@ main(int argc, char* argv[]) { cout << "[b10-dhcp4] Initiating DHCPv4 server operation." << endl; + /// @todo: pass verbose to the actul server once logging is implemented ControlledDhcpv4Srv* server = new ControlledDhcpv4Srv(port_number); server->run(); delete server; diff --git a/src/bin/dhcp4/tests/ctrl_dhcp4_srv_unittest.cc b/src/bin/dhcp4/tests/ctrl_dhcp4_srv_unittest.cc index 080425665e..5af0cc9857 100644 --- a/src/bin/dhcp4/tests/ctrl_dhcp4_srv_unittest.cc +++ b/src/bin/dhcp4/tests/ctrl_dhcp4_srv_unittest.cc @@ -1,4 +1,4 @@ -// Copyright (C) 2011 Internet Systems Consortium, Inc. ("ISC") +// Copyright (C) 2012 Internet Systems Consortium, Inc. ("ISC") // // Permission to use, copy, modify, and/or distribute this software for any // purpose with or without fee is hereby granted, provided that the above @@ -64,7 +64,7 @@ TEST_F(CtrlDhcpv4SrvTest, commands) { ConstElementPtr comment = parseAnswer(rcode, result); EXPECT_EQ(1, rcode); // expect failure (no such command as blah) - // case 1: send shutdown command without any parameters + // case 2: send shutdown command without any parameters result = ControlledDhcpv4Srv::execDhcpv4ServerCommand("shutdown", params); comment = parseAnswer(rcode, result); EXPECT_EQ(0, rcode); // expect success @@ -73,7 +73,7 @@ TEST_F(CtrlDhcpv4SrvTest, commands) { ConstElementPtr x(new isc::data::IntElement(pid)); params->set("pid", x); - // case 2: send shutdown command with 1 parameter: pid + // case 3: send shutdown command with 1 parameter: pid result = ControlledDhcpv4Srv::execDhcpv4ServerCommand("shutdown", params); comment = parseAnswer(rcode, result); EXPECT_EQ(0, rcode); // expect success From ed478d2878ef583a1f37da8a588f70f4bf47a9b9 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Fri, 22 Jun 2012 15:36:27 +0200 Subject: [PATCH 03/69] [1708] Data reception over many interfaces (IPv6) added --- src/bin/dhcp6/dhcp6.spec | 16 ++- src/bin/dhcp6/dhcp6_srv.cc | 4 +- src/lib/dhcp/iface_mgr.cc | 146 ++++++++++++++++------- src/lib/dhcp/iface_mgr.h | 4 +- src/lib/dhcp/tests/iface_mgr_unittest.cc | 2 +- 5 files changed, 125 insertions(+), 47 deletions(-) diff --git a/src/bin/dhcp6/dhcp6.spec b/src/bin/dhcp6/dhcp6.spec index 05c3529865..2a82a2d3a0 100644 --- a/src/bin/dhcp6/dhcp6.spec +++ b/src/bin/dhcp6/dhcp6.spec @@ -1,6 +1,6 @@ { "module_spec": { - "module_name": "dhcp6", + "module_name": "Dhcp6", "module_description": "DHCPv6 server daemon", "config_data": [ { "item_name": "interface", @@ -9,6 +9,18 @@ "item_default": "eth0" } ], - "commands": [] + "commands": [ + { + "command_name": "shutdown", + "command_description": "Shuts down DHCPv6 server.", + "command_args": [ + { + "item_name": "pid", + "item_type": "integer", + "item_optional": true + } + ] + } + ] } } diff --git a/src/bin/dhcp6/dhcp6_srv.cc b/src/bin/dhcp6/dhcp6_srv.cc index 55a925c1ed..d28d9b7d21 100644 --- a/src/bin/dhcp6/dhcp6_srv.cc +++ b/src/bin/dhcp6/dhcp6_srv.cc @@ -79,9 +79,11 @@ void Dhcpv6Srv::shutdown() { bool Dhcpv6Srv::run() { while (!shutdown_) { + /// @todo: calculate actual timeout once we have lease database + int timeout = 1000; // client's message and server's response - Pkt6Ptr query = IfaceMgr::instance().receive6(); + Pkt6Ptr query = IfaceMgr::instance().receive6(timeout); Pkt6Ptr rsp; if (query) { diff --git a/src/lib/dhcp/iface_mgr.cc b/src/lib/dhcp/iface_mgr.cc index 508413dfe8..3167ae039c 100644 --- a/src/lib/dhcp/iface_mgr.cc +++ b/src/lib/dhcp/iface_mgr.cc @@ -693,13 +693,12 @@ IfaceMgr::receive4(uint32_t timeout) { const SocketInfo* candidate = 0; IfaceCollection::const_iterator iface; - fd_set sockets; - FD_ZERO(&sockets); int maxfd = 0; - stringstream names; + FD_ZERO(&sockets); + /// @todo: marginal performance optimization. We could create the set once /// and then use its copy for select(). Please note that select() modifies /// provided set to indicated which sockets have something to read. @@ -858,9 +857,108 @@ IfaceMgr::receive4(uint32_t timeout) { return (pkt); } -Pkt6Ptr IfaceMgr::receive6() { - uint8_t buf[RCVBUFSIZE]; +Pkt6Ptr IfaceMgr::receive6(uint32_t timeout) { + const SocketInfo* candidate = 0; + fd_set sockets; + int maxfd = 0; + stringstream names; + + FD_ZERO(&sockets); + + /// @todo: marginal performance optimization. We could create the set once + /// and then use its copy for select(). Please note that select() modifies + /// provided set to indicated which sockets have something to read. + IfaceCollection::const_iterator iface; + for (iface = ifaces_.begin(); iface != ifaces_.end(); ++iface) { + + for (SocketCollection::const_iterator s = iface->sockets_.begin(); + s != iface->sockets_.end(); ++s) { + + // Only deal with IPv4 addresses. + if (s->addr_.getFamily() == AF_INET6) { + names << s->sockfd_ << "(" << iface->getName() << ") "; + + // Add this socket to listening set + FD_SET(s->sockfd_, &sockets); + if (maxfd < s->sockfd_) { + maxfd = s->sockfd_; + } + } + } + } + + // if there is session socket registered... + if (session_socket_ != INVALID_SOCKET) { + // at it to the set as well + FD_SET(session_socket_, &sockets); + if (maxfd < session_socket_) + maxfd = session_socket_; + names << session_socket_ << "(session)"; + } + + cout << "Trying to receive data on sockets:" << names.str() + << ".Timeout is " << timeout << " seconds." << endl; + + /// @todo: implement sub-second precision one day + struct timeval select_timeout; + select_timeout.tv_sec = timeout; + select_timeout.tv_usec = 0; + + int result = select(maxfd + 1, &sockets, NULL, NULL, &select_timeout); + + if (result == 0) { + // nothing received and timeout has been reached + return (Pkt6Ptr()); // NULL + } else if (result < 0) { + cout << "Socket read error: " << strerror(errno) << endl; + + /// @todo: perhaps throw here? + return (Pkt6Ptr()); // NULL + } + + // Let's find out which socket has the data + if ((session_socket_ != INVALID_SOCKET) && (FD_ISSET(session_socket_, &sockets))) { + // something received over session socket + cout << "BIND10 command or config available over session socket." << endl; + + if (session_callback_) { + // in theory we could call io_service.run_one() here, instead of + // implementing callback mechanism, but that would introduce + // asiolink dependency to libdhcp++ and that is something we want + // to avoid (see CPE market and out long term plans for minimalistic + // implementations. + session_callback_(); + } + + return (Pkt6Ptr()); // NULL + } + + // Let's find out which interface/socket has the data + for (iface = ifaces_.begin(); iface != ifaces_.end(); ++iface) { + for (SocketCollection::const_iterator s = iface->sockets_.begin(); + s != iface->sockets_.end(); ++s) { + if (FD_ISSET(s->sockfd_, &sockets)) { + candidate = &(*s); + break; + } + } + if (candidate) { + break; + } + } + + if (!candidate) { + cout << "Received data over unknown socket." << endl; + return (Pkt6Ptr()); // NULL + } + + cout << "Trying to receive over UDP6 socket " << candidate->sockfd_ << " bound to " + << candidate->addr_.toText() << "/port=" << candidate->port_ << " on " + << iface->getFullName() << endl; + + // Now we have a socket, let's get some data from it! + uint8_t buf[RCVBUFSIZE]; memset(&control_buf_[0], 0, control_buf_len_); struct sockaddr_in6 from; memset(&from, 0, sizeof(from)); @@ -892,43 +990,7 @@ Pkt6Ptr IfaceMgr::receive6() { m.msg_control = &control_buf_[0]; m.msg_controllen = control_buf_len_; - /// TODO: Need to move to select() and pool over - /// all available sockets. For now, we just take the - /// first interface and use first socket from it. - IfaceCollection::const_iterator iface = ifaces_.begin(); - const SocketInfo* candidate = 0; - while (iface != ifaces_.end()) { - for (SocketCollection::const_iterator s = iface->sockets_.begin(); - s != iface->sockets_.end(); ++s) { - if (s->addr_.getFamily() != AF_INET6) { - continue; - } - if (s->addr_.getAddress().to_v6().is_multicast()) { - candidate = &(*s); - break; - } - if (!candidate) { - candidate = &(*s); // it's not multicast, but it's better than nothing - } - } - if (candidate) { - break; - } - ++iface; - } - if (iface == ifaces_.end()) { - isc_throw(Unexpected, "No suitable IPv6 interfaces detected. Can't receive anything."); - } - - if (!candidate) { - isc_throw(Unexpected, "Interface " << iface->getFullName() - << " does not have any sockets open."); - } - - cout << "Trying to receive over UDP6 socket " << candidate->sockfd_ << " bound to " - << candidate->addr_.toText() << "/port=" << candidate->port_ << " on " - << iface->getFullName() << endl; - int result = recvmsg(candidate->sockfd_, &m, 0); + result = recvmsg(candidate->sockfd_, &m, 0); struct in6_addr to_addr; memset(&to_addr, 0, sizeof(to_addr)); diff --git a/src/lib/dhcp/iface_mgr.h b/src/lib/dhcp/iface_mgr.h index 7fa2e855f4..48f43a6507 100644 --- a/src/lib/dhcp/iface_mgr.h +++ b/src/lib/dhcp/iface_mgr.h @@ -345,8 +345,10 @@ public: /// to not wait infinitely, but rather do something useful /// (e.g. remove expired leases) /// + /// @param timeout specifies timeout (in seconds) + /// /// @return Pkt6 object representing received packet (or NULL) - Pkt6Ptr receive6(); + Pkt6Ptr receive6(uint32_t timeout); /// @brief Tries to receive IPv4 packet over open IPv4 sockets. /// diff --git a/src/lib/dhcp/tests/iface_mgr_unittest.cc b/src/lib/dhcp/tests/iface_mgr_unittest.cc index e7ccb680ba..df74d93eba 100644 --- a/src/lib/dhcp/tests/iface_mgr_unittest.cc +++ b/src/lib/dhcp/tests/iface_mgr_unittest.cc @@ -309,7 +309,7 @@ TEST_F(IfaceMgrTest, sendReceive6) { EXPECT_EQ(true, ifacemgr->send(sendPkt)); - rcvPkt = ifacemgr->receive6(); + rcvPkt = ifacemgr->receive6(10); ASSERT_TRUE(rcvPkt); // received our own packet From b524d5e024eda8dd37d51c0eb87f6b2ff2af9196 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Fri, 22 Jun 2012 16:14:23 +0200 Subject: [PATCH 04/69] [1708] msgq connection in dhcp{4,6} is now optional. --- ChangeLog | 9 +++++++++ src/bin/dhcp4/main.cc | 26 ++++++++++++++++++++++--- src/bin/dhcp4/tests/dhcp4_test.py | 13 +++++++++++++ src/bin/dhcp6/main.cc | 32 ++++++++++++++++++++++--------- src/bin/dhcp6/tests/dhcp6_test.py | 18 +++++++++++++++-- 5 files changed, 84 insertions(+), 14 deletions(-) diff --git a/ChangeLog b/ChangeLog index 8fb267fc41..a8d0f49fe0 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,12 @@ +451. [func] tomek + b10-dhcp6: DHCPv6 server component is now integrated into + BIND10 framework. It can be started from BIND10 (using bindctl) + and can receive commands. The only supported command for now + is 'Dhcp6 shutdown'. + b10-dhcp4: Command line-switch '-s' to disable msgq was added. + b10-dhcp6: Command line-switch '-s' to disable msgq was added. + (Trac #1708, git tbd) + 450. [func]* tomek b10-dhcp4: DHCPv4 server component is now integrated into BIND10 framework. It can be started from BIND10 (using bindctl) diff --git a/src/bin/dhcp4/main.cc b/src/bin/dhcp4/main.cc index ab3ae1bc1d..472f46f467 100644 --- a/src/bin/dhcp4/main.cc +++ b/src/bin/dhcp4/main.cc @@ -39,6 +39,7 @@ usage() { cerr << "Usage: b10-dhcp4 [-v]" << endl; cerr << "\t-v: verbose output" << endl; + cerr << "\t-s: stand-alone mode (don't connect to BIND10)" << endl; cerr << "\t-p number: specify non-standard port number 1-65535 " << "(useful for testing only)" << endl; exit(EXIT_FAILURE); @@ -51,13 +52,17 @@ main(int argc, char* argv[]) { bool verbose_mode = false; // should server be verbose? int port_number = DHCP4_SERVER_PORT; // The default. any other values are // useful for testing only. + bool stand_alone = false; // should be connect to BIND10 msgq? - while ((ch = getopt(argc, argv, "vp:")) != -1) { + while ((ch = getopt(argc, argv, "vsp:")) != -1) { switch (ch) { case 'v': verbose_mode = true; isc::log::denabled = true; break; + case 's': + stand_alone = true; + break; case 'p': port_number = strtol(optarg, NULL, 10); if (port_number == 0) { @@ -77,8 +82,9 @@ main(int argc, char* argv[]) { (verbose_mode ? isc::log::DEBUG : isc::log::INFO), isc::log::MAX_DEBUG_LEVEL, NULL); - cout << "b10-dhcp4: My pid=" << getpid() << ", binding to port " - << port_number << ", verbose " << (verbose_mode?"yes":"no") << endl; + cout << "b10-dhcp4: My pid=" << getpid() << ", binding to port " + << port_number << ", verbose " << (verbose_mode?"yes":"no") + << ", stand-alone=" << (stand_alone?"yes":"no") << endl; if (argc - optind > 0) { usage(); @@ -92,6 +98,20 @@ main(int argc, char* argv[]) { /// @todo: pass verbose to the actul server once logging is implemented ControlledDhcpv4Srv* server = new ControlledDhcpv4Srv(port_number); + + if (!stand_alone) { + try { + server->establishSession(); + } catch (const std::exception& ex) { + cerr << "Failed to establish BIND10 session. " + "Running in stand-alone mode:" << ex.what() << endl; + // Let's continue. It is useful to have the ability to run + // DHCP server in stand-alone mode, e.g. for testing + } + } else { + cout << "Skipping connection to the BIND10 msgq." << endl; + } + server->run(); delete server; server = NULL; diff --git a/src/bin/dhcp4/tests/dhcp4_test.py b/src/bin/dhcp4/tests/dhcp4_test.py index 18d23fff8d..065c80c2ad 100644 --- a/src/bin/dhcp4/tests/dhcp4_test.py +++ b/src/bin/dhcp4/tests/dhcp4_test.py @@ -166,5 +166,18 @@ class TestDhcpv4Daemon(unittest.TestCase): # Check that there is an error message about invalid port number printed on stderr self.assertEqual( str(output).count("opening sockets on port 10057"), 1) + def test_skip_msgq(self): + print("Check that connection to BIND10 msgq can be disabled.") + + (returncode, output, error) = self.runDhcp4(['../b10-dhcp4', '-s', '-p', '10057']) + + # When invalid port number is specified, return code must not be success + # TODO: Temporarily commented out as socket binding on systems that do not have + # interface detection implemented currently fails. + # self.assertTrue(returncode == 0) + + # Check that there is an error message about invalid port number printed on stderr + self.assertEqual( str(output).count("Skipping connection to the BIND10 msgq."), 1) + if __name__ == '__main__': unittest.main() diff --git a/src/bin/dhcp6/main.cc b/src/bin/dhcp6/main.cc index 1cd52913bf..96e2dd14dd 100644 --- a/src/bin/dhcp6/main.cc +++ b/src/bin/dhcp6/main.cc @@ -39,6 +39,7 @@ usage() { cerr << "Usage: b10-dhcp6 [-v]" << endl; cerr << "\t-v: verbose output" << endl; + cerr << "\t-s: stand-alone mode (don't connect to BIND10)" << endl; cerr << "\t-p number: specify non-standard port number 1-65535 " << "(useful for testing only)" << endl; exit(EXIT_FAILURE); @@ -51,13 +52,17 @@ main(int argc, char* argv[]) { int port_number = DHCP6_SERVER_PORT; // The default. Any other values are // useful for testing only. bool verbose_mode = false; // Should server be verbose? + bool stand_alone = false; // should be connect to BIND10 msgq? - while ((ch = getopt(argc, argv, "vp:")) != -1) { + while ((ch = getopt(argc, argv, "vsp:")) != -1) { switch (ch) { case 'v': verbose_mode = true; isc::log::denabled = true; break; + case 's': + stand_alone = true; + break; case 'p': port_number = strtol(optarg, NULL, 10); if (port_number == 0) { @@ -78,7 +83,8 @@ main(int argc, char* argv[]) { isc::log::MAX_DEBUG_LEVEL, NULL); cout << "b10-dhcp6: My pid=" << getpid() << ", binding to port " - << port_number << ", verbose " << (verbose_mode?"yes":"no") << endl; + << port_number << ", verbose " << (verbose_mode?"yes":"no") + << ", stand-alone=" << (stand_alone?"yes":"no") << endl; if (argc - optind > 0) { usage(); @@ -90,18 +96,26 @@ main(int argc, char* argv[]) { cout << "b10-dhcp6: Initiating DHCPv6 server operation." << endl; + /// @todo: pass verbose to the actual server once logging is implemented ControlledDhcpv6Srv* server = new ControlledDhcpv6Srv(port_number); + + if (!stand_alone) { + try { + server->establishSession(); + } catch (const std::exception& ex) { + cerr << "Failed to establish BIND10 session. " + "Running in stand-alone mode:" << ex.what() << endl; + // Let's continue. It is useful to have the ability to run + // DHCP server in stand-alone mode, e.g. for testing + } + } else { + cout << "Skipping connection to the BIND10 msgq." << endl; + } + server->run(); delete server; server = NULL; - cout << "[b10-dhcp6] Initiating DHCPv6 operation." << endl; - - /// @todo: pass verbose to the actual server once logging is implemented - Dhcpv6Srv* srv = new Dhcpv6Srv(port_number); - - srv->run(); - } catch (const std::exception& ex) { cerr << "[b10-dhcp6] Server failed: " << ex.what() << endl; ret = EXIT_FAILURE; diff --git a/src/bin/dhcp6/tests/dhcp6_test.py b/src/bin/dhcp6/tests/dhcp6_test.py index 230588fd73..e84bad198c 100644 --- a/src/bin/dhcp6/tests/dhcp6_test.py +++ b/src/bin/dhcp6/tests/dhcp6_test.py @@ -158,7 +158,7 @@ class TestDhcpv6Daemon(unittest.TestCase): def test_portnumber_nonroot(self): print("Check that specifying unprivileged port number will work.") - (returncode, output, error) = self.runCommand(['../b10-dhcp6', '-p', '10057']) + (returncode, output, error) = self.runCommand(['../b10-dhcp6', '-p', '10547']) # When invalid port number is specified, return code must not be success # TODO: Temporarily commented out as socket binding on systems that do not have @@ -166,7 +166,21 @@ class TestDhcpv6Daemon(unittest.TestCase): # self.assertTrue(returncode == 0) # Check that there is a message on stdout about opening proper port - self.assertEqual( str(output).count("opening sockets on port 10057"), 1) + self.assertEqual( str(output).count("opening sockets on port 10547"), 1) + + def test_skip_msgq(self): + print("Check that connection to BIND10 msgq can be disabled.") + + (returncode, output, error) = self.runDhcp4(['../b10-dhcp6', '-s', '-p', '10547']) + + # When invalid port number is specified, return code must not be success + # TODO: Temporarily commented out as socket binding on systems that do not have + # interface detection implemented currently fails. + # self.assertTrue(returncode == 0) + + # Check that there is an error message about invalid port number printed on stderr + self.assertEqual( str(output).count("Skipping connection to the BIND10 msgq."), 1) + if __name__ == '__main__': unittest.main() From ffb9c0ae7609347a66ae4997b1cf5c6250d83256 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Tue, 10 Jul 2012 19:03:07 +0200 Subject: [PATCH 05/69] [1958] Added basic implementation of perfdhcp StatsMgr and unit tests. --- tests/tools/perfdhcp/Makefile.am | 1 + tests/tools/perfdhcp/stats_mgr.h | 254 ++++++++++++++++++ tests/tools/perfdhcp/tests/Makefile.am | 1 + .../perfdhcp/tests/stats_mgr_unittest.cc | 73 +++++ 4 files changed, 329 insertions(+) create mode 100644 tests/tools/perfdhcp/stats_mgr.h create mode 100644 tests/tools/perfdhcp/tests/stats_mgr_unittest.cc diff --git a/tests/tools/perfdhcp/Makefile.am b/tests/tools/perfdhcp/Makefile.am index 6ebc00f92a..9fa0054e8f 100644 --- a/tests/tools/perfdhcp/Makefile.am +++ b/tests/tools/perfdhcp/Makefile.am @@ -24,6 +24,7 @@ libperfdhcp___la_SOURCES += localized_option.h libperfdhcp___la_SOURCES += perf_pkt6.cc perf_pkt6.h libperfdhcp___la_SOURCES += perf_pkt4.cc perf_pkt4.h libperfdhcp___la_SOURCES += pkt_transform.cc pkt_transform.h +libperfdhcp___la_SOURCES += stats_mgr.h libperfdhcp___la_CXXFLAGS = $(AM_CXXFLAGS) if USE_CLANGPP diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h new file mode 100644 index 0000000000..d041c9fb26 --- /dev/null +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -0,0 +1,254 @@ +// Copyright (C) 2012 Internet Systems Consortium, Inc. ("ISC") +// +// Permission to use, copy, modify, and/or distribute this software for any +// purpose with or without fee is hereby granted, provided that the above +// copyright notice and this permission notice appear in all copies. +// +// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH +// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY +// AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, +// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM +// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE +// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR +// PERFORMANCE OF THIS SOFTWARE. + +#ifndef __STATS_MGR_H +#define __STATS_MGR_H + +#include + +#include +#include +#include +#include +#include +#include + +#include + +namespace isc { +namespace perfdhcp { + +/// \brief Statistics Manager +/// +/// This class template is a storage for various performance statistics +/// collected during performance tests execution with perfdhcp tool. +/// +/// Statistics Manager holds lists of sent and received packets and +/// groups them into exchanges. For example: DHCPDISCOVER message and +/// corresponding DHCPOFFER messages belong to one exchange, DHCPREQUEST +/// and corresponding DHCPACK message belong to another exchange etc. +/// In order to update statistics for a particular exchange type, client +/// class passes sent and received packets. Internally, Statistics Manager +/// tries to match transaction id of received packet with sent packet +/// stored on the list of sent packets. When packets are matched the +/// round trip time can be calculated. +/// +/// \tparam T class representing DHCPv4 or DHCPv6 packet. +template +class StatsMgr : public boost::noncopyable { +public: + + /// DHCP packet exchange types. + enum ExchangeType { + XCHG_DO, ///< DHCPv4 DISCOVER-OFFER + XCHG_RA, ///< DHCPv4 REQUEST-ACK + XCHG_SA, ///< DHCPv6 SOLICIT-ADVERTISE + XCHG_RR ///< DHCPv6 REQUEST-REPLY + }; + + /// \brief Exchange Statistics. + /// + /// This class collects statistics for exchanges. Parent class + /// may define number of different packet exchanges like: + /// DHCPv4 DISCOVER-OFFER, DHCPv6 SOLICIT-ADVERTISE etc. Performance + /// statistics will be collected for each of those separately in + /// corresponding instance of ExchangeStats. + class ExchangeStats { + public: + + /// \brief List of packets (sent or received). + /// + /// List of packets based on multi index container allows efficient + /// search of packets based on their sequence (order in which they + /// were inserted) as well as based on packet transaction id. + typedef boost::multi_index_container< + boost::shared_ptr, + boost::multi_index::indexed_by< + boost::multi_index::sequenced<>, + boost::multi_index::ordered_unique< + boost::multi_index::const_mem_fun< + T, uint32_t, &T::getTransid> + > + > + > PktList; + + /// Packet list iterator for sequencial access to elements. + typedef typename PktList::iterator PktListIterator; + /// Packet list index to search packets using transaction id. + typedef typename PktList::template nth_index<1>::type + PktListTransidIndex; + /// Packet list iterator to access packets using transaction id. + typedef typename PktListTransidIndex::iterator PktListTransidIterator; + + /// \brief Constructor + /// + /// \param xchg_type exchange type + ExchangeStats(const ExchangeType xchg_type) + : xchg_type_(xchg_type) { + sent_packets_cache_ = sent_packets_.begin(); + } + + /// \brief Add new packet to list of sent packets. + /// + /// Method adds new packet to list of sent packets. + /// + /// \param packet packet object to be appended. + void appendSent(const boost::shared_ptr packet) { + sent_packets_.template get<0>().push_back(packet); + } + + /// \brief Find packet on the list of sent packets. + /// + /// Method finds packet with specified transaction id on the list + /// of sent packets. It is used to match received packet with + /// corresponding sent packet. + /// Since packets from the server most often come in the same order + /// as they were sent by client, this method will first check if + /// next sent packet matches. If it doesn't, function will search + /// the packet using indexing by transaction id. This reduces + /// packet search time significantly. + /// + /// \param transid transaction id of the packet to search + /// \throw isc::Unexpected if packet could not be found + /// \return packet having specified transaction id + boost::shared_ptr findSent(const uint32_t transid) { + if (sent_packets_.size() == 0) { + isc_throw(Unexpected, "Sent packets list is empty."); + } else if (sent_packets_cache_ == sent_packets_.end()) { + sent_packets_cache_ = sent_packets_.begin(); + } + + bool packet_found = false; + if ((*sent_packets_cache_)->getTransid() == transid) { + packet_found = true; + } else { + PktListTransidIndex& idx = sent_packets_.template get<1>(); + PktListTransidIterator it = idx.find(transid); + if (it != idx.end()) { + packet_found = true; + sent_packets_cache_ = sent_packets_.template project<0>(it); + } + } + + if (!packet_found) { + isc_throw(Unexpected, "Unable to find sent packet."); + } + + boost::shared_ptr sent_packet(*sent_packets_cache_); + ++sent_packets_cache_; + return sent_packet; + } + + double getMinDelay() const { return min_delay_; } + double getMaxDelay() const { return max_delay_; } + double getSumDelay() const { return sum_delay_; } + double getSquareSumDelay() const { return square_sum_delay_; } + private: + + /// \brief Private default constructor. + /// + /// Default constructor is private because we want client + /// class to specify exchange type explicitely. + ExchangeStats(); + + ExchangeType xchg_type_; ///< Packet exchange type. + PktList sent_packets_; ///< List of sent packets. + + /// Iterator pointing to the packet on sent list which will most + /// likely match next received packet. This is based on the + /// assumption that server responds in order to incoming packets. + PktListIterator sent_packets_cache_; + + PktList rcvd_packets_; ///< List of received packets. + + double min_delay_; ///< Minimum delay between sent + ///< and received packets. + double max_delay_; ///< Maximum delay between sent + ///< and received packets. + double sum_delay_; ///< Sum of delays between sent + ///< and received packets. + double square_sum_delay_; ///< Square sum of delays between + ///< sent and recived packets. + }; + + /// Pointer to ExchangeStats. + typedef boost::shared_ptr ExchangeStatsPtr; + /// Map containing all specified exchange types. + typedef typename std::map ExchangesMap; + /// Iterator poiting to \ref ExchangesMap + typedef typename ExchangesMap::iterator ExchangesMapIterator; + + /// \brief Specify new exchange type. + /// + /// This method creates new \ref ExchangeStats object that will + /// collect statistics data from packets exchange of the specified + /// type. + /// + /// \param xchg_type exchange type. + /// \throw isc::BadValue if exchange of specified type exists. + void addExchangeStats(const ExchangeType xchg_type) { + if (exchanges_.find(xchg_type) != exchanges_.end()) { + isc_throw(BadValue, "Exchange of specified type already added."); + } + exchanges_[xchg_type] = ExchangeStatsPtr(new ExchangeStats(xchg_type)); + } + + /// \brief Adds new packet to the sent packets list. + /// + /// Method adds new packet to the sent packets list. + /// Packets are added to the list sequentially and + /// most often read sequentially. + /// + /// \param xchg_type exchange type. + /// \param packet packet to be added to the list + /// \throw isc::BadValue if invalid exchange type specified. + void passSentPacket(const ExchangeType xchg_type, + const boost::shared_ptr packet) { + ExchangesMapIterator it = exchanges_.find(xchg_type); + if (it == exchanges_.end()) { + isc_throw(BadValue, "Packets exchange not specified"); + } + it->second->appendSent(packet); + } + + /// \brief Add new received packet and match with sent packet. + /// + /// Method adds new packet to the list of received packets. It + /// also searches for corresponding packet on the list of sent + /// packets. When packets are matched the statistics counters + /// are updated accordingly for the particular exchange type. + /// + /// \param xchg_type exchange type. + /// \param packet received packet + /// \throw isc::BadValue if invalid exchange type specified. + /// \throw isc::Unexpected if corresponding packet was not + /// found on the list of sent packets. + void passRcvdPacket(const ExchangeType xchg_type, + const boost::shared_ptr packet) { + ExchangesMapIterator it = exchanges_.find(xchg_type); + if (it == exchanges_.end()) { + isc_throw(BadValue, "Packets exchange not specified"); + } + boost::shared_ptr sent_packet + = it->second->findSent(packet->getTransid()); + } + +private: + ExchangesMap exchanges_; ///< Map of exchange types. +}; + +} // namespace perfdhcp +} // namespace isc + +#endif // __STATS_MGR_H diff --git a/tests/tools/perfdhcp/tests/Makefile.am b/tests/tools/perfdhcp/tests/Makefile.am index ab1251c5c9..55bb70903f 100644 --- a/tests/tools/perfdhcp/tests/Makefile.am +++ b/tests/tools/perfdhcp/tests/Makefile.am @@ -21,6 +21,7 @@ run_unittests_SOURCES += command_options_unittest.cc run_unittests_SOURCES += perf_pkt6_unittest.cc run_unittests_SOURCES += perf_pkt4_unittest.cc run_unittests_SOURCES += localized_option_unittest.cc +run_unittests_SOURCES += stats_mgr_unittest.cc run_unittests_SOURCES += $(top_builddir)/tests/tools/perfdhcp/command_options.cc run_unittests_SOURCES += $(top_builddir)/tests/tools/perfdhcp/pkt_transform.cc run_unittests_SOURCES += $(top_builddir)/tests/tools/perfdhcp/perf_pkt6.cc diff --git a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc new file mode 100644 index 0000000000..a275ed541d --- /dev/null +++ b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc @@ -0,0 +1,73 @@ +// Copyright (C) 2012 Internet Systems Consortium, Inc. ("ISC") +// +// Permission to use, copy, modify, and/or distribute this software for any +// purpose with or without fee is hereby granted, provided that the above +// copyright notice and this permission notice appear in all copies. +// +// THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH +// REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY +// AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, +// INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM +// LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE +// OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR +// PERFORMANCE OF THIS SOFTWARE. + +#include + +#include +#include +#include +#include + +#include + +#include "../stats_mgr.h" + +using namespace std; +using namespace isc; +using namespace isc::dhcp; +using namespace isc::perfdhcp; + +namespace { + +typedef StatsMgr StatsMgr4; +typedef StatsMgr StatsMgr6; + +const uint32_t common_transid = 123; + +class StatsMgrTest : public ::testing::Test { +public: + StatsMgrTest() { + } +}; + +TEST_F(StatsMgrTest, Constructor) { + boost::scoped_ptr stats_mgr(new StatsMgr4()); +} + +TEST_F(StatsMgrTest, Exchanges) { + boost::scoped_ptr stats_mgr(new StatsMgr4()); + boost::shared_ptr sent_packet(new Pkt4(DHCPDISCOVER, common_transid)); + boost::shared_ptr rcvd_packet(new Pkt4(DHCPOFFER, common_transid)); + EXPECT_THROW(stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet), BadValue); + EXPECT_THROW(stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet), BadValue); + + stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); + EXPECT_THROW(stats_mgr->passSentPacket(StatsMgr4::XCHG_RA, sent_packet), BadValue); + EXPECT_THROW(stats_mgr->passRcvdPacket(StatsMgr4::XCHG_RA, rcvd_packet), BadValue); + + EXPECT_NO_THROW(stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet)); + EXPECT_NO_THROW(stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet)); +} + +TEST_F(StatsMgrTest, SendReceiveSimple) { + boost::scoped_ptr stats_mgr(new StatsMgr4()); + boost::shared_ptr sent_packet(new Pkt4(DHCPDISCOVER, common_transid)); + boost::shared_ptr rcvd_packet(new Pkt4(DHCPOFFER, common_transid)); + stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); + ASSERT_NO_THROW(stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet)); + EXPECT_NO_THROW(stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet)); + EXPECT_NO_THROW(stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet)); +} + +} From e9f6a894ed38cdd6bd3165c67a0be64482e1a47d Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Tue, 10 Jul 2012 21:48:56 +0200 Subject: [PATCH 06/69] [1958] Updating RTT times for sent/received packets. --- tests/tools/perfdhcp/stats_mgr.h | 72 +++++++++++++++++++++++++++++++- 1 file changed, 70 insertions(+), 2 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index d041c9fb26..c5ee70bf9b 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -23,6 +23,7 @@ #include #include #include +#include #include @@ -150,15 +151,78 @@ public: return sent_packet; } + /// \brief Update delay counters. + /// + /// Method updates delay counters based on timestamps of + /// sent and received packets. + /// + /// \param sent_packet sent packet + /// \param rcvd_packet received packet + /// \throw isc::Unexpected if failed to calculate timestamps + void updateDelays(const boost::shared_ptr sent_packet, + const boost::shared_ptr rcvd_packet) { + boost::posix_time::ptime sent_time = sent_packet->getTimestamp(); + boost::posix_time::ptime rcvd_time = rcvd_packet->getTimestamp(); + + if (sent_time.is_not_a_date_time() || + rcvd_time.is_not_a_date_time()) { + isc_throw(Unexpected, + "Timestamp must be set for sent and " + "received packet to measure RTT"); + } + boost::posix_time::time_period period(sent_time, rcvd_time); + double delta = + static_cast(period.length().total_nanoseconds()) / 1e9; + + if (delta < 0) { + isc_throw(Unexpected, "Sent packet's timestamp must not be " + "greater than received packet's timestamp"); + } + + if (delta < min_delay_) { + min_delay_ = delta; + } + if (delta > max_delay_) { + max_delay_ = delta; + } + sum_delay_ += delta; + square_sum_delay_ += delta * delta; + } + + /// \brief Return minumum delay between sent and received packet. + /// + /// Method returns minimum delay between sent and received packet. + /// + /// \return minimum delay between packets. double getMinDelay() const { return min_delay_; } + + /// \brief Return maxmimum delay between sent and received packet. + /// + /// Method returns maximum delay between sent and received packet. + /// + /// \return maximum delay between packets. double getMaxDelay() const { return max_delay_; } + + /// \brief Return sum of delays between sent and received packets. + /// + /// Method returns sum of delays between sent and received packets. + /// + /// \return sum of delays between sent and received packets. double getSumDelay() const { return sum_delay_; } + + /// \brief Return square sum of delays between sent and received + /// packets. + /// + /// Method returns square sum of delays between sent and received + /// packets. + /// + /// \return square sum of delays between sent and received packets. double getSquareSumDelay() const { return square_sum_delay_; } private: /// \brief Private default constructor. /// - /// Default constructor is private because we want client + /// Default constructor is private because we want the client /// class to specify exchange type explicitely. ExchangeStats(); @@ -240,10 +304,14 @@ public: if (it == exchanges_.end()) { isc_throw(BadValue, "Packets exchange not specified"); } + ExchangeStatsPtr xchg_stats = it->second; boost::shared_ptr sent_packet - = it->second->findSent(packet->getTransid()); + = xchg_stats->findSent(packet->getTransid()); + + xchg_stats->updateDelays(sent_packet, packet); } + private: ExchangesMap exchanges_; ///< Map of exchange types. }; From c76267ee3aa8434b218954619a9d261fb42a6810 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Wed, 11 Jul 2012 14:40:54 +0200 Subject: [PATCH 07/69] [1958] Added perfdhcp orphans counters. --- tests/tools/perfdhcp/stats_mgr.h | 76 +++++++++++++---- .../perfdhcp/tests/stats_mgr_unittest.cc | 83 +++++++++++++++---- 2 files changed, 127 insertions(+), 32 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index c5ee70bf9b..b9da312e52 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -15,6 +15,7 @@ #ifndef __STATS_MGR_H #define __STATS_MGR_H +#include #include #include @@ -96,7 +97,12 @@ public: /// /// \param xchg_type exchange type ExchangeStats(const ExchangeType xchg_type) - : xchg_type_(xchg_type) { + : xchg_type_(xchg_type), + min_delay_(std::numeric_limits::max()), + max_delay_(0.), + sum_delay_(0.), + orphans_(0), + square_sum_delay_(0.) { sent_packets_cache_ = sent_packets_.begin(); } @@ -121,11 +127,12 @@ public: /// packet search time significantly. /// /// \param transid transaction id of the packet to search - /// \throw isc::Unexpected if packet could not be found - /// \return packet having specified transaction id + /// \return packet having specified transaction or NULL if packet + /// not found boost::shared_ptr findSent(const uint32_t transid) { if (sent_packets_.size() == 0) { - isc_throw(Unexpected, "Sent packets list is empty."); + ++orphans_; + return boost::shared_ptr(); } else if (sent_packets_cache_ == sent_packets_.end()) { sent_packets_cache_ = sent_packets_.begin(); } @@ -143,7 +150,8 @@ public: } if (!packet_found) { - isc_throw(Unexpected, "Unable to find sent packet."); + ++orphans_; + return boost::shared_ptr(); } boost::shared_ptr sent_packet(*sent_packets_cache_); @@ -218,6 +226,15 @@ public: /// /// \return square sum of delays between sent and received packets. double getSquareSumDelay() const { return square_sum_delay_; } + + /// \brief Return number of orphant packets. + /// + /// Method returns number of received packets that had no matching + /// sent packet. It is possible that such packet was late or not + /// for us. + /// + /// \return number of orphant received packets. + uint64_t getOrphans() const { return orphans_; } private: /// \brief Private default constructor. @@ -244,6 +261,8 @@ public: ///< and received packets. double square_sum_delay_; ///< Square sum of delays between ///< sent and recived packets. + + uint64_t orphans_; ///< Number of orphant received packets. }; /// Pointer to ExchangeStats. @@ -251,7 +270,7 @@ public: /// Map containing all specified exchange types. typedef typename std::map ExchangesMap; /// Iterator poiting to \ref ExchangesMap - typedef typename ExchangesMap::iterator ExchangesMapIterator; + typedef typename ExchangesMap::const_iterator ExchangesMapIterator; /// \brief Specify new exchange type. /// @@ -279,11 +298,8 @@ public: /// \throw isc::BadValue if invalid exchange type specified. void passSentPacket(const ExchangeType xchg_type, const boost::shared_ptr packet) { - ExchangesMapIterator it = exchanges_.find(xchg_type); - if (it == exchanges_.end()) { - isc_throw(BadValue, "Packets exchange not specified"); - } - it->second->appendSent(packet); + ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); + xchg_stats->appendSent(packet); } /// \brief Add new received packet and match with sent packet. @@ -300,19 +316,45 @@ public: /// found on the list of sent packets. void passRcvdPacket(const ExchangeType xchg_type, const boost::shared_ptr packet) { + ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); + boost::shared_ptr sent_packet + = xchg_stats->findSent(packet->getTransid()); + + if (sent_packet) { + xchg_stats->updateDelays(sent_packet, packet); + } + } + + /// \brief Return number of orphant packets. + /// + /// Method returns number of orphant packets for specified + /// exchange type. + /// + /// \param xchg_type exchange type. + /// \throw isc::BadValue if invalid exchange type specified. + /// \return number of orphant packets so far. + uint64_t getOrphans(const ExchangeType xchg_type) const { + ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); + return xchg_stats->getOrphans(); + } +private: + + /// \brief Return exchange stats object for given exchange type + /// + /// Method returns exchange stats object for given exchange type. + /// + /// \param xchg_type exchange type. + /// \throw isc::BadValue if invalid exchange type specified. + /// \return exchange stats object. + ExchangeStatsPtr getExchangeStats(const ExchangeType xchg_type) const { ExchangesMapIterator it = exchanges_.find(xchg_type); if (it == exchanges_.end()) { isc_throw(BadValue, "Packets exchange not specified"); } ExchangeStatsPtr xchg_stats = it->second; - boost::shared_ptr sent_packet - = xchg_stats->findSent(packet->getTransid()); - - xchg_stats->updateDelays(sent_packet, packet); + return xchg_stats; } - -private: ExchangesMap exchanges_; ///< Map of exchange types. }; diff --git a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc index a275ed541d..bf590fe64a 100644 --- a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc +++ b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc @@ -39,6 +39,13 @@ class StatsMgrTest : public ::testing::Test { public: StatsMgrTest() { } + + Pkt4* createPacket4(const uint8_t msg_type, + const uint32_t transid) { + Pkt4* pkt = new Pkt4(msg_type, transid); + pkt->updateTimestamp(); + return pkt; + } }; TEST_F(StatsMgrTest, Constructor) { @@ -47,27 +54,73 @@ TEST_F(StatsMgrTest, Constructor) { TEST_F(StatsMgrTest, Exchanges) { boost::scoped_ptr stats_mgr(new StatsMgr4()); - boost::shared_ptr sent_packet(new Pkt4(DHCPDISCOVER, common_transid)); - boost::shared_ptr rcvd_packet(new Pkt4(DHCPOFFER, common_transid)); - EXPECT_THROW(stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet), BadValue); - EXPECT_THROW(stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet), BadValue); - - stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); - EXPECT_THROW(stats_mgr->passSentPacket(StatsMgr4::XCHG_RA, sent_packet), BadValue); - EXPECT_THROW(stats_mgr->passRcvdPacket(StatsMgr4::XCHG_RA, rcvd_packet), BadValue); + boost::shared_ptr sent_packet(createPacket4(DHCPDISCOVER, + common_transid)); + boost::shared_ptr rcvd_packet(createPacket4(DHCPOFFER, + common_transid)); + EXPECT_THROW( + stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet), + BadValue + ); + EXPECT_THROW( + stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet), + BadValue + ); - EXPECT_NO_THROW(stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet)); - EXPECT_NO_THROW(stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet)); + stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); + EXPECT_THROW( + stats_mgr->passSentPacket(StatsMgr4::XCHG_RA, sent_packet), + BadValue + ); + EXPECT_THROW( + stats_mgr->passRcvdPacket(StatsMgr4::XCHG_RA, rcvd_packet), + BadValue + ); + + EXPECT_NO_THROW( + stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet) + ); + EXPECT_NO_THROW( + stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet) + ); } TEST_F(StatsMgrTest, SendReceiveSimple) { boost::scoped_ptr stats_mgr(new StatsMgr4()); - boost::shared_ptr sent_packet(new Pkt4(DHCPDISCOVER, common_transid)); - boost::shared_ptr rcvd_packet(new Pkt4(DHCPOFFER, common_transid)); + boost::shared_ptr sent_packet(createPacket4(DHCPDISCOVER, + common_transid)); + boost::shared_ptr rcvd_packet(createPacket4(DHCPOFFER, + common_transid)); stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); - ASSERT_NO_THROW(stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet)); - EXPECT_NO_THROW(stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet)); - EXPECT_NO_THROW(stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet)); + ASSERT_NO_THROW( + stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet) + ); + EXPECT_NO_THROW( + stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet) + ); + EXPECT_NO_THROW( + stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet) + ); +} + +TEST_F(StatsMgrTest, Orphans) { + const int packets_num = 6; + boost::scoped_ptr stats_mgr(new StatsMgr4()); + stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); + + for (int i = 0; i < packets_num; i += 2) { + boost::shared_ptr sent_packet(createPacket4(DHCPDISCOVER, i)); + ASSERT_NO_THROW( + stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet) + ); + } + for (int i = 0; i < packets_num; ++i) { + boost::shared_ptr rcvd_packet(createPacket4(DHCPOFFER, i)); + ASSERT_NO_THROW( + stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet); + ); + } + EXPECT_EQ(packets_num / 2, stats_mgr->getOrphans(StatsMgr4::XCHG_DO)); } } From 7017538fe54478e47fe1bdda055d3545a5d70f66 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Thu, 12 Jul 2012 09:34:37 +0200 Subject: [PATCH 08/69] [1958] Instrumentation to monitor unordered packet lookups. --- tests/tools/perfdhcp/stats_mgr.h | 132 ++++++++++++++++-- .../perfdhcp/tests/stats_mgr_unittest.cc | 27 ++++ 2 files changed, 149 insertions(+), 10 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index b9da312e52..507e6fd11f 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -21,9 +21,10 @@ #include #include #include -#include +#include #include #include +#include #include #include @@ -78,9 +79,10 @@ public: boost::shared_ptr, boost::multi_index::indexed_by< boost::multi_index::sequenced<>, - boost::multi_index::ordered_unique< - boost::multi_index::const_mem_fun< - T, uint32_t, &T::getTransid> + boost::multi_index::hashed_non_unique< + boost::multi_index::const_mem_fun< + T, uint32_t, &T::getTransid + > > > > PktList; @@ -102,7 +104,10 @@ public: max_delay_(0.), sum_delay_(0.), orphans_(0), - square_sum_delay_(0.) { + square_sum_delay_(0.), + ordered_lookups_(0), + unordered_lookup_size_sum_(0), + unordered_lookups_(0) { sent_packets_cache_ = sent_packets_.begin(); } @@ -110,11 +115,20 @@ public: /// /// Method adds new packet to list of sent packets. /// - /// \param packet packet object to be appended. + /// \param packet packet object to be added. void appendSent(const boost::shared_ptr packet) { sent_packets_.template get<0>().push_back(packet); } + /// \brief Add new packet to list of received packets. + /// + /// Method adds new packet to list of received packets. + /// + /// \param packet packet object to be added. + void appendRcvd(const boost::shared_ptr packet) { + rcvd_packets_.template get<0>().push_back(packet); + } + /// \brief Find packet on the list of sent packets. /// /// Method finds packet with specified transaction id on the list @@ -139,13 +153,22 @@ public: bool packet_found = false; if ((*sent_packets_cache_)->getTransid() == transid) { + ++ordered_lookups_; packet_found = true; } else { PktListTransidIndex& idx = sent_packets_.template get<1>(); - PktListTransidIterator it = idx.find(transid); - if (it != idx.end()) { - packet_found = true; - sent_packets_cache_ = sent_packets_.template project<0>(it); + std::pair p = + idx.equal_range(transid); + ++unordered_lookups_; + unordered_lookup_size_sum_ += std::distance(p.first, p.second); + for (PktListTransidIterator it = p.first; it != p.second; + ++it) { + if ((*it)->getTransid() == transid) { + packet_found = true; + sent_packets_cache_ = + sent_packets_.template project<0>(it); + break; + } } } @@ -235,6 +258,40 @@ public: /// /// \return number of orphant received packets. uint64_t getOrphans() const { return orphans_; } + + /// \brief Return average unordered lookup set size. + /// + /// Method returns average unordered lookup set size. + /// This value is changes every time \findSet function uses + /// unordered packet lookup using transaction id. + /// + /// \return average unordered lookup set size. + double getAvgUnorderedLookupSetSize() const { + return static_cast(unordered_lookup_size_sum_) / + static_cast(unordered_lookups_); + } + + /// \brief Return number of unordered sent packets lookups + /// + /// Method returns number of unordered sent packet lookups. + /// Unordered lookup is used when received packet was sent + /// out of order by server - transaction id of received + /// packet does not match transaction id of next sent packet. + /// + /// \return number of unordered lookups. + uint64_t getUnorderedLookups() const { return unordered_lookups_; } + + + /// \brief Return number of ordered sent packets lookups + /// + /// Method returns number of ordered sent packet lookups. + /// Ordered lookup is used when packets are received in the + /// same order as they were sent to the server. + /// If packets are skipped or received out of order, lookup + /// function will use unordered lookup (with hash table). + /// + /// \return number of ordered lookups. + uint64_t getOrderedLookups() const { return ordered_lookups_; } private: /// \brief Private default constructor. @@ -263,6 +320,19 @@ public: ///< sent and recived packets. uint64_t orphans_; ///< Number of orphant received packets. + + /// Sum of unordered lookup sets. Needed to calculate mean size of + /// lookup set. It is desired that number of unordered lookups is + /// minimal for performance reasons. Tracking number of lookups and + /// mean size of the lookup set should give idea of packets serach + /// complexity. + uint64_t unordered_lookup_size_sum_; + + uint64_t unordered_lookups_; ///< Number of unordered sent packets + ///< lookups. + uint64_t ordered_lookups_; ///< Number of ordered sent packets + ///< lookups. + }; /// Pointer to ExchangeStats. @@ -322,6 +392,7 @@ public: if (sent_packet) { xchg_stats->updateDelays(sent_packet, packet); + xchg_stats->appendRcvd(packet); } } @@ -337,6 +408,47 @@ public: ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); return xchg_stats->getOrphans(); } + + /// \brief Return average unordered lookup set size. + /// + /// Method returns average unordered lookup set size. + /// This value is changes every time \findSet function uses + /// unordered packet lookup using transaction id. + /// + /// \param xchg_type exchange type. + /// \return average unordered lookup set size. + double getAvgUnorderedLookupSetSize(const ExchangeType xchg_type) const { + ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); + return xchg_stats->getAvgUnorderedLookupSetSize(); + } + + /// \brief Return number of unordered sent packets lookups + /// + /// Method returns number of unordered sent packet lookups. + /// Unordered lookup is used when received packet was sent + /// out of order by server - transaction id of received + /// packet does not match transaction id of next sent packet. + /// + /// \return number of unordered lookups. + uint64_t getUnorderedLookups(const ExchangeType xchg_type) const { + ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); + return xchg_stats->getUnorderedLookups(); + } + + /// \brief Return number of ordered sent packets lookups + /// + /// Method returns number of ordered sent packet lookups. + /// Ordered lookup is used when packets are received in the + /// same order as they were sent to the server. + /// If packets are skipped or received out of order, lookup + /// function will use unordered lookup (with hash table). + /// + /// \return number of ordered lookups. + uint64_t getOrderedLookups(const ExchangeType xchg_type) const { + ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); + return xchg_stats->getOrderedLookups(); + } + private: /// \brief Return exchange stats object for given exchange type diff --git a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc index bf590fe64a..8d6ee8398a 100644 --- a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc +++ b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc @@ -103,6 +103,33 @@ TEST_F(StatsMgrTest, SendReceiveSimple) { ); } +TEST_F(StatsMgrTest, SendReceiveUnordered) { + const int packets_num = 10; + boost::scoped_ptr stats_mgr(new StatsMgr4()); + stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); + + uint32_t transid[packets_num] = { 64322, 100203, 1, 232324, 6786, 23, 4523, 777883, 98082, 3 }; + for (int i = 0; i < packets_num; ++i) { + // uint32_t transid = i & 0xFFFFFFFE | !(i & 1); + boost::shared_ptr sent_packet(createPacket4(DHCPDISCOVER, + transid[i])); + ASSERT_NO_THROW( + stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet) + ); + } + + for (int i = 0; i < packets_num; ++i) { + boost::shared_ptr rcvd_packet(createPacket4(DHCPDISCOVER, + transid[packets_num - 1 - i])); + ASSERT_NO_THROW( + stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet); + ); + } + EXPECT_EQ(0, stats_mgr->getOrphans(StatsMgr4::XCHG_DO)); + EXPECT_EQ(10, stats_mgr->getUnorderedLookups(StatsMgr4::XCHG_DO)); + std::cout << stats_mgr->getAvgUnorderedLookupSetSize(StatsMgr4::XCHG_DO) << std::endl; +} + TEST_F(StatsMgrTest, Orphans) { const int packets_num = 6; boost::scoped_ptr stats_mgr(new StatsMgr4()); From df4d0be872ff9ff4e120e156b71d0abc25df0319 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Thu, 12 Jul 2012 16:49:10 +0200 Subject: [PATCH 09/69] [1958] Added custom transaction id hashing in perfdhcp stats manager. --- tests/tools/perfdhcp/stats_mgr.h | 12 ++++++++---- tests/tools/perfdhcp/tests/stats_mgr_unittest.cc | 3 +-- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index 507e6fd11f..b224fa7d8b 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -24,7 +24,7 @@ #include #include #include -#include +#include #include #include @@ -70,6 +70,10 @@ public: class ExchangeStats { public: + static uint32_t transid_hash(boost::shared_ptr packet) { + return packet->getTransid() & 1023; + } + /// \brief List of packets (sent or received). /// /// List of packets based on multi index container allows efficient @@ -80,8 +84,8 @@ public: boost::multi_index::indexed_by< boost::multi_index::sequenced<>, boost::multi_index::hashed_non_unique< - boost::multi_index::const_mem_fun< - T, uint32_t, &T::getTransid + boost::multi_index::global_fun< + boost::shared_ptr, uint32_t, &ExchangeStats::transid_hash > > > @@ -158,7 +162,7 @@ public: } else { PktListTransidIndex& idx = sent_packets_.template get<1>(); std::pair p = - idx.equal_range(transid); + idx.equal_range(transid & 1023); ++unordered_lookups_; unordered_lookup_size_sum_ += std::distance(p.first, p.second); for (PktListTransidIterator it = p.first; it != p.second; diff --git a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc index 8d6ee8398a..2d6ab14d54 100644 --- a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc +++ b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc @@ -108,9 +108,8 @@ TEST_F(StatsMgrTest, SendReceiveUnordered) { boost::scoped_ptr stats_mgr(new StatsMgr4()); stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); - uint32_t transid[packets_num] = { 64322, 100203, 1, 232324, 6786, 23, 4523, 777883, 98082, 3 }; + uint32_t transid[packets_num] = { 1, 1024, 2, 1025, 3, 1026, 4, 1027, 5, 1028 }; for (int i = 0; i < packets_num; ++i) { - // uint32_t transid = i & 0xFFFFFFFE | !(i & 1); boost::shared_ptr sent_packet(createPacket4(DHCPDISCOVER, transid[i])); ASSERT_NO_THROW( From 5d2116c14a18c5aa2f41fa75e0f893ad8a645bf3 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Thu, 19 Jul 2012 16:48:19 +0200 Subject: [PATCH 10/69] [1958] Provided access to all basic counters and created corresponding unit tests. --- tests/tools/perfdhcp/stats_mgr.h | 282 ++++++++++++++---- .../perfdhcp/tests/stats_mgr_unittest.cc | 191 +++++++++++- 2 files changed, 408 insertions(+), 65 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index b224fa7d8b..0fed0ae0a2 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -85,7 +85,9 @@ public: boost::multi_index::sequenced<>, boost::multi_index::hashed_non_unique< boost::multi_index::global_fun< - boost::shared_ptr, uint32_t, &ExchangeStats::transid_hash + boost::shared_ptr, + uint32_t, + &ExchangeStats::transid_hash > > > @@ -111,7 +113,9 @@ public: square_sum_delay_(0.), ordered_lookups_(0), unordered_lookup_size_sum_(0), - unordered_lookups_(0) { + unordered_lookups_(0), + sent_packets_num_(0), + rcvd_packets_num_(0) { sent_packets_cache_ = sent_packets_.begin(); } @@ -121,6 +125,7 @@ public: /// /// \param packet packet object to be added. void appendSent(const boost::shared_ptr packet) { + ++sent_packets_num_; sent_packets_.template get<0>().push_back(packet); } @@ -130,62 +135,10 @@ public: /// /// \param packet packet object to be added. void appendRcvd(const boost::shared_ptr packet) { + ++rcvd_packets_num_; rcvd_packets_.template get<0>().push_back(packet); } - /// \brief Find packet on the list of sent packets. - /// - /// Method finds packet with specified transaction id on the list - /// of sent packets. It is used to match received packet with - /// corresponding sent packet. - /// Since packets from the server most often come in the same order - /// as they were sent by client, this method will first check if - /// next sent packet matches. If it doesn't, function will search - /// the packet using indexing by transaction id. This reduces - /// packet search time significantly. - /// - /// \param transid transaction id of the packet to search - /// \return packet having specified transaction or NULL if packet - /// not found - boost::shared_ptr findSent(const uint32_t transid) { - if (sent_packets_.size() == 0) { - ++orphans_; - return boost::shared_ptr(); - } else if (sent_packets_cache_ == sent_packets_.end()) { - sent_packets_cache_ = sent_packets_.begin(); - } - - bool packet_found = false; - if ((*sent_packets_cache_)->getTransid() == transid) { - ++ordered_lookups_; - packet_found = true; - } else { - PktListTransidIndex& idx = sent_packets_.template get<1>(); - std::pair p = - idx.equal_range(transid & 1023); - ++unordered_lookups_; - unordered_lookup_size_sum_ += std::distance(p.first, p.second); - for (PktListTransidIterator it = p.first; it != p.second; - ++it) { - if ((*it)->getTransid() == transid) { - packet_found = true; - sent_packets_cache_ = - sent_packets_.template project<0>(it); - break; - } - } - } - - if (!packet_found) { - ++orphans_; - return boost::shared_ptr(); - } - - boost::shared_ptr sent_packet(*sent_packets_cache_); - ++sent_packets_cache_; - return sent_packet; - } - /// \brief Update delay counters. /// /// Method updates delay counters based on timestamps of @@ -206,6 +159,9 @@ public: "received packet to measure RTT"); } boost::posix_time::time_period period(sent_time, rcvd_time); + // We don't bother calculating deltas in nanoseconds. It is much + // more convenient to use seconds instead because we are going to + // sum them up. double delta = static_cast(period.length().total_nanoseconds()) / 1e9; @@ -214,16 +170,111 @@ public: "greater than received packet's timestamp"); } + // Record the minimum delay between sent and received packets. if (delta < min_delay_) { min_delay_ = delta; } + // Record the maximum delay between sent and received packets. if (delta > max_delay_) { max_delay_ = delta; } + // Update delay sum and square sum. That will be used to calculate + // mean delays. sum_delay_ += delta; square_sum_delay_ += delta * delta; } + /// \brief Find packet on the list of sent packets. + /// + /// Method finds packet with specified transaction id on the list + /// of sent packets. It is used to match received packet with + /// corresponding sent packet. + /// Since packets from the server most often come in the same order + /// as they were sent by client, this method will first check if + /// next sent packet matches. If it doesn't, function will search + /// the packet using indexing by transaction id. This reduces + /// packet search time significantly. + /// + /// \param transid transaction id of the packet to search + /// \return packet having specified transaction or NULL if packet + /// not found + boost::shared_ptr findSent(const uint32_t transid) { + if (sent_packets_.size() == 0) { + // List of sent packets is empty so there is no sense + // to continue looking fo the packet. It also means + // that the received packet we got has no corresponding + // sent packet so orphans counter has to be updated. + ++orphans_; + return boost::shared_ptr(); + } else if (sent_packets_cache_ == sent_packets_.end()) { + // Even if there are still many unmatched packets on the + // list we might hit the end of it because of unordered + // lookups. The next logical step is to reset cache. + sent_packets_cache_ = sent_packets_.begin(); + } + + // With this variable we will be signalling success or failure + // to find the packet. + bool packet_found = false; + // Most likely responses are sent from the server in the same + // order as client's requests to the server. We are caching + // next sent packet and first try to match with it the next + // incoming packet. We are successful if there is no + // packet drop or out of order packets sent. This is actually + // the fastest way to look for packets. + if ((*sent_packets_cache_)->getTransid() == transid) { + ++ordered_lookups_; + packet_found = true; + } else { + // If we are here, it means that we were unable to match the + // next incoming packet with next sent packet so we need to + // take a little more expensive approach to look packets using + // alternative index (transaction id & 1023). + PktListTransidIndex& idx = sent_packets_.template get<1>(); + // Packets are grouped using trasaction id masking with value + // of 1023. For instance, packets with transaction id equal to + // 1, 1024 ... will belong to the same group (a.k.a. bucket). + // When using alternative index we don't find the packet but + // bucket of packets and need to iterate through the bucket + // to find the one that has desired transaction id. + std::pair p = + idx.equal_range(transid & 1023); + // We want to keep statistics of unordered lookups to make + // sure that there is a right balance before number of + // unordered lookups and ordered lookups. If number of unordered + // lookups is high it may mean that many packets are lost or + // sent out of order. + ++unordered_lookups_; + // We also want to keep the mean value of the bucket. The lower + // bucket size the better. If bucket sizes appear to big we + // might want to increase number of buckets. + unordered_lookup_size_sum_ += std::distance(p.first, p.second); + for (PktListTransidIterator it = p.first; it != p.second; + ++it) { + if ((*it)->getTransid() == transid) { + packet_found = true; + sent_packets_cache_ = + sent_packets_.template project<0>(it); + break; + } + } + } + + if (!packet_found) { + // If we are here, it means that both ordered lookup and + // unordered lookup failed. Searched packet is not on the list. + ++orphans_; + return boost::shared_ptr(); + } + + boost::shared_ptr sent_packet(*sent_packets_cache_); + // If packet was found, we assume it will be never searched + // again. We want to delete this packet from the list to + // improve performance of future searches. + sent_packets_cache_ = eraseSent(sent_packets_cache_); + return sent_packet; + } + /// \brief Return minumum delay between sent and received packet. /// /// Method returns minimum delay between sent and received packet. @@ -271,6 +322,9 @@ public: /// /// \return average unordered lookup set size. double getAvgUnorderedLookupSetSize() const { + if (unordered_lookups_ == 0) { + return 0.; + } return static_cast(unordered_lookup_size_sum_) / static_cast(unordered_lookups_); } @@ -285,7 +339,6 @@ public: /// \return number of unordered lookups. uint64_t getUnorderedLookups() const { return unordered_lookups_; } - /// \brief Return number of ordered sent packets lookups /// /// Method returns number of ordered sent packet lookups. @@ -296,6 +349,25 @@ public: /// /// \return number of ordered lookups. uint64_t getOrderedLookups() const { return ordered_lookups_; } + + /// \brief Return total number of sent packets + /// + /// Method returns total number of sent packets. + /// + /// \return number of sent packets. + uint64_t getSentPacketsNum() const { + return sent_packets_num_; + } + + /// \brief Return total number of received packets + /// + /// Method returns total number of received packets. + /// + /// \return number of received packets. + uint64_t getRcvdPacketsNum() const { + return rcvd_packets_num_; + } + private: /// \brief Private default constructor. @@ -304,6 +376,18 @@ public: /// class to specify exchange type explicitely. ExchangeStats(); + + /// \brief Erase packet from the list of sent packets. + /// + /// Method erases packet from the list of sent packets. + /// + /// \param it iterator pointing to packet to be erased. + /// \return iterator pointing to packet following erased + /// packet or sent_packets_.end() if packet not found. + PktListIterator eraseSent(const PktListIterator it) { + return sent_packets_.template get<0>().erase(it); + } + ExchangeType xchg_type_; ///< Packet exchange type. PktList sent_packets_; ///< List of sent packets. @@ -337,6 +421,8 @@ public: uint64_t ordered_lookups_; ///< Number of ordered sent packets ///< lookups. + uint64_t sent_packets_num_; ///< Total number of sent packets. + uint64_t rcvd_packets_num_; ///< Total number of received packets. }; /// Pointer to ExchangeStats. @@ -400,6 +486,59 @@ public: } } + /// \brief Return minumum delay between sent and received packet. + /// + /// Method returns minimum delay between sent and received packet + /// for specified exchange type. + /// + /// \param xchg_type exchange type. + /// \throw isc::BadValue if invalid exchange type specified. + /// \return minimum delay between packets. + double getMinDelay(const ExchangeType xchg_type) const { + ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); + return xchg_stats->getMinDelay(); + } + + /// \brief Return maxmimum delay between sent and received packet. + /// + /// Method returns maximum delay between sent and received packet + /// for specified exchange type. + /// + /// \param xchg_type exchange type. + /// \throw isc::BadValue if invalid exchange type specified. + /// \return maximum delay between packets. + double getMaxDelay(const ExchangeType xchg_type) const { + ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); + return xchg_stats->getMaxDelay(); + } + + /// \brief Return sum of delays between sent and received packets. + /// + /// Method returns sum of delays between sent and received packets + /// for specified exchange type. + /// + /// \param xchg_type exchange type. + /// \throw isc::BadValue if invalid exchange type specified. + /// \return sum of delays between sent and received packets. + double getSumDelay(const ExchangeType xchg_type) const { + ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); + return xchg_stats->getSumDelay(); + } + + /// \brief Return square sum of delays between sent and received + /// packets. + /// + /// Method returns square sum of delays between sent and received + /// packets for specified exchange type. + /// + /// \param xchg_type exchange type. + /// \throw isc::BadValue if invalid exchange type specified. + /// \return square sum of delays between sent and received packets. + double getSquareSumDelay(const ExchangeType xchg_type) const { + ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); + return xchg_stats->getSquareSumDelay(); + } + /// \brief Return number of orphant packets. /// /// Method returns number of orphant packets for specified @@ -420,6 +559,7 @@ public: /// unordered packet lookup using transaction id. /// /// \param xchg_type exchange type. + /// \throw isc::BadValue if invalid exchange type specified. /// \return average unordered lookup set size. double getAvgUnorderedLookupSetSize(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); @@ -433,6 +573,8 @@ public: /// out of order by server - transaction id of received /// packet does not match transaction id of next sent packet. /// + /// \param xchg_type exchange type. + /// \throw isc::BadValue if invalid exchange type specified. /// \return number of unordered lookups. uint64_t getUnorderedLookups(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); @@ -447,14 +589,40 @@ public: /// If packets are skipped or received out of order, lookup /// function will use unordered lookup (with hash table). /// + /// \param xchg_type exchange type. + /// \throw isc::BadValue if invalid exchange type specified. /// \return number of ordered lookups. uint64_t getOrderedLookups(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); return xchg_stats->getOrderedLookups(); } -private: + /// \brief Return total number of sent packets + /// + /// Method returns total number of sent packets for specified + /// exchange type. + /// + /// \param xchg_type exchange type. + /// \throw isc::BadValue if invalid exchange type specified. + /// \return number of sent packets. + uint64_t getSentPacketsNum(const ExchangeType xchg_type) const { + ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); + return xchg_stats->getSentPacketsNum(); + } + /// \brief Return total number of received packets + /// + /// Method returns total number of received packets for specified + /// exchange type. + /// + /// \param xchg_type exchange type. + /// \throw isc::BadValue if invalid exchange type specified. + /// \return number of received packets. + uint64_t getRcvdPacketsNum(const ExchangeType xchg_type) const { + ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); + return xchg_stats->getRcvdPacketsNum(); + } +private: /// \brief Return exchange stats object for given exchange type /// /// Method returns exchange stats object for given exchange type. diff --git a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc index 2d6ab14d54..38cd1c28ac 100644 --- a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc +++ b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc @@ -16,6 +16,7 @@ #include #include +#include #include #include @@ -43,21 +44,79 @@ public: Pkt4* createPacket4(const uint8_t msg_type, const uint32_t transid) { Pkt4* pkt = new Pkt4(msg_type, transid); + // Packet timestamp is normally updated by interface + // manager on packets reception or send. Unit tests + // do not use interface manager so we need to do it + // ourselfs. pkt->updateTimestamp(); return pkt; } + + Pkt6* createPacket6(const uint8_t msg_type, + const uint32_t transid) { + Pkt6* pkt = new Pkt6(msg_type, transid); + // Packet timestamp is normally updated by interface + // manager on packets reception or send. Unit tests + // do not use interface manager so we need to do it + // ourselfs. + pkt->updateTimestamp(); + return pkt; + } + + void passMultiplePackets6(const boost::shared_ptr stats_mgr, + const StatsMgr6::ExchangeType xchg_type, + const uint8_t packet_type, + const int num_packets, + const bool receive = false) { + for (int i = 0; i < num_packets; ++i) { + boost::shared_ptr + packet(createPacket6(packet_type, i)); + + if (receive) { + ASSERT_NO_THROW( + stats_mgr->passRcvdPacket(xchg_type, packet); + ); + } else { + ASSERT_NO_THROW( + stats_mgr->passSentPacket(xchg_type, packet) + ); + } + } + } + }; TEST_F(StatsMgrTest, Constructor) { boost::scoped_ptr stats_mgr(new StatsMgr4()); + stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); + EXPECT_EQ( + std::numeric_limits::max(), + stats_mgr->getMinDelay(StatsMgr4::XCHG_DO) + ); + EXPECT_EQ(0, stats_mgr->getMaxDelay(StatsMgr4::XCHG_DO)); + EXPECT_EQ(0, stats_mgr->getSumDelay(StatsMgr4::XCHG_DO)); + EXPECT_EQ(0, stats_mgr->getOrphans(StatsMgr4::XCHG_DO)); + EXPECT_EQ(0, stats_mgr->getSquareSumDelay(StatsMgr4::XCHG_DO)); + EXPECT_EQ(0, stats_mgr->getOrderedLookups(StatsMgr4::XCHG_DO)); + EXPECT_EQ(0, stats_mgr->getUnorderedLookups(StatsMgr4::XCHG_DO)); + EXPECT_EQ(0, stats_mgr->getSentPacketsNum(StatsMgr4::XCHG_DO)); + EXPECT_EQ(0, stats_mgr->getRcvdPacketsNum(StatsMgr4::XCHG_DO)); + + double avg_size = 0.; + ASSERT_NO_THROW( + avg_size = stats_mgr->getAvgUnorderedLookupSetSize(StatsMgr4::XCHG_DO); + ); + EXPECT_EQ(0., avg_size); } -TEST_F(StatsMgrTest, Exchanges) { +TEST_F(StatsMgrTest, Exchange) { boost::scoped_ptr stats_mgr(new StatsMgr4()); boost::shared_ptr sent_packet(createPacket4(DHCPDISCOVER, common_transid)); boost::shared_ptr rcvd_packet(createPacket4(DHCPOFFER, common_transid)); + // This is expected to throw because XCHG_DO was not yet + // added to Stats Manager for tracking. EXPECT_THROW( stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet), BadValue @@ -67,7 +126,10 @@ TEST_F(StatsMgrTest, Exchanges) { BadValue ); + // Adding DISCOVER-OFFER exchanges to be tracked by Stats Manager. stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); + // The following two attempts are expected to throw because + // invalid exchange types are passed (XCHG_RA instead of XCHG_DO) EXPECT_THROW( stats_mgr->passSentPacket(StatsMgr4::XCHG_RA, sent_packet), BadValue @@ -77,6 +139,8 @@ TEST_F(StatsMgrTest, Exchanges) { BadValue ); + // The following two attempts are expected to run fine because + // right exchange type is specified. EXPECT_NO_THROW( stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet) ); @@ -85,6 +149,46 @@ TEST_F(StatsMgrTest, Exchanges) { ); } +TEST_F(StatsMgrTest, MultipleExchanges) { + boost::shared_ptr stats_mgr(new StatsMgr6()); + stats_mgr->addExchangeStats(StatsMgr6::XCHG_SA); + stats_mgr->addExchangeStats(StatsMgr6::XCHG_RR); + + // Simulate sending number of solicit packets. + const int solicit_packets_num = 10; + passMultiplePackets6(stats_mgr, StatsMgr6::XCHG_SA, DHCPV6_SOLICIT, + solicit_packets_num); + + // Simulate sending number of request packets. It is important that + // number of request packets is different then number of solicit + // packets. We can now check if right number packets went to + // the right exchange type group. + const int request_packets_num = 5; + passMultiplePackets6(stats_mgr, StatsMgr6::XCHG_RR, DHCPV6_REQUEST, + request_packets_num); + + // Check if all packets are successfuly passed to packet lists. + EXPECT_EQ(solicit_packets_num, + stats_mgr->getSentPacketsNum(StatsMgr6::XCHG_SA)); + EXPECT_EQ(request_packets_num, + stats_mgr->getSentPacketsNum(StatsMgr6::XCHG_RR)); + + // Simulate reception of multiple packets for both SOLICIT-ADVERTISE + // and REQUEST-REPLY exchanges. Assume no packet drops. + const bool receive_packets = true; + passMultiplePackets6(stats_mgr, StatsMgr6::XCHG_SA, DHCPV6_ADVERTISE, + solicit_packets_num, receive_packets); + + passMultiplePackets6(stats_mgr, StatsMgr6::XCHG_RR, DHCPV6_REPLY, + request_packets_num, receive_packets); + + // Verify that all received packets are counted. + EXPECT_EQ(solicit_packets_num, + stats_mgr->getRcvdPacketsNum(StatsMgr6::XCHG_SA)); + EXPECT_EQ(request_packets_num, + stats_mgr->getRcvdPacketsNum(StatsMgr6::XCHG_RR)); +} + TEST_F(StatsMgrTest, SendReceiveSimple) { boost::scoped_ptr stats_mgr(new StatsMgr4()); boost::shared_ptr sent_packet(createPacket4(DHCPDISCOVER, @@ -92,15 +196,22 @@ TEST_F(StatsMgrTest, SendReceiveSimple) { boost::shared_ptr rcvd_packet(createPacket4(DHCPOFFER, common_transid)); stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); + // The following attempt is expected to pass becase the right + // exchange type is used. ASSERT_NO_THROW( stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet) ); - EXPECT_NO_THROW( + // It is ok, to pass to received packets here. First one will + // be matched with sent packet. The latter one will not be + // matched with sent packet but orphans counter will simply + // increase. + ASSERT_NO_THROW( stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet) ); - EXPECT_NO_THROW( + ASSERT_NO_THROW( stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet) ); + EXPECT_EQ(1, stats_mgr->getOrphans(StatsMgr4::XCHG_DO)); } TEST_F(StatsMgrTest, SendReceiveUnordered) { @@ -108,7 +219,9 @@ TEST_F(StatsMgrTest, SendReceiveUnordered) { boost::scoped_ptr stats_mgr(new StatsMgr4()); stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); - uint32_t transid[packets_num] = { 1, 1024, 2, 1025, 3, 1026, 4, 1027, 5, 1028 }; + // Transaction ids of 10 packets to be sent and received. + uint32_t transid[packets_num] = + { 1, 1024, 2, 1025, 3, 1026, 4, 1027, 5, 1028 }; for (int i = 0; i < packets_num; ++i) { boost::shared_ptr sent_packet(createPacket4(DHCPDISCOVER, transid[i])); @@ -117,16 +230,24 @@ TEST_F(StatsMgrTest, SendReceiveUnordered) { ); } + // We are simulating that received packets are coming in reverse order: + // 1028, 5, 1027 .... for (int i = 0; i < packets_num; ++i) { - boost::shared_ptr rcvd_packet(createPacket4(DHCPDISCOVER, - transid[packets_num - 1 - i])); + boost::shared_ptr + rcvd_packet(createPacket4(DHCPDISCOVER, + transid[packets_num - 1 - i])); ASSERT_NO_THROW( stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet); ); } + // All packets are expected to match (we did not drop any) EXPECT_EQ(0, stats_mgr->getOrphans(StatsMgr4::XCHG_DO)); - EXPECT_EQ(10, stats_mgr->getUnorderedLookups(StatsMgr4::XCHG_DO)); - std::cout << stats_mgr->getAvgUnorderedLookupSetSize(StatsMgr4::XCHG_DO) << std::endl; + // Most of the time we have to do unordered lookups except for the last + // one. Packets are removed from the sent list every time we have a match + // so eventually we come up with the single packet that caching iterator + // is pointing to. This is counted as ordered lookup. + EXPECT_EQ(1, stats_mgr->getOrderedLookups(StatsMgr4::XCHG_DO)); + EXPECT_EQ(9, stats_mgr->getUnorderedLookups(StatsMgr4::XCHG_DO)); } TEST_F(StatsMgrTest, Orphans) { @@ -134,19 +255,73 @@ TEST_F(StatsMgrTest, Orphans) { boost::scoped_ptr stats_mgr(new StatsMgr4()); stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); + // We skip every second packet to simulate drops. for (int i = 0; i < packets_num; i += 2) { boost::shared_ptr sent_packet(createPacket4(DHCPDISCOVER, i)); ASSERT_NO_THROW( stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet) ); } + // We pass all received packets. for (int i = 0; i < packets_num; ++i) { boost::shared_ptr rcvd_packet(createPacket4(DHCPOFFER, i)); ASSERT_NO_THROW( stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet); ); } + // The half of received packets are expected not to have matching + // sent packet. EXPECT_EQ(packets_num / 2, stats_mgr->getOrphans(StatsMgr4::XCHG_DO)); } +TEST_F(StatsMgrTest, Delays) { + boost::scoped_ptr stats_mgr(new StatsMgr4()); + stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); + + boost::shared_ptr sent_packet(createPacket4(DHCPDISCOVER, + common_transid)); + ASSERT_NO_THROW( + stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet) + ); + + // There is way to differentiate timstamps of two packets other than + // sleep for before we create another packet. Packet is using current + // time to update its timestamp. + // Sleeping for two seconds will guarantee that delay between packets + // will be greater than 1 second. Note that posix time value is + // transformed to double value and it makes it hard to determine + // actual value to expect. + std::cout << "Sleeping for 2 seconds to test packet delays" << std::endl; + sleep(2); + + boost::shared_ptr rcvd_packet(createPacket4(DHCPOFFER, + common_transid)); + ASSERT_NO_THROW( + stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet); + ); + + // Calculate period between packets. + boost::posix_time::ptime sent_time = sent_packet->getTimestamp(); + boost::posix_time::ptime rcvd_time = rcvd_packet->getTimestamp(); + + ASSERT_FALSE(sent_time.is_not_a_date_time()); + ASSERT_FALSE(rcvd_time.is_not_a_date_time()); + + boost::posix_time::time_period period(sent_time, rcvd_time); + + // Initially min delay is equal to MAX_DOUBLE. After first packets + // are passed, it is expected to set to actual value. + EXPECT_LT(stats_mgr->getMinDelay(StatsMgr4::XCHG_DO), + std::numeric_limits::max()); + EXPECT_GT(stats_mgr->getMinDelay(StatsMgr4::XCHG_DO), 1); + + // Max delay is supposed to the same value as mininimum + // or maximum delay. + EXPECT_GT(stats_mgr->getMaxDelay(StatsMgr4::XCHG_DO), 1); + + // Delay sums are now the same as minimum or maximum delay. + EXPECT_GT(stats_mgr->getSumDelay(StatsMgr4::XCHG_DO), 1); + EXPECT_GT(stats_mgr->getSquareSumDelay(StatsMgr4::XCHG_DO), 1); +} + } From f3a0b65d587b0758416bfb596a9ad696a6c9fb25 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Thu, 19 Jul 2012 23:28:13 +0200 Subject: [PATCH 11/69] [1958] Added custom counters and unit tests. --- tests/tools/perfdhcp/stats_mgr.h | 119 +++++++++++++++++- .../perfdhcp/tests/stats_mgr_unittest.cc | 38 ++++++ 2 files changed, 156 insertions(+), 1 deletion(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index 0fed0ae0a2..fffdf21c9f 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -52,6 +52,68 @@ template class StatsMgr : public boost::noncopyable { public: + /// \brief Custom Counter + /// + /// This class represents custom statistics counters. Client class + /// may create unlimited number of counters. Such counters are + /// being stored in map in Statistics Manager and access using + /// unique string key. + class CustomCounter { + public: + /// \brief Constructor. + /// + /// This constructor sets counter name. This name is used in + /// log file to report value of each counter. + /// + /// \param name name of the counter used in log file. + CustomCounter(const std::string& name) : + counter_(0), + name_(name) { }; + + /// \brief Increment operator. + const CustomCounter& operator++() { + ++counter_; + return *this; + } + + /// \brief Increment operator. + const CustomCounter& operator++(int) { + CustomCounter& this_counter(*this); + operator++(); + return this_counter; + } + + /// \brief Return counter value. + /// + /// Method returns counter value. + /// + /// \return counter value. + uint64_t getValue() const { + return counter_; + } + + /// \brief Return counter name. + /// + /// Method returns counter name. + /// + /// \return counter name. + const std::string& getName() const { + return name_; + } + private: + /// \brief Default constructor. + /// + /// Default constrcutor is private because we don't want client + /// class to call it because we want client class to specify + /// counter's name. + CustomCounter() { }; + + uint64_t counter_; ///< Counter's value. + std::string name_; ///< Counter's name. + }; + + typedef typename boost::shared_ptr CustomCounterPtr; + /// DHCP packet exchange types. enum ExchangeType { XCHG_DO, ///< DHCPv4 DISCOVER-OFFER @@ -431,6 +493,16 @@ public: typedef typename std::map ExchangesMap; /// Iterator poiting to \ref ExchangesMap typedef typename ExchangesMap::const_iterator ExchangesMapIterator; + /// Map containing custom counters. + typedef typename std::map CustomCountersMap; + /// Iterator for \ref CustomCountersMap. + typedef typename CustomCountersMap::iterator CustomCountersMapIterator; + + /// \brief Constructor. + StatsMgr() + : exchanges_(), + custom_counters_() { + } /// \brief Specify new exchange type. /// @@ -447,6 +519,49 @@ public: exchanges_[xchg_type] = ExchangeStatsPtr(new ExchangeStats(xchg_type)); } + /// \brief Add named custom uint64 counter. + /// + /// Method creates new named counter and stores in counter's map under + /// key specified here as short_name. + /// + /// \param short_name key to use to access counter in the map. + /// \param long_name name of the counter presented in the log file. + void addCustomCounter(const std::string& short_name, + const std::string& long_name) { + if (custom_counters_.find(short_name) != custom_counters_.end()) { + isc_throw(BadValue, + "Custom counter " << short_name << " already added."); + } + custom_counters_[short_name] = + CustomCounterPtr(new CustomCounter(long_name)); + } + + /// \brief Return specified counter. + /// + /// Method returns specified counter. + /// + /// \param counter_key key poiting to the counter in the counters map. + /// \return pointer to specified counter object. + CustomCounterPtr getCounter(const std::string& counter_key) { + CustomCountersMapIterator it = custom_counters_.find(counter_key); + if (it == custom_counters_.end()) { + isc_throw(BadValue, + "Custom counter " << counter_key << "does not exist"); + } + return it->second; + } + + /// \brief Increment specified counter. + /// + /// Increement counter value by one. + /// + /// \param counter_key key poitinh to the counter in the counters map. + /// \return pointer to specified counter after incrementation. + const CustomCounter& IncrementCounter(const std::string& counter_key) { + CustomCounterPtr counter = getCounter(counter_key); + return ++(*counter); + } + /// \brief Adds new packet to the sent packets list. /// /// Method adds new packet to the sent packets list. @@ -622,6 +737,7 @@ public: ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); return xchg_stats->getRcvdPacketsNum(); } + private: /// \brief Return exchange stats object for given exchange type /// @@ -639,7 +755,8 @@ private: return xchg_stats; } - ExchangesMap exchanges_; ///< Map of exchange types. + ExchangesMap exchanges_; ///< Map of exchange types. + CustomCountersMap custom_counters_; ///< Map with custom counters. }; } // namespace perfdhcp diff --git a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc index 38cd1c28ac..98e277dee0 100644 --- a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc +++ b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc @@ -324,4 +324,42 @@ TEST_F(StatsMgrTest, Delays) { EXPECT_GT(stats_mgr->getSquareSumDelay(StatsMgr4::XCHG_DO), 1); } +TEST_F(StatsMgrTest, CustomCounters) { + boost::scoped_ptr stats_mgr(new StatsMgr4()); + + // Specify counter keys and names. + const std::string too_short_key("tooshort"); + const std::string too_short_name("Too short packets"); + const std::string too_late_key("toolate"); + const std::string too_late_name("Packets sent too late"); + + // Add two custom counters. + stats_mgr->addCustomCounter(too_short_key, too_short_name); + stats_mgr->addCustomCounter(too_late_key, too_late_name); + + // Increment one of the counters 10 times. + const uint64_t tooshort_num = 10; + for (uint64_t i = 0; i < tooshort_num; ++i) { + stats_mgr->IncrementCounter(too_short_key); + } + + // Increment another counter by 5 times. + const uint64_t toolate_num = 5; + for (uint64_t i = 0; i < toolate_num; ++i) { + stats_mgr->IncrementCounter(too_late_key); + } + + // Check counter's current value and name. + StatsMgr4::CustomCounterPtr tooshort_counter = + stats_mgr->getCounter(too_short_key); + EXPECT_EQ(too_short_name, tooshort_counter->getName()); + EXPECT_EQ(tooshort_num, tooshort_counter->getValue()); + + // Check counter's current value and name. + StatsMgr4::CustomCounterPtr toolate_counter = + stats_mgr->getCounter(too_late_key); + EXPECT_EQ(too_late_name, toolate_counter->getName()); + EXPECT_EQ(toolate_num, toolate_counter->getValue()); +} + } From bd1c6a351401bbd9b88a86c6e159083e4421398b Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Thu, 19 Jul 2012 23:39:36 +0200 Subject: [PATCH 12/69] [1958] Fixed minor issues with comments in stats_mgr.h --- tests/tools/perfdhcp/stats_mgr.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index fffdf21c9f..3c12286cd6 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -280,7 +280,7 @@ public: bool packet_found = false; // Most likely responses are sent from the server in the same // order as client's requests to the server. We are caching - // next sent packet and first try to match with it the next + // next sent packet and first try to match it with the next // incoming packet. We are successful if there is no // packet drop or out of order packets sent. This is actually // the fastest way to look for packets. @@ -293,16 +293,16 @@ public: // take a little more expensive approach to look packets using // alternative index (transaction id & 1023). PktListTransidIndex& idx = sent_packets_.template get<1>(); - // Packets are grouped using trasaction id masking with value + // Packets are grouped using trasaction id masked with value // of 1023. For instance, packets with transaction id equal to // 1, 1024 ... will belong to the same group (a.k.a. bucket). // When using alternative index we don't find the packet but - // bucket of packets and need to iterate through the bucket + // bucket of packets and we need to iterate through the bucket // to find the one that has desired transaction id. std::pair p = idx.equal_range(transid & 1023); // We want to keep statistics of unordered lookups to make - // sure that there is a right balance before number of + // sure that there is a right balance between number of // unordered lookups and ordered lookups. If number of unordered // lookups is high it may mean that many packets are lost or // sent out of order. From a14dd1bb813a03e9cedecc44b237c24fc0dbe08e Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Fri, 20 Jul 2012 21:05:00 +0200 Subject: [PATCH 13/69] [1958] Prefer const iterators and pointers where possible. --- tests/tools/perfdhcp/stats_mgr.h | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index 3c12286cd6..cae102930b 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -23,7 +23,6 @@ #include #include #include -#include #include #include @@ -132,7 +131,7 @@ public: class ExchangeStats { public: - static uint32_t transid_hash(boost::shared_ptr packet) { + static uint32_t transid_hash(const boost::shared_ptr packet) { return packet->getTransid() & 1023; } @@ -156,12 +155,12 @@ public: > PktList; /// Packet list iterator for sequencial access to elements. - typedef typename PktList::iterator PktListIterator; + typedef typename PktList::const_iterator PktListIterator; /// Packet list index to search packets using transaction id. typedef typename PktList::template nth_index<1>::type PktListTransidIndex; /// Packet list iterator to access packets using transaction id. - typedef typename PktListTransidIndex::iterator PktListTransidIterator; + typedef typename PktListTransidIndex::const_iterator PktListTransidIterator; /// \brief Constructor /// @@ -209,8 +208,8 @@ public: /// \param sent_packet sent packet /// \param rcvd_packet received packet /// \throw isc::Unexpected if failed to calculate timestamps - void updateDelays(const boost::shared_ptr sent_packet, - const boost::shared_ptr rcvd_packet) { + void updateDelays(const boost::shared_ptr sent_packet, + const boost::shared_ptr rcvd_packet) { boost::posix_time::ptime sent_time = sent_packet->getTimestamp(); boost::posix_time::ptime rcvd_time = rcvd_packet->getTimestamp(); @@ -496,7 +495,7 @@ public: /// Map containing custom counters. typedef typename std::map CustomCountersMap; /// Iterator for \ref CustomCountersMap. - typedef typename CustomCountersMap::iterator CustomCountersMapIterator; + typedef typename CustomCountersMap::const_iterator CustomCountersMapIterator; /// \brief Constructor. StatsMgr() From f973e64243cb9b616201247f86caffa8965a6928 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Mon, 23 Jul 2012 15:10:40 +0200 Subject: [PATCH 14/69] [1708] Fixed python test for skip msgq connection. --- src/bin/dhcp6/tests/dhcp6_test.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/bin/dhcp6/tests/dhcp6_test.py b/src/bin/dhcp6/tests/dhcp6_test.py index e84bad198c..123b10599d 100644 --- a/src/bin/dhcp6/tests/dhcp6_test.py +++ b/src/bin/dhcp6/tests/dhcp6_test.py @@ -171,7 +171,7 @@ class TestDhcpv6Daemon(unittest.TestCase): def test_skip_msgq(self): print("Check that connection to BIND10 msgq can be disabled.") - (returncode, output, error) = self.runDhcp4(['../b10-dhcp6', '-s', '-p', '10547']) + (returncode, output, error) = self.runCommand(['../b10-dhcp6', '-s', '-p', '10547']) # When invalid port number is specified, return code must not be success # TODO: Temporarily commented out as socket binding on systems that do not have From 98c3fe102ae4b4e8769d2bf9412a5e5ab9225926 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Mon, 23 Jul 2012 16:22:42 +0200 Subject: [PATCH 15/69] [1958] Use hashing function for indexing and unordered search. --- tests/tools/perfdhcp/stats_mgr.h | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index cae102930b..7732bce8eb 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -131,7 +131,7 @@ public: class ExchangeStats { public: - static uint32_t transid_hash(const boost::shared_ptr packet) { + static uint32_t hashTransid(const boost::shared_ptr packet) { return packet->getTransid() & 1023; } @@ -148,7 +148,7 @@ public: boost::multi_index::global_fun< boost::shared_ptr, uint32_t, - &ExchangeStats::transid_hash + &ExchangeStats::hashTransid > > > @@ -259,7 +259,7 @@ public: /// \param transid transaction id of the packet to search /// \return packet having specified transaction or NULL if packet /// not found - boost::shared_ptr findSent(const uint32_t transid) { + boost::shared_ptr findSent(const boost::shared_ptr rcvd_packet) { if (sent_packets_.size() == 0) { // List of sent packets is empty so there is no sense // to continue looking fo the packet. It also means @@ -283,7 +283,7 @@ public: // incoming packet. We are successful if there is no // packet drop or out of order packets sent. This is actually // the fastest way to look for packets. - if ((*sent_packets_cache_)->getTransid() == transid) { + if ((*sent_packets_cache_)->getTransid() == rcvd_packet->getTransid()) { ++ordered_lookups_; packet_found = true; } else { @@ -299,7 +299,7 @@ public: // bucket of packets and we need to iterate through the bucket // to find the one that has desired transaction id. std::pair p = - idx.equal_range(transid & 1023); + idx.equal_range(hashTransid(rcvd_packet)); // We want to keep statistics of unordered lookups to make // sure that there is a right balance between number of // unordered lookups and ordered lookups. If number of unordered @@ -312,7 +312,7 @@ public: unordered_lookup_size_sum_ += std::distance(p.first, p.second); for (PktListTransidIterator it = p.first; it != p.second; ++it) { - if ((*it)->getTransid() == transid) { + if ((*it)->getTransid() == rcvd_packet->getTransid()) { packet_found = true; sent_packets_cache_ = sent_packets_.template project<0>(it); @@ -592,7 +592,7 @@ public: const boost::shared_ptr packet) { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); boost::shared_ptr sent_packet - = xchg_stats->findSent(packet->getTransid()); + = xchg_stats->findSent(packet); if (sent_packet) { xchg_stats->updateDelays(sent_packet, packet); From e1d0d73d043c58305c1affbee3abe52cdee92289 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Mon, 23 Jul 2012 16:35:54 +0200 Subject: [PATCH 16/69] [1708] DHCPv{4,6} start-up fixes: - better exception handling in dhcpv{4,6}Srv constructors - port number is now parsed with boost::lexical_cast - new tests for port numbers implemented - ControlledDhcpv{4,6}Srv objects are now automatic - runDhcpv4() method renamed to runCommand() to better prepare for upcoming 4/6 test merge. --- src/bin/dhcp4/dhcp4_srv.cc | 19 ++++++++++------- src/bin/dhcp4/main.cc | 20 ++++++++++-------- src/bin/dhcp4/tests/dhcp4_test.py | 34 +++++++++++++++++++++++++------ src/bin/dhcp6/dhcp6_srv.cc | 29 +++++++++++++------------- src/bin/dhcp6/main.cc | 19 ++++++++++------- src/bin/dhcp6/tests/dhcp6_test.py | 26 ++++++++++++++++++++--- 6 files changed, 102 insertions(+), 45 deletions(-) diff --git a/src/bin/dhcp4/dhcp4_srv.cc b/src/bin/dhcp4/dhcp4_srv.cc index 4fb0a771a4..a86b010792 100644 --- a/src/bin/dhcp4/dhcp4_srv.cc +++ b/src/bin/dhcp4/dhcp4_srv.cc @@ -37,16 +37,21 @@ const std::string HARDCODED_SERVER_ID = "192.0.2.1"; Dhcpv4Srv::Dhcpv4Srv(uint16_t port) { cout << "Initialization: opening sockets on port " << port << endl; - // first call to instance() will create IfaceMgr (it's a singleton) - // it may throw something if things go wrong - IfaceMgr::instance(); + try { + // first call to instance() will create IfaceMgr (it's a singleton) + // it may throw something if things go wrong + IfaceMgr::instance(); - /// @todo: instantiate LeaseMgr here once it is imlpemented. - IfaceMgr::instance().printIfaces(); + /// @todo: instantiate LeaseMgr here once it is imlpemented. - IfaceMgr::instance().openSockets4(port); + IfaceMgr::instance().openSockets4(port); - setServerID(); + setServerID(); + } catch (const std::exception &e) { + cerr << "Error during DHCPv4 server startup: " << e.what() << endl; + shutdown_ = true; + return; + } shutdown_ = false; } diff --git a/src/bin/dhcp4/main.cc b/src/bin/dhcp4/main.cc index 472f46f467..1087cc7fb6 100644 --- a/src/bin/dhcp4/main.cc +++ b/src/bin/dhcp4/main.cc @@ -17,6 +17,7 @@ #include #include #include +#include using namespace std; using namespace isc::dhcp; @@ -64,8 +65,14 @@ main(int argc, char* argv[]) { stand_alone = true; break; case 'p': - port_number = strtol(optarg, NULL, 10); - if (port_number == 0) { + try { + port_number = boost::lexical_cast(optarg); + } catch (const boost::bad_lexical_cast &) { + cerr << "Failed to parse port number: [" << optarg + << "], 1-65535 allowed." << endl; + usage(); + } + if (port_number <= 0 || port_number > 65535) { cerr << "Failed to parse port number: [" << optarg << "], 1-65535 allowed." << endl; usage(); @@ -97,11 +104,11 @@ main(int argc, char* argv[]) { cout << "[b10-dhcp4] Initiating DHCPv4 server operation." << endl; /// @todo: pass verbose to the actul server once logging is implemented - ControlledDhcpv4Srv* server = new ControlledDhcpv4Srv(port_number); + ControlledDhcpv4Srv server(port_number); if (!stand_alone) { try { - server->establishSession(); + server.establishSession(); } catch (const std::exception& ex) { cerr << "Failed to establish BIND10 session. " "Running in stand-alone mode:" << ex.what() << endl; @@ -112,10 +119,7 @@ main(int argc, char* argv[]) { cout << "Skipping connection to the BIND10 msgq." << endl; } - server->run(); - delete server; - server = NULL; - + server.run(); } catch (const std::exception& ex) { cerr << "[b10-dhcp4] Server failed: " << ex.what() << endl; ret = EXIT_FAILURE; diff --git a/src/bin/dhcp4/tests/dhcp4_test.py b/src/bin/dhcp4/tests/dhcp4_test.py index 065c80c2ad..935bba645a 100644 --- a/src/bin/dhcp4/tests/dhcp4_test.py +++ b/src/bin/dhcp4/tests/dhcp4_test.py @@ -34,7 +34,7 @@ class TestDhcpv4Daemon(unittest.TestCase): def tearDown(self): pass - def runDhcp4(self, params, wait=1): + def runCommand(self, params, wait=1): """ This method runs dhcp4 and returns a touple: (returncode, stdout, stderr) """ @@ -127,14 +127,14 @@ class TestDhcpv4Daemon(unittest.TestCase): print("Note: Purpose of some of the tests is to check if DHCPv4 server can be started,") print(" not that is can bind sockets correctly. Please ignore binding errors.") - (returncode, output, error) = self.runDhcp4(["../b10-dhcp4", "-v"]) + (returncode, output, error) = self.runCommand(["../b10-dhcp4", "-v"]) self.assertEqual( str(output).count("[b10-dhcp4] Initiating DHCPv4 server operation."), 1) def test_portnumber_0(self): print("Check that specifying port number 0 is not allowed.") - (returncode, output, error) = self.runDhcp4(['../b10-dhcp4', '-p', '0']) + (returncode, output, error) = self.runCommand(['../b10-dhcp4', '-p', '0']) # When invalid port number is specified, return code must not be success self.assertTrue(returncode != 0) @@ -145,7 +145,7 @@ class TestDhcpv4Daemon(unittest.TestCase): def test_portnumber_missing(self): print("Check that -p option requires a parameter.") - (returncode, output, error) = self.runDhcp4(['../b10-dhcp4', '-p']) + (returncode, output, error) = self.runCommand(['../b10-dhcp4', '-p']) # When invalid port number is specified, return code must not be success self.assertTrue(returncode != 0) @@ -153,10 +153,32 @@ class TestDhcpv4Daemon(unittest.TestCase): # Check that there is an error message about invalid port number printed on stderr self.assertEqual( str(error).count("option requires an argument"), 1) + def test_portnumber_invalid1(self): + print("Check that -p option is check against bogus port number (999999).") + + (returncode, output, error) = self.runCommand(['../b10-dhcp4', '-p','999999']) + + # When invalid port number is specified, return code must not be success + self.assertTrue(returncode != 0) + + # Check that there is an error message about invalid port number printed on stderr + self.assertEqual( str(error).count("Failed to parse port number"), 1) + + def test_portnumber_invalid2(self): + print("Check that -p option is check against bogus port number (123garbage).") + + (returncode, output, error) = self.runCommand(['../b10-dhcp4', '-p','123garbage']) + + # When invalid port number is specified, return code must not be success + self.assertTrue(returncode != 0) + + # Check that there is an error message about invalid port number printed on stderr + self.assertEqual( str(error).count("Failed to parse port number"), 1) + def test_portnumber_nonroot(self): print("Check that specifying unprivileged port number will work.") - (returncode, output, error) = self.runDhcp4(['../b10-dhcp4', '-p', '10057']) + (returncode, output, error) = self.runCommand(['../b10-dhcp4', '-s', '-p', '10057']) # When invalid port number is specified, return code must not be success # TODO: Temporarily commented out as socket binding on systems that do not have @@ -169,7 +191,7 @@ class TestDhcpv4Daemon(unittest.TestCase): def test_skip_msgq(self): print("Check that connection to BIND10 msgq can be disabled.") - (returncode, output, error) = self.runDhcp4(['../b10-dhcp4', '-s', '-p', '10057']) + (returncode, output, error) = self.runCommand(['../b10-dhcp4', '-s', '-p', '10057']) # When invalid port number is specified, return code must not be success # TODO: Temporarily commented out as socket binding on systems that do not have diff --git a/src/bin/dhcp6/dhcp6_srv.cc b/src/bin/dhcp6/dhcp6_srv.cc index d28d9b7d21..9b43f5374f 100644 --- a/src/bin/dhcp6/dhcp6_srv.cc +++ b/src/bin/dhcp6/dhcp6_srv.cc @@ -45,24 +45,25 @@ Dhcpv6Srv::Dhcpv6Srv(uint16_t port) { // first call to instance() will create IfaceMgr (it's a singleton) // it may throw something if things go wrong try { - IfaceMgr::instance(); + + if (IfaceMgr::instance().countIfaces() == 0) { + cout << "Failed to detect any network interfaces. Aborting." << endl; + shutdown_ = true; + return; + } + + IfaceMgr::instance().openSockets6(port); + + setServerID(); + + /// @todo: instantiate LeaseMgr here once it is imlpemented. + } catch (const std::exception &e) { - cout << "Failed to instantiate InterfaceManager:" << e.what() << ". Aborting." << endl; + cerr << "Error during DHCPv4 server startup: " << e.what() << endl; shutdown_ = true; + return; } - if (IfaceMgr::instance().countIfaces() == 0) { - cout << "Failed to detect any network interfaces. Aborting." << endl; - shutdown_ = true; - } - - // Now try to open IPv6 sockets on detected interfaces. - IfaceMgr::instance().openSockets6(port); - - /// @todo: instantiate LeaseMgr here once it is imlpemented. - - setServerID(); - shutdown_ = false; } diff --git a/src/bin/dhcp6/main.cc b/src/bin/dhcp6/main.cc index 96e2dd14dd..aebee90699 100644 --- a/src/bin/dhcp6/main.cc +++ b/src/bin/dhcp6/main.cc @@ -17,6 +17,7 @@ #include #include #include +#include using namespace std; using namespace isc::dhcp; @@ -64,8 +65,14 @@ main(int argc, char* argv[]) { stand_alone = true; break; case 'p': - port_number = strtol(optarg, NULL, 10); - if (port_number == 0) { + try { + port_number = boost::lexical_cast(optarg); + } catch (const boost::bad_lexical_cast &) { + cerr << "Failed to parse port number: [" << optarg + << "], 1-65535 allowed." << endl; + usage(); + } + if (port_number <= 0 || port_number > 65535) { cerr << "Failed to parse port number: [" << optarg << "], 1-65535 allowed." << endl; usage(); @@ -97,11 +104,11 @@ main(int argc, char* argv[]) { cout << "b10-dhcp6: Initiating DHCPv6 server operation." << endl; /// @todo: pass verbose to the actual server once logging is implemented - ControlledDhcpv6Srv* server = new ControlledDhcpv6Srv(port_number); + ControlledDhcpv6Srv server(port_number); if (!stand_alone) { try { - server->establishSession(); + server.establishSession(); } catch (const std::exception& ex) { cerr << "Failed to establish BIND10 session. " "Running in stand-alone mode:" << ex.what() << endl; @@ -112,9 +119,7 @@ main(int argc, char* argv[]) { cout << "Skipping connection to the BIND10 msgq." << endl; } - server->run(); - delete server; - server = NULL; + server.run(); } catch (const std::exception& ex) { cerr << "[b10-dhcp6] Server failed: " << ex.what() << endl; diff --git a/src/bin/dhcp6/tests/dhcp6_test.py b/src/bin/dhcp6/tests/dhcp6_test.py index 123b10599d..399c370742 100644 --- a/src/bin/dhcp6/tests/dhcp6_test.py +++ b/src/bin/dhcp6/tests/dhcp6_test.py @@ -155,17 +155,38 @@ class TestDhcpv6Daemon(unittest.TestCase): # Check that there is an error message about invalid port number printed on stderr self.assertEqual( str(error).count("option requires an argument"), 1) + def test_portnumber_invalid1(self): + print("Check that -p option is check against bogus port number (999999).") + + (returncode, output, error) = self.runCommand(['../b10-dhcp6', '-p','999999']) + + # When invalid port number is specified, return code must not be success + self.assertTrue(returncode != 0) + + # Check that there is an error message about invalid port number printed on stderr + self.assertEqual( str(error).count("Failed to parse port number"), 1) + + def test_portnumber_invalid2(self): + print("Check that -p option is check against bogus port number (123garbage).") + + (returncode, output, error) = self.runCommand(['../b10-dhcp6', '-p','123garbage']) + + # When invalid port number is specified, return code must not be success + self.assertTrue(returncode != 0) + + # Check that there is an error message about invalid port number printed on stderr + self.assertEqual( str(error).count("Failed to parse port number"), 1) + def test_portnumber_nonroot(self): print("Check that specifying unprivileged port number will work.") - (returncode, output, error) = self.runCommand(['../b10-dhcp6', '-p', '10547']) + (returncode, output, error) = self.runCommand(['../b10-dhcp6', '-s', '-p', '10547']) # When invalid port number is specified, return code must not be success # TODO: Temporarily commented out as socket binding on systems that do not have # interface detection implemented currently fails. # self.assertTrue(returncode == 0) - # Check that there is a message on stdout about opening proper port self.assertEqual( str(output).count("opening sockets on port 10547"), 1) def test_skip_msgq(self): @@ -178,7 +199,6 @@ class TestDhcpv6Daemon(unittest.TestCase): # interface detection implemented currently fails. # self.assertTrue(returncode == 0) - # Check that there is an error message about invalid port number printed on stderr self.assertEqual( str(output).count("Skipping connection to the BIND10 msgq."), 1) From b594cc0f845bf8c2fa7238aa8d153f642a3337ff Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Mon, 23 Jul 2012 16:58:43 +0200 Subject: [PATCH 17/69] [1958] Check for null pointers. --- tests/tools/perfdhcp/stats_mgr.h | 40 ++++++++++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index 7732bce8eb..bfebfb2765 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -131,7 +131,20 @@ public: class ExchangeStats { public: + /// \brief Hash transaction id of the packet. + /// + /// Function hashes transaction id of the packet. Hashing is + /// non-unique. Many packets may have the same hash value and thus + /// they belong to the same packet buckets. Packet buckets are + /// used for unordered packets search with multi index container. + /// + /// \param packet packet which transaction id is to be hashed. + /// \throw isc::BadValue if packet is null. + /// \return transaction id hash. static uint32_t hashTransid(const boost::shared_ptr packet) { + if (!packet) { + isc_throw(BadValue, "Packet is null"); + } return packet->getTransid() & 1023; } @@ -185,7 +198,11 @@ public: /// Method adds new packet to list of sent packets. /// /// \param packet packet object to be added. + /// \throw isc::BadValue if packet is null. void appendSent(const boost::shared_ptr packet) { + if (!packet) { + isc_throw(BadValue, "Packet is null"); + } ++sent_packets_num_; sent_packets_.template get<0>().push_back(packet); } @@ -195,7 +212,11 @@ public: /// Method adds new packet to list of received packets. /// /// \param packet packet object to be added. + /// \throw isc::BadValue if packet is null. void appendRcvd(const boost::shared_ptr packet) { + if (!packet) { + isc_throw(BadValue, "Packet is null"); + } ++rcvd_packets_num_; rcvd_packets_.template get<0>().push_back(packet); } @@ -207,9 +228,17 @@ public: /// /// \param sent_packet sent packet /// \param rcvd_packet received packet + /// \throw isc::BadValue if sent or received packet is null. /// \throw isc::Unexpected if failed to calculate timestamps void updateDelays(const boost::shared_ptr sent_packet, const boost::shared_ptr rcvd_packet) { + if (!sent_packet) { + isc_throw(BadValue, "Sent packet is null"); + } + if (!rcvd_packet) { + isc_throw(BadValue, "Received packet is null"); + } + boost::posix_time::ptime sent_time = sent_packet->getTimestamp(); boost::posix_time::ptime rcvd_time = rcvd_packet->getTimestamp(); @@ -257,9 +286,14 @@ public: /// packet search time significantly. /// /// \param transid transaction id of the packet to search + /// \throw isc::BadValue if received packet is null. /// \return packet having specified transaction or NULL if packet /// not found boost::shared_ptr findSent(const boost::shared_ptr rcvd_packet) { + if (!rcvd_packet) { + isc_throw(BadValue, "Received packet is null"); + } + if (sent_packets_.size() == 0) { // List of sent packets is empty so there is no sense // to continue looking fo the packet. It also means @@ -569,7 +603,8 @@ public: /// /// \param xchg_type exchange type. /// \param packet packet to be added to the list - /// \throw isc::BadValue if invalid exchange type specified. + /// \throw isc::BadValue if invalid exchange type specified or + /// packet is null. void passSentPacket(const ExchangeType xchg_type, const boost::shared_ptr packet) { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); @@ -585,7 +620,8 @@ public: /// /// \param xchg_type exchange type. /// \param packet received packet - /// \throw isc::BadValue if invalid exchange type specified. + /// \throw isc::BadValue if invalid exchange type specified + /// or packet is null. /// \throw isc::Unexpected if corresponding packet was not /// found on the list of sent packets. void passRcvdPacket(const ExchangeType xchg_type, From c69039221efe837760e74c93f5bb23cb19810a48 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Mon, 23 Jul 2012 18:45:05 +0200 Subject: [PATCH 18/69] [1958] Doxygen warnings eliminated for perfdhcp. --- tests/tools/perfdhcp/stats_mgr.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index bfebfb2765..8e6c1097a9 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -285,7 +285,7 @@ public: /// the packet using indexing by transaction id. This reduces /// packet search time significantly. /// - /// \param transid transaction id of the packet to search + /// \param rcvd_packet received packet to be matched with sent packet. /// \throw isc::BadValue if received packet is null. /// \return packet having specified transaction or NULL if packet /// not found @@ -412,8 +412,8 @@ public: /// \brief Return average unordered lookup set size. /// /// Method returns average unordered lookup set size. - /// This value is changes every time \findSet function uses - /// unordered packet lookup using transaction id. + /// This value changes every time \ref ExchangeStats::findSent + /// function performs unordered packet lookup. /// /// \return average unordered lookup set size. double getAvgUnorderedLookupSetSize() const { @@ -705,8 +705,8 @@ public: /// \brief Return average unordered lookup set size. /// /// Method returns average unordered lookup set size. - /// This value is changes every time \findSet function uses - /// unordered packet lookup using transaction id. + /// This value changes every time \ref ExchangeStats::findSent + /// function performs unordered packet lookup. /// /// \param xchg_type exchange type. /// \throw isc::BadValue if invalid exchange type specified. From 48755af6bfc85aa10e5eea2da1bde9478914e8df Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Wed, 25 Jul 2012 13:19:46 +0200 Subject: [PATCH 19/69] [1958] Added perfdhcp statistics printing functions. --- src/lib/dhcp/pkt6.h | 2 +- tests/tools/perfdhcp/stats_mgr.h | 303 ++++++++++++++---- .../perfdhcp/tests/stats_mgr_unittest.cc | 131 +++++--- 3 files changed, 339 insertions(+), 97 deletions(-) diff --git a/src/lib/dhcp/pkt6.h b/src/lib/dhcp/pkt6.h index 2612f27046..b3a35679ef 100644 --- a/src/lib/dhcp/pkt6.h +++ b/src/lib/dhcp/pkt6.h @@ -139,7 +139,7 @@ public: /// Returns value of transaction-id field /// /// @return transaction-id - uint32_t getTransid() { return (transid_); }; + uint32_t getTransid() const { return (transid_); }; /// Adds an option to this packet. /// diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index 8e6c1097a9..6a3614990c 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -24,6 +24,7 @@ #include #include #include +#include #include #include @@ -87,7 +88,7 @@ public: /// Method returns counter value. /// /// \return counter value. - uint64_t getValue() const { + unsigned long long getValue() const { return counter_; } @@ -107,8 +108,8 @@ public: /// counter's name. CustomCounter() { }; - uint64_t counter_; ///< Counter's value. - std::string name_; ///< Counter's name. + unsigned long long counter_; ///< Counter's value. + std::string name_; ///< Counter's name. }; typedef typename boost::shared_ptr CustomCounterPtr; @@ -163,17 +164,31 @@ public: uint32_t, &ExchangeStats::hashTransid > + >, + boost::multi_index::hashed_non_unique< + boost::multi_index::const_mem_fun< + T, + uint32_t, + &T::getTransid + > > > > PktList; /// Packet list iterator for sequencial access to elements. typedef typename PktList::const_iterator PktListIterator; - /// Packet list index to search packets using transaction id. + /// Packet list index to search packets using transaction id hash. typedef typename PktList::template nth_index<1>::type + PktListTransidHashIndex; + /// Packet list iterator to access packets using transaction id hash. + typedef typename PktListTransidHashIndex::const_iterator + PktListTransidHashIterator; + /// Packet list index to search packets using transaction id. + typedef typename PktList::template nth_index<2>::type PktListTransidIndex; /// Packet list iterator to access packets using transaction id. - typedef typename PktListTransidIndex::const_iterator PktListTransidIterator; + typedef typename PktListTransidIndex::const_iterator + PktListTransidIterator; /// \brief Constructor /// @@ -218,7 +233,7 @@ public: isc_throw(BadValue, "Packet is null"); } ++rcvd_packets_num_; - rcvd_packets_.template get<0>().push_back(packet); + rcvd_packets_.push_back(packet); } /// \brief Update delay counters. @@ -325,14 +340,14 @@ public: // next incoming packet with next sent packet so we need to // take a little more expensive approach to look packets using // alternative index (transaction id & 1023). - PktListTransidIndex& idx = sent_packets_.template get<1>(); + PktListTransidHashIndex& idx = sent_packets_.template get<1>(); // Packets are grouped using trasaction id masked with value // of 1023. For instance, packets with transaction id equal to // 1, 1024 ... will belong to the same group (a.k.a. bucket). // When using alternative index we don't find the packet but // bucket of packets and we need to iterate through the bucket // to find the one that has desired transaction id. - std::pair p = + std::pair p = idx.equal_range(hashTransid(rcvd_packet)); // We want to keep statistics of unordered lookups to make // sure that there is a right balance between number of @@ -344,7 +359,7 @@ public: // bucket size the better. If bucket sizes appear to big we // might want to increase number of buckets. unordered_lookup_size_sum_ += std::distance(p.first, p.second); - for (PktListTransidIterator it = p.first; it != p.second; + for (PktListTransidHashIterator it = p.first; it != p.second; ++it) { if ((*it)->getTransid() == rcvd_packet->getTransid()) { packet_found = true; @@ -384,21 +399,39 @@ public: /// \return maximum delay between packets. double getMaxDelay() const { return max_delay_; } - /// \brief Return sum of delays between sent and received packets. + /// \brief Return avarage packet delay. /// - /// Method returns sum of delays between sent and received packets. + /// Method returns average packet delay. If no packets have been + /// received for this exchange avg delay can't be calculated and + /// thus method throws exception. /// - /// \return sum of delays between sent and received packets. - double getSumDelay() const { return sum_delay_; } + /// \throw isc::InvalidOperation if no packets for this exchange + /// have been received yet. + /// \return average packet delay. + double getAvgDelay() const { + if (sum_delay_ == 0) { + isc_throw(InvalidOperation, "no packets received"); + } + return sum_delay_ / rcvd_packets_num_; + } - /// \brief Return square sum of delays between sent and received - /// packets. + /// \brief Return standard deviation of packet delay. /// - /// Method returns square sum of delays between sent and received - /// packets. + /// Method returns standard deviation of packet delay. If no + /// packets have been received for this exchange, the standard + /// deviation can't be calculated and thus method throws + /// exception. /// - /// \return square sum of delays between sent and received packets. - double getSquareSumDelay() const { return square_sum_delay_; } + /// \throw isc::InvalidOperation if number of received packets + /// for the exchange is equal to zero. + /// \return standard deviation of packet delay. + double getStdDevDelay() const { + if (rcvd_packets_num_ == 0) { + isc_throw(InvalidOperation, "no packets received"); + } + return sqrt(square_sum_delay_ / rcvd_packets_num_ - + getAvgDelay() * getAvgDelay()); + } /// \brief Return number of orphant packets. /// @@ -407,7 +440,7 @@ public: /// for us. /// /// \return number of orphant received packets. - uint64_t getOrphans() const { return orphans_; } + unsigned long long getOrphans() const { return orphans_; } /// \brief Return average unordered lookup set size. /// @@ -415,10 +448,12 @@ public: /// This value changes every time \ref ExchangeStats::findSent /// function performs unordered packet lookup. /// + /// \throw isc::InvalidOperation if there have been no unordered + /// lookups yet. /// \return average unordered lookup set size. double getAvgUnorderedLookupSetSize() const { if (unordered_lookups_ == 0) { - return 0.; + isc_throw(InvalidOperation, "no unordered lookups"); } return static_cast(unordered_lookup_size_sum_) / static_cast(unordered_lookups_); @@ -432,7 +467,7 @@ public: /// packet does not match transaction id of next sent packet. /// /// \return number of unordered lookups. - uint64_t getUnorderedLookups() const { return unordered_lookups_; } + unsigned long long getUnorderedLookups() const { return unordered_lookups_; } /// \brief Return number of ordered sent packets lookups /// @@ -443,14 +478,14 @@ public: /// function will use unordered lookup (with hash table). /// /// \return number of ordered lookups. - uint64_t getOrderedLookups() const { return ordered_lookups_; } + unsigned long long getOrderedLookups() const { return ordered_lookups_; } /// \brief Return total number of sent packets /// /// Method returns total number of sent packets. /// /// \return number of sent packets. - uint64_t getSentPacketsNum() const { + unsigned long long getSentPacketsNum() const { return sent_packets_num_; } @@ -459,10 +494,58 @@ public: /// Method returns total number of received packets. /// /// \return number of received packets. - uint64_t getRcvdPacketsNum() const { + unsigned long long getRcvdPacketsNum() const { return rcvd_packets_num_; } + //// \brief Print timestamps for sent and received packets. + /// + /// Method prints timestamps for all sent and received packets for + /// packet exchange. + /// + /// \throw isc::InvalidOperation if found packet with no timestamp set. + void printTimestamps() { + // Iterate through all received packets. + for (PktListIterator it = rcvd_packets_.begin(); + it != rcvd_packets_.end(); + ++it) { + boost::shared_ptr rcvd_packet = *it; + // Search for corresponding sent packet using transaction id + // of received packet. + PktListTransidIndex& idx = archived_packets_.template get<2>(); + PktListTransidIterator it_archived = + idx.find(rcvd_packet->getTransid()); + // This should not happen that there is no corresponding + // sent packet. If it does however, we just drop the packet. + if (it_archived != idx.end()) { + boost::shared_ptr sent_packet = *it_archived; + // Get sent and received packet times. + boost::posix_time::ptime sent_time = + sent_packet->getTimestamp(); + boost::posix_time::ptime rcvd_time = + rcvd_packet->getTimestamp(); + // All sent and received packets should have timestamps + // set but if there is a bug somewhere and packet does + // not have timestamp we want to catch this here. + if (sent_time.is_not_a_date_time() || + rcvd_time.is_not_a_date_time()) { + isc_throw(InvalidOperation, "packet time is not set"); + } + // Calculate durations of packets from beginning of epoch. + boost::posix_time::ptime + epoch_time(boost::posix_time::min_date_time); + boost::posix_time::time_period + sent_period(epoch_time, sent_time); + boost::posix_time::time_period + rcvd_period(epoch_time, rcvd_time); + // Print timestamps for sent and received packet. + printf("sent/received times: %s / %s\n", + boost::posix_time::to_iso_string(sent_period.length()).c_str(), + boost::posix_time::to_iso_string(rcvd_period.length()).c_str()); + } + } + } + private: /// \brief Private default constructor. @@ -471,7 +554,6 @@ public: /// class to specify exchange type explicitely. ExchangeStats(); - /// \brief Erase packet from the list of sent packets. /// /// Method erases packet from the list of sent packets. @@ -480,7 +562,14 @@ public: /// \return iterator pointing to packet following erased /// packet or sent_packets_.end() if packet not found. PktListIterator eraseSent(const PktListIterator it) { - return sent_packets_.template get<0>().erase(it); + // We don't want to keep list of all sent packets + // because it will affect packet lookup performance. + // If packet is matched with received packet we + // move it to list of archived packets. List of + // archived packets may be used for diagnostics + // when test is completed. + archived_packets_.push_back(*it); + return sent_packets_.template get<0>().erase(it); } ExchangeType xchg_type_; ///< Packet exchange type. @@ -493,6 +582,11 @@ public: PktList rcvd_packets_; ///< List of received packets. + /// List of archived packets. All sent packets that have + /// been matched with received packet are moved to this + /// list for diagnostics purposes. + PktList archived_packets_; + double min_delay_; ///< Minimum delay between sent ///< and received packets. double max_delay_; ///< Maximum delay between sent @@ -502,22 +596,22 @@ public: double square_sum_delay_; ///< Square sum of delays between ///< sent and recived packets. - uint64_t orphans_; ///< Number of orphant received packets. + unsigned long long orphans_; ///< Number of orphant received packets. /// Sum of unordered lookup sets. Needed to calculate mean size of /// lookup set. It is desired that number of unordered lookups is /// minimal for performance reasons. Tracking number of lookups and /// mean size of the lookup set should give idea of packets serach /// complexity. - uint64_t unordered_lookup_size_sum_; + unsigned long long unordered_lookup_size_sum_; - uint64_t unordered_lookups_; ///< Number of unordered sent packets + unsigned long long unordered_lookups_; ///< Number of unordered sent packets ///< lookups. - uint64_t ordered_lookups_; ///< Number of ordered sent packets + unsigned long long ordered_lookups_; ///< Number of ordered sent packets ///< lookups. - uint64_t sent_packets_num_; ///< Total number of sent packets. - uint64_t rcvd_packets_num_; ///< Total number of received packets. + unsigned long long sent_packets_num_; ///< Total number of sent packets. + unsigned long long rcvd_packets_num_; ///< Total number of received packets. }; /// Pointer to ExchangeStats. @@ -662,31 +756,26 @@ public: return xchg_stats->getMaxDelay(); } - /// \brief Return sum of delays between sent and received packets. + /// \brief Return avarage packet delay. /// - /// Method returns sum of delays between sent and received packets - /// for specified exchange type. + /// Method returns average packet delay for specified + /// exchange type. /// - /// \param xchg_type exchange type. - /// \throw isc::BadValue if invalid exchange type specified. - /// \return sum of delays between sent and received packets. - double getSumDelay(const ExchangeType xchg_type) const { + /// \return average packet delay. + double getAvgDelay(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); - return xchg_stats->getSumDelay(); + return xchg_stats->getAvgDelay(); } - /// \brief Return square sum of delays between sent and received - /// packets. + /// \brief Return standard deviation of packet delay. /// - /// Method returns square sum of delays between sent and received - /// packets for specified exchange type. + /// Method returns standard deviation of packet delay + /// for specified exchange type. /// - /// \param xchg_type exchange type. - /// \throw isc::BadValue if invalid exchange type specified. - /// \return square sum of delays between sent and received packets. - double getSquareSumDelay(const ExchangeType xchg_type) const { + /// \return standard deviation of packet delay. + double getStdDevDelay(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); - return xchg_stats->getSquareSumDelay(); + return xchg_stats->getStdDevDelay(); } /// \brief Return number of orphant packets. @@ -697,7 +786,7 @@ public: /// \param xchg_type exchange type. /// \throw isc::BadValue if invalid exchange type specified. /// \return number of orphant packets so far. - uint64_t getOrphans(const ExchangeType xchg_type) const { + unsigned long long getOrphans(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); return xchg_stats->getOrphans(); } @@ -726,7 +815,7 @@ public: /// \param xchg_type exchange type. /// \throw isc::BadValue if invalid exchange type specified. /// \return number of unordered lookups. - uint64_t getUnorderedLookups(const ExchangeType xchg_type) const { + unsigned long long getUnorderedLookups(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); return xchg_stats->getUnorderedLookups(); } @@ -742,7 +831,7 @@ public: /// \param xchg_type exchange type. /// \throw isc::BadValue if invalid exchange type specified. /// \return number of ordered lookups. - uint64_t getOrderedLookups(const ExchangeType xchg_type) const { + unsigned long long getOrderedLookups(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); return xchg_stats->getOrderedLookups(); } @@ -755,7 +844,7 @@ public: /// \param xchg_type exchange type. /// \throw isc::BadValue if invalid exchange type specified. /// \return number of sent packets. - uint64_t getSentPacketsNum(const ExchangeType xchg_type) const { + unsigned long long getSentPacketsNum(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); return xchg_stats->getSentPacketsNum(); } @@ -768,11 +857,115 @@ public: /// \param xchg_type exchange type. /// \throw isc::BadValue if invalid exchange type specified. /// \return number of received packets. - uint64_t getRcvdPacketsNum(const ExchangeType xchg_type) const { + unsigned long long getRcvdPacketsNum(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); return xchg_stats->getRcvdPacketsNum(); } + /// \brief Return name of the exchange. + /// + /// Method returns name of the specified exchange type. + /// This function is mainly for logging purposes. + /// + /// \param xchg_type exchange type. + /// \return string representing name of the exchange. + std::string exchangeToString(ExchangeType xchg_type) const { + switch(xchg_type) { + case XCHG_DO: + return "DISCOVER-OFFER"; + case XCHG_RA: + return "REQUEST-ACK"; + case XCHG_SA: + return "SOLICIT-ADVERTISE"; + case XCHG_RR: + return "REQUEST-REPLY"; + default: + return "Unknown exchange type"; + } + } + + /// \brief Print main statistics for all exchange types. + /// + /// Method prints main statistics for all exchange types. + /// Statistics includes: number of sent and received packets, + /// number of dropped packets and number of orphans. + void printMainStats() const { + for (ExchangesMapIterator it = exchanges_.begin(); + it != exchanges_.end(); + ++it) { + ExchangeStatsPtr xchg_stats = it->second; + printf("***Statistics for packet exchange %s***\n" + "sent: %llu, received: %llu\n" + "drops: %lld, orphans: %llu\n\n", + exchangeToString(it->first).c_str(), + xchg_stats->getSentPacketsNum(), + xchg_stats->getRcvdPacketsNum(), + xchg_stats->getRcvdPacketsNum() + - xchg_stats->getSentPacketsNum(), + xchg_stats->getOrphans()); + } + } + + /// \brief Print round trip time packets statistics. + /// + /// Method prints round trip time packets statistics. Statistics + /// includes minimum packet delay, maximum packet delay, average + /// packet delay and standard deviation of delays. Packet delay + /// is a duration between sending a packet to server and receiving + /// response from server. + void printRTTStats() const { + for (ExchangesMapIterator it = exchanges_.begin(); + it != exchanges_.end(); + ++it) { + ExchangeStatsPtr xchg_stats = it->second; + printf("***Round trip time Statistics for packet exchange %s***\n" + "min delay: %.3f\n" + "avg delay: %.3f\n" + "max delay: %.3f\n" + "std deviation: %.3f\n", + exchangeToString(it->first).c_str(), + xchg_stats->getMinDelay(), + xchg_stats->getAvgDelay(), + xchg_stats->getMaxDelay(), + xchg_stats->getStdDevDelay()); + } + } + + /// \brief Print timestamps of all packets. + /// + /// Method prints timestamps of all sent and received + /// packets for all defined exchange types. + /// + /// \throw isc::InvalidOperation if one of the packets has + /// no timestamp value set. + void printTimestamps() const { + for (ExchangesMapIterator it = exchanges_.begin(); + it != exchanges_.end(); + ++it) { + ExchangeStatsPtr xchg_stats = it->second; + printf("***Timestamps for packets in exchange %s***\n", + exchangeToString(it->first).c_str()); + xchg_stats->printTimestamps(); + } + } + + /// \brief Print names and values of custom counters. + /// + /// Method prints names and values of custom counters. Custom counters + /// are defined by client class for tracking different statistics. + void printCustomCounters() const { + if (custom_counters_.size() > 0) { + printf("***Various statistics counters***\n"); + } + for (CustomCountersMapIterator it = custom_counters_.begin(); + it != custom_counters_.end(); + ++it) { + CustomCounterPtr counter = it->second; + printf("%s: %llu\n", counter->getName().c_str(), + counter->getValue()); + } + } + private: /// \brief Return exchange stats object for given exchange type /// diff --git a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc index 98e277dee0..dad544e594 100644 --- a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc +++ b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc @@ -41,6 +41,13 @@ public: StatsMgrTest() { } + /// \brief Create DHCPv4 packet. + /// + /// Method creates DHCPv4 packet and updates its timestamp. + /// + /// \param msg_type DHCPv4 message type. + /// \param transid transaction id for the packet. + /// \return DHCPv4 packet. Pkt4* createPacket4(const uint8_t msg_type, const uint32_t transid) { Pkt4* pkt = new Pkt4(msg_type, transid); @@ -52,6 +59,13 @@ public: return pkt; } + /// \brief Create DHCPv6 packet. + /// + /// Method creates DHCPv6 packet and updates its timestamp. + /// + /// \param msg_type DHCPv6 message type. + /// \param transid transaction id. + /// \return DHCPv6 packet. Pkt6* createPacket6(const uint8_t msg_type, const uint32_t transid) { Pkt6* pkt = new Pkt6(msg_type, transid); @@ -63,6 +77,16 @@ public: return pkt; } + /// \brief Pass multiple DHCPv6 packets to Statistics Manager. + /// + /// Method simulates sending or receiving multiple DHCPv6 packets. + /// + /// \param stats_mgr Statistics Manager instance to be used. + /// \param xchg_type packet exchange types. + /// \param packet_type DHCPv6 packet type. + /// \param num_packets packets to be passed to Statistics Manager. + /// \param receive simulated packets are received (if true) + /// or sent (if false) void passMultiplePackets6(const boost::shared_ptr stats_mgr, const StatsMgr6::ExchangeType xchg_type, const uint8_t packet_type, @@ -84,6 +108,49 @@ public: } } + /// \brief Simulate DHCPv4 DISCOVER-OFFER with delay. + /// + /// Method simulates DHCPv4 DISCOVER-OFFER exchange. The OFFER packet + /// creation is delayed by the specified number of seconds. This imposes + /// different packet timestamps and affects delay counters in Statistics + /// Manager. + /// + /// \param stats_mgr Statistics Manager instance. + /// \param delay delay in seconds between DISCOVER and OFFER packets. + void passDOPacketsWithDelay(const boost::shared_ptr stats_mgr, + unsigned int delay, + uint32_t transid) { + boost::shared_ptr sent_packet(createPacket4(DHCPDISCOVER, + transid)); + ASSERT_NO_THROW( + stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet) + ); + + // There is way to differentiate timstamps of two packets other than + // sleep for before we create another packet. Packet is using current + // time to update its timestamp. + // Sleeping for X seconds will guarantee that delay between packets + // will be greater than 1 second. Note that posix time value is + // transformed to double value and it makes it hard to determine + // actual value to expect. + std::cout << "Sleeping for " << delay << "s to test packet delays" + << std::endl; + sleep(delay); + + boost::shared_ptr rcvd_packet(createPacket4(DHCPOFFER, + transid)); + ASSERT_NO_THROW( + stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet); + ); + + // Calculate period between packets. + boost::posix_time::ptime sent_time = sent_packet->getTimestamp(); + boost::posix_time::ptime rcvd_time = rcvd_packet->getTimestamp(); + + ASSERT_FALSE(sent_time.is_not_a_date_time()); + ASSERT_FALSE(rcvd_time.is_not_a_date_time()); + } + }; TEST_F(StatsMgrTest, Constructor) { @@ -94,19 +161,17 @@ TEST_F(StatsMgrTest, Constructor) { stats_mgr->getMinDelay(StatsMgr4::XCHG_DO) ); EXPECT_EQ(0, stats_mgr->getMaxDelay(StatsMgr4::XCHG_DO)); - EXPECT_EQ(0, stats_mgr->getSumDelay(StatsMgr4::XCHG_DO)); EXPECT_EQ(0, stats_mgr->getOrphans(StatsMgr4::XCHG_DO)); - EXPECT_EQ(0, stats_mgr->getSquareSumDelay(StatsMgr4::XCHG_DO)); EXPECT_EQ(0, stats_mgr->getOrderedLookups(StatsMgr4::XCHG_DO)); EXPECT_EQ(0, stats_mgr->getUnorderedLookups(StatsMgr4::XCHG_DO)); EXPECT_EQ(0, stats_mgr->getSentPacketsNum(StatsMgr4::XCHG_DO)); EXPECT_EQ(0, stats_mgr->getRcvdPacketsNum(StatsMgr4::XCHG_DO)); - double avg_size = 0.; - ASSERT_NO_THROW( - avg_size = stats_mgr->getAvgUnorderedLookupSetSize(StatsMgr4::XCHG_DO); - ); - EXPECT_EQ(0., avg_size); + EXPECT_THROW(stats_mgr->getAvgDelay(StatsMgr4::XCHG_DO), InvalidOperation); + EXPECT_THROW(stats_mgr->getStdDevDelay(StatsMgr4::XCHG_DO), + InvalidOperation); + EXPECT_THROW(stats_mgr->getAvgUnorderedLookupSetSize(StatsMgr4::XCHG_DO), + InvalidOperation); } TEST_F(StatsMgrTest, Exchange) { @@ -275,39 +340,14 @@ TEST_F(StatsMgrTest, Orphans) { } TEST_F(StatsMgrTest, Delays) { - boost::scoped_ptr stats_mgr(new StatsMgr4()); + + boost::shared_ptr stats_mgr(new StatsMgr4()); stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); - boost::shared_ptr sent_packet(createPacket4(DHCPDISCOVER, - common_transid)); - ASSERT_NO_THROW( - stats_mgr->passSentPacket(StatsMgr4::XCHG_DO, sent_packet) - ); - - // There is way to differentiate timstamps of two packets other than - // sleep for before we create another packet. Packet is using current - // time to update its timestamp. - // Sleeping for two seconds will guarantee that delay between packets - // will be greater than 1 second. Note that posix time value is - // transformed to double value and it makes it hard to determine - // actual value to expect. - std::cout << "Sleeping for 2 seconds to test packet delays" << std::endl; - sleep(2); - - boost::shared_ptr rcvd_packet(createPacket4(DHCPOFFER, - common_transid)); - ASSERT_NO_THROW( - stats_mgr->passRcvdPacket(StatsMgr4::XCHG_DO, rcvd_packet); - ); - - // Calculate period between packets. - boost::posix_time::ptime sent_time = sent_packet->getTimestamp(); - boost::posix_time::ptime rcvd_time = rcvd_packet->getTimestamp(); - - ASSERT_FALSE(sent_time.is_not_a_date_time()); - ASSERT_FALSE(rcvd_time.is_not_a_date_time()); - - boost::posix_time::time_period period(sent_time, rcvd_time); + // Send DISCOVER, wait 2s and receive OFFER. This will affect + // counters in Stats Manager. + const unsigned int delay1 = 2; + passDOPacketsWithDelay(stats_mgr, 2, common_transid); // Initially min delay is equal to MAX_DOUBLE. After first packets // are passed, it is expected to set to actual value. @@ -320,8 +360,15 @@ TEST_F(StatsMgrTest, Delays) { EXPECT_GT(stats_mgr->getMaxDelay(StatsMgr4::XCHG_DO), 1); // Delay sums are now the same as minimum or maximum delay. - EXPECT_GT(stats_mgr->getSumDelay(StatsMgr4::XCHG_DO), 1); - EXPECT_GT(stats_mgr->getSquareSumDelay(StatsMgr4::XCHG_DO), 1); + EXPECT_GT(stats_mgr->getAvgDelay(StatsMgr4::XCHG_DO), 1); + + // Simulate another DISCOVER-OFFER exchange with delay between + // sent and received packets. Delay is now shorter than earlier + // so standard deviation of delay will now increase. + const unsigned int delay2 = 1; + passDOPacketsWithDelay(stats_mgr, delay2, common_transid + 1); + // Standard deviation is expected to be non-zero. + EXPECT_GT(stats_mgr->getStdDevDelay(StatsMgr4::XCHG_DO), 0); } TEST_F(StatsMgrTest, CustomCounters) { @@ -360,6 +407,8 @@ TEST_F(StatsMgrTest, CustomCounters) { stats_mgr->getCounter(too_late_key); EXPECT_EQ(too_late_name, toolate_counter->getName()); EXPECT_EQ(toolate_num, toolate_counter->getValue()); -} + +} + } From 16e49803425ea35550a9787987215e5344ffe528 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Mon, 30 Jul 2012 11:11:23 +0200 Subject: [PATCH 20/69] [1958] Make packet types const and passing them by reference. --- tests/tools/perfdhcp/stats_mgr.h | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index 6a3614990c..934ba59f2c 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -142,7 +142,7 @@ public: /// \param packet packet which transaction id is to be hashed. /// \throw isc::BadValue if packet is null. /// \return transaction id hash. - static uint32_t hashTransid(const boost::shared_ptr packet) { + static uint32_t hashTransid(const boost::shared_ptr& packet) { if (!packet) { isc_throw(BadValue, "Packet is null"); } @@ -155,12 +155,12 @@ public: /// search of packets based on their sequence (order in which they /// were inserted) as well as based on packet transaction id. typedef boost::multi_index_container< - boost::shared_ptr, + boost::shared_ptr, boost::multi_index::indexed_by< boost::multi_index::sequenced<>, boost::multi_index::hashed_non_unique< boost::multi_index::global_fun< - boost::shared_ptr, + const boost::shared_ptr&, uint32_t, &ExchangeStats::hashTransid > @@ -214,7 +214,7 @@ public: /// /// \param packet packet object to be added. /// \throw isc::BadValue if packet is null. - void appendSent(const boost::shared_ptr packet) { + void appendSent(const boost::shared_ptr& packet) { if (!packet) { isc_throw(BadValue, "Packet is null"); } @@ -228,7 +228,7 @@ public: /// /// \param packet packet object to be added. /// \throw isc::BadValue if packet is null. - void appendRcvd(const boost::shared_ptr packet) { + void appendRcvd(const boost::shared_ptr& packet) { if (!packet) { isc_throw(BadValue, "Packet is null"); } @@ -245,8 +245,8 @@ public: /// \param rcvd_packet received packet /// \throw isc::BadValue if sent or received packet is null. /// \throw isc::Unexpected if failed to calculate timestamps - void updateDelays(const boost::shared_ptr sent_packet, - const boost::shared_ptr rcvd_packet) { + void updateDelays(const boost::shared_ptr& sent_packet, + const boost::shared_ptr& rcvd_packet) { if (!sent_packet) { isc_throw(BadValue, "Sent packet is null"); } @@ -304,7 +304,7 @@ public: /// \throw isc::BadValue if received packet is null. /// \return packet having specified transaction or NULL if packet /// not found - boost::shared_ptr findSent(const boost::shared_ptr rcvd_packet) { + boost::shared_ptr findSent(const boost::shared_ptr& rcvd_packet) { if (!rcvd_packet) { isc_throw(BadValue, "Received packet is null"); } @@ -315,7 +315,7 @@ public: // that the received packet we got has no corresponding // sent packet so orphans counter has to be updated. ++orphans_; - return boost::shared_ptr(); + return boost::shared_ptr(); } else if (sent_packets_cache_ == sent_packets_.end()) { // Even if there are still many unmatched packets on the // list we might hit the end of it because of unordered @@ -374,10 +374,10 @@ public: // If we are here, it means that both ordered lookup and // unordered lookup failed. Searched packet is not on the list. ++orphans_; - return boost::shared_ptr(); + return boost::shared_ptr(); } - boost::shared_ptr sent_packet(*sent_packets_cache_); + boost::shared_ptr sent_packet(*sent_packets_cache_); // If packet was found, we assume it will be never searched // again. We want to delete this packet from the list to // improve performance of future searches. @@ -509,7 +509,7 @@ public: for (PktListIterator it = rcvd_packets_.begin(); it != rcvd_packets_.end(); ++it) { - boost::shared_ptr rcvd_packet = *it; + boost::shared_ptr rcvd_packet = *it; // Search for corresponding sent packet using transaction id // of received packet. PktListTransidIndex& idx = archived_packets_.template get<2>(); @@ -518,7 +518,7 @@ public: // This should not happen that there is no corresponding // sent packet. If it does however, we just drop the packet. if (it_archived != idx.end()) { - boost::shared_ptr sent_packet = *it_archived; + boost::shared_ptr sent_packet = *it_archived; // Get sent and received packet times. boost::posix_time::ptime sent_time = sent_packet->getTimestamp(); @@ -700,7 +700,7 @@ public: /// \throw isc::BadValue if invalid exchange type specified or /// packet is null. void passSentPacket(const ExchangeType xchg_type, - const boost::shared_ptr packet) { + const boost::shared_ptr& packet) { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); xchg_stats->appendSent(packet); } @@ -719,9 +719,9 @@ public: /// \throw isc::Unexpected if corresponding packet was not /// found on the list of sent packets. void passRcvdPacket(const ExchangeType xchg_type, - const boost::shared_ptr packet) { + const boost::shared_ptr& packet) { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); - boost::shared_ptr sent_packet + boost::shared_ptr sent_packet = xchg_stats->findSent(packet); if (sent_packet) { From 793457876fc93510db9814cd70c4ab2cc8a918b6 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Mon, 30 Jul 2012 12:02:28 +0200 Subject: [PATCH 21/69] [1958] Replaced printf with cout. --- tests/tools/perfdhcp/stats_mgr.h | 76 ++++++++++++++++---------------- 1 file changed, 39 insertions(+), 37 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index 934ba59f2c..c606ca12ac 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -505,6 +505,9 @@ public: /// /// \throw isc::InvalidOperation if found packet with no timestamp set. void printTimestamps() { + // We will be using boost::posix_time extensivelly here + using namespace boost::posix_time; + // Iterate through all received packets. for (PktListIterator it = rcvd_packets_.begin(); it != rcvd_packets_.end(); @@ -520,10 +523,8 @@ public: if (it_archived != idx.end()) { boost::shared_ptr sent_packet = *it_archived; // Get sent and received packet times. - boost::posix_time::ptime sent_time = - sent_packet->getTimestamp(); - boost::posix_time::ptime rcvd_time = - rcvd_packet->getTimestamp(); + ptime sent_time = sent_packet->getTimestamp(); + ptime rcvd_time = rcvd_packet->getTimestamp(); // All sent and received packets should have timestamps // set but if there is a bug somewhere and packet does // not have timestamp we want to catch this here. @@ -532,16 +533,14 @@ public: isc_throw(InvalidOperation, "packet time is not set"); } // Calculate durations of packets from beginning of epoch. - boost::posix_time::ptime - epoch_time(boost::posix_time::min_date_time); - boost::posix_time::time_period - sent_period(epoch_time, sent_time); - boost::posix_time::time_period - rcvd_period(epoch_time, rcvd_time); + ptime epoch_time(min_date_time); + time_period sent_period(epoch_time, sent_time); + time_period rcvd_period(epoch_time, rcvd_time); // Print timestamps for sent and received packet. - printf("sent/received times: %s / %s\n", - boost::posix_time::to_iso_string(sent_period.length()).c_str(), - boost::posix_time::to_iso_string(rcvd_period.length()).c_str()); + std::cout << "sent / received: " + << to_iso_string(sent_period.length()) + << " / " << to_iso_string(rcvd_period.length()) + << std::endl; } } } @@ -894,15 +893,16 @@ public: it != exchanges_.end(); ++it) { ExchangeStatsPtr xchg_stats = it->second; - printf("***Statistics for packet exchange %s***\n" - "sent: %llu, received: %llu\n" - "drops: %lld, orphans: %llu\n\n", - exchangeToString(it->first).c_str(), - xchg_stats->getSentPacketsNum(), - xchg_stats->getRcvdPacketsNum(), - xchg_stats->getRcvdPacketsNum() - - xchg_stats->getSentPacketsNum(), - xchg_stats->getOrphans()); + unsigned long long drops = xchg_stats->getRcvdPacketsNum() - + xchg_stats->getSentPacketsNum(); + std::cout << "***Statistics for packet exchange " + << exchangeToString(it->first) << "***" << std::endl + << "sent: " << xchg_stats->getSentPacketsNum() << ", " + << "received: " << xchg_stats->getRcvdPacketsNum() + << std::endl + << "drops: " << drops << ", orphans: " + << xchg_stats->getOrphans() + << std::endl << std::endl; } } @@ -914,20 +914,22 @@ public: /// is a duration between sending a packet to server and receiving /// response from server. void printRTTStats() const { + using namespace std; for (ExchangesMapIterator it = exchanges_.begin(); it != exchanges_.end(); ++it) { ExchangeStatsPtr xchg_stats = it->second; - printf("***Round trip time Statistics for packet exchange %s***\n" - "min delay: %.3f\n" - "avg delay: %.3f\n" - "max delay: %.3f\n" - "std deviation: %.3f\n", - exchangeToString(it->first).c_str(), - xchg_stats->getMinDelay(), - xchg_stats->getAvgDelay(), - xchg_stats->getMaxDelay(), - xchg_stats->getStdDevDelay()); + cout << "***Round trip time Statistics for packet exchange " + << exchangeToString(it->first) << "***" << endl + << fixed << setprecision(3) + << "min delay: " << xchg_stats->getMinDelay() * 1e3 + << " ms" << endl + << "avg delay: " << xchg_stats->getAvgDelay() * 1e3 + << " ms" << endl + << "max delay: " << xchg_stats->getMaxDelay() * 1e3 + << " ms" << endl + << "std deviation: " << xchg_stats->getStdDevDelay() * 1e3 + << " ms" << endl << endl; } } @@ -943,9 +945,10 @@ public: it != exchanges_.end(); ++it) { ExchangeStatsPtr xchg_stats = it->second; - printf("***Timestamps for packets in exchange %s***\n", - exchangeToString(it->first).c_str()); + std::cout << "***Timestamps for packets in exchange " + << exchangeToString(it->first) << "***" << std::endl; xchg_stats->printTimestamps(); + std::cout << std::endl; } } @@ -955,14 +958,13 @@ public: /// are defined by client class for tracking different statistics. void printCustomCounters() const { if (custom_counters_.size() > 0) { - printf("***Various statistics counters***\n"); + std::cout << "***Various statistics counters***" << std::endl; } for (CustomCountersMapIterator it = custom_counters_.begin(); it != custom_counters_.end(); ++it) { CustomCounterPtr counter = it->second; - printf("%s: %llu\n", counter->getName().c_str(), - counter->getValue()); + std::cout << counter->getName() << ": " << counter->getValue() << std::endl; } } From e1491225037840283b198a6042d90939d52906fe Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Mon, 30 Jul 2012 12:04:39 +0200 Subject: [PATCH 22/69] [1958] Replaced unsigned long long with uin64_t --- tests/tools/perfdhcp/stats_mgr.h | 38 ++++++++++++++++---------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index c606ca12ac..26bb7042ef 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -88,7 +88,7 @@ public: /// Method returns counter value. /// /// \return counter value. - unsigned long long getValue() const { + uint64_t getValue() const { return counter_; } @@ -108,7 +108,7 @@ public: /// counter's name. CustomCounter() { }; - unsigned long long counter_; ///< Counter's value. + uint64_t counter_; ///< Counter's value. std::string name_; ///< Counter's name. }; @@ -440,7 +440,7 @@ public: /// for us. /// /// \return number of orphant received packets. - unsigned long long getOrphans() const { return orphans_; } + uint64_t getOrphans() const { return orphans_; } /// \brief Return average unordered lookup set size. /// @@ -467,7 +467,7 @@ public: /// packet does not match transaction id of next sent packet. /// /// \return number of unordered lookups. - unsigned long long getUnorderedLookups() const { return unordered_lookups_; } + uint64_t getUnorderedLookups() const { return unordered_lookups_; } /// \brief Return number of ordered sent packets lookups /// @@ -478,14 +478,14 @@ public: /// function will use unordered lookup (with hash table). /// /// \return number of ordered lookups. - unsigned long long getOrderedLookups() const { return ordered_lookups_; } + uint64_t getOrderedLookups() const { return ordered_lookups_; } /// \brief Return total number of sent packets /// /// Method returns total number of sent packets. /// /// \return number of sent packets. - unsigned long long getSentPacketsNum() const { + uint64_t getSentPacketsNum() const { return sent_packets_num_; } @@ -494,7 +494,7 @@ public: /// Method returns total number of received packets. /// /// \return number of received packets. - unsigned long long getRcvdPacketsNum() const { + uint64_t getRcvdPacketsNum() const { return rcvd_packets_num_; } @@ -595,22 +595,22 @@ public: double square_sum_delay_; ///< Square sum of delays between ///< sent and recived packets. - unsigned long long orphans_; ///< Number of orphant received packets. + uint64_t orphans_; ///< Number of orphant received packets. /// Sum of unordered lookup sets. Needed to calculate mean size of /// lookup set. It is desired that number of unordered lookups is /// minimal for performance reasons. Tracking number of lookups and /// mean size of the lookup set should give idea of packets serach /// complexity. - unsigned long long unordered_lookup_size_sum_; + uint64_t unordered_lookup_size_sum_; - unsigned long long unordered_lookups_; ///< Number of unordered sent packets + uint64_t unordered_lookups_; ///< Number of unordered sent packets ///< lookups. - unsigned long long ordered_lookups_; ///< Number of ordered sent packets + uint64_t ordered_lookups_; ///< Number of ordered sent packets ///< lookups. - unsigned long long sent_packets_num_; ///< Total number of sent packets. - unsigned long long rcvd_packets_num_; ///< Total number of received packets. + uint64_t sent_packets_num_; ///< Total number of sent packets. + uint64_t rcvd_packets_num_; ///< Total number of received packets. }; /// Pointer to ExchangeStats. @@ -785,7 +785,7 @@ public: /// \param xchg_type exchange type. /// \throw isc::BadValue if invalid exchange type specified. /// \return number of orphant packets so far. - unsigned long long getOrphans(const ExchangeType xchg_type) const { + uint64_t getOrphans(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); return xchg_stats->getOrphans(); } @@ -814,7 +814,7 @@ public: /// \param xchg_type exchange type. /// \throw isc::BadValue if invalid exchange type specified. /// \return number of unordered lookups. - unsigned long long getUnorderedLookups(const ExchangeType xchg_type) const { + uint64_t getUnorderedLookups(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); return xchg_stats->getUnorderedLookups(); } @@ -830,7 +830,7 @@ public: /// \param xchg_type exchange type. /// \throw isc::BadValue if invalid exchange type specified. /// \return number of ordered lookups. - unsigned long long getOrderedLookups(const ExchangeType xchg_type) const { + uint64_t getOrderedLookups(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); return xchg_stats->getOrderedLookups(); } @@ -843,7 +843,7 @@ public: /// \param xchg_type exchange type. /// \throw isc::BadValue if invalid exchange type specified. /// \return number of sent packets. - unsigned long long getSentPacketsNum(const ExchangeType xchg_type) const { + uint64_t getSentPacketsNum(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); return xchg_stats->getSentPacketsNum(); } @@ -856,7 +856,7 @@ public: /// \param xchg_type exchange type. /// \throw isc::BadValue if invalid exchange type specified. /// \return number of received packets. - unsigned long long getRcvdPacketsNum(const ExchangeType xchg_type) const { + uint64_t getRcvdPacketsNum(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); return xchg_stats->getRcvdPacketsNum(); } @@ -893,7 +893,7 @@ public: it != exchanges_.end(); ++it) { ExchangeStatsPtr xchg_stats = it->second; - unsigned long long drops = xchg_stats->getRcvdPacketsNum() - + uint64_t drops = xchg_stats->getRcvdPacketsNum() - xchg_stats->getSentPacketsNum(); std::cout << "***Statistics for packet exchange " << exchangeToString(it->first) << "***" << std::endl From f40e523668cd8332238b7369470403e8b66b6f35 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Mon, 30 Jul 2012 12:21:24 +0200 Subject: [PATCH 23/69] [1958] Code cleanup: naming, getters layout and return statements. --- tests/tools/perfdhcp/stats_mgr.h | 92 +++++++++++++++----------------- 1 file changed, 42 insertions(+), 50 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index 26bb7042ef..4c3d1e08f4 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -73,14 +73,14 @@ public: /// \brief Increment operator. const CustomCounter& operator++() { ++counter_; - return *this; + return(*this); } /// \brief Increment operator. const CustomCounter& operator++(int) { CustomCounter& this_counter(*this); operator++(); - return this_counter; + return(this_counter); } /// \brief Return counter value. @@ -88,18 +88,14 @@ public: /// Method returns counter value. /// /// \return counter value. - uint64_t getValue() const { - return counter_; - } + uint64_t getValue() const { return counter_; } /// \brief Return counter name. /// /// Method returns counter name. /// /// \return counter name. - const std::string& getName() const { - return name_; - } + const std::string& getName() const { return name_; } private: /// \brief Default constructor. /// @@ -146,7 +142,7 @@ public: if (!packet) { isc_throw(BadValue, "Packet is null"); } - return packet->getTransid() & 1023; + return(packet->getTransid() & 1023); } /// \brief List of packets (sent or received). @@ -199,7 +195,7 @@ public: max_delay_(0.), sum_delay_(0.), orphans_(0), - square_sum_delay_(0.), + sum_delay_squared_(0.), ordered_lookups_(0), unordered_lookup_size_sum_(0), unordered_lookups_(0), @@ -286,7 +282,7 @@ public: // Update delay sum and square sum. That will be used to calculate // mean delays. sum_delay_ += delta; - square_sum_delay_ += delta * delta; + sum_delay_squared_ += delta * delta; } /// \brief Find packet on the list of sent packets. @@ -315,7 +311,7 @@ public: // that the received packet we got has no corresponding // sent packet so orphans counter has to be updated. ++orphans_; - return boost::shared_ptr(); + return(boost::shared_ptr()); } else if (sent_packets_cache_ == sent_packets_.end()) { // Even if there are still many unmatched packets on the // list we might hit the end of it because of unordered @@ -374,7 +370,7 @@ public: // If we are here, it means that both ordered lookup and // unordered lookup failed. Searched packet is not on the list. ++orphans_; - return boost::shared_ptr(); + return(boost::shared_ptr()); } boost::shared_ptr sent_packet(*sent_packets_cache_); @@ -382,7 +378,7 @@ public: // again. We want to delete this packet from the list to // improve performance of future searches. sent_packets_cache_ = eraseSent(sent_packets_cache_); - return sent_packet; + return(sent_packet); } /// \brief Return minumum delay between sent and received packet. @@ -390,14 +386,14 @@ public: /// Method returns minimum delay between sent and received packet. /// /// \return minimum delay between packets. - double getMinDelay() const { return min_delay_; } + double getMinDelay() const { return(min_delay_); } /// \brief Return maxmimum delay between sent and received packet. /// /// Method returns maximum delay between sent and received packet. /// /// \return maximum delay between packets. - double getMaxDelay() const { return max_delay_; } + double getMaxDelay() const { return(max_delay_); } /// \brief Return avarage packet delay. /// @@ -412,7 +408,7 @@ public: if (sum_delay_ == 0) { isc_throw(InvalidOperation, "no packets received"); } - return sum_delay_ / rcvd_packets_num_; + return(sum_delay_ / rcvd_packets_num_); } /// \brief Return standard deviation of packet delay. @@ -429,8 +425,8 @@ public: if (rcvd_packets_num_ == 0) { isc_throw(InvalidOperation, "no packets received"); } - return sqrt(square_sum_delay_ / rcvd_packets_num_ - - getAvgDelay() * getAvgDelay()); + return(sqrt(sum_delay_squared_ / rcvd_packets_num_ - + getAvgDelay() * getAvgDelay())); } /// \brief Return number of orphant packets. @@ -440,7 +436,7 @@ public: /// for us. /// /// \return number of orphant received packets. - uint64_t getOrphans() const { return orphans_; } + uint64_t getOrphans() const { return(orphans_); } /// \brief Return average unordered lookup set size. /// @@ -455,8 +451,8 @@ public: if (unordered_lookups_ == 0) { isc_throw(InvalidOperation, "no unordered lookups"); } - return static_cast(unordered_lookup_size_sum_) / - static_cast(unordered_lookups_); + return(static_cast(unordered_lookup_size_sum_) / + static_cast(unordered_lookups_)); } /// \brief Return number of unordered sent packets lookups @@ -467,7 +463,7 @@ public: /// packet does not match transaction id of next sent packet. /// /// \return number of unordered lookups. - uint64_t getUnorderedLookups() const { return unordered_lookups_; } + uint64_t getUnorderedLookups() const { return(unordered_lookups_); } /// \brief Return number of ordered sent packets lookups /// @@ -478,25 +474,21 @@ public: /// function will use unordered lookup (with hash table). /// /// \return number of ordered lookups. - uint64_t getOrderedLookups() const { return ordered_lookups_; } + uint64_t getOrderedLookups() const { return(ordered_lookups_); } /// \brief Return total number of sent packets /// /// Method returns total number of sent packets. /// /// \return number of sent packets. - uint64_t getSentPacketsNum() const { - return sent_packets_num_; - } + uint64_t getSentPacketsNum() const { return(sent_packets_num_); } /// \brief Return total number of received packets /// /// Method returns total number of received packets. /// /// \return number of received packets. - uint64_t getRcvdPacketsNum() const { - return rcvd_packets_num_; - } + uint64_t getRcvdPacketsNum() const { return(rcvd_packets_num_); } //// \brief Print timestamps for sent and received packets. /// @@ -568,7 +560,7 @@ public: // archived packets may be used for diagnostics // when test is completed. archived_packets_.push_back(*it); - return sent_packets_.template get<0>().erase(it); + return(sent_packets_.template get<0>().erase(it)); } ExchangeType xchg_type_; ///< Packet exchange type. @@ -592,7 +584,7 @@ public: ///< and received packets. double sum_delay_; ///< Sum of delays between sent ///< and received packets. - double square_sum_delay_; ///< Square sum of delays between + double sum_delay_squared_; ///< Squared sum of delays between ///< sent and recived packets. uint64_t orphans_; ///< Number of orphant received packets. @@ -674,7 +666,7 @@ public: isc_throw(BadValue, "Custom counter " << counter_key << "does not exist"); } - return it->second; + return(it->second); } /// \brief Increment specified counter. @@ -685,7 +677,7 @@ public: /// \return pointer to specified counter after incrementation. const CustomCounter& IncrementCounter(const std::string& counter_key) { CustomCounterPtr counter = getCounter(counter_key); - return ++(*counter); + return(++(*counter)); } /// \brief Adds new packet to the sent packets list. @@ -739,7 +731,7 @@ public: /// \return minimum delay between packets. double getMinDelay(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); - return xchg_stats->getMinDelay(); + return(xchg_stats->getMinDelay()); } /// \brief Return maxmimum delay between sent and received packet. @@ -752,7 +744,7 @@ public: /// \return maximum delay between packets. double getMaxDelay(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); - return xchg_stats->getMaxDelay(); + return(xchg_stats->getMaxDelay()); } /// \brief Return avarage packet delay. @@ -763,7 +755,7 @@ public: /// \return average packet delay. double getAvgDelay(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); - return xchg_stats->getAvgDelay(); + return(xchg_stats->getAvgDelay()); } /// \brief Return standard deviation of packet delay. @@ -774,7 +766,7 @@ public: /// \return standard deviation of packet delay. double getStdDevDelay(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); - return xchg_stats->getStdDevDelay(); + return(xchg_stats->getStdDevDelay()); } /// \brief Return number of orphant packets. @@ -787,7 +779,7 @@ public: /// \return number of orphant packets so far. uint64_t getOrphans(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); - return xchg_stats->getOrphans(); + return(xchg_stats->getOrphans()); } /// \brief Return average unordered lookup set size. @@ -801,7 +793,7 @@ public: /// \return average unordered lookup set size. double getAvgUnorderedLookupSetSize(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); - return xchg_stats->getAvgUnorderedLookupSetSize(); + return(xchg_stats->getAvgUnorderedLookupSetSize()); } /// \brief Return number of unordered sent packets lookups @@ -816,7 +808,7 @@ public: /// \return number of unordered lookups. uint64_t getUnorderedLookups(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); - return xchg_stats->getUnorderedLookups(); + return(xchg_stats->getUnorderedLookups()); } /// \brief Return number of ordered sent packets lookups @@ -832,7 +824,7 @@ public: /// \return number of ordered lookups. uint64_t getOrderedLookups(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); - return xchg_stats->getOrderedLookups(); + return(xchg_stats->getOrderedLookups()); } /// \brief Return total number of sent packets @@ -845,7 +837,7 @@ public: /// \return number of sent packets. uint64_t getSentPacketsNum(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); - return xchg_stats->getSentPacketsNum(); + return(xchg_stats->getSentPacketsNum()); } /// \brief Return total number of received packets @@ -858,7 +850,7 @@ public: /// \return number of received packets. uint64_t getRcvdPacketsNum(const ExchangeType xchg_type) const { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); - return xchg_stats->getRcvdPacketsNum(); + return(xchg_stats->getRcvdPacketsNum()); } /// \brief Return name of the exchange. @@ -871,15 +863,15 @@ public: std::string exchangeToString(ExchangeType xchg_type) const { switch(xchg_type) { case XCHG_DO: - return "DISCOVER-OFFER"; + return("DISCOVER-OFFER"); case XCHG_RA: - return "REQUEST-ACK"; + return("REQUEST-ACK"); case XCHG_SA: - return "SOLICIT-ADVERTISE"; + return("SOLICIT-ADVERTISE"); case XCHG_RR: - return "REQUEST-REPLY"; + return("REQUEST-REPLY"); default: - return "Unknown exchange type"; + return("Unknown exchange type"); } } @@ -982,7 +974,7 @@ private: isc_throw(BadValue, "Packets exchange not specified"); } ExchangeStatsPtr xchg_stats = it->second; - return xchg_stats; + return(xchg_stats); } ExchangesMap exchanges_; ///< Map of exchange types. From 0048a810b5641bb7b35f7ccf790846cee6f89697 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Mon, 30 Jul 2012 13:26:53 +0200 Subject: [PATCH 24/69] [1958] Improved presentation of statistics counters. --- tests/tools/perfdhcp/stats_mgr.h | 94 +++++++++++++++++--------------- 1 file changed, 50 insertions(+), 44 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index 4c3d1e08f4..4d819546a5 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -490,6 +490,37 @@ public: /// \return number of received packets. uint64_t getRcvdPacketsNum() const { return(rcvd_packets_num_); } + /// \brief Print main statistics for packet exchange. + /// + /// Method prints main statistics for particular exchange. + /// Statistics includes: number of sent and received packets, + /// number of dropped packets and number of orphans. + void printMainStats() const { + using namespace std; + uint64_t drops = getRcvdPacketsNum() - getSentPacketsNum(); + cout << "sent packets: " << getSentPacketsNum() << endl + << "received packets: " << getRcvdPacketsNum() << endl + << "drops: " << drops << endl + << "orphans: " << getOrphans() << endl; + } + + /// \brief Print round trip time packets statistics. + /// + /// Method prints round trip time packets statistics. Statistics + /// includes minimum packet delay, maximum packet delay, average + /// packet delay and standard deviation of delays. Packet delay + /// is a duration between sending a packet to server and receiving + /// response from server. + void printRTTStats() const { + using namespace std; + cout << fixed << setprecision(3) + << "min delay: " << getMinDelay() * 1e3 << " ms" << endl + << "avg delay: " << getAvgDelay() * 1e3 << " ms" << endl + << "max delay: " << getMaxDelay() * 1e3 << " ms" << endl + << "std deviation: " << getStdDevDelay() * 1e3 << " ms" + << endl; + } + //// \brief Print timestamps for sent and received packets. /// /// Method prints timestamps for all sent and received packets for @@ -875,53 +906,27 @@ public: } } - /// \brief Print main statistics for all exchange types. + /// \brief Print statistics counters for all exchange types. /// - /// Method prints main statistics for all exchange types. - /// Statistics includes: number of sent and received packets, - /// number of dropped packets and number of orphans. - void printMainStats() const { + /// Method prints statistics for all exchange types. + /// Statistics includes: + /// - number of sent and received packets + /// - number of dropped packets and number of orphans + /// - minimum packets delay, + /// - average packets delay, + /// - maximum packets delay, + /// - standard deviation of packets delay. + void printStats() const { for (ExchangesMapIterator it = exchanges_.begin(); it != exchanges_.end(); ++it) { ExchangeStatsPtr xchg_stats = it->second; - uint64_t drops = xchg_stats->getRcvdPacketsNum() - - xchg_stats->getSentPacketsNum(); - std::cout << "***Statistics for packet exchange " - << exchangeToString(it->first) << "***" << std::endl - << "sent: " << xchg_stats->getSentPacketsNum() << ", " - << "received: " << xchg_stats->getRcvdPacketsNum() - << std::endl - << "drops: " << drops << ", orphans: " - << xchg_stats->getOrphans() - << std::endl << std::endl; - } - } - - /// \brief Print round trip time packets statistics. - /// - /// Method prints round trip time packets statistics. Statistics - /// includes minimum packet delay, maximum packet delay, average - /// packet delay and standard deviation of delays. Packet delay - /// is a duration between sending a packet to server and receiving - /// response from server. - void printRTTStats() const { - using namespace std; - for (ExchangesMapIterator it = exchanges_.begin(); - it != exchanges_.end(); - ++it) { - ExchangeStatsPtr xchg_stats = it->second; - cout << "***Round trip time Statistics for packet exchange " - << exchangeToString(it->first) << "***" << endl - << fixed << setprecision(3) - << "min delay: " << xchg_stats->getMinDelay() * 1e3 - << " ms" << endl - << "avg delay: " << xchg_stats->getAvgDelay() * 1e3 - << " ms" << endl - << "max delay: " << xchg_stats->getMaxDelay() * 1e3 - << " ms" << endl - << "std deviation: " << xchg_stats->getStdDevDelay() * 1e3 - << " ms" << endl << endl; + std::cout << "***Statistics for: " << exchangeToString(it->first) + << "***" << std::endl; + xchg_stats->printMainStats(); + std::cout << std::endl; + xchg_stats->printRTTStats(); + std::cout << std::endl; } } @@ -937,8 +942,9 @@ public: it != exchanges_.end(); ++it) { ExchangeStatsPtr xchg_stats = it->second; - std::cout << "***Timestamps for packets in exchange " - << exchangeToString(it->first) << "***" << std::endl; + std::cout << "***Timestamps for packets: " + << exchangeToString(it->first) + << "***" << std::endl; xchg_stats->printTimestamps(); std::cout << std::endl; } From 3b17dac1ab05a635f39f73316be62f4783a0182f Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Mon, 30 Jul 2012 14:57:05 +0200 Subject: [PATCH 25/69] [1958] Better documentation if multi_index_container, removed redundant index. --- tests/tools/perfdhcp/stats_mgr.h | 139 ++++++++++++++++++++++--------- 1 file changed, 98 insertions(+), 41 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index 4d819546a5..b523b63fdf 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -149,7 +149,70 @@ public: /// /// List of packets based on multi index container allows efficient /// search of packets based on their sequence (order in which they - /// were inserted) as well as based on packet transaction id. + /// were inserted) as well as based on their hashed transaction id. + /// The first index (sequenced) provides the way to use container + /// as a regular list (including iterators, removal of elements from + /// the middle of the collection etc.). This index is meant to be used + /// more frequently than the latter one and it is based on the + /// assumption that responses from the DHCP server are received in + /// order. In this case, when next packet is received it can be + /// matched with next packet on the list of sent packets. This + /// prevents intensive searches on the list of sent packets every + /// time new packet arrives. In many cases however packets can be + /// dropped by the server or may be sent out of order and we still + /// want to have ability to search packets using transaction id. + /// The second index can be used for this purpose. This index is + /// hashing transaction ids using custom function \ref hashTransid. + /// Note that other possibility would be to simply specify index + /// that uses transaction id directly (instead of hashing with + /// \ref hashTransid). In this case however we have chosen to use + /// hashing function because it shortens the index size to just + /// 1023 values maximum. Search operation on this index generally + /// returns the range of packets that have the same transaction id + /// hash assigned but most often these ranges will be short so further + /// search within a range to find a packet with pacrticular transaction + /// id will not be intensive. + /// + /// Example 1: Add elements to the list + /// \code + /// PktList packets_collection(); + /// boost::shared_ptr pkt1(new Pkt4(...)); + /// boost::shared_ptr pkt2(new Pkt4(...)); + /// // Add new packet to the container, it will be available through + /// // both indexes + /// packets_collection.push_back(pkt1); + /// // Here is another way to add packet to the container. The result + /// // is exactly the same as previously. + /// packets_collection.template get<0>().push_back(pkt2); + /// \endcode + /// + /// Example 2: Access elements through sequencial index + /// \code + /// PktList packets_collection(); + /// ... # Add elements to the container + /// for (PktListIterator it = packets_collection.begin(); + /// it != packets_collection.end(); + /// ++it) { + /// boost::shared_ptr pkt = *it; + /// # Do something with packet; + /// } + /// \endcode + /// + /// Example 3: Access elements through hashed index + /// \code + /// // Get the instance of the second search index. + /// PktListTransidHashIndex& idx = sent_packets_.template get<1>(); + /// // Get the range (bucket) of packets sharing the same transaction + /// // id hash. + /// std::pair p = + /// idx.equal_range(hashTransid(rcvd_packet)); + /// // Iterate through the returned bucket. + /// for (PktListTransidHashIterator it = p.first; it != p.second; + /// ++it) { + /// boost::shared_ptr pkt = *it; + /// ... # Do something with the packet (e.g. check transaction id) + /// } + /// \endcode typedef boost::multi_index_container< boost::shared_ptr, boost::multi_index::indexed_by< @@ -160,13 +223,6 @@ public: uint32_t, &ExchangeStats::hashTransid > - >, - boost::multi_index::hashed_non_unique< - boost::multi_index::const_mem_fun< - T, - uint32_t, - &T::getTransid - > > > > PktList; @@ -179,12 +235,6 @@ public: /// Packet list iterator to access packets using transaction id hash. typedef typename PktListTransidHashIndex::const_iterator PktListTransidHashIterator; - /// Packet list index to search packets using transaction id. - typedef typename PktList::template nth_index<2>::type - PktListTransidIndex; - /// Packet list iterator to access packets using transaction id. - typedef typename PktListTransidIndex::const_iterator - PktListTransidIterator; /// \brief Constructor /// @@ -536,34 +586,39 @@ public: it != rcvd_packets_.end(); ++it) { boost::shared_ptr rcvd_packet = *it; - // Search for corresponding sent packet using transaction id - // of received packet. - PktListTransidIndex& idx = archived_packets_.template get<2>(); - PktListTransidIterator it_archived = - idx.find(rcvd_packet->getTransid()); - // This should not happen that there is no corresponding - // sent packet. If it does however, we just drop the packet. - if (it_archived != idx.end()) { - boost::shared_ptr sent_packet = *it_archived; - // Get sent and received packet times. - ptime sent_time = sent_packet->getTimestamp(); - ptime rcvd_time = rcvd_packet->getTimestamp(); - // All sent and received packets should have timestamps - // set but if there is a bug somewhere and packet does - // not have timestamp we want to catch this here. - if (sent_time.is_not_a_date_time() || - rcvd_time.is_not_a_date_time()) { - isc_throw(InvalidOperation, "packet time is not set"); + PktListTransidHashIndex& idx = + archived_packets_.template get<1>(); + std::pair p = + idx.equal_range(hashTransid(rcvd_packet)); + for (PktListTransidHashIterator it_archived = p.first; + it_archived != p.second; + ++it) { + if ((*it_archived)->getTransid() == + rcvd_packet->getTransid()) { + boost::shared_ptr sent_packet = *it_archived; + // Get sent and received packet times. + ptime sent_time = sent_packet->getTimestamp(); + ptime rcvd_time = rcvd_packet->getTimestamp(); + // All sent and received packets should have timestamps + // set but if there is a bug somewhere and packet does + // not have timestamp we want to catch this here. + if (sent_time.is_not_a_date_time() || + rcvd_time.is_not_a_date_time()) { + isc_throw(InvalidOperation, "packet time is not set"); + } + // Calculate durations of packets from beginning of epoch. + ptime epoch_time(min_date_time); + time_period sent_period(epoch_time, sent_time); + time_period rcvd_period(epoch_time, rcvd_time); + // Print timestamps for sent and received packet. + std::cout << "sent / received: " + << to_iso_string(sent_period.length()) + << " / " + << to_iso_string(rcvd_period.length()) + << std::endl; + break; } - // Calculate durations of packets from beginning of epoch. - ptime epoch_time(min_date_time); - time_period sent_period(epoch_time, sent_time); - time_period rcvd_period(epoch_time, rcvd_time); - // Print timestamps for sent and received packet. - std::cout << "sent / received: " - << to_iso_string(sent_period.length()) - << " / " << to_iso_string(rcvd_period.length()) - << std::endl; } } } @@ -591,6 +646,8 @@ public: // archived packets may be used for diagnostics // when test is completed. archived_packets_.push_back(*it); + // get<0>() template returns sequencial index to + // container. return(sent_packets_.template get<0>().erase(it)); } From 731fac5042e88fa7611b0f99feace1e1b05fe4a0 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Mon, 30 Jul 2012 15:10:02 +0200 Subject: [PATCH 26/69] [1958] Renamed member to next_sent_ to reflect the purpose of it. --- tests/tools/perfdhcp/stats_mgr.h | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index b523b63fdf..84d61e0321 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -251,7 +251,7 @@ public: unordered_lookups_(0), sent_packets_num_(0), rcvd_packets_num_(0) { - sent_packets_cache_ = sent_packets_.begin(); + next_sent_ = sent_packets_.begin(); } /// \brief Add new packet to list of sent packets. @@ -362,11 +362,11 @@ public: // sent packet so orphans counter has to be updated. ++orphans_; return(boost::shared_ptr()); - } else if (sent_packets_cache_ == sent_packets_.end()) { + } else if (next_sent_ == sent_packets_.end()) { // Even if there are still many unmatched packets on the // list we might hit the end of it because of unordered - // lookups. The next logical step is to reset cache. - sent_packets_cache_ = sent_packets_.begin(); + // lookups. The next logical step is to reset iterator. + next_sent_ = sent_packets_.begin(); } // With this variable we will be signalling success or failure @@ -378,7 +378,7 @@ public: // incoming packet. We are successful if there is no // packet drop or out of order packets sent. This is actually // the fastest way to look for packets. - if ((*sent_packets_cache_)->getTransid() == rcvd_packet->getTransid()) { + if ((*next_sent_)->getTransid() == rcvd_packet->getTransid()) { ++ordered_lookups_; packet_found = true; } else { @@ -409,7 +409,7 @@ public: ++it) { if ((*it)->getTransid() == rcvd_packet->getTransid()) { packet_found = true; - sent_packets_cache_ = + next_sent_ = sent_packets_.template project<0>(it); break; } @@ -423,11 +423,11 @@ public: return(boost::shared_ptr()); } - boost::shared_ptr sent_packet(*sent_packets_cache_); + boost::shared_ptr sent_packet(*next_sent_); // If packet was found, we assume it will be never searched // again. We want to delete this packet from the list to // improve performance of future searches. - sent_packets_cache_ = eraseSent(sent_packets_cache_); + next_sent_ = eraseSent(next_sent_); return(sent_packet); } @@ -657,7 +657,7 @@ public: /// Iterator pointing to the packet on sent list which will most /// likely match next received packet. This is based on the /// assumption that server responds in order to incoming packets. - PktListIterator sent_packets_cache_; + PktListIterator next_sent_; PktList rcvd_packets_; ///< List of received packets. @@ -747,6 +747,7 @@ public: /// Method returns specified counter. /// /// \param counter_key key poiting to the counter in the counters map. + /// The short counter name has to be used to access counter. /// \return pointer to specified counter object. CustomCounterPtr getCounter(const std::string& counter_key) { CustomCountersMapIterator it = custom_counters_.find(counter_key); From 0ef001aacb7a9da735d418504168420ddac0c3df Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Mon, 30 Jul 2012 15:15:27 +0200 Subject: [PATCH 27/69] [1958] Use EXPECT_DOUBLE_EQ for doubles. --- tests/tools/perfdhcp/tests/stats_mgr_unittest.cc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc index dad544e594..1611105885 100644 --- a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc +++ b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc @@ -156,11 +156,11 @@ public: TEST_F(StatsMgrTest, Constructor) { boost::scoped_ptr stats_mgr(new StatsMgr4()); stats_mgr->addExchangeStats(StatsMgr4::XCHG_DO); - EXPECT_EQ( + EXPECT_DOUBLE_EQ( std::numeric_limits::max(), stats_mgr->getMinDelay(StatsMgr4::XCHG_DO) ); - EXPECT_EQ(0, stats_mgr->getMaxDelay(StatsMgr4::XCHG_DO)); + EXPECT_DOUBLE_EQ(0, stats_mgr->getMaxDelay(StatsMgr4::XCHG_DO)); EXPECT_EQ(0, stats_mgr->getOrphans(StatsMgr4::XCHG_DO)); EXPECT_EQ(0, stats_mgr->getOrderedLookups(StatsMgr4::XCHG_DO)); EXPECT_EQ(0, stats_mgr->getUnorderedLookups(StatsMgr4::XCHG_DO)); From 2175b280a686cc4aec2cf72c21f7ce839bf609ee Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Mon, 30 Jul 2012 21:52:47 +0200 Subject: [PATCH 28/69] [1958] Added optional packets archiving and unit test for printing stats. --- tests/tools/perfdhcp/stats_mgr.h | 149 ++++++++++++++---- .../perfdhcp/tests/stats_mgr_unittest.cc | 36 +++++ 2 files changed, 156 insertions(+), 29 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index 84d61e0321..4ca257614b 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -239,7 +239,7 @@ public: /// \brief Constructor /// /// \param xchg_type exchange type - ExchangeStats(const ExchangeType xchg_type) + ExchangeStats(const ExchangeType xchg_type, const bool archive_enabled) : xchg_type_(xchg_type), min_delay_(std::numeric_limits::max()), max_delay_(0.), @@ -250,7 +250,11 @@ public: unordered_lookup_size_sum_(0), unordered_lookups_(0), sent_packets_num_(0), - rcvd_packets_num_(0) { + rcvd_packets_num_(0), + sent_packets_(), + rcvd_packets_(), + archived_packets_(), + archive_enabled_(archive_enabled) { next_sent_ = sent_packets_.begin(); } @@ -278,7 +282,6 @@ public: if (!packet) { isc_throw(BadValue, "Packet is null"); } - ++rcvd_packets_num_; rcvd_packets_.push_back(packet); } @@ -423,6 +426,7 @@ public: return(boost::shared_ptr()); } + ++rcvd_packets_num_; boost::shared_ptr sent_packet(*next_sent_); // If packet was found, we assume it will be never searched // again. We want to delete this packet from the list to @@ -455,7 +459,7 @@ public: /// have been received yet. /// \return average packet delay. double getAvgDelay() const { - if (sum_delay_ == 0) { + if (rcvd_packets_num_ == 0) { isc_throw(InvalidOperation, "no packets received"); } return(sum_delay_ / rcvd_packets_num_); @@ -563,21 +567,38 @@ public: /// response from server. void printRTTStats() const { using namespace std; - cout << fixed << setprecision(3) - << "min delay: " << getMinDelay() * 1e3 << " ms" << endl - << "avg delay: " << getAvgDelay() * 1e3 << " ms" << endl - << "max delay: " << getMaxDelay() * 1e3 << " ms" << endl - << "std deviation: " << getStdDevDelay() * 1e3 << " ms" - << endl; + try { + cout << fixed << setprecision(3) + << "min delay: " << getMinDelay() * 1e3 << " ms" << endl + << "avg delay: " << getAvgDelay() * 1e3 << " ms" << endl + << "max delay: " << getMaxDelay() * 1e3 << " ms" << endl + << "std deviation: " << getStdDevDelay() * 1e3 << " ms" + << endl; + } catch (const Exception& e) { + cout << "Unavailable! No packets received." << endl; + } } //// \brief Print timestamps for sent and received packets. /// /// Method prints timestamps for all sent and received packets for - /// packet exchange. + /// packet exchange. In order to run this method the packets + /// archiving mode has to be enabled during object constructions. + /// Otherwise sent packets are not stored during tests execution + /// and this method has no ability to get and print their timestamps. /// - /// \throw isc::InvalidOperation if found packet with no timestamp set. + /// \throw isc::InvalidOperation if found packet with no timestamp or + /// if packets archive mode is disabled. void printTimestamps() { + // If archive mode is disabled there is no sense to proceed + // because we don't have packets and their timestamps. + if (!archive_enabled_) { + isc_throw(isc::InvalidOperation, + "packets archive mode is disabled"); + } + if (rcvd_packets_num_ == 0) { + std::cout << "Unavailable! No packets received." << std::endl; + } // We will be using boost::posix_time extensivelly here using namespace boost::posix_time; @@ -605,7 +626,8 @@ public: // not have timestamp we want to catch this here. if (sent_time.is_not_a_date_time() || rcvd_time.is_not_a_date_time()) { - isc_throw(InvalidOperation, "packet time is not set"); + isc_throw(InvalidOperation, + "packet time is not set"); } // Calculate durations of packets from beginning of epoch. ptime epoch_time(min_date_time); @@ -639,13 +661,15 @@ public: /// \return iterator pointing to packet following erased /// packet or sent_packets_.end() if packet not found. PktListIterator eraseSent(const PktListIterator it) { - // We don't want to keep list of all sent packets - // because it will affect packet lookup performance. - // If packet is matched with received packet we - // move it to list of archived packets. List of - // archived packets may be used for diagnostics - // when test is completed. - archived_packets_.push_back(*it); + if (archive_enabled_) { + // We don't want to keep list of all sent packets + // because it will affect packet lookup performance. + // If packet is matched with received packet we + // move it to list of archived packets. List of + // archived packets may be used for diagnostics + // when test is completed. + archived_packets_.push_back(*it); + } // get<0>() template returns sequencial index to // container. return(sent_packets_.template get<0>().erase(it)); @@ -666,6 +690,19 @@ public: /// list for diagnostics purposes. PktList archived_packets_; + /// Indicates all packets have to be preserved after matching. + /// By default this is disabled which means that when received + /// packet is matched with sent packet both are deleted. This + /// is important when test is executed for extended period of + /// time and high memory usage might be the issue. + /// When timestamps listing is specified from the command line + /// (using diagnostics selector), all packets have to be preserved + /// so as the printing method may read their timestamps and + /// print it to user. In such usage model it will be rare to + /// run test for extended period of time so it should be fine + /// to keep all packets archived throughout the test. + bool archive_enabled_; + double min_delay_; ///< Minimum delay between sent ///< and received packets. double max_delay_; ///< Maximum delay between sent @@ -704,10 +741,32 @@ public: /// Iterator for \ref CustomCountersMap. typedef typename CustomCountersMap::const_iterator CustomCountersMapIterator; + /// \brief Default constructor. + /// + /// This constructor by default disables packets archiving mode. + /// In this mode all packets from the list of sent packets are + /// moved to list of archived packets once they have been matched + /// with received packets. This is required if it has been selected + /// from the command line to print timestamps for all packets after + /// the test. If this is not selected archiving should be disabled + /// for performance reasons and to avoid waste of memory for storing + /// large list of archived packets. + StatsMgr() : + exchanges_(), + custom_counters_(), + archive_enabled_(false) { + } + /// \brief Constructor. - StatsMgr() - : exchanges_(), - custom_counters_() { + /// + /// Use this constructor to set packets archive mode. + /// + /// \param archive_enabled true indicates that packets + /// archive mode is enabled. + StatsMgr(const bool archive_enabled) : + exchanges_(), + custom_counters_(), + archive_enabled_(archive_enabled) { } /// \brief Specify new exchange type. @@ -722,7 +781,8 @@ public: if (exchanges_.find(xchg_type) != exchanges_.end()) { isc_throw(BadValue, "Exchange of specified type already added."); } - exchanges_[xchg_type] = ExchangeStatsPtr(new ExchangeStats(xchg_type)); + exchanges_[xchg_type] = + ExchangeStatsPtr(new ExchangeStats(xchg_type, archive_enabled_)); } /// \brief Add named custom uint64 counter. @@ -806,7 +866,9 @@ public: if (sent_packet) { xchg_stats->updateDelays(sent_packet, packet); - xchg_stats->appendRcvd(packet); + if (archive_enabled_) { + xchg_stats->appendRcvd(packet); + } } } @@ -974,7 +1036,14 @@ public: /// - average packets delay, /// - maximum packets delay, /// - standard deviation of packets delay. + /// + /// \throw isc::InvalidOperation if no exchange type added to + /// track statistics. void printStats() const { + if (exchanges_.size() == 0) { + isc_throw(isc::InvalidOperation, + "no exchange type added for tracking"); + } for (ExchangesMapIterator it = exchanges_.begin(); it != exchanges_.end(); ++it) { @@ -994,8 +1063,16 @@ public: /// packets for all defined exchange types. /// /// \throw isc::InvalidOperation if one of the packets has - /// no timestamp value set. + /// no timestamp value set or if packets archive mode is + /// disabled. + /// + /// \throw isc::InvalidOperation if no exchange type added to + /// track statistics or packets archive mode is disabled. void printTimestamps() const { + if (exchanges_.size() == 0) { + isc_throw(isc::InvalidOperation, + "no exchange type added for tracking"); + } for (ExchangesMapIterator it = exchanges_.begin(); it != exchanges_.end(); ++it) { @@ -1012,19 +1089,23 @@ public: /// /// Method prints names and values of custom counters. Custom counters /// are defined by client class for tracking different statistics. + /// + /// \throw isc::InvalidOperation if no custom counters added for tracking. void printCustomCounters() const { - if (custom_counters_.size() > 0) { - std::cout << "***Various statistics counters***" << std::endl; + if (custom_counters_.size() == 0) { + isc_throw(isc::InvalidOperation, "no custom counters specified"); } for (CustomCountersMapIterator it = custom_counters_.begin(); it != custom_counters_.end(); ++it) { CustomCounterPtr counter = it->second; - std::cout << counter->getName() << ": " << counter->getValue() << std::endl; + std::cout << counter->getName() << ": " << counter->getValue() + << std::endl; } } private: + /// \brief Return exchange stats object for given exchange type /// /// Method returns exchange stats object for given exchange type. @@ -1043,6 +1124,16 @@ private: ExchangesMap exchanges_; ///< Map of exchange types. CustomCountersMap custom_counters_; ///< Map with custom counters. + + /// Indicates that packets from list of sent packets should be + /// archived (moved to list of archived packets) once they are + /// matched with received packets. This is required when it has + /// been selected from the command line to print packets' + /// timestamps after test. This may affect performance and + /// consume large amount of memory when the test is running + /// for extended period of time and many packets have to be + /// archived. + bool archive_enabled_; }; } // namespace perfdhcp diff --git a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc index 1611105885..2233847bb1 100644 --- a/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc +++ b/tests/tools/perfdhcp/tests/stats_mgr_unittest.cc @@ -410,5 +410,41 @@ TEST_F(StatsMgrTest, CustomCounters) { } +TEST_F(StatsMgrTest, PrintStats) { + std::cout << "This unit test is checking statistics printing " + << "capabilities. It is expected that some counters " + << "will be printed during this test. It may also " + << "cause spurious errors." << std::endl; + boost::shared_ptr stats_mgr(new StatsMgr6()); + stats_mgr->addExchangeStats(StatsMgr6::XCHG_SA); + + // Simulate sending and receiving one packet. Otherwise printing + // functions will complain about lack of packets. + const int packets_num = 1; + passMultiplePackets6(stats_mgr, StatsMgr6::XCHG_SA, DHCPV6_SOLICIT, + packets_num); + passMultiplePackets6(stats_mgr, StatsMgr6::XCHG_SA, DHCPV6_ADVERTISE, + packets_num, true); + + // This function will print statistics even if packets are not + // archived because it relies on counters. There is at least one + // exchange needed to count the average delay and std deviation. + EXPECT_NO_THROW(stats_mgr->printStats()); + + // Printing timestamps is expected to fail because by default we + // disable packets archiving mode. Without packets we can't get + // timestamps. + EXPECT_THROW(stats_mgr->printTimestamps(), isc::InvalidOperation); + + // Now, we create another statistics manager instance and enable + // packets archiving mode. + const bool archive_packets = true; + boost::shared_ptr stats_mgr2(new StatsMgr6(archive_packets)); + stats_mgr2->addExchangeStats(StatsMgr6::XCHG_SA); + + // Timestamps should now get printed because packets have been preserved. + EXPECT_NO_THROW(stats_mgr2->printTimestamps()); +} + } From f9abe2be61d8660f54e9b191940817093f3e902b Mon Sep 17 00:00:00 2001 From: Michal 'vorner' Vaner Date: Fri, 3 Aug 2012 21:01:06 +0200 Subject: [PATCH 29/69] [2066] Explain ACLs Not complete. Split done at the end of the day, without too much logic at which point. --- doc/guide/bind10-guide.xml | 129 +++++++++++++++++++++++++++++++++++++ 1 file changed, 129 insertions(+) diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml index 040b109799..0b97cc8180 100644 --- a/doc/guide/bind10-guide.xml +++ b/doc/guide/bind10-guide.xml @@ -1302,6 +1302,135 @@ TODO + + Common configuration elements + + + Some things are configured in the same or similar manner across + many modules. So we show them here in one place. + + +

+ ACLs + + + An ACL, or Access Control List, is a way to describe if a request + is allowed or disallowed. The principle is, there's a list of rules. + A request either matches the rule or it doesn't. If it matches, + a decision is made and no more ACL processing happens. If it does + not match, the processing moves to the next rule. If there are + no more rules, a default action is taken. + + + + Each element in the ACL is a dictionary. There's exactly one + action element. This element describes the + decision taken when the rule matches. If it is set to + "ACCEPT", the request processed. In case of "REJECT", the request + is not processed and an error message is sent back (what it means + depends on which ACL it is). The action of "DROP" does not + process the request and sends no answer — pretending the + request never came (which means the sender might resend it). + + + + Other elements inside the rule describe the properties a request + must have to match. The request must match all the properties + in the check. + + +
+ Matching properties + + + The first thing you can check against is the source address + of request. The name is from and the value + is a string containing either a single IPv4 or IPv6 address, + or a range in the usual slash notation (eg. "192.0.2.0/24"). + + + + The other is TSIG key by which the message was signed. The ACL + contains only the name (under the name "key"), the key itself + must be stored in the global keyring. + + + + More properties to match are planned — the destination + address, ports, matches against the packet content. + +
+ +
+ More complicated matches + + + From time to time, you need to express something more complex + than just a single address or key. + + + + You can specify a list of values instead of single value. Then + the property needs to match at least one of the values listed + — so you can say "from": ["192.0.2.0/24", + "2001:db8::/32"] to match any address in the ranges + set aside for documentation. The keys or any future properties + will work in a similar way. + + + + If that is not enough, you can compose the matching conditions + to logical expressions. They are called "ANY", "ALL" and "NOT". + The "ANY" and "ALL" ones contain lists of subexpressions — + each subexpression is a similar dictionary, just not containing + the "action" element. The "NOT" contains single subexpression. + Their function should be obvious — "NOT" matches if and + only if the subexpression does not match. The "ALL" matches exactly + when each of the subexpressions matches and "ANY" when at least + one matches. + +
+ +
+ Examples + + + All the examples here is just the JSON representing the ACL, + nicely formatted and split across lines. They are out of any + surrounding context. This is similar to what you'd get from + config show_json called on the entry containing + the ACL. + + + + In the first example, the ACL accepts queries from two known hosts. + Each host has an IP addresses (both IPv4 and IPv6) and a TSIG + key. Other queries are politely rejected. The last entry in the list + has no conditions — making it match any query. + + [ + { + "from": ["192.0.2.1", "2001:db8::1"], + "key": "first.key", + "action": "ACCEPT + }, + { + "from": ["192.0.2.2", "2001:db8::2"], + "key": "second.key", + "action": "ACCEPT" + }, + { + "action": "REJECT" + } +] + + + +
+
+ + Authoritative Server From 46ad312845e75e7b1e8481efe5c84ef6dd98939d Mon Sep 17 00:00:00 2001 From: Mukund Sivaraman Date: Sat, 4 Aug 2012 02:10:42 +0530 Subject: [PATCH 30/69] [2171] Remove redundant mkdir (already exists in cfgmgr) Besides, loadzone doesn't do anything with this directory. --- src/bin/loadzone/Makefile.am | 4 ---- 1 file changed, 4 deletions(-) diff --git a/src/bin/loadzone/Makefile.am b/src/bin/loadzone/Makefile.am index a235d68bd0..74386ae891 100644 --- a/src/bin/loadzone/Makefile.am +++ b/src/bin/loadzone/Makefile.am @@ -20,10 +20,6 @@ b10-loadzone: b10-loadzone.py -e "s|@@LIBEXECDIR@@|$(pkglibexecdir)|" b10-loadzone.py >$@ chmod a+x $@ -install-data-local: - $(mkinstalldirs) $(DESTDIR)/@localstatedir@/@PACKAGE@ -# TODO: permissions handled later - EXTRA_DIST += tests/normal/README EXTRA_DIST += tests/normal/dsset-subzone.example.com EXTRA_DIST += tests/normal/example.com From b71313c089c3a2dce333061b5b243555743561c8 Mon Sep 17 00:00:00 2001 From: Mukund Sivaraman Date: Sat, 4 Aug 2012 02:11:41 +0530 Subject: [PATCH 31/69] [2171] Set g+s,g+w permissions on localstatedir This is usually $prefix/var/bind10-devel/. We set mode to 2770 oct. --- src/bin/cfgmgr/Makefile.am | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/bin/cfgmgr/Makefile.am b/src/bin/cfgmgr/Makefile.am index aee78cf995..4a6fc0d30a 100644 --- a/src/bin/cfgmgr/Makefile.am +++ b/src/bin/cfgmgr/Makefile.am @@ -26,8 +26,9 @@ b10-cfgmgr: b10-cfgmgr.py install-data-local: $(mkinstalldirs) $(DESTDIR)/@localstatedir@/@PACKAGE@ -# TODO: permissions handled later +install-data-hook: + -chmod 2770 $(DESTDIR)/@localstatedir@/@PACKAGE@ CLEANDIRS = __pycache__ From 83e0fb268518baeccd6ece9b1c252c01f6925aae Mon Sep 17 00:00:00 2001 From: Michal 'vorner' Vaner Date: Mon, 6 Aug 2012 12:01:11 +0200 Subject: [PATCH 32/69] [2066] docs/ACLs: Some examples and how to bindctl it --- doc/guide/bind10-guide.xml | 63 +++++++++++++++++++++++++++++++++++++- 1 file changed, 62 insertions(+), 1 deletion(-) diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml index 0b97cc8180..65606c3d4b 100644 --- a/doc/guide/bind10-guide.xml +++ b/doc/guide/bind10-guide.xml @@ -1426,7 +1426,68 @@ TODO ] - + + Now we show two ways to accept only the queries from private ranges. + This is the same as rejecting anything that is outside. + + [ + { + "from": [ + "10.0.0.0/8", + "172.16.0.0/12", + "192.168.0.0/16", + "fc00::/7" + ], + "action": "ACCEPT" + }, + { + "action": "REJECT" + } +] + + [ + { + "NOT": { + "ANY": [ + {"from": "10.0.0.0/8"}, + {"from": "172.16.0.0/12"}, + {"from": "192.168.0.0/16"}, + {"from": "fc00::/7"} + ] + }, + "action": "REJECT" + }, + { + "action": "ACCEPT" + } +] + +
+ +
+ Interaction with <command>bindctl</command> + + + Currently, bindctl has hard time coping with + the variable nature of the ACL syntax. This technical limitation + makes it impossible to edit parts of the entries. You need to + set the whole entry at once, providing the whole JSON value. + + + + This limitation is planned to be solved soon at least partially. + + + + You'd do something like this to create the second example. + Note that the whole JSON must be on a single line. + + > config add somewhere/acl +> config set somewhere/acl[0] { "from": [ "10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16", "fc00::/7" ], "action": "ACCEPT" } +> config add somewhere/acl +> config set somewhere/acl[1] { "action": "REJECT" } +> config commit +
From 762239bdb617fcd2a3a737470f52aa5b5b70ccb9 Mon Sep 17 00:00:00 2001 From: Michal 'vorner' Vaner Date: Mon, 6 Aug 2012 13:40:11 +0200 Subject: [PATCH 33/69] [2066] docs: Update links to the ACL chapter Instead of talking the same over and over again, we now link the common ACL chapter. --- doc/guide/bind10-guide.xml | 116 +++---------------------------------- 1 file changed, 7 insertions(+), 109 deletions(-) diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml index 65606c3d4b..5df1c5c405 100644 --- a/doc/guide/bind10-guide.xml +++ b/doc/guide/bind10-guide.xml @@ -2073,37 +2073,22 @@ http://bind10.isc.org/wiki/ScalableZoneLoadDesign#a7.2UpdatingaZone can be used to control accessibility of the outbound zone transfer service. By default, b10-xfrout allows any clients to - perform zone transfers for any zones: + perform zone transfers for any zones. > config show Xfrout/transfer_acl Xfrout/transfer_acl[0] {"action": "ACCEPT"} any (default) - You can change this to, for example, rejecting all transfer - requests by default while allowing requests for the transfer - of zone "example.com" from 192.0.2.1 and 2001:db8::1 as follows: + You can configure it the same as any ACL + (). - > config set Xfrout/transfer_acl[0] {"action": "REJECT"} -> config add Xfrout/zone_config -> config set Xfrout/zone_config[0]/origin "example.com" -> config set Xfrout/zone_config[0]/transfer_acl [{"action": "ACCEPT", "from": "192.0.2.1"}, - {"action": "ACCEPT", "from": "2001:db8::1"}] -> config commit - - - In the above example the lines - for were divided for - readability. In the actual input it must be in a single line. - - If you want to require TSIG in access control, a system wide TSIG "key ring" must be configured. - For example, to change the previous example to allowing requests - from 192.0.2.1 signed by a TSIG with a key name of - "key.example", you'll need to do this: + In this example, we allow client matching both the IP address + and key. > config set tsig_keys/keys ["key.example:<base64-key>"] @@ -2320,29 +2305,7 @@ what is XfroutClient xfr_client?? - Multiple rules can be specified in the ACL, and an ACL rule - can consist of multiple constraints, such as a combination of - IP address and TSIG. - The following configuration sequence will add a new rule to - the ACL created in the above example. This additional rule - allows update requests sent from a client - using TSIG key name of "key.example" (different from the - key used in the previous example) and has an IPv6 address of ::1. - -> config add DDNS/zones[0]/update_acl {"action": "ACCEPT", "from": "::1", "key": "key.example"} -> config show DDNS/zones[0]/update_acl -DDNS/zones[0]/update_acl[0] {"action": "ACCEPT", "key": "key.example.org"} any (modified) -DDNS/zones[0]/update_acl[1] {"action": "ACCEPT", "from": "::1", "key": "key.example"} any (modified) -> config commit - - (Note the "add" in the first line. Before this sequence, we - have had only entry in zones[0]/update_acl. - The add command with a value (rule) adds - a new entry and sets it to the given rule. - - Due to a limitation of the current implementation, it doesn't - work if you first try to just add a new entry and then set it to - a given rule.) + Full description of ACLs can be found in . @@ -2357,21 +2320,6 @@ DDNS/zones[0]/update_acl[1] {"action": "ACCEPT", "from": "::1", "key": "key. should have a TSIG key in its constraints. - - The ACL rules will be checked in the listed order, and the - first matching one will apply. - If none of the rules matches, the default rule will apply, - which is rejecting any requests in the case of - b10-ddns. - - - - - Other actions than "ACCEPT", namely "REJECT" and "DROP", can be - used, too. - See about their effects. - - Currently update ACL can only control updates per zone basis; it's not possible to specify access control with higher @@ -2511,59 +2459,9 @@ DDNS/zones[0]/update_acl[1] {"action": "ACCEPT", "from": "::1", "key": "key. DNS queries from the localhost (127.0.0.1 and ::1). The configuration may be used to reject, drop, or allow specific IPs or networks. - This configuration list is first match. + See . - - The configuration's item may be - set to ACCEPT to allow the incoming query, - REJECT to respond with a DNS REFUSED return - code, or DROP to ignore the query without - any response (such as a blackhole). For more information, - see the respective debugging messages: RESOLVER_QUERY_ACCEPTED, - RESOLVER_QUERY_REJECTED, - and RESOLVER_QUERY_DROPPED. - - - - The required configuration's item is set - to an IPv4 or IPv6 address, addresses with an network mask, or to - the special lowercase keywords any6 (for - any IPv6 address) or any4 (for any IPv4 - address). - - - - - - For example to allow the 192.168.1.0/24 - network to use your recursive name server, at the - bindctl prompt run: - - - -> config add Resolver/query_acl -> config set Resolver/query_acl[2]/action "ACCEPT" -> config set Resolver/query_acl[2]/from "192.168.1.0/24" -> config commit - - - (Replace the 2 - as needed; run config show - Resolver/query_acl if needed.) - - - This prototype access control configuration - syntax may be changed. -
From d130356fbcc92d9b2220146fbfc11b9dc6f993c0 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Mon, 6 Aug 2012 08:22:00 -0700 Subject: [PATCH 34/69] [2172] Add OSX support to isc-sysinfo Mostly the same as FreeBSD, except for swap usage --- src/lib/python/isc/sysinfo/sysinfo.py | 78 +++++++++++++++++ .../python/isc/sysinfo/tests/sysinfo_test.py | 83 +++++++++++++++++++ 2 files changed, 161 insertions(+) diff --git a/src/lib/python/isc/sysinfo/sysinfo.py b/src/lib/python/isc/sysinfo/sysinfo.py index 0663a2e021..69032f7bab 100644 --- a/src/lib/python/isc/sysinfo/sysinfo.py +++ b/src/lib/python/isc/sysinfo/sysinfo.py @@ -414,6 +414,82 @@ class SysInfoFreeBSD(SysInfoBSD): except (subprocess.CalledProcessError, OSError): self._net_connections = 'Warning: "netstat -nr" command failed.\n' +class SysInfoOSX(SysInfoBSD): + """OS X (Darwin) implementation of the SysInfo class. + See the SysInfo class documentation for more information. + """ + def __init__(self): + super().__init__() + + # Don't know how to gather these + self._mem_cached = -1 + self._mem_buffers = -1 + + try: + s = subprocess.check_output(['sysctl', '-n', 'kern.smp.active']) + self._platform_is_smp = int(s.decode('utf-8').strip()) > 0 + except (subprocess.CalledProcessError, OSError): + pass + + try: + s = subprocess.check_output(['sysctl', '-n', 'kern.boottime']) + t = s.decode('utf-8').strip() + r = re.match('^\{\s+sec\s+\=\s+(\d+),.*', t) + if r: + sec = time.time() - int(r.group(1)) + self._uptime = int(round(sec)) + except (subprocess.CalledProcessError, OSError): + pass + + try: + s = subprocess.check_output(['sysctl', '-n', 'vm.loadavg']) + l = s.decode('utf-8').strip() + r = re.match('^\{(.*)\}$', l) + if r: + la = r.group(1).strip().split(' ') + else: + la = l.split(' ') + if len(la) >= 3: + self._loadavg = [float(la[0]), float(la[1]), float(la[2])] + except (subprocess.CalledProcessError, OSError): + pass + + try: + s = subprocess.check_output(['vm_stat']) + lines = s.decode('utf-8').split('\n') + # store all values in a dict + values = {} + page_size = 4096 + page_size_re = re.compile('page size of [0-9]+ bytes') + for line in lines: + page_size_m = page_size_re.match(line) + if page_size_m: + page_size = int(page_size_m.group(1)) + else: + key, _, value = line.partition(':') + values[key] = value.strip()[:-1] + self._mem_free = int(values['Pages free']) * page_size + except (subprocess.CalledProcessError, OSError): + pass + + try: + s = subprocess.check_output(['sysctl', '-n', 'vm.swapusage']) + l = s.decode('utf-8').strip() + r = re.match('^total = (\d+\.\d+)M\s+used = (\d+\.\d+)M\s+free = (\d+\.\d+)M', l) + if r: + self._mem_swap_total = float(r.group(1).strip()) * 1024 + self._mem_swap_free = float(r.group(3).strip()) * 1024 + except (subprocess.CalledProcessError, OSError): + pass + + try: + s = subprocess.check_output(['netstat', '-nr']) + self._net_routing_table = s.decode('utf-8') + except (subprocess.CalledProcessError, OSError): + self._net_connections = 'Warning: "netstat -nr" command failed.\n' + + + class SysInfoTestcase(SysInfo): def __init__(self): super().__init__() @@ -429,6 +505,8 @@ def SysInfoFromFactory(): return SysInfoOpenBSD() elif osname == 'FreeBSD': return SysInfoFreeBSD() + elif osname == 'Darwin': + return SysInfoOSX() elif osname == 'BIND10Testcase': return SysInfoTestcase() else: diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index 4d712cdf46..2d07a9959f 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -155,6 +155,41 @@ def _my_freebsd_subprocess_check_output(command): else: assert False, 'Unhandled command' +def _my_osx_platform_system(): + return 'Darwin' + +def _my_osx_os_sysconf(key): + if key == 'SC_NPROCESSORS_CONF': + return 91 + assert False, 'Unhandled key' + +def _my_osx_subprocess_check_output(command): + assert type(command) == list, 'command argument is not a list' + if command == ['hostname']: + return b'daemon.example.com\n' + elif command == ['sysctl', '-n', 'hw.physmem']: + return b'987654321\n' + elif command == ['ifconfig']: + return b'qB2osV6vUOjqm3P/+tQ4d92xoYz8/U8P9v3KWRpNwlI=\n' + elif command == ['netstat', '-s']: + return b'osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n' + elif command == ['netstat', '-an']: + return b'Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n' + elif command == ['sysctl', '-n', 'kern.smp.active']: + return b'0\n' + elif command == ['sysctl', '-n', 'kern.boottime']: + return bytes('{ sec = ' + str(int(time.time() - 76632)) + ', usec = 0 }\n', 'utf-8') + elif command == ['sysctl', '-n', 'vm.loadavg']: + return b'{ 0.2 0.4 0.6 }\n' + elif command == ['vm_stat']: + return b'Pages free: 12345.\n' + elif command == ['sysctl', '-n', 'vm.swapusage']: + return b'total = 18432.00M used = 17381.23M free = 1050.77M\n' + elif command == ['netstat', '-nr']: + return b'XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n' + else: + assert False, 'Unhandled command: ' + str(command) + class SysInfoTest(unittest.TestCase): def test_sysinfo(self): """Test that the various methods on SysInfo exist and return data.""" @@ -358,5 +393,53 @@ class SysInfoTest(unittest.TestCase): os.sysconf = old_os_sysconf subprocess.check_output = old_subprocess_check_output + def test_sysinfo_osx(self): + """Tests the OS X implementation of SysInfo. Note that this + tests deep into the implementation, and not just the + interfaces.""" + + # Don't run this test on platform other than FreeBSD as some + # system calls may not even be available. + osname = platform.system() + if osname != 'Darwin': + return + + # Save and replace existing implementations of library functions + # with mock ones for testing. + old_platform_system = platform.system + platform.system = _my_osx_platform_system + old_os_sysconf = os.sysconf + os.sysconf = _my_osx_os_sysconf + old_subprocess_check_output = subprocess.check_output + subprocess.check_output = _my_osx_subprocess_check_output + + s = SysInfoFromFactory() + self.assertEqual(91, s.get_num_processors()) + self.assertEqual('daemon.example.com', s.get_platform_hostname()) + self.assertFalse(s.get_platform_is_smp()) + + self.assertLess(abs(76632 - s.get_uptime()), 4) + self.assertEqual([0.2, 0.4, 0.6], s.get_loadavg()) + self.assertEqual(987654321, s.get_mem_total()) + self.assertEqual((12345 * 4096), s.get_mem_free()) + self.assertEqual(-1, s.get_mem_cached()) + self.assertEqual(-1, s.get_mem_buffers()) + self.assertEqual(18874368.0, s.get_mem_swap_total()) + self.assertEqual(1075988.48, s.get_mem_swap_free()) + self.assertRegexpMatches(s.get_platform_distro(), '^Darwin\s+.*') + + # These test that the corresponding tools are being called (and + # no further processing is done on this data). Please see the + # implementation functions at the top of this file. + self.assertEqual('qB2osV6vUOjqm3P/+tQ4d92xoYz8/U8P9v3KWRpNwlI=\n', s.get_net_interfaces()) + self.assertEqual('XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n', s.get_net_routing_table()) + self.assertEqual('osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n', s.get_net_stats()) + self.assertEqual('Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n', s.get_net_connections()) + + # Restore original implementations. + platform.system = old_platform_system + os.sysconf = old_os_sysconf + subprocess.check_output = old_subprocess_check_output + if __name__ == "__main__": unittest.main() From 9759be762e000ccdf9e17ca890fdb990bd773c02 Mon Sep 17 00:00:00 2001 From: Marcin Siodelski Date: Tue, 7 Aug 2012 15:29:40 +0100 Subject: [PATCH 35/69] [1958] Implemented changes suggested in the second code review. --- tests/tools/perfdhcp/stats_mgr.h | 31 +++++++++++++------------------ 1 file changed, 13 insertions(+), 18 deletions(-) diff --git a/tests/tools/perfdhcp/stats_mgr.h b/tests/tools/perfdhcp/stats_mgr.h index 4ca257614b..245c69e3cf 100644 --- a/tests/tools/perfdhcp/stats_mgr.h +++ b/tests/tools/perfdhcp/stats_mgr.h @@ -88,14 +88,14 @@ public: /// Method returns counter value. /// /// \return counter value. - uint64_t getValue() const { return counter_; } + uint64_t getValue() const { return(counter_); } /// \brief Return counter name. /// /// Method returns counter name. /// /// \return counter name. - const std::string& getName() const { return name_; } + const std::string& getName() const { return(name_); } private: /// \brief Default constructor. /// @@ -239,6 +239,8 @@ public: /// \brief Constructor /// /// \param xchg_type exchange type + /// \param archive_enabled if true packets archive mode is enabled. + /// In this mode all packets are stored throughout the test execution. ExchangeStats(const ExchangeType xchg_type, const bool archive_enabled) : xchg_type_(xchg_type), min_delay_(std::numeric_limits::max()), @@ -338,7 +340,7 @@ public: sum_delay_squared_ += delta * delta; } - /// \brief Find packet on the list of sent packets. + /// \brief Match received packet with the corresponding sent packet. /// /// Method finds packet with specified transaction id on the list /// of sent packets. It is used to match received packet with @@ -353,7 +355,7 @@ public: /// \throw isc::BadValue if received packet is null. /// \return packet having specified transaction or NULL if packet /// not found - boost::shared_ptr findSent(const boost::shared_ptr& rcvd_packet) { + boost::shared_ptr matchPackets(const boost::shared_ptr& rcvd_packet) { if (!rcvd_packet) { isc_throw(BadValue, "Received packet is null"); } @@ -426,6 +428,8 @@ public: return(boost::shared_ptr()); } + // Packet is matched so we count it. We don't count unmatched packets + // as they are counted as orphans with a separate counter. ++rcvd_packets_num_; boost::shared_ptr sent_packet(*next_sent_); // If packet was found, we assume it will be never searched @@ -495,7 +499,7 @@ public: /// \brief Return average unordered lookup set size. /// /// Method returns average unordered lookup set size. - /// This value changes every time \ref ExchangeStats::findSent + /// This value changes every time \ref ExchangeStats::matchPackets /// function performs unordered packet lookup. /// /// \throw isc::InvalidOperation if there have been no unordered @@ -741,7 +745,7 @@ public: /// Iterator for \ref CustomCountersMap. typedef typename CustomCountersMap::const_iterator CustomCountersMapIterator; - /// \brief Default constructor. + /// \brief Constructor. /// /// This constructor by default disables packets archiving mode. /// In this mode all packets from the list of sent packets are @@ -751,19 +755,10 @@ public: /// the test. If this is not selected archiving should be disabled /// for performance reasons and to avoid waste of memory for storing /// large list of archived packets. - StatsMgr() : - exchanges_(), - custom_counters_(), - archive_enabled_(false) { - } - - /// \brief Constructor. - /// - /// Use this constructor to set packets archive mode. /// /// \param archive_enabled true indicates that packets /// archive mode is enabled. - StatsMgr(const bool archive_enabled) : + StatsMgr(const bool archive_enabled = false) : exchanges_(), custom_counters_(), archive_enabled_(archive_enabled) { @@ -862,7 +857,7 @@ public: const boost::shared_ptr& packet) { ExchangeStatsPtr xchg_stats = getExchangeStats(xchg_type); boost::shared_ptr sent_packet - = xchg_stats->findSent(packet); + = xchg_stats->matchPackets(packet); if (sent_packet) { xchg_stats->updateDelays(sent_packet, packet); @@ -936,7 +931,7 @@ public: /// \brief Return average unordered lookup set size. /// /// Method returns average unordered lookup set size. - /// This value changes every time \ref ExchangeStats::findSent + /// This value changes every time \ref ExchangeStats::matchPackets /// function performs unordered packet lookup. /// /// \param xchg_type exchange type. From 371d3158521b1072115c55cac20b61ab9e605ea2 Mon Sep 17 00:00:00 2001 From: JINMEI Tatuya Date: Tue, 7 Aug 2012 16:56:52 -0700 Subject: [PATCH 36/69] [2172] fixed a trivial typo --- src/lib/python/isc/sysinfo/tests/sysinfo_test.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index 2d07a9959f..ccd8c9e1c0 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -398,7 +398,7 @@ class SysInfoTest(unittest.TestCase): tests deep into the implementation, and not just the interfaces.""" - # Don't run this test on platform other than FreeBSD as some + # Don't run this test on platform other than OS X as some # system calls may not even be available. osname = platform.system() if osname != 'Darwin': From ecb6a3aad71647bc11305fa8b8789a60ed591cfb Mon Sep 17 00:00:00 2001 From: Mukund Sivaraman Date: Wed, 8 Aug 2012 19:15:00 +0530 Subject: [PATCH 37/69] [master] Add logger_lockfile to .gitignore --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 3480cb68f5..dd6903ca89 100644 --- a/.gitignore +++ b/.gitignore @@ -35,3 +35,4 @@ TAGS /coverage-cpp-html /dns++.pc /report.info +/logger_lockfile From a47c2eae8dd926d05a7ba27f1cde67ac98006673 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Wed, 8 Aug 2012 21:05:25 +0200 Subject: [PATCH 38/69] [2172] rearrange freebsd/osx hierarchy --- src/lib/python/isc/sysinfo/sysinfo.py | 88 +++++++++++---------------- 1 file changed, 35 insertions(+), 53 deletions(-) diff --git a/src/lib/python/isc/sysinfo/sysinfo.py b/src/lib/python/isc/sysinfo/sysinfo.py index 69032f7bab..5f9b97ffab 100644 --- a/src/lib/python/isc/sysinfo/sysinfo.py +++ b/src/lib/python/isc/sysinfo/sysinfo.py @@ -349,9 +349,9 @@ class SysInfoOpenBSD(SysInfoBSD): except (subprocess.CalledProcessError, OSError): self._net_routing_table = 'Warning: "route -n show" command failed.\n' -class SysInfoFreeBSD(SysInfoBSD): - """FreeBSD implementation of the SysInfo class. - See the SysInfo class documentation for more information. +class SysInfoFreeBSDOSX(SysInfoBSD): + """Shared code for the FreeBSD and OS X implementations of the SysInfo + class. See the SysInfo class documentation for more information. """ def __init__(self): super().__init__() @@ -360,12 +360,6 @@ class SysInfoFreeBSD(SysInfoBSD): self._mem_cached = -1 self._mem_buffers = -1 - try: - s = subprocess.check_output(['sysctl', '-n', 'kern.smp.active']) - self._platform_is_smp = int(s.decode('utf-8').strip()) > 0 - except (subprocess.CalledProcessError, OSError): - pass - try: s = subprocess.check_output(['sysctl', '-n', 'kern.boottime']) t = s.decode('utf-8').strip() @@ -389,6 +383,25 @@ class SysInfoFreeBSD(SysInfoBSD): except (subprocess.CalledProcessError, OSError): pass + try: + s = subprocess.check_output(['netstat', '-nr']) + self._net_routing_table = s.decode('utf-8') + except (subprocess.CalledProcessError, OSError): + self._net_connections = 'Warning: "netstat -nr" command failed.\n' + +class SysInfoFreeBSD(SysInfoFreeBSDOSX): + """FreeBSD implementation of the SysInfo class. + See the SysInfo class documentation for more information. + """ + def __init__(self): + super().__init() + + try: + s = subprocess.check_output(['sysctl', '-n', 'kern.smp.active']) + self._platform_is_smp = int(s.decode('utf-8').strip()) > 0 + except (subprocess.CalledProcessError, OSError): + pass + try: s = subprocess.check_output(['vmstat', '-H']) lines = s.decode('utf-8').split('\n') @@ -408,49 +421,22 @@ class SysInfoFreeBSD(SysInfoBSD): except (subprocess.CalledProcessError, OSError): pass - try: - s = subprocess.check_output(['netstat', '-nr']) - self._net_routing_table = s.decode('utf-8') - except (subprocess.CalledProcessError, OSError): - self._net_connections = 'Warning: "netstat -nr" command failed.\n' -class SysInfoOSX(SysInfoBSD): + +class SysInfoOSX(SysInfoFreeBSDOSX): """OS X (Darwin) implementation of the SysInfo class. See the SysInfo class documentation for more information. """ def __init__(self): super().__init__() - # Don't know how to gather these - self._mem_cached = -1 - self._mem_buffers = -1 - + # note; this call overrides the value already set when hw.physmem + # was read. However, on OSX, physmem is not necessarily the correct + # value. But since it does not fail and does work on most BSD's, it's + # left in the base class and overwritten here try: - s = subprocess.check_output(['sysctl', '-n', 'kern.smp.active']) - self._platform_is_smp = int(s.decode('utf-8').strip()) > 0 - except (subprocess.CalledProcessError, OSError): - pass - - try: - s = subprocess.check_output(['sysctl', '-n', 'kern.boottime']) - t = s.decode('utf-8').strip() - r = re.match('^\{\s+sec\s+\=\s+(\d+),.*', t) - if r: - sec = time.time() - int(r.group(1)) - self._uptime = int(round(sec)) - except (subprocess.CalledProcessError, OSError): - pass - - try: - s = subprocess.check_output(['sysctl', '-n', 'vm.loadavg']) - l = s.decode('utf-8').strip() - r = re.match('^\{(.*)\}$', l) - if r: - la = r.group(1).strip().split(' ') - else: - la = l.split(' ') - if len(la) >= 3: - self._loadavg = [float(la[0]), float(la[1]), float(la[2])] + s = subprocess.check_output(['sysctl', '-n', 'hw.memsize']) + self._mem_total = int(s.decode('utf-8').strip()) except (subprocess.CalledProcessError, OSError): pass @@ -459,7 +445,7 @@ class SysInfoOSX(SysInfoBSD): lines = s.decode('utf-8').split('\n') # store all values in a dict values = {} - page_size = 4096 + page_size = None page_size_re = re.compile('page size of [0-9]+ bytes') for line in lines: page_size_m = page_size_re.match(line) @@ -468,7 +454,10 @@ class SysInfoOSX(SysInfoBSD): else: key, _, value = line.partition(':') values[key] = value.strip()[:-1] - self._mem_free = int(values['Pages free']) * page_size + # Only calculate memory if page size is known + if page_size is not None: + self._mem_free = int(values['Pages free']) * page_size + + int(values['Pages speculative']) * page_size except (subprocess.CalledProcessError, OSError): pass @@ -482,13 +471,6 @@ class SysInfoOSX(SysInfoBSD): except (subprocess.CalledProcessError, OSError): pass - try: - s = subprocess.check_output(['netstat', '-nr']) - self._net_routing_table = s.decode('utf-8') - except (subprocess.CalledProcessError, OSError): - self._net_connections = 'Warning: "netstat -nr" command failed.\n' - - class SysInfoTestcase(SysInfo): def __init__(self): From 121e490ee2758601ee3443fcd4b53686f7283701 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Wed, 8 Aug 2012 12:37:07 -0700 Subject: [PATCH 39/69] [2172] fix page_size regex --- src/lib/python/isc/sysinfo/sysinfo.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/lib/python/isc/sysinfo/sysinfo.py b/src/lib/python/isc/sysinfo/sysinfo.py index 5f9b97ffab..37446f9b7c 100644 --- a/src/lib/python/isc/sysinfo/sysinfo.py +++ b/src/lib/python/isc/sysinfo/sysinfo.py @@ -446,7 +446,7 @@ class SysInfoOSX(SysInfoFreeBSDOSX): # store all values in a dict values = {} page_size = None - page_size_re = re.compile('page size of [0-9]+ bytes') + page_size_re = re.compile('.*page size of ([0-9]+) bytes') for line in lines: page_size_m = page_size_re.match(line) if page_size_m: @@ -456,7 +456,7 @@ class SysInfoOSX(SysInfoFreeBSDOSX): values[key] = value.strip()[:-1] # Only calculate memory if page size is known if page_size is not None: - self._mem_free = int(values['Pages free']) * page_size + + self._mem_free = int(values['Pages free']) * page_size +\ int(values['Pages speculative']) * page_size except (subprocess.CalledProcessError, OSError): pass From 0d4243fc59eca849a010bde05a4d18ba19c1fd92 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Wed, 8 Aug 2012 12:54:22 -0700 Subject: [PATCH 40/69] [2172] fix osx sysinfo tests for previous change --- src/lib/python/isc/sysinfo/tests/sysinfo_test.py | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index ccd8c9e1c0..6b7c799d1e 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -169,6 +169,9 @@ def _my_osx_subprocess_check_output(command): return b'daemon.example.com\n' elif command == ['sysctl', '-n', 'hw.physmem']: return b'987654321\n' + elif command == ['sysctl', '-n', 'hw.memsize']: + # Something different than physmem + return b'123456789\n' elif command == ['ifconfig']: return b'qB2osV6vUOjqm3P/+tQ4d92xoYz8/U8P9v3KWRpNwlI=\n' elif command == ['netstat', '-s']: @@ -182,7 +185,7 @@ def _my_osx_subprocess_check_output(command): elif command == ['sysctl', '-n', 'vm.loadavg']: return b'{ 0.2 0.4 0.6 }\n' elif command == ['vm_stat']: - return b'Pages free: 12345.\n' + return b'Mach Virtual Memory Statistics: (page size of 4096 bytes)\nPages free: 12345.\nPages speculative: 11111.\n' elif command == ['sysctl', '-n', 'vm.swapusage']: return b'total = 18432.00M used = 17381.23M free = 1050.77M\n' elif command == ['netstat', '-nr']: @@ -420,8 +423,8 @@ class SysInfoTest(unittest.TestCase): self.assertLess(abs(76632 - s.get_uptime()), 4) self.assertEqual([0.2, 0.4, 0.6], s.get_loadavg()) - self.assertEqual(987654321, s.get_mem_total()) - self.assertEqual((12345 * 4096), s.get_mem_free()) + self.assertEqual(123456789, s.get_mem_total()) + self.assertEqual((23456 * 4096), s.get_mem_free()) self.assertEqual(-1, s.get_mem_cached()) self.assertEqual(-1, s.get_mem_buffers()) self.assertEqual(18874368.0, s.get_mem_swap_total()) From 665f4873d2c67ffeb1b6acc73b5336f9c43f39ee Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Thu, 9 Aug 2012 00:16:34 +0200 Subject: [PATCH 41/69] [2172] refactor sysinfo tests --- src/lib/python/isc/sysinfo/sysinfo.py | 2 +- .../python/isc/sysinfo/tests/sysinfo_test.py | 127 +++++++++--------- 2 files changed, 66 insertions(+), 63 deletions(-) diff --git a/src/lib/python/isc/sysinfo/sysinfo.py b/src/lib/python/isc/sysinfo/sysinfo.py index 37446f9b7c..d79e77cd32 100644 --- a/src/lib/python/isc/sysinfo/sysinfo.py +++ b/src/lib/python/isc/sysinfo/sysinfo.py @@ -394,7 +394,7 @@ class SysInfoFreeBSD(SysInfoFreeBSDOSX): See the SysInfo class documentation for more information. """ def __init__(self): - super().__init() + super().__init__() try: s = subprocess.check_output(['sysctl', '-n', 'kern.smp.active']) diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index 6b7c799d1e..1b03e993fc 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -20,6 +20,13 @@ import platform import subprocess import time +# Long list of different values used throughout the tests +NPROCESSORS_LINUX = 42 +NPROCESSORS_OPENBSD = 43 +NPROCESSORS_FREEBSD = 44 +NPROCESSORS_OSX = 45 + + def _my_testcase_platform_system(): return 'BIND10Testcase' @@ -28,7 +35,7 @@ def _my_linux_platform_system(): def _my_linux_os_sysconf(key): if key == 'SC_NPROCESSORS_CONF': - return 42 + return NPROCESSORS_LINUX assert False, 'Unhandled key' class MyLinuxFile: @@ -92,106 +99,102 @@ def _my_openbsd_platform_system(): def _my_openbsd_os_sysconf(key): if key == 'SC_NPROCESSORS_CONF': - return 53 + return NPROCESSORS_OPENBSD assert False, 'Unhandled key' -def _my_openbsd_subprocess_check_output(command): +def _my_bsd_subprocess_check_output(command): + '''subprocess output for all bsd types''' assert type(command) == list, 'command argument is not a list' if command == ['hostname']: return b'blowfish.example.com\n' - elif command == ['sysctl', '-n', 'kern.boottime']: - return bytes(str(int(time.time() - 76632)), 'utf-8') - elif command == ['sysctl', '-n', 'vm.loadavg']: - return b'0.7 0.9 0.8\n' elif command == ['sysctl', '-n', 'hw.physmem']: return b'543214321\n' - elif command == ['vmstat']: - return b' procs memory page disks traps cpu\n r b w avm fre flt re pi po fr sr wd0 cd0 int sys cs us sy id\n 0 0 0 121212 123456 47 0 0 0 0 0 2 0 2 80 14 0 1 99\n' - elif command == ['swapctl', '-s', '-k']: - return b'total: 553507 1K-blocks allocated, 2 used, 553505 available' elif command == ['ifconfig']: return b'qB2osV6vUOjqm3P/+tQ4d92xoYz8/U8P9v3KWRpNwlI=\n' - elif command == ['route', '-n', 'show']: - return b'XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n' elif command == ['netstat', '-s']: return b'osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n' elif command == ['netstat', '-an']: return b'Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n' else: - assert False, 'Unhandled command' + return None + +def _my_openbsd_subprocess_check_output(command): + assert type(command) == list, 'command argument is not a list' + bsd_output = _my_bsd_subprocess_check_output(command) + if command == ['sysctl', '-n', 'kern.boottime']: + return bytes(str(int(time.time() - 76632)), 'utf-8') + elif command == ['sysctl', '-n', 'vm.loadavg']: + return b'0.7 0.9 0.8\n' + elif command == ['vmstat']: + return b' procs memory page disks traps cpu\n r b w avm fre flt re pi po fr sr wd0 cd0 int sys cs us sy id\n 0 0 0 121212 123456 47 0 0 0 0 0 2 0 2 80 14 0 1 99\n' + elif command == ['swapctl', '-s', '-k']: + return b'total: 553507 1K-blocks allocated, 2 used, 553505 available' + elif command == ['route', '-n', 'show']: + return b'XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n' + else: + if bsd_output is not None: + return bsd_output + else: + assert False, 'Unhandled command' def _my_freebsd_platform_system(): return 'FreeBSD' def _my_freebsd_os_sysconf(key): if key == 'SC_NPROCESSORS_CONF': - return 91 + return NPROCESSORS_FREEBSD assert False, 'Unhandled key' -def _my_freebsd_subprocess_check_output(command): +def _my_freebsd_osx_subprocess_check_output(command): + '''subprocess output shared for freebsd and osx''' assert type(command) == list, 'command argument is not a list' - if command == ['hostname']: - return b'daemon.example.com\n' - elif command == ['sysctl', '-n', 'kern.smp.active']: - return b'1\n' - elif command == ['sysctl', '-n', 'kern.boottime']: + if command == ['sysctl', '-n', 'kern.boottime']: return bytes('{ sec = ' + str(int(time.time() - 76632)) + ', usec = 0 }\n', 'utf-8') elif command == ['sysctl', '-n', 'vm.loadavg']: return b'{ 0.2 0.4 0.6 }\n' - elif command == ['sysctl', '-n', 'hw.physmem']: - return b'987654321\n' + elif command == ['netstat', '-nr']: + return b'XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n' + else: + return _my_bsd_subprocess_check_output(command) + +def _my_freebsd_subprocess_check_output(command): + assert type(command) == list, 'command argument is not a list' + if command == ['sysctl', '-n', 'kern.smp.active']: + return b'1\n' elif command == ['vmstat', '-H']: return b' procs memory page disks traps cpu\n r b w avm fre flt re pi po fr sr wd0 cd0 int sys cs us sy id\n 0 0 0 343434 123456 47 0 0 0 0 0 2 0 2 80 14 0 1 99\n' elif command == ['swapctl', '-s', '-k']: return b'Total: 1013216 0\n' - elif command == ['ifconfig']: - return b'qB2osV6vUOjqm3P/+tQ4d92xoYz8/U8P9v3KWRpNwlI=\n' - elif command == ['netstat', '-nr']: - return b'XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n' - elif command == ['netstat', '-s']: - return b'osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n' - elif command == ['netstat', '-an']: - return b'Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n' else: - assert False, 'Unhandled command' + freebsd_osx_output = _my_freebsd_osx_subprocess_check_output(command) + if freebsd_osx_output is not None: + return freebsd_osx_output + else: + assert False, 'Unhandled command' def _my_osx_platform_system(): return 'Darwin' def _my_osx_os_sysconf(key): if key == 'SC_NPROCESSORS_CONF': - return 91 + return NPROCESSORS_OSX assert False, 'Unhandled key' def _my_osx_subprocess_check_output(command): assert type(command) == list, 'command argument is not a list' - if command == ['hostname']: - return b'daemon.example.com\n' - elif command == ['sysctl', '-n', 'hw.physmem']: - return b'987654321\n' - elif command == ['sysctl', '-n', 'hw.memsize']: - # Something different than physmem + if command == ['sysctl', '-n', 'hw.memsize']: + # Something different than physmem from bsd return b'123456789\n' - elif command == ['ifconfig']: - return b'qB2osV6vUOjqm3P/+tQ4d92xoYz8/U8P9v3KWRpNwlI=\n' - elif command == ['netstat', '-s']: - return b'osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n' - elif command == ['netstat', '-an']: - return b'Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n' - elif command == ['sysctl', '-n', 'kern.smp.active']: - return b'0\n' - elif command == ['sysctl', '-n', 'kern.boottime']: - return bytes('{ sec = ' + str(int(time.time() - 76632)) + ', usec = 0 }\n', 'utf-8') - elif command == ['sysctl', '-n', 'vm.loadavg']: - return b'{ 0.2 0.4 0.6 }\n' elif command == ['vm_stat']: return b'Mach Virtual Memory Statistics: (page size of 4096 bytes)\nPages free: 12345.\nPages speculative: 11111.\n' elif command == ['sysctl', '-n', 'vm.swapusage']: return b'total = 18432.00M used = 17381.23M free = 1050.77M\n' - elif command == ['netstat', '-nr']: - return b'XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n' else: - assert False, 'Unhandled command: ' + str(command) + freebsd_osx_output = _my_freebsd_osx_subprocess_check_output(command) + if freebsd_osx_output is not None: + return freebsd_osx_output + else: + assert False, 'Unhandled command' class SysInfoTest(unittest.TestCase): def test_sysinfo(self): @@ -273,7 +276,7 @@ class SysInfoTest(unittest.TestCase): subprocess.check_output = _my_linux_subprocess_check_output s = SysInfoFromFactory() - self.assertEqual(42, s.get_num_processors()) + self.assertEqual(NPROCESSORS_LINUX, s.get_num_processors()) self.assertEqual('myhostname', s.get_platform_hostname()) self.assertTrue(s.get_platform_is_smp()) self.assertEqual(86401, s.get_uptime()) @@ -321,7 +324,7 @@ class SysInfoTest(unittest.TestCase): subprocess.check_output = _my_openbsd_subprocess_check_output s = SysInfoFromFactory() - self.assertEqual(53, s.get_num_processors()) + self.assertEqual(NPROCESSORS_OPENBSD, s.get_num_processors()) self.assertEqual('blowfish.example.com', s.get_platform_hostname()) self.assertFalse(s.get_platform_is_smp()) @@ -369,14 +372,14 @@ class SysInfoTest(unittest.TestCase): subprocess.check_output = _my_freebsd_subprocess_check_output s = SysInfoFromFactory() - self.assertEqual(91, s.get_num_processors()) - self.assertEqual('daemon.example.com', s.get_platform_hostname()) + self.assertEqual(NPROCESSORS_FREEBSD, s.get_num_processors()) + self.assertEqual('blowfish.example.com', s.get_platform_hostname()) self.assertTrue(s.get_platform_is_smp()) self.assertLess(abs(76632 - s.get_uptime()), 4) self.assertEqual([0.2, 0.4, 0.6], s.get_loadavg()) - self.assertEqual(987654321, s.get_mem_total()) - self.assertEqual(987654321 - (343434 * 1024), s.get_mem_free()) + self.assertEqual(543214321, s.get_mem_total()) + self.assertEqual(543214321 - (343434 * 1024), s.get_mem_free()) self.assertEqual(-1, s.get_mem_cached()) self.assertEqual(-1, s.get_mem_buffers()) self.assertEqual(1037533184, s.get_mem_swap_total()) @@ -417,8 +420,8 @@ class SysInfoTest(unittest.TestCase): subprocess.check_output = _my_osx_subprocess_check_output s = SysInfoFromFactory() - self.assertEqual(91, s.get_num_processors()) - self.assertEqual('daemon.example.com', s.get_platform_hostname()) + self.assertEqual(NPROCESSORS_OSX, s.get_num_processors()) + self.assertEqual('blowfish.example.com', s.get_platform_hostname()) self.assertFalse(s.get_platform_is_smp()) self.assertLess(abs(76632 - s.get_uptime()), 4) From 0b2ed3a094e9c083d22df78c89c503c80d4e2dbf Mon Sep 17 00:00:00 2001 From: JINMEI Tatuya Date: Wed, 8 Aug 2012 23:55:08 -0700 Subject: [PATCH 42/69] [2066] missing double-quote --- doc/guide/bind10-guide.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml index 5df1c5c405..97437c8d12 100644 --- a/doc/guide/bind10-guide.xml +++ b/doc/guide/bind10-guide.xml @@ -1413,7 +1413,7 @@ TODO { "from": ["192.0.2.1", "2001:db8::1"], "key": "first.key", - "action": "ACCEPT + "action": "ACCEPT" }, { "from": ["192.0.2.2", "2001:db8::2"], From 17f47f632edf3ca466ad2e5320c4211f75a4ed11 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Thu, 9 Aug 2012 13:48:51 +0200 Subject: [PATCH 43/69] [2172] comments and internal documentation --- .../python/isc/sysinfo/tests/sysinfo_test.py | 30 +++++++++++++++---- 1 file changed, 25 insertions(+), 5 deletions(-) diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index 1b03e993fc..6c4d046974 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -20,7 +20,8 @@ import platform import subprocess import time -# Long list of different values used throughout the tests +# different fake 'number of processors' values used for the different +# operating systems NPROCESSORS_LINUX = 42 NPROCESSORS_OPENBSD = 43 NPROCESSORS_FREEBSD = 44 @@ -102,11 +103,30 @@ def _my_openbsd_os_sysconf(key): return NPROCESSORS_OPENBSD assert False, 'Unhandled key' +# For the BSD types, there is a hierarchy that mostly resembles the +# class hierarchy in the sysinfo library; +# These are output strings of commands that sysinfo calls +# +# The test hierarchy is used as follows: +# Each operating system has its own _my__subprocess_check_output +# call. If the call is not found, it calls it's 'parent' (e.g. +# for openbsd that is my_bsd_subprocesses_check_output). +# +# If that returns None, the call had no test value and the test fails +# (and needs to be updated). +# The child classes are checked first so that they can override +# output from the parents, if necessary. +# +# Some parents have their own parent +# (e.g. _my_freebsd_osx_subprocess_check_output), in that case, +# if they do not recognize the command, they simply return whatever +# their parent returns + def _my_bsd_subprocess_check_output(command): '''subprocess output for all bsd types''' assert type(command) == list, 'command argument is not a list' if command == ['hostname']: - return b'blowfish.example.com\n' + return b'test.example.com\n' elif command == ['sysctl', '-n', 'hw.physmem']: return b'543214321\n' elif command == ['ifconfig']: @@ -325,7 +345,7 @@ class SysInfoTest(unittest.TestCase): s = SysInfoFromFactory() self.assertEqual(NPROCESSORS_OPENBSD, s.get_num_processors()) - self.assertEqual('blowfish.example.com', s.get_platform_hostname()) + self.assertEqual('test.example.com', s.get_platform_hostname()) self.assertFalse(s.get_platform_is_smp()) self.assertLess(abs(76632 - s.get_uptime()), 4) @@ -373,7 +393,7 @@ class SysInfoTest(unittest.TestCase): s = SysInfoFromFactory() self.assertEqual(NPROCESSORS_FREEBSD, s.get_num_processors()) - self.assertEqual('blowfish.example.com', s.get_platform_hostname()) + self.assertEqual('test.example.com', s.get_platform_hostname()) self.assertTrue(s.get_platform_is_smp()) self.assertLess(abs(76632 - s.get_uptime()), 4) @@ -421,7 +441,7 @@ class SysInfoTest(unittest.TestCase): s = SysInfoFromFactory() self.assertEqual(NPROCESSORS_OSX, s.get_num_processors()) - self.assertEqual('blowfish.example.com', s.get_platform_hostname()) + self.assertEqual('test.example.com', s.get_platform_hostname()) self.assertFalse(s.get_platform_is_smp()) self.assertLess(abs(76632 - s.get_uptime()), 4) From e107153e1d8515dafd02be789e81b9d8c38d728d Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Thu, 9 Aug 2012 14:17:12 +0200 Subject: [PATCH 44/69] [2172] don't make tests themselves platformdependent But fake every platform always; all platform-dependent calls should be prepared by the test setup anyway --- .../python/isc/sysinfo/tests/sysinfo_test.py | 60 ++++++++++--------- 1 file changed, 33 insertions(+), 27 deletions(-) diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index 6c4d046974..a7fa365bcb 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -103,6 +103,9 @@ def _my_openbsd_os_sysconf(key): return NPROCESSORS_OPENBSD assert False, 'Unhandled key' +def _my_openbsd_platform_uname(): + return ('OpenBSD', 'test.example.com', '5.0', '', 'amd64') + # For the BSD types, there is a hierarchy that mostly resembles the # class hierarchy in the sysinfo library; # These are output strings of commands that sysinfo calls @@ -165,6 +168,9 @@ def _my_freebsd_os_sysconf(key): return NPROCESSORS_FREEBSD assert False, 'Unhandled key' +def _my_freebsd_platform_uname(): + return ('FreeBSD', 'freebsd', '8.2-RELEASE', '', 'i386') + def _my_freebsd_osx_subprocess_check_output(command): '''subprocess output shared for freebsd and osx''' assert type(command) == list, 'command argument is not a list' @@ -195,6 +201,9 @@ def _my_freebsd_subprocess_check_output(command): def _my_osx_platform_system(): return 'Darwin' +def _my_osx_platform_uname(): + return ('Darwin', 'test.example.com', '10.6.0', '', '') + def _my_osx_os_sysconf(key): if key == 'SC_NPROCESSORS_CONF': return NPROCESSORS_OSX @@ -278,12 +287,6 @@ class SysInfoTest(unittest.TestCase): tests deep into the implementation, and not just the interfaces.""" - # Don't run this test on platform other than Linux as some - # system calls may not even be available. - osname = platform.system() - if osname != 'Linux': - return - # Save and replace existing implementations of library functions # with mock ones for testing. old_platform_system = platform.system @@ -328,12 +331,6 @@ class SysInfoTest(unittest.TestCase): tests deep into the implementation, and not just the interfaces.""" - # Don't run this test on platform other than OpenBSD as some - # system calls may not even be available. - osname = platform.system() - if osname != 'OpenBSD': - return - # Save and replace existing implementations of library functions # with mock ones for testing. old_platform_system = platform.system @@ -342,6 +339,8 @@ class SysInfoTest(unittest.TestCase): os.sysconf = _my_openbsd_os_sysconf old_subprocess_check_output = subprocess.check_output subprocess.check_output = _my_openbsd_subprocess_check_output + old_os_uname = os.uname + os.uname = _my_openbsd_platform_uname s = SysInfoFromFactory() self.assertEqual(NPROCESSORS_OPENBSD, s.get_num_processors()) @@ -356,7 +355,12 @@ class SysInfoTest(unittest.TestCase): self.assertEqual(-1, s.get_mem_buffers()) self.assertEqual(566791168, s.get_mem_swap_total()) self.assertEqual(566789120, s.get_mem_swap_free()) - self.assertRegexpMatches(s.get_platform_distro(), '^OpenBSD\s+.*') + # Try new regex assertion (which replaced the deprecated + # assertRegexpMatches. If it is not available, use the old one + try: + self.assertRegex(s.get_platform_distro(), '^OpenBSD\s+.*') + except AttributeError: + self.assertRegexpMatches(s.get_platform_distro(), '^OpenBSD\s+.*') # These test that the corresponding tools are being called (and # no further processing is done on this data). Please see the @@ -376,12 +380,6 @@ class SysInfoTest(unittest.TestCase): tests deep into the implementation, and not just the interfaces.""" - # Don't run this test on platform other than FreeBSD as some - # system calls may not even be available. - osname = platform.system() - if osname != 'FreeBSD': - return - # Save and replace existing implementations of library functions # with mock ones for testing. old_platform_system = platform.system @@ -390,6 +388,8 @@ class SysInfoTest(unittest.TestCase): os.sysconf = _my_freebsd_os_sysconf old_subprocess_check_output = subprocess.check_output subprocess.check_output = _my_freebsd_subprocess_check_output + old_os_uname = os.uname + os.uname = _my_freebsd_platform_uname s = SysInfoFromFactory() self.assertEqual(NPROCESSORS_FREEBSD, s.get_num_processors()) @@ -404,7 +404,12 @@ class SysInfoTest(unittest.TestCase): self.assertEqual(-1, s.get_mem_buffers()) self.assertEqual(1037533184, s.get_mem_swap_total()) self.assertEqual(1037533184, s.get_mem_swap_free()) - self.assertRegexpMatches(s.get_platform_distro(), '^FreeBSD\s+.*') + # Try new regex assertion (which replaced the deprecated + # assertRegexpMatches. If it is not available, use the old one + try: + self.assertRegex(s.get_platform_distro(), '^FreeBSD\s+.*') + except AttributeError: + self.assertRegexpMatches(s.get_platform_distro(), '^FreeBSD\s+.*') # These test that the corresponding tools are being called (and # no further processing is done on this data). Please see the @@ -424,12 +429,6 @@ class SysInfoTest(unittest.TestCase): tests deep into the implementation, and not just the interfaces.""" - # Don't run this test on platform other than OS X as some - # system calls may not even be available. - osname = platform.system() - if osname != 'Darwin': - return - # Save and replace existing implementations of library functions # with mock ones for testing. old_platform_system = platform.system @@ -438,6 +437,8 @@ class SysInfoTest(unittest.TestCase): os.sysconf = _my_osx_os_sysconf old_subprocess_check_output = subprocess.check_output subprocess.check_output = _my_osx_subprocess_check_output + old_os_uname = os.uname + os.uname = _my_osx_platform_uname s = SysInfoFromFactory() self.assertEqual(NPROCESSORS_OSX, s.get_num_processors()) @@ -452,7 +453,12 @@ class SysInfoTest(unittest.TestCase): self.assertEqual(-1, s.get_mem_buffers()) self.assertEqual(18874368.0, s.get_mem_swap_total()) self.assertEqual(1075988.48, s.get_mem_swap_free()) - self.assertRegexpMatches(s.get_platform_distro(), '^Darwin\s+.*') + # Try new regex assertion (which replaced the deprecated + # assertRegexpMatches. If it is not available, use the old one + try: + self.assertRegex(s.get_platform_distro(), '^Darwin\s+.*') + except AttributeError: + self.assertRegexpMatches(s.get_platform_distro(), '^Darwin\s+.*') # These test that the corresponding tools are being called (and # no further processing is done on this data). Please see the From cf59b08af0e9db877300b8dd81fb3679d4dc2b93 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Thu, 9 Aug 2012 14:34:28 +0200 Subject: [PATCH 45/69] [2172] save and restore syscalls in setup/teardown --- .../python/isc/sysinfo/tests/sysinfo_test.py | 61 ++++++------------- 1 file changed, 20 insertions(+), 41 deletions(-) diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index a7fa365bcb..d9e3cc680c 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -226,6 +226,22 @@ def _my_osx_subprocess_check_output(command): assert False, 'Unhandled command' class SysInfoTest(unittest.TestCase): + + def setUp(self): + # Save existing implementations of library functions + # (they are replaced in the tests) + self.old_platform_system = platform.system + self.old_os_sysconf = os.sysconf + self.old_open = __builtins__.open + self.old_subprocess_check_output = subprocess.check_output + + def tearDown(self): + # Restore the library functions + platform.system = self.old_platform_system + os.sysconf = self.old_os_sysconf + __builtins__.open = self.old_open + subprocess.check_output = self.old_subprocess_check_output + def test_sysinfo(self): """Test that the various methods on SysInfo exist and return data.""" @@ -287,15 +303,11 @@ class SysInfoTest(unittest.TestCase): tests deep into the implementation, and not just the interfaces.""" - # Save and replace existing implementations of library functions + # Replace existing implementations of library functions # with mock ones for testing. - old_platform_system = platform.system platform.system = _my_linux_platform_system - old_os_sysconf = os.sysconf os.sysconf = _my_linux_os_sysconf - old_open = __builtins__.open __builtins__.open = _my_linux_open - old_subprocess_check_output = subprocess.check_output subprocess.check_output = _my_linux_subprocess_check_output s = SysInfoFromFactory() @@ -320,26 +332,16 @@ class SysInfoTest(unittest.TestCase): self.assertEqual('osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n', s.get_net_stats()) self.assertEqual('Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n', s.get_net_connections()) - # Restore original implementations. - platform.system = old_platform_system - os.sysconf = old_os_sysconf - __builtins__.open = old_open - subprocess.check_output = old_subprocess_check_output - def test_sysinfo_openbsd(self): """Tests the OpenBSD implementation of SysInfo. Note that this tests deep into the implementation, and not just the interfaces.""" - # Save and replace existing implementations of library functions + # Replace existing implementations of library functions # with mock ones for testing. - old_platform_system = platform.system platform.system = _my_openbsd_platform_system - old_os_sysconf = os.sysconf os.sysconf = _my_openbsd_os_sysconf - old_subprocess_check_output = subprocess.check_output subprocess.check_output = _my_openbsd_subprocess_check_output - old_os_uname = os.uname os.uname = _my_openbsd_platform_uname s = SysInfoFromFactory() @@ -370,25 +372,16 @@ class SysInfoTest(unittest.TestCase): self.assertEqual('osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n', s.get_net_stats()) self.assertEqual('Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n', s.get_net_connections()) - # Restore original implementations. - platform.system = old_platform_system - os.sysconf = old_os_sysconf - subprocess.check_output = old_subprocess_check_output - def test_sysinfo_freebsd(self): """Tests the FreeBSD implementation of SysInfo. Note that this tests deep into the implementation, and not just the interfaces.""" - # Save and replace existing implementations of library functions + # Replace existing implementations of library functions # with mock ones for testing. - old_platform_system = platform.system platform.system = _my_freebsd_platform_system - old_os_sysconf = os.sysconf os.sysconf = _my_freebsd_os_sysconf - old_subprocess_check_output = subprocess.check_output subprocess.check_output = _my_freebsd_subprocess_check_output - old_os_uname = os.uname os.uname = _my_freebsd_platform_uname s = SysInfoFromFactory() @@ -419,25 +412,16 @@ class SysInfoTest(unittest.TestCase): self.assertEqual('osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n', s.get_net_stats()) self.assertEqual('Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n', s.get_net_connections()) - # Restore original implementations. - platform.system = old_platform_system - os.sysconf = old_os_sysconf - subprocess.check_output = old_subprocess_check_output - def test_sysinfo_osx(self): """Tests the OS X implementation of SysInfo. Note that this tests deep into the implementation, and not just the interfaces.""" - # Save and replace existing implementations of library functions + # Replace existing implementations of library functions # with mock ones for testing. - old_platform_system = platform.system platform.system = _my_osx_platform_system - old_os_sysconf = os.sysconf os.sysconf = _my_osx_os_sysconf - old_subprocess_check_output = subprocess.check_output subprocess.check_output = _my_osx_subprocess_check_output - old_os_uname = os.uname os.uname = _my_osx_platform_uname s = SysInfoFromFactory() @@ -468,10 +452,5 @@ class SysInfoTest(unittest.TestCase): self.assertEqual('osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n', s.get_net_stats()) self.assertEqual('Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n', s.get_net_connections()) - # Restore original implementations. - platform.system = old_platform_system - os.sysconf = old_os_sysconf - subprocess.check_output = old_subprocess_check_output - if __name__ == "__main__": unittest.main() From 1f3ce585601a7c861443221e53f49e71431a15ae Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Thu, 9 Aug 2012 15:01:55 +0200 Subject: [PATCH 46/69] [2172] refactor check of general bsd values too --- .../python/isc/sysinfo/tests/sysinfo_test.py | 60 +++++++------------ 1 file changed, 23 insertions(+), 37 deletions(-) diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index d9e3cc680c..f01319088f 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -332,6 +332,21 @@ class SysInfoTest(unittest.TestCase): self.assertEqual('osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n', s.get_net_stats()) self.assertEqual('Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n', s.get_net_connections()) + def check_bsd_values(self, s): + # check values shared by all bsd implementations + self.assertEqual('test.example.com', s.get_platform_hostname()) + self.assertLess(abs(76632 - s.get_uptime()), 4) + self.assertEqual(-1, s.get_mem_cached()) + self.assertEqual(-1, s.get_mem_buffers()) + + # These test that the corresponding tools are being called (and + # no further processing is done on this data). Please see the + # implementation functions at the top of this file. + self.assertEqual('qB2osV6vUOjqm3P/+tQ4d92xoYz8/U8P9v3KWRpNwlI=\n', s.get_net_interfaces()) + self.assertEqual('XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n', s.get_net_routing_table()) + self.assertEqual('osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n', s.get_net_stats()) + self.assertEqual('Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n', s.get_net_connections()) + def test_sysinfo_openbsd(self): """Tests the OpenBSD implementation of SysInfo. Note that this tests deep into the implementation, and not just the @@ -346,17 +361,16 @@ class SysInfoTest(unittest.TestCase): s = SysInfoFromFactory() self.assertEqual(NPROCESSORS_OPENBSD, s.get_num_processors()) - self.assertEqual('test.example.com', s.get_platform_hostname()) - self.assertFalse(s.get_platform_is_smp()) - self.assertLess(abs(76632 - s.get_uptime()), 4) + self.check_bsd_values(s) + self.assertEqual([0.7, 0.9, 0.8], s.get_loadavg()) + self.assertFalse(s.get_platform_is_smp()) self.assertEqual(543214321, s.get_mem_total()) self.assertEqual(543214321 - (121212 * 1024), s.get_mem_free()) - self.assertEqual(-1, s.get_mem_cached()) - self.assertEqual(-1, s.get_mem_buffers()) self.assertEqual(566791168, s.get_mem_swap_total()) self.assertEqual(566789120, s.get_mem_swap_free()) + # Try new regex assertion (which replaced the deprecated # assertRegexpMatches. If it is not available, use the old one try: @@ -364,14 +378,6 @@ class SysInfoTest(unittest.TestCase): except AttributeError: self.assertRegexpMatches(s.get_platform_distro(), '^OpenBSD\s+.*') - # These test that the corresponding tools are being called (and - # no further processing is done on this data). Please see the - # implementation functions at the top of this file. - self.assertEqual('qB2osV6vUOjqm3P/+tQ4d92xoYz8/U8P9v3KWRpNwlI=\n', s.get_net_interfaces()) - self.assertEqual('XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n', s.get_net_routing_table()) - self.assertEqual('osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n', s.get_net_stats()) - self.assertEqual('Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n', s.get_net_connections()) - def test_sysinfo_freebsd(self): """Tests the FreeBSD implementation of SysInfo. Note that this tests deep into the implementation, and not just the @@ -386,15 +392,13 @@ class SysInfoTest(unittest.TestCase): s = SysInfoFromFactory() self.assertEqual(NPROCESSORS_FREEBSD, s.get_num_processors()) - self.assertEqual('test.example.com', s.get_platform_hostname()) self.assertTrue(s.get_platform_is_smp()) - self.assertLess(abs(76632 - s.get_uptime()), 4) + self.check_bsd_values(s) + self.assertEqual([0.2, 0.4, 0.6], s.get_loadavg()) self.assertEqual(543214321, s.get_mem_total()) self.assertEqual(543214321 - (343434 * 1024), s.get_mem_free()) - self.assertEqual(-1, s.get_mem_cached()) - self.assertEqual(-1, s.get_mem_buffers()) self.assertEqual(1037533184, s.get_mem_swap_total()) self.assertEqual(1037533184, s.get_mem_swap_free()) # Try new regex assertion (which replaced the deprecated @@ -404,14 +408,6 @@ class SysInfoTest(unittest.TestCase): except AttributeError: self.assertRegexpMatches(s.get_platform_distro(), '^FreeBSD\s+.*') - # These test that the corresponding tools are being called (and - # no further processing is done on this data). Please see the - # implementation functions at the top of this file. - self.assertEqual('qB2osV6vUOjqm3P/+tQ4d92xoYz8/U8P9v3KWRpNwlI=\n', s.get_net_interfaces()) - self.assertEqual('XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n', s.get_net_routing_table()) - self.assertEqual('osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n', s.get_net_stats()) - self.assertEqual('Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n', s.get_net_connections()) - def test_sysinfo_osx(self): """Tests the OS X implementation of SysInfo. Note that this tests deep into the implementation, and not just the @@ -426,15 +422,13 @@ class SysInfoTest(unittest.TestCase): s = SysInfoFromFactory() self.assertEqual(NPROCESSORS_OSX, s.get_num_processors()) - self.assertEqual('test.example.com', s.get_platform_hostname()) self.assertFalse(s.get_platform_is_smp()) - self.assertLess(abs(76632 - s.get_uptime()), 4) + self.check_bsd_values(s) + self.assertEqual([0.2, 0.4, 0.6], s.get_loadavg()) self.assertEqual(123456789, s.get_mem_total()) self.assertEqual((23456 * 4096), s.get_mem_free()) - self.assertEqual(-1, s.get_mem_cached()) - self.assertEqual(-1, s.get_mem_buffers()) self.assertEqual(18874368.0, s.get_mem_swap_total()) self.assertEqual(1075988.48, s.get_mem_swap_free()) # Try new regex assertion (which replaced the deprecated @@ -444,13 +438,5 @@ class SysInfoTest(unittest.TestCase): except AttributeError: self.assertRegexpMatches(s.get_platform_distro(), '^Darwin\s+.*') - # These test that the corresponding tools are being called (and - # no further processing is done on this data). Please see the - # implementation functions at the top of this file. - self.assertEqual('qB2osV6vUOjqm3P/+tQ4d92xoYz8/U8P9v3KWRpNwlI=\n', s.get_net_interfaces()) - self.assertEqual('XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n', s.get_net_routing_table()) - self.assertEqual('osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n', s.get_net_stats()) - self.assertEqual('Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n', s.get_net_connections()) - if __name__ == "__main__": unittest.main() From 05cff66f9a287b2363ae0bc314024ab19f344e55 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Thu, 9 Aug 2012 15:22:00 +0200 Subject: [PATCH 47/69] [2172] use None instead of -1 for unknown values and update calling tool to not show anything for some values where it is None (i.e. where they have no relevance on the system) --- src/bin/sysinfo/sysinfo.py.in | 23 +++++++------ src/lib/python/isc/sysinfo/sysinfo.py | 25 +++++--------- .../python/isc/sysinfo/tests/sysinfo_test.py | 34 +++++++++---------- 3 files changed, 38 insertions(+), 44 deletions(-) diff --git a/src/bin/sysinfo/sysinfo.py.in b/src/bin/sysinfo/sysinfo.py.in index 1e77acb27e..9e2d9d03d8 100755 --- a/src/bin/sysinfo/sysinfo.py.in +++ b/src/bin/sysinfo/sysinfo.py.in @@ -57,8 +57,8 @@ def main(): s = SysInfoFromFactory() - f.write('BIND 10 ShowTech tool\n') - f.write('=====================\n') + f.write('ISC Sysinfo tool\n') + f.write('================\n') f.write('\nCPU\n'); f.write(' + Number of processors: %d\n' % (s.get_num_processors())) @@ -69,12 +69,13 @@ def main(): f.write(' + Distribution: %s\n' % (s.get_platform_distro())) f.write(' + Kernel version: %s\n' % (s.get_platform_version())) - f.write(' + SMP kernel: ') - if s.get_platform_is_smp(): - f.write('yes') - else: - f.write('no') - f.write('\n') + if s.get_platform_is_smp() is not None: + f.write(' + SMP kernel: ') + if s.get_platform_is_smp(): + f.write('yes') + else: + f.write('no') + f.write('\n') f.write(' + Machine name: %s\n' % (s.get_platform_machine())) f.write(' + Hostname: %s\n' % (s.get_platform_hostname())) @@ -86,8 +87,10 @@ def main(): f.write('\nMemory\n'); f.write(' + Total: %d bytes\n' % (s.get_mem_total())) f.write(' + Free: %d bytes\n' % (s.get_mem_free())) - f.write(' + Cached: %d bytes\n' % (s.get_mem_cached())) - f.write(' + Buffers: %d bytes\n' % (s.get_mem_buffers())) + if s.get_mem_cached() is not None: + f.write(' + Cached: %d bytes\n' % (s.get_mem_cached())) + if s.get_mem_buffers() is not None: + f.write(' + Buffers: %d bytes\n' % (s.get_mem_buffers())) f.write(' + Swap total: %d bytes\n' % (s.get_mem_swap_total())) f.write(' + Swap free: %d bytes\n' % (s.get_mem_swap_free())) diff --git a/src/lib/python/isc/sysinfo/sysinfo.py b/src/lib/python/isc/sysinfo/sysinfo.py index d79e77cd32..1ccf6c2a62 100644 --- a/src/lib/python/isc/sysinfo/sysinfo.py +++ b/src/lib/python/isc/sysinfo/sysinfo.py @@ -25,21 +25,21 @@ import time class SysInfo: def __init__(self): - self._num_processors = -1 + self._num_processors = None self._endianness = 'Unknown' self._hostname = '' self._platform_name = 'Unknown' self._platform_version = 'Unknown' self._platform_machine = 'Unknown' self._platform_is_smp = False - self._uptime = -1 + self._uptime = None self._loadavg = [-1.0, -1.0, -1.0] - self._mem_total = -1 - self._mem_free = -1 - self._mem_cached = -1 - self._mem_buffers = -1 - self._mem_swap_total = -1 - self._mem_swap_free = -1 + self._mem_total = None + self._mem_free = None + self._mem_cached = None + self._mem_buffers = None + self._mem_swap_total = None + self._mem_swap_free = None self._platform_distro = 'Unknown' self._net_interfaces = 'Unknown\n' self._net_routing_table = 'Unknown\n' @@ -303,11 +303,6 @@ class SysInfoOpenBSD(SysInfoBSD): def __init__(self): super().__init__() - # Don't know how to gather these - self._platform_is_smp = False - self._mem_cached = -1 - self._mem_buffers = -1 - try: s = subprocess.check_output(['sysctl', '-n', 'kern.boottime']) t = s.decode('utf-8').strip() @@ -356,10 +351,6 @@ class SysInfoFreeBSDOSX(SysInfoBSD): def __init__(self): super().__init__() - # Don't know how to gather these - self._mem_cached = -1 - self._mem_buffers = -1 - try: s = subprocess.check_output(['sysctl', '-n', 'kern.boottime']) t = s.decode('utf-8').strip() diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index f01319088f..209693216a 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -246,21 +246,21 @@ class SysInfoTest(unittest.TestCase): """Test that the various methods on SysInfo exist and return data.""" s = SysInfo() - self.assertEqual(-1, s.get_num_processors()) + self.assertEqual(None, s.get_num_processors()) self.assertEqual('Unknown', s.get_endianness()) self.assertEqual('', s.get_platform_hostname()) self.assertEqual('Unknown', s.get_platform_name()) self.assertEqual('Unknown', s.get_platform_version()) self.assertEqual('Unknown', s.get_platform_machine()) self.assertFalse(s.get_platform_is_smp()) - self.assertEqual(-1, s.get_uptime()) + self.assertEqual(None, s.get_uptime()) self.assertEqual([-1.0, -1.0, -1.0], s.get_loadavg()) - self.assertEqual(-1, s.get_mem_total()) - self.assertEqual(-1, s.get_mem_free()) - self.assertEqual(-1, s.get_mem_cached()) - self.assertEqual(-1, s.get_mem_buffers()) - self.assertEqual(-1, s.get_mem_swap_total()) - self.assertEqual(-1, s.get_mem_swap_free()) + self.assertEqual(None, s.get_mem_total()) + self.assertEqual(None, s.get_mem_free()) + self.assertEqual(None, s.get_mem_cached()) + self.assertEqual(None, s.get_mem_buffers()) + self.assertEqual(None, s.get_mem_swap_total()) + self.assertEqual(None, s.get_mem_swap_free()) self.assertEqual('Unknown', s.get_platform_distro()) self.assertEqual('Unknown\n', s.get_net_interfaces()) self.assertEqual('Unknown\n', s.get_net_routing_table()) @@ -275,7 +275,7 @@ class SysInfoTest(unittest.TestCase): platform.system = _my_testcase_platform_system s = SysInfoFromFactory() - self.assertEqual(-1, s.get_num_processors()) + self.assertEqual(None, s.get_num_processors()) self.assertEqual('bigrastafarian', s.get_endianness()) self.assertEqual('', s.get_platform_hostname()) self.assertEqual('b10test', s.get_platform_name()) @@ -284,12 +284,12 @@ class SysInfoTest(unittest.TestCase): self.assertFalse(s.get_platform_is_smp()) self.assertEqual(131072, s.get_uptime()) self.assertEqual([-1.0, -1.0, -1.0], s.get_loadavg()) - self.assertEqual(-1, s.get_mem_total()) - self.assertEqual(-1, s.get_mem_free()) - self.assertEqual(-1, s.get_mem_cached()) - self.assertEqual(-1, s.get_mem_buffers()) - self.assertEqual(-1, s.get_mem_swap_total()) - self.assertEqual(-1, s.get_mem_swap_free()) + self.assertEqual(None, s.get_mem_total()) + self.assertEqual(None, s.get_mem_free()) + self.assertEqual(None, s.get_mem_cached()) + self.assertEqual(None, s.get_mem_buffers()) + self.assertEqual(None, s.get_mem_swap_total()) + self.assertEqual(None, s.get_mem_swap_free()) self.assertEqual('Unknown', s.get_platform_distro()) self.assertEqual('Unknown\n', s.get_net_interfaces()) self.assertEqual('Unknown\n', s.get_net_routing_table()) @@ -336,8 +336,8 @@ class SysInfoTest(unittest.TestCase): # check values shared by all bsd implementations self.assertEqual('test.example.com', s.get_platform_hostname()) self.assertLess(abs(76632 - s.get_uptime()), 4) - self.assertEqual(-1, s.get_mem_cached()) - self.assertEqual(-1, s.get_mem_buffers()) + self.assertEqual(None, s.get_mem_cached()) + self.assertEqual(None, s.get_mem_buffers()) # These test that the corresponding tools are being called (and # no further processing is done on this data). Please see the From 7520a6838ab90cf969ad0656ecc702ec0ce3af13 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Thu, 9 Aug 2012 15:25:35 +0200 Subject: [PATCH 48/69] [2172] set some more members to None initially --- src/lib/python/isc/sysinfo/sysinfo.py | 4 ++-- src/lib/python/isc/sysinfo/tests/sysinfo_test.py | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/src/lib/python/isc/sysinfo/sysinfo.py b/src/lib/python/isc/sysinfo/sysinfo.py index 1ccf6c2a62..24b92237d3 100644 --- a/src/lib/python/isc/sysinfo/sysinfo.py +++ b/src/lib/python/isc/sysinfo/sysinfo.py @@ -31,9 +31,9 @@ class SysInfo: self._platform_name = 'Unknown' self._platform_version = 'Unknown' self._platform_machine = 'Unknown' - self._platform_is_smp = False + self._platform_is_smp = None self._uptime = None - self._loadavg = [-1.0, -1.0, -1.0] + self._loadavg = None self._mem_total = None self._mem_free = None self._mem_cached = None diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index 209693216a..47db5aefc0 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -254,7 +254,7 @@ class SysInfoTest(unittest.TestCase): self.assertEqual('Unknown', s.get_platform_machine()) self.assertFalse(s.get_platform_is_smp()) self.assertEqual(None, s.get_uptime()) - self.assertEqual([-1.0, -1.0, -1.0], s.get_loadavg()) + self.assertEqual(None, s.get_loadavg()) self.assertEqual(None, s.get_mem_total()) self.assertEqual(None, s.get_mem_free()) self.assertEqual(None, s.get_mem_cached()) @@ -283,7 +283,7 @@ class SysInfoTest(unittest.TestCase): self.assertEqual('Unknown', s.get_platform_machine()) self.assertFalse(s.get_platform_is_smp()) self.assertEqual(131072, s.get_uptime()) - self.assertEqual([-1.0, -1.0, -1.0], s.get_loadavg()) + self.assertEqual(None, s.get_loadavg()) self.assertEqual(None, s.get_mem_total()) self.assertEqual(None, s.get_mem_free()) self.assertEqual(None, s.get_mem_cached()) From 3fc727f1addbcbab236ffa9f27cab70427fa2205 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Thu, 9 Aug 2012 16:20:24 +0200 Subject: [PATCH 49/69] [2172] generalize only-write-value-if-known --- src/bin/sysinfo/sysinfo.py.in | 47 ++++++++++++++++++++--------------- 1 file changed, 27 insertions(+), 20 deletions(-) diff --git a/src/bin/sysinfo/sysinfo.py.in b/src/bin/sysinfo/sysinfo.py.in index 9e2d9d03d8..1de2a781a0 100755 --- a/src/bin/sysinfo/sysinfo.py.in +++ b/src/bin/sysinfo/sysinfo.py.in @@ -32,6 +32,15 @@ def usage(): file=sys.stderr) exit(1) +def write_value(out, fmt, call): + '''Helper function for standard value writing. + Writes the result from the call in the given format to out. + Does not write anything if the result of the call is None. + ''' + value = call() + if value is not None: + out.write(fmt % value) + def main(): try: opts, args = getopt.getopt(sys.argv[1:], "o:h", \ @@ -61,13 +70,13 @@ def main(): f.write('================\n') f.write('\nCPU\n'); - f.write(' + Number of processors: %d\n' % (s.get_num_processors())) - f.write(' + Endianness: %s\n' % (s.get_endianness())) + write_value(f, ' + Number of processors: %d\n', s.get_num_processors) + write_value(f, ' + Endianness: %s\n', s.get_endianness) f.write('\nPlatform\n'); - f.write(' + Operating system: %s\n' % (s.get_platform_name())) - f.write(' + Distribution: %s\n' % (s.get_platform_distro())) - f.write(' + Kernel version: %s\n' % (s.get_platform_version())) + write_value(f, ' + Operating system: %s\n', s.get_platform_name) + write_value(f, ' + Distribution: %s\n', s.get_platform_distro) + write_value(f, ' + Kernel version: %s\n', s.get_platform_version) if s.get_platform_is_smp() is not None: f.write(' + SMP kernel: ') @@ -77,22 +86,20 @@ def main(): f.write('no') f.write('\n') - f.write(' + Machine name: %s\n' % (s.get_platform_machine())) - f.write(' + Hostname: %s\n' % (s.get_platform_hostname())) - f.write(' + Uptime: %d seconds\n' % (s.get_uptime())) + write_value(f, ' + Machine name: %s\n', s.get_platform_machine) + write_value(f, ' + Hostname: %s\n', s.get_platform_hostname) + write_value(f, ' + Uptime: %d seconds\n', s.get_uptime) l = s.get_loadavg() f.write(' + Loadavg: %f %f %f\n' % (l[0], l[1], l[2])) f.write('\nMemory\n'); - f.write(' + Total: %d bytes\n' % (s.get_mem_total())) - f.write(' + Free: %d bytes\n' % (s.get_mem_free())) - if s.get_mem_cached() is not None: - f.write(' + Cached: %d bytes\n' % (s.get_mem_cached())) - if s.get_mem_buffers() is not None: - f.write(' + Buffers: %d bytes\n' % (s.get_mem_buffers())) - f.write(' + Swap total: %d bytes\n' % (s.get_mem_swap_total())) - f.write(' + Swap free: %d bytes\n' % (s.get_mem_swap_free())) + write_value(f, ' + Total: %d bytes\n', s.get_mem_total) + write_value(f, ' + Free: %d bytes\n', s.get_mem_free) + write_value(f, ' + Cached: %d bytes\n', s.get_mem_cached) + write_value(f, ' + Buffers: %d bytes\n', s.get_mem_buffers) + write_value(f, ' + Swap total: %d bytes\n', s.get_mem_swap_total) + write_value(f, ' + Swap free: %d bytes\n', s.get_mem_swap_free) f.write('\n\nNetwork\n'); f.write('-------\n\n'); @@ -100,19 +107,19 @@ def main(): f.write('Interfaces\n') f.write('~~~~~~~~~~\n\n') - f.write(s.get_net_interfaces()) + write_value(f, '%s', s.get_net_interfaces) f.write('\nRouting table\n') f.write('~~~~~~~~~~~~~\n\n') - f.write(s.get_net_routing_table()) + write_value(f, '%s', s.get_net_routing_table) f.write('\nStatistics\n') f.write('~~~~~~~~~~\n\n') - f.write(s.get_net_stats()) + write_value(f, '%s', s.get_net_stats) f.write('\nConnections\n') f.write('~~~~~~~~~~~\n\n') - f.write(s.get_net_connections()) + write_value(f, '%s', s.get_net_connections) try: if os.getuid() != 0: From 1bb96c51fe720e5ef8466aa0b2eb1619bd949204 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Thu, 9 Aug 2012 16:24:18 +0200 Subject: [PATCH 50/69] [2172] use 3-tuple instead of fixed-size list for loadavg --- src/bin/sysinfo/sysinfo.py.in | 3 +-- src/lib/python/isc/sysinfo/sysinfo.py | 6 +++--- 2 files changed, 4 insertions(+), 5 deletions(-) diff --git a/src/bin/sysinfo/sysinfo.py.in b/src/bin/sysinfo/sysinfo.py.in index 1de2a781a0..24cf309a31 100755 --- a/src/bin/sysinfo/sysinfo.py.in +++ b/src/bin/sysinfo/sysinfo.py.in @@ -90,8 +90,7 @@ def main(): write_value(f, ' + Hostname: %s\n', s.get_platform_hostname) write_value(f, ' + Uptime: %d seconds\n', s.get_uptime) - l = s.get_loadavg() - f.write(' + Loadavg: %f %f %f\n' % (l[0], l[1], l[2])) + write_value(f, ' + Loadavg: %f %f %f\n', s.get_loadavg) f.write('\nMemory\n'); write_value(f, ' + Total: %d bytes\n', s.get_mem_total) diff --git a/src/lib/python/isc/sysinfo/sysinfo.py b/src/lib/python/isc/sysinfo/sysinfo.py index 24b92237d3..c3238142a9 100644 --- a/src/lib/python/isc/sysinfo/sysinfo.py +++ b/src/lib/python/isc/sysinfo/sysinfo.py @@ -164,7 +164,7 @@ class SysInfoLinux(SysInfoPOSIX): with open('/proc/loadavg') as f: l = f.read().strip().split(' ') if len(l) >= 3: - self._loadavg = [float(l[0]), float(l[1]), float(l[2])] + self._loadavg = (float(l[0]), float(l[1]), float(l[2])) with open('/proc/meminfo') as f: m = f.readlines() @@ -315,7 +315,7 @@ class SysInfoOpenBSD(SysInfoBSD): s = subprocess.check_output(['sysctl', '-n', 'vm.loadavg']) l = s.decode('utf-8').strip().split(' ') if len(l) >= 3: - self._loadavg = [float(l[0]), float(l[1]), float(l[2])] + self._loadavg = (float(l[0]), float(l[1]), float(l[2])) except (subprocess.CalledProcessError, OSError): pass @@ -370,7 +370,7 @@ class SysInfoFreeBSDOSX(SysInfoBSD): else: la = l.split(' ') if len(la) >= 3: - self._loadavg = [float(la[0]), float(la[1]), float(la[2])] + self._loadavg = (float(la[0]), float(la[1]), float(la[2])) except (subprocess.CalledProcessError, OSError): pass From 0336d2de9661d70f20fd4a7ce88413f03441884d Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Thu, 9 Aug 2012 16:26:24 +0200 Subject: [PATCH 51/69] [2172] forgot to update test --- src/lib/python/isc/sysinfo/tests/sysinfo_test.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index 47db5aefc0..37c391e252 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -315,7 +315,7 @@ class SysInfoTest(unittest.TestCase): self.assertEqual('myhostname', s.get_platform_hostname()) self.assertTrue(s.get_platform_is_smp()) self.assertEqual(86401, s.get_uptime()) - self.assertEqual([0.1, 0.2, 0.3], s.get_loadavg()) + self.assertEqual((0.1, 0.2, 0.3), s.get_loadavg()) self.assertEqual(3157884928, s.get_mem_total()) self.assertEqual(891383808, s.get_mem_free()) self.assertEqual(1335152640, s.get_mem_cached()) @@ -364,7 +364,7 @@ class SysInfoTest(unittest.TestCase): self.check_bsd_values(s) - self.assertEqual([0.7, 0.9, 0.8], s.get_loadavg()) + self.assertEqual((0.7, 0.9, 0.8), s.get_loadavg()) self.assertFalse(s.get_platform_is_smp()) self.assertEqual(543214321, s.get_mem_total()) self.assertEqual(543214321 - (121212 * 1024), s.get_mem_free()) @@ -396,7 +396,7 @@ class SysInfoTest(unittest.TestCase): self.check_bsd_values(s) - self.assertEqual([0.2, 0.4, 0.6], s.get_loadavg()) + self.assertEqual((0.2, 0.4, 0.6), s.get_loadavg()) self.assertEqual(543214321, s.get_mem_total()) self.assertEqual(543214321 - (343434 * 1024), s.get_mem_free()) self.assertEqual(1037533184, s.get_mem_swap_total()) @@ -426,7 +426,7 @@ class SysInfoTest(unittest.TestCase): self.check_bsd_values(s) - self.assertEqual([0.2, 0.4, 0.6], s.get_loadavg()) + self.assertEqual((0.2, 0.4, 0.6), s.get_loadavg()) self.assertEqual(123456789, s.get_mem_total()) self.assertEqual((23456 * 4096), s.get_mem_free()) self.assertEqual(18874368.0, s.get_mem_swap_total()) From 97ad12066c71c308e0411b66a16e1ad272d5733f Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Thu, 9 Aug 2012 16:42:17 +0200 Subject: [PATCH 52/69] [2172] minor cleanup --- src/lib/python/isc/sysinfo/tests/sysinfo_test.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index 37c391e252..01a78d4ceb 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -143,7 +143,6 @@ def _my_bsd_subprocess_check_output(command): def _my_openbsd_subprocess_check_output(command): assert type(command) == list, 'command argument is not a list' - bsd_output = _my_bsd_subprocess_check_output(command) if command == ['sysctl', '-n', 'kern.boottime']: return bytes(str(int(time.time() - 76632)), 'utf-8') elif command == ['sysctl', '-n', 'vm.loadavg']: @@ -155,6 +154,7 @@ def _my_openbsd_subprocess_check_output(command): elif command == ['route', '-n', 'show']: return b'XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n' else: + bsd_output = _my_bsd_subprocess_check_output(command) if bsd_output is not None: return bsd_output else: From fa831b22d22077fb08e509f60cec2c33b9ff9214 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Fri, 10 Aug 2012 10:59:10 +0200 Subject: [PATCH 53/69] [2184] support add/remove for any type deriving 'actual' type from the data that has been set; no change for primitive type, treat as lists if list, treat as named_set if dict. Elements of named sets and lists are themselves considered to be of the any type --- src/lib/python/isc/config/ccsession.py | 49 +++++--- src/lib/python/isc/config/config_data.py | 5 + .../python/isc/config/tests/ccsession_test.py | 106 +++++++++++++++--- 3 files changed, 127 insertions(+), 33 deletions(-) diff --git a/src/lib/python/isc/config/ccsession.py b/src/lib/python/isc/config/ccsession.py index 703d1968eb..276a4e1ae0 100644 --- a/src/lib/python/isc/config/ccsession.py +++ b/src/lib/python/isc/config/ccsession.py @@ -144,7 +144,7 @@ class ModuleCCSession(ConfigData): module, and one to update the configuration run-time. These callbacks are called when 'check_command' is called on the ModuleCCSession""" - + def __init__(self, spec_file_name, config_handler, command_handler, cc_session=None, handle_logging_config=True, socket_file = None): @@ -178,9 +178,9 @@ class ModuleCCSession(ConfigData): """ module_spec = isc.config.module_spec_from_file(spec_file_name) ConfigData.__init__(self, module_spec) - + self._module_name = module_spec.get_module_name() - + self.set_config_handler(config_handler) self.set_command_handler(command_handler) @@ -248,7 +248,7 @@ class ModuleCCSession(ConfigData): returns nothing. It calls check_command_without_recvmsg() to parse the received message. - + If nonblock is True, it just checks if there's a command and does nothing if there isn't. If nonblock is False, it waits until it arrives. It temporarily sets timeout to infinity, @@ -265,7 +265,7 @@ class ModuleCCSession(ConfigData): """Parse the given message to see if there is a command or a configuration update. Calls the corresponding handler functions if present. Responds on the channel if the - handler returns a message.""" + handler returns a message.""" # should we default to an answer? success-by-default? unhandled error? if msg is not None and not 'result' in msg: answer = None @@ -314,7 +314,7 @@ class ModuleCCSession(ConfigData): answer = create_answer(1, str(exc)) if answer: self._session.group_reply(env, answer) - + def set_config_handler(self, config_handler): """Set the config handler for this module. The handler is a function that takes the full configuration and handles it. @@ -521,7 +521,7 @@ class UIModuleCCSession(MultiConfigData): if not cur_list: cur_list = [] - if value is None: + if value is None and "list_item_spec" in module_spec: if "item_default" in module_spec["list_item_spec"]: value = module_spec["list_item_spec"]["item_default"] @@ -572,8 +572,14 @@ class UIModuleCCSession(MultiConfigData): if module_spec is None: raise isc.cc.data.DataNotFoundError("Unknown item " + str(identifier)) + # for type any, we determine the 'type' by what value is set + # (which would be either list or dict) + cur_value, _ = self.get_value(identifier) + type_any = module_spec['item_type'] == 'any' + # the specified element must be a list or a named_set - if 'list_item_spec' in module_spec: + if 'list_item_spec' in module_spec or\ + (type_any and type(cur_value) == list): value = None # in lists, we might get the value with spaces, making it # the third argument. In that case we interpret both as @@ -583,11 +589,13 @@ class UIModuleCCSession(MultiConfigData): value_str += set_value_str value = isc.cc.data.parse_value_str(value_str) self._add_value_to_list(identifier, value, module_spec) - elif 'named_set_item_spec' in module_spec: + elif 'named_set_item_spec' in module_spec or\ + (type_any and type(cur_value) == dict): item_name = None item_value = None if value_str is not None: - item_name = isc.cc.data.parse_value_str(value_str) + #item_name = isc.cc.data.parse_value_str(value_str) + item_name = value_str if set_value_str is not None: item_value = isc.cc.data.parse_value_str(set_value_str) else: @@ -643,14 +651,25 @@ class UIModuleCCSession(MultiConfigData): if value_str is not None: value = isc.cc.data.parse_value_str(value_str) - if 'list_item_spec' in module_spec: - if value is not None: + # for type any, we determine the 'type' by what value is set + # (which would be either list or dict) + cur_value, _ = self.get_value(identifier) + type_any = module_spec['item_type'] == 'any' + + # there's two forms of 'remove from list'; the remove-value-from-list + # form, and the 'remove-by-index' form. We can recognize the second + # case by value is None + if 'list_item_spec' in module_spec or\ + (type_any and type(cur_value) == list) or\ + value is None: + if not type_any and value is not None: isc.config.config_data.check_type(module_spec['list_item_spec'], value) self._remove_value_from_list(identifier, value) - elif 'named_set_item_spec' in module_spec: - self._remove_value_from_named_set(identifier, value) + elif 'named_set_item_spec' in module_spec or\ + (type_any and type(cur_value) == dict): + self._remove_value_from_named_set(identifier, value_str) else: - raise isc.cc.data.DataNotFoundError(str(identifier) + " is not a list or a named_set") + raise isc.cc.data.DataNotFoundError(str(identifier) + " is not a list or a named_set " + str(module_spec) + " and type " + str(type(cur_value))) diff --git a/src/lib/python/isc/config/config_data.py b/src/lib/python/isc/config/config_data.py index 174e98c911..3a28c6b5d5 100644 --- a/src/lib/python/isc/config/config_data.py +++ b/src/lib/python/isc/config/config_data.py @@ -204,6 +204,9 @@ def find_spec_part(element, identifier, strict_identifier = True): # always want the 'full' spec of the item for id_part in id_parts[:-1]: cur_el = _find_spec_part_single(cur_el, id_part) + # As soon as we find 'any', return that + if cur_el["item_type"] == "any": + return cur_el if strict_identifier and spec_part_is_list(cur_el) and\ not isc.cc.data.identifier_has_list_index(id_part): raise isc.cc.data.DataNotFoundError(id_part + @@ -742,6 +745,8 @@ class MultiConfigData: # list cur_list = cur_value for list_index in list_indices: + if type(cur_list) != list: + raise isc.cc.data.DataTypeError(id + " is not a list") if list_index >= len(cur_list): raise isc.cc.data.DataNotFoundError("No item " + str(list_index) + " in " + id_part) diff --git a/src/lib/python/isc/config/tests/ccsession_test.py b/src/lib/python/isc/config/tests/ccsession_test.py index d1060bf008..c08edcc67f 100644 --- a/src/lib/python/isc/config/tests/ccsession_test.py +++ b/src/lib/python/isc/config/tests/ccsession_test.py @@ -33,7 +33,7 @@ class TestHelperFunctions(unittest.TestCase): self.assertRaises(ModuleCCSessionError, parse_answer, { 'result': [] }) self.assertRaises(ModuleCCSessionError, parse_answer, { 'result': [ 'not_an_rcode' ] }) self.assertRaises(ModuleCCSessionError, parse_answer, { 'result': [ 1, 2 ] }) - + rcode, val = parse_answer({ 'result': [ 0 ] }) self.assertEqual(0, rcode) self.assertEqual(None, val) @@ -107,7 +107,7 @@ class TestModuleCCSession(unittest.TestCase): def spec_file(self, file): return self.data_path + os.sep + file - + def create_session(self, spec_file_name, config_handler = None, command_handler = None, cc_session = None): return ModuleCCSession(self.spec_file(spec_file_name), @@ -335,7 +335,7 @@ class TestModuleCCSession(unittest.TestCase): self.assertEqual(len(fake_session.message_queue), 1) self.assertEqual({'result': [1, 'No config_data specification']}, fake_session.get_message('Spec1', None)) - + def test_check_command3(self): fake_session = FakeModuleCCSession() mccs = self.create_session("spec2.spec", None, None, fake_session) @@ -348,7 +348,7 @@ class TestModuleCCSession(unittest.TestCase): self.assertEqual(len(fake_session.message_queue), 1) self.assertEqual({'result': [0]}, fake_session.get_message('Spec2', None)) - + def test_check_command4(self): fake_session = FakeModuleCCSession() mccs = self.create_session("spec2.spec", None, None, fake_session) @@ -361,7 +361,7 @@ class TestModuleCCSession(unittest.TestCase): self.assertEqual(len(fake_session.message_queue), 1) self.assertEqual({'result': [1, 'aaa should be an integer']}, fake_session.get_message('Spec2', None)) - + def test_check_command5(self): fake_session = FakeModuleCCSession() mccs = self.create_session("spec2.spec", None, None, fake_session) @@ -374,7 +374,7 @@ class TestModuleCCSession(unittest.TestCase): self.assertEqual(len(fake_session.message_queue), 1) self.assertEqual({'result': [1, 'aaa should be an integer']}, fake_session.get_message('Spec2', None)) - + def test_check_command6(self): fake_session = FakeModuleCCSession() mccs = self.create_session("spec2.spec", None, None, fake_session) @@ -460,7 +460,7 @@ class TestModuleCCSession(unittest.TestCase): self.assertEqual(len(fake_session.message_queue), 1) self.assertEqual({'result': [1, 'No config_data specification']}, fake_session.get_message('Spec1', None)) - + def test_check_command_without_recvmsg2(self): "copied from test_check_command3" fake_session = FakeModuleCCSession() @@ -474,7 +474,7 @@ class TestModuleCCSession(unittest.TestCase): self.assertEqual(len(fake_session.message_queue), 1) self.assertEqual({'result': [0]}, fake_session.get_message('Spec2', None)) - + def test_check_command_without_recvmsg3(self): "copied from test_check_command7" fake_session = FakeModuleCCSession() @@ -487,7 +487,7 @@ class TestModuleCCSession(unittest.TestCase): mccs.check_command_without_recvmsg(cmd, env) self.assertEqual({'result': [0]}, fake_session.get_message('Spec2', None)) - + def test_check_command_block_timeout(self): """Check it works if session has timeout and it sets it back.""" def cmd_check(mccs, session): @@ -893,22 +893,22 @@ class fakeUIConn(): def set_get_answer(self, name, answer): self.get_answers[name] = answer - + def set_post_answer(self, name, answer): self.post_answers[name] = answer - + def send_GET(self, name, arg = None): if name in self.get_answers: return self.get_answers[name] else: return {} - + def send_POST(self, name, arg = None): if name in self.post_answers: return self.post_answers[name] else: return fakeAnswer() - + class TestUIModuleCCSession(unittest.TestCase): def setUp(self): @@ -919,9 +919,9 @@ class TestUIModuleCCSession(unittest.TestCase): def spec_file(self, file): return self.data_path + os.sep + file - - def create_uccs2(self, fake_conn): - module_spec = isc.config.module_spec_from_file(self.spec_file("spec2.spec")) + + def create_uccs(self, fake_conn, specfile="spec2.spec"): + module_spec = isc.config.module_spec_from_file(self.spec_file(specfile)) fake_conn.set_get_answer('/module_spec', { module_spec.get_module_name(): module_spec.get_full_spec()}) fake_conn.set_get_answer('/config_data', { 'version': BIND10_CONFIG_DATA_VERSION }) return UIModuleCCSession(fake_conn) @@ -989,7 +989,7 @@ class TestUIModuleCCSession(unittest.TestCase): def test_add_remove_value(self): fake_conn = fakeUIConn() - uccs = self.create_uccs2(fake_conn) + uccs = self.create_uccs(fake_conn) self.assertRaises(isc.cc.data.DataNotFoundError, uccs.add_value, 1, "a") self.assertRaises(isc.cc.data.DataNotFoundError, uccs.add_value, "no_such_item", "a") @@ -1020,6 +1020,76 @@ class TestUIModuleCCSession(unittest.TestCase): self.assertRaises(isc.cc.data.DataTypeError, uccs.remove_value, "Spec2/item5", None) + # Test adding and removing values for type = any + def test_add_remove_value_any(self): + fake_conn = fakeUIConn() + uccs = self.create_uccs(fake_conn, "spec40.spec") + + # Test item set of basic types + items = [ 1234, "foo", True, False ] + items_as_str = [ '1234', 'foo', 'true', 'false' ] + + def test_fails(): + self.assertRaises(isc.cc.data.DataNotFoundError, uccs.add_value, "Spec40/item1", "foo") + self.assertRaises(isc.cc.data.DataNotFoundError, uccs.add_value, "Spec40/item1", "foo", "bar") + self.assertRaises(isc.cc.data.DataNotFoundError, uccs.remove_value, "Spec40/item1", "foo") + self.assertRaises(isc.cc.data.DataTypeError, uccs.remove_value, "Spec40/item1[0]", None) + + # A few helper functions to perform a number of tests + # (to repeat the same test for nested data) + def check_list(identifier): + for item in items_as_str: + uccs.add_value(identifier, item) + self.assertEqual((items, 1), uccs.get_value(identifier)) + + # Removing from list should work in both ways + uccs.remove_value(identifier, "foo") + uccs.remove_value(identifier + "[1]", None) + self.assertEqual(([1234, False], 1), uccs.get_value(identifier)) + + # As should item indexing + self.assertEqual((1234, 1), uccs.get_value(identifier + "[0]")) + self.assertEqual((False, 1), uccs.get_value(identifier + "[1]")) + + def check_named_set(identifier): + for item in items_as_str: + # use string version as key as well + uccs.add_value(identifier, item, item) + + self.assertEqual((1234, 1), uccs.get_value(identifier + "/1234")) + self.assertEqual((True, 1), uccs.get_value(identifier + "/true")) + + for item in items_as_str: + # use string version as key as well + uccs.remove_value(identifier, item) + + + # should fail when set to value of primitive type + for item in items: + uccs.set_value("Spec40/item1", item) + test_fails() + + # When set to list, add and remove should work, and its elements + # should be considered of type 'any' themselves. + uccs.set_value("Spec40/item1", []) + check_list("Spec40/item1") + + # When set to dict, it should have the behaviour of a named set + uccs.set_value("Spec40/item1", {}) + check_named_set("Spec40/item1") + + # And, or course, we may need nesting. + uccs.set_value("Spec40/item1", { "foo": {}, "bar": [] }) + check_named_set("Spec40/item1/foo") + check_list("Spec40/item1/bar") + uccs.set_value("Spec40/item1", [ {}, [] ] ) + check_named_set("Spec40/item1[0]") + check_list("Spec40/item1[1]") + uccs.set_value("Spec40/item1", [[[[[[]]]]]] ) + check_list("Spec40/item1[0][0][0][0][0]") + uccs.set_value("Spec40/item1", { 'a': { 'a': { 'a': {} } } } ) + check_named_set("Spec40/item1/a/a/a") + def test_add_dup_value(self): fake_conn = fakeUIConn() uccs = self.create_uccs_listtest(fake_conn) @@ -1101,7 +1171,7 @@ class TestUIModuleCCSession(unittest.TestCase): def test_commit(self): fake_conn = fakeUIConn() - uccs = self.create_uccs2(fake_conn) + uccs = self.create_uccs(fake_conn) uccs.commit() uccs._local_changes = {'Spec2': {'item5': [ 'a' ]}} uccs.commit() From 7e93ff50b09f9bb0a121fd4336102f1875d89ac3 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Fri, 10 Aug 2012 14:12:14 +0200 Subject: [PATCH 54/69] [2184] Fix issue of null default value Found another issue which became more apparent now (but I think we've seen before); get_default_value() returns None if there is no default set, however, in some cases, None can be the actual default (datasources params for instance). This fix makes sure a default value of None is not confused with no value at all (which resulted in the error 'Error: data_sources/classes/IN[X]/params not found' --- src/lib/config/tests/testdata/spec40.spec | 9 +++++++++ src/lib/python/isc/config/config_data.py | 12 ++++++++++-- src/lib/python/isc/config/tests/ccsession_test.py | 12 ++++++++++++ 3 files changed, 31 insertions(+), 2 deletions(-) diff --git a/src/lib/config/tests/testdata/spec40.spec b/src/lib/config/tests/testdata/spec40.spec index f778fc09a2..6fbec10f3f 100644 --- a/src/lib/config/tests/testdata/spec40.spec +++ b/src/lib/config/tests/testdata/spec40.spec @@ -6,6 +6,15 @@ "item_type": "any", "item_optional": false, "item_default": "asdf" + }, + { "item_name": "item2", + "item_type": "any", + "item_optional": true + }, + { "item_name": "item3", + "item_type": "any", + "item_optional": true, + "item_default": null } ] } diff --git a/src/lib/python/isc/config/config_data.py b/src/lib/python/isc/config/config_data.py index 3a28c6b5d5..2bc6edbee3 100644 --- a/src/lib/python/isc/config/config_data.py +++ b/src/lib/python/isc/config/config_data.py @@ -556,7 +556,6 @@ class MultiConfigData: if 'item_default' in spec: # one special case, named_set if spec['item_type'] == 'named_set': - print("is " + id_part + " in named set?") return spec['item_default'] else: return spec['item_default'] @@ -585,6 +584,14 @@ class MultiConfigData: value = self.get_default_value(identifier) if value is not None: return value, self.DEFAULT + else: + # get_default_value returns None for both + # the cases where there is no default, and where + # it is set to null, so we need to catch the latter + spec_part = self.find_spec_part(identifier) + if spec_part and 'item_default' in spec_part and\ + spec_part['item_default'] is None: + return None, self.DEFAULT return None, self.NONE def _append_value_item(self, result, spec_part, identifier, all, first = False): @@ -650,7 +657,8 @@ class MultiConfigData: all) else: value, status = self.get_value(identifier) - if status == self.NONE and not spec_part['item_optional']: + if status == self.NONE and not spec_part['item_optional']:# and\ + #not ('item_default' in spec_part): raise isc.cc.data.DataNotFoundError(identifier + " not found") entry = _create_value_map_entry(identifier, diff --git a/src/lib/python/isc/config/tests/ccsession_test.py b/src/lib/python/isc/config/tests/ccsession_test.py index c08edcc67f..0101d50fb1 100644 --- a/src/lib/python/isc/config/tests/ccsession_test.py +++ b/src/lib/python/isc/config/tests/ccsession_test.py @@ -1020,6 +1020,18 @@ class TestUIModuleCCSession(unittest.TestCase): self.assertRaises(isc.cc.data.DataTypeError, uccs.remove_value, "Spec2/item5", None) + # Check that the difference between no default and default = null + # is recognized + def test_default_null(self): + fake_conn = fakeUIConn() + uccs = self.create_uccs(fake_conn, "spec40.spec") + (value, status) = uccs.get_value("/Spec40/item2") + self.assertIsNone(value) + self.assertEqual(uccs.NONE, status) + (value, status) = uccs.get_value("/Spec40/item3") + self.assertIsNone(value) + self.assertEqual(uccs.DEFAULT, status) + # Test adding and removing values for type = any def test_add_remove_value_any(self): fake_conn = fakeUIConn() From a5813028510249dcc4febe0bddf2e206f1c1039e Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Fri, 10 Aug 2012 14:37:40 +0200 Subject: [PATCH 55/69] [2184] update guide for params any type fix --- doc/guide/bind10-guide.html | 263 ++++++++------ doc/guide/bind10-guide.txt | 629 +++++++++++++++++---------------- doc/guide/bind10-guide.xml | 15 +- doc/guide/bind10-messages.html | 2 +- 4 files changed, 497 insertions(+), 412 deletions(-) diff --git a/doc/guide/bind10-guide.html b/doc/guide/bind10-guide.html index 5f7eeb9495..5c3b4c3df8 100644 --- a/doc/guide/bind10-guide.html +++ b/doc/guide/bind10-guide.html @@ -1,4 +1,4 @@ -BIND 10 Guide

BIND 10 Guide

Administrator Reference for BIND 10

This is the reference guide for BIND 10 version +BIND 10 Guide

BIND 10 Guide

Administrator Reference for BIND 10

This is the reference guide for BIND 10 version 20120712.

Copyright © 2010-2012 Internet Systems Consortium, Inc.

Abstract

BIND 10 is a framework that features Domain Name System (DNS) suite and Dynamic Host Configuration Protocol (DHCP) servers with development managed by Internet Systems Consortium (ISC). @@ -10,9 +10,9 @@ The most up-to-date version of this document (in PDF, HTML, and plain text formats), along with other documents for BIND 10, can be found at http://bind10.isc.org/docs. -

Table of Contents

Preface
1. Acknowledgements
1. Introduction
1.1. Supported Platforms
1.2. Required Software at Run-time
1.3. Starting and Stopping the Server
1.4. Managing BIND 10
2. Installation
2.1. Packages
2.2. Install Hierarchy
2.3. Building Requirements
2.4. Quick start
2.5. Installation from source
2.5.1. Download Tar File
2.5.2. Retrieve from Git
2.5.3. Configure before the build
2.5.4. Build
2.5.5. Install
3. Starting BIND10 with bind10
3.1. Starting BIND 10
3.2. Configuration to start processes
4. Command channel
5. Configuration manager
6. Remote control daemon
6.1. Configuration specification for b10-cmdctl
7. Control and configure user interface
8. Authoritative Server
8.1. Server Configurations
8.2. Data Source Backends
8.2.1. In-memory Data Source
8.2.2. In-memory Data Source with SQLite3 Backend
8.2.3. Reloading an In-memory Data Source
8.2.4. Disabling In-memory Data Sources
8.3. Loading Master Zones Files
9. Incoming Zone Transfers
9.1. Configuration for Incoming Zone Transfers
9.2. Enabling IXFR
9.3. Secondary Manager
9.4. Trigger an Incoming Zone Transfer Manually
9.5. Incoming Transfers with In-memory Datasource
10. Outbound Zone Transfers
11. Dynamic DNS Update
11.1. Enabling Dynamic Update
11.2. Access Control
11.3. Miscellaneous Operational Issues
12. Recursive Name Server
12.1. Access Control
12.2. Forwarding
13. DHCPv4 Server
13.1. DHCPv4 Server Usage
13.2. DHCPv4 Server Configuration
13.3. Supported standards
13.4. DHCPv4 Server Limitations
14. DHCPv6 Server
14.1. DHCPv6 Server Usage
14.2. DHCPv6 Server Configuration
14.3. Supported DHCPv6 Standards
14.4. DHCPv6 Server Limitations
15. libdhcp++ library
15.1. Interface detection
15.2. DHCPv4/DHCPv6 packet handling
16. Statistics
17. Logging
17.1. Logging configuration
17.1.1. Loggers
17.1.2. Output Options
17.1.3. Example session
17.2. Logging Message Format

List of Tables

3.1. Special startup components

Preface

Table of Contents

1. Acknowledgements

1. Acknowledgements

ISC would like to acknowledge generous support for +

Table of Contents

Preface
1. Acknowledgements
1. Introduction
1.1. Supported Platforms
1.2. Required Software at Run-time
1.3. Starting and Stopping the Server
1.4. Managing BIND 10
2. Installation
2.1. Packages
2.2. Install Hierarchy
2.3. Building Requirements
2.4. Quick start
2.5. Installation from source
2.5.1. Download Tar File
2.5.2. Retrieve from Git
2.5.3. Configure before the build
2.5.4. Build
2.5.5. Install
3. Starting BIND10 with bind10
3.1. Starting BIND 10
3.2. Configuration to start processes
4. Command channel
5. Configuration manager
6. Remote control daemon
6.1. Configuration specification for b10-cmdctl
7. Control and configure user interface
8. Authoritative Server
8.1. Server Configurations
8.2. Data Source Backends
8.2.1. Data source types
8.2.2. Examples
8.3. Loading Master Zones Files
9. Incoming Zone Transfers
9.1. Configuration for Incoming Zone Transfers
9.2. Enabling IXFR
9.3. Secondary Manager
9.4. Trigger an Incoming Zone Transfer Manually
9.5. Incoming Transfers with In-memory Datasource
10. Outbound Zone Transfers
11. Dynamic DNS Update
11.1. Enabling Dynamic Update
11.2. Access Control
11.3. Miscellaneous Operational Issues
12. Recursive Name Server
12.1. Access Control
12.2. Forwarding
13. DHCPv4 Server
13.1. DHCPv4 Server Usage
13.2. DHCPv4 Server Configuration
13.3. Supported standards
13.4. DHCPv4 Server Limitations
14. DHCPv6 Server
14.1. DHCPv6 Server Usage
14.2. DHCPv6 Server Configuration
14.3. Supported DHCPv6 Standards
14.4. DHCPv6 Server Limitations
15. libdhcp++ library
15.1. Interface detection
15.2. DHCPv4/DHCPv6 packet handling
16. Statistics
17. Logging
17.1. Logging configuration
17.1.1. Loggers
17.1.2. Output Options
17.1.3. Example session
17.2. Logging Message Format

List of Tables

3.1. Special startup components

Preface

Table of Contents

1. Acknowledgements

1. Acknowledgements

ISC would like to acknowledge generous support for BIND 10 development of DHCPv4 and DHCPv6 components provided - by Comcast.

Chapter 1. Introduction

Table of Contents

1.1. Supported Platforms
1.2. Required Software at Run-time
1.3. Starting and Stopping the Server
1.4. Managing BIND 10

+ by Comcast.

Chapter 1. Introduction

Table of Contents

1.1. Supported Platforms
1.2. Required Software at Run-time
1.3. Starting and Stopping the Server
1.4. Managing BIND 10

BIND is the popular implementation of a DNS server, developer interfaces, and DNS tools. BIND 10 is a rewrite of BIND 9 and ISC DHCP. @@ -25,7 +25,7 @@

This guide covers the experimental prototype of BIND 10 version 20120712. -

1.1. Supported Platforms

+

1.1. Supported Platforms

BIND 10 builds have been tested on (in no particular order) Debian GNU/Linux 6 and unstable, Ubuntu 9.10, NetBSD 5, Solaris 10 and 11, FreeBSD 7 and 8, CentOS Linux 5.3, @@ -174,7 +174,7 @@ documentation and code examples. -

Chapter 2. Installation

Table of Contents

2.1. Packages
2.2. Install Hierarchy
2.3. Building Requirements
2.4. Quick start
2.5. Installation from source
2.5.1. Download Tar File
2.5.2. Retrieve from Git
2.5.3. Configure before the build
2.5.4. Build
2.5.5. Install

2.1. Packages

+

Chapter 2. Installation

Table of Contents

2.1. Packages
2.2. Install Hierarchy
2.3. Building Requirements
2.4. Quick start
2.5. Installation from source
2.5.1. Download Tar File
2.5.2. Retrieve from Git
2.5.3. Configure before the build
2.5.4. Build
2.5.5. Install

2.1. Packages

Some operating systems or softare package vendors may provide ready-to-use, pre-built software packages for the BIND 10 suite. @@ -283,14 +283,14 @@ downloadable tar file or via BIND 10's Git code revision control service. (It may also be available in pre-compiled ready-to-use packages from operating system vendors.) -

2.5.1. Download Tar File

+

2.5.1. Download Tar File

Downloading a release tar file is the recommended method to obtain the source code.

The BIND 10 releases are available as tar file downloads from ftp://ftp.isc.org/isc/bind10/. Periodic development snapshots may also be available. -

2.5.2. Retrieve from Git

+

2.5.2. Retrieve from Git

Downloading this "bleeding edge" code is recommended only for developers or advanced users. Using development code in a production environment is not recommended. @@ -325,7 +325,7 @@ autoheader, automake, and related commands. -

2.5.3. Configure before the build

+

2.5.3. Configure before the build

BIND 10 uses the GNU Build System to discover build environment details. To generate the makefiles using the defaults, simply run: @@ -356,12 +356,12 @@

If the configure fails, it may be due to missing or old dependencies. -

2.5.4. Build

+

2.5.4. Build

After the configure step is complete, to build the executables from the C++ code and prepare the Python scripts, run: 

$ make

-

2.5.5. Install

+

2.5.5. Install

To install the BIND 10 executables, support files, and documentation, run: 

$ make install

@@ -432,7 +432,7 @@ started in a special way, with the value of special used for them: -

Table 3.1. Special startup components

ComponentSpecialDescription
b10-authauthAuthoritative DNS server
b10-resolverresolverDNS resolver
b10-cmdctlcmdctlCommand control (remote control interface)


+

Table 3.1. Special startup components

ComponentSpecialDescription
b10-authauthAuthoritative DNS server
b10-resolverresolverDNS resolver
b10-cmdctlcmdctlCommand control (remote control interface)


The kind specifies how a failure of the component should be handled. If it is set to @@ -661,13 +661,13 @@ the details and relays (over a b10-msgq command channel) the configuration on to the specified module.

-

Chapter 8. Authoritative Server

Table of Contents

8.1. Server Configurations
8.2. Data Source Backends
8.2.1. In-memory Data Source
8.2.2. In-memory Data Source with SQLite3 Backend
8.2.3. Reloading an In-memory Data Source
8.2.4. Disabling In-memory Data Sources
8.3. Loading Master Zones Files

+

Chapter 8. Authoritative Server

Table of Contents

8.1. Server Configurations
8.2. Data Source Backends
8.2.1. Data source types
8.2.2. Examples
8.3. Loading Master Zones Files

The b10-auth is the authoritative DNS server. It supports EDNS0, DNSSEC, IPv6, and SQLite3 and in-memory zone data backends. Normally it is started by the bind10 master process. -

8.1. Server Configurations

+

8.1. Server Configurations

b10-auth is configured via the b10-cfgmgr configuration manager. The module name is “Auth”. @@ -766,86 +766,127 @@ can use various data source backends. if configured.)

-

8.2. Data Source Backends

Note

- For the development prototype release, b10-auth - supports a SQLite3 data source backend and in-memory data source - backend. +

8.2. Data Source Backends

+ Bind 10 has the concept of data sources. A data source is a place + where authoritative zone data reside and where they can be served + from. This can be a master file, a database or something completely + different. +

+ Once a query arrives, b10-auth goes through a + configured list of data sources and finds the one containing a best + matching zone. From the equally good ones, the first one is taken. + This data source is then used to answer the query. +

Note

+ In the development prototype release, b10-auth + can serve data from a SQLite3 data source backend and from master + files. Upcoming versions will be able to use multiple different data sources, such as MySQL and Berkeley DB.

- By default, the SQLite3 backend uses the data file located at + The configuration is located in data_sources/classes. Each item there + represents one RR class and a list used to answer queries for that + class. The default contains two classes. The CH class contains a static + data source — one that serves things like + “AUTHORS.BIND.”. The IN class contains single SQLite3 + data source with database file located at /usr/local/var/bind10-devel/zone.sqlite3. - (The full path is what was defined at build configure time for - --localstatedir. - The default is /usr/local/var/.) - This data file location may be changed by defining the - “database_file” configuration. -

8.2.1. In-memory Data Source

+

+ Each data source has several options. The first one is + type, which specifies the type of data source to + use. Valid types include the ones listed below, but bind10 uses + dynamically loaded modules for them, so there may be more in your + case. This option is mandatory. +

+ Another option is params. This option is type + specific; it holds different data depending on the type + above. Also, depending on the type, it could be possible to omit it. +

+ There are two options related to the so-called cache. If you enable + cache, zone data from the data source are loaded into memory. + Then, when answering a query, b10-auth looks + into the memory only instead of the data source, which speeds + answering up. The first option is cache-enable, + a boolean value turning the cache on and off (off is the default). + The second one, cache-zones, is a list of zone + origins to load into in-memory. Remember that zones in the data source + not listed here will not be loaded and will not be available at all. +

8.2.1. Data source types

+ As mentioned, the type used by default is “sqlite3”. + It has single configuration option inside params + — database_file, which contains the path + to the sqlite3 file containing the data. +

+ Another type is called “MasterFiles”. This one is + slightly special. The data are stored in RFC1034 master files. + Because answering directly from them would be impractical, + this type mandates the cache to be enabled. Also, the list of + zones (cache-zones) should be omitted. The + params is a dictionary mapping from zone + origins to the files they reside in. +

8.2.2. Examples

+ As this is one of the more complex configurations of Bind10, + we show some examples. They all assume they start with default + configuration. +

+ First, let's disable the static data source + (“VERSION.BIND” and friends). As it is the only + data source in the CH class, we can remove the whole class. - The following commands to bindctl - provide an example of configuring an in-memory data - source containing the “example.com” zone - with the zone file named “example.com.zone”: + 

> config remove data_sources/classes CH
+> config commit

+

+ Another one, let's say our default data source contains zones + “example.org.” and “example.net.”. + We want them to be served from memory to make the answering + faster. - - - 

> config add Auth/datasources
-> config set Auth/datasources[0]/type "memory"
-> config add Auth/datasources[0]/zones
-> config set Auth/datasources[0]/zones[0]/origin "example.com"
-> config set Auth/datasources[0]/zones[0]/file "example.com.zone"
+          
> config set data_sources/classes/IN[0]/cache-enable true
+> config add data_sources/classes/IN[0]/cache-zones example.org.
+> config add data_sources/classes/IN[0]/cache-zones example.net.
 > config commit

 
-	  The authoritative server will begin serving it immediately
-	  after the zone data is loaded from the master text file.
-

8.2.2. In-memory Data Source with SQLite3 Backend

+ Now every time the zone in the data source is changed by the + operator, Bind10 needs to be told to reload it, by + 

> Auth loadzone example.org

+ You don't need to do this when the zone is modified by + XfrIn, it does so automatically. +

+ Now, the last example is when there are master files we want to + serve in addition to whatever is inside the sqlite3 database. - The following commands to bindctl - provide an example of configuring an in-memory data - source containing the “example.org” zone - with a SQLite3 backend file named “example.org.sqlite3”: - - - - 

> config add Auth/datasources
-> config set Auth/datasources[1]/type "memory"
-> config add Auth/datasources[1]/zones
-> config set Auth/datasources[1]/zones[0]/origin "example.org"
-> config set Auth/datasources[1]/zones[0]/file "example.org.sqlite3"
-> config set Auth/datasources[1]/zones[0]/filetype "sqlite3"
+          
> config add data_sources/classes/IN
+> config set data_sources/classes/IN[1]/type MasterFiles
+> config set data_sources/classes/IN[1]/cache-enable true
+> config set data_sources/classes/IN[1]/params { "example.org": "/path/to/example.org", "example.com": "/path/to/example.com" }
 > config commit

 
-	  The authoritative server will begin serving it immediately
-	  after the zone data is loaded from the database file.
-

8.2.3. Reloading an In-memory Data Source

- Use the Auth loadzone command in - bindctl to reload a changed master - file into memory; for example: + Initially, a map value has to be set, but this value may be an + empty map. After that, key/value pairs can be added with 'config + add' and keys can be removed with 'config remove'. The initial + value may be an empty map, but it has to be set before zones are + added or removed. - 

> Auth loadzone origin="example.com"
-

+ 

+> config set data_sources/classes/IN[1]/params {}
+> config add data_sources/classes/IN[1]/params another.example.org /path/to/another.example.org
+> config add data_sources/classes/IN[1]/params another.example.com /path/to/another.example.com
+> config remove data_sources/classes/IN[1]/params another.example.org
+

-

8.2.4. Disabling In-memory Data Sources

- By default, the memory data source is disabled; it must be - configured explicitly. To disable all the in-memory zones, - specify a null list for Auth/datasources: - - - - 

> config set Auth/datasources/ []
-> config commit

-

- The following example stops serving a specific zone: - - 

> config remove Auth/datasources[0]/zones[0]
-> config commit

- - (Replace the list number(s) in - datasources[0] - and/or zones[0] - for the relevant zone as needed.) - -

8.3. Loading Master Zones Files

+ bindctl. To reload a zone, you the same command + as above. +

Note

+ There's also Auth/database_file configuration + variable, pointing to a sqlite3 database file. This is no longer + used by b10-auth, but it is left in place for + now, since other modules use it. Once b10-xfrin, + b10-xfrout and b10-ddns + are ported to the new configuration, this will disappear. But for + now, make sure that if you use any of these modules, the new + and old configuration correspond. The defaults are consistent, so + unless you tweaked either the new or the old configuration, you're + good. +

8.3. Loading Master Zones Files

RFC 1035 style DNS master zone files may imported into a BIND 10 SQLite3 data source by using the b10-loadzone utility. @@ -874,7 +915,7 @@ can use various data source backends. If you reload a zone already existing in the database, all records from that prior zone disappear and a whole new set appears. -

Chapter 9. Incoming Zone Transfers

Table of Contents

9.1. Configuration for Incoming Zone Transfers
9.2. Enabling IXFR
9.3. Secondary Manager
9.4. Trigger an Incoming Zone Transfer Manually
9.5. Incoming Transfers with In-memory Datasource

+

Chapter 9. Incoming Zone Transfers

Table of Contents

9.1. Configuration for Incoming Zone Transfers
9.2. Enabling IXFR
9.3. Secondary Manager
9.4. Trigger an Incoming Zone Transfer Manually
9.5. Incoming Transfers with In-memory Datasource

Incoming zones are transferred using the b10-xfrin process which is started by bind10. When received, the zone is stored in the corresponding BIND 10 @@ -888,7 +929,7 @@ can use various data source backends. IXFR. Due to some implementation limitations of the current development release, however, it only tries AXFR by default, and care should be taken to enable IXFR. -

9.1. Configuration for Incoming Zone Transfers

+

9.1. Configuration for Incoming Zone Transfers

In practice, you need to specify a list of secondary zones to enable incoming zone transfers for these zones (you can still trigger a zone transfer manually, without a prior configuration @@ -904,7 +945,7 @@ can use various data source backends. > config commit

(We assume there has been no zone configuration before). -

9.2. Enabling IXFR

+

9.2. Enabling IXFR

As noted above, b10-xfrin uses AXFR for zone transfers by default. To enable IXFR for zone transfers for a particular zone, set the use_ixfr @@ -953,13 +994,13 @@ can use various data source backends. (i.e. no SOA record for it), b10-zonemgr will automatically tell b10-xfrin to transfer the zone in. -

9.4. Trigger an Incoming Zone Transfer Manually

+

9.4. Trigger an Incoming Zone Transfer Manually

To manually trigger a zone transfer to retrieve a remote zone, you may use the bindctl utility. For example, at the bindctl prompt run: 

> Xfrin retransfer zone_name="foo.example.org" master=192.0.2.99

-

9.5. Incoming Transfers with In-memory Datasource

+

9.5. Incoming Transfers with In-memory Datasource

In the case of an incoming zone transfer, the received zone is first stored in the corresponding BIND 10 datasource. In case the secondary zone is served by an in-memory datasource @@ -970,7 +1011,7 @@ can use various data source backends. The administrator doesn't have to do anything for b10-auth to serve the new version of the zone, except for the configuration such as the one described in - Section 8.2.2, “In-memory Data Source with SQLite3 Backend”. + Section 8.2, “Data Source Backends”.

Chapter 10. Outbound Zone Transfers

The b10-xfrout process is started by bind10. @@ -1015,7 +1056,7 @@ Xfrout/transfer_acl[0] {"action": "ACCEPT"} any (default)

TSIGs in the incoming messages and to sign responses.

Note

The way to specify zone specific configuration (ACLs, etc) is likely to be changed. -

Chapter 11. Dynamic DNS Update

Table of Contents

11.1. Enabling Dynamic Update
11.2. Access Control
11.3. Miscellaneous Operational Issues

+

Chapter 11. Dynamic DNS Update

Table of Contents

11.1. Enabling Dynamic Update
11.2. Access Control
11.3. Miscellaneous Operational Issues

BIND 10 supports the server side of the Dynamic DNS Update (DDNS) protocol as defined in RFC 2136. This service is provided by the b10-ddns @@ -1034,11 +1075,11 @@ Xfrout/transfer_acl[0] {"action": "ACCEPT"} any (default)

notify b10-xfrout so that other secondary servers will be notified via the DNS NOTIFY protocol. In addition, if b10-auth serves the updated - zone from its in-memory cache (as described in - Section 8.2.2, “In-memory Data Source with SQLite3 Backend”), + zone (as described in + Section 8.2, “Data Source Backends”), b10-ddns will also notify b10-auth so that b10-auth - will re-cache the updated zone content. + will re-cache the updated zone content if necessary.

The b10-ddns component supports requests over both UDP and TCP, and both IPv6 and IPv4; for TCP requests, @@ -1062,7 +1103,7 @@ Xfrout/transfer_acl[0] {"action": "ACCEPT"} any (default)

this feature.

-

11.1. Enabling Dynamic Update

+

11.1. Enabling Dynamic Update

First off, it must be made sure that a few components on which b10-ddns depends are configured to run, which are b10-auth @@ -1123,7 +1164,7 @@ Xfrout/transfer_acl[0] {"action": "ACCEPT"} any (default)

to shutdown gracefully this parameter should also be specified.

-

11.2. Access Control

+

11.2. Access Control

By default, b10-ddns rejects any update requests from any clients by returning a REFUSED response. To allow updates to take effect, an access control rule @@ -1221,7 +1262,7 @@ DDNS/zones[0]/update_acl[1] {"action": "ACCEPT", "from": "::1", "key": "key. arbitrary clients. There have been other troubles that could have been avoided if the ACL could be checked before the prerequisite check. -

11.3. Miscellaneous Operational Issues

+

11.3. Miscellaneous Operational Issues

Unlike BIND 9, BIND 10 currently does not support automatic re-signing of DNSSEC-signed zone when it's updated via DDNS. It could be possible to re-sign the updated zone afterwards @@ -1263,7 +1304,7 @@ DDNS/zones[0]/update_acl[1] {"action": "ACCEPT", "from": "::1", "key": "key. IXFR. This is done automatically; it does not require specific configuration to make this possible. -

Chapter 12. Recursive Name Server

Table of Contents

12.1. Access Control
12.2. Forwarding

+

Chapter 12. Recursive Name Server

Table of Contents

12.1. Access Control
12.2. Forwarding

The b10-resolver process is started by bind10. @@ -1297,7 +1338,7 @@ DDNS/zones[0]/update_acl[1] {"action": "ACCEPT", "from": "::1", "key": "key.

(Replace the “2” as needed; run “config show - Resolver/listen_on” if needed.)

12.1. Access Control

+ Resolver/listen_on” if needed.)

12.1. Access Control

By default, the b10-resolver daemon only accepts DNS queries from the localhost (127.0.0.1 and ::1). The Resolver/query_acl configuration may @@ -1330,7 +1371,7 @@ DDNS/zones[0]/update_acl[1] {"action": "ACCEPT", "from": "::1", "key": "key.

(Replace the “2” as needed; run “config show Resolver/query_acl” if needed.)

Note

This prototype access control configuration - syntax may be changed.

12.2. Forwarding

+ syntax may be changed.

12.2. Forwarding

To enable forwarding, the upstream address and port must be configured to forward queries to, such as: @@ -1621,7 +1662,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

} }

-

Chapter 17. Logging

Table of Contents

17.1. Logging configuration
17.1.1. Loggers
17.1.2. Output Options
17.1.3. Example session
17.2. Logging Message Format

17.1. Logging configuration

+

Chapter 17. Logging

Table of Contents

17.1. Logging configuration
17.1.1. Loggers
17.1.2. Output Options
17.1.3. Example session
17.2. Logging Message Format

17.1. Logging configuration

The logging system in BIND 10 is configured through the Logging module. All BIND 10 modules will look at the @@ -1630,7 +1671,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

-

17.1.1. Loggers

+

17.1.1. Loggers

Within BIND 10, a message is logged through a component called a "logger". Different parts of BIND 10 log messages @@ -1651,7 +1692,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

(what to log), and the output_options (where to log). -

17.1.1.1. name (string)

+

17.1.1.1. name (string)

Each logger in the system has a name, the name being that of the component using it to log messages. For instance, if you want to configure logging for the resolver module, @@ -1724,7 +1765,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

“Auth.cache” logger will appear in the output with a logger name of “b10-auth.cache”). -

17.1.1.2. severity (string)

+

17.1.1.2. severity (string)

This specifies the category of messages logged. Each message is logged with an associated severity which @@ -1740,7 +1781,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

-

17.1.1.3. output_options (list)

+

17.1.1.3. output_options (list)

Each logger can have zero or more output_options. These specify where log @@ -1750,7 +1791,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

The other options for a logger are: -

17.1.1.4. debuglevel (integer)

+

17.1.1.4. debuglevel (integer)

When a logger's severity is set to DEBUG, this value specifies what debug messages should be printed. It ranges @@ -1759,7 +1800,7 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

If severity for the logger is not DEBUG, this value is ignored. -

17.1.1.5. additive (true or false)

+

17.1.1.5. additive (true or false)

If this is true, the output_options from the parent will be used. For example, if there are two @@ -1773,18 +1814,18 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

-

17.1.2. Output Options

+

17.1.2. Output Options

The main settings for an output option are the destination and a value called output, the meaning of which depends on the destination that is set. -

17.1.2.1. destination (string)

+

17.1.2.1. destination (string)

The destination is the type of output. It can be one of: -

  • console
  • file
  • syslog

17.1.2.2. output (string)

+

  • console
  • file
  • syslog

17.1.2.2. output (string)

Depending on what is set as the output destination, this value is interpreted as follows: @@ -1814,12 +1855,12 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

The other options for output_options are: -

17.1.2.2.1. flush (true of false)

+

17.1.2.2.1. flush (true of false)

Flush buffers after each log message. Doing this will reduce performance but will ensure that if the program terminates abnormally, all messages up to the point of termination are output. -

17.1.2.2.2. maxsize (integer)

+

17.1.2.2.2. maxsize (integer)

Only relevant when destination is file, this is maximum file size of output files in bytes. When the maximum size is reached, the file is renamed and a new file opened. @@ -1828,11 +1869,11 @@ const std::string HARDCODED_DNS_SERVER = "2001:db8:1::1";

etc.)

If this is 0, no maximum file size is used. -

17.1.2.2.3. maxver (integer)

+

17.1.2.2.3. maxver (integer)

Maximum number of old log files to keep around when rolling the output file. Only relevant when destination is “file”. -

17.1.3. Example session

+

17.1.3. Example session

In this example we want to set the global logging to write to the file /var/log/my_bind10.log, @@ -1993,7 +2034,7 @@ Logging/loggers[0]/output_options[0]/maxver 8 integer (modified) And every module will now be using the values from the logger named “*”. -

17.2. Logging Message Format

+

17.2. Logging Message Format

Each message written by BIND 10 to the configured logging destinations comprises a number of components that identify the origin of the message and, if the message indicates diff --git a/doc/guide/bind10-guide.txt b/doc/guide/bind10-guide.txt index 02cb343d37..b6d7f8fbad 100644 --- a/doc/guide/bind10-guide.txt +++ b/doc/guide/bind10-guide.txt @@ -4,7 +4,7 @@ Administrator Reference for BIND 10 This is the reference guide for BIND 10 version 20120712. - Copyright (c) 2010-2012 Internet Systems Consortium, Inc. + Copyright © 2010-2012 Internet Systems Consortium, Inc. Abstract @@ -81,14 +81,9 @@ Administrator Reference for BIND 10 8.2. Data Source Backends - 8.2.1. In-memory Data Source + 8.2.1. Data source types - 8.2.2. In-memory Data Source with SQLite3 - Backend - - 8.2.3. Reloading an In-memory Data Source - - 8.2.4. Disabling In-memory Data Sources + 8.2.2. Examples 8.3. Loading Master Zones Files @@ -170,12 +165,12 @@ Preface 1. Acknowledgements -1. Acknowledgements +1. Acknowledgements ISC would like to acknowledge generous support for BIND 10 development of DHCPv4 and DHCPv6 components provided by Comcast. -Chapter 1. Introduction +Chapter 1. Introduction Table of Contents @@ -197,7 +192,7 @@ Chapter 1. Introduction This guide covers the experimental prototype of BIND 10 version 20120712. -1.1. Supported Platforms +1.1. Supported Platforms BIND 10 builds have been tested on (in no particular order) Debian GNU/Linux 6 and unstable, Ubuntu 9.10, NetBSD 5, Solaris 10 and 11, @@ -206,13 +201,13 @@ Chapter 1. Introduction planned for BIND 10 to build, install and run on Windows and standard Unix-type platforms. -1.2. Required Software at Run-time +1.2. Required Software at Run-time Running BIND 10 uses various extra software which may not be provided in some operating systems' default installations nor standard packages collections. You may need to install this required software separately. - (For the build requirements, also see Section 2.3, "Building - Requirements".) + (For the build requirements, also see Section 2.3, “Building + Requirements”.) BIND 10 requires at least Python 3.1 (http://www.python.org/). It also works with Python 3.2. @@ -232,7 +227,7 @@ Chapter 1. Introduction included with Python). Python modules need to be built for the corresponding Python 3. -1.3. Starting and Stopping the Server +1.3. Starting and Stopping the Server BIND 10 is modular. Part of this modularity is accomplished using multiple cooperating processes which, together, provide the server functionality. @@ -245,50 +240,49 @@ Chapter 1. Introduction processes as needed. The processes started by the bind10 command have names starting with "b10-", including: - o b10-auth -- Authoritative DNS server. This process serves DNS - requests. - o b10-cfgmgr -- Configuration manager. This process maintains all of the + o b10-auth — Authoritative DNS server. This process serves DNS requests. + o b10-cfgmgr — Configuration manager. This process maintains all of the configuration for BIND 10. - o b10-cmdctl -- Command and control service. This process allows - external control of the BIND 10 system. - o b10-ddns -- Dynamic DNS update service. This process is used to handle + o b10-cmdctl — Command and control service. This process allows external + control of the BIND 10 system. + o b10-ddns — Dynamic DNS update service. This process is used to handle incoming DNS update requests to allow granted clients to update zones for which BIND 10 is serving as a primary server. - o b10-msgq -- Message bus daemon. This process coordinates communication + o b10-msgq — Message bus daemon. This process coordinates communication between all of the other BIND 10 processes. - o b10-resolver -- Recursive name server. This process handles incoming + o b10-resolver — Recursive name server. This process handles incoming DNS queries and provides answers from its cache or by recursively doing remote lookups. - o b10-sockcreator -- Socket creator daemon. This process creates sockets + o b10-sockcreator — Socket creator daemon. This process creates sockets used by network-listening BIND 10 processes. - o b10-stats -- Statistics collection daemon. This process collects and + o b10-stats — Statistics collection daemon. This process collects and reports statistics data. - o b10-stats-httpd -- HTTP server for statistics reporting. This process + o b10-stats-httpd — HTTP server for statistics reporting. This process reports statistics data in XML format over HTTP. - o b10-xfrin -- Incoming zone transfer service. This process is used to + o b10-xfrin — Incoming zone transfer service. This process is used to transfer a new copy of a zone into BIND 10, when acting as a secondary server. - o b10-xfrout -- Outgoing zone transfer service. This process is used to + o b10-xfrout — Outgoing zone transfer service. This process is used to handle transfer requests to send a local zone to a remote secondary server. - o b10-zonemgr -- Secondary zone manager. This process keeps track of + o b10-zonemgr — Secondary zone manager. This process keeps track of timers and other necessary information for BIND 10 to act as a slave server. These are ran by bind10 and do not need to be manually started independently. -1.4. Managing BIND 10 +1.4. Managing BIND 10 Once BIND 10 is running, a few commands are used to interact directly with the system: - o bindctl -- Interactive administration interface. This is a low-level + o bindctl — Interactive administration interface. This is a low-level command-line tool which allows a developer or an experienced administrator to control BIND 10. - o b10-loadzone -- Zone file loader. This tool will load standard + o b10-loadzone — Zone file loader. This tool will load standard masterfile-format zone files into BIND 10. - o b10-cmdctl-usermgr -- User access control. This tool allows an + o b10-cmdctl-usermgr — User access control. This tool allows an administrator to authorize additional users to manage BIND 10. The tools and modules are covered in full detail in this guide. In @@ -298,7 +292,7 @@ Chapter 1. Introduction Python for the message bus, configuration backend, and, of course, DNS. These include detailed developer documentation and code examples. -Chapter 2. Installation +Chapter 2. Installation Table of Contents @@ -322,7 +316,7 @@ Chapter 2. Installation 2.5.5. Install -2.1. Packages +2.1. Packages Some operating systems or softare package vendors may provide ready-to-use, pre-built software packages for the BIND 10 suite. @@ -332,28 +326,28 @@ Chapter 2. Installation FreeBSD ports, NetBSD pkgsrc, and Debian testing package collections provide all the prerequisite packages. -2.2. Install Hierarchy +2.2. Install Hierarchy The following is the standard, common layout of the complete BIND 10 installation: - o bin/ -- general tools and diagnostic clients. - o etc/bind10-devel/ -- configuration files. - o lib/ -- libraries and python modules. - o libexec/bind10-devel/ -- executables that a user wouldn't normally run + o bin/ — general tools and diagnostic clients. + o etc/bind10-devel/ — configuration files. + o lib/ — libraries and python modules. + o libexec/bind10-devel/ — executables that a user wouldn't normally run directly and are not run independently. These are the BIND 10 modules which are daemons started by the bind10 tool. - o sbin/ -- commands used by the system administrator. - o share/bind10-devel/ -- configuration specifications. - o share/doc/bind10-devel/ -- this guide and other supplementary + o sbin/ — commands used by the system administrator. + o share/bind10-devel/ — configuration specifications. + o share/doc/bind10-devel/ — this guide and other supplementary documentation. - o share/man/ -- manual pages (online documentation). - o var/bind10-devel/ -- data source and configuration databases. + o share/man/ — manual pages (online documentation). + o var/bind10-devel/ — data source and configuration databases. -2.3. Building Requirements +2.3. Building Requirements - In addition to the run-time requirements (listed in Section 1.2, "Required - Software at Run-time"), building BIND 10 from source code requires various + In addition to the run-time requirements (listed in Section 1.2, “Required + Software at Run-time”), building BIND 10 from source code requires various development include headers and program development tools. Note @@ -378,7 +372,7 @@ Chapter 2. Installation http://bind10.isc.org/wiki/SystemSpecificNotes for system-specific installation tips. -2.4. Quick start +2.4. Quick start Note @@ -389,48 +383,48 @@ Chapter 2. Installation To quickly get started with BIND 10, follow these steps. - 1. Install required run-time and build dependencies. - 2. Download the BIND 10 source tar file from +  1. Install required run-time and build dependencies. +  2. Download the BIND 10 source tar file from ftp://ftp.isc.org/isc/bind10/. - 3. Extract the tar file: +  3. Extract the tar file: $ gzcat bind10-VERSION.tar.gz | tar -xvf - - 4. Go into the source and run configure: +  4. Go into the source and run configure: $ cd bind10-VERSION $ ./configure - 5. Build it: +  5. Build it: $ make - 6. Install it (to default /usr/local): +  6. Install it (to default /usr/local): $ make install - 7. Start the server: +  7. Start the server: $ /usr/local/sbin/bind10 - 8. Test it; for example: +  8. Test it; for example: $ dig @127.0.0.1 -c CH -t TXT authors.bind - 9. Load desired zone file(s), for example: +  9. Load desired zone file(s), for example: $ b10-loadzone your.zone.example.org - 10. Test the new zone. + 10. Test the new zone. -2.5. Installation from source +2.5. Installation from source BIND 10 is open source software written in C++ and Python. It is freely available in source code form from ISC as a downloadable tar file or via BIND 10's Git code revision control service. (It may also be available in pre-compiled ready-to-use packages from operating system vendors.) - 2.5.1. Download Tar File + 2.5.1. Download Tar File Downloading a release tar file is the recommended method to obtain the source code. @@ -439,7 +433,7 @@ Chapter 2. Installation ftp://ftp.isc.org/isc/bind10/. Periodic development snapshots may also be available. - 2.5.2. Retrieve from Git + 2.5.2. Retrieve from Git Downloading this "bleeding edge" code is recommended only for developers or advanced users. Using development code in a production environment is @@ -454,7 +448,7 @@ Chapter 2. Installation The latest development code (and temporary experiments and un-reviewed code) is available via the BIND 10 code revision control system. This is powered by Git and all the BIND 10 development is public. The leading - development is done in the "master" branch. + development is done in the “master” branch. The code can be checked out from git://git.bind10.isc.org/bind10; for example: @@ -467,7 +461,7 @@ Chapter 2. Installation the --install switch. This will run autoconf, aclocal, libtoolize, autoheader, automake, and related commands. - 2.5.3. Configure before the build + 2.5.3. Configure before the build BIND 10 uses the GNU Build System to discover build environment details. To generate the makefiles using the defaults, simply run: @@ -502,14 +496,14 @@ Chapter 2. Installation If the configure fails, it may be due to missing or old dependencies. - 2.5.4. Build + 2.5.4. Build After the configure step is complete, to build the executables from the C++ code and prepare the Python scripts, run: $ make - 2.5.5. Install + 2.5.5. Install To install the BIND 10 executables, support files, and documentation, run: @@ -519,7 +513,7 @@ Chapter 2. Installation The install step may require superuser privileges. -Chapter 3. Starting BIND10 with bind10 +Chapter 3. Starting BIND10 with bind10 Table of Contents @@ -547,7 +541,7 @@ Chapter 3. Starting BIND10 with bind10 b10-cmdctl for administration tools to communicate with the system, and b10-stats for statistics collection. -3.1. Starting BIND 10 +3.1. Starting BIND 10 To start the BIND 10 service, simply run bind10. Run it with the --verbose switch to get additional debugging or diagnostic output. @@ -556,9 +550,9 @@ Chapter 3. Starting BIND10 with bind10 If the setproctitle Python module is detected at start up, the process names for the Python-based daemons will be renamed to better identify them - instead of just "python". This is not needed on some operating systems. + instead of just “python”. This is not needed on some operating systems. -3.2. Configuration to start processes +3.2. Configuration to start processes The processes to be used can be configured for bind10 to start, with the exception of the required b10-sockcreator, b10-msgq and b10-cfgmgr @@ -574,7 +568,7 @@ Chapter 3. Starting BIND10 with bind10 > config set Boss/components/b10-resolver/priority 10 > config commit - Now, what it means. We add an entry called "b10-resolver". It is both a + Now, what it means. We add an entry called “b10-resolver”. It is both a name used to reference this component in the configuration and the name of the process to start. Then we set some parameters on how to start it. @@ -583,7 +577,7 @@ Chapter 3. Starting BIND10 with bind10 a usual way. This is the list of components that need to be started in a special way, with the value of special used for them: - Table 3.1. Special startup components + Table 3.1. Special startup components +----------------------------------------------------------------------+ | Component | Special | Description | @@ -596,11 +590,11 @@ Chapter 3. Starting BIND10 with bind10 +----------------------------------------------------------------------+ The kind specifies how a failure of the component should be handled. If it - is set to "dispensable" (the default unless you set something else), it - will get started again if it fails. If it is set to "needed" and it fails + is set to “dispensable” (the default unless you set something else), it + will get started again if it fails. If it is set to “needed” and it fails at startup, the whole bind10 shuts down and exits with an error exit code. But if it fails some time later, it is just started again. If you set it - to "core", you indicate that the system is not usable without the + to “core”, you indicate that the system is not usable without the component and if such component fails, the system shuts down no matter when the failure happened. This is the behaviour of the core components (the ones you can't turn off), but you can declare any other components as @@ -616,7 +610,7 @@ Chapter 3. Starting BIND10 with bind10 address. It is the address used by the component on the b10-msgq message bus. The special components already know their address, but the usual ones don't. The address is by convention the thing after b10-, with the first - letter capitalized (eg. b10-stats would have "Stats" as its address). + letter capitalized (eg. b10-stats would have “Stats” as its address). The last one is process. It is the name of the process to be started. It defaults to the name of the component if not set, but you can use this to @@ -652,11 +646,11 @@ Chapter 3. Starting BIND10 with bind10 a situation, so it would probably not do what you want. Such support is yet to be implemented. -Chapter 4. Command channel +Chapter 4. Command channel The BIND 10 components use the b10-msgq message routing daemon to communicate with other BIND 10 components. The b10-msgq implements what is - called the "Command Channel". Processes intercommunicate by sending + called the “Command Channel”. Processes intercommunicate by sending messages on the command channel. Example messages include shutdown, get configurations, and set configurations. This Command Channel is not used for DNS message passing. It is used only to control and monitor the BIND @@ -667,7 +661,7 @@ Chapter 4. Command channel /usr/local/var/bind10-devel/msg_socket for this interprocess communication. -Chapter 5. Configuration manager +Chapter 5. Configuration manager The configuration manager, b10-cfgmgr, handles all BIND 10 system configuration. It provides persistent storage for configuration, and @@ -679,7 +673,7 @@ Chapter 5. Configuration manager The administrator doesn't connect to it directly, but uses a user interface to communicate with the configuration manager via b10-cmdctl's - REST-ful interface. b10-cmdctl is covered in Chapter 6, Remote control + REST-ful interface. b10-cmdctl is covered in Chapter 6, Remote control daemon. Note @@ -701,10 +695,10 @@ Chapter 5. Configuration manager The configuration manager does not have any command line arguments. Normally it is not started manually, but is automatically started using - the bind10 master process (as covered in Chapter 3, Starting BIND10 with + the bind10 master process (as covered in Chapter 3, Starting BIND10 with bind10). -Chapter 6. Remote control daemon +Chapter 6. Remote control daemon Table of Contents @@ -717,7 +711,7 @@ Chapter 6. Remote control daemon When b10-cmdctl starts, it firsts asks b10-cfgmgr about what modules are running and what their configuration is (over the b10-msgq channel). Then - it will start listening on HTTPS for clients -- the user interface -- such + it will start listening on HTTPS for clients — the user interface — such as bindctl. b10-cmdctl directly sends commands (received from the user interface) to @@ -744,7 +738,7 @@ Chapter 6. Remote control daemon /usr/local/etc/bind10-devel/cmdctl-accounts.csv. This comma-delimited file lists the accounts with a user name, hashed password, and salt. (A sample file is at /usr/local/share/bind10-devel/cmdctl-accounts.csv. It contains - the user named "root" with the password "bind10".) + the user named “root” with the password “bind10”.) The administrator may create a user account with the b10-cmdctl-usermgr tool. @@ -755,7 +749,7 @@ Chapter 6. Remote control daemon connection is stateless and times out in 1200 seconds by default. This can be redefined by using the --idle-timeout command line argument. -6.1. Configuration specification for b10-cmdctl +6.1. Configuration specification for b10-cmdctl The configuration items for b10-cmdctl are: accounts_file which defines the path to the user accounts database (the default is @@ -765,7 +759,7 @@ Chapter 6. Remote control daemon defines the path to the PEM private key file (the default is /usr/local/etc/bind10-devel/cmdctl-keyfile.pem). -Chapter 7. Control and configure user interface +Chapter 7. Control and configure user interface Note @@ -785,7 +779,7 @@ Chapter 7. Control and configure user interface b10-cfgmgr which then stores the details and relays (over a b10-msgq command channel) the configuration on to the specified module. -Chapter 8. Authoritative Server +Chapter 8. Authoritative Server Table of Contents @@ -793,13 +787,9 @@ Chapter 8. Authoritative Server 8.2. Data Source Backends - 8.2.1. In-memory Data Source + 8.2.1. Data source types - 8.2.2. In-memory Data Source with SQLite3 Backend - - 8.2.3. Reloading an In-memory Data Source - - 8.2.4. Disabling In-memory Data Sources + 8.2.2. Examples 8.3. Loading Master Zones Files @@ -807,10 +797,10 @@ Chapter 8. Authoritative Server IPv6, and SQLite3 and in-memory zone data backends. Normally it is started by the bind10 master process. -8.1. Server Configurations +8.1. Server Configurations b10-auth is configured via the b10-cfgmgr configuration manager. The - module name is "Auth". The configuration data items are: + module name is “Auth”. The configuration data items are: database_file This is an optional string to define the path to find the SQLite3 @@ -819,10 +809,10 @@ Chapter 8. Authoritative Server datasources datasources configures data sources. The list items include: type - to define the required data source type (such as "memory"); class - to optionally select the class (it defaults to "IN"); and zones to - define the file path name, the filetype ("sqlite3" to load from a - SQLite3 database file or "text" to load from a master text file), + to define the required data source type (such as “memory”); class + to optionally select the class (it defaults to “IN”); and zones to + define the file path name, the filetype (“sqlite3” to load from a + SQLite3 database file or “text” to load from a master text file), and the origin (default domain). By default, this is empty. Note @@ -852,7 +842,7 @@ Chapter 8. Authoritative Server There are plans to solve the problem such that the server handles it by itself. But until it is actually implemented, it is - recommended to alter the configuration -- remove the wildcard + recommended to alter the configuration — remove the wildcard addresses and list all addresses explicitly. Then the server will answer on the same interface the request came on, preserving the correct address. @@ -867,9 +857,9 @@ Chapter 8. Authoritative Server loadzone loadzone tells b10-auth to load or reload a zone file. The arguments include: class which optionally defines the class (it - defaults to "IN"); origin is the domain name of the zone; and + defaults to “IN”); origin is the domain name of the zone; and datasrc optionally defines the type of datasource (it defaults to - "memory"). + “memory”). Note @@ -885,79 +875,123 @@ Chapter 8. Authoritative Server argument to select the process ID to stop. (Note that the BIND 10 boss process may restart this service if configured.) -8.2. Data Source Backends +8.2. Data Source Backends + + Bind 10 has the concept of data sources. A data source is a place where + authoritative zone data reside and where they can be served from. This can + be a master file, a database or something completely different. + + Once a query arrives, b10-auth goes through a configured list of data + sources and finds the one containing a best matching zone. From the + equally good ones, the first one is taken. This data source is then used + to answer the query. Note - For the development prototype release, b10-auth supports a SQLite3 data - source backend and in-memory data source backend. Upcoming versions will + In the development prototype release, b10-auth can serve data from a + SQLite3 data source backend and from master files. Upcoming versions will be able to use multiple different data sources, such as MySQL and Berkeley DB. - By default, the SQLite3 backend uses the data file located at - /usr/local/var/bind10-devel/zone.sqlite3. (The full path is what was - defined at build configure time for --localstatedir. The default is - /usr/local/var/.) This data file location may be changed by defining the - "database_file" configuration. + The configuration is located in data_sources/classes. Each item there + represents one RR class and a list used to answer queries for that class. + The default contains two classes. The CH class contains a static data + source — one that serves things like “AUTHORS.BIND.”. The IN class + contains single SQLite3 data source with database file located at + /usr/local/var/bind10-devel/zone.sqlite3. - 8.2.1. In-memory Data Source + Each data source has several options. The first one is type, which + specifies the type of data source to use. Valid types include the ones + listed below, but bind10 uses dynamically loaded modules for them, so + there may be more in your case. This option is mandatory. - The following commands to bindctl provide an example of configuring an - in-memory data source containing the "example.com" zone with the zone file - named "example.com.zone": + Another option is params. This option is type specific; it holds different + data depending on the type above. Also, depending on the type, it could be + possible to omit it. - > config add Auth/datasources - > config set Auth/datasources[0]/type "memory" - > config add Auth/datasources[0]/zones - > config set Auth/datasources[0]/zones[0]/origin "example.com" - > config set Auth/datasources[0]/zones[0]/file "example.com.zone" + There are two options related to the so-called cache. If you enable cache, + zone data from the data source are loaded into memory. Then, when + answering a query, b10-auth looks into the memory only instead of the data + source, which speeds answering up. The first option is cache-enable, a + boolean value turning the cache on and off (off is the default). The + second one, cache-zones, is a list of zone origins to load into in-memory. + Remember that zones in the data source not listed here will not be loaded + and will not be available at all. + + 8.2.1. Data source types + + As mentioned, the type used by default is “sqlite3”. It has single + configuration option inside params — database_file, which contains the + path to the sqlite3 file containing the data. + + Another type is called “MasterFiles”. This one is slightly special. The + data are stored in RFC1034 master files. Because answering directly from + them would be impractical, this type mandates the cache to be enabled. + Also, the list of zones (cache-zones) should be omitted. The params is a + dictionary mapping from zone origins to the files they reside in. + + 8.2.2. Examples + + As this is one of the more complex configurations of Bind10, we show some + examples. They all assume they start with default configuration. + + First, let's disable the static data source (“VERSION.BIND” and friends). + As it is the only data source in the CH class, we can remove the whole + class. + + > config remove data_sources/classes CH > config commit - The authoritative server will begin serving it immediately after the zone - data is loaded from the master text file. + Another one, let's say our default data source contains zones + “example.org.” and “example.net.”. We want them to be served from memory + to make the answering faster. - 8.2.2. In-memory Data Source with SQLite3 Backend - - The following commands to bindctl provide an example of configuring an - in-memory data source containing the "example.org" zone with a SQLite3 - backend file named "example.org.sqlite3": - - > config add Auth/datasources - > config set Auth/datasources[1]/type "memory" - > config add Auth/datasources[1]/zones - > config set Auth/datasources[1]/zones[0]/origin "example.org" - > config set Auth/datasources[1]/zones[0]/file "example.org.sqlite3" - > config set Auth/datasources[1]/zones[0]/filetype "sqlite3" + > config set data_sources/classes/IN[0]/cache-enable true + > config add data_sources/classes/IN[0]/cache-zones example.org. + > config add data_sources/classes/IN[0]/cache-zones example.net. > config commit - The authoritative server will begin serving it immediately after the zone - data is loaded from the database file. + Now every time the zone in the data source is changed by the operator, + Bind10 needs to be told to reload it, by - 8.2.3. Reloading an In-memory Data Source + > Auth loadzone example.org - Use the Auth loadzone command in bindctl to reload a changed master file - into memory; for example: + You don't need to do this when the zone is modified by XfrIn, it does so + automatically. - > Auth loadzone origin="example.com" + Now, the last example is when there are master files we want to serve in + addition to whatever is inside the sqlite3 database. - 8.2.4. Disabling In-memory Data Sources - - By default, the memory data source is disabled; it must be configured - explicitly. To disable all the in-memory zones, specify a null list for - Auth/datasources: - - > config set Auth/datasources/ [] + > config add data_sources/classes/IN + > config set data_sources/classes/IN[1]/type MasterFiles + > config set data_sources/classes/IN[1]/cache-enable true + > config set data_sources/classes/IN[1]/params { "example.org": "/path/to/example.org", "example.com": "/path/to/example.com" } > config commit - The following example stops serving a specific zone: + Initially, a map value has to be set, but this value may be an empty map. + After that, key/value pairs can be added with 'config add' and keys can be + removed with 'config remove'. The initial value may be an empty map, but + it has to be set before zones are added or removed. - > config remove Auth/datasources[0]/zones[0] - > config commit + > config set data_sources/classes/IN[1]/params {} + > config add data_sources/classes/IN[1]/params another.example.org /path/to/another.example.org + > config add data_sources/classes/IN[1]/params another.example.com /path/to/another.example.com + > config remove data_sources/classes/IN[1]/params another.example.org - (Replace the list number(s) in datasources[0] and/or zones[0] for the - relevant zone as needed.) -8.3. Loading Master Zones Files + bindctl. To reload a zone, you the same command as above. + + Note + + There's also Auth/database_file configuration variable, pointing to a + sqlite3 database file. This is no longer used by b10-auth, but it is left + in place for now, since other modules use it. Once b10-xfrin, b10-xfrout + and b10-ddns are ported to the new configuration, this will disappear. But + for now, make sure that if you use any of these modules, the new and old + configuration correspond. The defaults are consistent, so unless you + tweaked either the new or the old configuration, you're good. + +8.3. Loading Master Zones Files RFC 1035 style DNS master zone files may imported into a BIND 10 SQLite3 data source by using the b10-loadzone utility. @@ -988,7 +1022,7 @@ Chapter 8. Authoritative Server If you reload a zone already existing in the database, all records from that prior zone disappear and a whole new set appears. -Chapter 9. Incoming Zone Transfers +Chapter 9. Incoming Zone Transfers Table of Contents @@ -1012,7 +1046,7 @@ Chapter 9. Incoming Zone Transfers implementation limitations of the current development release, however, it only tries AXFR by default, and care should be taken to enable IXFR. -9.1. Configuration for Incoming Zone Transfers +9.1. Configuration for Incoming Zone Transfers In practice, you need to specify a list of secondary zones to enable incoming zone transfers for these zones (you can still trigger a zone @@ -1029,7 +1063,7 @@ Chapter 9. Incoming Zone Transfers (We assume there has been no zone configuration before). -9.2. Enabling IXFR +9.2. Enabling IXFR As noted above, b10-xfrin uses AXFR for zone transfers by default. To enable IXFR for zone transfers for a particular zone, set the use_ixfr @@ -1052,7 +1086,7 @@ Chapter 9. Incoming Zone Transfers be implemented in a near future version, at which point we will enable IXFR by default. -9.3. Secondary Manager +9.3. Secondary Manager The b10-zonemgr process is started by bind10. It keeps track of SOA refresh, retry, and expire timers and other details for BIND 10 to perform @@ -1077,14 +1111,14 @@ Chapter 9. Incoming Zone Transfers for it), b10-zonemgr will automatically tell b10-xfrin to transfer the zone in. -9.4. Trigger an Incoming Zone Transfer Manually +9.4. Trigger an Incoming Zone Transfer Manually To manually trigger a zone transfer to retrieve a remote zone, you may use the bindctl utility. For example, at the bindctl prompt run: > Xfrin retransfer zone_name="foo.example.org" master=192.0.2.99 -9.5. Incoming Transfers with In-memory Datasource +9.5. Incoming Transfers with In-memory Datasource In the case of an incoming zone transfer, the received zone is first stored in the corresponding BIND 10 datasource. In case the secondary zone @@ -1094,9 +1128,9 @@ Chapter 9. Incoming Zone Transfers The administrator doesn't have to do anything for b10-auth to serve the new version of the zone, except for the configuration such as the one - described in Section 8.2.2, "In-memory Data Source with SQLite3 Backend". + described in Section 8.2, “Data Source Backends”. -Chapter 10. Outbound Zone Transfers +Chapter 10. Outbound Zone Transfers The b10-xfrout process is started by bind10. When the b10-auth authoritative DNS server receives an AXFR or IXFR request, b10-auth @@ -1145,7 +1179,7 @@ Chapter 10. Outbound Zone Transfers The way to specify zone specific configuration (ACLs, etc) is likely to be changed. -Chapter 11. Dynamic DNS Update +Chapter 11. Dynamic DNS Update Table of Contents @@ -1166,10 +1200,9 @@ Chapter 11. Dynamic DNS Update successful update, REFUSED if rejected due to ACL check, etc). If the zone has been changed as a result, it will internally notify b10-xfrout so that other secondary servers will be notified via the DNS NOTIFY protocol. In - addition, if b10-auth serves the updated zone from its in-memory cache (as - described in Section 8.2.2, "In-memory Data Source with SQLite3 Backend"), - b10-ddns will also notify b10-auth so that b10-auth will re-cache the - updated zone content. + addition, if b10-auth serves the updated zone (as described in + Section 8.2, “Data Source Backends”), b10-ddns will also notify b10-auth + so that b10-auth will re-cache the updated zone content if necessary. The b10-ddns component supports requests over both UDP and TCP, and both IPv6 and IPv4; for TCP requests, however, it terminates the TCP connection @@ -1182,7 +1215,7 @@ Chapter 11. Dynamic DNS Update As of this writing b10-ddns does not support update forwarding for secondary zones. If it receives an update request for a secondary zone, it - will immediately return a "not implemented" response. + will immediately return a “not implemented” response. Note @@ -1190,7 +1223,7 @@ Chapter 11. Dynamic DNS Update supported. But currently it's considered a lower priority task and there is no specific plan of implementing this feature. -11.1. Enabling Dynamic Update +11.1. Enabling Dynamic Update First off, it must be made sure that a few components on which b10-ddns depends are configured to run, which are b10-auth and b10-zonemgr. In @@ -1239,7 +1272,7 @@ Chapter 11. Dynamic DNS Update b10-ddns would start and work without specifying it. But for it to shutdown gracefully this parameter should also be specified. -11.2. Access Control +11.2. Access Control By default, b10-ddns rejects any update requests from any clients by returning a REFUSED response. To allow updates to take effect, an access @@ -1253,7 +1286,7 @@ Chapter 11. Dynamic DNS Update The zone's origin name class - The RR class of the zone (normally "IN", and in that case can be + The RR class of the zone (normally “IN”, and in that case can be omitted in configuration) update_acl @@ -1264,7 +1297,7 @@ Chapter 11. Dynamic DNS Update In general, an update ACL rule that allows an update request should be configured with a TSIG key. This is an example update ACL that allows - updates to the zone named "example.org" (of default RR class "IN") from + updates to the zone named “example.org” (of default RR class “IN”) from clients that send requests signed with a TSIG whose key name is "key.example.org" (and refuses all others): @@ -1273,7 +1306,7 @@ Chapter 11. Dynamic DNS Update > config add DDNS/zones[0]/update_acl {"action": "ACCEPT", "key": "key.example.org"} > config commit - The TSIG key must be configured system wide (see Chapter 10, Outbound Zone + The TSIG key must be configured system wide (see Chapter 10, Outbound Zone Transfers.) Multiple rules can be specified in the ACL, and an ACL rule can consist of @@ -1309,7 +1342,7 @@ Chapter 11. Dynamic DNS Update which is rejecting any requests in the case of b10-ddns. Other actions than "ACCEPT", namely "REJECT" and "DROP", can be used, too. - See Chapter 12, Recursive Name Server about their effects. + See Chapter 12, Recursive Name Server about their effects. Currently update ACL can only control updates per zone basis; it's not possible to specify access control with higher granularity such as for @@ -1329,7 +1362,7 @@ Chapter 11. Dynamic DNS Update clients. There have been other troubles that could have been avoided if the ACL could be checked before the prerequisite check. -11.3. Miscellaneous Operational Issues +11.3. Miscellaneous Operational Issues Unlike BIND 9, BIND 10 currently does not support automatic re-signing of DNSSEC-signed zone when it's updated via DDNS. It could be possible to @@ -1338,20 +1371,20 @@ Chapter 11. Dynamic DNS Update operation. In general, it's not advisable to allow DDNS for a signed zone at this moment. - Also unlike BIND 9, it's currently not possible to "freeze" a zone + Also unlike BIND 9, it's currently not possible to “freeze” a zone temporarily in order to suspend DDNS while you manually update the zone. If you need to make manual updates to a dynamic zone, you'll need to temporarily reject any updates to the zone via the update ACLs. Dynamic updates are only applicable to primary zones. In order to avoid updating secondary zones via DDNS requests, b10-ddns refers to the - "secondary_zones" configuration of b10-zonemgr. Zones listed in - "secondary_zones" will never be updated via DDNS regardless of the update + “secondary_zones” configuration of b10-zonemgr. Zones listed in + “secondary_zones” will never be updated via DDNS regardless of the update ACL configuration; b10-ddns will return a NOTAUTH (server not authoritative for the zone) response. If you have a "conceptual" secondary zone whose content is a copy of some external source but is not updated via the standard zone transfers and therefore not listed in - "secondary_zones", be careful not to allow DDNS for the zone; it would be + “secondary_zones”, be careful not to allow DDNS for the zone; it would be quite likely to lead to inconsistent state between different servers. Normally this should not be a problem because the default update ACL rejects any update requests, but you may want to take an extra care about @@ -1362,7 +1395,7 @@ Chapter 11. Dynamic DNS Update can be retrieved in the form of outbound IXFR. This is done automatically; it does not require specific configuration to make this possible. -Chapter 12. Recursive Name Server +Chapter 12. Recursive Name Server Table of Contents @@ -1393,26 +1426,26 @@ Chapter 12. Recursive Name Server > config set Resolver/listen_on[2]/port 53 > config commit - (Replace the "2" as needed; run "config show Resolver/listen_on" if + (Replace the “2” as needed; run “config show Resolver/listen_on” if needed.) -12.1. Access Control +12.1. Access Control By default, the b10-resolver daemon only accepts DNS queries from the localhost (127.0.0.1 and ::1). The Resolver/query_acl configuration may be used to reject, drop, or allow specific IPs or networks. This configuration list is first match. - The configuration's action item may be set to "ACCEPT" to allow the - incoming query, "REJECT" to respond with a DNS REFUSED return code, or - "DROP" to ignore the query without any response (such as a blackhole). For + The configuration's action item may be set to “ACCEPT” to allow the + incoming query, “REJECT” to respond with a DNS REFUSED return code, or + “DROP” to ignore the query without any response (such as a blackhole). For more information, see the respective debugging messages: RESOLVER_QUERY_ACCEPTED, RESOLVER_QUERY_REJECTED, and RESOLVER_QUERY_DROPPED. The required configuration's from item is set to an IPv4 or IPv6 address, addresses with an network mask, or to the special lowercase keywords - "any6" (for any IPv6 address) or "any4" (for any IPv4 address). + “any6” (for any IPv6 address) or “any4” (for any IPv4 address). For example to allow the 192.168.1.0/24 network to use your recursive name server, at the bindctl prompt run: @@ -1422,14 +1455,14 @@ Chapter 12. Recursive Name Server > config set Resolver/query_acl[2]/from "192.168.1.0/24" > config commit - (Replace the "2" as needed; run "config show Resolver/query_acl" if + (Replace the “2” as needed; run “config show Resolver/query_acl” if needed.) Note This prototype access control configuration syntax may be changed. -12.2. Forwarding +12.2. Forwarding To enable forwarding, the upstream address and port must be configured to forward queries to, such as: @@ -1445,7 +1478,7 @@ Chapter 12. Recursive Name Server > config set Resolver/forward_addresses [] > config commit -Chapter 13. DHCPv4 Server +Chapter 13. DHCPv4 Server Table of Contents @@ -1465,7 +1498,7 @@ Chapter 13. DHCPv4 Server clients. Even though principles of both DHCPv4 and DHCPv6 are somewhat similar, these are two radically different protocols. BIND10 offers server implementations for both DHCPv4 and DHCPv6. This chapter is about DHCP for - IPv4. For a description of the DHCPv6 server, see Chapter 14, DHCPv6 + IPv4. For a description of the DHCPv6 server, see Chapter 14, DHCPv6 Server. The DHCPv4 server component is currently under intense development. You @@ -1473,7 +1506,7 @@ Chapter 13. DHCPv4 Server developers mailing list. The DHCPv4 and DHCPv6 components in BIND10 architecture are internally - code named "Kea". + code named “Kea”. Note @@ -1481,17 +1514,17 @@ Chapter 13. DHCPv4 Server servers. That means that while they are capable of performing DHCP configuration, they are not fully functional yet. In particular, neither has functional lease databases. This means that they will assign the same, - fixed, hardcoded addresses to any client that will ask. See Section 13.4, - "DHCPv4 Server Limitations" and Section 14.4, "DHCPv6 Server Limitations" + fixed, hardcoded addresses to any client that will ask. See Section 13.4, + “DHCPv4 Server Limitations” and Section 14.4, “DHCPv6 Server Limitations” for detailed description. -13.1. DHCPv4 Server Usage +13.1. DHCPv4 Server Usage BIND10 provides the DHCPv4 server component since December 2011. It is a skeleton server and can be described as an early prototype that is not fully functional yet. It is mature enough to conduct first tests in lab - environment, but it has significant limitations. See Section 13.4, "DHCPv4 - Server Limitations" for details. + environment, but it has significant limitations. See Section 13.4, “DHCPv4 + Server Limitations” for details. b10-dhcp4 is a BIND10 component and is being run under BIND10 framework. To add a DHCPv4 process to the set of running BIND10 services, you can use @@ -1524,7 +1557,7 @@ Chapter 13. DHCPv4 Server started directly, but rather via bind10. Please be aware of this planned change. -13.2. DHCPv4 Server Configuration +13.2. DHCPv4 Server Configuration The DHCPv4 server does not have a lease database implemented yet nor any support for configuration, so every time the same set of configuration @@ -1545,55 +1578,55 @@ Chapter 13. DHCPv4 Server Lease database and configuration support is planned for 2012. -13.3. Supported standards +13.3. Supported standards The following standards and draft standards are currently supported: - o RFC2131: Supported messages are DISCOVER, OFFER, REQUEST, and ACK. - o RFC2132: Supported options are: PAD (0), END(255), Message Type(53), + o RFC2131: Supported messages are DISCOVER, OFFER, REQUEST, and ACK. + o RFC2132: Supported options are: PAD (0), END(255), Message Type(53), DHCP Server Identifier (54), Domain Name (15), DNS Servers (6), IP Address Lease Time (51), Subnet mask (1), and Routers (3). -13.4. DHCPv4 Server Limitations +13.4. DHCPv4 Server Limitations These are the current limitations of the DHCPv4 server software. Most of them are reflections of the early stage of development and should be - treated as "not implemented yet", rather than actual limitations. + treated as “not implemented yet”, rather than actual limitations. - o During initial IPv4 node configuration, the server is expected to send + o During initial IPv4 node configuration, the server is expected to send packets to a node that does not have IPv4 address assigned yet. The server requires certain tricks (or hacks) to transmit such packets. This is not implemented yet, therefore DHCPv4 server supports relayed traffic only (that is, normal point to point communication). - o b10-dhcp4 provides a single, fixed, hardcoded lease to any client that + o b10-dhcp4 provides a single, fixed, hardcoded lease to any client that asks. There is no lease manager implemented. If two clients request addresses, they will both get the same fixed address. - o b10-dhcp4 does not support any configuration mechanisms yet. The whole + o b10-dhcp4 does not support any configuration mechanisms yet. The whole configuration is currently hardcoded. The only way to tweak - configuration is to directly modify source code. See see Section 13.2, - "DHCPv4 Server Configuration" for details. - o Upon start, the server will open sockets on all interfaces that are + configuration is to directly modify source code. See see Section 13.2, + “DHCPv4 Server Configuration” for details. + o Upon start, the server will open sockets on all interfaces that are not loopback, are up and running and have IPv4 address. - o PRL (Parameter Request List, a list of options requested by a client) + o PRL (Parameter Request List, a list of options requested by a client) is currently ignored and server assigns DNS SERVER and DOMAIN NAME options. - o b10-dhcp4 does not support BOOTP. That is a design choice. This + o b10-dhcp4 does not support BOOTP. That is a design choice. This limitation is permanent. If you have legacy nodes that can't use DHCP and require BOOTP support, please use latest version of ISC DHCP http://www.isc.org/software/dhcp. - o Interface detection is currently working on Linux only. See - Section 15.1, "Interface detection" for details. - o b10-dhcp4 does not verify that assigned address is unused. According + o Interface detection is currently working on Linux only. See + Section 15.1, “Interface detection” for details. + o b10-dhcp4 does not verify that assigned address is unused. According to RFC2131, the allocating server should verify that address is no used by sending ICMP echo request. - o Address renewal (RENEW), rebinding (REBIND), confirmation (CONFIRM), + o Address renewal (RENEW), rebinding (REBIND), confirmation (CONFIRM), duplication report (DECLINE) and release (RELEASE) are not supported yet. - o DNS Update is not supported yet. - o -v (verbose) command line option is currently the default, and cannot + o DNS Update is not supported yet. + o -v (verbose) command line option is currently the default, and cannot be disabled. -Chapter 14. DHCPv6 Server +Chapter 14. DHCPv6 Server Table of Contents @@ -1608,14 +1641,14 @@ Chapter 14. DHCPv6 Server Dynamic Host Configuration Protocol for IPv6 (DHCPv6) is specified in RFC3315. BIND10 provides DHCPv6 server implementation that is described in this chapter. For a description of the DHCPv4 server implementation, see - Chapter 13, DHCPv4 Server. + Chapter 13, DHCPv4 Server. The DHCPv6 server component is currently under intense development. You may want to check out BIND10 DHCP (Kea) wiki and recent posts on BIND10 developers mailing list. The DHCPv4 and DHCPv6 components in BIND10 architecture are internally - code named "Kea". + code named “Kea”. Note @@ -1623,17 +1656,17 @@ Chapter 14. DHCPv6 Server servers. That means that while they are capable of performing DHCP configuration, they are not fully functional yet. In particular, neither has functional lease databases. This means that they will assign the same, - fixed, hardcoded addresses to any client that will ask. See Section 13.4, - "DHCPv4 Server Limitations" and Section 14.4, "DHCPv6 Server Limitations" + fixed, hardcoded addresses to any client that will ask. See Section 13.4, + “DHCPv4 Server Limitations” and Section 14.4, “DHCPv6 Server Limitations” for detailed description. -14.1. DHCPv6 Server Usage +14.1. DHCPv6 Server Usage BIND10 provides the DHCPv6 server component since September 2011. It is a skeleton server and can be described as an early prototype that is not fully functional yet. It is mature enough to conduct first tests in lab - environment, but it has significant limitations. See Section 14.4, "DHCPv6 - Server Limitations" for details. + environment, but it has significant limitations. See Section 14.4, “DHCPv6 + Server Limitations” for details. The DHCPv6 server is implemented as b10-dhcp6 daemon. As it is not configurable yet, it is fully autonomous, that is it does not interact @@ -1659,7 +1692,7 @@ Chapter 14. DHCPv6 Server started directly, but rather via bind10. Please be aware of this planned change. -14.2. DHCPv6 Server Configuration +14.2. DHCPv6 Server Configuration The DHCPv6 server does not have lease database implemented yet or any support for configuration, so every time the same set of configuration @@ -1679,50 +1712,50 @@ Chapter 14. DHCPv6 Server Lease database and configuration support is planned for 2012. -14.3. Supported DHCPv6 Standards +14.3. Supported DHCPv6 Standards The following standards and draft standards are currently supported: - o RFC3315: Supported messages are SOLICIT, ADVERTISE, REQUEST, and + o RFC3315: Supported messages are SOLICIT, ADVERTISE, REQUEST, and REPLY. Supported options are SERVER_ID, CLIENT_ID, IA_NA, and IAADDRESS. - o RFC3646: Supported option is DNS_SERVERS. + o RFC3646: Supported option is DNS_SERVERS. -14.4. DHCPv6 Server Limitations +14.4. DHCPv6 Server Limitations These are the current limitations of the DHCPv6 server software. Most of them are reflections of the early stage of development and should be - treated as "not implemented yet", rather than actual limitations. + treated as “not implemented yet”, rather than actual limitations. - o Relayed traffic is not supported. - o b10-dhcp6 provides a single, fixed, hardcoded lease to any client that + o Relayed traffic is not supported. + o b10-dhcp6 provides a single, fixed, hardcoded lease to any client that asks. There is no lease manager implemented. If two clients request addresses, they will both get the same fixed address. - o b10-dhcp6 does not support any configuration mechanisms yet. The whole + o b10-dhcp6 does not support any configuration mechanisms yet. The whole configuration is currently hardcoded. The only way to tweak - configuration is to directly modify source code. See see Section 14.2, - "DHCPv6 Server Configuration" for details. - o Upon start, the server will open sockets on all interfaces that are + configuration is to directly modify source code. See see Section 14.2, + “DHCPv6 Server Configuration” for details. + o Upon start, the server will open sockets on all interfaces that are not loopback, are up, running and are multicast capable and have IPv6 address. Support for multiple interfaces is not coded in reception routines yet, so if you are running this code on a machine that has many interfaces and b10-dhcp6 happens to listen on wrong interface, the easiest way to work around this problem is to turn down other interfaces. This limitation will be fixed shortly. - o ORO (Option Request Option, a list of options requested by a client) + o ORO (Option Request Option, a list of options requested by a client) is currently ignored and server assigns DNS SERVER option. - o Temporary addresses are not supported yet. - o Prefix delegation is not supported yet. - o Address renewal (RENEW), rebinding (REBIND), confirmation (CONFIRM), + o Temporary addresses are not supported yet. + o Prefix delegation is not supported yet. + o Address renewal (RENEW), rebinding (REBIND), confirmation (CONFIRM), duplication report (DECLINE) and release (RELEASE) are not supported yet. - o DNS Update is not supported yet. - o Interface detection is currently working on Linux only. See - Section 15.1, "Interface detection" for details. - o -v (verbose) command line option is currently the default, and cannot + o DNS Update is not supported yet. + o Interface detection is currently working on Linux only. See + Section 15.1, “Interface detection” for details. + o -v (verbose) command line option is currently the default, and cannot be disabled. -Chapter 15. libdhcp++ library +Chapter 15. libdhcp++ library Table of Contents @@ -1740,7 +1773,7 @@ Chapter 15. libdhcp++ library is designed to be portable, universal library useful for any kind of DHCP-related software. -15.1. Interface detection +15.1. Interface detection Both DHCPv4 and DHCPv6 components share network interface detection routines. Interface detection is currently only supported on Linux @@ -1751,11 +1784,11 @@ Chapter 15. libdhcp++ library lo0) can be easily predicted. Please contact BIND10 development team if you are interested in running DHCP components on systems other than Linux. -15.2. DHCPv4/DHCPv6 packet handling +15.2. DHCPv4/DHCPv6 packet handling TODO: Describe packet handling here, with pointers to wiki -Chapter 16. Statistics +Chapter 16. Statistics The b10-stats process is started by bind10. It periodically collects statistics data from various modules and aggregates it. @@ -1787,7 +1820,7 @@ Chapter 16. Statistics } -Chapter 17. Logging +Chapter 17. Logging Table of Contents @@ -1801,13 +1834,13 @@ Chapter 17. Logging 17.2. Logging Message Format -17.1. Logging configuration +17.1. Logging configuration The logging system in BIND 10 is configured through the Logging module. All BIND 10 modules will look at the configuration in Logging to see what should be logged and to where. - 17.1.1. Loggers + 17.1.1. Loggers Within BIND 10, a message is logged through a component called a "logger". Different parts of BIND 10 log messages through different loggers, and @@ -1820,78 +1853,78 @@ Chapter 17. Logging (the component that is generating the messages), the severity (what to log), and the output_options (where to log). - 17.1.1.1. name (string) + 17.1.1.1. name (string) Each logger in the system has a name, the name being that of the component using it to log messages. For instance, if you want to configure logging - for the resolver module, you add an entry for a logger named "Resolver". + for the resolver module, you add an entry for a logger named “Resolver”. This configuration will then be used by the loggers in the Resolver module, and all the libraries used by it. If you want to specify logging for one specific library within the module, you set the name to module.library. For example, the logger used by the - nameserver address store component has the full name of "Resolver.nsas". + nameserver address store component has the full name of “Resolver.nsas”. If there is no entry in Logging for a particular library, it will use the configuration given for the module. To illustrate this, suppose you want the cache library to log messages of severity DEBUG, and the rest of the resolver code to log messages of severity INFO. To achieve this you specify two loggers, one with the name - "Resolver" and severity INFO, and one with the name "Resolver.cache" with + “Resolver” and severity INFO, and one with the name “Resolver.cache” with severity DEBUG. As there are no entries for other libraries (e.g. the - nsas), they will use the configuration for the module ("Resolver"), so + nsas), they will use the configuration for the module (“Resolver”), so giving the desired behavior. - One special case is that of a module name of "*" (asterisks), which is + One special case is that of a module name of “*” (asterisks), which is interpreted as any module. You can set global logging options by using this, including setting the logging configuration for a library that is - used by multiple modules (e.g. "*.config" specifies the configuration + used by multiple modules (e.g. “*.config” specifies the configuration library code in whatever module is using it). If there are multiple logger specifications in the configuration that might match a particular logger, the specification with the more specific logger name takes precedence. For example, if there are entries for for - both "*" and "Resolver", the resolver module -- and all libraries it uses - -- will log messages according to the configuration in the second entry - ("Resolver"). All other modules will use the configuration of the first - entry ("*"). If there was also a configuration entry for "Resolver.cache", + both “*” and “Resolver”, the resolver module — and all libraries it uses — + will log messages according to the configuration in the second entry + (“Resolver”). All other modules will use the configuration of the first + entry (“*”). If there was also a configuration entry for “Resolver.cache”, the cache library within the resolver would use that in preference to the - entry for "Resolver". + entry for “Resolver”. One final note about the naming. When specifying the module name within a logger, use the name of the module as specified in bindctl, e.g. - "Resolver" for the resolver module, "Xfrout" for the xfrout module, etc. + “Resolver” for the resolver module, “Xfrout” for the xfrout module, etc. When the message is logged, the message will include the name of the logger generating the message, but with the module name replaced by the name of the process implementing the module (so for example, a message - generated by the "Auth.cache" logger will appear in the output with a - logger name of "b10-auth.cache"). + generated by the “Auth.cache” logger will appear in the output with a + logger name of “b10-auth.cache”). - 17.1.1.2. severity (string) + 17.1.1.2. severity (string) This specifies the category of messages logged. Each message is logged with an associated severity which may be one of the following (in descending order of severity): - o FATAL - o ERROR - o WARN - o INFO - o DEBUG + o FATAL + o ERROR + o WARN + o INFO + o DEBUG When the severity of a logger is set to one of these values, it will only log messages of that severity, and the severities above it. The severity may also be set to NONE, in which case all messages from that logger are inhibited. - 17.1.1.3. output_options (list) + 17.1.1.3. output_options (list) Each logger can have zero or more output_options. These specify where log messages are sent to. These are explained in detail below. The other options for a logger are: - 17.1.1.4. debuglevel (integer) + 17.1.1.4. debuglevel (integer) When a logger's severity is set to DEBUG, this value specifies what debug messages should be printed. It ranges from 0 (least verbose) to 99 (most @@ -1899,80 +1932,80 @@ Chapter 17. Logging If severity for the logger is not DEBUG, this value is ignored. - 17.1.1.5. additive (true or false) + 17.1.1.5. additive (true or false) If this is true, the output_options from the parent will be used. For - example, if there are two loggers configured; "Resolver" and - "Resolver.cache", and additive is true in the second, it will write the - log messages not only to the destinations specified for "Resolver.cache", + example, if there are two loggers configured; “Resolver” and + “Resolver.cache”, and additive is true in the second, it will write the + log messages not only to the destinations specified for “Resolver.cache”, but also to the destinations as specified in the output_options in the - logger named "Resolver". + logger named “Resolver”. - 17.1.2. Output Options + 17.1.2. Output Options The main settings for an output option are the destination and a value called output, the meaning of which depends on the destination that is set. - 17.1.2.1. destination (string) + 17.1.2.1. destination (string) The destination is the type of output. It can be one of: - o console - o file - o syslog + o console + o file + o syslog - 17.1.2.2. output (string) + 17.1.2.2. output (string) Depending on what is set as the output destination, this value is interpreted as follows: - destination is "console" + destination is “console” - The value of output must be one of "stdout" (messages printed to - standard output) or "stderr" (messages printed to standard error). + The value of output must be one of “stdout” (messages printed to + standard output) or “stderr” (messages printed to standard error). - Note: if output is set to "stderr" and a lot of messages are + Note: if output is set to “stderr” and a lot of messages are produced in a short time (e.g. if the logging level is set to DEBUG), you may occasionally see some messages jumbled up together. This is due to a combination of the way that messages are written to the screen and the unbuffered nature of the standard error stream. If this occurs, it is recommended that - output be set to "stdout". + output be set to “stdout”. - destination is "file" + destination is “file” The value of output is interpreted as a file name; log messages will be appended to this file. - destination is "syslog" + destination is “syslog” The value of output is interpreted as the syslog facility (e.g. local0) that should be used for log messages. The other options for output_options are: - 17.1.2.2.1. flush (true of false) + 17.1.2.2.1. flush (true of false) Flush buffers after each log message. Doing this will reduce performance but will ensure that if the program terminates abnormally, all messages up to the point of termination are output. - 17.1.2.2.2. maxsize (integer) + 17.1.2.2.2. maxsize (integer) Only relevant when destination is file, this is maximum file size of output files in bytes. When the maximum size is reached, the file is renamed and a new file opened. (For example, a ".1" is appended to the - name -- if a ".1" file exists, it is renamed ".2", etc.) + name — if a ".1" file exists, it is renamed ".2", etc.) If this is 0, no maximum file size is used. - 17.1.2.2.3. maxver (integer) + 17.1.2.2.3. maxver (integer) Maximum number of old log files to keep around when rolling the output - file. Only relevant when destination is "file". + file. Only relevant when destination is “file”. - 17.1.3. Example session + 17.1.3. Example session In this example we want to set the global logging to write to the file /var/log/my_bind10.log, at severity WARN. We want the authoritative server @@ -2071,9 +2104,9 @@ Chapter 17. Logging > config remove Logging/loggers[1] > config commit - And every module will now be using the values from the logger named "*". + And every module will now be using the values from the logger named “*”. -17.2. Logging Message Format +17.2. Logging Message Format Each message written by BIND 10 to the configured logging destinations comprises a number of components that identify the origin of the message diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml index c5df326db8..70ca345bb2 100644 --- a/doc/guide/bind10-guide.xml +++ b/doc/guide/bind10-guide.xml @@ -1611,8 +1611,19 @@ can use various data source backends. > config set data_sources/classes/IN[1]/params { "example.org": "/path/to/example.org", "example.com": "/path/to/example.com" } > config commit - Unfortunately, due to current technical limitations, the params must - be set as one JSON blob, it can't be edited in + Initially, a map value has to be set, but this value may be an + empty map. After that, key/value pairs can be added with 'config + add' and keys can be removed with 'config remove'. The initial + value may be an empty map, but it has to be set before zones are + added or removed. + + +> config set data_sources/classes/IN[1]/params {} +> config add data_sources/classes/IN[1]/params another.example.org /path/to/another.example.org +> config add data_sources/classes/IN[1]/params another.example.com /path/to/another.example.com +> config remove data_sources/classes/IN[1]/params another.example.org + + bindctl. To reload a zone, you the same command as above. diff --git a/doc/guide/bind10-messages.html b/doc/guide/bind10-messages.html index 37c742b7e5..29d4ba98f1 100644 --- a/doc/guide/bind10-messages.html +++ b/doc/guide/bind10-messages.html @@ -1,4 +1,4 @@ -BIND 10 Messages Manual

BIND 10 Messages Manual

This is the messages manual for BIND 10 version +BIND 10 Messages Manual

BIND 10 Messages Manual

This is the messages manual for BIND 10 version 20120712.

Copyright © 2011-2012 Internet Systems Consortium, Inc.

Abstract

BIND 10 is a Domain Name System (DNS) suite managed by Internet Systems Consortium (ISC). It includes DNS libraries and modular components for controlling authoritative and From 1bcbd2eeafca5c1e1fbed6a5e8eed17487bbbedd Mon Sep 17 00:00:00 2001 From: Michal 'vorner' Vaner Date: Fri, 10 Aug 2012 15:39:59 +0200 Subject: [PATCH 56/69] [2066] Hopefully more understandable description of ACLs Including something like more formal grammar of them. --- doc/guide/bind10-guide.xml | 51 +++++++++++++++++++++++++++----------- 1 file changed, 36 insertions(+), 15 deletions(-) diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml index 97437c8d12..772bd82c43 100644 --- a/doc/guide/bind10-guide.xml +++ b/doc/guide/bind10-guide.xml @@ -1316,27 +1316,48 @@ TODO An ACL, or Access Control List, is a way to describe if a request is allowed or disallowed. The principle is, there's a list of rules. - A request either matches the rule or it doesn't. If it matches, - a decision is made and no more ACL processing happens. If it does - not match, the processing moves to the next rule. If there are - no more rules, a default action is taken. + Each rule is a name-value mapping (a dictionary, in the JSON + terminology). Each rule must contain exactly one mapping called + "action", which describes what should happen if the rule applies. + There may be more mappings, calld matches, which describe the + conditions under which the rule applies. - Each element in the ACL is a dictionary. There's exactly one - action element. This element describes the - decision taken when the rule matches. If it is set to - "ACCEPT", the request processed. In case of "REJECT", the request - is not processed and an error message is sent back (what it means - depends on which ACL it is). The action of "DROP" does not - process the request and sends no answer — pretending the - request never came (which means the sender might resend it). + When there's a query, the first rule is examined. If it matches, the + action in it is taken. If not, next rule is examined. If there are no + more rules to examine, a default action is taken. - Other elements inside the rule describe the properties a request - must have to match. The request must match all the properties - in the check. + There are three possible "action" values. The "ACCEPT" value means + the query is handled. If it is "REJECT", the query is not answered, + but a polite error message is sent back (if that makes sense in the + context). The "DROP" action acts like a black hole. The query is + not answered and no error message is sent. + + + + If there are multiple matching conditions inside the rule, all of + them must be satisfied for the rule to apply. This can be used, + for example, to require the query to be signed by a TSIG key and + originate from given address. + + + + This is encoded in form of JSON. Semi-formal description could look + something like this. It is described in more details below. + + ACL := [ RULE, RULE, ... ] +RULE := { "action": "ACCEPT"|"REJECT"|"DROP", MATCH, MATCH, ... } +RULE_RAW := { MATCH, MATCH, ... } +MATCH := FROM_MATCH|KEY_MATCH|NOT_MATCH|OR_MATCH|AND_MATCH|... +FROM_MATCH := "from": "<ip range> +KEY_MATCH := "key": "<key name> +NOT_MATCH := "NOT": RULE_RAW +OR_MATCH := "ANY": [ RULE_RAW, RULE_RAW, ... ] +AND_MATCH := "ALL": [ RULE_RAW, RULE_RAW, ... ] +

From f533afad3cbb047c6a5d4225720ff7b9dccafbc9 Mon Sep 17 00:00:00 2001 From: Michal 'vorner' Vaner Date: Mon, 13 Aug 2012 13:08:02 +0200 Subject: [PATCH 57/69] [2066] Add default values to ACL rules It otherwise complained during config add somewhere/acl, that there's no value. The defaults differ from module to module. Also, some reindentation and removal of tabs is added. --- src/bin/ddns/ddns.spec | 9 ++--- src/bin/resolver/resolver.spec.pre.in | 48 ++++++++++----------------- src/bin/xfrout/xfrout.spec.pre.in | 6 ++-- 3 files changed, 26 insertions(+), 37 deletions(-) diff --git a/src/bin/ddns/ddns.spec b/src/bin/ddns/ddns.spec index 70611e690f..3061cdc18d 100644 --- a/src/bin/ddns/ddns.spec +++ b/src/bin/ddns/ddns.spec @@ -12,8 +12,8 @@ "item_type": "map", "item_optional": true, "item_default": { - "origin": "", - "class": "IN", + "origin": "", + "class": "IN", "update_acl": [] }, "map_item_spec": [ @@ -33,11 +33,12 @@ "item_name": "update_acl", "item_type": "list", "item_optional": false, - "item_default": [], + "item_default": [], "list_item_spec": { "item_name": "acl_element", "item_type": "any", - "item_optional": true + "item_optional": true, + "item_default": {"action": "REJECT"} } } ] diff --git a/src/bin/resolver/resolver.spec.pre.in b/src/bin/resolver/resolver.spec.pre.in index d6bb22675b..138f4e38bb 100644 --- a/src/bin/resolver/resolver.spec.pre.in +++ b/src/bin/resolver/resolver.spec.pre.in @@ -116,37 +116,23 @@ }, { "item_name": "query_acl", - "item_type": "list", - "item_optional": false, - "item_default": [ - { - "action": "ACCEPT", - "from": "127.0.0.1" - }, - { - "action": "ACCEPT", - "from": "::1" - } - ], - "list_item_spec": { - "item_name": "rule", - "item_type": "map", - "item_optional": false, - "item_default": {}, - "map_item_spec": [ - { - "item_name": "action", - "item_type": "string", - "item_optional": false, - "item_default": "" - }, - { - "item_name": "from", - "item_type": "string", - "item_optional": false, - "item_default": "" - } - ] + "item_type": "list", + "item_optional": false, + "item_default": [ + { + "action": "ACCEPT", + "from": "127.0.0.1" + }, + { + "action": "ACCEPT", + "from": "::1" + } + ], + "list_item_spec": { + "item_name": "rule", + "item_type": "any", + "item_optional": false, + "item_default": {"action": "REJECT"} } } ], diff --git a/src/bin/xfrout/xfrout.spec.pre.in b/src/bin/xfrout/xfrout.spec.pre.in index dfcc6d98f4..6b113b0bf2 100644 --- a/src/bin/xfrout/xfrout.spec.pre.in +++ b/src/bin/xfrout/xfrout.spec.pre.in @@ -17,7 +17,8 @@ { "item_name": "acl_element", "item_type": "any", - "item_optional": true + "item_optional": true, + "item_default": {"action": "ACCEPT"} } }, { @@ -80,7 +81,8 @@ { "item_name": "acl_element", "item_type": "any", - "item_optional": true + "item_optional": true, + "item_default": {"action": "ACCEPT"} } } ] From 3483f241339bb3926b9a38b092f44579bd92469b Mon Sep 17 00:00:00 2001 From: Michal 'vorner' Vaner Date: Mon, 13 Aug 2012 13:12:58 +0200 Subject: [PATCH 58/69] [2066] ACL Docs: More tweaks by review * Note that TSIG key is DNS specific. * Language fix. * Returned an example to the resolver. --- doc/guide/bind10-guide.xml | 34 +++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-) diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml index 772bd82c43..dd63313e1a 100644 --- a/doc/guide/bind10-guide.xml +++ b/doc/guide/bind10-guide.xml @@ -1352,8 +1352,10 @@ TODO RULE := { "action": "ACCEPT"|"REJECT"|"DROP", MATCH, MATCH, ... } RULE_RAW := { MATCH, MATCH, ... } MATCH := FROM_MATCH|KEY_MATCH|NOT_MATCH|OR_MATCH|AND_MATCH|... -FROM_MATCH := "from": "<ip range> -KEY_MATCH := "key": "<key name> +FROM_MATCH := "from": [RANGE, RANGE, RANGE, ...] | RANGE +RANGE := "<ip range> +KEY_MATCH := "key": [KEY, KEY, KEY, ...] | KEY +KEY := "<key name> NOT_MATCH := "NOT": RULE_RAW OR_MATCH := "ANY": [ RULE_RAW, RULE_RAW, ... ] AND_MATCH := "ALL": [ RULE_RAW, RULE_RAW, ... ] @@ -1373,8 +1375,9 @@ AND_MATCH := "ALL": [ RULE_RAW, RULE_RAW, ... ] The other is TSIG key by which the message was signed. The ACL contains only the name (under the name "key"), the key itself - must be stored in the global keyring. + must be stored in the global keyring. This property is applicable only + to the DNS context. @@ -2101,7 +2104,7 @@ http://bind10.isc.org/wiki/ScalableZoneLoadDesign#a7.2UpdatingaZone Xfrout/transfer_acl[0] {"action": "ACCEPT"} any (default) - You can configure it the same as any ACL + You can configure it in the same way as any ACL (). @@ -2483,6 +2486,27 @@ what is XfroutClient xfr_client?? See . + + To allow queries on the "192.168.1.1" interface, these commands could be + issued. Note that we don't set the value of the third rule -- in the case + of resolver, rejecting all queries is the default value of new rule. And + that last final rule is not needed too, as the default, when a query falls + off the list, is rejection. + +> config show Resolver/query_acl +Resolver/query_acl[0] {"action": "ACCEPT", "from": "127.0.0.1"} any (default) +Resolver/query_acl[1] {"action": "ACCEPT", "from": "::1"} any (default) +> config add Resolver/query_acl +> config set Resolver/query_acl[2] {"action": "ACCEPT", "from": "192.168.1.1/24"} +> config add Resolver/query_acl +> config show Resolver/query_acl +Resolver/query_acl[0] {"action": "ACCEPT", "from": "127.0.0.1"} any (modified) +Resolver/query_acl[1] {"action": "ACCEPT", "from": "::1"} any (modified) +Resolver/query_acl[2] {"action": "ACCEPT", "from": "192.168.1.1/24"} any (modified) +Resolver/query_acl[3] {"action": "REJECT"} any (modified) +> config commit + +
From d1ee05cf0b7f6fe2b6cdc1b7b18ee6ca5aa258e2 Mon Sep 17 00:00:00 2001 From: Michal 'vorner' Vaner Date: Mon, 13 Aug 2012 14:05:54 +0200 Subject: [PATCH 59/69] [2066] Fix lettuce The fix that allowed resolver to have TSIG keys and other future checks made the setting of action impossible. This will work once again when we merge #2184, but for now, we need the fix. --- tests/lettuce/features/resolver_basic.feature | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/lettuce/features/resolver_basic.feature b/tests/lettuce/features/resolver_basic.feature index 409210186e..47fc12304c 100644 --- a/tests/lettuce/features/resolver_basic.feature +++ b/tests/lettuce/features/resolver_basic.feature @@ -27,10 +27,10 @@ Feature: Basic Resolver A query for l.root-servers.net. should have rcode REFUSED # Test whether acl ACCEPT works - When I set bind10 configuration Resolver/query_acl[0]/action to ACCEPT + When I set bind10 configuration Resolver/query_acl[0] to {"action": "ACCEPT", "from": "127.0.0.1"} # This address is currently hardcoded, so shouldn't cause outside traffic A query for l.root-servers.net. should have rcode NOERROR # Check whether setting the ACL to reject again works - When I set bind10 configuration Resolver/query_acl[0]/action to REJECT + When I set bind10 configuration Resolver/query_acl[0] to {"action": "REJECT", "from": "127.0.0.1"} A query for l.root-servers.net. should have rcode REFUSED From be7f08f05aa29fda91f4b214d660d3bfea12ba45 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Mon, 13 Aug 2012 14:26:00 +0200 Subject: [PATCH 60/69] [2172] set mem_info to None in OSX before overwriting --- src/lib/python/isc/sysinfo/sysinfo.py | 1 + 1 file changed, 1 insertion(+) diff --git a/src/lib/python/isc/sysinfo/sysinfo.py b/src/lib/python/isc/sysinfo/sysinfo.py index c3238142a9..1a79acade0 100644 --- a/src/lib/python/isc/sysinfo/sysinfo.py +++ b/src/lib/python/isc/sysinfo/sysinfo.py @@ -425,6 +425,7 @@ class SysInfoOSX(SysInfoFreeBSDOSX): # was read. However, on OSX, physmem is not necessarily the correct # value. But since it does not fail and does work on most BSD's, it's # left in the base class and overwritten here + self._mem_total = None try: s = subprocess.check_output(['sysctl', '-n', 'hw.memsize']) self._mem_total = int(s.decode('utf-8').strip()) From ffc684f617ecb3967bc544da507e718bb1709d65 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Mon, 13 Aug 2012 14:26:40 +0200 Subject: [PATCH 61/69] [2172] only use 'distro' value on linux --- src/lib/python/isc/sysinfo/sysinfo.py | 19 ++++++++++----- .../python/isc/sysinfo/tests/sysinfo_test.py | 24 +++---------------- 2 files changed, 16 insertions(+), 27 deletions(-) diff --git a/src/lib/python/isc/sysinfo/sysinfo.py b/src/lib/python/isc/sysinfo/sysinfo.py index 1a79acade0..8090d4fbe5 100644 --- a/src/lib/python/isc/sysinfo/sysinfo.py +++ b/src/lib/python/isc/sysinfo/sysinfo.py @@ -36,16 +36,20 @@ class SysInfo: self._loadavg = None self._mem_total = None self._mem_free = None - self._mem_cached = None - self._mem_buffers = None self._mem_swap_total = None self._mem_swap_free = None - self._platform_distro = 'Unknown' self._net_interfaces = 'Unknown\n' self._net_routing_table = 'Unknown\n' self._net_stats = 'Unknown\n' self._net_connections = 'Unknown\n' + # The following are Linux speicific, and should eventually be removed + # from this level; for now we simply default to None (so they won't + # be printed) + self._platform_distro = None + self._mem_cached = None + self._mem_buffers = None + def get_num_processors(self): """Returns the number of processors. This is the number of hyperthreads when hyper-threading is enabled. @@ -77,7 +81,12 @@ class SysInfo: return self._platform_is_smp def get_platform_distro(self): - """Returns the name of the OS distribution in use.""" + """Returns the name of the OS distribution in use. + + Note: the concept of 'distribution' is Linux specific. This shouldn't + be at this level. + + """ return self._platform_distro def get_uptime(self): @@ -276,8 +285,6 @@ class SysInfoBSD(SysInfoPOSIX): except (subprocess.CalledProcessError, OSError): pass - self._platform_distro = self._platform_name + ' ' + self._platform_version - try: s = subprocess.check_output(['ifconfig']) self._net_interfaces = s.decode('utf-8') diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index 01a78d4ceb..60d5f9e863 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -261,7 +261,7 @@ class SysInfoTest(unittest.TestCase): self.assertEqual(None, s.get_mem_buffers()) self.assertEqual(None, s.get_mem_swap_total()) self.assertEqual(None, s.get_mem_swap_free()) - self.assertEqual('Unknown', s.get_platform_distro()) + self.assertEqual(None, s.get_platform_distro()) self.assertEqual('Unknown\n', s.get_net_interfaces()) self.assertEqual('Unknown\n', s.get_net_routing_table()) self.assertEqual('Unknown\n', s.get_net_stats()) @@ -290,7 +290,7 @@ class SysInfoTest(unittest.TestCase): self.assertEqual(None, s.get_mem_buffers()) self.assertEqual(None, s.get_mem_swap_total()) self.assertEqual(None, s.get_mem_swap_free()) - self.assertEqual('Unknown', s.get_platform_distro()) + self.assertEqual(None, s.get_platform_distro()) self.assertEqual('Unknown\n', s.get_net_interfaces()) self.assertEqual('Unknown\n', s.get_net_routing_table()) self.assertEqual('Unknown\n', s.get_net_stats()) @@ -338,6 +338,7 @@ class SysInfoTest(unittest.TestCase): self.assertLess(abs(76632 - s.get_uptime()), 4) self.assertEqual(None, s.get_mem_cached()) self.assertEqual(None, s.get_mem_buffers()) + self.assertEqual(None, s.get_platform_distro()) # These test that the corresponding tools are being called (and # no further processing is done on this data). Please see the @@ -371,13 +372,6 @@ class SysInfoTest(unittest.TestCase): self.assertEqual(566791168, s.get_mem_swap_total()) self.assertEqual(566789120, s.get_mem_swap_free()) - # Try new regex assertion (which replaced the deprecated - # assertRegexpMatches. If it is not available, use the old one - try: - self.assertRegex(s.get_platform_distro(), '^OpenBSD\s+.*') - except AttributeError: - self.assertRegexpMatches(s.get_platform_distro(), '^OpenBSD\s+.*') - def test_sysinfo_freebsd(self): """Tests the FreeBSD implementation of SysInfo. Note that this tests deep into the implementation, and not just the @@ -401,12 +395,6 @@ class SysInfoTest(unittest.TestCase): self.assertEqual(543214321 - (343434 * 1024), s.get_mem_free()) self.assertEqual(1037533184, s.get_mem_swap_total()) self.assertEqual(1037533184, s.get_mem_swap_free()) - # Try new regex assertion (which replaced the deprecated - # assertRegexpMatches. If it is not available, use the old one - try: - self.assertRegex(s.get_platform_distro(), '^FreeBSD\s+.*') - except AttributeError: - self.assertRegexpMatches(s.get_platform_distro(), '^FreeBSD\s+.*') def test_sysinfo_osx(self): """Tests the OS X implementation of SysInfo. Note that this @@ -431,12 +419,6 @@ class SysInfoTest(unittest.TestCase): self.assertEqual((23456 * 4096), s.get_mem_free()) self.assertEqual(18874368.0, s.get_mem_swap_total()) self.assertEqual(1075988.48, s.get_mem_swap_free()) - # Try new regex assertion (which replaced the deprecated - # assertRegexpMatches. If it is not available, use the old one - try: - self.assertRegex(s.get_platform_distro(), '^Darwin\s+.*') - except AttributeError: - self.assertRegexpMatches(s.get_platform_distro(), '^Darwin\s+.*') if __name__ == "__main__": unittest.main() From db05d3bab1a3ef3bdfe2cd10c453cc4b6ca7e421 Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Mon, 13 Aug 2012 14:27:37 +0200 Subject: [PATCH 62/69] [2172] use netstat instead of route on openBSD --- src/lib/python/isc/sysinfo/sysinfo.py | 18 ++++++------------ .../python/isc/sysinfo/tests/sysinfo_test.py | 6 ++---- 2 files changed, 8 insertions(+), 16 deletions(-) diff --git a/src/lib/python/isc/sysinfo/sysinfo.py b/src/lib/python/isc/sysinfo/sysinfo.py index 8090d4fbe5..97b9e59253 100644 --- a/src/lib/python/isc/sysinfo/sysinfo.py +++ b/src/lib/python/isc/sysinfo/sysinfo.py @@ -303,6 +303,12 @@ class SysInfoBSD(SysInfoPOSIX): except (subprocess.CalledProcessError, OSError): self._net_connections = 'Warning: "netstat -an" command failed.\n' + try: + s = subprocess.check_output(['netstat', '-nr']) + self._net_routing_table = s.decode('utf-8') + except (subprocess.CalledProcessError, OSError): + self._net_connections = 'Warning: "netstat -nr" command failed.\n' + class SysInfoOpenBSD(SysInfoBSD): """OpenBSD implementation of the SysInfo class. See the SysInfo class documentation for more information. @@ -345,12 +351,6 @@ class SysInfoOpenBSD(SysInfoBSD): except (subprocess.CalledProcessError, OSError): pass - try: - s = subprocess.check_output(['route', '-n', 'show']) - self._net_routing_table = s.decode('utf-8') - except (subprocess.CalledProcessError, OSError): - self._net_routing_table = 'Warning: "route -n show" command failed.\n' - class SysInfoFreeBSDOSX(SysInfoBSD): """Shared code for the FreeBSD and OS X implementations of the SysInfo class. See the SysInfo class documentation for more information. @@ -381,12 +381,6 @@ class SysInfoFreeBSDOSX(SysInfoBSD): except (subprocess.CalledProcessError, OSError): pass - try: - s = subprocess.check_output(['netstat', '-nr']) - self._net_routing_table = s.decode('utf-8') - except (subprocess.CalledProcessError, OSError): - self._net_connections = 'Warning: "netstat -nr" command failed.\n' - class SysInfoFreeBSD(SysInfoFreeBSDOSX): """FreeBSD implementation of the SysInfo class. See the SysInfo class documentation for more information. diff --git a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py index 60d5f9e863..0add0367e2 100644 --- a/src/lib/python/isc/sysinfo/tests/sysinfo_test.py +++ b/src/lib/python/isc/sysinfo/tests/sysinfo_test.py @@ -138,6 +138,8 @@ def _my_bsd_subprocess_check_output(command): return b'osuxbrcc1g9VgaF4yf3FrtfodrfATrbSnjhqhuQSAs8=\n' elif command == ['netstat', '-an']: return b'Z+w0lwa02/T+5+EIio84rrst/Dtizoz/aL9Im7J7ESA=\n' + elif command == ['netstat', '-nr']: + return b'XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n' else: return None @@ -151,8 +153,6 @@ def _my_openbsd_subprocess_check_output(command): return b' procs memory page disks traps cpu\n r b w avm fre flt re pi po fr sr wd0 cd0 int sys cs us sy id\n 0 0 0 121212 123456 47 0 0 0 0 0 2 0 2 80 14 0 1 99\n' elif command == ['swapctl', '-s', '-k']: return b'total: 553507 1K-blocks allocated, 2 used, 553505 available' - elif command == ['route', '-n', 'show']: - return b'XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n' else: bsd_output = _my_bsd_subprocess_check_output(command) if bsd_output is not None: @@ -178,8 +178,6 @@ def _my_freebsd_osx_subprocess_check_output(command): return bytes('{ sec = ' + str(int(time.time() - 76632)) + ', usec = 0 }\n', 'utf-8') elif command == ['sysctl', '-n', 'vm.loadavg']: return b'{ 0.2 0.4 0.6 }\n' - elif command == ['netstat', '-nr']: - return b'XfizswwNA9NkXz6K36ZExpjV08Y5IXkHI8jjDSV+5Nc=\n' else: return _my_bsd_subprocess_check_output(command) From e05046fbd6b546b3da756d15a0dd7b64828499ce Mon Sep 17 00:00:00 2001 From: Jelte Jansen Date: Mon, 13 Aug 2012 15:50:36 +0200 Subject: [PATCH 63/69] [2184] removed dead code and fixed bad error --- src/lib/python/isc/config/ccsession.py | 3 +-- src/lib/python/isc/config/config_data.py | 3 +-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/src/lib/python/isc/config/ccsession.py b/src/lib/python/isc/config/ccsession.py index 276a4e1ae0..a95316d6bd 100644 --- a/src/lib/python/isc/config/ccsession.py +++ b/src/lib/python/isc/config/ccsession.py @@ -594,7 +594,6 @@ class UIModuleCCSession(MultiConfigData): item_name = None item_value = None if value_str is not None: - #item_name = isc.cc.data.parse_value_str(value_str) item_name = value_str if set_value_str is not None: item_value = isc.cc.data.parse_value_str(set_value_str) @@ -669,7 +668,7 @@ class UIModuleCCSession(MultiConfigData): (type_any and type(cur_value) == dict): self._remove_value_from_named_set(identifier, value_str) else: - raise isc.cc.data.DataNotFoundError(str(identifier) + " is not a list or a named_set " + str(module_spec) + " and type " + str(type(cur_value))) + raise isc.cc.data.DataNotFoundError(str(identifier) + " is not a list or a named_set") diff --git a/src/lib/python/isc/config/config_data.py b/src/lib/python/isc/config/config_data.py index 2bc6edbee3..413d052b0b 100644 --- a/src/lib/python/isc/config/config_data.py +++ b/src/lib/python/isc/config/config_data.py @@ -657,8 +657,7 @@ class MultiConfigData: all) else: value, status = self.get_value(identifier) - if status == self.NONE and not spec_part['item_optional']:# and\ - #not ('item_default' in spec_part): + if status == self.NONE and not spec_part['item_optional']: raise isc.cc.data.DataNotFoundError(identifier + " not found") entry = _create_value_map_entry(identifier, From 77e9624b4ea99f4654d58392f82e3ae02facf216 Mon Sep 17 00:00:00 2001 From: Tomek Mrugalski Date: Mon, 13 Aug 2012 16:16:23 +0200 Subject: [PATCH 64/69] [master] commit-id updated. --- ChangeLog | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/ChangeLog b/ChangeLog index 11176b625d..fbffe449e9 100644 --- a/ChangeLog +++ b/ChangeLog @@ -5,7 +5,7 @@ is 'Dhcp6 shutdown'. b10-dhcp4: Command line-switch '-s' to disable msgq was added. b10-dhcp6: Command line-switch '-s' to disable msgq was added. - (Trac #1708, git tbd) + (Trac #1708, git e0d7c52a71414f4de1361b09d3c70431c96daa3f) 458. [build]* jinmei BIND 10 now relies on Boost offset_ptr, which caused some new From 33ea36e96957ee97641a4f5fb281e974c79dba45 Mon Sep 17 00:00:00 2001 From: JINMEI Tatuya Date: Mon, 13 Aug 2012 11:31:54 -0700 Subject: [PATCH 65/69] [2066] editorial fix: missing closing double-quotes --- doc/guide/bind10-guide.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml index dd63313e1a..0fe0ceb3ef 100644 --- a/doc/guide/bind10-guide.xml +++ b/doc/guide/bind10-guide.xml @@ -1353,9 +1353,9 @@ RULE := { "action": "ACCEPT"|"REJECT"|"DROP", MATCH, MATCH, ... } RULE_RAW := { MATCH, MATCH, ... } MATCH := FROM_MATCH|KEY_MATCH|NOT_MATCH|OR_MATCH|AND_MATCH|... FROM_MATCH := "from": [RANGE, RANGE, RANGE, ...] | RANGE -RANGE := "<ip range> +RANGE := "<ip range>"; KEY_MATCH := "key": [KEY, KEY, KEY, ...] | KEY -KEY := "<key name> +KEY := "<key name>"; NOT_MATCH := "NOT": RULE_RAW OR_MATCH := "ANY": [ RULE_RAW, RULE_RAW, ... ] AND_MATCH := "ALL": [ RULE_RAW, RULE_RAW, ... ] From 112b9731c47d04769ed1de4eaed90752da8f285a Mon Sep 17 00:00:00 2001 From: JINMEI Tatuya Date: Tue, 14 Aug 2012 12:27:33 -0700 Subject: [PATCH 66/69] [2066] corrected small errors in the previous fix --- doc/guide/bind10-guide.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml index 0fe0ceb3ef..64a6040d02 100644 --- a/doc/guide/bind10-guide.xml +++ b/doc/guide/bind10-guide.xml @@ -1353,9 +1353,9 @@ RULE := { "action": "ACCEPT"|"REJECT"|"DROP", MATCH, MATCH, ... } RULE_RAW := { MATCH, MATCH, ... } MATCH := FROM_MATCH|KEY_MATCH|NOT_MATCH|OR_MATCH|AND_MATCH|... FROM_MATCH := "from": [RANGE, RANGE, RANGE, ...] | RANGE -RANGE := "<ip range>"; +RANGE := "<ip range>" KEY_MATCH := "key": [KEY, KEY, KEY, ...] | KEY -KEY := "<key name>"; +KEY := "<key name>" NOT_MATCH := "NOT": RULE_RAW OR_MATCH := "ANY": [ RULE_RAW, RULE_RAW, ... ] AND_MATCH := "ALL": [ RULE_RAW, RULE_RAW, ... ] From c22e36e2ffb2a1e4586d0838c81aa44efb3bdd3e Mon Sep 17 00:00:00 2001 From: JINMEI Tatuya Date: Tue, 14 Aug 2012 12:27:46 -0700 Subject: [PATCH 67/69] [2066] added a note that list form of ACL match value doesn't work. referring to #2066. --- doc/guide/bind10-guide.xml | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml index 64a6040d02..00f7e3aa46 100644 --- a/doc/guide/bind10-guide.xml +++ b/doc/guide/bind10-guide.xml @@ -1403,6 +1403,18 @@ AND_MATCH := "ALL": [ RULE_RAW, RULE_RAW, ... ] will work in a similar way. + + + The list form is currently rejected due to an + implementation bug. There is a plan to fix it relatively + soon, so the syntax is kept here, but note that it won't + work until the bug is fixed. To keep track of the status + of the issue, see + Trac #2191. + Until then, the value must be a single string. + + + If that is not enough, you can compose the matching conditions to logical expressions. They are called "ANY", "ALL" and "NOT". From a5c9a0e9e86ef3ff1ce2ae8fc63640e7a94c63fa Mon Sep 17 00:00:00 2001 From: JINMEI Tatuya Date: Tue, 14 Aug 2012 12:31:06 -0700 Subject: [PATCH 68/69] [2066] revised text about xfrout acl config per my own suggestion --- doc/guide/bind10-guide.xml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml index 00f7e3aa46..32592d16c8 100644 --- a/doc/guide/bind10-guide.xml +++ b/doc/guide/bind10-guide.xml @@ -2115,11 +2115,6 @@ http://bind10.isc.org/wiki/ScalableZoneLoadDesign#a7.2UpdatingaZone > config show Xfrout/transfer_acl Xfrout/transfer_acl[0] {"action": "ACCEPT"} any (default) - - You can configure it in the same way as any ACL - (). - - If you want to require TSIG in access control, a system wide TSIG "key ring" must be configured. @@ -2135,6 +2130,11 @@ Xfrout/transfer_acl[0] {"action": "ACCEPT"} any (default) will use the system wide keyring to check TSIGs in the incoming messages and to sign responses. + + For further details on ACL configuration, see + . + + The way to specify zone specific configuration (ACLs, etc) is likely to be changed. From aa78dee2dc53f87be18838e1c383778f4b6e9308 Mon Sep 17 00:00:00 2001 From: JINMEI Tatuya Date: Tue, 14 Aug 2012 12:34:37 -0700 Subject: [PATCH 69/69] [2066] revised resolver ACL text per my own suggestion --- doc/guide/bind10-guide.xml | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/doc/guide/bind10-guide.xml b/doc/guide/bind10-guide.xml index 32592d16c8..f585855f1f 100644 --- a/doc/guide/bind10-guide.xml +++ b/doc/guide/bind10-guide.xml @@ -2499,24 +2499,26 @@ what is XfroutClient xfr_client?? - To allow queries on the "192.168.1.1" interface, these commands could be - issued. Note that we don't set the value of the third rule -- in the case - of resolver, rejecting all queries is the default value of new rule. And - that last final rule is not needed too, as the default, when a query falls - off the list, is rejection. + The following session is an example of extending the ACL to also + allow queries from 192.0.2.0/24: > config show Resolver/query_acl Resolver/query_acl[0] {"action": "ACCEPT", "from": "127.0.0.1"} any (default) Resolver/query_acl[1] {"action": "ACCEPT", "from": "::1"} any (default) > config add Resolver/query_acl -> config set Resolver/query_acl[2] {"action": "ACCEPT", "from": "192.168.1.1/24"} +> config set Resolver/query_acl[2] {"action": "ACCEPT", "from": "192.0.2.0/24"} > config add Resolver/query_acl > config show Resolver/query_acl Resolver/query_acl[0] {"action": "ACCEPT", "from": "127.0.0.1"} any (modified) Resolver/query_acl[1] {"action": "ACCEPT", "from": "::1"} any (modified) -Resolver/query_acl[2] {"action": "ACCEPT", "from": "192.168.1.1/24"} any (modified) +Resolver/query_acl[2] {"action": "ACCEPT", "from": "192.0.2.0/24"} any (modified) Resolver/query_acl[3] {"action": "REJECT"} any (modified) > config commit + Note that we didn't set the value of the last final rule + (query_acl[3]) -- in the case of resolver, rejecting all queries is + the default value of a new rule. In fact, this rule can even be + omitted completely, as the default, when a query falls off the list, + is rejection.